Kaizen 228 - Process Large-Scale Migrated Data Using Catalyst Solutions

Howdy, tech wizards!

This week’s Kaizen explores how Catalyst Solutions in the Zoho CRM Developer Hub help import large volumes of data into Zoho CRM while improving data quality and simplifying the migration process.

Why Catalyst Solutions?

Catalyst Solutions provide a library of independent, ready-to-use backend solutions called CodeLibs. The CRM Bulk Data Processor CodeLib helps handle complex data operations on large datasets. Following are the key benefits:

1. Highly Scalable: Processes large datasets efficiently, supporting up to 200,000 records per API call.
2. Automation Made Simple: Automate complex tasks like segmentation, data cleansing and lead scoring with minimal effort.  
3. Effortless Integration: Comes with pre-configured Catalyst resources such as serverless functions and data stores that integrate directly with Zoho CRM.
   

Business Scenario

Zylker, a manufacturing company, migrated over 3 lakh Lead records into Zoho CRM from spreadsheets. 

Post-migration, the Lead data showed the following inconsistencies and challenges:

1. Phone number field did not include country codes, or contained incorrect country codes. This inconsistency made it difficult for sales teams to contact Leads reliably.
2. Identifying potential Leads for follow-up required manual effort and marketing team has to run generic campaigns without lead segmentation. 

Using workflows before record creation to fix phone numbers for this volume would result in 3 lakh individual workflow executions, which can slow imports and cause partial failures.

Solution Overview

Zylker can use the CRM Bulk Data Processor available in Catalyst Solutions to address these challenges at scale.

The solution works as follows:

1. Lead records are fetched in bulk from Zoho CRM using the Bulk Read API.
2. The fetched data is temporarily stored to evaluate the priority and quality of each record. Based on this evaluation, the records are also segmented accordingly.
3. The Country field value is used to identify and append the correct country code to the phone number.
4. The processed records are updated back into Zoho CRM using Bulk Write API.

All operations are executed using pre-configured Catalyst resources provided by the CodeLib. You can explore the Bulk Data Flow in Catalyst help page to understand how each resource participates in the processing pipeline.

Prerequisites

Before you begin, ensure the following:

1. Log into your Zoho CRM and navigate to Setup > Customization > Modules and Fields > Leads.

Create the following custom Single Line fields to map existing record data in Zoho CRM. 

1. Last Activity Days
2. Web Engagement Score

Refer to the Working with Custom Fields help page for detailed guide. 

2. Create the following custom Single Line fields to store the calculated values during bulk processing:

1. Lead Score Value
2. Lead Segment
3. Sales Priority
4. Data Quality Status
   

3. Go to Setup > Developer Hub > Catalyst Solutions > Zoho CRM Bulk Data Processor. 

4. Click Go to Catalyst and create a Catalyst account if required.

5. Refer to the Catalyst CLI Installation Guide and install it on your local system.

6. Create a new folder on your local system, which will act as a local project directory. Use the following command:

mkdir -p <folder_name>

7. Navigate to the project folder in your terminal and execute the following command: 

catalyst init

8. Follow the prompts to select and initialize your desired project from the Catalyst console. 

When prompted to choose a feature, press Enter and proceed without choosing any feature.

9. Run the following command to install the CRM Bulk Data Processor CodeLib.

catalyst codelib:install https://github.com/catalystbyzoho/codelib-zoho-crm-bulk-processor

Step 1: Create a Cron Job

1. Open the Catalyst Console and navigate to the same project in which you have installed the CRM Bulk Data Processor CodeLib.

2. Go to Cloud Scale > Triggers > Cron and click Create Cron.

3. Fill in the following details:

1. Provide a Name and Description for the cron job. 
2. Select Function and choose BulkJobScheduler from the dropdown in the Target Function field.
3. Enter the following parameters and values:
   
   MODULES -> Leads
   FIELDS_TO_BE_PROCESSED -> Last_Name, Company, Mobile, Country, Designation, No_of_Employees, Web_Engagement_Score, Lead_Score_Value, Lead_Segment, Sales_Priority, Data_Quality_Status
   
   Use GET Modules Metadata API and GET Fields Metadata API to get the API names of the required module and fields.
    
4. Choose the Schedule Type as One Time for our use case.
   

Refer to the Implementing Cron Jobs Help Guide for more details.

Step 2: Configure Zoho CRM API Credentials 

1. Create a self-client application in Zoho's API Console.

2. Generate a Grant Token with the following scopes:

1. ZohoFiles.files.ALL
2. ZohoCRM.bulk.ALL
3. ZohoCRM.modules.ALL
4. ZohoCRM.settings.ALL
5. ZohoCRM.org.ALL
   

3. Follow the OAuth Help Section for a step-by-step guide to generate Access and Refresh tokens for your Zoho CRM organization.

Store all the API credentials securely for later use.

4. To allow Catalyst to access Zoho CRM data, configure the stored API credentials as environment variables in the catalyst-config.json file within the BulkDataProcessor function.

For detailed guidance, refer to the Configure Function Components help guide in CRM Bulk Data Processor. 

These environment variables are used by Catalyst Connectors to create access tokens for establishing secure connection between Zoho CRM and Catalyst.

Step 3: Add Business Logic

1. Go to com.processor.record.ZCRMRecordsProcessorImpl.java file of the BulkDataProcessor in the functions directory.

2. Override the ZCRMRecordsProcessor method with the business logic.

The complete code sample for this use case is available on GitHub for reference.

Following is the major custom logic used here:

Use the GET Fields Metadata API to fetch the API names of the fields required throughout the custom logic.

Phone Number Normalization:

This can be implemented in two stages, Country standardization and Phone Number formatting. 

For country standardization, CountryStandardizerUtil maps different variations of the Country field in records to a single standardized country identifier. These variations may include, 

1. Country Names (India, Unites States, United Kingdom)
2. Abbreviations (Ind, IN, USA, US, UK)
3. Calling Codes (+91, +1, +44)
   

With this, we can ensure different interpretations of the same country are interpreted consistently during the process. 

Once the country is standardized, Phone Numbers are formatted to E.164 format in the PhoneNormalizerUtil. 

It removes all non-numeric characters and converts the Phone input to E.164 format with the respective country codes. 

public class PhoneNormalizerUtil {     private static final Pattern NON_DIGIT = Pattern.compile("[^0-9]");     public static Optional<String> normalizeToE164(String rawPhone, String countryCode) {         if (rawPhone == null || rawPhone.trim().isEmpty()) {             return Optional.empty();         }         String digits = NON_DIGIT.matcher(rawPhone).replaceAll("");         if (rawPhone.startsWith("+") && digits.length() >= 10) {             return Optional.of("+" + digits);         }         switch (countryCode) {             case "IN":                 return normalizeIndia(digits);             case "US":                 return normalizeUS(digits);             case "UK":                 return normalizeUK(digits);             default:                 return Optional.empty();         }     }     private static Optional<String> normalizeIndia(String digits) {         if (digits.length() == 10) {             return Optional.of("+91" + digits);         }         if (digits.startsWith("91") && digits.length() == 12) {             return Optional.of("+" + digits);         }         return Optional.empty();     }     private static Optional<String> normalizeUS(String digits) {         if (digits.length() == 10) {             return Optional.of("+1" + digits);         }         if (digits.startsWith("1") && digits.length() == 11) {             return Optional.of("+" + digits);         }         return Optional.empty();     }     private static Optional<String> normalizeUK(String digits) {         if (digits.startsWith("44") && digits.length() == 12) {             return Optional.of("+" + digits);         }         if (digits.length() == 10) {             return Optional.of("+44" + digits);         }         return Optional.empty();     } }

Lead Priority and Segmentation

The lead priority is calculated using the Title, No of employees, Last Activity Days and Web Engagement Score fields in the record.  

Each field value contributes a predefined number of points:

1. If the job title indicates a senior role such as CEO, CTO, Director, or VP, 25 points are added.
2. If the company size is 200 employees or more, 20 points are added.
3. If the Lead has been active within the last 7 days, 15 points are added.
4. If the web engagement score is 70 or higher, 10 points are added.

The sum of all applicable factors is returned as an integer and based on this score the lead's priority and segmentation is assigned as follows:

1. High (Hot): score ≥ 70
2. Medium (Warm): score ≥ 40
3. Low (Cold): score < 40
   

public static int calculateLeadScore(             String jobTitle,             Integer companySize,             Integer lastActivityDays,             Integer webEngagementScore) {         int score = 0;         // Decision Maker         if (jobTitle != null) {             String title = jobTitle.toLowerCase();             if (title.contains("ceo") || title.contains("cto")                     || title.contains("director") || title.contains("vp")) {                 score += 25;             }         }         // Company Size         if (companySize != null && companySize >= 200) {             score += 20;         }         // Recency         if (lastActivityDays != null && lastActivityDays <= 7) {             score += 15;         }         // Engagement         if (webEngagementScore != null && webEngagementScore >= 70) {             score += 10;         }         return score;     }     public static String deriveSegment(int score) {         if (score >= 70) return "Hot";         if (score >= 40) return "Warm";         return "Cold";     }     public static String derivePriority(String segment) {         switch (segment) {             case "Hot":                 return "High";             case "Warm":                 return "Medium";             default:                 return "Low";         }     }

Lead Quality:

Lead quality is evaluated based on the validity of the Mobile and Country fields in the records.

1. Clean: Both field values are valid.
2. Critical: Both field values are invalid.
3. Needs Review: Only one of the field values is invalid.
   

Write Back to Records:

The processed data is written back to Zoho CRM as follows:

1. Mobile: Formatted phone numbers.
2. Lead Score: Total score calculated from all applicable fields.
3. Lead Segment: Segment derived from the lead score.
4. Sales Priority: Priority assigned based on the lead segment.
5. Data Quality Status: Overall quality of the Lead record.
   

if (normalizedMobile.isPresent()) { data.put("Mobile", normalizedMobile.get()); } data.put("Lead_Score", leadScore); data.put("Lead_Segment", leadSegment); data.put("Sales_Priority", salesPriority); data.put("Data_Quality_Status", dataQualityStatus); }

Step 4: Deploy to Development

Run the following command to deploy your local customizations to the Development Environment of Catalyst.

catalyst deploy

You can find the complete code sample on GitHub for reference.

Try it Out!

With the changes deployed to the Development environment, let us now test the solution.

We hope this Kaizen helps you import large volumes of data into Zoho CRM while improving data quality and lead intelligence using the CRM Bulk Data Processor.

Have questions or suggestions? Drop them in the comments or write to us at  support@zohocrm.com. 

We will circle back to you next Friday with another interesting topic. 

On to Better Building!

--------------------------------------------------------------------------------------------------------------------------

Previous Kaizen: Client Script Support for List Page (Canvas) | Kaizen Collection: Directory

--------------------------------------------------------------------------------------------------------------------------

• Topic Participants

• Kiruthiga G

  

• Sticky Posts

• Kaizen #198: Using Client Script for Custom Validation in Blueprint

  Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

• Kaizen #226: Using ZRC in Client Script

  Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In this

• Kaizen #222 - Client Script Support for Notes Related List

  Hello everyone! Welcome to another week of Kaizen. The final Kaizen post of the year 2025 is here! With the new Client Script support for the Notes Related List, you can validate, enrich, and manage notes across modules. In this post, we’ll explore how

• Kaizen #217 - Actions APIs : Tasks

  Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.

• Kaizen #216 - Actions APIs : Email Notifications

  Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are

• Recent Topics

• How can i download and secure all my mails from the archive folders?

  Hi, i would like to download all my mails from my archive folders and secure it on my external HDD. Is this possible? Thx. amir

• How to open filtered report in popup using Zoho Creator Deluge?

  First report: There is so many records in Report, If I click one record, pop up is occur. Second report (Pop up): there is also so many record data, and this pop up is also Report, not Form. First report: It has got "Sales Order" field. when I click any

• Make panel configuration interface wider

  Hi there, The same way you changed the custom function editor's interface wider, it would be nice to be able to edit panels in pages using the full width of the screen rather than the currently max-width: 1368px. Is there a reason for having the configuration panel not taking the full width? Its impossible at this width to edit panels that have a lot of elements. Please change it to 100% so we can better edit the layouts. Thanks! B.

• Can you default reports/charts to report the current week?

  Our data table maintains two years of data. Management wants certain report to automatically filter the report to the latest calendar week. I know I can do this manually with filters but I want the report to automatically default to the latest calendar

• Rendering PDF to view on page

  My company upload lots of PDF files onto Zoho. But every time we open it, it downloads the file instead of viewing it on the web page. Does Zoho allow uploaded PDF files to be rendered to view on web page yet? I've been trying to use <embed> or <object> but it cannot be loaded.  (similar thread: https://help.zoho.com/portal/community/topic/how-to-open-a-pdf-file-of-a-view-in-preview-mode)

• Overlapping Reports in Dashboards

  It's rare, but occasionally it would be a good feature if I were able to overlap reports, either fully or partially in the Dashboards. Also, then having the ability to move objects to the front or rear, or make them transparent/translucent would be good

• Feature request - pin or flag note

  Hi, It would be great if you could either pin or flag one or more notes so that they remain visible when there are a bunch of notes and some get hidden in the list. Sometimes you are looking for a particular name that gets lost in a bunch of less important

• Admin guide: Handling Mailbox Settings for better user management

  Managing day-to-day email scenarios, such as supporting users with multiple email addresses, ensuring uninterrupted email access during employee absences, enabling secure mailbox sharing, and enforcing organizational security and compliance, can be challenging

• Cisco Webex Calling Intergration

  Hi Guys, Our organisation is looking at a move from Salesforce to Zoho. We have found there is no support for Cisco Webex Calling however? Is there a way to enable this or are there any apps which can provide this? Thanks!

• Designing a practical Zoho setup for a small business: lessons from a real implementation

  I recently finished setting up a Zoho-based operating system for a small but growing consumer beauty business (GlowAtHomeBeauty), and I wanted to share a practical takeaway for other founders and implementers. The business wasn’t failing because of lack

• DKIM (Marketing emails) UNVERIFIED (Zoho One)

  I'm having a problem with Zoho One verifying my Marketing Email DKIM Record for MYFINISHERPHOTOS.COM. I have removed and re-entered the ownership, DKIM (Transactional emails), SPF and Marketing DKIM and all of them come back verified except the DKIM (Marketing

• Zoho Recruit Community Meet-up - India

  Namaste, India. 🙏🏼 The Zoho Recruit team is hitting the road—and we 're absolutely excited behind the scenes. Join us for the Zoho Recruit India Meet-up 2026, a morning designed to make your recruiting life easier (and a lot more effective). Date City

• Generate a Zoho Sign link

  From time to time I get a response "I never received your you e-document for electronic signature" is there a way to generate a Zoho Sign link to share.

• Zoho Social - Cliq Integration / Bot

  Dear community / zoho, I am looking for a way to create a bot within Zoho Cliq to update my colleagues about our Zoho Social activities. For example, if a new post is published, it would be great if this post automatically would be shared in our social

• CRM gets location smart with the all new Map View: visualize records, locate records within any radius, and more

  Hello all, We've introduced a new way to work with location data in Zoho CRM: the Map View. Instead of scrolling through endless lists, your records now appear as pins on a map. Built on top of the all-new address field and powered by Mappls (MapMyIndia),

• Is it possible to create a word cloud chart in ZoHo Analystics?

  Hi there, I have a volume of transaction text that I would like to analyse using word cloud (or other approcah to detect and present word frequency in a dataset). For example, I have 50,000 records describing menu items in restaurants. I want to be able

• How to Fix the Corrupted Outlook 2019 .pst file on Windows safely?

  There are multiple reasons to get corrupted PST files (due to a power failure, system crash, or forced shutdown) and several other reasons. If You are using this ScanePST.EXE Microsoft inbuilt recovery tool, it only supports the minor corruption issue

• [Webinar] A recap of Zoho Writer in 2025

  Hi Zoho Writer users, We're excited to announce Zoho Writer's webinar for January 2026: A recap of Zoho Writer in 2025. This webinar will provide a recap of the features and enhancements we added in 2025 to enhance your productivity. Choose your preferred

• How to drag row(s) or column(s)?

  Hi. Selecting a row or column and then dragging it to a new position does not seem to work. Am i missing something or this is just not possible in Zoho Sheet? Cheers, Jay

• Building Toppings #5 - Creating and configuring custom service connections in Bigin Toppings

  Hello Biginners, Integrating Bigin with external applications extends its capabilities and enables customized functionalities. In our last post, we saw how to create a default service connection. Today, we'll see how to create a custom service connection

• Admin asked me for Backend Details when I wanted to verify my ZeptoMail Account

  Please provide the backend details where you will be adding the SMTP/API information of ZeptoMail Who knows what this means?

• Optimising CRM-Projects workflows to manage requests, using Forms as an intermediary

  Is it possible to create a workflow between three apps with traceability between them all? We send information from Zoho CRM Deals over to Zoho Projects for project management and execution. We have used a lookup of sorts to create tasks in the past,

• Conditional fields when converting a Lead and creating a Deal

  Hi, On my Deal page I have a field which has a rule against it. Depending on the value entered, depends on which further fields are displayed. When I convert a Lead and select for a Deal to be created as well, all fields are shown, regardless of the value

• ATE Session on Payment Gateways: Our experts are live now. Post your questions now!

  Hello everyone, Our experts are all excited to answer all your questions related to payment workflows. Please feel free to join this session and learn more about this topic. If you have a query at anytime, please post them here.

• Upload data deleted all Zoho form data that we manage

  Good morning. Let me introduce myself, I'm Iky from Indonesia. I'm experiencing an error or problem using Zoho Forms. I manage Zoho Forms, but I previously encountered an error when I misclicked the delete button in the upload format. It apparently deleted

• ZOHO FORMにURL表示ができない

  初心者です。 ZOHO FORM で宿泊者名簿を作っています。 ゲストが、URLをクリックするとStripeで支払いができるようにURLを表示をしたいのですが、 上手くできません。 やり方が分かる方、ぜひ教えてください。

• Custom module - change from autonumber to name

  I fear I know the answer to this already, but thought I'd ask the question. I created a custom module and instead of having a name as being the primary field, I changed it to an auto-number. I didn't realise that all searches would only show this reference.

• No Automatic Spacing on the Notebook App?

  When I'm adding to notes on the app, I have to add spaces between words myself, rather than it automatically doing it. All my other apps add spacing, so it must be something with Zoho. Is there a setting I need to change, or something else I can do so

• Holidays - Cannot Enter Two Holidays on Same Day

  I have a fairly common setup, where part-time employees receive 1/2 day's pay on a holiday and full-time employees receive a full day's pay. Historically, I've been able to accommodate this by entering two separate holidays, one that covers full-time

• Zoho Bookings and Survey Integration through Flow

  I am trying to set up flows where once an appointment is marked as completed in Zoho Bookings, the applicable survey form would be sent to the customer. Problem is, I cannot customise flows wherein if Consultation A is completed, Survey Form A would be

• Campaigns set up and execution assistance

  Hello Community, Can someone recommend a professional who can assist with the completion of my set up and deployment of Campaigns? Looking for a person or company that is not going to ask for big dollars up-front without a guarantee of performance to

• Card Location in Zobot

  Hello, when using the “Location” card in a codeless builder Zobot, the behavior in WhatsApp is inconsistent. When asking the user to share their location, they can type a message, which will return the message “Sorry, the entered location is invalid.

• Zobot with Plugs

  Hello, I am having a problem with Zobot using Plugs. Here is my current flow: When I run the flow, I should immediately see the messages from the initial cards (Send Message cards), then after running the plug, and finally, see the messages after the

• Kaizen #223 - File Manager in CRM Widget Using ZRC Methods

  Hello, CRM Wizards! Here is what we are improving this week with Kaizen. we will explore the new ZRC (Zoho Request Client) introduced in Widget SDK v1.5, and learn how to use it to build a Related List Widget that integrates with Zoho WorkDrive. It helps

• Remove Powered by Zoho at the footer

  Hi, I've read two past tickets regarding this but it seems that the instructions given are outdated. I assume the layout keeps on changing, which makes it frustrating for me to search high and low. Please let me know how exactly do I do this now? Th

• Error AS101 when adding new email alias

  Hi, I am trying to add apple@(mydomain).com The error AS101 is shown while I try to add the alias.

• No Need To Fix Something That Is Working

  Zoho Books is a great financial tool which helps businesses to become more efficient and productive with day-to-day operations. As such, every change, upgrade, improvement needs to be carefully thought before implemented in the software and I'm sure Zoho

• Creating a task, i can not work out how to get ID for What_Id

  hi From Module A function I map Module B record reference membershipid (ours). I need Module B Zoho ID to create the related to on task. All examples i've seen start with the Zoho ID. void automation.LTM_Assign_Dispute_Task(String membershipid) { try

• Using email "importance" as workflow-criteria

  I'd like to set up a workflow that triggers if an incoming email has been flagged as "high importance" but I'm not seeing any way to do that. Hopefully I'm just missing something obvious...?

• This domain is not allowed to add. Please contact support-as@zohocorp.com for further details

  I am trying to setup the free version of Zoho Mail. When I tried to add my domain, theselfreunion.com I got the error message that is the subject of this Topic. I've read your other community forum topics, and this is NOT a free domain. So what is the

• Next Page