Kaizen 228 - Process Large-Scale Migrated Data Using Catalyst Solutions

Howdy, tech wizards!

This week’s Kaizen explores how Catalyst Solutions in the Zoho CRM Developer Hub help import large volumes of data into Zoho CRM while improving data quality and simplifying the migration process.

Why Catalyst Solutions?

Catalyst Solutions provide a library of independent, ready-to-use backend solutions called CodeLibs. The CRM Bulk Data Processor CodeLib helps handle complex data operations on large datasets. Following are the key benefits:

1. Highly Scalable: Processes large datasets efficiently, supporting up to 200,000 records per API call.
2. Automation Made Simple: Automate complex tasks like segmentation, data cleansing and lead scoring with minimal effort.  
3. Effortless Integration: Comes with pre-configured Catalyst resources such as serverless functions and data stores that integrate directly with Zoho CRM.
   

Business Scenario

Zylker, a manufacturing company, migrated over 3 lakh Lead records into Zoho CRM from spreadsheets. 

Post-migration, the Lead data showed the following inconsistencies and challenges:

1. Phone number field did not include country codes, or contained incorrect country codes. This inconsistency made it difficult for sales teams to contact Leads reliably.
2. Identifying potential Leads for follow-up required manual effort and marketing team has to run generic campaigns without lead segmentation. 

Using workflows before record creation to fix phone numbers for this volume would result in 3 lakh individual workflow executions, which can slow imports and cause partial failures.

Solution Overview

Zylker can use the CRM Bulk Data Processor available in Catalyst Solutions to address these challenges at scale.

The solution works as follows:

1. Lead records are fetched in bulk from Zoho CRM using the Bulk Read API.
2. The fetched data is temporarily stored to evaluate the priority and quality of each record. Based on this evaluation, the records are also segmented accordingly.
3. The Country field value is used to identify and append the correct country code to the phone number.
4. The processed records are updated back into Zoho CRM using Bulk Write API.

All operations are executed using pre-configured Catalyst resources provided by the CodeLib. You can explore the Bulk Data Flow in Catalyst help page to understand how each resource participates in the processing pipeline.

Prerequisites

Before you begin, ensure the following:

1. Log into your Zoho CRM and navigate to Setup > Customization > Modules and Fields > Leads.

Create the following custom Single Line fields to map existing record data in Zoho CRM. 

1. Last Activity Days
2. Web Engagement Score

Refer to the Working with Custom Fields help page for detailed guide. 

2. Create the following custom Single Line fields to store the calculated values during bulk processing:

1. Lead Score Value
2. Lead Segment
3. Sales Priority
4. Data Quality Status
   

3. Go to Setup > Developer Hub > Catalyst Solutions > Zoho CRM Bulk Data Processor. 

4. Click Go to Catalyst and create a Catalyst account if required.

5. Refer to the Catalyst CLI Installation Guide and install it on your local system.

6. Create a new folder on your local system, which will act as a local project directory. Use the following command:

mkdir -p <folder_name>

7. Navigate to the project folder in your terminal and execute the following command: 

catalyst init

8. Follow the prompts to select and initialize your desired project from the Catalyst console. 

When prompted to choose a feature, press Enter and proceed without choosing any feature.

9. Run the following command to install the CRM Bulk Data Processor CodeLib.

catalyst codelib:install https://github.com/catalystbyzoho/codelib-zoho-crm-bulk-processor

Step 1: Create a Cron Job

1. Open the Catalyst Console and navigate to the same project in which you have installed the CRM Bulk Data Processor CodeLib.

2. Go to Cloud Scale > Triggers > Cron and click Create Cron.

3. Fill in the following details:

1. Provide a Name and Description for the cron job. 
2. Select Function and choose BulkJobScheduler from the dropdown in the Target Function field.
3. Enter the following parameters and values:
   
   MODULES -> Leads
   FIELDS_TO_BE_PROCESSED -> Last_Name, Company, Mobile, Country, Designation, No_of_Employees, Web_Engagement_Score, Lead_Score_Value, Lead_Segment, Sales_Priority, Data_Quality_Status
   
   Use GET Modules Metadata API and GET Fields Metadata API to get the API names of the required module and fields.
    
4. Choose the Schedule Type as One Time for our use case.
   

Refer to the Implementing Cron Jobs Help Guide for more details.

Step 2: Configure Zoho CRM API Credentials 

1. Create a self-client application in Zoho's API Console.

2. Generate a Grant Token with the following scopes:

1. ZohoFiles.files.ALL
2. ZohoCRM.bulk.ALL
3. ZohoCRM.modules.ALL
4. ZohoCRM.settings.ALL
5. ZohoCRM.org.ALL
   

3. Follow the OAuth Help Section for a step-by-step guide to generate Access and Refresh tokens for your Zoho CRM organization.

Store all the API credentials securely for later use.

4. To allow Catalyst to access Zoho CRM data, configure the stored API credentials as environment variables in the catalyst-config.json file within the BulkDataProcessor function.

For detailed guidance, refer to the Configure Function Components help guide in CRM Bulk Data Processor. 

These environment variables are used by Catalyst Connectors to create access tokens for establishing secure connection between Zoho CRM and Catalyst.

Step 3: Add Business Logic

1. Go to com.processor.record.ZCRMRecordsProcessorImpl.java file of the BulkDataProcessor in the functions directory.

2. Override the ZCRMRecordsProcessor method with the business logic.

The complete code sample for this use case is available on GitHub for reference.

Following is the major custom logic used here:

Use the GET Fields Metadata API to fetch the API names of the fields required throughout the custom logic.

Phone Number Normalization:

This can be implemented in two stages, Country standardization and Phone Number formatting. 

For country standardization, CountryStandardizerUtil maps different variations of the Country field in records to a single standardized country identifier. These variations may include, 

1. Country Names (India, Unites States, United Kingdom)
2. Abbreviations (Ind, IN, USA, US, UK)
3. Calling Codes (+91, +1, +44)
   

With this, we can ensure different interpretations of the same country are interpreted consistently during the process. 

Once the country is standardized, Phone Numbers are formatted to E.164 format in the PhoneNormalizerUtil. 

It removes all non-numeric characters and converts the Phone input to E.164 format with the respective country codes. 

public class PhoneNormalizerUtil {     private static final Pattern NON_DIGIT = Pattern.compile("[^0-9]");     public static Optional<String> normalizeToE164(String rawPhone, String countryCode) {         if (rawPhone == null || rawPhone.trim().isEmpty()) {             return Optional.empty();         }         String digits = NON_DIGIT.matcher(rawPhone).replaceAll("");         if (rawPhone.startsWith("+") && digits.length() >= 10) {             return Optional.of("+" + digits);         }         switch (countryCode) {             case "IN":                 return normalizeIndia(digits);             case "US":                 return normalizeUS(digits);             case "UK":                 return normalizeUK(digits);             default:                 return Optional.empty();         }     }     private static Optional<String> normalizeIndia(String digits) {         if (digits.length() == 10) {             return Optional.of("+91" + digits);         }         if (digits.startsWith("91") && digits.length() == 12) {             return Optional.of("+" + digits);         }         return Optional.empty();     }     private static Optional<String> normalizeUS(String digits) {         if (digits.length() == 10) {             return Optional.of("+1" + digits);         }         if (digits.startsWith("1") && digits.length() == 11) {             return Optional.of("+" + digits);         }         return Optional.empty();     }     private static Optional<String> normalizeUK(String digits) {         if (digits.startsWith("44") && digits.length() == 12) {             return Optional.of("+" + digits);         }         if (digits.length() == 10) {             return Optional.of("+44" + digits);         }         return Optional.empty();     } }

Lead Priority and Segmentation

The lead priority is calculated using the Title, No of employees, Last Activity Days and Web Engagement Score fields in the record.  

Each field value contributes a predefined number of points:

1. If the job title indicates a senior role such as CEO, CTO, Director, or VP, 25 points are added.
2. If the company size is 200 employees or more, 20 points are added.
3. If the Lead has been active within the last 7 days, 15 points are added.
4. If the web engagement score is 70 or higher, 10 points are added.

The sum of all applicable factors is returned as an integer and based on this score the lead's priority and segmentation is assigned as follows:

1. High (Hot): score ≥ 70
2. Medium (Warm): score ≥ 40
3. Low (Cold): score < 40
   

public static int calculateLeadScore(             String jobTitle,             Integer companySize,             Integer lastActivityDays,             Integer webEngagementScore) {         int score = 0;         // Decision Maker         if (jobTitle != null) {             String title = jobTitle.toLowerCase();             if (title.contains("ceo") || title.contains("cto")                     || title.contains("director") || title.contains("vp")) {                 score += 25;             }         }         // Company Size         if (companySize != null && companySize >= 200) {             score += 20;         }         // Recency         if (lastActivityDays != null && lastActivityDays <= 7) {             score += 15;         }         // Engagement         if (webEngagementScore != null && webEngagementScore >= 70) {             score += 10;         }         return score;     }     public static String deriveSegment(int score) {         if (score >= 70) return "Hot";         if (score >= 40) return "Warm";         return "Cold";     }     public static String derivePriority(String segment) {         switch (segment) {             case "Hot":                 return "High";             case "Warm":                 return "Medium";             default:                 return "Low";         }     }

Lead Quality:

Lead quality is evaluated based on the validity of the Mobile and Country fields in the records.

1. Clean: Both field values are valid.
2. Critical: Both field values are invalid.
3. Needs Review: Only one of the field values is invalid.
   

Write Back to Records:

The processed data is written back to Zoho CRM as follows:

1. Mobile: Formatted phone numbers.
2. Lead Score: Total score calculated from all applicable fields.
3. Lead Segment: Segment derived from the lead score.
4. Sales Priority: Priority assigned based on the lead segment.
5. Data Quality Status: Overall quality of the Lead record.
   

if (normalizedMobile.isPresent()) { data.put("Mobile", normalizedMobile.get()); } data.put("Lead_Score", leadScore); data.put("Lead_Segment", leadSegment); data.put("Sales_Priority", salesPriority); data.put("Data_Quality_Status", dataQualityStatus); }

Step 4: Deploy to Development

Run the following command to deploy your local customizations to the Development Environment of Catalyst.

catalyst deploy

You can find the complete code sample on GitHub for reference.

Try it Out!

With the changes deployed to the Development environment, let us now test the solution.

We hope this Kaizen helps you import large volumes of data into Zoho CRM while improving data quality and lead intelligence using the CRM Bulk Data Processor.

Have questions or suggestions? Drop them in the comments or write to us at  support@zohocrm.com. 

We will circle back to you next Friday with another interesting topic. 

On to Better Building!

--------------------------------------------------------------------------------------------------------------------------

Previous Kaizen: Client Script Support for List Page (Canvas) | Kaizen Collection: Directory

--------------------------------------------------------------------------------------------------------------------------

• Topic Participants

• Kiruthiga G

  

• Sticky Posts

• Announcing the new SKILL.md for Zoho CRM and the updated OAS repository!

  We are introducing a new zoho-crm skill to make working with Zoho CRM Developer tools (like APIs, functions, widgets, client scripts, queries etc) easier and faster, with the help of AI in your preferred AI harness like Claude Code, Codex, Cursor, VSCode

• Kaizen #256 - Build an Arrival Readiness Web Tab in Zoho CRM

  Hi everyone! Welcome back to the Kaizen series! In the post, we discuss a use case in hospitality industry: how an Arrival Readiness web tab widget can be used to let reception staff identify and resolve issues before arrival of guests. Use case In the

• Kaizen #198: Using Client Script for Custom Validation in Blueprint

  Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

• Kaizen #226: Using ZRC in Client Script

  Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In this

• Kaizen #222 - Client Script Support for Notes Related List

  Hello everyone! Welcome to another week of Kaizen. The final Kaizen post of the year 2025 is here! With the new Client Script support for the Notes Related List, you can validate, enrich, and manage notes across modules. In this post, we’ll explore how

• Recent Topics

• Check out in Meetings

  Why there is no check out in Meetings of Zoho CRM, very difficult to track

• Admin Logging in as another User

  How can a Super Admin login as another user. For example, I have a sales rep that is having issues with their Accounts and I want to view their Zoho Account with out having to do a GTM and sharing screens. Latest Update (27th April 2026): With the early

• How do I get to "Admin Panel"

  I just started a Cliq account. I am the sole administrator. As I read the documentation, I keep seeing reference to an "Admin Panel". I see no way to access this from my Cliq account. How do I access this panel?

• 【初開催！ オンライン】Zoho Campaigns メール配信の基礎・活用勉強会を開催します！ 10/15 参加無料

  ユーザーの皆さま、こんにちは。 Zoho コミュニティグループの中野です。 Zoho ユーザーコミュニティ初となる、メールマーケティングをテーマにしたオンライン勉強会を開催します！ ▶︎イベント詳細・参加登録はこちら 今回取り上げるサービスは「Zoho Campaigns」です。 「Zoho CRMとZoho Campaignsのどちらからメールを配信すればよいか分からない」 「一斉配信はできるようになったけれど、現在のリスト管理や設定が正しいか不安」 「メールを送るだけで終わり、配信後の改善につなげられていない」

• Lookups and Custom Modules

  I created a Custom Module that I'm calling "Item Purchasing Group". This module will store information for groups of Items that must be purchased together (including minimum overall quantities and/or values, etc.). This will have a one-to-many relationship

• Cliq iOS can't see shared screen

  Hello, I had this morning a video call with a colleague. She is using Cliq Desktop MacOS and wanted to share her screen with me. I'm on iPad. I noticed, while she shared her screen, I could only see her video, but not the shared screen... Does Cliq iOS is able to display shared screen, or is it somewhere else to be found ? Regards

• Introducing Smart Fields in Zoho Forms

  Hello form builders! We are excited to introduce Smart Fields in Zoho Forms - a faster way to add calculations to your forms without writing formulas. Instead of creating multiple fields and configuring complex calculations yourself, you can now drag

• Checkbox not displaying correctly in subform

  Greetings: I have shared my app (wv) with support.  I am having problems with a checkbox field not displaying correctly in a subform.  The main form is called volunteers and the subform is called volunteers_subform.  The field in question is called roles. When you access the subform by itself the checkbox lookup displays as checkboxes.  But when you view it in the main volunteers form, it displays as a multi-select dropdown instead.   I have attached jpegs showing the difference.   Any thoughts?

• Unbundle feature for composite items

  We receive composite items from our vendors and sell them either individually or create other composite items out of them. So, there is a lot of bundling and unbundling involved with our composite items. Previously, this feature was supported in form

• The reason I switched away from Zoho Notebook

  My main reason for switching to Zoho was driven by three core principles: moving away from US-based products, keeping my data within India as much as possible, and supporting Indian companies. With that intent, I’ve been actively de-Googling my digital

• Valid characters for use in email addresses using Zoho's apps/APIs

  We have found an issue with the + sign character in zoho subs - we are allowed to create a customer with an email address that has a plus sign but the API doesn't allow the + so the billing portal is not created. We are going to prevent users from using

• Expand Zoho Analytics Spatial Capabitlities

  Dear Analytics team Thanks for the great product - I'm a big fan on Analytics and the monthly release highlights is one of my favourite emails to receive. I'd like to request expansion and improvement of Zoho Analytics spatial data capabilities, namely:

• Automate customer follow-ups with Sequences in Bigin

  Greetings, We hope you're all doing well! We're happy to announce a new feature that we've added to Bigin. Following up with every lead at the right time can be challenging, especially when your team is managing multiple conversations across emails, calls,

• Dashboards for Customers

  Is it possible to build dashboards for each customers in the community for their tickets?

• Displaying only unread tickets in ticket view

  Hello, I was wondering if someone might be able to help me with this one. We use filters to display our ticket list, typically using a saved filter which displays the tickets which are overdue or due today. What I'd really like is another filter that

• Adding Custom Status Options in Work Order Action Menu

  Hello FSM Team, We would like to inquire if it is possible to add more custom options in the Work Order action/status dropdown (e.g., Cancel, Terminate, Non-Billable, Void, etc.). Currently, the available options are limited, and we are unable to customize

• Can we have an automated TDS Receivable Recon with Form 26AS of Income Tax Portal ?

  Can we have an automated TDS Receivable Recon with Form 26AS of Income Tax Portal ?

• Tag Limi?

  is there any way to create more than 20 tags? 

• prefill_data for Creator + Merge & Sign

  Hello, Is there an option to use prefill_data for signer fields from creator? I see the option for CRM but it will not work with creator? I have tried passing the value in the signer data, and that also does not work. Alternatively, is there a way to

• Delug script

  I have been looking at auto-update a amount (home currency) field from another module. Zoho native multicurrency was used in the other module (we have 4 here). Custom script was input with no error, but the field was not updated on trigger. Script as

• Can I Build a Gaming Website Using Zoho? Need a Quick Roadmap

  Hi Dear Members, I want to create a gaming website using Zoho. Is it possible? My goal is to build a sleek, unique, easy-to-navigate, and fast-loading website for a gaming project. I want the site to look professional, user-friendly, and optimized for

• Remove Due Date from Statements

  Is it possible to remove the Due Date on the Invoice transation line on statements? I have figured work arounds to get the Invoice detail on the Statement but cannot work out haw to simplify the stament by removing the "- due on xx/xx/2026" - it really

• What Should an Agent-Native Project Workspace Look Like

  AI is moving beyond assistants that simply answer questions. With tools like MCP, agents can now interact directly with project data and perform actions. That makes me wonder: what should an agent-native project workspace actually look like? For me, it

• unable to join meeting due to speaker issue

  unable to join meeting due to speaker issue

• How to add formatting in zoho.cliq.postToUser(...) message?

  In a CRM Deluge function, I'm trying to use the message formatting guidelines given here: https://www.zoho.com/deluge/help/cliq/posting-to-zoho-cliq.html#message-formats My message is: message: #Title text. The result in Cliq is: #Title text. (no large

• Marketing Tip #50: Sell digital products and use them to grow your store

  Physical products need packaging, shipping, and stock management. Digital products need none of that. Once created, they can be sold an unlimited number of times, delivered instantly, and never go out of stock. What counts as a digital product? A digital

• Turn off Mobile Optimize

  Hello is it possible to stop the automatic mobile optimization of my site as it looks much better on a mobile in full site format rather than mobile format. Thanks

• How to select from pricebook when creating a salesorder or quote

  I am creating a sales order and when selecting the Products I do not see any where to select from pricebooks. How do i associate this to my orders?

• API - Available Stock Definitions

  Okay, Zoho team... your copywriters fell down on the job for this one :) I think these warrant a bit more explanation as to what they include and what they don't.

• Cross-application Deluge calls fail when application link name starts with a number

  I encountered an issue when calling a custom function in another Zoho Creator application within the same account. The target application’s URL/application link name began with a number, similar to: 27_example_application The following cross-application

• Transformez vos webinaires en opérations automatisées

  Organiser un webinaire, c’est bien plus que passer en direct. Il faut gérer les inscriptions, suivre les participants, envoyer les enregistrements, mettre à jour le CRM… et s’assurer que chaque étape se déroule au bon moment. L’automatisation des workflows

• Marketing Tip #1: Optimize item titles for SEO

  Your item title is the first thing both Google and shoppers notice. Instead of a generic “Leather Bag,” go for something detailed like “Handcrafted Leather Laptop Bag – Durable & Stylish.” This helps your items rank better in search results and instantly

• How can I filter inactive/disable agent tickets?

  Hello, We have an user-agent that left the company and we inactive/disabled his agent. So, now I can no longer filter or search tickets that he is the current owner, or even create a rule to reassign tickets to another person when the ticket is reopen.

• How do i move multiple tickets to a different department?

  Hello, i have several tickets that have been assigned to the wrong department.  I am talking about hundreds of automatically generated ones that come from a separate system. How can i select them all at once to move them to another department in one go? I can select them in "unsassigned open tickets view" but i can't find a "move to another department" option. I also can't seem to assign multiple tickets to the same agent in that same view. Could somebody advice?

• Add Reauthentication Option for Zoho Bug Tracker Integration in Zoho Desk

  Hello Zoho Desk Team, We hope you're doing well. We would like to request an enhancement to the Zoho Bug Tracker integration within Zoho Desk. Current Limitation: At the moment, there is no option to reauthenticate the Zoho Bug Tracker integration in

• Create Package From A Picklist

  Dear Zoho, Can it be made possible to create a package from a picklist? The reason our company makes a picklist is for that to become a package Our sales orders have 600-1000 items I hope that makes it clear that it's hard to delete 990 items when we

• Migrating from Zoho Checkout to Zoho Billing

  I have an active Zoho Checkout account where I'm charging customers for subscriptions. Now I'm looking to upgrade and only use Zoho BIlling, in which my account in on read only mode at the moment. I can see is also tracking the customers and its payments.

• Step up your lead generation game with online forms

  Hello fellow form builders,   Generating leads for your sales team gets easier when you integrate your CRM with online forms. To help show you how to use forms as part of your lead generation strategy, we have exclusive tips and tricks on how to collect, update and prioritize lead information in Zoho CRM based on form responses.  1. Create new leads If CRM software is designed to help you handle your business easily, that means getting data into it should also be easy. Check out the article on how

• API - Bank Accounts | Reconciliation Discrepancy

  Unless I'm reading it wrong, the documentation for the Bank Reconciliations API is inconsistent with itself. https://www.zoho.com/books/api/v3/bank-accounts/#create-a-bank-reconciliation The text definition says that the transaction_id should be a string,

• another little issue

  So to get over showing prices with VAT we used the price lists and made the prices inclusive. The problem is now on the items it wants to show the orginal retail value which makes no sense. Ideas please

• Next Page