Kaizen 228 - Process Large-Scale Migrated Data Using Catalyst Solutions

Howdy, tech wizards!

This week’s Kaizen explores how Catalyst Solutions in the Zoho CRM Developer Hub help import large volumes of data into Zoho CRM while improving data quality and simplifying the migration process.

Why Catalyst Solutions?

Catalyst Solutions provide a library of independent, ready-to-use backend solutions called CodeLibs. The CRM Bulk Data Processor CodeLib helps handle complex data operations on large datasets. Following are the key benefits:

1. Highly Scalable: Processes large datasets efficiently, supporting up to 200,000 records per API call.
2. Automation Made Simple: Automate complex tasks like segmentation, data cleansing and lead scoring with minimal effort.  
3. Effortless Integration: Comes with pre-configured Catalyst resources such as serverless functions and data stores that integrate directly with Zoho CRM.
   

Business Scenario

Zylker, a manufacturing company, migrated over 3 lakh Lead records into Zoho CRM from spreadsheets. 

Post-migration, the Lead data showed the following inconsistencies and challenges:

1. Phone number field did not include country codes, or contained incorrect country codes. This inconsistency made it difficult for sales teams to contact Leads reliably.
2. Identifying potential Leads for follow-up required manual effort and marketing team has to run generic campaigns without lead segmentation. 

Using workflows before record creation to fix phone numbers for this volume would result in 3 lakh individual workflow executions, which can slow imports and cause partial failures.

Solution Overview

Zylker can use the CRM Bulk Data Processor available in Catalyst Solutions to address these challenges at scale.

The solution works as follows:

1. Lead records are fetched in bulk from Zoho CRM using the Bulk Read API.
2. The fetched data is temporarily stored to evaluate the priority and quality of each record. Based on this evaluation, the records are also segmented accordingly.
3. The Country field value is used to identify and append the correct country code to the phone number.
4. The processed records are updated back into Zoho CRM using Bulk Write API.

All operations are executed using pre-configured Catalyst resources provided by the CodeLib. You can explore the Bulk Data Flow in Catalyst help page to understand how each resource participates in the processing pipeline.

Prerequisites

Before you begin, ensure the following:

1. Log into your Zoho CRM and navigate to Setup > Customization > Modules and Fields > Leads.

Create the following custom Single Line fields to map existing record data in Zoho CRM. 

1. Last Activity Days
2. Web Engagement Score

Refer to the Working with Custom Fields help page for detailed guide. 

2. Create the following custom Single Line fields to store the calculated values during bulk processing:

1. Lead Score Value
2. Lead Segment
3. Sales Priority
4. Data Quality Status
   

3. Go to Setup > Developer Hub > Catalyst Solutions > Zoho CRM Bulk Data Processor. 

4. Click Go to Catalyst and create a Catalyst account if required.

5. Refer to the Catalyst CLI Installation Guide and install it on your local system.

6. Create a new folder on your local system, which will act as a local project directory. Use the following command:

mkdir -p <folder_name>

7. Navigate to the project folder in your terminal and execute the following command: 

catalyst init

8. Follow the prompts to select and initialize your desired project from the Catalyst console. 

When prompted to choose a feature, press Enter and proceed without choosing any feature.

9. Run the following command to install the CRM Bulk Data Processor CodeLib.

catalyst codelib:install https://github.com/catalystbyzoho/codelib-zoho-crm-bulk-processor

Step 1: Create a Cron Job

1. Open the Catalyst Console and navigate to the same project in which you have installed the CRM Bulk Data Processor CodeLib.

2. Go to Cloud Scale > Triggers > Cron and click Create Cron.

3. Fill in the following details:

1. Provide a Name and Description for the cron job. 
2. Select Function and choose BulkJobScheduler from the dropdown in the Target Function field.
3. Enter the following parameters and values:
   
   MODULES -> Leads
   FIELDS_TO_BE_PROCESSED -> Last_Name, Company, Mobile, Country, Designation, No_of_Employees, Web_Engagement_Score, Lead_Score_Value, Lead_Segment, Sales_Priority, Data_Quality_Status
   
   Use GET Modules Metadata API and GET Fields Metadata API to get the API names of the required module and fields.
    
4. Choose the Schedule Type as One Time for our use case.
   

Refer to the Implementing Cron Jobs Help Guide for more details.

Step 2: Configure Zoho CRM API Credentials 

1. Create a self-client application in Zoho's API Console.

2. Generate a Grant Token with the following scopes:

1. ZohoFiles.files.ALL
2. ZohoCRM.bulk.ALL
3. ZohoCRM.modules.ALL
4. ZohoCRM.settings.ALL
5. ZohoCRM.org.ALL
   

3. Follow the OAuth Help Section for a step-by-step guide to generate Access and Refresh tokens for your Zoho CRM organization.

Store all the API credentials securely for later use.

4. To allow Catalyst to access Zoho CRM data, configure the stored API credentials as environment variables in the catalyst-config.json file within the BulkDataProcessor function.

For detailed guidance, refer to the Configure Function Components help guide in CRM Bulk Data Processor. 

These environment variables are used by Catalyst Connectors to create access tokens for establishing secure connection between Zoho CRM and Catalyst.

Step 3: Add Business Logic

1. Go to com.processor.record.ZCRMRecordsProcessorImpl.java file of the BulkDataProcessor in the functions directory.

2. Override the ZCRMRecordsProcessor method with the business logic.

The complete code sample for this use case is available on GitHub for reference.

Following is the major custom logic used here:

Use the GET Fields Metadata API to fetch the API names of the fields required throughout the custom logic.

Phone Number Normalization:

This can be implemented in two stages, Country standardization and Phone Number formatting. 

For country standardization, CountryStandardizerUtil maps different variations of the Country field in records to a single standardized country identifier. These variations may include, 

1. Country Names (India, Unites States, United Kingdom)
2. Abbreviations (Ind, IN, USA, US, UK)
3. Calling Codes (+91, +1, +44)
   

With this, we can ensure different interpretations of the same country are interpreted consistently during the process. 

Once the country is standardized, Phone Numbers are formatted to E.164 format in the PhoneNormalizerUtil. 

It removes all non-numeric characters and converts the Phone input to E.164 format with the respective country codes. 

public class PhoneNormalizerUtil {     private static final Pattern NON_DIGIT = Pattern.compile("[^0-9]");     public static Optional<String> normalizeToE164(String rawPhone, String countryCode) {         if (rawPhone == null || rawPhone.trim().isEmpty()) {             return Optional.empty();         }         String digits = NON_DIGIT.matcher(rawPhone).replaceAll("");         if (rawPhone.startsWith("+") && digits.length() >= 10) {             return Optional.of("+" + digits);         }         switch (countryCode) {             case "IN":                 return normalizeIndia(digits);             case "US":                 return normalizeUS(digits);             case "UK":                 return normalizeUK(digits);             default:                 return Optional.empty();         }     }     private static Optional<String> normalizeIndia(String digits) {         if (digits.length() == 10) {             return Optional.of("+91" + digits);         }         if (digits.startsWith("91") && digits.length() == 12) {             return Optional.of("+" + digits);         }         return Optional.empty();     }     private static Optional<String> normalizeUS(String digits) {         if (digits.length() == 10) {             return Optional.of("+1" + digits);         }         if (digits.startsWith("1") && digits.length() == 11) {             return Optional.of("+" + digits);         }         return Optional.empty();     }     private static Optional<String> normalizeUK(String digits) {         if (digits.startsWith("44") && digits.length() == 12) {             return Optional.of("+" + digits);         }         if (digits.length() == 10) {             return Optional.of("+44" + digits);         }         return Optional.empty();     } }

Lead Priority and Segmentation

The lead priority is calculated using the Title, No of employees, Last Activity Days and Web Engagement Score fields in the record.  

Each field value contributes a predefined number of points:

1. If the job title indicates a senior role such as CEO, CTO, Director, or VP, 25 points are added.
2. If the company size is 200 employees or more, 20 points are added.
3. If the Lead has been active within the last 7 days, 15 points are added.
4. If the web engagement score is 70 or higher, 10 points are added.

The sum of all applicable factors is returned as an integer and based on this score the lead's priority and segmentation is assigned as follows:

1. High (Hot): score ≥ 70
2. Medium (Warm): score ≥ 40
3. Low (Cold): score < 40
   

public static int calculateLeadScore(             String jobTitle,             Integer companySize,             Integer lastActivityDays,             Integer webEngagementScore) {         int score = 0;         // Decision Maker         if (jobTitle != null) {             String title = jobTitle.toLowerCase();             if (title.contains("ceo") || title.contains("cto")                     || title.contains("director") || title.contains("vp")) {                 score += 25;             }         }         // Company Size         if (companySize != null && companySize >= 200) {             score += 20;         }         // Recency         if (lastActivityDays != null && lastActivityDays <= 7) {             score += 15;         }         // Engagement         if (webEngagementScore != null && webEngagementScore >= 70) {             score += 10;         }         return score;     }     public static String deriveSegment(int score) {         if (score >= 70) return "Hot";         if (score >= 40) return "Warm";         return "Cold";     }     public static String derivePriority(String segment) {         switch (segment) {             case "Hot":                 return "High";             case "Warm":                 return "Medium";             default:                 return "Low";         }     }

Lead Quality:

Lead quality is evaluated based on the validity of the Mobile and Country fields in the records.

1. Clean: Both field values are valid.
2. Critical: Both field values are invalid.
3. Needs Review: Only one of the field values is invalid.
   

Write Back to Records:

The processed data is written back to Zoho CRM as follows:

1. Mobile: Formatted phone numbers.
2. Lead Score: Total score calculated from all applicable fields.
3. Lead Segment: Segment derived from the lead score.
4. Sales Priority: Priority assigned based on the lead segment.
5. Data Quality Status: Overall quality of the Lead record.
   

if (normalizedMobile.isPresent()) { data.put("Mobile", normalizedMobile.get()); } data.put("Lead_Score", leadScore); data.put("Lead_Segment", leadSegment); data.put("Sales_Priority", salesPriority); data.put("Data_Quality_Status", dataQualityStatus); }

Step 4: Deploy to Development

Run the following command to deploy your local customizations to the Development Environment of Catalyst.

catalyst deploy

You can find the complete code sample on GitHub for reference.

Try it Out!

With the changes deployed to the Development environment, let us now test the solution.

We hope this Kaizen helps you import large volumes of data into Zoho CRM while improving data quality and lead intelligence using the CRM Bulk Data Processor.

Have questions or suggestions? Drop them in the comments or write to us at  support@zohocrm.com. 

We will circle back to you next Friday with another interesting topic. 

On to Better Building!

--------------------------------------------------------------------------------------------------------------------------

Previous Kaizen: Client Script Support for List Page (Canvas) | Kaizen Collection: Directory

--------------------------------------------------------------------------------------------------------------------------

• Topic Participants

• Kiruthiga G

  

• Sticky Posts

• Kaizen #198: Using Client Script for Custom Validation in Blueprint

  Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

• Kaizen #226: Using ZRC in Client Script

  Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In this

• Kaizen #222 - Client Script Support for Notes Related List

  Hello everyone! Welcome to another week of Kaizen. The final Kaizen post of the year 2025 is here! With the new Client Script support for the Notes Related List, you can validate, enrich, and manage notes across modules. In this post, we’ll explore how

• Kaizen #217 - Actions APIs : Tasks

  Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.

• Kaizen #216 - Actions APIs : Email Notifications

  Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are

• Recent Topics

• Canvas Detail View Related List Sorting

  Hello, I am having an issue finding a way to sort a related list within a canvas detail view. I have sorted the related list on the page layout associated with the canvas view, but that does not transfer to the canvas view. What am I missing?

• Cross Module Filtering – Use Fields from Lookup modules in Custom Views criteria and Advanced Filters

  Hello everyone, Zoho CRM now enables you to achieve deeper filtering of records in a module, using fields of a lookup, thereby enhancing your data management experience manifold. This filtering based on lookup module fields is now available in advanced

• Recent enhancements made to workflow rules

  Greetings, I hope all of you are doing well. We're happy to announce a few recent enhancements we've made to Bigin's workflows. We'll go over each one in detail, but for now these updates can be broadly categorized into two parts: Enhancements to workflow

• How do I get my cell height to auto adjust?

  I'm evaluating Zoho Sheet as a replacement for Google Spreadsheet. Using latest version of Safari, and experiencing the same issues in Chrome. A few bugs/things I can't figure out: Auto adjusting cell height when importing from Google Docs or creating a new document Turning off the auto date formatting for X.Y.Z where they are all integers.. (This is extremely important, as our requirements are written as outlines.. "X.Y.Z" corresponds to a requirement ID.. which is then turning into a date) The

• Marketing Tip #36: Optimize your online store for AI search

  Search is changing fast. Along with traditional search engines, AI-powered search and recommendation experiences are becoming a bigger part of how customers discover products online. Stores that are well-organized, descriptive, and consistent are more

• Create a "My saved custom themes" section in Zoho Forms

  Hi! I created lots of forms for my company that we embed on the website with a custon design. It is a pain not to be able to save a custom layout as a template and just not to have to do it again! So could you add a gallery like : "My saved themes" ?

• Automating CRM backup storage?

  Hi there, We've recently set up automatic backups for our Zoho CRM account. We were hoping that the backup functionality would not require any manual work on our end, but it seems that we are always required to download the backups ourselves, store them,

• Introducing Spotlight Forms

  Hey form builders! If someone opens your form, sees the wall of fields ahead, and quietly closes the tab. It may not be because the questions were hard. It could be because the experience felt like too much. Which is why we have now introduced a new form

• BUG and HANGUP - Add Row with Fields DOUBLES the amount of rows instead of Adding Just 1 Row

  As it says in the title, there is a bug with forms generated with Zoho Writer where the Add Row With Fields ends up DOUBLING the amount of rows instead of Adding just 1 row.

• Forms cannot be accessed.

  https://forms.zoho.com/ is not available, please help to fix

• Digest Mai - Un résumé de ce qui s'est passé le mois dernier sur Community

  Chers utilisateurs, Un nouveau mois se termine au sein de la communauté Zoho France. Découvrons ensemble un résumé des activités du mois de mai. Quatre ans après son lancement, Zoho Marketing Plus continue d’évoluer avec une ambition claire : offrir aux

• [URGENT] Cannot access Functions tab in CRM

  Navigating to /settings/functions/myFunctions gives this error message: "Sorry, something went wrong. Please try again later." I raised this issue with Zoho Support on Monday (3 days ago) but have not heard back. I'm sure it's clear how important it is

• Important changes to note for Zoho Sign users in Saudi Arabia

  Dear Zoho Sign users, If your Zoho Sign account is hosted in our Saudi Arabia data centre, here's an important update on digital signature certificates in the Kingdom of Saudi Arabia. What's changing Going forward, the "Sign via Zoho Sign" option will

• Zoho Desk MCP doesn't expose all functions

  Hello, I'd like to be able to draft (rather than send) ticket replies using Claude Cowork. However, the Zoho Desk MCP doesn't currently offer that, despite it being available in the API (https://desk.zoho.com/DeskAPIDocument#Threads#Threads_DraftEmailReply).

• Number of Reopn

  Hi Zoho, Is there any appropriate API call for This URL "http://support.zoho.com/api/v1/dashboards/reopenedTickets?...." what I thought is the resulting output of this call has data for number of reopen... "https://desk.zoho.com/api/v1/tickets/" + Ticket_ID

• Can I use merge tags with a conditional clause?

  Similar to Mailchimp's "IF / ELSE / ENDIF" functionality, I want to be able to display a sentence with a personalised field only if that merge tag is not empty for the given record. Is this possible on Zoho Campaigns? See http://kb.mailchimp.com/article/how-do-conditional-smart-merge-tags-work for how it works on Mailchimp. Thanks Phil

• Question - why no way to input a 'tool description' and 'tree of 'tools'

  Every business is different, with different business processes. To be truly useful Zoho MCP needs to have user editable tool descriptions (or ruleset) and a 'tool tree' so that the LLM is context aware when being used. For example, the tool description

• Personalize your booking pages with Custom CSS

  Greetings from the Zoho Bookings team! We’re introducing Custom CSS for Zoho Bookings, designed to give you complete control over the look and feel of your booking pages. With this new feature, you can upload your own CSS file to customize colors, fonts,

• CRM API v9?

  v8 has been around for a while. any plans / estimates for v9?

• Calendar report with order options and more quick view templates

  I think many of us regularly work with calendar-style reports. It would be great to be able to customize the quick view with new templates and have options to sort the entries for each day of the calendar by different criteria. I think this is an interesting

• Kaizen #125 Manipulating Multi-Select Lookup fields (MxN) using Zoho CRM APIs

  Hello everyone! Welcome back to another week of Kaizen. In last week's post in the Kaizen series, we discussed how subforms work in Zoho CRM and how to manipulate subform data using Zoho CRM APIs. In this post, we will discuss how to manipulate a multi-select

• Zoho Books and TRAINING SALES receipt label for eTims?

  Hi, Can Zoho Books implement TRAINING SALES receipts and push them to eTims for test? In other words how can we send to Zoho or even create in Zoho training mode invoices and TEST the workflow POS>Zoho Books>eTims and back without actually registering

• How do I post a new question in Zoho Community forums?

  Hi everyone, I’m new to the Zoho Community and I’m trying to figure out how to properly create and publish a new topic in the forum. When I visit the community page, I can’t clearly find the option like “Add Topic” or “Post Question.” Could someone guide

• Any Zoho Books users in the Kenyan Hospitality industry? How to set service items for eTims?

  Hello, We are opening a coffee shop in Kenya and would like to know if there are any Zoho books users in hospitality service industry in Kenya? We would love to know: 1. how do you cope with the absence of the mandatory Tourism Levy 2% tax option? 2.

• Cloning Item With Images Or The Option With Images

  Hello, when I clone an item, I expect the images to carry over to the cloned item, however this is not the case in Inventory. Please make it possible for the images to get cloned or at least can we get a pop up asking if we want to clone the images as

• Zoho Books | Product updates | May 2026

  Hello users, We're back with the latest updates and enhancements we've rolled out in Zoho Books. From sales tax automation to scanning receipts for free, explore the updates designed to upgrade your bookkeeping experience. Sales Tax Automation [US & Canada

• Does Zoho Learn integrate with Zoho Connect,People,Workdrive,Project,Desk?

  Can we propose Zoho LEarn as a centralised Knowledge Portal tool that can get synched with the other Zoho products and serve as a central Knowledge repository?

• Request to Update Billing Information and Payment Method

  Hello, I’m using Zoho and I would like to update the billing information and change the payment card to our company card. Could you please let me know how I can do this? Thank you in advance for your help.

• Zoho Analytics "Esc" key problem

  I frequently use the Escape (Esc) key while building dashboards, reports, and writing SQL queries. Since the recent updates to Zoho Analytics, the Esc key no longer behaves as expected. When writing SQL queries, pressing Esc to dismiss a suggestion now

• Zoho Analytics Filter Bug

  I encountered a bug where typing the letter "A" in the drop-down filter of a table or query table causes the drop-down to close unexpectedly. For example, when typing "Today", the drop-down list closes as soon as "a" is entered. I tested this on another

• Using Email Triggers on Zoho Flow

  Hello, I'm sending the email to create the variables as this article says: https://help.zoho.com/portal/en/kb/flow/user-guide/create-a-flow/articles/email-trigger#How_email_trigger_works But the collection of the variables only seems to work when the

• How to customize the "Placeholder Text" separately from the "Field Label" on the Booking Form?

  Hi, I am currently customizing the Booking Form for one of my Workspaces in Zoho Bookings, and I need some help adjusting a custom text field. Right now, when I create a custom text field, the gray "placeholder text" inside the text box automatically

• Why don't Zia agents support file uploads?

  I am trying to build a Zia Agent that allows uploading of a PDF file and uses the GLM5 model to process it and extract information. But agents.zoho.com has no way to enable file uploads on the agent. Additionally, GLM5 based agents keep outputting their

• Pasting Images in Zoho Desk ignores cursor location

  My team has reported an issue which started recently where when we paste an image into a new or existing reply or comment, the pasted image seems to ignore the current cursor location instead paste itself at the last character present in the reply/comment,

• 'Pinned' notes feature of a pipeline record

  Hi team, Could you please implement a feature which will allow users to pin different notes so that they will appear at the very top of the notes tab in a pipeline record. Sometimes we have a wide range of notes on a record which means more important

• Announcing new features in Trident for Mac (1.37.0)

  Hello everyone! We’re excited to introduce the latest updates to Trident, which are designed to take workplace communication to the next level. Let’s dive into the details. Import EML archives directly into Trident. You can now import EML archives into

• #10 Bill While You Sleep

  A consultant is reviewing last month's work. Client meetings? Done. Deliverables? Sent. Support requests? Resolved. Then they realize something. "I have completed the work... but I haven't billed the client yet." The work was completed. The client was

• Access images from form submission in power automate

  Images from form submission show up as links in power automate. How do I access the image data?

• Kaizen #245 - Real Time Signal Alerts for High-Value Abandoned Checkouts

  Howdy, Tech Wizards! Welcome back to another week of Kaizen. In this post, we will build a real-time abandoned checkout notification system using Stripe, Zoho CRM Functions, Sales Signals, and Widgets. When a customer abandons a high-value purchase, Zoho

• Unable to attach Fillable File Upload field to Merge Template ever since UI update

  Ever since the new UI update, the field for Attachments for sending document for Signing in Writer has had an issue where trying to add a Fillable item in the Attachment field ends up always becoming a "Choose a File From Drive" option instead. No matter

• Next Page