Kaizen 228 - Process Large-Scale Migrated Data Using Catalyst Solutions
Howdy, tech wizards!
This week’s Kaizen explores how Catalyst Solutions in the Zoho CRM Developer Hub help import large volumes of data into Zoho CRM while improving data quality and simplifying the migration process.
Why Catalyst Solutions?
Catalyst Solutions provide a library of independent, ready-to-use backend solutions called CodeLibs. The CRM Bulk Data Processor CodeLib helps handle complex data operations on large datasets. Following are the key benefits:
- Highly Scalable: Processes large datasets efficiently, supporting up to 200,000 records per API call.
- Automation Made Simple: Automate complex tasks like segmentation, data cleansing and lead scoring with minimal effort.
- Effortless Integration: Comes with pre-configured Catalyst resources such as serverless functions and data stores that integrate directly with Zoho CRM.
Business Scenario
Zylker, a manufacturing company, migrated over 3 lakh Lead records into Zoho CRM from spreadsheets.
Post-migration, the Lead data showed the following inconsistencies and challenges:
- Phone number field did not include country codes, or contained incorrect country codes. This inconsistency made it difficult for sales teams to contact Leads reliably.
- Identifying potential Leads for follow-up required manual effort and marketing team has to run generic campaigns without lead segmentation.
Using workflows before record creation to fix phone numbers for this volume would result in 3 lakh individual workflow executions, which can slow imports and cause partial failures.
Solution Overview
Zylker can use the CRM Bulk Data Processor available in Catalyst Solutions to address these challenges at scale.
The solution works as follows:
- Lead records are fetched in bulk from Zoho CRM using the Bulk Read API.
- The fetched data is temporarily stored to evaluate the priority and quality of each record. Based on this evaluation, the records are also segmented accordingly.
- The Country field value is used to identify and append the correct country code to the phone number.
- The processed records are updated back into Zoho CRM using Bulk Write API.
All operations are executed using pre-configured Catalyst resources provided by the CodeLib. You can explore the Bulk Data Flow in Catalyst help page to understand how each resource participates in the processing pipeline.
Prerequisites
Before you begin, ensure the following:
1. Log into your Zoho CRM and navigate to Setup > Customization > Modules and Fields > Leads.
Create the following custom Single Line fields to map existing record data in Zoho CRM.
- Last Activity Days
- Web Engagement Score
Refer to the Working with Custom Fields help page for detailed guide.
2. Create the following custom Single Line fields to store the calculated values during bulk processing:
- Lead Score Value
- Lead Segment
- Sales Priority
- Data Quality Status
3. Go to Setup > Developer Hub > Catalyst Solutions > Zoho CRM Bulk Data Processor.
4. Click Go to Catalyst and create a Catalyst account if required.
5. Refer to the Catalyst CLI Installation Guide and install it on your local system.
6. Create a new folder on your local system, which will act as a local project directory. Use the following command:
mkdir -p <folder_name> |
7. Navigate to the project folder in your terminal and execute the following command:
catalyst init |
8. Follow the prompts to select and initialize your desired project from the Catalyst console.
When prompted to choose a feature, press Enter and proceed without choosing any feature.
9. Run the following command to install the CRM Bulk Data Processor CodeLib.
catalyst codelib:install https://github.com/catalystbyzoho/codelib-zoho-crm-bulk-processor |
Step 1: Create a Cron Job
1. Open the Catalyst Console and navigate to the same project in which you have installed the CRM Bulk Data Processor CodeLib.
2. Go to Cloud Scale > Triggers > Cron and click Create Cron.
3. Fill in the following details:
- Provide a Name and Description for the cron job.
- Select Function and choose BulkJobScheduler from the dropdown in the Target Function field.
- Enter the following parameters and values:
MODULES -> Leads
FIELDS_TO_BE_PROCESSED -> Last_Name, Company, Mobile, Country, Designation, No_of_Employees, Web_Engagement_Score, Lead_Score_Value, Lead_Segment, Sales_Priority, Data_Quality_Status
Use GET Modules Metadata API and GET Fields Metadata API to get the API names of the required module and fields.
- Choose the Schedule Type as One Time for our use case.
Refer to the Implementing Cron Jobs Help Guide for more details.
Step 2: Configure Zoho CRM API Credentials
1. Create a self-client application in Zoho's API Console.
2. Generate a Grant Token with the following scopes:
- ZohoFiles.files.ALL
- ZohoCRM.bulk.ALL
- ZohoCRM.modules.ALL
- ZohoCRM.settings.ALL
- ZohoCRM.org.ALL
3. Follow the OAuth Help Section for a step-by-step guide to generate Access and Refresh tokens for your Zoho CRM organization.
Store all the API credentials securely for later use.
4. To allow Catalyst to access Zoho CRM data, configure the stored API credentials as environment variables in the catalyst-config.json file within the BulkDataProcessor function.
For detailed guidance, refer to the Configure Function Components help guide in CRM Bulk Data Processor.
These environment variables are used by Catalyst Connectors to create access tokens for establishing secure connection between Zoho CRM and Catalyst.
Step 3: Add Business Logic
1. Go to com.processor.record.ZCRMRecordsProcessorImpl.java file of the BulkDataProcessor in the functions directory.
2. Override the ZCRMRecordsProcessor method with the business logic.
The complete code sample for this use case is available on GitHub for reference.
Following is the major custom logic used here:
Use the GET Fields Metadata API to fetch the API names of the fields required throughout the custom logic.
Phone Number Normalization:
This can be implemented in two stages, Country standardization and Phone Number formatting.
For country standardization, CountryStandardizerUtil maps different variations of the Country field in records to a single standardized country identifier. These variations may include,
- Country Names (India, Unites States, United Kingdom)
- Abbreviations (Ind, IN, USA, US, UK)
- Calling Codes (+91, +1, +44)
With this, we can ensure different interpretations of the same country are interpreted consistently during the process.
Once the country is standardized, Phone Numbers are formatted to E.164 format in the PhoneNormalizerUtil.
It removes all non-numeric characters and converts the Phone input to E.164 format with the respective country codes.
public class PhoneNormalizerUtil { private static final Pattern NON_DIGIT = Pattern.compile("[^0-9]"); public static Optional<String> normalizeToE164(String rawPhone, String countryCode) { if (rawPhone == null || rawPhone.trim().isEmpty()) { return Optional.empty(); } String digits = NON_DIGIT.matcher(rawPhone).replaceAll(""); if (rawPhone.startsWith("+") && digits.length() >= 10) { return Optional.of("+" + digits); } switch (countryCode) { case "IN": return normalizeIndia(digits); case "US": return normalizeUS(digits); case "UK": return normalizeUK(digits); default: return Optional.empty(); } } private static Optional<String> normalizeIndia(String digits) { if (digits.length() == 10) { return Optional.of("+91" + digits); } if (digits.startsWith("91") && digits.length() == 12) { return Optional.of("+" + digits); } return Optional.empty(); } private static Optional<String> normalizeUS(String digits) { if (digits.length() == 10) { return Optional.of("+1" + digits); } if (digits.startsWith("1") && digits.length() == 11) { return Optional.of("+" + digits); } return Optional.empty(); } private static Optional<String> normalizeUK(String digits) { if (digits.startsWith("44") && digits.length() == 12) { return Optional.of("+" + digits); } if (digits.length() == 10) { return Optional.of("+44" + digits); } return Optional.empty(); } } |
Lead Priority and Segmentation
The lead priority is calculated using the Title, No of employees, Last Activity Days and Web Engagement Score fields in the record.
Each field value contributes a predefined number of points:
- If the job title indicates a senior role such as CEO, CTO, Director, or VP, 25 points are added.
- If the company size is 200 employees or more, 20 points are added.
- If the Lead has been active within the last 7 days, 15 points are added.
- If the web engagement score is 70 or higher, 10 points are added.
The sum of all applicable factors is returned as an integer and based on this score the lead's priority and segmentation is assigned as follows:
- High (Hot): score ≥ 70
- Medium (Warm): score ≥ 40
- Low (Cold): score < 40
public static int calculateLeadScore( String jobTitle, Integer companySize, Integer lastActivityDays, Integer webEngagementScore) { int score = 0; // Decision Maker if (jobTitle != null) { String title = jobTitle.toLowerCase(); if (title.contains("ceo") || title.contains("cto") || title.contains("director") || title.contains("vp")) { score += 25; } } // Company Size if (companySize != null && companySize >= 200) { score += 20; } // Recency if (lastActivityDays != null && lastActivityDays <= 7) { score += 15; } // Engagement if (webEngagementScore != null && webEngagementScore >= 70) { score += 10; } return score; } public static String deriveSegment(int score) { if (score >= 70) return "Hot"; if (score >= 40) return "Warm"; return "Cold"; } public static String derivePriority(String segment) { switch (segment) { case "Hot": return "High"; case "Warm": return "Medium"; default: return "Low"; } } |
Lead Quality:
Lead quality is evaluated based on the validity of the Mobile and Country fields in the records.
- Clean: Both field values are valid.
- Critical: Both field values are invalid.
- Needs Review: Only one of the field values is invalid.
Write Back to Records:
The processed data is written back to Zoho CRM as follows:
- Mobile: Formatted phone numbers.
- Lead Score: Total score calculated from all applicable fields.
- Lead Segment: Segment derived from the lead score.
- Sales Priority: Priority assigned based on the lead segment.
- Data Quality Status: Overall quality of the Lead record.
if (normalizedMobile.isPresent()) { data.put("Mobile", normalizedMobile.get()); } data.put("Lead_Score", leadScore); data.put("Lead_Segment", leadSegment); data.put("Sales_Priority", salesPriority); data.put("Data_Quality_Status", dataQualityStatus); } |
Step 4: Deploy to Development
Run the following command to deploy your local customizations to the Development Environment of Catalyst.
catalyst deploy |
You can find the complete code sample on GitHub for reference.
Try it Out!
With the changes deployed to the Development environment, let us now test the solution.
We hope this Kaizen helps you import large volumes of data into Zoho CRM while improving data quality and lead intelligence using the CRM Bulk Data Processor.
Have questions or suggestions? Drop them in the comments or write to us at support@zohocrm.com.
We will circle back to you next Friday with another interesting topic.
On to Better Building!
--------------------------------------------------------------------------------------------------------------------------
Previous Kaizen: Client Script Support for List Page (Canvas) | Kaizen Collection: Directory
--------------------------------------------------------------------------------------------------------------------------
Access your files securely from anywhere
Centralize Knowledge. Transform Learning.
All-in-one knowledge management and training platform for your employees and customers.
New to Zoho Recruit?
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Kiruthiga G
Sticky Posts
Kaizen #198: Using Client Script for Custom Validation in Blueprint
Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Kaizen #226: Using ZRC in Client Script
Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In thisKaizen #222 - Client Script Support for Notes Related List
Hello everyone! Welcome to another week of Kaizen. The final Kaizen post of the year 2025 is here! With the new Client Script support for the Notes Related List, you can validate, enrich, and manage notes across modules. In this post, we’ll explore howKaizen #217 - Actions APIs : Tasks
Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.Kaizen #216 - Actions APIs : Email Notifications
Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
Access query string from incoming webhook URL in Zoho Flow
I have a fairly straightforward use case, but cannot find anything conclusive on this subject. How can I simply access the query string of the incoming webhook request in Zoho Flow? I am attaching the Deal ID to the end of the request like: https://flow.zoho.com/762439902/flow/webhook/incoming?zapikey=1001.0dec2d7dd15080c464a13925275f0129.448c23ee219c545c89f5a178456cfc8b&isdebug=false&dealID=${Deals.DealUnable to access my Zoho forms account
For some days now, I haven't had access to my Zoho Forms account. I keep getting an error that says, "You are an inactive user in your organization" via the mobile app and "You don't have permission to access this organization" via the web. I was removedIdentify Unused Deluge Functions in Zoho CRM
Hello Zoho Support Team, I would like to request a feature that helps administrators identify unused or rarely used Deluge functions in Zoho CRM. Currently, Deluge functions are distributed across different areas, and there is no centralized overviewUser Automation: User based workflow rules & webhooks
User management is an undeniable part of project management and requires adequate monitoring. As teams grow and projects multiply, manual coordination for updating users & permissions becomes difficult and can give way to errors. User automation in ZohoUpdate on V2 API End-of-Life Timeline
Dear Users, Earlier this year, we shared the launch of the V3 APIs and requested users to migrate from the older V2 APIs by December 2025. We have received valuable feedback from our users and partners regarding their migration timelines. We are happyRun a workflow after record merge
Hello, We are wondering if there is a way to trigger a workflow after two records are merged? We have a sms service that use to text back and forth with Prospects and Contacts. To make it easy for our users, we automatically create them in this otherLive webinar | The Evolution of Zoho Show: 2025 Feature Recap
2025 has been a year of exciting updates and features for Zoho Show. From advanced design and formatting tools to smarter presentation delivery, enhanced collaboration, and AI-powered features—along with the launch of our desktop apps, Show has continuedBatch Tracking custom function
Hi fellow zoho users, We have our ecommerce site setup to shopify. I have some products I want to add batch tracking too, my only issue is that it breaks our shopify workflow (create sales order, invoice, payment, package) because it requires us to manuallyInclude Suggested Articles in New Ticket Email
When someone creates a new ticket, is there any way to include recommended knowledge base articles (like the ones Zia Suggests) in the email notification to the user? When they're waiting for a reply, it would be nice to have sent them some recommendedItem Bulk Edit - Allow for Reorder Level
We're implementing a process for using the Reorder Level field for Items, and I have to go through and add this value to a huge chunk of our Items. It's driving me bonkers that I have to do this individually through the UI rather than bulk updating. ItSharing my portal URL with clients outside the project
Hi I need help making my project public for anyone to check on my task. I'm a freelance artist and I use trello to keep track on my client's projects however I wanted to do an upgrade. Went on here and so far I'm loving it. However, I'm having an issue sharing my url to those to see progress. They said they needed an account to access my project. How do I fix this? Without them needing an account.Zoho Flow + QuickBooks Estimates – Line items not created from CRM subform
Hi everyone, I’m trying to create QuickBooks Estimates from Zoho CRM Quotes using Zoho Flow. I’m aware that Zoho Flow’s native “Create Estimate” action does not support multiple line items, so I followed the community guidance for Invoices using a customSAP S/4 HANA Integration via ZOHO Flow
Hi How do I change the SAP Client ID while creating a new connection ? MY client ID is 421. I believe the client id should be an input field for length 3 numbers rather then a drop down to select from a fixed set of values. ThanksVLOOKUP FUNCTION PROBLEM
Hello, i have a problem with the VLOOKUP function in my sheet http://public.sheet.zoho.com/public/tonimoreno/indicemasacorporal2 in cell D2. This function doesn't work correctly and always returns the last content of the range. Can you help me?Zoho Flow Completely Broken?
Since December 13th, 100% of our flows have stopped working completely. Nothing is working or triggering??? I called support & sat on the line with them for 2 HOURS until they finally concluded it was a back-end problem on their end?? Now i'm callingPrint / Export full Dashboard page
Hello Zoho team, It is currently only possible to print/export one component of a dashboard at a time. Requesting the option to print/export a full dashboard page with all components to a pdf or picture. Doing through the browser doesn't give good results.emails duplicated in my outlook client
Hello I am just new here and I have a litle problem that I do not find how to solve. I sumarise here after. Your help would be very appreciated. I have just recently contracted this zoho mail servicesservices as an initial evaluation yesterday night. I then: - opened a free account - created 1 initial email address ..........@domain.com - created a second email address ..........@domain.com - proceeded with the config instructions (checking the ownership of my domain, changing MX etc... ) asHow to check task starting time in zoho mobile app?
I am using the Zoho mobile app on my iPhone, and I am not able to see the task's starting and finishing times. When opening the Task information / Details, I can only see the Task start date and due date, but not the time. Is there any way to check aCreating an extension in Sigma: Zoho CRM isn't selectable as a service
I have 2 Zoho accounts which are part of 2 different workplaces. One workplace is able to select 'Zoho CRM' as a service option when creating a new extension, the other one is not. I'm not sure what the exact differences are between them. What do I needList of packaged components and if they are upgradable
Hello, In reference to the article Components and Packaging in Zoho Vertical Studio, can you provide an overview of what these are. Can you also please provide a list of of components that are considered Packaged and also whether they are Upgradable?RouteIQ for Zoho FSM
Beste, Zou wel top zijn dat we een RouteIQ hebben voor FSM aangezien we constant moeten zien wat de beste route is voor onze monteurs. Nu moeten we een speciale aparte programma hebben om de beste route te berrekenen voor onze monteurs aangezien de planningExport Tickets from the Help Center
Hello everyone! We are now allowing end users to export tickets directly from the help center. The ticket export allows users to: Filter tickets by Department, Priority, and Account before exporting. Download files from Export History (Setup > Data administrationIs it possible to remove filtering options?
My CRM has a lot of custom fields that should not be used in filters or views. Fields that are automated and exist only to store temporary values that get used in functions. These create a lot of noise in the list of fields to filter. Isn't there anyFunction #18: Associate invoice templates automatically based on customer language
For businesses dealing with a diverse linguistic clientele, it becomes crucial to send out invoices in the customer's preferred language to ensure effective communication. This requirement can be handled in Zoho Books by creating invoice templates inChanging field types
Question im a Zoho CRM user and curious, if I change a flied type from single or multi line text to a URL field type will i lose the data in the field or will it be converted to website link automatically. ThanksQuote PDF – Header image stopped rendering suddenly
Hello Zoho Support Team, We are facing an issue with Quote PDF templates in Zoho Books. Previously, images added inside the Quote header HTML were rendering correctly. However, recently the header image is no longer appearing, while header text stillChanging Default PDF Name
Is it possible to change the default name of a PDF? As of right now, all of my quotes are named 'QT_$QuoteNumber' (i,e: 'QT_19803471298374) - would it be possible to change this to: '$CompanyName - $AccountName - $QuoteNumber' for instance?Tip 2: Recursive functions in Deluge: How to dynamically run a function for a specified number of times.
Hi folks, As part of the Zoho Creator - Tips and Tricks series every fortnight, we are back today with a new tip on Recursive functions. Let us first quickly understand what Recursive functions are: A function that calls itself one or more times is known as a Recursive function. That is, you can execute a function to perform a particular action a specific number of times. And, at the end of each iteration, a new output is generated. Recursive functions are commonly used by programmers as it enablesFunction #48: Manage fixed installment payments using Zoho Books
Hello everyone, and welcome back to our series! Businesses offer installment payment options to their customers, particularly for expensive purchases, to ease the financial burden on them. By breaking down the total cost into smaller, more manageableSpotlight series #6: The Show app for Android TV has a new look!
Hello everyone! We are delighted to introduce our revamped and redesigned Show app for Android TV. Smart TVs are exploding in popularity. Android TV alone has over 110 million active monthly devices. Zoho Show, as part of a constant effort to improveCan’t create alias “Email alias already exists”
Hi, I’m trying to create the following alias for my domain email. contact@ personal@ private@ But I get a email alias already exists. I had a soho account previously which I deleted which I suspect is causing the issue. Can someone assist? Kind RegaAppraisals - Order of Previous Reviews Should Be Newest to Oldest
The new Zoho People layout generally does a decent job at providing the necessary information for performing appraisals of employee performance. One example of this is the Previous Reviews section: This information is helpful when conducting a review.Paging through API results. a major gap in your documentation.
There is no way for me. to get all of my data through a single API call. Typically REST APis have mechanisms for paging through API results. But the documentation for the API I am using: https://desk.zoho.com/DeskAPIDocument#Introduction Has no mentionRecurring invoices were generated with old template...
I have recurring invoices setup. Ones generated on 10.20.2025 used the modified template I had used. But for some reason, on the ones created on 11/20/2025, it seems the invoices created reverted to the previous version of the invoice template. Notably,Tip #54- Exploring the Files Icon in Zoho Assist- 'Insider Insights'
As we’re already in mid-December, it’s a good time to take a closer look at one of the most useful options in the Zoho Assist remote support dashboard—the Files icon. To get started, log in to Zoho Assist using your credentials. Once you’re in, navigateTip#47: Estimation planner
Sprint planning becomes easier, smoother, more collaborative and more accurate with the Estimation Planner extension. Most work items involve multiple users, and each user's role and effort vary. To provide an unbiased and fair estimation point to theTime Tracking on Iphone doesn't stop
When I start a time tracking session the timer starts as expected. However when I hit stop, the timer remains on that project. The only buttons available are discard and start. Start runs the timer more and discard says it will throw away the data fromZoho Sign - Zoho CRM extension upgrade
Hi everyone, We've updated Zoho Sign extension for Zoho CRM with significant internal changes. Impact on existing Zoho Sign extension users Users using the extension without customization If you are using the integration without implementing Zoho Sign'sGroup by Owner/ Owner Kanban /Group by Custom Field
Hi, We are missing the option of viewing Tasks grouped by their owner, as it is an essential function for us to manage our employees' working tasks. We find it hard to inspect what are the tasks that our employees are working on. We are seeking an optionFlow using way too many tasks
I built a flow that is using way too many tasks for what it is doing. I reached out to support a week ago but never heard back so hoping the community can help. Does a filter count as a task? I am seeing tons of records filtered with < 1 Sec Time TakenNext Page