EU DC Partial Outage Resolved: A Detailed RCA
Incident Summary
Due to an overload on one of the nodes in the EU DC for Zoho Desk, the system was unable to handle the heavy load, causing a slowdown in requests and resulting in a partial outage for customers with data residing in that node.
Due to an overload on one of the nodes in the EU DC for Zoho Desk, the system was unable to handle the heavy load, causing a slowdown in requests and resulting in a partial outage for customers with data residing in that node.
On May 2, 8:26 AM, CEST , the incident was identified by our team. Our engineers determined the root cause and initiated efforts to address it by adjusting our system configurations. During this time, we began transferring high-traffic organizations to other nodes to minimize impact on the affected node. Although our secondary adjustments mitigated the issue and enabled the portals to load and work by May 2, 03:19 PM, CEST, it was not completely resolved.
After completing the scheduled movement of top-traffic generating organizations, we deployed a bug fix to prevent the system from holding connections for extended periods. Stability was fully restored on May 3, 12:40 PM, CEST. The incident and its history were also captured on the service availability page (status.zoho.eu).
Technical Breakdown
We identified that our system was experiencing slowness, leading to a significant number of users experiencing difficulty in connecting to Desk. This issue directly impacted the availability and performance of our service.
Further analysis using our monitoring system and log data indicated that the high traffic was legitimate and not due to a DDoS attack. Also, this incident was solely related to a load-surge issue, and there was no data loss or data impact caused by the partial outage. Our engineering team worked to optimize our system configuration to handle the traffic, but it did not completely resolve the issue. In addition, we moved top traffic generating organizations from the impacted node to minimize traffic, resulting in some improvement.
Furthermore, we identified a code bug that held connections for an extended period of time and quickly deployed a live build to rectify the issue.
Furthermore, we identified a code bug that held connections for an extended period of time and quickly deployed a live build to rectify the issue.
Timeline (in CEST)
May 2, 09:26 AM | Incident identified |
May 2, 11:40 AM | Root cause identified - high number of connections |
May 2, 12:15 PM | System configurations tuned |
May 2, 01:51 PM | Started moving top-traffic generating orgs |
May 2, 03:54 PM | Second-level system configuration tuning |
May 3, 12:12 PM | Preparation of build to fix code bug began |
May 3, 12:29 PM | Movement of top-traffic generating orgs completed |
May 3, 02:40 PM | Bug fix build went live, stabilizing the system. |
Future Preventive Measures to Avoid Recurrence of the Issue
- Relocating selected organizations to other nodes to keep the connection count to the affected node at a minimum.
- Monitoring the system connections proactively and re-balancing them as necessary.
- Setting a lower connection threshold to receive early notifications when breached and take prompt action to avoid customer impact.
- Incorporate a code-check configuration rule to prevent code from holding connections for extended periods of time before being shipped to production.
Regards,
Zoho Desk Team
Access your files securely from anywhere
Centralize Knowledge. Transform Learning.
All-in-one knowledge management and training platform for your employees and customers.
New to Zoho Recruit?
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Ash | Zoho Desk
Stéphane LAMBERT
Shivani | Zoho Desk
Sticky Posts
Zoho Desk Partners with Microsoft's M365 Copilot for seamless customer service experiences
Hello Zoho Desk users, We are happy to announce that Zoho Desk has partnered with Microsoft's M365 to empower customer service teams with enhanced capabilities and seamless experiences for agents. Microsoft announced their partnership during their keynoteWhatsApp pricing changes: Pay per message starting July 1, 2025
Starting July 1, 2025, WhatsApp is shifting from conversation-based pricing to per-message billing. That means every business-initiated message you send will count. Not just the first one in a 24-hour window. Pricing updates on the WhatsApp Business PlatformLive Webinar - Work smarter with Zoho Desk and Zoho Workplace integration
Hello customers! Zoho Desk and Zoho Workplace are coming together for a webinar on 14th May, 2024. Zoho Workplace is a suite of productivity apps for email, chat, docs, calls, and more at one single place. Zoho Desk is closely integrated with a few toolsApple iOS 17 and iPadOS 17 updates for Zoho Desk users
Hello Zoho Desk users! Apple recently announced the release of iOS 17 and iPad OS 17. These latest OS updates will help you stay productive and efficient, through interactive and seamless user experiences. Zoho Desk has incorporated the updates to helpZoho Desk Cheat Sheet For The Year-End
Check out these Zoho Desk best practices to end this year on a high and have a great one ahead! #1 Set Business (Holiday) Hours - If you have limited working hours, please make sure you restrict your business hours or set them as holidays for the coming days. Let your customers know when you will, and won't, be available. #2 Update the Annual Holiday List - Check the holidays for the new year and update the holiday schedule. Usually, holidays from the current year will be carried over for the next
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
Sign Out
Hello, I have a doubt with Permalink. I have a view created with critera to show only the records belonging to the user who has sign in. The problem is that the different users use the same computer and some times the user login keep signed, and when I send the permalink (by email) of the view, the users enter with other login signed. Moreover the permalink view doesn�t allow to do a log out. May I add something in the permalink to request always sign in? Sorry for my English. Many thanks! RegardsAnnouncing new features in Trident for Windows (v.1.39.4.0)
Hello Community! Trident for Windows just received a major update, with a range of capabilities that focuses on strengthening communication and simplifying workflows. Let’s dive into what’s new! Upload email attachments to WorkDrive. Until now, you couldIntroducing Automatic Field Addition (Text Tags) in Zoho Sign
Hello, Today we are excited to announce the general availability of automatic field addition (text tags) feature in Zoho Sign. Now, you can now add text tags in the content of your documents and Zoho Sign will automatically add the corresponding fields when they are uploaded for the signing process. For example: when you add text tags to your sales orders, new employee contracts, and NDAs, Zoho Sign will add the corresponding fields when these documents are uploaded for the signing process. If youIntroducing Built-in Telephony in Zoho Recruit
We’re excited to introduce Built-in Telephony in Zoho Recruit, designed to make recruiter–candidate communication faster, simpler, and fully traceable. These capabilities help you reduce app switching, handle inbound calls efficiently, and keep everyWrite-Off multiple invoices and tax calculation
Good evening, I have many invoices which are long overdue and I do not expect them to be paid. I believe I should write them off. I did some tests and I have some questions: - I cannot find a way to write off several invoices together. How can I do that,Splitting Transactions in Zoho Books
I have read in past forum posts that the ability to split bank transactions would likely be implemented - it's definitely a typical accounting program feature. I'm new to Zoho and thought I'd found nirvana until I realized this feature doesn't seem toStatement Aging On Cutomer Statement
Hello, Is it possible to put aging on customer statements? Current 1-30days 31-60days 61-90days 91-120days Over 120 days. See attached image from another accounting package. Many customers pay off a statement and clear older invoices.Unveiling the next iteration of Ask Zia in Zoho CRM: An all-new chat interface, conversation history, actions, and much more
Your CRM assistant just leveled up. Zoho CRM's Ask Zia functionality now offers a more conversational and context-aware experience to help you not just understand your data, but act on it—all from one chat window. With its redesigned interface and expandedResponse rate and time on social media
Hello, I just want to know if it's possible to manage the response rate and response time from my social media on zoho social ? I don't see any statistical reports on the online scoreboard ? Thank you in advance for your response and sorry if the question has already been postedWhatsapp BOT with CRM
Hello, how do you use Whatsapp integrations in zoho CRM?Ability to translate Zoho CRM Kiosks
Hi team, Is support for translating kiosk text and screen names in the Zoho CRM translation tool planned on the roadmap? Thanks,Whatsapp Limitation Questions
Good day, I would like to find out about the functionality or possibility of all the below points within the Zoho/WhatsApp integration. Will WhatsApp buttons ever be possible in the future? Will WhatsApp Re-directs to different users be possible basedEditing the list of Categories in the Categorize Manually section of Banking in Zoho Books
Hi, I need to create two new Categories called Withdrawals and Deposits to categorize payments in a bank account. How do I edit the Categories list?Can I write a check in Zoho Books with no associated bill?
This currently does not seem possible, and I have a client that desperately needs this function if I am able to convert them with Quickbooks. Thank you in advance for your reply.Internal Fillable Contract with Zoho Writer (Before Sending to Client)
Hi everyone, I’m trying to automate the following process in Zoho CRM and would appreciate some guidance. Process: When a Deal moves to a specific stage, CRM triggers an automation. CRM sends a contract template to an internal team member so they can[Free Webinar] Intelligent document processing with Zoho RPA
Hello everyone! Greetings from the Zoho RPA team! We're excited to invite you to our upcoming webinar on intelligent document processing with Zoho RPA, where we'll introduce powerful new capabilities designed to make your automation journey smarter, faster,Enhancements for Currencies in Zoho CRM: Automatic exchange rate updates, options to update record exchange rates, and more
The multi-currency feature helps you track currencies region-wise. This can apply to Sales, CTC, or any other currency-related data. You can record amounts in a customer’s local currency, while the CRM automatically converts them to your home currencyWhen I schedule calendar appointments in zoho and invite external emails, they do not receive invites
Hello, We have recently transitioned to zoho and are having a problem with the calendar feature. When we schedule new calendar appointments in zoho the invite emails aren't being sent to the external users that we list in participants. However, this worksZoho People > Performance Management > Appraisal cycle
Hello All I am using this 2 users to test out how it work on Performance Management User 1 - Reportee User 2 - Reporting Manager : Li Ting Haley User 1 : Self Appraisal Error How do i fix this error?Send emails directly via Cases module
Greetings all, The ability to send emails from the Cases module, which users have been eagerly anticipating, is now available, just like in the other modules. In Zoho CRM, Cases is a module specifically designed for managing support tickets. If your organizationWebinar Alert: Supercharge landing pages with data insights | Zoho LandingPage
Every visitor to your landing page leaves behind valuable data, but are you leveraging it to improve conversions? Join our expert-led Landing Page Analytics webinar to learn how to track, analyze, and optimize landing page performance with Zoho LandingPage’sBulk Deletion of Zoho Projects Using Node.js and Zoho Projects API
Zoho Projects currently does not provide a built-in option to delete multiple projects in bulk from the UI. When working with testing environments or large numbers of temporary projects, deleting them one by one becomes time-consuming. To address this,Show unsubscribed contacts ?
Hello, I would like to display the unsubscribed contacts. Unfortunately, I do not have this subscription type as described in the documentation (https://help.zoho.com/portal/en/kb/marketing-automation-2-0/user-guide/contacts/contact-management/articles/subscription-type-24-1-2024#Subscription_Type_field.)Track Marketing Automation Campaigns in Zoho CRM
Hello, I've been searching but haven't found the exact answer to this question. I am looking to track Marketing Automation email campaigns and activities inside of Zoho CRM. Use Case: Action: Prospect Submits A Lead Form Outcomes: Prospect created inValidation rule for Date field
The condition settings for a Date field are are absolutlly usless. Conditions can only be set for a specific date, which is logically ineffective in most cases. When setting a condition for a Date field, users usually need to compare the value relativeEasily map shift data fields during user imports
Greetings all, You can now include all your shift-related data for your users without any hassle during user imports. With this enhancement, the Map Import Fields to Zoho CRM option includes all shift-related fields: Current Shift, Next Shift, and ShiftUnable to send
Hello, I am unble to send any single email during the whole time due to the Zoho IP 136.143.188.16 being bloked by SpamCop.net Please help can somebody help me?Adding new data to the sheet with most recent data being added to top row.
I am using a form that I built from within sheets. When the form is submitted, is it possible to have the data entered in (for example) row 2 (since row 1 is the heading) instead of being added into the next available row which should be hundreds ofHow do you arrange order in which the speakers are listed in a session once they have been selected?
Probably another simple thing I've missed but I can't find how to arrange the order in which the speakers are listed in a session once they have been selected. We usually want the speakers listed alphabetically by last name, but sometimes not. Once theProduct Updates in Zoho Workplace applications | February 2026
Hello Workplace Community, Let’s take a look at the new features and enhancements that went live across all Workplace applications for the month of February. Zoho Mail Organize Personal Notes with Collections You can now create collections under My PersonalIntroducing note actions and dynamic visibility in Kiosk Studio
Hello all, We are introducing enhancements to Kiosk Studio that will improve the product scope and meet your custom needs more precisely. What's new? Add notes as Actions: You can add notes to CRM records as kiosk Actions, as well as insert merge fieldsRegarding the Recipient Email change
I was not being able to change the Recipient email. Kindly resolve the problemRequest to Customize Module Bar Placement in New Zoho CRM UI
Hello Support and Zoho Community, I've been exploring the new UI of Zoho CRM "For Everyone" and have noticed a potential concern for my users. We are accustomed to having the module names displayed across the top, which made navigation more intuitivekanban view for client portal
Are kanban views an option for client portals? Access to Kanban views in the client portals would solve some mobile-compliant issues I have with the UI. Kanban functions very nicely on mobile and would be a super asset for my clients and vendors as they【Zoho CRM】サンドボックスのアップデート：メール送信トレイ機能の追加
ユーザーの皆さま、こんにちは。コミュニティチームの藤澤です。 CRMのサンドボックス（テスト環境）にメールの送信トレイ機能が追加されました。 本番環境でメール配信の前に、サンドボックスで送信される全てのメールを確認・検証できます。ワークフロー、承認プロセス、一括送信など、あらゆる送信パターンに対応しています。 「メールの送信トレイ」機能を使うと、顧客へ送信する前にメールの内容を事前確認できます。項目の欠落や書式の乱れなど、あらゆる問題をサンドボックス内でチェックできるため、本番環境でのトラブルを未然に防ぐことに役立ちます。Force mandatory entry on one of two fields on Contacts
We are finding our users aren't always entering a phone number or email address of a contact. We would like to make these fields mandatory but realize they won't always have both pieces of information, but should at least have one. Is there a way to makeHow can I prevent having recepients from being added as contacts in Zoho Desk?
How can I prevent having recepients from being automatically added as contacts in Zoho Desk? There's no option to disable this.Agent Concern
would like to ask the difference between an agent and a light agent. can a light agent close a ticket. thank you.Train Zoho Answer Bot Based on Customer
Hi all, Is it currently possible to mark Help Centre articles to a specific customer, and restrict the answer bot to only use relevant information if it is either marked as "General", or tagged for the specific customer in question? We currently haveNon-depreciating fixed asset
Hi! There are non-depreciable fixed assets (e.g. land). It would be very useful to be able to create a new type of fixed asset (within the fixed assets module) with a ‘No depreciation’ depreciation method. There is always the option of recording landNext Page