WorkDrive server upgrade incident: Zoho Flow and Deluge user API calls affected in US DC (Issue fixed)

WorkDrive server upgrade incident: Zoho Flow and Deluge user API calls affected in US DC (Issue fixed)

Hello everyone,

We recently encountered an API call failure issue in the Zoho Flow and Deluge apps. The details of the issue are as follows:

About the Issue
  • The WorkDrive API and WorkDrive Integration tasks in Deluge returned error responses.
  • Zoho Flow tasks that were configured with WorkDrive actions failed.
Cause
The issue was caused by a bug introduced by the latest upgrade, when we attempted to modify the logic for auditing API actions.

Impacted users
  • Zoho Flow users who have configured WorkDrive actions in the Zoho Flow service in the US DC.
  • Zoho Deluge users who have used WorkDrive integration tasks or WorkDrive APIs using invokeURL task from any Zoho services in the US DC.
    Note: US DC refers to users whose accounts are mapped to the US data center.
Issue duration
July 18, 2023 04:54 AM PDT to July 18, 2023 11:57 PM PDT

Detection and resolution time
We detected the issue on July 18, 2023 at 5:20 PM PDT and successfully fixed it by July 18, 2023 at 11:57 PM PDT.

Root cause analysis

Nature of the issue
During one of our planned upgrades to our WorkDrive servers on July 18, 2023, an unintended bug was introduced, leading to API call failures when Zoho Flow attempted to communicate with WorkDrive. Similarly, Zoho Deluge's integration tasks and APIs faced issues, resulting in error responses. The root cause was traced back to our efforts to modify the logic for auditing API actions, particularly the auditing of API requests for actions like creating files or folders.

Impact
Only users who had their data stored in the US DC were affected by this issue, and they might have encountered unexpected behavior in API responses. It's important to note that, although some API actions returned failure statuses, some actions partially succeeded.

Resolution
Our development team worked promptly to resolve the bug, and we have released an immediate hotfix update to rectify the problem.

Preventive measures
We sincerely apologize for any inconvenience caused during this period. To prevent similar incidents in the future, we have implemented additional checks and balances in our upgrade procedures to ensure the stability and reliability of the system.

Thank you for your understanding. We remain committed to providing you with the best possible service. If you have any questions or concerns, please feel free to drop them in the comment section below.

    Access your files securely from anywhere







                          Zoho Developer Community






                                                • Desk Community Learning Series


                                                • Digest


                                                • Functions


                                                • Meetups


                                                • Kbase


                                                • Resources


                                                • Glossary


                                                • Desk Marketplace


                                                • MVP Corner


                                                • Word of the Day


                                                • Ask the Experts



                                                          • Sticky Posts

                                                          • Ask the Experts: Five-hour live Q&A session with Zoho WorkDrive product experts

                                                            Have questions about WorkDrive 5.0? Let’s talk! We recently launched Zoho WorkDrive 5.0, packed with powerful updates to help your team work smarter, stay secure, and get more value from your business content. From content-centric workflows and AI-powered
                                                          • WorkDrive API Documentation

                                                            WorkDrive provides users and developers an extensive set of APIs to help integrate functionalities of Zoho WorkDrive with other Zoho applications and third-party tools. We have published the official WorkDrive API Documentation page for all external users.
                                                          • March 15, 2023: Zoho Docs is discontinued

                                                            As of today (March 15, 2023) Zoho Docs is discontinued for all users. We would like to thank our customers for trusting us for so many years! Going forward, we're confident you'll enjoy using Zoho WorkDrive for all your advanced file management and collaboration
                                                          • Support for auto-upgrade in TrueSync (for Windows)

                                                            WorkDrive TrueSync app now supports auto-upgrading to the latest version for Windows OS. You must manually download and install the TrueSync app version 3.4.0 to avail this feature. Download the latest TrueSync app for Windows (version 3.4.0) Supported
                                                          • Introducing Collections

                                                            Create collection links to easily request any number of users to upload files to a particular folder. Once users upload their files, you will be able to view their submissions and manage their files effortlessly. Users don't need a WorkDrive account to


                                                          Manage your brands on social media



                                                                Zoho TeamInbox Resources



                                                                    Zoho CRM Plus Resources

                                                                      Zoho Books Resources


                                                                        Zoho Subscriptions Resources

                                                                          Zoho Projects Resources


                                                                            Zoho Sprints Resources


                                                                              Qntrl Resources


                                                                                Zoho Creator Resources



                                                                                    Zoho CRM Resources

                                                                                    • CRM Community Learning Series

                                                                                      CRM Community Learning Series


                                                                                    • Kaizen

                                                                                      Kaizen

                                                                                    • Functions

                                                                                      Functions

                                                                                    • Meetups

                                                                                      Meetups

                                                                                    • Kbase

                                                                                      Kbase

                                                                                    • Resources

                                                                                      Resources

                                                                                    • Digest

                                                                                      Digest

                                                                                    • CRM Marketplace

                                                                                      CRM Marketplace

                                                                                    • MVP Corner

                                                                                      MVP Corner







                                                                                        Design. Discuss. Deliver.

                                                                                        Create visually engaging stories with Zoho Show.

                                                                                        Get Started Now


                                                                                          Zoho Show Resources


                                                                                            Zoho Writer Writer

                                                                                            Get Started. Write Away!

                                                                                            Writer is a powerful online word processor, designed for collaborative work.

                                                                                              Zoho CRM コンテンツ




                                                                                                Nederlandse Hulpbronnen


                                                                                                    ご検討中の方




                                                                                                          • Recent Topics

                                                                                                          • javax.mail.authenticationfailedexception 535 authentication failed

                                                                                                            Hi, I am facing 535 authentication failed error when trying to send email from zoho desktop as well as in webmail. Can you suggest to fix this issue,. Regards, Rekha
                                                                                                          • Pocket from Mozilla is closing shop. Don’t lose your favorites . Move them to Zoho Mail Bookmarks now! 📥🔖

                                                                                                            The end of Pocket shouldn't mean the end of your important links and content. Easily import them into Zoho Mail's Bookmarks and continue right where you left off. You can bring over your entire Saves, Collections, and tags just the way they are. Bookmarks
                                                                                                          • Zoho Sign: need to leave document pending for up to a year, or maybe there's a better way?

                                                                                                            I have zoho one, maybe there's a better way to do this with another service than sending a zoho sign template from zoho crm. At the end of the day this requirement is due to regulations, no matter how dumb it may seem. I'm just looking for a way of getting
                                                                                                          • 'Add Tax To Amount' not reflected in Invoice

                                                                                                            Hi Zoho Support, I'm experiencing an issue with tax calculation display in my invoice template. Despite having "Add tax to amount" box checked in the template settings, the Amount column is not showing the tax-inclusive total for line items. Current behaviour:
                                                                                                          • To Do: shareable task links without login

                                                                                                            Hi! I’m using Zoho Mail and ToDo in my daily work, and I’ve run into one limitation that’s a real blocker for me. Right now, to share tasks with managers or directors, they need to have a Zoho account and be added to a group. In practice, many of them
                                                                                                          • Separate Items & Services

                                                                                                            Hi, please separate items and services into different categories. Thank you
                                                                                                          • Unable to edit or delete email address

                                                                                                            I signed up for free Zoho today. I usually am pretty good at understanding and configuring things like this, but your interface baffles me, and your online help is cryptic to say the least. I have spent hours just trying to set up a couple of email accounts in Zoho before pointing my domain MX records to Zoho. I solved some other issues on my own, but I can't figure out this latest problem: I have created two email addresses in Zoho. Let's call the first one myname@mydomain.com and the second one
                                                                                                          • Mastering Zia Match Scores | Let's Talk Recruit

                                                                                                            Feeling overwhelmed by hundreds of resumes for every job? You’re not alone! Welcome back to Let’s Talk Recruit, where we break down Zoho Recruit’s features and hiring best practices into simple, actionable insights for recruiters. Imagine having an assistant
                                                                                                          • We are unable to process your request now. Please try again after sometime or contact support@zohoaccounts.com

                                                                                                            I cannot sign up and return the error of we are unable to process your request now. Please try again after sometime or contact support@zohoaccounts.com
                                                                                                          • Multi-currency - What's cooking ?

                                                                                                            Hi,       We have been doing this feature for sometime and we would like to give you some glimpses of it.  Working with Multi Currency :        Multicurrency support gives you the ability to handle business transactions in multiple currencies. You can define a base currency for your organization and add more currencies with exchange rates based on the base currency.  Setup :        From the setup page, you can manage all the currencies supported by your organization.       Currencies page        
                                                                                                          • Integrating Chatbot with Zoho Creator Application

                                                                                                            Is it possible to integrate a chatbot with a Zoho Creator application?
                                                                                                          • How to reduce programmatically the image uploaded by user?

                                                                                                            I need a function that will automatically reduce the pixel dimension to 800 x 600 pixels / 180 resolution or (approx. 1.37MB) of image uploaded by user from digital camera, for example, 2271 x 1704 pixels /180 resolution or approx. 11.1MB. After the user selected the image, the function will able to detect if pixels is above 800x600, process the photo (crop/ reduce) and resume upload. Need help...  
                                                                                                          • Dark mode for Zoho Creator / Zoho CRM Code editor

                                                                                                            Hi Team, Is there any plans for Dark mode in Zoho creator / Zoho Crm code editor and development pages in pipeline?
                                                                                                          • Is there a way to make a button scroll down?

                                                                                                            Looking to have a button on a landing page scroll down to another section on the page. Any recomendations outside of coding?
                                                                                                          • Collective-booking event not added to all staff calendars

                                                                                                            We assign two staff to certain events. When the client books this event, it adds it to one staff calendar (the 'organiser') but not the other. How can I ensure all staff assigned to a collective booking get the event in their calendar? (A side note: it
                                                                                                          • ZOHO Android Client

                                                                                                            Hi, I installed the Android app, but it had an issue, so I reinstalled it. I was able to add multiple accounts, but now when I add the next account, it just duplicates the one I already have and will not even allow me to enter the info for another account.
                                                                                                          • I'd like to suggest a feature enhancement for SalesIQ that would greatly improve the user experience across different channels.

                                                                                                            Hello Zoho Team, Current Limitation: When I enable the pre-chat form under Brands > Flow Controls to collect the visitor’s name and email, it gets applied globally across all channels, including WhatsApp, Messenger, and Instagram. This doesn't quite align
                                                                                                          • Enhance Barcode/QR Code scanner with bulk scanning or continuously scanning

                                                                                                            Dear Zoho Creator, As we all know, after each scan, the scanning frame closes. Imagine having 100 items; we would need to tap 100 times and wait roughly 1 second each time for the scanning frame to reopen on mobile web. It's not just about wasting time;
                                                                                                          • Managing Rental Sales in Zoho Inventory

                                                                                                            I am aware that Zoho Inventory is not yet set up to handle rental sales and invoicing. Is anyone using it for rentals anyway? I'd like to hear about how others have found work arounds to manage inventory of rental equipment, rental payments, etc. Th
                                                                                                          • Megamenu

                                                                                                            Finally! Megamenu's are now available in Zoho-Sites, after waiting for it and requesting it for years! BUT ... why am I asked to upgrade in order to use a megamenu? First: Zoho promised to always provide premium versions and options for all included Zoho-applications
                                                                                                          • Zoho Flow to Creator 3001 Respoonse

                                                                                                            I have updated my Flows with the new V2 connection to Zoho Creator, but now some Flows do not work. They take in data from a Webhook and are supposed to create a record in Creator, however creator returns a 3001 message along with a failure, but I cannot
                                                                                                          • File Upload to Work Drive While Adding Records in Zoho Creator Application

                                                                                                            Hi I am trying to set a file attachment field in zoho creator form, to enable the user to upload a scanned document from their local pc. The file should be uploaded to zoho workdrive and not to the default zoho creator storage. The file link should be
                                                                                                          • Why not possible to generate?

                                                                                                            Using this https://desk.zoho.com/DeskAPIDocument#TicketCount#TicketCount_Getticketcountbyfield on my ZML script url :"https://desk.zoho.com/api/v1/ticketsCountByFieldValues?departmentId=XXXXXXXXXXX&accountId!=XXXXXXXXX&customField1=cf_country_1:XXXXXX&field=overDue"
                                                                                                          • email

                                                                                                            Hi My crm email is not working, can you check, I have zoho one account.
                                                                                                          • Need option to see Mass Emails & Cadences in Gmail Outbox OR a dedicated Zoho Outbox

                                                                                                            Hi everyone, Right now, when we send 1:1 emails from gmail (with gmail API connected to Zoho CRM), those emails appear both in gmail's sent folder and in Zoho CRM. That works well. But when we send Mass Emails or Cadence emails form Zoho CRM, they are
                                                                                                          • I can't found API for Sales Receipts

                                                                                                            Hello May you please help me to find an API document for Sales Receipts to get data and retrive a custom fields like Invoice and credit notes Regards
                                                                                                          • Kaizen #205 - Answering Your Questions | Managing Picklists and Enabling History Tracking via Zoho CRM APIs

                                                                                                            Hello everyone! Welcome back to another post in our Kaizen series. In this post, we will look at how you can manage picklist fields in Zoho CRM using APIs. This topic was raised as feedback to Kaizen #200, so we are taking it up here with more details.
                                                                                                          • Multiple Vendor SKUs

                                                                                                            One of the big concerns we have with ZOHO Inventory is lack of Vendor Skus like many other inventory software packages offer. Being able to have multiple vendor skus for the same product would be HUGE! It would populate the appropriate vendor Sku for
                                                                                                          • Internally created tickets

                                                                                                            Hi there When tickets are created internally on-behalf of customers - there is nothing to show that the ticket was created by an internal agent. This means, that it's easy for our agents to confuse tickets which were created by internal team members and
                                                                                                          • Automatically change website passwords

                                                                                                            Hi everyone, We just switched to a Professional package to also use the "Automatically change website passwords" function. But I cannot find anything about it, how to use it, anywhere. Does anyone know how I can use this function? Best, Caspar
                                                                                                          • Change Invoice Prices for an Effective Date

                                                                                                            Hi, It would be a really good feature to be able to change the prices on invoices/recurring invoices from an effective date in the event of price increases. For instance, I am in the process of increasing prices that will be effective from a specific
                                                                                                          • "Other Current Asset" accounts as "Paid Through" accounts in Expense

                                                                                                            It would be incredibly useful to be able to assign accounts of type Other Current Asset as Paid Through accounts in Expense. Currently, Other Current Liability are permitted as Paid Through Accounts. This makes sense, as Credit Cards are current liabilities.
                                                                                                          • Multi column open text questions that allows respondents to add rows for additional information

                                                                                                            I need to create a question that has 2 columns with open text, but I also need to allow respondents to click a "+" button, or something similar, so that they can add additional information if they choose to. I've tried using the Multiple Textboxes type
                                                                                                          • Bot Filtering & Apple Mail Privacy Protection Compliance in Zoho Campaigns

                                                                                                            Dear Campaigns Users, The wait is over! We’re excited to announce that the enhanced bot filtering feature is now live in Zoho Campaigns. This update brings greater accuracy to your email campaign reports by distinguishing real user engagement from automated
                                                                                                          • Découvrons les détails qui simplifient vos journées de travail avec Trident

                                                                                                            Nous nous installons dans des routines efficaces et rodées avec le temps. Chaque matin, nous ouvrons nos e-mails, passons aux messages, consultons notre agenda, puis attaquons nos tâches. Ce processus nous semble maîtrisé, mais est-il réellement optimisé
                                                                                                          • Issue with Purchase Rate Showing as “0” After Importing Items List

                                                                                                            Dear Zoho Books Support Team, Good day. I’m reaching out regarding an issue I’m facing while importing my items list into Zoho Books. Despite mapping all fields correctly and including the purchase price for each product in my Excel file, the Purchase
                                                                                                          • API for Task Entity in Zoho Books

                                                                                                            I’m working on automating task creation in Zoho Books via a custom button in the Bills Module. The goal is to create a task in the Tasks Module and assign it to the Finance Team, so they can track progress efficiently. While reviewing Zoho Books documentation,
                                                                                                          • create invoice in zoho books from the zoho forms

                                                                                                            Is there a native way to have create invoice in zoho books, when zoho form is completed?
                                                                                                          • Email undelivered

                                                                                                            GOod Day I am always receiving an uncategorized-bounce to my email. I am not sure why this is happening.
                                                                                                          • Add inventory_valuation_method to items endpooints

                                                                                                            To ensure consistent item creation it would be helpful to have the inventory_valuation_method (FIFO vs WAC) be able to be set at item creation or as an update (consistent with current behavior where it is not allowed for items with existing transactions)
                                                                                                          • Next Page