Unresponsiveness in Zoho Cliq

Unresponsiveness in Zoho Cliq

Time of post: 6:56 am PDT, 19 July 2023
Dear Cliq users,

We have encountered an unprecedented issue in our messaging servers. The details of the issue are as follows:
  • Impacted users : 1/4th of users in US DC (users who access cliq services through cliq.zoho.com)
  • Issue detection time :  4:22 AM PDT [4:52 pm IST], 19 July 2023
  • Issue : Unresponsive UI, Inaccessability of chats
  • Cause : Slow responsivenes in two of our messaging servers due to an unforeseen surge in requests.

We are currently working on resolving it with high priority. We will provide an update here in 30 mins. We apologize for any inconvenience caused.

Update 1 [8:39 am PDT, 19th July]: We have identified specific functions within the system that were causing issues in sending message. To ensure proper functionality, we have implemented restrictions on these functions. We are pleased to report that we have observed improvements in message delivery as a result. We will continue monitoring the situation and provide you with an update in the next 60 minutes. Thank you for your patience.

Update 2 [10:31 am PDT, 19th July]: We are pleased to inform you that the issue with our servers has been successfully resolved, and the Zoho Cliq services are now operating smoothly for all users. Our team is actively monitoring the system to ensure the absence of any further issues. 
We sincerely apologize for any inconvenience this may have caused and extend our gratitude for your patience while we resolved the matter.

Update 3 [2: 53 am PDT, 20th July]: We encountered a recurring issue with one of our servers, which persisted from 11:41 pm PDT, 19th July, to 12:14 am PDT, 20th July. However, thanks to our diligent server monitoring team, the problem was swiftly addressed, and services have now been fully restored to their normal operation.

Update 4 [5:17 am PDT, 20th July]: We regret to inform you that we experienced a similar issue with one of our servers, which occurred between 4:02 am PDT, 20th July, and 5:01 am PDT, 20th July. The good news is that our dedicated team is actively working on resolving this matter, and as a result, Cliq services should now be stable for most users. However, we want to be transparent about the situation, and some users may still encounter partial slowness in message processing. Rest assured, our team is diligently addressing this remaining concern, and we will provide you with another update on the progress within the next hour.

Update 5 [
6:59 am PDT, 20th July]: Our team has been diligently working on a patch to address the issue with our servers. As a result, Cliq services are in the process of being restored to normal functionality, and we expect everything to be up and running smoothly very soon. We sincerely apologize for any inconvenience this may have caused and appreciate your understanding and patience during this time.

Update 6 [7:19 am PDT, 20th July]: I am pleased to inform you that our team has successfully resolved the issue with our servers. For the past 20 minutes (starting from 7:00 am PDT, 20th July), our systems have remained stable. We are actively monitoring the servers to ensure that this incident doesn't recur in the future. As of now, Cliq services are expected to work normally for all users. You can proceed with your tasks and collaborations without any concerns.

Analysis report:

On July 19, 2023, at 3:49 AM PDT,  there was a problem in our network which caused a drop in live connections between clients and servers. As a result of this, our clients were creating new connections, where as the backend servers were trying to repair the
existing connections. This caused a surge in two of our servers, which resulted in partial downtime.

During the partial downtime the followings functionalities would have been affected.
  1. Chat related functionalities
  2. Propagation of user status
  3. Notifications
To fix this issue temporarily, some of the internal functionalities were throttled. This allowed us to control the surge of connections and process the load.
Counter measures :
  1. To prevent this from happening again, the logic for repairing failed connections has been modified, so that no surges should be observed in any of our servers. We hope to push it to production as soon as possible.
  2. We are also working on segregated processing of functionalities to prevent the availability issues due to heavy load.



Thank you once again for your understanding.

Best regards,
Poorvik 

    • Sticky Posts

    • New in Zoho Chat : Search for contacts, files, links & conversations with the all new powerful 'Smart Search' bar.

      With the newly revamped 'Smart Search' bar in Zoho Chat, we have made your search for contacts, chats, files and links super quick and easy using Search Quantifiers.   Search for a contact or specific conversations using quantifiers, such as, from: @user_name - to find chats or channel conversations received from a specific user. to: @user_name - to find chats or channel conversations sent to a specific user. in: #channel_name - to find a particular instance in a channel. in: #chat_name - to find
    • New in Zoho Chat: Threaded conversation at its finest best

      Perform effective team communication in Zoho Chat with our new 'Reply' option.   Converse and stay focussed on the parent conversation, rather than getting entangled in the web of several, never-ending sub threads.   To reply to a certain message, all you need to do is hover to the left hand side of the message. Then, click on the three dots to open a pop up menu. Here, click on the Reply button and type the reply to the message in the compose box and press Enter.   Voila, that was pretty simple. 
    • Changes in Cliq iOS app notification due to iOS 13 and Xcode 11

      Hello everyone! With the iOS 13 update, Apple has updated its policy on usage of VoIP push notifications. Over the past few months, we tried our best to provide a similar experience with the updated policy.  Changes in iOS 13:  With iOS 13, Apple mandates all VoIP push notifications to be reported to the CallKit framework as a new call. If a VoIP push notification is not reported to the CallKit within a designated time window, iOS will terminate the app. If enough VoIP push notifications are not
    • What's new in Zoho Cliq - June 2020 updates

      Hello again, everyone! I'm back to share with you the recent feature improvements and updates that we've pulled together for enhancing your experience in Cliq. Here's what's new this June for you all in Cliq's web and iOS app! New on Cliq Web: Drag and drop files to a chat in your left side panel   Now you can drag and drop attachments from your open conversation window to a specific chat or channel in the left side menu without opening it. Swift up actions and collaborate efficiently with Cliq's
    • 4 Things You Should Do Once You Get Started with Cliq

      Hey there, new user!  You've successfully logged in and set up your organization and you're all set to start working. What's next? Buckle up because here are 4 essential things you need to do first in order to get the most out of your Cliq experience:   1. Invite your colleagues   Now that you've set up your Cliq for business, you need to bring in all your employees, of course, because how else can you collaborate with them?   To invite your colleagues to Cliq, head on over to the Admin Panel which
      • Recent Topics

      • Add zoho calendar to google calendar

        Hi I keep seeing instructions on how to sync Zoho CRM calendar with google calendar but no instructions on how to view Zoho calendar in my google calendar.
      • Paragraph spacings in Zoho Notebook are double-spaced only. Is there a way to keep the original spacing format?

        Sorry I don't know the proper terminologies but the best way I can put it is all single-spaced paragraphs that I copy to Zoho Notebook become double-spaced paragraphs. It's very inconvenient and counterproductive when copying and pasting things from and
      • Changing line space in Notebook

        My lines are double spaced. How do i change to single space?
      • Promote a layout to standard so I can delete one layout

        Similar to this and many other topics: https://help.zoho.com/portal/en/community/topic/set-layout-as-standard The problem, every model has a standard layout, which is what we use most of the time... But as the business grows and evolves we may have to
      • Workdrive on Android - Gallery Photo Backups

        Hello, Is there any way of backing up the photos on my android phone directly to a specific folder on Workdrive? Assuming i have the workdrive app installed on the phone in question. Emma
      • Auto sync Photo storage

        Hello I am new to Zoho Workdrive and was wondering if the is a way of automatically syncing photos on my Android phone to my workdrive as want to move away from Google? Thanks
      • Is anyone experience missing functions in the new UI until hard refresh?

        The set of functions including search is almost always missing upon loading an app in the new UI. If you refresh, the functions return. (see second screenshot) Anyone else experiencing this?
      • Conversion Rate – Won Deals over Assigned Prospects

        Hello, I would like assistance configuring a KPI in Zoho Analytics titled: Objective of the calculation: Number of Won Deals divided by Total number of assigned prospects (not only converted prospects). Important clarification: The denominator must include
      • Not receiving New Ticket Emails

        Hello! The company I work for uses the Zoho ticketing system, however, I've noticed I'm not receiving email notifications when new tickets are published. I have admin rights to see these tickets on Zoho Desk and respond to them, but am not receiving the
      • Can't update a field on a parent form from a child form's On Success workflow

        Title: Can't update a field on a parent form from a child form's On Success workflow Hi everyone, I'm building a simple Golf Lesson Tracker app and I'm stuck on what should be a basic operation — updating a number field on a parent form when a child form
      • Approvals in Zoho Creator

        Hi, This is Surya, in one of  my creator application I have a form called job posting, and I created an approval process for that form. When a user submits that form the record directly adding to that form's report, even it is in the review for approval.
      • Upload from Zoho Creator File Upload field to OpenAI Vector Store

        I’ve struggled for quite a while to get this working properly. For a long time we relied on Azure Functions as a workaround to handle file transfers between Zoho Creator and OpenAI Vector Stores. It worked, but added unnecessary infrastructure and complexity.
      • Calendar start time limit

        Hi all, Has anyone found a magical way to filter calendar reports to start from, let's say 8:00, and not 0:00? Or at least to auto scroll there like Google Calendar for example?
      • Enable Free External Collaboration on Notecards in Zoho Notebook

        Hi Zoho Notebook Team, I would like to suggest a feature enhancement regarding external collaboration in Zoho Notebook. Currently, we can share notes with external users, and they are able to view the content without any issue. However, when these external
      • EU Problem

        Hi all, we've been facing issues with the Europe data center for two days. It's starting to disrupt our daily workflow. Any word on when this will be resolved?
      • How can I clone my website in order to try new templates?

        I want to try a new template for my existing Zoho website without the risk of changing the current site or accidentally publishing the wrong template. Therefore I thought a 'clone website'-option should be somewhere, but I can't find it. How do I try
      • Is it possible to update multiple work items at once?

        Hi, Is it possible to update the statuses/release tags of multiple work items at once? I came across this thread that says it's not possible at the moment, but it was related to the Sprints mobile app. Does this feature not exist in the website as well?
      • Custom Module Missing from Roles & Permissions List

        Hi Zoho Community, I created a new Custom Module in Zoho Expense. The module is fully built and I can see it in the Module Builder (Settings > Customization > Modules). However, I am unable to deploy this to my users because the module does not appear
      • Need Customer Item Inward Module along with QC

        Need Customer Item Inward Module along with QC 1. Using Transfer Orders hit the item balance sheet 2. Items without inventory it becomes difficult for tracking purpose. 3. Custom Modules become tedious to capture multiple items, item subforms are not
      • Zoho CRM Quotes – Subform and PDF/Writer Limitations

        Hello, I am encountering the following limitations in Zoho CRM Quotes: Custom product images cannot be uploaded in the subform – the image upload field cannot be added; only the file upload field is available. File upload placeholders cannot be used in
      • Apple Messages for Business in Omnichannel communications?

        Hello, Apple launched "Apple Messages for Business" but Zoho CRM or Zoho Desk don't appear in the list of possible integrators. Zoho already promotes https://www.zoho.com/crm/omnichannel.html Omni Channel integration, but Apple Messages does not yet appear.
      • Possible to Turn Off Automatic Notifications for Approvals?

        Hello, This is another question regarding the approval process. First a bit of background: Each of our accounts is assigned a rank based on potential sales. In Zoho, the account rank field is a drop-down with the 5 rank levels and is located on the account
      • Will be possible to create a merge mail template for products?

        Hi, we would need to create a mail merge template for products (native) module. Will be possibile? or do you have a smart solutions to merge products data with a mail merge? thanks Chris
      • Maximum incoming email (attachment size) limit

        I see that the maximum size of received emails can now be set to 20MB, however I'd still like to see it be larger. It's embarrassing when a client sends me an email with a large attachment and I can't receive it. 50, even 100 MB seems reasonable these
      • [Free Webinar] Learning Table Series - Multi-currency management in Zoho Creator

        Hello everyone, We’re excited to invite you to another edition of the Learning Table Series webinar. As you may already know, we've moved to a purpose-based approach in the Learning Table Series this year. Each session now focuses on how a Zoho Creator
      • Outlook/Hotmail Blocking Zoho SMTP IPs (S3150)

        We are currently facing a serious deliverability issue with Zoho SMTP while sending transactional OTP emails for our production application. Emails sent to Outlook / Hotmail addresses are being rejected with the following error: 550 - 5.7.1 Unfortunately,
      • Outlook is blocking incoming mail

        Outlook is blocking all emails sent from the Zoho server. ERROR CODE :550 - 5.7.1 Unfortunately, messages from [136.143.169.51] weren't sent. Please contact your Internet service provider since part of their network is on our block list (S3150). It looks
      • LESS_THAN_MIN_OCCURANCE - code 2945

        Hi I'm trying to post a customer record to creator API and getting this error message. So cryptic. Can someone please help? Thanks Varun
      • Ranking by group in report

        Dears, I am new to Zoho Analytics and I would like to ask you guys help to creating a ranking column. Report type: Pivot Matter: I want to create a ranking column on the right of Percentage. This ranking is group by column Type, and ranking by Final Score/Percent.
      • How do I add a project template to a pre-existing project?

        Help....How do I add a project template to a pre-existing project? Right now I have to delete the old project, make a new one and add the project template at the time of creating the new project.
      • How to add line breaks in zoho.cliq.postToUser(...) message?

        In a CRM function using Deluge I'm sending this message and attempting to add some line breaks but they are ignored. Is there another way to add these breaks? My message: message: New urgent task\nDescription \nThis is a fake description.\n A new line?
      • Passing the image/file uploaded in form to openai api

        I'm trying to use the OpenAI's new vision feature where we can send image through Api. What I want is the user to upload an image in the form and send this image to OpenAI. But I can't access this image properly in deluge script. There are also some constraints
      • Users may not pick the fields to be shown as columns in the Choose Account window when creating a new Deal record

        Hi there, by talking with other users I found out that I, as an Admin, am the only one who can pick fields to be shown as columns in the Choose Account window when creating a new Deal record. In fact, if other users click on the "Add Column" symbol on
      • {Action Required} Re-authenticate your Google Accounts to Continue Data Sync

        Hello Users! To align with Google’s latest updates on how apps access files in Google Drive, we’ve enhanced our integration to comply with the updated security and privacy standards, ensuring safer and more reliable access to your data. With this update,
      • Why is the ability Customize Calls module so limited?

        Why can't I add additional sections? why can't I add other field types than the very limited subset that zoho allows? Why can I only add fields to the outbound/inbound call sections and not to the Call Information section?
      • Report Template - How to remove page break after each record?

        Hi, We have report template for a list report. It looks good at screen. But when printing, it creates a page break after each record. How to remove the it? So we can print multiple records in same page. Please look at the attached screenshots. Report Template Report Print Preview
      • Calendar report with order options and more quick view templates

        I think many of us regularly work with calendar-style reports. It would be great to be able to customize the quick view with new templates and have options to sort the entries for each day of the calendar by different criteria. I think this is an interesting
      • Sender Email Configuration Error.

        Hello Team, Hope you are all doing well. We are in the process of creating the Zoho FSM environment in the UAE. When we try to add the sender email address “techsupportuae@stryker.com”, we receive the error message: “Error occurred while sending mail
      • Shall we play a game?

        Presenting the very first game created using ZOHO Creator: Tic-Tac-Toe (or noughts and crosses) I made this to challenge myself and employ some of the new features of ZOHO Creator. I must admit that the code is very literal and not too elegant. There are plans to improve on the machine AI and streamline the code over time. Currently the code makes extensive use of functions for the machine "AI" - there are 12 of these.   The machine AI can be tricked, so to counteract that I made it exceedingly arrogant
      • CRM notes

        I want to be able to add notes to a task that do not necessarily get rolled up into an account or contact.   For example, I tasks to work on a Court Order for John Doe divorce account.  There might be lots of updates (in the form of notes) that employees
      • Next Page