TransMail - Duplicate notification emails

TransMail - Duplicate notification emails

Dear users, 

Thanks for your continuous support to Zoho and our services. 

We would like to apologize for a minor inconvenience caused to some of you, where you might have received duplicate copies of the same notification or alert emails triggered from our services. 

Zoho TransMail is the transactional email service used by Zoho Services,  to send transactional emails (notifications/ alerts/ service messages) to our users. These Zoho services use the APIs of TransMail to trigger emails via the Transmail servers. Please find the details of the incident and the rootcause below. 

Issue details: 
Transmail triggered duplicate copies of emails instead of sending a single email. We found that some emails were duplicated due to a technical issue in one of our servers. We identified the root cause of the issue and fixed it and also rolled out the fix to minimize the impact. 

Issue duration: 1 hours 25 minutes (01:50 to 03:17PDT) 

Affected services: 
Zoho Desk
Zoho Blueprint
Zoho CRM
Zoho People
Zoho ShowTime
Site 24x7

Impact:
Around 100 K transactional emails/ notification emails were duplicated to the recipients including all the above services. We are deeply sorry for the trouble. 

A total of around 1.6 M emails were duplicated and placed in the SMTP out queue. However we acted immediately and blocked the emails triggered from Desk to external servers. Around 1.5 M emails were blocked and the duplicate emails from Desk were blocked. When blocking, some normal emails were also blocked and removed from the queue to minimize the impact. 
So some users may have missed certain notification emails due to this fix. 

Root-cause Analysis: 
We were unable to perform updates in one of our servers due to a memory issue affecting a dependent application. So even after an email was sent, we were unable to mark the email as processed. Hence the emails were being retried again, until the error is corrected and the email is marked as sent. 

So the emails were duplicated as they were repeatedly processed.

We apologize profusely for the inconvenience caused for you due to the duplication of notification emails from our services. We are also taking required precautionary steps to ensure that similar issues will not recur in future. 

Thanks,
TransMail team






                            Zoho Desk Resources

                            • Desk Community Learning Series


                            • Digest


                            • Functions


                            • Meetups


                            • Kbase


                            • Resources


                            • Glossary


                            • Desk Marketplace


                            • MVP Corner


                            • Word of the Day



                                Zoho Marketing Automation
                                        • Sticky Posts

                                        • TransMail Overview

                                          Note: TransMail is now ZeptoMail. Click here to know more. Hey there! It's been 5 months since we launched Zoho's transactional email service — TransMail. We've had great response and onboarded many users. I'd like to now give an overview of TransMail
                                        • TransMail has a new name—ZeptoMail!

                                          Tried navigating to TransMail's community forum but see a different name now? That's because TransMail has a new name. TransMail is now ZeptoMail! As we’ve grown from an internal service used mainly by other Zoho products to an up-and-coming competitor
                                        • Welcome to TransMail community

                                          Note: TransMail is now ZeptoMail. Click here to know more. Hi everyone, Welcome to TransMail's community forum. Though TransMail has been in use inside Zoho for nearly half a decade, we only recently launched it as a service to all of you. We'd firstly


                                        Manage your brands on social media



                                                Zoho TeamInbox Resources

                                                  Zoho DataPrep Resources



                                                    Zoho CRM Plus Resources

                                                      Zoho Books Resources


                                                        Zoho Subscriptions Resources

                                                          Zoho Projects Resources


                                                            Zoho Sprints Resources


                                                              Qntrl Resources


                                                                Zoho Creator Resources


                                                                  Zoho WorkDrive Resources



                                                                    Zoho Campaigns Resources

                                                                      Zoho CRM Resources

                                                                      • CRM Community Learning Series

                                                                        CRM Community Learning Series


                                                                      • Tips

                                                                        Tips

                                                                      • Functions

                                                                        Functions

                                                                      • Meetups

                                                                        Meetups

                                                                      • Kbase

                                                                        Kbase

                                                                      • Resources

                                                                        Resources

                                                                      • Digest

                                                                        Digest

                                                                      • CRM Marketplace

                                                                        CRM Marketplace

                                                                      • MVP Corner

                                                                        MVP Corner




                                                                            Design. Discuss. Deliver.

                                                                            Create visually engaging stories with Zoho Show.

                                                                            Get Started Now