1. Overview

Zia Root Cause Analysis (Zia RCA) analyzes every test case in an environment as soon as its run finishes. It tags each test case automatically and presents the analysis in plain language, removing the need to open every test case individually to review its logs and run history.

Zia RCA can also generate a full analysis for that same environment, including a health score, a failure breakdown, recurring patterns, and a prioritized list of fixes. The analysis gives you a structured view of the environment's health and which fixes to prioritize, without needing to review every failure individually.

1.1 How Zia RCA analyzes your test runs

A test plan run executes a set of test cases against one or more environments, such as different browsers, operating systems, or screen resolutions. Zia RCA analyzes these results at two levels, for each test case in the environment and for the overall environment.

1.1.1 Test case level analysis

Once Zia RCA is enabled for a test plan, Zia analyzes each test case in an environment automatically as soon as its run finishes. For a failed test case, this identifies why it failed and suggests a possible fix. For a passed test case, it highlights a key finding and recommended improvements.

Every test case, whether passed or failed, also includes the following details:

1. Severity: Indicates the likely impact of the result.
2. Confidence: Shows how certain Zia is about its analysis.
3. Run history: The test case's execution history over its last 10 runs.
4. Stability: Indicates how consistently the test case has passed.
5. Pattern: Indicates whether the test case is repeating a failure it has shown before failing for the first time, or continuing to pass consistently.
6. Performance: Shows how the test case's duration compares with the median duration of its recent runs.

For details about these fields, see Zia analysis for a test case

Types of Zia tags

Every test case receives one or more tags that capture its priority, stability, pattern, or performance characteristics. These Zia tags consist of one primary tag representing its highest-priority classification, plus any additional tags that also apply.

Zia can assign tags in the following categories:

1. Priority: Indicates the likely impact of the classification.
2. Stability: Shows how consistently the test case has passed in recent runs.
3. Pattern: Flags whether the test case has failed in the same way before.
4. Performance: Compares the test case's duration with the median duration of its recent runs. 
   

To learn more about tags, see Tag categories in Zia RCA

1.1.2 Environment level analysis

Once every test case in the environment has been individually analyzed, Zia analyzes the environment as a whole. This builds on what's already known about each test case, its failure cause or improvement classification, its Zia Tag, and its run history. Combining all of this produces a single view of the environment, a Health Score, a breakdown of failures, recurring patterns across the run, and a prioritized list of fixes.

Apart from the environment's total, passed, and failed test case counts, Zia RCA also maintains the flaky count once this analysis is generated.

Note: If even one test case in the environment could not be individually analyzed by Zia, the environment analysis does not run automatically and must be started manually.

For details about what the environment analysis includes, see Zia analysis for an environment

1.2 Data sent to your AI model

Zia RCA sends data to the configured AI model both when automatically tagging a test case and when generating a full analysis for an environment. The data includes the logs, test step text, and run history required to classify each test case.

When Zia RCA is configured with a third-party AI integration, such as OpenAI, Gemini, or Claude, the data leaves Zoho's infrastructure and is sent to the respective AI provider. When Zoho GenAI is configured, the data remains within Zoho's infrastructure. Learn more

For test cases containing sensitive information, evaluate the applicable data governance requirements before configuring a third-party AI provider.

Usage quotas

Usage depends on the AI provider powering Zia. If Zia is using Zoho GenAI, standard platform AI quotas apply at the organization and user level. If a third-party provider is configured, the usage will be managed by the respective service provider.

2. Tag categories in Zia RCA

Every tag Zia assigns belongs to one of four categories, Priority, Stability, Pattern, or Performance. Each category groups tags that describe the same kind of concern, for example, every Stability tag says something about how consistently a test case has passed.

When multiple tags apply to a test case, Zia uses a fixed display order to determine which tag is shown as the Primary tag. The first applicable tag in this order is shown as the Primary tag, and the remaining applicable tags are displayed in the same order.

The display order is defined for individual tags rather than tag categories. Tags from different categories can therefore have different positions in the display order. For example, a Pattern tag can outrank a Stability tag, and a Priority tag rank below tags from other categories.

3. Zia analysis for a test case

Every analyzed test case has a full analysis available inside the preview page, shown as the Zia Analysis. The analysis card contains the Zia Tag in order, the analysis summary, and the parameters Zia assessed.

Note: After enabling Zia RCA, at least one previous run is required to perform the analysis.

3.1 Analysis summary

The analysis summary provides every test case with a classification of its outcome, why it failed, or what could be improved even though it passed. A failed test case is assigned one Primary cause for failure, which explains why it failed. A passed test case instead receives improvement suggestions worth addressing even without a failure.

3.1.1 RCA for failed test cases

The analysis summary for a failed test case shows its Failure Cause alongside Zia's Primary cause and Possible fix for that specific failure.

Types of Failure cause

Each failed test case is assigned one Failure Cause, shown as its Primary Cause. This identifies why the test case failed and anchors the possible fix Zia suggests.

1. Primary cause: A cause statement explaining why the test case failed. For example, "The element could not be found using its current locator, which suggests the page structure has changed."
2. Possible fix: The issue identified by Zia and the recommended resolution. For example, "The button's ID appears to have changed since the last passing run. Update the locator to use the new ID." 
   

3.1.2 RCA for passed test cases

The analysis summary for a passed test case shows its Improvement category alongside Zia's Key finding and recommended Improvements for that specific test case.

Types of Improvement categories

A passed test case will display suggestions for improvement. Each suggested improvement is grouped under one of the following dimensions.

What Zia found

1. Key finding: A statement identifying what could be improved about the test case. For example, "This test case passed, but response time has increased 40% over the last 5 runs."
2. Improvements: For each finding, a list of recommended actions that can be taken to improve the test case.  
   

3.2 Parameters assessed during analysis

1. Severity: Indicates the potential impact of the results as High, Medium, or Low. For example, an Application Bug blocking a core workflow is typically High, while a minor Timing Issue affecting a single step is more likely to be Low.
2. Confidence: Zia's level of confidence in its analysis, shown as a percentage. For low-confidence analysis, the execution logs should be reviewed before acting on the recommendation.
3. Run history: The visual indicator of each run's outcome, up to the last 10 runs, shown in sequence, along with the most recent result.
4. Success rate: The percentage of runs that passed, out of up to 10 of its most recent runs, along with the most recent pass or failure status.
5. Stability: Shows how consistently the test case has passed across up to its last 10 runs.
6. Pattern: Indicated whether the test case's result reflects a known pattern, such as a first-time failure, a repeated one, or a stable run.
7. Performance: The test case's duration compared with the median duration of up to its last 10 runs, including both passed and failed runs.

For information about where the full analysis appears and how to open it, see Viewing the Zia analysis for a test case

4. Zia analysis for an environment

Zia RCA operates for the environment as a whole. As part of this, every test case in the environment is individually analyzed, as covered in Zia analysis for a test case, before performing the overall analysis.

Note:

1. If even one test case in the environment could not be individually analyzed, the environment analysis does not run automatically, they must be started manually. Learn more
2. Only one analysis can run per environment at a time. Starting an analysis in a different environment does not interrupt one already running elsewhere.
3. Once generated, an environment analysis does not refresh automatically. Run the test plan again to perform a new analysis.
4. Deleting a test plan run also deletes its Zia RCA output for that run, including its test case analysis and the environment analysis. This does not affect Zia RCA output for any other run of the same test plan.

4.1 Components of the environment analysis

The environment analysis is made up of the following components:

1. 4.1.1 Analysis overview
2. 4.1.2 Analysis breakdown   
   

4.1.1 Analysis overview

The environment analysis gives you insights into the run's health and highlights issues that have changed since the previous comparable run. The analysis card displays the Run Health Score and headline numbers for new failures, newly flaky test cases, regressions, and estimated time to fix. 

Run Health Score

The Run Health Score indicates the overall health of the environment's test results, from 0 to 100.

How the Run Health Score is calculated

The Run Health Score starts at 100 and applies deductions for failed test cases, regressions, flaky test cases, and failure causes:

Health Score = round( 100 − (FailureRate × 60) − min(Regressions × 2, 20) − min(FlakyCount × 1.5, 15) − min(SeriousCauseCount × 1, 10) )

where:

1. FailureRate: The proportion of failed test cases, expressed between 0 and 1, so its deduction naturally maxes out at 60 points.
2. Regressions: Test cases that previously passed consistently but have recently started to fail. Each regression deducts 2 points, up to a maximum deduction of 20 points.
3. FlakyCount: Test cases that alternate between passing and failing across recent runs, with no script changes in between. Each deducts 1.5 points, up to a maximum of 15.
4. SeriousCauseCount: The number of failed test cases with a serious Failure Cause, such as Application Bug or Infrastructure Issue. Each deducts 1 point, up to a maximum of 10.

Note:

1. The score is rounded to the nearest whole number after all deductions are applied. If the resulting score is below 0, the score is set to 0. The Run Health Score therefore always falls between 0 and 100.
2. A run with no test cases receives a score of 100 because the failure rate, regression count, flaky count, and serious-cause count are all zero.

4.1.2 Analysis breakdown

The environment analysis presents its findings across four tabs:

A. Summary

A summary provides a narrative overview of the analysis, highlighting the most significant failure patterns, contributing factors, and reliability risks identified in the run.

This provides a quantitative view of the environment's test results, including the number of failed, passed, not analyzed, and flaky test cases. It also shows the dominant cause among failures, and how many test cases have a Zia Analysis

Note:

1. The "not analyzed" count can include test cases that never ran, as well as test cases that ran but could not be analyzed.
2. When multiple failure causes are tied for the highest count, No dominant cause is displayed.

B. Failure Distribution

Shows how failed test cases are distributed across different failure causes as a bar chart. Each bar's length represents that cause's share of the environment's total failures. Together, the bars account for every failed test case.

Key insights

These insights counts how many test cases across the environment carry each Priority, Stability, Pattern, and Performance tag. This reveals whether the same kind of concern, such as flakiness or a performance issue, recurs across multiple test cases in the environment. For the full list of tags, see Tag categories in Zia RCA.

C. Recurring Patterns

Recurring patterns lists observations describing failures that share a common root cause across multiple test cases in the environment. Each pattern points to a shared underlying cause that a single corrective action could address, rather than requiring a separate fix for every affected test case. For example, several test cases failing while waiting on the same slow-loading element, or a set of failures all tracing back to mismatched test data.

D. Recommended Actions

These actions lists a prioritized set of fixes based on the findings from the environment analysis. Each action is ranked according to its expected impact and assigned to the team responsible for addressing it, such as DevOps, QA, or Development. Recommendations can include fixing application defects, updating test scripts, updating locators, addressing environment or infrastructure problems, and investigating recurring failures.

Note:

1. During analysis, Zia heals any broken locators it detects so the analyzed run can complete. To apply a healed locator to future runs as well, it needs to be approved through the same approval flow used by Self-healing for live preview and Test Plan runs. Learn how to review and approve proposed locators
2. Deleting a test plan run also deletes its Zia RCA output for that run, including its test case analysis and the environment analysis. This does not affect Zia RCA output for any other run of the same test plan.

For information about generating an environment analysis, see Viewing the Zia analysis for an environment

5. Metrics in Zia RCA

Zia RCA reports metrics that summarize a run's results, at both the run and environment level. Total, Passed, and Failed are basic run results that QEngine tracks for every run. Flaky and Run Health Score are specific to Zia RCA, produced as part of the environment analysis

6. Points to note

1. Usage depends on the AI provider powering Zia. If Zia is using Zoho GenAI, standard platform AI quotas apply at the organization and user level. If a third-party provider is configured, the usage will be managed by the respective service provider.
2. Only one environment analysis can run at a time for a given environment. Initiating an analysis in a different environment does not interrupt the one already in progress.
3. An environment analysis does not refresh automatically once generated. Run the test plan again to perform a new analysis.
4. Deleting a test plan run also deletes its Zia RCA output for that run, including the test case and the environment analysis. This does not affect Zia RCA output for any other run of the same test plan.
5. The Health Score reflects only failed test cases. Tags and findings on passed test cases do not affect the score.
6. A locator Zia fixes during analysis only applies to future runs once it's approved. Approval follows the same process as Self-healing for live preview and test plan runs. See Self-healing in Zoho QEngine

7. FAQ

8. Related links

1. Configuring Zia for Zoho QEngine
2. Test Plans
3. Results
4. Run History
5. Performance metrics in Zoho QEngine
6. Configure and view Zia RCA in Zoho QEngine

9. What's next