# OpenRCA ![Category](https://img.shields.io/badge/Category-Agent-1f6feb) ![Metric](https://img.shields.io/badge/Metric-RCA%20Accuracy-2da44e) ![Tasks](https://img.shields.io/badge/Tasks-335-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-low-d29922) ![Created](https://img.shields.io/badge/Created-2025-555) [![Paper](https://img.shields.io/badge/Paper-OpenReview-8c1b13)](https://openreview.net/forum?id=M4qNIzQYpd) [![GitHub](https://img.shields.io/badge/GitHub-microsoft%2FOpenRCA-181717?logo=github&logoColor=white)](https://github.com/microsoft/OpenRCA) > **Quick answer:** OpenRCA is a Microsoft-created benchmark asking "Can Large Language Models Locate the Root Cause of Software Failures?", built from 335 real failures across three enterprise software systems (Telecom, Bank, and Market) with over 68GB of accompanying logs, metrics, and traces that a model must analyze to identify the true root cause. ## At a Glance **What it tests:** Whether an LLM (or agent) can take a natural-language query plus large volumes of raw telemetry data and correctly diagnose the root cause of a real production software failure. **Why it matters:** Root-cause analysis (RCA) is one of the most time-consuming tasks in enterprise software operations; OpenRCA directly tests whether current LLMs can meaningfully accelerate this process using the same messy, large-scale telemetry human engineers rely on. **Known limitations:** With 68GB+ of logs, metrics, and traces per system, OpenRCA is heavily constrained by context-window and retrieval limitations, meaning scores reflect an entangled mix of root-cause reasoning ability and an agent's data-retrieval/tool-use strategy. ## What OpenRCA Measures OpenRCA draws on 335 real incidents across three enterprise systems — Telecom, Bank, and Market (the latter two further split into cloudbed-1/cloudbed-2 sub-environments) — each paired with a natural-language query and the full telemetry record (logs, metrics, and distributed traces) surrounding the failure. Given this data, a model must identify the true root cause of the incident, mirroring the diagnostic workflow of a human site-reliability or DevOps engineer. Because the underlying telemetry for each incident can span gigabytes of raw data, OpenRCA is explicitly designed to test agentic systems' ability to search, filter, and reason over large volumes of operational data rather than answering from a small, pre-filtered context window. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Agent / root cause analysis from enterprise telemetry | | Metric | Root cause identification accuracy (%) | | Number of tasks | 335 failures across 3 enterprise systems | | Saturation | Low — large telemetry volumes and multi-step diagnosis make this a hard, unsaturated task for current LLMs and agents | | Created by | Junjielong Xu, Qinan Zhang, Zhiqing Zhong, Shilin He, Chaoyun Zhang, Qingwei Lin, Dan Pei, Pinjia He, Dongmei Zhang, Qi Zhang (Microsoft) | | Source paper | [Xu et al. 2025, ICLR](https://openreview.net/forum?id=M4qNIzQYpd) | | GitHub | [microsoft/OpenRCA](https://github.com/microsoft/OpenRCA) | ## How OpenRCA Is Scored A model's output — its identified root cause for a given incident query — is compared against the ground-truth root cause recorded for that incident; the headline metric is the percentage of the 335 incidents where the model's diagnosis matches the documented true root cause. ## State-of-the-Art Results > OpenRCA's authors note the benchmark is used to explore how far agent systems can go in locating real software root causes, with current LLM-based approaches leaving substantial headroom against the large-scale, multi-system telemetry involved. See the [official GitHub repository](https://github.com/microsoft/OpenRCA) for the current per-model breakdown. ## OpenRCA on Benchgen > No Benchgen results yet — [be the first to run OpenRCA](/benchmarks/junjielong-xu-qinan-zhang-zhiqing-zhong-shilin-he-chaoyun-zhang-qingwei-lin-dan-pei-pinjia-he-dongmei-zhang-qi-zhang-microsoft/openrca). ## OpenRCA vs Other Benchmarks | Benchmark | What it tests | Tasks | Saturation | |---|---|---|---| | **OpenRCA** | Root-cause analysis from enterprise telemetry | 335 | low | | [WideSearch](/benchmarks/ryan-wong-jiawei-wang-junjie-zhao-li-chen-yan-gao-long-zhang-xuan-zhou-zuo-wang-kai-xiang-et-al-bytedance-seed/widesearch) | Broad, verifiable web information-gathering | 200 | very low | | [Terminal-Bench Hard](/benchmarks/mike-a-merrill-alexander-g-shaw-nicholas-carlini-et-al-stanford-laude-institute/terminal-bench-hard) | Hard, realistic CLI agent tasks | 89 | low | OpenRCA is distinguished from other agentic benchmarks by centering on operational telemetry analysis at enterprise scale, rather than web search or terminal command execution, making it a distinct diagnostic-reasoning signal for AIOps-style agents. ## Run OpenRCA on Your Model Benchgen lets teams evaluate their own model against OpenRCA's real enterprise-incident dataset, surfacing whether it can reliably diagnose root causes from large-scale operational telemetry. ## Frequently Asked Questions

What is OpenRCA?

OpenRCA is a Microsoft-created benchmark testing whether LLMs can locate the root cause of real software failures using 335 enterprise incidents and their accompanying logs, metrics, and traces.

What does a good OpenRCA score look like?

Because OpenRCA involves diagnosing real incidents from 68GB+ of raw telemetry per system, a good score is one that meaningfully exceeds naive or majority-class baselines across the 335-incident set, reflecting genuine root-cause reasoning rather than pattern-matching on query text alone.

Who created OpenRCA?

OpenRCA was created by Junjielong Xu and collaborators at Microsoft, published at ICLR 2025.

OpenRCA — Leaderboard

OpenRCA Leaderboard