Written by: Nimesh Chakravarthi, Co-founder & CTO, Struct | Last updated: August 24, 2026
Key Takeaways for Automated RCA
-
Automated root cause analysis ingests alerts, metrics, logs, traces, and code changes, then correlates signals in near real time and ranks evidence-backed hypotheses.
-
Trace context propagation with OpenTelemetry joins metrics, logs, and traces on a shared
trace_id, so responders avoid manual timestamp correlation. -
Defensible hypotheses rely on temporal precedence, dependency-graph proximity, explanatory coverage, and recurrence before surfacing to responders.
-
The seven-step reference architecture spans alert intake through incident resolution verification and code-agent handoff, creating a repeatable on-call pipeline.
-
Struct connects to your existing stack in under 10 minutes and delivers Slack-native automated RCA with incident resolution verification out of the box.
How Automated RCA Correlates Metrics, Logs, and Traces
Automated RCA correlates signals by propagating a shared identifier, the trace_id, across every telemetry type. This shared context lets the engine join a metric alert, the offending trace, and the exact log line without timestamp guessing. Trace context must propagate across service hops using the W3C traceparent header, every log line inside a span must carry trace_id and span_id, and metric points must carry trace_id as exemplars. OpenTelemetry SDKs expose these values automatically.
The OpenTelemetry Collector accepts traces, metrics, and logs on a single OTLP receiver (gRPC on port 4317 or HTTP on port 4318) and routes them through separate pipelines with batch and memory_limiter processors to one or more backends. This unified collector removes the need for three separate ingestion pipelines. Common production failures include async context loss across queue boundaries, sampling mismatches where logs retain trace_id values for traces that were never stored, and log shippers that strip trace fields.
A minimal OpenTelemetry collector pipeline wired to an LLM hypothesis engine looks like this:
receivers: otlp: protocols: grpc: endpoint: "0.0.0.0:4317" processors: batch: {} memory_limiter: limit_mib: 512 exporters: # Fan out to your existing backend (Datadog, Grafana, Prometheus remote_write) datadog: api: key: ${DD_API_KEY} # Sidecar exporter streams correlated bundles to the RCA hypothesis engine otlphttp/rca_engine: endpoint: "https://rca-engine.internal/v1/otlp" headers: Authorization: "Bearer ${RCA_ENGINE_TOKEN}" service: pipelines: traces: receivers: [otlp] processors: [memory_limiter, batch] exporters: [datadog, otlphttp/rca_engine] metrics: receivers: [otlp] processors: [memory_limiter, batch] exporters: [datadog, otlphttp/rca_engine] logs: receivers: [otlp] processors: [memory_limiter, batch] exporters: [datadog, otlphttp/rca_engine] The RCA engine receives a correlated bundle that includes the trace, metric exemplars, and log lines sharing the same trace_id. It then passes this bundle to an LLM hypothesis loop that ranks candidates before any human is paged.
Evidence an RCA Engine Should Require Before Acting
An RCA engine should require temporal precedence, dependency-graph proximity, explanatory coverage, and recurrence before promoting a hypothesis to the responder. Automated RCA systems score candidate causes across these dimensions and display which evidence raised or lowered each rank.
Four criteria work together to define a defensible hypothesis, and an RCA engine must evaluate all four before surfacing a candidate:
-
Temporal precedence: The suspected cause must precede the failure signal. If the deployment happened after the error spike, it cannot be the root cause.
-
Dependency-graph proximity: Topology-based suppression walks the dependency graph upward to mark alerts tracing to a failing ancestor as suppressed. Even when timing aligns, the candidate must sit in the causal path of the failure to avoid blaming an unrelated peer service.
-
Explanatory coverage: The hypothesis must account for all affected entities. A top candidate should explain symptoms across explored branches rather than only one noisy service.
-
Recurrence: A reliable automated RCA workflow tests whether a candidate also occurs during healthy periods and whether another common upstream factor explains both the candidate and the symptom. This final check prevents the engine from promoting spurious correlations.
Evidence-linked reasoning requires every root-cause conclusion to attach at least one verifiable telemetry, log, or source-data artifact before the conclusion reaches responders. This safeguard prevents hallucinated service names or nonexistent endpoints from reaching the on-call queue.
Seven-Step Reference Architecture for Automated RCA
This architecture covers the full pipeline from alert intake through incident resolution verification and code-agent handoff. Each step maps to a concrete engineering action your team can implement.
Step 1: Alert Intake from Slack and PagerDuty
Configure your alerting channels such as Slack, PagerDuty, Linear, or Sentry as the single entry point. Every alert that fires triggers the pipeline automatically, and the investigation starts without manual acknowledgement.
Step 2: Data Ingestion via OpenTelemetry and Vendor APIs
The pipeline ingests metrics, logs, traces, and cloud events through the OpenTelemetry Collector configuration shown earlier. Vendor API adapters pull supplementary context from Datadog, AWS CloudWatch, GCP Logging, Azure Traces, Grafana, and Prometheus remote_read endpoints in parallel.
Step 3: Change-Correlation Windows for Recent Deploys
A complete incident evidence bundle must include deployment and configuration history revealing recent change candidates. The pipeline queries GitHub commit history, deploy records, and feature-flag changes within a configurable lookback window, typically 30 to 60 minutes before alert onset, and attaches them to the correlated telemetry bundle.
Step 4: Hypothesis Generation with Confidence Scoring
The RCA engine generates ranked hypotheses using a combination of causal graph traversal and LLM-based reasoning. Candidates confirmed by multiple causal ranking methods receive the highest consolidated confidence scores.
|
Confidence Tier |
Signal Coverage |
Recommended Action |
|---|---|---|
|
High (≥ 0.85) |
Metrics, traces, and logs all corroborate, and temporal precedence is confirmed |
Surface to responder with suggested fix, and mark as eligible for automated remediation |
|
Medium (0.60–0.84) |
Two of three signal types corroborate, and change correlation is present |
Surface to responder for review, and hold automated action |
|
Low (< 0.60) |
Single signal type or contradictory evidence |
Return “needs investigation” rather than emitting a low-confidence hypothesis |
Step 5: Noise Reduction and Incident Grouping
AI-driven event correlation groups related alerts into a single incident, transforming, for example, 47 alerts into one correlated incident. Topology-based suppression marks downstream symptomatic alerts as children of the root cause, which prevents duplicate pages.
Step 6: Incident Resolution Verification Loop
After a fix is applied, the pipeline re-samples live observability data on a configurable settling window. Resolution is recorded only after verification of sustained compliance across re-samples on key signal types. If verification fails, the incident remains open and escalates to the on-call engineer rather than closing silently.
Step 7: Handoff to Code Agents or PR Creation
Once the root cause is confirmed and resolution verified, the pipeline hands off the full evidence bundle, including correlated telemetry, ranked hypothesis, and fix suggestion, to a code agent or generates a pull request directly. The AWS Deep Research RCA reference architecture demonstrates parallel hypothesis investigation by dynamically spawned subagents, with context enrichment from runbooks and past incidents stored in a knowledge base. Your team can replicate this pattern with any LangGraph-compatible agent framework or connect Struct to handle the entire pipeline in under 10 minutes.
See how Struct automates code agent handoff
Comparison of Automated RCA Platforms for On-Call Teams
Now that the seven-step reference architecture is clear, the next decision is which platform can deliver these capabilities without requiring you to build every component from scratch. The table below compares four platforms on the dimensions most relevant to on-call engineering teams at Series A to C SaaS companies. Use this comparison to see which platform aligns with your observability stack and whether you need out-of-the-box incident resolution verification or plan to build custom pipelines. Pricing reflects publicly available information as of August 2026. Contact each vendor for current enterprise quotes.
|
Platform |
Pricing Model |
Key Integrations |
Stated Limitation |
Best-Fit Audience |
|---|---|---|---|---|
|
Dynatrace Davis AI |
DPS consumption pricing, with enterprise contracts available for full AIOps |
Full-stack Dynatrace telemetry, with limited cross-vendor log ingestion without OneAgent |
Requires Dynatrace instrumentation across the stack, and cross-vendor correlation is constrained without OneAgent deployment |
Large enterprises already standardized on Dynatrace for APM and infrastructure |
|
Datadog Watchdog / Bits AI |
Per-host and per-log pricing, with AI features bundled at higher tiers |
Native Datadog metrics, logs, traces, APM, and synthetics. Cross-stack investigation into Sentry, GitHub, and cloud logging requires layering an additional tool |
RCA scope limited to Datadog-ingested telemetry, with no native Slack-first investigation workflow or incident resolution verification loop |
Teams already paying for Datadog who want in-platform anomaly detection without a separate tool |
|
Grafana (Grafana OnCall + Asserts) |
Open-source core free. Grafana Cloud Pro starts at a $19/month platform fee plus usage-based charges, with enterprise pricing on request |
Prometheus, Loki, Tempo, OpenTelemetry, and a broad plugin ecosystem |
Automated hypothesis ranking and resolution verification require custom configuration, with no out-of-the-box LLM-driven RCA pipeline |
Teams running self-hosted Prometheus and Loki stacks who prefer open-source tooling and have engineering bandwidth to build custom pipelines |
|
Struct |
Startup tier, up to 5 users and 30 issues per month, free. Growth tier with unlimited users, 200 issues per month, and white-glove onboarding, with a 30-day risk-free pilot included |
Requires access to logs via integrations. Full on-premise VPC-isolated deployment is not currently supported for organizations with zero-egress policies |
Series A to C SaaS engineering teams, typically 15 to 80 engineers, running Datadog or Grafana plus Slack who need Slack-native automated RCA with incident resolution verification out of the box |
Struct Implementation After the Neutral Architecture
After you map the seven-step architecture, Struct connects the entire pipeline to your existing stack in under 10 minutes. You authenticate your alert source such as Slack or PagerDuty, your code repository such as GitHub, and your observability context such as Datadog, GCP, AWS CloudWatch, or Grafana. Auto-investigations start immediately, with no YAML pipelines to write and no dashboards to configure.
Arcana, a Series B fintech with 40 engineers, reduced median investigation time from 30 minutes to 2 minutes, reclaimed 56 developer hours per month, and ran 2,100 plus automated investigations monthly with an 85 to 90 percent helpful rate after integrating Struct with Sentry, GitHub, GCP Cloud Logging, and Slack. Senior engineer hours on investigation dropped substantially after adding Struct on top of Datadog.
By the time you open your laptop after a 3 AM page, Struct has already correlated logs, mapped the timeline, identified the root cause, and posted a dynamically generated dashboard to your Slack thread. Struct performs regression analysis, correlates anomalies, generates impact summaries and incident reports, and integrates with tools like Slack, GitHub, and observability platforms for quick deployment in minutes. Large-scale customers report triage time reductions of about 80 percent.
Struct’s composable runbook architecture lets you paste your existing on-call runbook directly into the platform. The AI follows your exact operational procedures, including specific correlation ID formats, escalation paths, and blast-radius checks, for every alert that fires. Junior engineers gain a reliable starting point without relying on tribal knowledge.
Start automating your runbook in 10 minutes
Incident Resolution Verification in Practice
Incident resolution verification provides a closed-loop process that confirms an incident is genuinely resolved by re-sampling live observability data after a fix is applied, not just clearing the alert. Most platforms stop at hypothesis generation, while incident resolution verification closes the loop.
Struct’s Incident Tracker, launched August 3, 2026, runs an approximately one-minute automated verification loop against your observability data after each remediation action. The loop re-samples the same metrics, error rates, and latency distributions that triggered the original alert. If verification fails its first re-sample, the loop holds for one full settling window and re-samples again rather than immediately re-triggering the action, and three consecutive failures escalate to a human engineer.
This approach matters because a reboot acknowledgement does not prove sensing, connectivity, or business recovery, and verification of resolution must rely on evidence independent of command acknowledgement. Struct’s incident resolution verification treats silence as an unknown, not a success, and keeps the incident open until live telemetry confirms the fix held.
Google’s closed-loop incident resolution workflow demonstrated at DASH spans seven steps from Datadog detection through root-cause analysis, fix generation, and automated verification that confirms resolution against live telemetry data. Struct delivers the same pattern natively inside Slack without requiring a custom CLI session.
Azure Application Insights Live Metrics displays telemetry with one-second latency for fix validation during release. Struct’s verification loop integrates with Azure, Datadog, and GCP to pull that same live signal automatically, so you are not manually watching dashboards while a deploy rolls out.
Frequently Asked Questions
Does Struct send our logs outside our VPC?
Struct accesses your logs and observability context ephemerally via authenticated integrations with AWS CloudWatch, GCP Logging, Azure, and Datadog. Logs are processed during the investigation and are not stored permanently. Struct is SOC 2 Type II and HIPAA compliant, which covers the compliance requirements of most Series A to C SaaS and fintech companies. If your organization enforces a strict zero-egress policy requiring full on-premise deployment, Struct’s Enterprise tier includes sidecar and on-prem support options, so you can discuss your architecture with the team.
How does Struct handle runbook customization for our specific system?
You can paste your existing on-call runbook directly into Struct’s configuration. The platform supports custom instructions, proprietary correlation ID formats, and composable widgets that guarantee specific visual data is always pulled for defined alert types. When an alert fires, Struct follows your exact operational procedures, including escalation paths, blast-radius checks, and service-specific queries, rather than applying a generic investigation template. A junior engineer on their first on-call shift receives the same investigation quality as your most experienced SRE.
What telemetry quality does Struct require to produce accurate root cause analysis?
Struct relies on the observability data your stack already emits. The ideal setup includes an alerting trigger such as Slack, PagerDuty, or Sentry, structured logs with trace IDs in a cloud logging platform such as AWS CloudWatch, GCP, or Azure, and a code repository such as GitHub for change correlation. If your services emit OpenTelemetry-instrumented traces with the W3C traceparent propagation described earlier, hypothesis confidence scores will be significantly higher. If your logging is sparse or lacks trace context, Struct still investigates using available signals, and the helpful investigation rate improves directly with telemetry quality.
How quickly can a new engineer become productive on on-call with Struct?
New engineers can take on-call shifts from day one because Struct performs the first-pass investigation automatically before the engineer engages. By the time they open the Slack thread, Struct has already posted the correlated timeline, blast radius, root cause hypothesis, and suggested fix. The Slack-native conversational interface lets them ask follow-up questions, such as pulling logs from five minutes prior, testing an alternative hypothesis, or checking whether a specific user is affected, without switching tools or knowing which Datadog query to write. This workflow removes the tribal knowledge barrier that typically keeps new hires off the on-call rotation for months.
How does Struct’s incident resolution verification differ from a standard alert auto-resolve?
Standard alert auto-resolve closes a ticket when the alerting threshold drops below its trigger value and does not confirm that the underlying condition is fixed. Struct’s incident resolution verification re-samples the same metrics, error rates, and latency distributions that triggered the original alert after a fix is applied. It requires sustained compliance across multiple independent signal types before marking the incident resolved and keeps the incident open with escalation if verification fails. You avoid discovering at 9 AM that an incident resolved at 3 AM but the fix never actually held.
Conclusion: Moving to Evidence-Backed On-Call
The seven-step reference architecture, spanning alert intake, data ingestion via OpenTelemetry, change-correlation windows, confidence-ranked hypothesis generation, noise reduction, incident resolution verification, and code-agent handoff, gives your on-call team a repeatable pipeline that replaces manual log hunts with evidence-backed automation. Automated RCA pipelines have demonstrated root cause diagnosis in under six minutes on production EKS environments where manual correlation previously took hours.
Before implementing, audit two things: your current telemetry quality and your existing runbooks. Confirm that your services emit trace_id-tagged logs and OpenTelemetry traces, and confirm that your runbooks are documented well enough to encode into an automated pipeline. Both gaps are fixable in days, and both directly determine the accuracy of any automated RCA system you deploy.
Struct connects to your existing Datadog, Grafana, or Prometheus stack as an investigation layer, not a replacement, and delivers incident resolution verification out of the box in a 10-minute setup. Arcana’s results, described earlier, demonstrate the impact of automated RCA at scale. Your next on-call rotation does not need to start with a 45-minute log hunt.
