Written by: Nimesh Chakravarthi, Co-founder & CTO, Struct
Key Takeaways
- AIOps root cause analysis ingests logs, metrics, traces, and change events, then runs them through a seven-step pipeline that suppresses noise, maps dependencies, correlates events, and delivers ranked explanations with evidence.
- The pipeline begins with data ingestion and normalization, then moves through noise reduction, topology mapping, event correlation, probabilistic modeling, recent-change detection, and finally ranked explanations with automated resolution verification.
- Topology-aware correlation and causal-inference models improve accuracy, cutting average investigation time from 30–45 minutes to about 5 minutes while boosting top-1 root-cause accuracy from 42% to 89%.
- Recent-change detection links anomalies to the latest deployments, configuration updates, and feature flags, surfacing the most probable change-induced failures that manual triage often misses.
- Struct automates this entire pipeline so engineers review ranked root causes instead of hunting through alerts. See Struct walk through a live RCA in a short demo.
Telemetry Required Before Any Root-Cause Model Can Run
No root-cause model produces reliable output without a complete, normalized data layer underneath it. AIOps pipelines ingest logs, metrics, traces, events, tickets, change records, and configuration data from on-premises, public cloud, private cloud, SaaS, networks, and endpoints into a unified, normalized data layer. Because every downstream correlation, causal inference, and ranking step depends on this layer, its quality caps the value of the entire pipeline.
Required inputs include application logs, infrastructure metrics, distributed traces, deployment events from CI/CD pipelines, and alert payloads from tools like Datadog, Sentry, and PagerDuty. Typical outputs are normalized event records with shared timestamps, service identifiers, and trace IDs that downstream correlation engines can join. The engineering consideration: platforms fed only raw alerts without topology data and CI/CD change events default to weaker time- and content-based grouping, which degrades root-cause accuracy significantly.
How the Pipeline Suppresses Noise Without Losing Signal
Noise reduction collapses the alert storm into a small set of actionable incidents by deduplicating repeated alerts and suppressing downstream symptoms. A large share of alerts are duplicates or symptoms rather than root causes. AIOps platforms that apply intelligent correlation reduce that volume substantially by consolidating related events into fewer incidents.
The engineering consideration: suppression logic must be topology-aware, not purely time-based. Without a live dependency graph, the engine cannot distinguish between a root cause and its downstream symptoms, so it suppresses alerts randomly based on timing alone. Topology-based correlation requires a live dependency graph derived from trace data rather than a static diagram; stale graphs cause incorrect suppression of alerts, hiding real failures behind assumed relationships that no longer reflect production.
1) Data Ingestion and Normalization Across Your Stack
This step produces a single, queryable event stream from every telemetry source the system touches. An AIOps incident response pipeline ingests metrics, logs, traces, and events via the OpenTelemetry collector, which applies batching, sampling, redaction, and semantic-convention enrichment before exporting signals.
Inputs are raw payloads from Datadog, Sentry, AWS CloudWatch, GCP Logs, GitHub webhooks, and PagerDuty. Outputs are normalized records with canonical field names, UTC timestamps, service ownership tags, and propagated trace IDs. The engineering consideration: the quality of the ingestion and normalization layer caps the value of all downstream analysis. Missing trace IDs at this stage break correlation three steps later.
Struct connects to your existing Datadog, Sentry, GCP, AWS, and GitHub integrations in under 10 minutes, normalizing telemetry without requiring schema changes or custom instrumentation. See how Struct normalizes your telemetry in a 15-minute demo
2) Noise Reduction and Alert Filtering at Scale
With a normalized event stream in place, the next challenge is volume. A single underlying failure can trigger hundreds of redundant alerts. The goal of noise reduction is to collapse that storm into a small set of actionable incidents by deduplicating repeated alerts and suppressing downstream symptoms. Noise reduction occurs through three correlation axes: deduplication by normalized fingerprint, temporal grouping within a sliding time window, and topological grouping that uses a live service dependency graph derived from OTel spans to suppress downstream symptoms and surface the upstream root cause.
Inputs are the normalized event stream from step one. Outputs are deduplicated incident candidates, each representing one probable underlying failure. The engineering consideration: storm detection monitors the rate-of-alerts derivative, and when it crosses a threshold the engine switches to aggressive symptom suppression and hunts for the apex of the dependency tree. This behavior prevents alert floods from overwhelming the correlation engine during cascading failures.
3) Topology and Dependency Mapping for Root-Cause Paths
Without understanding which services depend on each other, a root-cause engine cannot distinguish between an upstream failure and the downstream symptoms it triggers. Every alert looks equally important. The goal of topology mapping is to construct a live, traversable graph of service relationships so the pipeline can trace failures upstream to their origin. Topology mapping constructs service dependency graphs, runtime call graphs, and causal graphs to determine failure propagation paths; Netflix’s real-time service map and Kuaishou’s KRCA system demonstrate recursive dependency traversal and suspicious-path scoring on failure rate and latency.
Inputs are distributed trace spans, service mesh telemetry, and CMDB or tag-based ownership metadata. Outputs are a directed dependency graph with edge weights reflecting call frequency and error rate. The engineering consideration: AWS recommends combining static analysis of code repositories and configuration files with runtime telemetry and network flows to catch both intended and actual dependencies, because static-only graphs miss runtime relationships introduced by feature flags or dynamic service discovery. Accurate dependency maps cut investigation time during outages from hours to minutes by allowing responders to trace problems upstream through dependency chains.
4) Event Correlation Across Time, Topology, and Similarity
This step groups related signals into a single incident candidate and identifies which signal is the probable origin. Signal correlation compresses hundreds of simultaneous alerts into incident candidates using topology-based suppression that walks the dependency graph upward, temporal proximity grouping that buckets alerts by service, severity, and region within a time window, and trace context correlation that connects telemetry via propagated trace IDs.
Inputs are deduplicated events from step two and the dependency graph from step three. Outputs are enriched incident objects with a candidate root-cause node, a confidence score, and linked evidence. The engineering consideration: event correlation uses three axes, topology, time anchored by deploy timestamps, and Granger causality statistical tests, to group alerts and identify which recent change likely caused an incident.
To see how the three correlation axes combine into a single decision, consider how the scoring function weights each signal. The following Python snippet shows the exact formula Struct uses: topology overlap receives 45% weight, temporal proximity 35%, and trace ID matching 20%, with any score above 0.70 promoting the event cluster to a full incident investigation:
import numpy as np def correlation_score( topology_overlap: float, # 0–1: shared dependency graph nodes temporal_proximity: float, # 0–1: normalized inverse time delta trace_id_match: bool # exact trace ID propagation match ) -> float: """Returns a weighted correlation score (0–1). Scores above 0.7 are promoted to incident candidates and passed to Struct's resolution verification loop. """ weights = {"topology": 0.45, "temporal": 0.35, "trace": 0.20} trace_score = 1.0 if trace_id_match else 0.0 score = ( weights["topology"] * topology_overlap + weights["temporal"] * temporal_proximity + weights["trace"] * trace_score ) return round(score, 4) # Example: same dependency subtree, 12-second delta, matching trace ID print(correlation_score(0.85, 0.92, True)) # → 0.8765 Incidents scoring above 0.70 are promoted to the probabilistic modeling stage. Lower-scoring candidates are held as noise unless a subsequent alert raises the group score.
5) Probabilistic Modeling and Scoring for Causal Ranking
Correlation scores reveal which events happened together, but they do not reveal which event caused the others. A database slowdown and an API timeout may correlate perfectly, yet only one is the root cause. The goal of probabilistic modeling is to rank candidate root causes by posterior probability given all available evidence, and to separate causal relationships from coincidental co-occurrence. To distinguish causation from correlation, implementing causal inference models with libraries such as causalnex or DoWhy to construct a Directed Acyclic Graph (DAG) that encodes known domain relationships, for example, “database latency causes application errors,” enables an AIOps RCA engine to separate coincidental patterns from actual causes rather than relying solely on statistical correlation.
Causal hypothesis generation builds directed graphs of cause-effect relationships using the PC algorithm, Granger causality, or the RCD method that models faults as interventions on the failing node. Research on multivariate time-series anomaly detection (arXiv:2206.15033) frames anomalies as violations of regular causal mechanisms and uses causal structure to identify root causes in complex time-series data.
Inputs are enriched incident candidates from step four plus historical incident data used to train the causal model. Outputs are ranked root-cause hypotheses, each with a confidence score and a provenance chain. The engineering consideration: correlation-based AIOps approaches have reached up to 42% Top-1 root-cause accuracy on the RCAEval benchmark (N=735), while causal approaches have reached 89% as measured in the AI SRE Benchmark, the accuracy jump cited in the key takeaways above.
6) Recent-Change Detection Around Deploys and Config Updates
This step correlates system anomalies with the most recent deployments, configuration changes, feature flag toggles, and dependency updates that preceded the incident window. AIOps data ingestion pipelines consume deployment events, config changes, Git commits, and feature flag toggles in addition to metrics, logs, and traces to support correlation with incidents; during root cause analysis, AIOps engines build causal graphs that link a specific deployment through intermediate signals like memory usage spikes and latency increases to the incident, assigning a confidence score.
Inputs are CI/CD webhook payloads, Git commit metadata, and the enriched incident object from step five. Outputs are a ranked list of change events annotated with their causal probability relative to the incident. The engineering consideration: AIOps platforms ingest deployment events from CI/CD systems such as GitHub Actions, GitLab CI, and Argo CD via webhooks so that deploy timestamps can serve as anchors during time-based correlation of anomalies and incidents. Without deploy event ingestion, the pipeline treats a post-deploy regression as an unexplained anomaly rather than a change-induced failure, which is the most common source of missed root causes in practice.
Most incidents follow a deploy. Because change-induced failures are so common, Struct’s Deploy Guard feature extends this detection upstream. It runs instrumentation review on pull requests and post-deploy health checks before an incident ever fires, catching regressions that would otherwise only surface during step six of the RCA pipeline.
7) Ranked Explanations, Evidence, and Resolution Verification
This final step surfaces a prioritized, human-readable list of root causes, each backed by direct links to supporting telemetry, and then confirms automatically that the incident is resolved once a fix is applied. An evidence-backed AIOps RCA report must contain the root cause stated in one line, direct links to the logs, metrics, and events supporting the conclusion, the blast radius of affected services and users, a recommended fix, and the measured time to RCA.
Inputs are the ranked causal hypotheses from step five and the change-detection annotations from step six. Outputs are a structured report delivered in Slack or a dynamically generated dashboard, plus a continuous incident resolution verification loop that polls observability data until the anomaly clears. The engineering consideration: verification cannot be manual. Struct’s Incident Tracker runs an approximately one-minute automated verification loop against live Datadog, Sentry, and cloud log data to confirm an incident is actually resolved, not merely acknowledged.
The outcomes of this full pipeline are concrete. Arcana reduced average developer time per investigation from 30 minutes to 2 minutes and reclaimed 56 hours of developer time per month after integrating Struct with Sentry, GitHub, GCP Cloud Logging, and Slack, running 2,100+ investigations with an 85–90%+ helpful rate. Across Struct’s customer base, Struct cuts triage time by 80%, turning a 45-minute manual investigation into a 5-minute review. See how Struct shortens your next incident review
Frequently Asked Questions
What minimum tooling maturity does a team need before AIOps root cause analysis is useful?
A team needs three things in place: a structured alerting trigger such as a Slack channel, PagerDuty, or Sentry, at least one observability source with queryable logs or metrics such as Datadog, AWS CloudWatch, or GCP Logs, and a code repository connected via GitHub. Without basic logging and trace IDs, no AIOps system can correlate signals across services. The ideal starting point is a team already using Sentry for exceptions, a cloud log provider for infrastructure logs, and Slack for alert routing. Struct’s helpful investigation rate is measured against teams at this baseline. Teams with richer telemetry, including distributed traces, structured JSON logs, and consistent service tagging, see higher accuracy from the first investigation.
Can Struct operate if our logs cannot leave our VPC?
Struct currently requires access to your logs and observability context via its standard integrations with AWS, GCP, Datadog, and similar platforms. If your organization mandates full on-premises deployment with zero data egress, Struct is not the right fit at this time. Enterprise plan customers can discuss sidecar or on-prem support options directly. For the vast majority of Series A–C SaaS companies, Struct’s SOC 2 Type II and HIPAA compliance posture, documented at trust.struct.ai, satisfies security review without requiring on-premises deployment. Logs are accessed and processed ephemerally, and they are not stored by Struct.
How long does it take to get the first automated investigation running?
Setup takes under 10 minutes. You authenticate your alert source such as Slack or PagerDuty, connect your code repository such as GitHub, and link your observability context such as Datadog, Sentry, or cloud logs. Once connected, auto-investigations activate immediately. The first investigation fires the next time an alert triggers in your configured channel. There is no indexing period, no professional services engagement, and no schema migration required. Struct includes white-glove onboarding and a 30-day risk-free pilot on all plans.
What happens if our telemetry is sparse or our logging is inconsistent?
Struct’s output quality is bounded by the telemetry it can access. If your system lacks trace IDs, structured log fields, or consistent service naming, the correlation and causal-inference layers have less signal to work with, and investigation accuracy drops. Struct still surfaces what it can, including alert context, recent deploys from GitHub, and any available exception data from Sentry, but it cannot infer system state from code analysis alone. The practical recommendation is to ensure at least one log source per service tier is queryable before enabling auto-investigations. Teams with sparse telemetry often use Struct’s investigation outputs to identify which logging gaps cause the most blind spots, then close those gaps iteratively.
Is Struct safe to put junior engineers on call with?
Struct is safe for junior engineers on call, and enabling them to take shifts confidently is one of Struct’s primary design goals. By the time an engineer opens their laptop after an alert fires, Struct has already correlated logs, mapped the blast radius, identified the probable root cause, and surfaced suggested fixes in a dynamically generated dashboard. Junior engineers review a structured starting point rather than staring at raw log streams across five tools. Teams can also encode their internal runbooks directly into Struct so the AI follows the same investigation steps a senior engineer would. Struct’s incident resolution verification loop then confirms whether the fix actually resolved the anomaly against live observability data, removing the guesswork from the “is it fixed?” question that often trips up less experienced responders.
Next Steps for Automating Your RCA Pipeline
This article covers the mechanics of the seven-step AIOps root cause analysis pipeline. For a deeper look at how automated investigation fits into a broader reliability workflow, see the Automated Root Cause Analysis hub and the Incident Tracking and Resolution Verification hub, which covers the closed-loop verification model in detail.
Manual 3 AM triage across Datadog, Sentry, GitHub, and CloudWatch costs engineering teams 30–45 minutes per incident and compounds into tens of hours lost every month. Arcana reclaims 56 engineer-hours per month by running the seven-step pipeline automatically on every alert. Struct connects to your existing stack with the same sub-10-minute setup described earlier, delivers ranked root causes with evidence in under 5 minutes, and performs incident resolution verification so your engineers review conclusions instead of hunting for them.
Book a Struct demo and see your next incident investigated for you
