Cloud & Platform · 2024

OpsPilot

One incident view that joins AWS telemetry, Kubernetes events and Terraform state.

My role: Systems architect & developer
When: 2024
Status: Active project
Stack: AWS SDK, Terraform, Kubernetes

OpsPilot cloud operations and telemetry assistant

Where it started

During cloud incidents, on-call engineers lose minutes cross-referencing CloudWatch alarms, Kubernetes pod events and Terraform state across a dozen browser tabs.

What I owned

Systems architect & developer

How I built it

Normalised AWS SNS, PagerDuty and Prometheus alerts into one IncidentContext schema behind a FastAPI gateway.
Queried AWS and Kubernetes APIs in parallel with asyncio, and matched alert timestamps against recent Terraform applies to surface drift.
Kept automation safe by gating runbook suggestions behind operator confirmation and scoping IAM to read-only.

[Alert Trigger: CloudWatch / Prometheus Alertmanager]
                       │
                       ▼
            [OpsPilot Ingestion Gateway (FastAPI)]
                       │
         ┌─────────────┼─────────────┐
         ▼             ▼             ▼
   [AWS Telemetry] [K8s Client] [Terraform State Engine]
   (CloudWatch/    (Pod Events, (State drift, lock status,
    EC2 status)     CrashLogs)   resource dependency tree)
                       │
                       ▼
          [Context Aggregator & Diagnostician]
                       │
                       ▼
       [Structured Incident Briefing & Remediation Runbook]

System architecture

Unified ingestion: webhook alerts from AWS SNS, PagerDuty and Prometheus normalised into one IncidentContext schema.
Terraform drift correlator: alert timestamps matched against the latest Terraform apply to detect infrastructure regressions.
Asynchronous querying: asyncio with connection pooling queries AWS and Kubernetes in parallel (<400ms aggregation).

Deterministic rule gating before LLM enrichment

Automated runbook suggestions never execute destructive cloud commands without explicit operator confirmation.

Least-privilege IAM

Credentials scoped to read-only CloudWatch, CloudTrail and EC2 Describe APIs, preventing privilege escalation.

Hardening: API credentials and kubeconfig contexts stored in isolated encrypted secret vaults with short-lived session tokens.

What changed

Estimated 60% faster initial triage by collating logs, events and recent infrastructure changes into a single briefing.

✓Estimated 60% reduction in initial triage time by collating logs, events and recent infrastructure changes into one view.
✓Verified detection of infrastructure regressions tied directly to recent Terraform applies.

AWS SDKTerraformKubernetesFastAPIPrometheus