IT Asset Discovery for Microservices in Bay Area Tech

At 5:48 UTC on March 1, 2026, engineers at Firetiger watched their own ingest service go down. Firetiger is a San Francisco startup that builds AI agents to monitor production software for other companies — running the same containerized, microservices-based infrastructure most Bay Area engineering teams now run on Kubernetes and Amazon ECS.

The failure had been building for 25 hours. A race condition during a pull request merge on February 27 canceled a build job partway through. A separate bug in the deploy pipeline’s artifact tracking read that canceled build as complete anyway. It pushed a rollout that updated the company’s ECS service definitions to point at the image from that same interrupted build. Existing containers kept serving traffic on the previous image, so the system ran normally for most of a day — until a container restarted, asked for the image its service definition named, and got a build that had stopped partway through compiling three days earlier.

Firetiger’s own monitoring caught the anomaly around 09:45 UTC. An alert-routing misconfiguration kept the page from reaching an on-call engineer for more than eight hours. Root cause landed at 18:00: the service definition, the record of what container image was supposed to be running, pointed at a partial build. A rebuild and redeploy at 18:15 closed the gap, and the ingest service recovered five minutes later.

The pattern outlasts one company’s build pipeline. A configuration record naming a container or a service is only as current as the moment someone wrote it, and in a containerized microservices environment that moment can be seconds old and already wrong. Bay Area engineering teams built most of the orchestration tooling the rest of the industry now runs on, and they carry the identical exposure everyone else running it does: infrastructure that changes faster than any inventory built to describe it.

The Bay Area engineering org this is actually about

Bay Area technology companies created most of the container orchestration tooling the industry runs. According to CNCF survey data compiled by Tigera, 91 percent of organizations using Kubernetes have more than 1,000 employees. Mid-market and growth-stage companies in the Bay Area run the same Kubernetes clusters, ECS fleets, and managed serverless functions as those large enterprises — typically with platform engineering teams of two to six people.

The infrastructure complexity is enterprise-grade. The operational headcount is not. Each service definition in that environment is a configuration item that can drift from what is actually running in the cluster, exactly the setup Firetiger’s own incident played out inside.

What IT asset discovery means for ephemeral microservices infrastructure

A configuration item in a traditional infrastructure environment lives for months or years. A physical server gets racked, added to inventory, and stays. Discovery automation scanning weekly maintains a reasonably accurate record.

Containers, pods, and serverless functions — the building blocks of a microservices architecture — operate on a different timescale. A Kubernetes pod lives minutes to hours. An ECS task restarts on a schedule. A Lambda invocation exists for milliseconds. Periodic scanning across a network range produces a record of what was running at the moment of the scan, not what the orchestrator is currently instructed to run. This is the same short-lived pattern that makes ephemeral container CMDB tracking a distinct discovery problem from a physical server refresh cycle.

The configuration item for a containerized service is not the running instance. It is the service definition: which container image, which registry, which tag. In Firetiger’s case, the ECS task definition referenced a container image tag that did not exist in the registry. The definition was the wrong record. Running containers continued serving the previous image correctly, masking the drift entirely. No network scan would have detected it because the problem lived in the definition layer, not in the instances already running.

This failure mode is the inverse of the challenge in fragile OT environments, where undiscovered assets are assets that were never documented at all. In containerized environments, the record existed. The record was wrong.

The Kubernetes CMDB discovery challenge is structural: the unit of configuration is the definition or manifest, not the instance it spawns.

Where it breaks, with evidence

Komodor’s 2025 Enterprise Kubernetes Report found that nearly 80 percent of production outages trace to system configuration changes. Enterprises running Kubernetes environments lose an average of 34 workdays per year to Kubernetes-related incidents and their follow-on recovery work. A separate finding from the same research indicates that 82 percent of container users run workloads with resource misalignment between what the service definition specifies and what the environment holds (vendor-sourced, cited for scale).

All three numbers share a root. The configuration record did not reflect what was actually deployed. The service definition said one thing. The cluster held another.

Firetiger’s incident makes the pattern specific. The ECS task definitions referenced an image tag with no corresponding artifact in the registry. Running tasks served the previous image correctly, masking the drift. The record was wrong for more than 25 hours before a container restart surfaced it. The alert-routing misconfiguration, introduced while testing a new notification feature the previous Friday, then added 8 more hours before an engineer began investigating. Total gap: 33 hours between the configuration record becoming inaccurate and the team knowing about it.

Why the Bay Area’s operating rhythm makes this worse

The Bay Area concentrates platform engineering talent into a narrow pool, and a handful of large technology companies absorb most of it. Mid-market and growth-stage companies build the same containerized infrastructure with one to three engineers often responsible for service definitions across multiple clusters and environments. Agentic coding tools have raised deploy velocity further: each additional change is another chance for a definition to reference a version that was never built or was deployed to the wrong environment. Firetiger’s race condition happened at ordinary startup deploy velocity — the same failure mode occurs more often as that velocity increases, and it stays invisible until an outage forces the question.

What breaks first when the record is wrong

The Firetiger post-mortem illustrates the incident response failure mode directly. Firetiger’s AI Issue Analysis agent detected and flagged the problem. The alert-routing misconfiguration, introduced while testing a new notification attribution feature the previous Friday, then hid that detection from the incident management channel. The team’s own tooling held the answer. The wrong record in the notification routing system prevented anyone from being told.

When engineers began investigating at 17:45, root cause took 15 minutes to confirm: the ECS task definition referenced an image tag that had never been built into a container image. The service definition was the record. The record was wrong. Identifying it required looking at what the definition actually contained.

A configuration record kept current through API-driven discovery of the ECS service definitions would have surfaced the mismatch at the moment the partial deploy occurred. The 25-hour silent window closes when discovery runs against the orchestration plane on a cadence tight enough to catch definition changes before a container restart does.

Change management produces the second failure mode. In the Firetiger incident, Terraform rejected the Lambda update as invalid while accepting the ECS mutation. The result was mixed state: one service definition updated, one not, with no record of which definitions had changed and which had been rolled back. Without a pre-change and post-change snapshot of the service definition layer, the team reconstructs what changed from the environment itself, not from a record.

Audit and compliance produce the third. PCI DSS, SOC 2, ISO 27001, and HIPAA each require a current, accurate inventory of in-scope system components and configurations. A Bay Area company carrying any of these obligations from containerized infrastructure cannot produce an accurate point-in-time snapshot from a CMDB last reconciled before the most recent deploy cycle.

What accurate discovery actually looks like here

Container environments require API-driven discovery because the configuration items that matter live in the orchestration plane, not on a network segment. An ECS cluster exposes its task definitions through the AWS API. A Kubernetes cluster exposes its Deployments, StatefulSets, ConfigMaps, and service accounts through the Kubernetes API. A managed cluster on EKS or AKS exposes the same resource types through the cloud provider’s control plane API. Network-range scanning does not reach these resources.

Treating the Deployment or Auto Scaling Group as the stable configuration item, rather than each ephemeral instance it spawns, resolves the lifecycle problem. The Deployment is the CI. Its replicas are running instances of that CI. When the Deployment’s container image reference changes, discovery captures that change at the definition level, not at the instance level where it would arrive too late.

High-frequency scheduled discovery is the practical operating cadence for this class of environment. A team shipping multiple deploys per day needs a scan cadence that closes the window between when a definition changes and when the CMDB reflects it. Event-driven discovery can supplement scheduled runs for changes that occur between scan windows, such as a Terraform apply that updates an ECS service definition mid-cycle.

The cloud discovery coverage challenge compounds across providers. Organizations running EKS alongside Azure AKS carry two separate control plane APIs. Discovery that reaches both and reconciles findings into one CI record per service definition produces a current, single-source inventory across a multi-cloud container environment.

Where a platform fits

Virima applies the API-driven discovery model above to microservices and container environments on AWS and Azure. Native AWS discovery covers ECS task definitions, ECS clusters, and EKS workloads. Native Azure discovery covers AKS clusters, node pools, and workload configurations. ViVID™ service maps build dependency context from those CI records and update as service definitions change. For the mapping side of this problem specifically, see how service mapping for microservices and cloud dependencies extends that dependency context across ephemeral compute. Multi-source reconciliation governs attribute authority when cloud provider APIs and other discovery methods observe the same asset. For Bay Area engineering teams carrying PCI DSS, SOC 2, ISO 27001, or HIPAA obligations from containerized infrastructure, that combination produces a configuration record that reflects what the orchestration plane actually holds.

The configuration record that was not there

Firetiger’s team found the root cause in 15 minutes once they started looking. The ECS task definition said to run an image that did not exist in the registry. The record identified the problem.

The 25 hours before that moment, and the 8 hours of misrouted alerts, were the same underlying problem appearing twice. Nobody had a reliable, current answer to what the service definition actually contained — the gap that discovery against the orchestration plane, not the network, is built to close.

Bay Area engineering teams running containerized microservices infrastructure at startup or mid-market scale carry this gap at higher deploy velocity than Firetiger’s build pipeline produced on March 1. The configuration record that exists before an incident is the only one that helps during it.

Frequently Asked Questions