Frequently Asked Questions

AI observability is the ability to see inside a model's decision process — capturing every model call, tool use, retrieved context, and reasoning step in a full trace. AI monitoring is the broader practice of tracking how a system performs over time using defined signals: accuracy trends, drift rates, cost patterns, error rates. Observability is what gives you the granular view of a single session or decision. Monitoring is what gives you the trend view across thousands of sessions. Both are necessary. Observability without monitoring means you can debug individual failures but can't see the patterns that precede them. Monitoring without observability means you know something changed but can't trace it back to a specific decision.

A real-time AI audit trail needs to capture more than access logs. It needs session-level traces that show the full chain of an agent's actions — what it read, what tools it called, what decisions it made, and what it produced — all with timestamps and identity context. For coding agents and other autonomous systems, that means capturing every MCP tool call, every data source accessed, every model call made in the session, and every output generated. A log entry that says "user accessed Jira" is insufficient for compliance purposes. Regulators and auditors need to be able to reconstruct what an agent did in a given session, why it did it, and whether that chain of actions was within policy boundaries. Systems that capture only final outputs are not audit-ready for agentic AI workflows.

Data drift is when the real-world inputs going into an AI model change significantly from the data the model was trained on. Models are built on a snapshot of data at a point in time. The world moves on, user behavior shifts, upstream data sources update — and the model is now operating on inputs it was never optimized for. The result is accuracy degradation that builds slowly and silently, often invisible until a significant failure surfaces. A churn prediction model trained before a major market shift, a recommendation engine whose product catalog changed, a support chatbot whose knowledge base is six months out of date — all of these are data drift problems. Monitoring for drift at the feature level and the embedding level gives teams the ability to catch this degradation early and retrain proactively.

AI monitoring covers twelve distinct signal categories, and no platform covers all of them equally. The core groups are performance signals (accuracy, latency, error rates), data drift signals (whether inputs are changing from what the model was trained on), output quality signals (hallucination rates, toxicity, relevance), cost and resource signals (token usage, API spend), user behavior signals, and system health signals. The right signals depend on your primary risk. If your biggest concern is output accuracy, prioritize performance and output quality monitoring. If you're running autonomous agents with live system access, cost signals and user behavior signals become critical. If you're in a regulated industry, audit trail and pipeline signals matter most. Buying a monitoring platform before identifying which signal gaps you actually have is how teams end up with expensive dashboards nobody acts on.

An MCP (Model Context Protocol) server is a layer that gives AI agents access to external data sources and tools — Jira, GitHub, Confluence, internal databases, APIs. When an agent connects through an MCP server, it can read, query, and act on the data those systems contain. The security risk is significant: MCP-connected agents can accumulate data access permissions that exceed those of any human developer on the team, and most organizations have no centralized audit trail for what those agents actually touched. A single misconfigured MCP server pointed at sensitive engineering data creates a privileged system account with zero governance controls on top of it. Organizations deploying MCP-connected coding agents need centralized MCP governance, session-level audit logging, and scoped permission enforcement before agents reach production systems.

Coding agent sprawl is what happens when an engineering organization adopts multiple AI coding tools — Cursor, Claude Code, Codex, and others — without a centralized layer to govern them. Each tool carries its own identity, its own data access permissions, and its own cost footprint, with no coordination between any of them. A developer connecting a coding agent to internal Jira tickets, GitHub repositories, and Confluence docs through an MCP server can accidentally create an agent with more data access than any human developer on the team — and no audit trail to show what it read or acted on. The risks are real: runaway inference costs, ungoverned data access, compliance exposure, and zero visibility for security or finance teams. Organizations need centralized identity, cost controls, and audit logging that cover every coding tool simultaneously, not tool-by-tool.

AI monitoring is the continuous observation of how an AI system behaves in production — tracking output quality, model drift, cost, latency, error rates, and user behavior patterns over time. AI governance defines who is accountable for the system, what policies apply to it, and how decisions about it get made. Monitoring generates the signals; governance uses those signals to make decisions and enforce accountability. A well-governed AI system has monitoring in place so that governance teams can see when something drifts, degrades, or violates policy. Without monitoring, governance operates on assumptions about what the system is doing rather than evidence.

AI governance defines the rules, accountability structures, and policies that determine how AI systems should operate. AI security enforces those rules at the moment of execution — blocking harmful actions, filtering dangerous inputs, and preventing data from leaving systems it shouldn't leave. Governance is the policy layer; security is the enforcement layer. A governance framework that says "agents cannot access customer PII without authorization" is useless without a security control that actually stops the agent from accessing that data in the first place. Most organizations need both, and they need them connected. Governance without security is a wish list. Security without governance is enforcement without a clear standard to enforce against.

Shadow AI refers to AI tools and models that employees use without IT or security approval. A team member signs up for an AI writing tool, pastes internal strategy documents into it, and sends the content to a third-party model's training pipeline — all without anyone in security knowing it happened. Shadow AI is widespread because the tools are easy to access and genuinely useful. Detection requires active scanning of browser traffic, endpoint activity, and cloud tenant connections for unauthorized LLM usage, plus a live registry of every sanctioned AI tool in the environment. Organizations that can't answer the question "what AI tools are running in our environment right now" have a shadow AI problem by definition.

Prompt injection is an attack where malicious instructions are hidden inside a user input or connected data source, tricking an AI model into overriding its original instructions. A support ticket with an embedded command, a document with encoded payloads, a database field with rogue instructions — all of them can redirect a model's behavior without triggering any conventional security alert. The risk is serious because AI models are trained to be helpful, which means they follow instructions. A model that reads a poisoned input and complies is doing exactly what it was designed to do. Organizations need input validation and prompt filtering controls sitting directly in the request path — before the model reads anything — to defend against this class of attack.