Forgebench was at the INC42 CTO Summit to listen to how technology leaders are thinking about AI as it moves from experimentation into production. What stood out was not simply the pace of adoption, but the operational questions emerging around it.
There was no shortage of discussion about artificial intelligence at the INC42 CTO Summit. That, by itself, would hardly have been news. AI has become so thoroughly embedded in the vocabulary of technology leadership that a gathering of CTOs without a conversation about models, agents and automation would have been the unusual event.
What was more revealing was the direction those conversations tended to take.
The question was less often whether an organization could make use of AI than what happened after it did. An agent may be able to perform a task, but in a production environment it must also contend with permissions, data, tools, existing workflows, and the systems the business already depends on. A model may be remarkably capable today, but there is little reason to assume that it will remain the preferred model tomorrow.
AI usage may begin with a handful of developers experimenting with new capabilities, but the economics look rather different when those capabilities spread across an organization.
The difficult questions are beginning to appear downstream of the technology itself.
The enterprise AI problem is beginning to look less like a problem of building AI and more like a problem of operating it.
The Difficult Part Begins After the Demo
An agent rarely operates in a greenfield environment.
It enters an organization that already has applications, APIs, data, identity systems, business workflows, and security boundaries. It may need to retrieve information from one system, invoke a tool in another, and use a model to determine what to do next. The complexity is not necessarily in any individual component. It lies in the relationships between them.
This was evident in discussions about building AI into systems that already operate at scale. An agent has to understand the systems, workflows, and constraints around it; it cannot simply be treated as an LLM-powered feature sitting alongside the existing stack.
That changes the architectural question.
With a conventional AI application, much of the attention goes toward the quality of the model's output. With an agent, the organization also has to understand the conditions under which that output was produced and the actions that followed it.
What context was available to the agent? Which tools could it invoke? What systems could those tools reach? What was it authorized to do? If it acted on behalf of a user or another system, under whose identity did it act? And once the action was taken, could someone reconstruct what happened?
The distinction becomes particularly important when AI moves from generating an answer to taking an action. The agent now has an operational identity, access to tools, and potentially consequential permissions. The summit discussions around agent identity, security, isolation, guardrails, and human intervention reflected this growing concern.
The agent is no longer merely part of the interface.
It has become another participant in the production architecture.
The Model Will Change. The System Has to Survive It.
There is another architectural characteristic of AI that is easy to underestimate: the model itself is becoming a moving part.
An application can remain useful while the model underneath it changes several times. Capability, latency, availability, cost, and workload requirements can all shift independently of the application. The model selected for a particular workload today may not be the model that makes sense tomorrow.
That makes model choice less of a permanent architectural decision and more of an operational one.
The question becomes: how do you make the model replaceable without making the application replaceable?
One answer is to treat the model as one capability available to the agent rather than as the fixed endpoint of the application's architecture. The agent may draw on one model or another depending on the workload, while context, tools, policies, and enterprise systems remain part of the surrounding environment.
That separation also makes observability and evaluation more important. A new model may be cheaper or more capable in a benchmark and still behave differently on the workload that matters to an organization. If the model changes, technology teams need to understand the effect on behavior, latency, quality, and cost.
One of the stronger ideas emerging from the summit was therefore to build for adaptability and allow models to change. Model switching, cost optimization, observability, separating context from the model, and verification all become connected parts of that architectural problem.
There is a broader implication here.
Traditional applications can often be designed around relatively stable underlying components. AI applications are being built in an environment where the underlying technology is changing much faster.
The application may have a lifecycle measured in years. The model beneath it may have a lifecycle measured in months.
Architecture has to account for that asymmetry.
The AI Bill Is Becoming an Operating Question
The economics of AI surfaced another question that goes beyond the invoice.
Measuring consumption is relatively straightforward: tokens, model calls, and infrastructure costs provide an obvious starting point. But consumption is not the same as value.
An organization can know precisely how many tokens it used and still have little idea what that consumption produced.
Conversations around AI economics raised questions about token budgets, usage measurement, the relationship between token output and feature output, where usage occurs, where value is delivered, and whether AI usage should be managed through individual or universal caps, role-based access, or tiered systems.
These questions point toward a different relationship:
Usage → Cost → Output → Value
The significance of that relationship becomes clearer as AI moves beyond a small group of technical users. Organizations want to democratise access to AI, but wider adoption creates more workloads, more models, more expenditure and more variation in how the technology is used.
The challenge is therefore not simply to establish a budget. It is to establish enough attribution to understand where that budget is going and enough context to determine whether the resulting output justifies it.
The token bill tells you what was consumed.
It does not, by itself, tell you what was accomplished.
That distinction will become increasingly important as organizations begin measuring AI not merely as infrastructure consumption, but against engineering delivery, product output, and business value.
Autonomy Is Not a Switch
The conversations around agents also suggested a more measured way of thinking about autonomy.
An agent that recommends an action operates under very different conditions than one that executes an approved action. An agent confined to a single workflow presents a different operational problem from one that can invoke tools across several enterprise systems.
Autonomy is therefore better understood as a progression: from human-controlled systems to assisted workflows, partially autonomous systems, and eventually highly autonomous systems. The controls have to evolve with that progression.
But those controls do not map one-to-one to the stages of autonomy.
Identity remains important at every stage. Permissions remain important at every stage. Observability does not suddenly appear when an agent becomes highly autonomous. What changes is the degree and sophistication of control required.
A system that recommends an action may need visibility and approval. A system that executes an action may additionally require tighter permissions and intervention mechanisms. A system that can coordinate activity across several systems may require stronger isolation, verification, and rollback.
This is why autonomy should be treated as a progression in which the control mechanisms accumulate and become more consequential, rather than as a sequence of discrete controls attached to discrete stages.
The principle is straightforward: as an agent is given more authority to act, the organization needs greater confidence in its identity, permissions, behavior, and ability to intervene.
When Software Starts Acting, Control Becomes Architecture
This may be the most consequential shift.
A conventional application can be represented, in broad terms, as:
Application → APIs → Infrastructure
AI introduces models into that environment. Agentic systems make the relationship more complex:
User / Intent → Agent
The agent then works with several surrounding capabilities:
Models · Context · Tools · Enterprise Systems
That distinction matters. Models are not necessarily the agent's final destination. They are one capability the agent uses, alongside context and tools, while enterprise systems provide the environment where those actions have consequences.
Around the entire system sits another layer:
Identity · Permissions · Observability · Cost · Verification · Intervention
These are not sequential steps. They are concurrent operating concerns.
Identity determines who or what is acting. Permissions determine what it can reach. Observability makes its behavior visible. Cost establishes what its activity consumes. Verification provides evidence that behavior and outcomes meet the required conditions. Intervention provides a mechanism for human control when the system reaches the limits of its authority.
Together, they form the operational basis for control.
That layer is easy to overlook because it does not produce the visible novelty of AI. It does something more prosaic: it makes increasingly capable systems understandable and governable.
And production AI needs that discipline because it does not behave quite like conventional software. Models can change. Outputs can vary. Context can change. Agents can behave differently depending on the situation. Underlying capabilities can evolve. The production environment therefore needs mechanisms to observe, verify, understand, and control behavior over time.
This is the problem space that feels increasingly important to us at Forgebench.
The industry has spent considerable energy making agents easier to build. The next problem is making them operable at scale without losing sight of who is acting, what that actor is permitted to do, which tools it can reach, what its activity costs, and what happened along the way.
That is the rationale for an AI control plane.
Forgebench is designed as the layer between the agents an organization builds and the providers and tools those agents call. It gives each agent an identity and credential, binds the tools and MCP servers it can reach, establishes policy and guardrails, provides observability and metering, and maintains the record of what happened.
The control plane does not replace the agent or the framework it was built in. The agents remain yours. What changes is the environment in which they operate: one in which identity, authorization, spend, observability, and the record are part of the system rather than scattered across it.
The conversations at the INC42 CTO Summit made the direction of travel clearer.
Enterprise AI has moved beyond the question of whether models are capable enough. It is now confronting the less glamorous, more consequential questions that appear when those models and agents become part of everyday production systems.
- Who is acting?
- What can they do?
- What did they do?
- What did it cost?
- What changed?
- And can the organization prove what happened?
Those are operational questions.
Increasingly, they are architectural ones too.
And that may be the next chapter of enterprise AI: not simply building systems that can act, but building the environment in which those systems can be trusted to operate.