5 Layers of an AI Agent: What Every Manager Needs to Know

“Most people still think an AI agent is just ChatGPT with a good prompt.” That is the opening line of Sunil Ramlochan’s article “The AI Agent Stack Is Not a Prompt. It’s a Production System”, and the author calls this belief “a comforting myth.” The useful truth, he argues, is different: a real agent is closer to a small operating system for getting work done. It has a brain, hands, memory, rules, logs, recovery plans, and someone accountable when the agent does the wrong thing.

The article’s thesis fits in a single line: an agent is an entire stack. Reliability comes from the architecture around it, while the model itself – or a clever prompt – is just one ingredient. The picture is an engineering one, so let us approach it from the other side: what in this stack actually concerns the manager who does not write code but decides whether to put an agent to work.

Let us clear up the main confusion first – the very one the author argues against. An “agent” works in a fundamentally different way from a chat interface: a prompt gives you one question and one answer. An agent receives a goal and works in a loop: it understands the task, decides what to do next, picks a tool, uses it, looks at the result, makes sense of what happened, and then continues, stops, or calls a human. The article openly references the OpenAI guide to building agents, which defines an agent as a system of models, tools, instructions, and constraints. This loop is what sets an agent apart from conventional automation: an agent does not reply with a single line – it sees the job through.

The author breaks this “small computer for work” into five layers. Let us walk through them with a manager’s eye.

Layer 1: The model – the brain, but not the whole body

The model is the part that reasons, plans, and writes. It gets all the attention because it looks like magic: stronger model – better reasoning, fewer obvious mistakes. But the author puts it bluntly: in production, the model is often not the hardest part.

A strong model will still fail if it has access to the wrong tools, cannot retrieve the right context, remembers outdated information, has no permission boundaries, its decisions cannot be audited through logs, or it was assigned a task that should have been handled by a plain algorithm. The comparison from the article is apt: the model is the head chef in a restaurant. Even an excellent chef needs clean ingredients, a working kitchen, orders, food-safety controls, and a manager who will shut things down when something goes wrong.

For a manager, two takeaways follow. First, the money side: behind the word “agent” is a stream of model calls, each of which costs money – nothing like a single fixed subscription. Second, expectations: do not assume that a more expensive model will compensate for weak architecture. “Pick the best model you can justify,” writes the author, but do not expect it to fill the holes in your process.

Layer 2: Tools – the agent’s hands and its biggest risk

Tools are what the agent uses to act on the outside world: search, APIs, databases, email, calendars, code execution, files, payment systems, internal business applications. This, writes the author, is where the agent stops being a text generator and becomes an executor. And this is where the “blast radius” grows sharply. It is precisely this transition that led, three years ago, to the BYOA – Bring Your Own Agent trend: the moment a tool gets access to real systems, organizational questions start outpacing technical ones.

A bad answer is one type of error. Writing junk into a database, sending an accidental email, processing a refund twice, leaking a file – that is an entirely different category. This is why tool boundaries matter so much. The author offers a list of questions worth writing down for anyone about to give an agent access to real systems: which tools does the agent have access to, which actions are read-only, which require approval, can the agent write to production systems, are calls logged, can an error be rolled back, and can malicious input trick the agent into misusing a tool.

The key insight of this layer: many agent failures are caused by brittle tool interactions – the model’s reasoning is barely involved. Once an agent has permission to act, it needs engineering discipline, and a carefully worded prompt alone will not cut it. The author’s practical principle: treat tools like production APIs – minimum privileges, strict schemas, confirmation steps, caution with write operations.

Interestingly, all of this – tools, permissions, confirmation workflows – sounds like an architecture conversation. But beneath the architecture lies something simpler: the ability to clearly articulate what the agent should do, in what context, and under what constraints. Without that articulation, even a well-built agent starts “filling in the blanks” – and that is precisely when tools become dangerous.

Layer 3: Memory and its lifecycle

“Memory sounds simple. Let the agent remember what is useful. Easy, right? Not quite,” writes the author. Memory comes in four varieties: short-term (what is happening in the current task), session (what happened in the current conversation), long-term semantic (knowledge reused over time), and transactional (state that must be accurate: orders, statuses, permissions, approvals). Short-term memory is essentially the model’s context window, and it has well-known limits: long prompts lose information in the middle, and the agent inherits this limitation.

This is one of the hardest layers because memory here operates as a constant judgment call: what to remember, what to forget, who can see it, how to prevent one user’s context from leaking into another’s session, how to stop stale data from poisoning future decisions. The author illustrates the risk vividly: a sales agent remembering a discount policy from last quarter; a medical agent mixing up two patients’ notes; a financial agent remembering an approval but not the condition it was tied to. The takeaway fits in one sentence: memory in an agent works like a database with consequences – treating it as a notepad is dangerous.

For a manager, this translates into an uncomfortable question for any vendor or contractor: what exactly does your agent remember between interactions, and who guarantees it will not mix up my clients? The author’s advice: give memory a lifecycle – data provenance, expiration dates, access rights, deletion rules, and separation of user, session, business, and system memory.

Layer 4: The runtime – where “thinking” becomes a loop

At the center of the stack sits the agent’s runtime – that very loop: understand the goal, decide what to do next, pick a tool, use it, evaluate the result, make sense of it, and continue, stop, or ask for help. The author links this pattern to the ReAct approach (reasoning and acting) and underscores a simple point: the agent sees the task through to completion – it does not end with a single response.

And this is exactly where things break. The loop can get stuck. A tool can return bad data. The model can misinterpret the result. A retry can accidentally execute the same action twice. That is why the runtime is about execution management – and the formula “the LLM just thinks” does not describe even half the work. The most important piece of advice for managers in the entire article is buried right here: do not design an agent as one long prompt. Design it as a workflow with state, checkpoints, stop conditions, and human review where needed.

This flips the frame. “Describe a process with state and checkpoints” is exactly the work a manager does when building a standard operating procedure or assigning a task to a team. The agent’s loop looks suspiciously like how a real employee works: receives a task, plans the steps, does something, looks at the result, adjusts. The difference is that an employee has common sense and a fear of making mistakes, while an agent has only the constraints you put in place beforehand.

Layer 5: Observability, safety, and governance

The final layer wraps the entire stack, and the author calls it the boundary between a demo and a system you can trust. This is the layer that lets a team answer: what the agent did, why it did it that way, which tool it called, what the tool returned, how much the run cost, whether it followed the rules, whether it made things up, whether it exposed private data, whether it got stuck in a loop, and whether a human approved the risky step. Without this layer, writes the author, debugging agents turns into folklore – the team argues about what happened instead of looking it up.

But then comes the most important distinction, and it is squarely a management one. “Observability tells you what happened. Governance decides what is allowed to happen.” A dashboard will show that the agent is about to do something risky. Governance determines who can stop it, under what conditions, and who owns that decision. The author directly references the risk classification from the OWASP Top 10 for LLMs (where prompt injection is named a major threat) and the NIST AI Risk Management Framework as guideposts.

Here is the point for you. When vendors paint a future where “agents manage complex processes,” the question of who is responsible when the agent makes a mistake quietly disappears. This layer answers honestly – at critical junctures, a human is responsible. The author’s advice: do not stop at logs. Define escalation rules, approval workflows, audit trails, policy checks, budget limits, and emergency shutdown conditions.

The weakest link: the seams between layers

Here the article takes a turn that is easy to miss – and should not be. The author writes that the hardest thing about the agent stack may not be any individual layer at all – but the connective tissue between them. The runtime picks a tool, the tool changes business state, the memory layer stores the result, the model interprets it, the observability layer logs the decision, the safety layer decides whether to permit the next step. If even one of these handoffs is poorly defined, the entire system becomes brittle.

And here the author names an unromantic truth: reliability comes from “boring” engineering things – state machines, idempotency keys, retries with safeguards, rate limits, permission scopes, tests, evaluation sets, audit logs, rollback paths, and human-in-the-loop checkpoints. An agent may feel intelligent, but in production, reliability comes from disciplined design – the model’s intelligence alone does not guarantee it.

For a manager, this picture of handoffs between layers is suspiciously familiar. Blurred responsibilities, context lost in handoffs, rising coordination overhead – these are the problems of any real team where a task passes through multiple hands. That is why both the author and the OpenAI guide recommend making the “one agent or several” decision the same way you decide whether to hire more people: start with one, squeeze everything you can out of it, and add complexity only when one stops coping. The same logic underpins Google Cloud’s report on the emerging role of the manager as a supervisor of an agent team – except the vendor draws five autonomous agents running today, while engineers honestly advise starting with one and well-defined seams between them.

What a manager should actually do with all this

The author closes the article with the slogan “Agents are not magic. They are managed autonomy” and offers a checklist: before deploying an agent, ask a question deeper than “does it work?” The right question is: “Can I trust this agent when it has real users, real data, real tools, real costs, and real consequences?” A demo survives on a lucky prompt. Production demands architecture, tests, permissions, memory discipline, observability, safeguards, and a clear owner.

Let us translate that into steps for a manager who does not write code but wants to understand whether they have a task suited for an agent – and whether they are ready for it.

Start by finding a candidate. Review your week and list processes that repeat, involve multiple steps, and contain forks of the “it depends” variety. Not one-off requests like drafting an email or summarizing a meeting – a prompt is enough for those, no agent needed. Processes specifically: handling incoming support requests, a recurring report pulling from multiple sources, initial resume screening. This is precisely Benedict Evans’ criterion: AI automates tasks well and judgments poorly, so an agent is justified where the process has room for judgment. A concrete example of this transition is agentic data analysis, where the difference between a chat and an agent becomes obvious on three real manager files.

Next, describe the process as a multi-step workflow – exactly what the article calls for. A single long prompt does not work here. Break the task into steps, attach a specific action and required tool to each one, describe edge cases, and be sure to mark which steps require a human decision. If you cannot clearly describe the process in words, no agent will execute it. That is the readiness test, and you can do it right now, in a regular chat.

Here is a prompt that turns your operating procedure into a draft workflow for a future agent – following the same five-layer logic the author describes. The example uses a typical support process – sorting morning requests; substitute your own:

A good result will flag refunds and payment disputes as mandatory human handoff points – if the model did not do that, the task was formulated too vaguely.

Once the draft is ready, hand it to a second model for review – let it find the gaps before you take the task to development. Below, a slightly “polished” draft of the same support process is used as an example; paste your own:

A good reviewer will immediately latch onto step 4: refunds under $50 processed by the agent on its own, with no checkpoint – a monetary action without human sign-off is the main gap in this draft.

Notice: both of these steps happen in a regular chat, before any talk of building a real agent. The costliest mistake is commissioning a complex agent for a process you could not clearly describe yourself.

The bottom line

“Managed autonomy” – that is the precise name for what a manager is actually deploying: the more freedom an agent gets, the more structure it needs around it, and designing that structure – including deciding where autonomy is acceptable and where human sign-off stays – is management work by nature. Accountability does not leave you; it shifts from execution to designing the checkpoints.

And the foundational skill underneath all of this is the same – whether for a single prompt or an agent. It is the ability to articulate precisely what you want, on what data, and with what constraints – essentially the same 5 elements of a good prompt, only applied to an entire process. Without that skill, neither a prompt nor an agent works. That is where it makes sense to start.

Related articles

Managers and AI: The Most Frequent Users – But Not for Managing

Why AI Pilots Die Between the Demo and the Factory Floor

40 GigaChat Case Studies vs the Benchmark: Checking Sber's Numbers

How to Implement AI in Manufacturing: A Step-by-Step Guide for Plant Leaders