Author(s): Jin Watanabe

Originally published on Towards AI.

AI agents fail not because models are weak, but because their environments are poorly designed. As AI systems move beyond simple chat and into autonomous execution, prompt engineering alone is no longer sufficient. Context engineering — how and when information is made available to an AI — has become the real core of agent development.

Introduction

2025 has been called the year of AI agents. We’ve moved beyond simple Q&A-style chat interactions to use cases where AI agents handle increasingly complex tasks.

While the importance of “prompt engineering” has been repeatedly emphasized, as AI agents have grown more sophisticated, “context engineering” has emerged as a broader concept that encompasses prompt engineering and has now become mainstream in AI agent development.

For those who are familiar with prompt engineering but only have a vague understanding of context engineering, this article summarizes the concept and importance of context engineering, which will continue to be central to AI agent development.

What is Context Engineering?

Context Engineering became a trending topic following this tweet by AI luminary Andrej Karpathy:

https://x.com/karpathy/status/1937902205765607626

Here, Karpathy points out the importance of context engineering as the art of designing what goes into an LLM’s context, beyond just prompt engineering.

For an overview of the elements involved in context engineering, this diagram by Philipp Schmid of Google DeepMind is helpful:

The New Skill in AI is Not Prompting, It's Context Engineering

Context Engineering is the new skill in AI. It is about providing the right information and tools, in the right format…

www.philschmid.de

Traditional prompt engineering focused on how to craft system prompts and user prompts. Context engineering is explained as a broader concept that includes prompt engineering plus short-term memory, long-term memory, RAG, tools, and structured outputs.

Context engineering itself is just a concept without a strict definition, and the components and technical approaches discussed within it are constantly evolving.

Therefore, it’s better to understand context engineering at a more abstract level to grasp its essential meaning. This will help you understand the whole picture faster and make it easier to catch up with future changes.

When someone who knows nothing about cars tries to learn about them, memorizing each component one by one — the engine, accelerator and brake, steering wheel, wipers — is very inefficient. But if you understand that a car is “something that carries people long distances,” you can naturally understand the necessity of the engine for power, the accelerator and brake for driving, and wipers for rainy conditions.

This way, when new things that didn’t exist when cars were invented — like navigation systems or child seats — come along, you can naturally catch up by reasoning backward from their necessity.

So what exactly is context engineering? In essence, context engineering is about “creating an environment where AI agents can work effectively.” The key point is how to create an environment that maximizes AI’s capabilities. Let’s think about this more intuitively.

Thinking of AI as a Person

Since discussions about AI tend to get sidetracked, let’s think of it in human terms.

Imagine your company has just hired an incredibly talented individual. Their background and skills are impeccable, they’re highly motivated, and you secretly think they might be more capable than anyone else in the company. This talented person has become your direct report (let’s call them “he” for convenience). He’s tremendously capable, but since it’s his first day, he naturally knows nothing about the company’s internal workings or operations.

How can you get the most out of his abilities?

First, the worst thing a manager can do is “provide no necessary information.” Some managers actually do this — thinking “he’s talented, so he can figure anything out,” they essentially abandon him without explaining anything about the work.

He’s talented and motivated, but he has no idea what’s expected in this workplace, what he’s allowed or not allowed to do, who handles what tasks, or who to ask for help. He becomes utterly confused. Every action becomes like a frame problem, and he can only take the safest, most non-committal actions.

This is a complete waste of talent. But the next problematic case is the opposite: “providing all information.”

You explain all organizational information, all internal systems, and give access to every document across all departments. You grant access to all data in internal databases, so there’s no information he can’t reach.

Does this enable him to perform his work smoothly? It’s better than providing no information, but he still can’t function well. Why? Because now there are “too many options.”

When to ask whom, which system to reference, whether to look at documents or extract from databases — with all these choices, there are too many possible actions, and he becomes paralyzed.

He’s probably so talented that he’ll try to push forward through trial and error, but since misses far outnumber hits, he’ll end up in a research hell of constant backtracking.

Too many options simultaneously increases investigation time and the possibility of rework, so it’s not necessarily a comfortable working environment.

So how can he fully demonstrate his exceptional abilities?

First, share the purpose of the work. Who is this work for and why? What level of quality is required? How much time can be spent? What’s allowed and what’s not? Share the overall picture of the work.

Then share the basic workflow.

The work consists of five major steps. In Step 1, extract basic data from System A and cross-reference with Database B values. In Step 2, conduct research and analysis using System C and Website D. In Step 3, reference past internal documents in Folder D… Explain the workflow overview and which systems and reference materials to use at each stage. Information like “if you’re stuck on XX, ask so-and-so” is also very helpful.

Then explain that the final output should be in PowerPoint, using these company design colors, that it’s for executives so keep it at a high level of abstraction, and that here’s last time’s presentation to reference. This dramatically increases clarity about the work.

Of course, since he’s talented, you don’t need to instruct every single action. Define a reasonable scope, and within that, he’ll make what he judges to be the best choices, and ask questions rather than proceeding blindly when unsure.

Even from a subordinate’s perspective, compared to a boss who gives no direction or dumps irrelevant information, a boss who provides appropriate information as described above creates a much more comfortable working environment. And this is the essence of context engineering.

In other words, just as it’s crucial to provide necessary information at the right time to even a talented human subordinate, context engineering for AI means guiding it to access the right information at the right time and setting up that environment.

With this perspective, the components of context engineering become naturally understandable.

First, the system prompt corresponds to purpose, sharing high-level policy. The user prompt is more specific instruction content.

Short-term memory is naturally needed to remember one’s actions and results. Long-term memory functions as a cross-task collection of best practices and knowledge.

RAG provides a mechanism to retrieve relevant information as needed. There are tools to support specific tasks including external information and web searches, and structured output mechanisms to specify the format of final outputs.

Though not mentioned above, sub-agent mechanisms naturally come into play when delegating specific tasks to work as a team.

Since the amount of information an LLM can hold at once (context window size) is limited, it’s important to determine what information to provide and when — just like humans have limits on how much information they can process at once.

In essence, the most important aspect of context engineering is not adding information, but “subtracting” it.

Many people hear that “AI agents autonomously select and filter information while performing tasks” and try to provide every piece of information and tool. This additive strategy doesn’t work.

When MCP became a hot topic, there was a tendency to promote that connecting all kinds of tools via MCP would make AI agents capable of anything. But this only makes the problems harder for AI to solve and leads to performance degradation.

Just as you would provide a comfortable working environment for a talented subordinate, context engineering — and the key to AI agent development — is not about indiscriminately adding information, but creating an environment where the right information is available at the right time.

By thinking “How can I create an environment where this AI agent can work effectively?” based on your use case, you’ll naturally arrive at the necessary technical elements and be able to consider approaches that haven’t even emerged yet.

Context Engineering is a Kind of Art

While intuitively understandable, since it’s just a concept, there’s no clear right answer. There are various best practices, but simply adopting all of them won’t guarantee improved AI agent quality.

As Andrej Karpathy pointed out that context engineering is essentially an art, AI agent development is becoming a kind of total artwork by the designer.

When properly considering how business flows should work given AI, the division of roles with humans, what degree of automation to aim for, what the UX for user interactions should be, and so on, it becomes necessary to examine not just the context window but whether to use an LLM at all. These are not independent areas but deeply interrelated domains.

While domain knowledge is naturally required, significant balance and comprehensive skills are demanded as an architect: the design ability to identify where to use AI agents while building the overall flow, internal AI agent design skills, and data engineering skills to organize the reference information itself.

This is one reason why AI agents often stall at PoC stage without reaching production level. Being able to do prompt engineering and AI implementation alone isn’t enough — overall design that maximizes AI’s potential is critical.

In other words, in the domain of LLM utilization, what’s required of AI engineers is creating environments where AI can work.

If you provide the right environment, AI will execute tasks at speeds and quality levels humans cannot achieve. No supercar can drive fast on poorly paved, bumpy roads.

The Pathology of Context

In the article mentioned above, Philipp Schmid of Google DeepMind states that “80% of failures in AI agent development stem from missing context information.” Let’s dig a bit deeper into why this isn’t simply about providing more information.

An important concept called “the pathology of context” is explained with four points in this article:

How Long Contexts Fail

Taking care of your context is the key to building successful agents. Just because there's a 1 million token context…

www.dbreunig.com

1. Context Poisoning

This refers to when incorrect information enters the context, that misinformation continues to be referenced in subsequent reasoning, causing the AI agent to take actions different from what was originally intended.

In an experiment where DeepMind’s Gemini 2.5 played Pokémon, once an incorrect game state or unachievable goal was included in the context, it would endlessly repeat nonsensical strategies based on that goal.

Context is memory and serves as a guideline for the AI agent, so once incorrect information enters, it’s difficult to correct and amplifies over time.

This is somewhat obvious, but when you pass miscellaneous information to an AI agent, if it inadvertently contains substantive errors, the AI agent will act based on that information unless it can logically determine it’s clearly wrong.

The quality of information in the context is more important than quantity.

2. Context Distraction

When too much information is crammed into AI at once, even if all of it is relevant, the massive context buries the most crucial instructions, leaving the AI unable to grasp what’s important.

As pointed out by “Lost in the Middle,” the maximum context length and the length at which practical reasoning is actually possible are different things, so stuffing the context with information is counterproductive.

This is equivalent to piling mountains of related documents in front of a subordinate, making it harder to find the most important information. You need to pass the minimum necessary information.

3. Context Confusion

This phenomenon occurs when **irrelevant information or tool definitions** are put into the context, causing the model to treat them as meaningful and degrading performance.

In experiments with tool connections via MCP, when only one tool was provided versus multiple tools including unnecessary ones, the latter case showed performance degradation across all models.

What’s put in the context is information the model cannot ignore, so it simply adds noise.

Some people try to share everything with subordinates, including irrelevant information, but irrelevant information is just noise that actually dulls judgment and leads to performance degradation.

4. Context Clash

This is when information within the context is contradictory and conflicting.

This requires attention with RAG as well. When documents with poor version control contain both new and old information, the reference information itself is contradictory, preventing the AI agent from making appropriate judgments.

You’ve probably experienced unorganized internal documents where multiple materials exist for the same content and it’s unclear which is authoritative.

If you think providing lots of information is always better and include outdated documents in the AI agent’s reference scope, it will retrieve old information too and either fail to make judgments or process whichever information it hits first as authoritative.

When having AI reference data including documents and tools, you need to ensure there are no contradictions in the underlying information.

Conclusion

What did you think?

Being overly influenced by the word “autonomous” in AI agents and dumping numerous pieces of information and tools all at once is, as we’ve seen, the approach that should be most avoided in AI agent development.

Context engineering, the key to AI agent development, is about providing the right context to AI agents at the right time and setting up conditions for AI agents to reference on their own. This is nothing other than creating an environment where AI agents can work effectively, just like humans.

It’s a kind of art, and since it’s becoming a domain of total artistry that includes business processes and data organization, it will be where consultants and architects can showcase their skills.

Furthermore, from the business side, having a project manager who gives precise instructions to subordinates and excels at management may lead to building better AI agents.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI