One Agent vs. Four Agents: The Pros and Cons of Context Isolation

Fun fact — 11x.ai lost a good chunk of reputation after fabricated case studies in its own marketing were discovered. The market's read on that wasn't "AI SDRs don't work," it was "don't ship one whose outputs a rep can't check." The whole industry narrative shifted from replace the SDR to give the SDR a tool they can actually trust going into a call — not one they have to fact-check afterward.

Reading about this gave me an idea to create an SDR research agent, where an inbound enquiry goes in, a sourced account brief comes out. The interesting problem isn't finding facts about a company; any agent with a search tool can do that. It's knowing which of those facts you're actually allowed to assert: every claim traces back to a source, every abstention is explicit, every conflict between sources gets surfaced instead of silently resolved.

The stack is LangChain and LangGraph for the agent, Tavily for web search — no vector database, no PGVector, nothing RAG-shaped. This is a live-research problem, not a retrieval one, and I wanted to see how that felt in Python specifically.

The plan

1. Create the tools, pass them to one agent, and eval the output against companies in the golden set.
2. Check that with an LLM judge.
3. Use that feedback to improve the agent and re-eval.
4. Swap in a Deep Agent with sub-agents and eval the same set.
5. Check that with an LLM judge.
6. Wire up tracing for real observability — I expected sub-agents to increase accuracy at the cost of more token usage, and wanted to actually see that trade rather than guess at it.
7. Use that feedback to improve the agent, re-eval, and start reducing cost.
8. Action the cost reductions.
9. Use that feedback to improve the agent again and re-eval.

01. The baseline: one agent, one shared context

Phase one of this thing is a plain LangGraph tool-calling loop. One agent, four tools (web_search, fetch_page, fetch_careers_page, lookup_funding), one shared conversation. I expected it to degrade, and it did — context filling up, the model attributing a funding figure to a source that was actually about headcount, citations drift. On the golden set (20 companies, hand-labelled), it settled around 7.1 tool calls per brief.

Four tools, deliberately not more. web_search and fetch_page cover general research. fetch_careers_page gets its own tool rather than being just another search query, because job postings are the strongest available signal for what a company is actually investing in — a company hiring two backend engineers for "document processing pipelines" is telling you its roadmap more honestly than its own marketing page does. lookup_funding is a funding-focused wrapper around the same search, kept separate so the sub-agent that owns funding research (more on that in section 03) has a tool named for exactly what it's supposed to do. The one rule that applies to all four: every tool returns a URL and a fetch timestamp alongside whatever it found, not just the content. That's not incidental — it's the only reason the citation chain in section 02 doesn't have to be reconstructed after the fact. A tool that can't say where a fact came from and when can't produce evidence, and a claim without evidence never reaches the output.

→ tools/fetch.py · tools/funding.py · tools/search.py

The fix, on paper, is well-known: give each research area its own isolated context so funding research can't bleed into hiring research. LangChain's Deep Agents package (create_deep_agent, on top of LangGraph) does exactly this — sub-agents, a virtual filesystem, a task tool for delegation. So phase four was: four sub-agents (company-profile, funding, hiring, news), each in its own context, each writing a findings file, with the parent reading only the four summaries and never the raw research.

→ agent_simple.py

02. The eval that makes every claim in this post checkable

Before touching any of the sub-agent stuff, I built a golden set: 20 companies, hand-labelled, and deliberately not 20 easy ones. ~14 are ordinary companies with discoverable facts. 3 are companies where a key fact is genuinely not public — private headcount, undisclosed funding — where the only correct answer is abstaining, and getting that subset right matters more than getting any of the easy ones right. 3 have known conflicting sources (the company site says one headcount, LinkedIn says another), where the correct behavior is surfacing the disagreement, not silently picking a side.

The metrics, on purpose, are almost all deterministic — no LLM judge required:

1factual accuracy vs. expected facts            no judge needed
2hallucination rate (schema violations)         no judge needed
3abstention correctness                         no judge needed
4conflict detection                             no judge needed
5steps / cost / latency per brief               no judge needed
6citation validity (source supports the claim)  yes — one cheap call per claim

That's not an accident, it's the actual design of the output schema — every claim carries an evidence list of source indices, so "did it abstain correctly" and "did it surface the conflict" become string/set membership checks, not vibes. The rules that make that hold are enforced in code, not left to the prompt to remember:

1kind == "verified"  -> len(evidence) >= 1
2kind == "inference" -> evidence references at least one verified claim's evidence
3every index in any evidence list must exist in sources

A brief that violates any of these gets kicked back to the agent for repair, with the specific violation named. Two failed repairs and the run fails loudly instead of shipping something that looks fine and isn't. The one metric that does need a judge (does the cited source actually support the claim) gets validated first, against 30 hand-labelled examples, before I trust a single number it produces. If the judge's own accuracy comes in under 80%, the citation-validity column gets printed with a warning attached rather than quietly presented as fact.

This is the part that made every "before/after" table in this post possible. Without it, "the fixes helped" is a feeling. With it, it's a specific citation-validity delta on a specific golden-set company, checked against a judge I'd already validated.

→ eval/run.py · eval/golden.json

03. Getting the parent and sub-agents set up

Everything below in this section lives in agent_deep.py.

Parallel dispatch isn't automatic. I assumed I'd be able to run four sub-agents in parallel, and that was just how task calls worked. It isn't — the docs are explicit that the main agent delegates via task tool calls "in a single turn to run them in parallel," which means parallelism only happens if the model chooses to emit four tool calls in one turn. Nothing structural forces it. I had to write that instruction into the parent's prompt by hand:

11. In a single turn, call the task tool four times — once each for
2company-profile, funding, hiring, and news (use those exact names as
3subagent_type). Do not research anything yourself directly; that's
4what the sub-agents are for.

→ agent_deep.py:129-132

Findings files don't write themselves. My original architecture assumed a sub-agent's output would land in findings/<area>.md automatically. It doesn't — a sub-agent's return value is plain text back to the parent unless its own system prompt explicitly calls write_file before finishing. Wasn't a deal breaker, but worth mentioning:

1When you are done, call write_file to save your findings to
2`findings/{filename}.md`. Structure it as a markdown list of facts,
3each with its source URL and fetched_at timestamp, so the parent
4agent can build citations from it directly.

→ agent_deep.py:80-85

There's a fifth sub-agent I didn't order. create_deep_agent auto-adds a general-purpose sub-agent to the task tool's options unless you either name one of your own subagents general-purpose or explicitly disable it. I did neither. So for a good chunk of this project, the parent had five delegation options, not four, and nothing in my prompt ever told it about the fifth one or scoped what it could do. More on why this mattered in a minute.

04. The bill arrives

First real smoke test, two companies, after all four sub-agents were wired up and the general-purpose leak was still live:

1                Notion Labs    Retool
2cost               $3.08        $2.54
3latency            311s         310s
4tool_calls          313          227
5model_calls          87           60

Compare that 227-313 to the 7.1 tool calls per brief from the single-agent baseline. That's a 30-45x multiplier, and on first glance it seems four sub-agents burning tokens independently is the known cost of isolation. But a multiplier that size isn't "more thorough," it's something being wrong.

The output quality was genuinely good, for what it's worth — real conflicts surfaced when compared to the single agent (a headcount disagreement, an ambiguous funding-round label), honest abstentions, well-numbered sources. So this wasn't a case of the architecture not working. It was a case of it working and costing far more than it should to get there.

05. Three fixes, and what each one actually saved

Three things, in order of how much they mattered.

No efficiency budget. The single-agent version's system prompt ends with an explicit instruction — a handful of well-chosen tool calls beats exhaustively searching, stop once you have enough. I'd never carried that line, or anything like it, into any of the four sub-agent prompts. Nothing told a sub-agent when "enough" was. Added to each one:

1Work efficiently: a handful of well-chosen tool calls beats
2exhaustively searching. You are one of four specialists researching
3this company in parallel — you do not need to verify every possible
4angle of your focus area, just enough to write a useful,
5honestly-hedged findings file. Once you have that, stop searching.

The math on the pre-fix version is ugly: 313 tool calls over 3 attempts (two repairs, maxed out every time), roughly 25 tool calls per sub-agent per attempt, against a whole-brief budget of ~7 for the entire single-agent version.

The fifth sub-agent, still uninvited. Confirmed via the harness's own task tool description — before the fix, the model literally saw general-purpose listed as a valid delegation target alongside my four real ones, description-free, scope-free. Turns out it only has filesystem tools by default (no web access), so it wasn't the redundant-research culprit I first suspected — but it was still an unaccounted, unprompted path the architecture wasn't supposed to have. Disabled with one call:

1register_harness_profile(
2    f"openai:{MODEL}",
3    HarnessProfile(general_purpose_subagent=GeneralPurposeSubagentProfile(enabled=False)),
4)

→ agent_deep.py:38-41

Checkpointed retries instead of full re-runs. This is the one I verified before trusting it, because it changes behavior in a way that's easy to get subtly wrong. The repair loop's malformed-date path — StructuredOutputValidationError, raised deep inside LangChain's own structured-output parsing — unwinds the whole agent.invoke() call with no result object. Without a checkpointer, my retry logic was falling back to the original bare enquiry message and starting over: all four sub-agents, from zero, every time a date came back formatted as "2021-10-08 (updated 2022-04-21)" instead of ISO 8601. And this happened on every single attempt in the first smoke test.

Before trusting a fix for that, I needed to know something the docs only implied: does a checkpoint survive an exception raised mid-graph, the way it survives a purpose-built interrupt() pause? Those are documented as the same mechanism, but I hadn't seen it confirmed for a raw exception specifically. So, another five-minute throwaway — a sub-agent whose tool appends to a real Python list, a parent told to delegate then deliberately call a tool that always raises:

1try:
2    agent.invoke({"messages": [{"role": "user", "content": "go"}]}, config=config)
3except RuntimeError:
4    pass
5# same thread_id, new message
6agent.invoke({"messages": [{"role": "user", "content": "ok"}]}, config=config)

call_log stayed at ['called'] across both invokes. The marker sub-agent did not re-run. Confirmed, not assumed — a checkpointer plus a stable thread_id means a crash mid-graph resumes from the last completed superstep, not from scratch, exactly like an interrupt would.

Put those first two fixes together and re-ran Retool:

1                    Before    After efficiency + no fifth sub-agent
2cost                $2.54     $1.20
3latency              310s      161s
4tool_calls            227       102
5model_calls            60        30

Roughly halved, across the board, output quality unchanged. And that's before the checkpointing fix even landed — that one doesn't show up as a flat percentage so much as it removes the failure mode entirely: a repair attempt that used to cost a full second (or third) run of four sub-agents now costs only the marginal retry of the synthesis step that actually failed.

→ agent_deep.py:211-217 (the checkpointer setup)

06. Wiring up the thing that should've come first

By the time I hit the cost blowup in the last section, I was debugging a black box: four sub-agents, each in their own context window, and the only visibility I had into what any of them actually did was squinting at a finished brief's source list and guessing sub-agent ownership from URL patterns and timestamps. Workable, and I did find a real duplicate fetch that way (retool.com/careers fetched three separate times by what were clearly three different code paths, 13 seconds apart) — but indirect, and indirect is exactly the kind of thing that hides a wrong number instead of surfacing it. I swapped the two phases. You can't fix what you can't see, and at that point I couldn't see anything.

So: LangSmith, env vars only, no code changes — LANGSMITH_TRACING=true, LANGSMITH_API_KEY. Except the first attempt 403'd on every single trace ingest, silently, while the agent itself ran fine. Turned out to be two separate things stacked on top of each other, and getting to the bottom of both without burning another agent run each time was its own small lesson:

• The API key was scoped to "Full Organization" instead of a specific workspace, which 403s on trace ingestion specifically (a documented LangSmith behavior — organization-scoped service keys need an explicit tenant header that the SDK doesn't send by default).
• Even after fixing that, still 403. Turned out the account was on the EU region and LANGSMITH_ENDPOINT was unset, defaulting to US. The maddening part: reads worked fine against the wrong endpoint the whole time (Client.info() kept succeeding), which is exactly why it took a minute to find — the failure only showed up on the write path.

07. What the trace actually showed

Once tracing worked, the real per-sub-agent breakdown for that Retool run:

1sub-agent          tool_calls  llm_calls   in_tok   out_tok    cost
2PARENT                      9          4    23452      4461  $0.074
3company-profile             15          6    77935      3058  $0.084
4funding                     14          7   108450      2728  $0.088
5hiring                      17          6    98627      3279  $0.102
6news                        21          7   125032      2782  $0.105

Roughly even across the four, which was reassuring — no single sub-agent was pathologically worse than the others, cost tracked fetch volume directly (the sub-agent that fetches the most pages pays the most per LLM call, because every new turn resends the whole growing conversation).

And then the trace turned up a bug I wouldn't have found any other way. lookup_funding calls web_search internally, which calls Tavily's own tool internally. Each of those inner calls is itself a traced LangChain tool, so one decision by the funding sub-agent — "call lookup_funding once" — was showing up as three separate tool-call entries. Across the whole trace: 100 counted, 76 real. Every tool_calls number I'd reported all session had been inflated by about 24%, and it had nothing to do with the agent's actual behavior — it was an artifact of how I'd wrapped Tavily.

Fixed with a general rule rather than a hardcoded exclusion list — only count a tool call whose parent isn't itself a tool call — so it holds for any future tool that wraps another tool, not just this one:

1def on_tool_end(self, output, *, run_id, parent_run_id=None, **kwargs):
2    if parent_run_id not in self._tool_run_ids:
3        self.tool_calls += 1

→ agent_deep.py:193-202

Verified for free, no LLM spend: invoke lookup_funding directly through the fixed callback, confirm it counts 1, not 3.

That fix matters beyond this one trace, because tool_calls isn't just a debugging number — it's one of the eval harness's own trajectory metrics, the same table that scores every golden-set run. A 24% inflation baked into a metric the eval prints as fact is exactly the kind of thing section 02 was supposed to prevent, and it slipped through anyway, because the bug lived in the instrumentation, not in anything the eval itself was checking. The eval can only be as honest as what feeds it.

08. Where it's left off

Back to the plan from the start of this post: steps one through six happened, more or less, if not in the tidy order this post's sections might suggest. Step nine — another eval pass after acting on the cost reductions, the actual 20-company delta this whole loop was building toward — is the one I stopped short of, on purpose, due to cost (this is a mock project, and I'm happy with the signal from the two companies I did take all the way through).

That's where it sits — and it's a good place to stop for now, not just a stopping point. On the two companies I ran end to end after all three fixes, the isolated version hit 100% citation validity, 100% factual accuracy, and surfaced real conflicts on both — a headcount disagreement, an ambiguous funding-round label — that the single-agent version had run on the same companies and missed entirely. It's costing roughly half what it did a session ago. And every bug in this post — the uninvited fifth sub-agent, the missing efficiency budget, a tool getting counted three times for one decision — got caught and fixed before it shipped anywhere, because the eval and the trace were both actually there to catch it.

The TLDR — build, judge, eval, repeat. Otherwise we're just vibecoding ;)