Ever written careful instructions for an A.I. agent, only to watch it ignore them an hour into a session? The fix lies in knowing WHERE your instructions should live. Read on for all the details...
WHY AGENTS "FORGET"
• Everything an agent knows lives in its context window, and every token costs money and (more subtly) attention.
• Long sessions trigger "compaction": The conversation gets summarized to free up room, and instructions given early can get squeezed out.
• Every steering method answers one question: How do I make an instruction cheap to carry and hard for the agent to lose?
THE STEERING TOOLKIT (drawn from Anthropic's Claude Code, but the ideas generalize)
1.) Always-on files (e.g., CLAUDE.md): Loaded every session and re-read after compaction. Keep them under 200 lines, give them an owner, review changes like code.
2.) Rules: Path-scoped constraints that load only when relevant files are touched.
3.) Skills: Procedures (deploy workflows, checklists) whose full text loads only when invoked.
4.) Subagents: side tasks that run in isolated context windows... only the final summary returns.
5.) Hooks: deterministic code that fires on lifecycle events. The big idea: An instruction is a probability while a hook is a guarantee. If something must *never* happen, enforce it with code, not prose.
THE INDUSTRY IS CONVERGING
• AGENTS.md (kicked off by OpenAI, stewarded by the The Linux Foundation and backed by Google, Microsoft and AWS) serves similar function to CLAUDE.md and is read natively by Codex, Cursor, GitHub Copilot, Gemini CLI and dozens more tools across 60,000+ repositories.
• ETH Zurich researchers studied 138 real-world repos: Developer-written instruction files improved agent task success ~4% and cut agent-introduced bugs by 35-55%.
• The same study found LLM-generated instruction files DECREASED success while raising inference costs by 20%+. These data suggest the value is the human judgment encoded in the file... **you can't delegate the steering wheel to the thing being steered**.
NOT A CODER? THE SAME FRAMEWORK APPLIES
• Custom instructions = your always-on file (keep it short).
• Projects and Gems = your path-scoped rules.
• Custom GPTs = the consumer cousin of skills.
BOTTOM LINE
• Match persistence to relevance: always-on files stay ruthlessly short; procedures and area-specific conventions load on demand.
• "Always" and "never" are signals you need a guardrail (deterministic enforcement), not an instruction.
• Treat steering files as code: owned, reviewed and pruned. An instructions file that grows without gardening dilutes the instructions that matter.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
Filtering by Tag: #aiAgents
How dbt Won Analytics Engineering, with dbt Lab’s CEO Tristan Handy
dbt is THE open-source tool that brought software-engineering rigor to data transformation; it's now used by over 100,000 teams. Today's rockstar guest, Tristan Handy, is CEO of dbt Labs, the company behind the movement.
More on Tristan:
• President and co-founder of "Fivetran + dbt Labs" (recent merger).
• Coined the term "analytics engineering".
• Over two decades of experience as a data practitioner working in both large enterprises and startups.
• His expertise and data industry best practices have influenced thousands of subscribers and listeners weekly via his newsletter (The Analytics Engineering Roundup) and The Analytics Engineering Podcast.
In this episode, Tristan explains:
• What dbt is.
• Why dbt Labs' the recent merger with Fivetran is a win for dbt users.
• Why skill files are so powerful.
• Why he turned down acquisition offers for years
• How the semantic layer keeps A.I. agents from confidently getting your metrics wrong
• ...and much more.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
Fable 5 as Advisor: Anthropic’s Two-Model Pattern for Smarter, Cheaper Agents
Want near-frontier A.I. agent quality at a fraction of the cost? Anthropic recently productized the Advisor Strategy that pairs a cheap "executor" model with a brilliant "advisor" to give you the best of both worlds:
HOW IT WORKS
• A fast, cheap model (e.g., Claude Haiku or Sonnet) runs the entire agent loop: calling tools, writing code, drafting output.
• A frontier model (e.g., Claude Opus or Fable) sits on standby as a "tool" the executor can consult (like a junior worker phoning their supervisor when unsure).
• Everything happens inside one API call: Anthropic's servers hand the advisor the full conversation transcript and return just 400-700 tokens of advice, making this fast and inexpensive (it's also usually only a one-line code change so it's easy to implement).
THE RESULTS
• Sonnet + Opus advisor beat Sonnet alone on the "SWE-bench Multilingual" benchmark by 2.7 percentage points while cutting cost per task by 11.9%. Better quality AND slightly lower cost.
• Unsurprisingly, the biggest gains come from pairing a very fast/cheap model with a much more capable advisor: For example, on BrowseComp (web research benchmark), Haiku alone scored 19.7%; Haiku + Opus advisor scored 41.2% (more than double!) at 85% less cost than Sonnet alone.
• Newest data, from last week: On "SWE-bench Pro", Sonnet 5 + a Fable 5 advisor captured ~92% of Fable's standalone performance at ~63% of its cost.
WHY IT WORKS
• The advisor's output is tiny relative to the whole task, and a good plan delivered early prevents wasted attempts and misguided tool calls.
• Unlike OpenAI's router (which dispatches queries to a model up front), the cheap model runs the show and escalates itself mid-task with full shared context.
PRACTICAL LESSONS
• Skip it for single-turn Q&A; it shines on long-horizon agentic work (like coding, research, computer use).
• Executors under-call the advisor by default so prompt them to consult it early (before committing to an approach) and late (before declaring the task done).
• Cap advisor output at ~2,000 tokens (~7x cost reduction, no quality loss) and enable prompt caching for long loops.
• The pattern is spreading: OpenRouter now offers a cross-provider version (e.g., a Google Gemini executor consulting Claude).
• Alternative design patterns such as having a powerful "orchestrator" (shown below the advisor pattern in the chart I included in this post) might work even more effectively for your use case so it could be worth comparing them.
BOTTOM LINE
Frontier A.I. progress is no longer just bigger models... it's smarter economics in composing the models we already have.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
TrueFoundry’s Nikunj Bajaj on How to Get $100M Returns on AI Agent Deployments
Imagine being able to deploy an AI agent and getting a return of over $100m from that single deployment. My guest today, Nikunj Bajaj, has facilitated that multiple times! Lots to learn from him, enjoy!
Nikunj:
• CEO and co-founder of TrueFoundry, a Bay Area-based startup that has raised over $20m to solve the thorniest problems that enterprises face when deploying agents.
• His clients include demanding organizations like NVIDIA and Siemens.
• Was previously ML tech lead at Facebook.
• Holds a master's in computer science from University of California, Berkeley.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.