Every major A.I. platform now offers two dials that shape what you get back: how large of a model you pick and how much effort it can invest. Here's what each dial does and how to use them together:
THE "MODEL SIZE" DIAL
• Model selection swaps which set of frozen weights handles your request. Weights are fixed at training time; nothing in your prompt changes them.
• Larger models encode more knowledge and capability, and each output token costs more.
• Your prompt steers predictions but doesn't teach: Paste in docs for a library the model has never seen and it will use them for that request, then retain nothing.
THE "EFFORT" DIAL
• Effort shapes all output tokens: reasoning, tool calls and messages to you.
• At high effort, a model reads more files, verifies more of its own work and pushes further before checking in. In one Anthropic comparison, the high-effort path generated ~7x more tokens to reach a higher-confidence answer.
• Effort sets how far a model is willing to travel, not how far it *must* (thus a well-trained model stops when it finds the bug rather than padding the bill).
WHICH DIAL SHOULD YOU TURN?
• First off, when output disappoints, try fixing your prompt and context. A vague request is the most common culprit for a disappointing output and *no* knob will fix that.
• Skipped files, unrun tests, an abandoned refactor? Those are examples of DILIGENCE FAILURE: in these cases, RAISE THE EFFORT.
• Full context, visible attempt, still confidently wrong? Those are examples of CAPABILITY FAILURE: in these cases, upgrade to a BIGGER MODEL.
COST
• Cheaper *per token* isn't always cheaper *per task*. On hard multi-step work, a large model reaches the quality bar in fewer steps, so total cost can come out lower than a small model grinding at the ceiling of its ability (see chart).
• On routine work, the equation flips: Both models get it right, so the big model's extra verification is wasted money. Drop down to a smaller model.
WHAT CAN YOU DO?
• Start with defaults; providers tune them to what most users want to spend. Treat effort as a preference for your kind of work, generally NOT as a task-by-task fiddle.
• Route routine work to smaller, cheaper models and reserve the frontier for problems that stretch it.
• This applies everywhere: Anthropic, OpenAI, Google and open-source reasoning-model providers have all converged on separate capability and effort controls.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
Filtering by Tag: #LLM
Alibaba’s Qwen3.8-Max: Open-Weight Model Surpasses Most American Frontier Labs
Chinese tech giant Alibaba is shipping Qwen3.8-Max: It'll be the largest open-weight A.I. model ever released and (as shown in chart) it's competing at the frontier alongside closed-source models from Anthropic and OpenAI. Here's what you need to know:
THE MODEL
• 2.4 trillion parameters in a mixture-of-experts architecture; only ~95B are active per token, so the headline number reflects capacity, not per-request compute.
• Accepts text, images and video, with a one-million-token context window (~750K words).
• Selectable low, medium or extra-high reasoning effort, so no paying for lengthy thinking traces when you need a quick lookup.
• All part of Alibaba's ~$53B, three-year bet on cloud and A.I. infrastructure.
THE BENCHMARKS
• Alibaba frames it as second only to Anthropic's Claude Fable 5; independent signals so far land in a similar neighborhood.
• Immediately became the highest-ranking Chinese model for text on the Arena leaderboard and ranked second globally on vision.
• Scored 86.6 on Terminal-Bench (agentic command-line tasks), ahead of both Claude Opus 4.8 and Fable 5.
• Vendor demos showcase multi-day autonomy: 10+ days coding unattended, plus reproducing an ML research paper from scratch and then beating 87% of 526 human teams in a live contest.
THE PRICE WAR
• $2 per million input tokens, $6 output and 25¢ cached input.
• Undercuts domestic rival Kimi K3 by more than half on output.
• Combined rate is less than a third of Claude Opus and under a quarter of GPT-5.6 Sol's... and cached-input pricing lets agentic workloads collapse toward the 25¢ floor.
IS IT SAFE TO USE A CHINESE MODEL?
• The risk depends less on the model and more on how your data reach it.
• Consumer apps and the hosted API route data through infrastructure governed by Chinese law, so keep anything sensitive or proprietary out of that.
• Safer: open weights hosted by a Western A.I. cloud (like Lightning AI) in your own jurisdiction.
• Safest: run the model on your own hardware: weights are inert files that can't phone home.
• Caveats: outputs reflect training under Chinese content regulations, so evaluate before trusting; practice supply-chain hygiene (official repos, checksums) and check whether your industry restricts Chinese-origin models.
BOTTOM LINE: Whatever your view on the geopolitics, the cost of experimenting at/near the frontier keeps falling and the control builders retain over their own stacks keeps rising. In my view, that's great news :)
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
Fable 5 as Advisor: Anthropic’s Two-Model Pattern for Smarter, Cheaper Agents
Want near-frontier A.I. agent quality at a fraction of the cost? Anthropic recently productized the Advisor Strategy that pairs a cheap "executor" model with a brilliant "advisor" to give you the best of both worlds:
HOW IT WORKS
• A fast, cheap model (e.g., Claude Haiku or Sonnet) runs the entire agent loop: calling tools, writing code, drafting output.
• A frontier model (e.g., Claude Opus or Fable) sits on standby as a "tool" the executor can consult (like a junior worker phoning their supervisor when unsure).
• Everything happens inside one API call: Anthropic's servers hand the advisor the full conversation transcript and return just 400-700 tokens of advice, making this fast and inexpensive (it's also usually only a one-line code change so it's easy to implement).
THE RESULTS
• Sonnet + Opus advisor beat Sonnet alone on the "SWE-bench Multilingual" benchmark by 2.7 percentage points while cutting cost per task by 11.9%. Better quality AND slightly lower cost.
• Unsurprisingly, the biggest gains come from pairing a very fast/cheap model with a much more capable advisor: For example, on BrowseComp (web research benchmark), Haiku alone scored 19.7%; Haiku + Opus advisor scored 41.2% (more than double!) at 85% less cost than Sonnet alone.
• Newest data, from last week: On "SWE-bench Pro", Sonnet 5 + a Fable 5 advisor captured ~92% of Fable's standalone performance at ~63% of its cost.
WHY IT WORKS
• The advisor's output is tiny relative to the whole task, and a good plan delivered early prevents wasted attempts and misguided tool calls.
• Unlike OpenAI's router (which dispatches queries to a model up front), the cheap model runs the show and escalates itself mid-task with full shared context.
PRACTICAL LESSONS
• Skip it for single-turn Q&A; it shines on long-horizon agentic work (like coding, research, computer use).
• Executors under-call the advisor by default so prompt them to consult it early (before committing to an approach) and late (before declaring the task done).
• Cap advisor output at ~2,000 tokens (~7x cost reduction, no quality loss) and enable prompt caching for long loops.
• The pattern is spreading: OpenRouter now offers a cross-provider version (e.g., a Google Gemini executor consulting Claude).
• Alternative design patterns such as having a powerful "orchestrator" (shown below the advisor pattern in the chart I included in this post) might work even more effectively for your use case so it could be worth comparing them.
BOTTOM LINE
Frontier A.I. progress is no longer just bigger models... it's smarter economics in composing the models we already have.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
What’s Left to Build When Software Is Free, with Chip Huyen
For today's landmark episode (#999!), I asked rockstar Chip Huyen to be my guest and she said "yes"! We discuss her book "A.I. Engineering" (the most popular O'Reilly book in 2025) and how the A.I. job landscape is shifting.
In case you haven't heard of her, more on Chip:
• Her most recent book is "AI Engineering", which was the most popular book in the O'Reilly platform last year.
• Previously wrote “Designing Machine Learning Systems”, which was also an O'Reilly mega-bestseller and was based on the Stanford University course she created and taught on the same topic.
• Is currently building a new stealth startup.
• Previously worked as VP of AI at Voltron Data, co-founder of Claypot AI, ML Engineer at Snorkel AI and Sr Deep Learning Engineer at NVIDIA.
• Holds a Master's in Computer Science from Stanford.
• Her invaluable posts have earned her over 300k followers on LinkedIn.
In this episode, Chip breaks down:
• What separates AI engineering from machine learning engineering.
• The case for a "start simple" workflow.
• The real costs of running LLMs in production.
• Physical AI.
• Robotics.
• World models.
• Why the durable problems worth solving are increasingly human ones.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
End-to-End Foundation Models for the Energy Industry, with Jazmia Henry
What does it take to build foundation LLMs from scratch today? Deeply impressive Jazmia Henry breaks down the four stages in today's episode, enjoy!
Jazmia:
• Holds degrees from Tulane University and Columbia University... and is partway through a PhD at the University of Oxford.
• Held a technical fellowship at Stanford University.
• Previously worked as a data strategist at Morgan Stanley, head of ML at The Motley Fool and a Lead Applied AI engineer at Microsoft.
• Published a top paper at NeurIPS, the world's most prestigious academic AI conference.
• Currently works as "Member of Technical Staff for AI/ML" at collide., a Texas-based startup that’s building AI infrastructure (including all aspects of specialized foundation models) for the energy industry.
Key topics covered in this episode include:
• What foundation models are.
• Her "full-stack" foundation-model building's four distinct stages.
• How reinforcement learning (RL) models are "bursty" because they idle the GPU during reward calculation and then dump enormous loads on it all at once.
• Reward hacking by RL models.
Thanks to Mark Freeman II for recommending Jazmia as a guest.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
How to Build AI-First Organizations, with Jacob Miller and Jeremy Mumford
After today's fun episode with Jacob and Jeremy — authors of the brand-new book "Architected Intelligence" — you’ll have all the key info to build successful AI features, AI products and AI-first companies. Enjoy!
Jeremy Mumford and Jacob Miller serve as Lead AI Engineer and Vice President of Platform Intelligence, respectively, at Pattern, a giant Utah-based tech company that IPO’ed on the Nasdaq exchange about six months ago.
Jacob and Jeremy's brand-new "Architected Intelligence" book was published by Wiley and this episode focuses almost exclusively on this invaluable book.
Episode highlights include:
• The "User Agnosticism Tenet", which means designing products and processes so they can be executed equally well by a human, an AI agent, or any hybrid combo.
• The shift in the "define-build-feedback" loop today where "building" is no longer the bottleneck, which means "definition" and "feedback" are where teams win or lose.
• Why workflows are deterministic, predictable, and cheaper than agents, and why the natural progression is skills first, then workflows, and only then agents.
• Why data engineering is the bedrock of AI engineering.
• Why velocity is the only durable moat in a world where everyone has access to the same frontier models.
Thanks to podcast superfan Jonathan Bown for recommending Jeremy and Jacob as guests!
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
Tokenmaxxing vs AI Hardware Bottlenecks
Humans (like Reinforcement Learning algos) can "reward hack": "Tokenmaxxing" being a perfect example, after employers started using "number of tokens" consumed as a proxy for developers' productivity.
Even if humans weren't engaging in this pointless time-, money- and energy-consuming behavior, however, demand for A.I. compute is so vast that everyone's scrambling to to make more available. Alas, four tricky hardware bottlenecks face us:
1. GPUs:
• NVIDIA data-center GPU lead times now run 36–52 weeks, with Blackwell chips sold out through mid-2026.
• The real choke point isn't fabrication: It's TSMC's "CoWoS" advanced packaging, which is sold out through 2026. Nvidia alone has locked up ~60% of CoWoS capacity through 2027.
2. High-Bandwidth Memory (HBM):
• Demand has quintupled since 2023, and only three companies (SK hynix, Samsung and Micron) make it.
• All three are sold out well into 2026 and new HBM factories take 18–24 months to come online.
3. CPUs:
• As workloads shift toward agentic AI, the CPU:GPU ratio jumps from ~1:12 (for GenAI-only chatbots) to 1:1.
• Intel's CFO says the server-CPU shortfall "starts with a B" — billions in unmet demand so server CPU prices are up 10–20% in just the past couple of months.
4. Electricity: Hyperscaler build-outs are now gated by grid interconnect (18–36 months) and transformer lead times.
THE BIG MISMATCH
• The top 5 hyperscalers alone (Alphabet, Amazon, Meta, Microsoft and Oracle) are on track for ~$725B in combined 2026 capex.
• That's roughly 6x the hyperscalers' 2022 spend, with ~75% going to A.I. infrastructure.
• Hardware suppliers, however, have grown capex by only ~50%.... a 6x increase in demand met by only a 50% increase in supply is a big mismatch!
REASONS FOR OPTIMISM
Demand will continue to be high but I'm optimistic we'll continue to squeeze more juice from every lemon because, e.g.:
• Algorithmic efficiency keeps improving — Google's TurboQuant recently briefly tanked memory stocks by promising to materially cut inference memory needs.
• LLM efficiency gains via mixture-of-experts and smarter inference scheduling continue to compound.
• The tokenmaxxing trend is a corporate farce that will fade.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
The Four Types of Memory Every AI Agent Needs, with Richmond Alake
To build an effective A.I. agent, getting its memory right is essential. In today's episode, our agent-memory guide is brilliant (and very funny!) machine-learning architect and engineer, Richmond Alake.
More on Richmond:
• Director of A.I. developer experience at Oracle.
• Previously roles include: staff developer advocate for AI/ML at MongoDB, ML architect at Slalom, writer for NVIDIA and computer-vision engineer at Loveshark.
• Holds a master's in ML and robotics from the University of Surrey.
In this episode, Richmond magnificently covers:
• How agent memory is the encapsulation of systems (embedding models, rerankers, databases, and LLMs) that allow AI agents to learn and adapt with new information over time, rather than starting from scratch every session.
• The four types of agent memory (all drawn from human cognition).
• Memory-first agent harnesses.
• Predictions for a flattening of AI engineering roles, where the future developer will need end-to-end understanding of the full agent stack.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.