A GitHub repo I starred in January does not run anymore. The tutorial I followed in February has a pinned issue that says "keys getting banned." The Mac mini I was going to buy is still on the wish list, which turns out to be the only part of this I got right.
This is not an anti-AI post. The tools are real, the tokens are real, the productivity wins are real. But most AI tutorials in 2026 are selling layers 1 and 4 of a four-layer stack and hoping you do not look at layers 2 and 3 until after your credit card is on file.
This post is a teardown of those four layers, with real numbers, and a checklist you can run on any tool before you commit hardware or a subscription to it.
The four layers of the hype stack
| Layer | What it is | Who pays for it |
|---|---|---|
| 1. UI wrapper | The app you interact with | You (subscription) |
| 2. Orchestration | Agent loop, tool calls, memory, retries | You (eventually, in tokens) |
| 3. Inference | The model behind the curtain | You (tokens) or the provider (loss) |
| 4. Hardware | Whatever runs the inference | Data center, or your desk |
Most hype cycles concentrate on layers 1 and 4 because those are the layers a YouTube thumbnail can sell. A slick dashboard on layer 1. A glowing Mac mini on layer 4. Layer 2 is engineering and hard to video. Layer 3 is invisible until the bill arrives.
Rule of thumb: If a tutorial spends more time unboxing a Mac mini than discussing the inference bill, layer 3 is where your surprise is going to land.
Layer 1: The wrapper
Call it what you want. Agent, copilot, assistant, IDE extension, self-hosted dashboard. The UX sits on top of someone else's model. That is not a bad thing. Wrappers are legitimate businesses. The problem is when the pitch implies the wrapper is the product, which it almost never is.
Audit questions for layer 1:
- Can I swap the model underneath without rewriting my workflow?
- What happens to my prompts, context, and memory if the provider changes ToS?
- Is there a ToS clause in the provider's terms that names this tool, or its category, as prohibited?
That last one bites harder than people realize. As of April 2026 there are several major provider ToS updates that explicitly prohibit certain third-party agentic tools. Users have reported keys being revoked. A wrapper that routes through a non-permitted path is one enforcement pass away from being a paperweight.
Layer 2: Orchestration
This is where the real engineering lives. Tool calling, retries, memory, sandboxing, error recovery, concurrency, cost control. Most OSS projects that show off a "watch it book my flight" demo are one degraded model or one retry storm away from the same flow taking forty minutes and four dollars in tokens.
Agent loops are where cost explodes. A single user message can trigger ten or fifteen tool calls, each with a full context window of system prompt, tool descriptions, and prior turns. Input tokens dominate. You do not see this on the demo.
# Simplified agent loop cost surface
def agent_run(user_msg):
context = load_memory() # N input tokens
while not done:
response = llm.call( # N input + M output tokens
system + tools + context
)
tool_result = run_tool(response)
context.append(tool_result) # grows N each iteration
return final_answer
Notice the loop. Each iteration includes the full context of every previous iteration. A ten-step agent run with a 30K-token system prompt and 4K-token tool results runs roughly 300K input tokens before you count output. At current Claude Sonnet rates that is about $0.90 for a single user request. Do that twenty times a day and you have replaced a $20 subscription with a $540 month.
Real numbers from the All-In podcast in February 2026:
- Jason Calacanis: his team's AI agents were running at $300 a day, roughly $110K annualized.
- Chamath Palihapitiya: started imposing token budgets on his developers. Direct quote, paraphrased: "I'll run out of money."
- Mark Cuban: eight Claude agents at $300 a day plus a human maintaining them at $200 a day costs more than the employee they replace.
These are people who can afford it. They are flinching.
Audit questions for layer 2:
- What is the worst-case token spend on a single request loop?
- Does the agent have a hard cap on tool-call iterations?
- Is retry logic fixed-interval or exponential? Exponential plus a flaky tool is where budgets die.
- Does the memory layer grow unbounded, or is it summarized?
Layer 3: Inference
Every tool you are looking at, self-hosted or otherwise, runs on someone's inference bill. When a subscription "includes" a feature, somebody is eating the difference. That works until it does not.
Recent, concrete, April 2026 examples:
- April 4, 2026: Anthropic revoked subscription billing for third-party tools. OpenClaw and similar tools that previously rode on a Pro/Max subscription now bill to Extra Usage.
- April 21, 2026: Anthropic A/B tested removing Claude Code from the $20 Pro plan for 2% of new signups. The pricing page briefly reflected the removal before clarification.
- 2025 and 2026 economics: OpenAI reportedly burned about $8 billion against $13 billion in revenue in 2025, with roughly $14 billion in losses projected for 2026. Sam Altman has publicly said OpenAI loses money on the $200/month ChatGPT Pro tier.
- GTC 2026: Jensen Huang put a gigawatt data center, amortized over 15 years, at roughly $40 billion empty.
Per-token prices have collapsed. GPT-4 input tokens went from $30 per million to $2.50 per million in eighteen months, an 80 to 90 percent drop. The floor is not zero, and consumption keeps rising faster than price falls.
| Metric | 2023 | 2025/2026 | Change |
|---|---|---|---|
| GPT-4-class input tokens | $30/M | ~$2.50/M | -90% |
| Avg enterprise AI spend | — | $85K/month | +36% YoY |
| Orgs spending >$100K/month | baseline | 2x baseline | doubled |
| Daily tokens processed (China alone) | 100B (Jan 2024) | 140T | ~1400x |
The structural problem: per-token costs are down 80-90% in 18 months, per-user consumption is up faster. Your effective monthly bill is not going down.
Audit questions for layer 3:
- If the provider moved every call to strict pay-per-token tomorrow, what would my monthly bill look like?
- Can I cap spend at the provider level, not just the app level?
- Does the tool work with a local fallback that preserves 80% of function, or does it hard-fail without the frontier model?
Layer 4: Hardware
The Mac mini posts, the "$2000 gets you your own AI" guides, the affiliate-stuffed "best LLM server" roundups. This is not a scam. Apple Silicon really is good for local inference. Unified memory, no PCIe copy penalty, low idle power, silent. All genuinely useful.
But the honest read on local models for agentic workflows in April 2026:
| Model | RAM needed | Speed on consumer Apple Silicon | Usable for agents? |
|---|---|---|---|
| Kimi K2 (1.8-bit quantized) | 245GB+ | 1-2 tokens/sec on Mac Studio 512GB | No, too slow |
| Qwen3-Coder-30B (MoE, 3B active) | 64GB | 10-15 tokens/sec on M4 Pro | Yes, for simple flows |
| GLM-4.7-Flash (9B active, 128K ctx) | 24GB | ~20 tokens/sec on M4 | Yes, for coding subset |
| GPT-OSS-20B | 24GB | ~15 tokens/sec on M4 | Yes, general purpose |
Local models are usable. They are not the same as frontier cloud models for multi-step agentic work, especially when the task needs long-horizon planning or serious tool use. A local 30B is a working tool. A local Kimi K2 is a science project unless you own a Mac Studio cluster.
Anyone telling you otherwise is usually selling you something (affiliate link, course, the same course with a new thumbnail).
Audit questions for layer 4:
- Do the benchmarks in the guide match the task I actually want to do, or are they summarization benchmarks being used to imply coding performance?
- What is the tokens-per-second floor for the model at the context length I'll actually use?
- If I only use the hardware four hours a day, is the cloud API still cheaper at my real usage?
Four signals a stack is on borrowed time
Run these before you commit.
- The pricing page changed silently in the last 30 days. Check the Wayback Machine. Compare.
- The tutorial's recent comments are mostly "this doesn't work for me." Filter by newest. If the top new comments are bug reports, the snapshot decayed.
- The GitHub issues tab has an open "API keys being banned" or "provider blocked us" thread. Search by keyword. This one is terminal.
- The tool routes through a provider whose ToS explicitly names it, its category, or its pattern. Find the ToS. Ctrl-F the project name.
Three of four true: the stack is late cycle, budget accordingly or skip.
One or two true: yellow flag, keep watching.
Zero: you might be early, which is also risky, but at least the risk is legible.
What to actually build on
Boring but it holds up:
- Prefer tools where the provider bills you directly. The bill is honest. The less abstraction between you and the token meter, the fewer surprises.
- Prefer stacks where swapping the underlying model is a config change, not a rewrite. Portability is the cheapest insurance you can buy.
- Treat GitHub stars like follower counts. Not meaningless, but not the product.
- Assume any third-party tool that rides a major provider's consumer subscription instead of an API key is temporary, and price that temporariness into your decision.
- Run the 30-day test. If a tutorial still has fresh comments saying "this works" a month out, it is probably safe. If not, skip.
None of this is anti-AI. It is pro not-getting-caught-out. The tools are real. The wins are real. The token bill is also real. The layer you are not looking at is usually the one that bites.
What signals have you seen that a tool is late cycle? Drop them in the comments. I am genuinely collecting patterns, because the next wave of this is already starting.
Top comments (4)
Your point maps pretty cleanly to Amazon’s consumer playbook.
They came in with artificially attractive economics: cheap books, cheap products, free-ish shipping through Prime, and a lot of UX work designed to get themselves wired directly into household purchasing loops. Once they had enough behavioral lock-in, the system got worse in the places that matter but are hard to audit. Shipping guarantees softened, prices absorbed the cost of the “free” parts, and the actual end-to-end cost of convenience became increasingly opaque. By the time you realize your replenishment flow is quietly extracting margin at every step, you’re already operationally dependent on it.
I think tokens are heading toward the same pattern.
They’re cheap right now because vendors are still subsidizing adoption, still normalizing API-first workflows, and still trying to become infrastructure before customers develop price discipline. The optimistic story is that Moore’s law, better chips, and inference efficiency will keep bending the curve down. The more likely story is that vendors use those gains to protect margin while introducing new forms of cost extraction.
Not necessarily through obvious “price went up” moves, either. More likely through invisible degradations:
That last part is the piece I think people are underrating. A lot of AI tooling today has “AWS bill from hell” energy, except hidden behind cheerful product language. The system says it is helping, optimizing, streamlining. What it is really doing is inserting abstractions between the developer and the cost surface.
So yeah, I share the skepticism. The current token economy feels less like a stable equilibrium and more like an onboarding subsidy. Once vendors are sufficiently embedded in people’s workflows, the pressure will be to capture more value, not less. And because most users won’t inspect routing, fallback behavior, retry loops, or effective per-task cost, a lot of that extraction can happen without ever appearing as a clean line-item price hike.
I would not assume this gets cheaper in the way people mean cheaper. It may get cheaper per token while getting more expensive per useful outcome.
That layer 3 audit question about what happens if the provider moves everything to pay-per-token tomorrow is the one that keeps me up. Not because I don't know the answer, but because I've seen the pattern before. It's the same shape as "free tier will cover you forever" optimism from the early cloud days, right before the S3 bill arrives.
What struck me reading this is that the entire stack is built on a polite fiction: that someone else is eating the margin. And the fictions are getting less polite. The Anthropic examples from this month are the canary. They're not being subtle about it.
I think the mental model I've landed on is that any tool I don't pay a metered bill for directly is a tool I should treat as having an implicit expiration date. Not because it's bad software, but because the economics underneath it are still being negotiated in real time. The wrapper isn't the product. The wrapper is a bet that the provider won't change the rules before the next funding round closes.
The GitHub stars-as-follower-counts point lands too. I've starred repos in January that I can't run in April. Not because the code rotted, but because the API surface underneath it shifted. The star is just a timestamp of when I was optimistic.
The question I keep coming back to: at what point does the portability insurance (layer 4 hardware) become cheaper than the recurring uncertainty premium on layers 1-3? For my actual usage patterns, I suspect the crossover point is closer than the YouTube thumbnails suggest, but further out than the "just buy a Mac mini" crowd admits. Somewhere in the boring middle, where the spreadsheet lives.
The Layer 2 cost math is the part nobody wants to talk about. I built an AI support agent for DeFi protocols where every user question can trigger 5-10 tool calls (transaction lookup, balance check, approval scan, gas estimate). Each call carries the full system prompt plus tool descriptions plus conversation history. A single user interaction can easily burn 100K+ input tokens before the agent gives a final answer.
At $0 MRR that's pure cash burn on every demo interaction. The "free tier" of 3 messages on my demo widget costs me real money per visitor. Most SaaS founders think about CAC in terms of ad spend. When your product is an AI agent, the demo itself is your biggest cost center.
Your agent loop code example is exactly right. The context growth per iteration is the silent killer. I ended up implementing aggressive caching on chain data responses so that if the same transaction gets looked up twice, the second call skips inference entirely and returns cached results. Doesn't solve the core problem but cuts repeat query costs to near zero.
The signal I'd add to your "borrowed time" checklist: if the tool's pricing model assumes a flat subscription but the underlying costs scale with usage, the math breaks the moment a power user shows up. Every flat-rate AI product is one whale user away from negative unit economics.
Great article! Really appreciate how you explained the concepts in a simple and structured way. This was both insightful and easy to follow. Thanks for sharing 🙌