On June 17, a developer posting as Tech with Mak dropped a nine-folder layout for a production AI architecture and called it "the most honest breakdown I've seen." The post crossed 55.4K views. I went in skeptical of the superlative and came out mostly agreeing, though not for the reason the thread leads with.
The honest part isn't the folder count. It's that the layout refuses to let you ship without measuring whether the thing actually works.
What's in the tree
The services/ folder splits the RAG pipeline, semantic cache, memory, query rewriter, and router into five files instead of one. agents/ holds a document grader, a decomposer, and an adaptive router, described as self-correcting by design. prompts/ are versioned, typed, and registered rather than hardcoded into call sites. security/ runs three guards (input, content, and output), not a single filter at the door.
None of that is exotic. Anyone who has pushed an LLM feature past a demo has felt the pain each folder addresses. Hardcoded prompts rot. A single content filter misses either the injection going in or the leak coming out. Splitting them is just admitting where the failures actually happen.
The layer everyone skips
evaluation/ is the one that earns the "honest" label. It holds a golden dataset, an offline eval, and an online monitor, and the thread is blunt that most people skip this entire layer and ship blind.
That's the whole argument, and it's correct. The difference between a demo and a production AI architecture is not the RAG pipeline or the semantic cache. Those make the demo feel good. It's whether you can tell, after a prompt change or a model swap, that quality went up instead of down. Without a golden dataset you are guessing, and most teams guess.
Observability, and the folder that's new
observability/ is the natural partner: per-stage tracing, feedback linked back to those traces, and cost tracked per query. Cost per query is the line I'd underline. Multi-step agent systems hide their spend across retries and tool calls, and "per query" is the only granularity a finance conversation respects.
The genuinely current idea is the .claude/ folder: agent context so your AI coding assistant knows the codebase before it touches a file. This part didn't exist in anyone's architecture diagram a year ago. Treating your AI coding assistant's working context as a committed, versioned part of the repo, not a setting buried in someone's editor, is where this layout points past the usual best-practices list.
The honesty is real, the novelty is oversold
Here's where I push back. Every individual layer is established practice, and assembling them into one tree is a useful synthesis of things teams already half-do. The post's framing ("Production is this") implies a single correct shape. There isn't one. A small team with one RAG endpoint does not need five service files on day one, and pretending otherwise is how you get people who own an evaluation harness and zero users.
What's worth noting is what gets attention here. The same week this careful breakdown circulated, the AI thread pulling far bigger numbers was a list of ten GitHub repos that scrape the entire internet, one of them past 130K stars. [1] The boring layers (evaluation, observability, security) are the ones that keep a system alive in production, and they are reliably the least viral.
If you're shipping LLM features this quarter
Don't adopt nine folders because a tweet did. Adopt the order of operations the tree implies. Build the evaluation layer first, with a golden dataset of the cases you actually care about, before you tune the semantic cache or add a third agent. Wire cost-per-query into your traces while the bill is still small enough to ignore, because that's exactly when nobody bothers. And commit your AI coding assistant's context to the repo so the next person, or the next agent, starts from what you already learned.
The folder count will change. The lesson under it won't: if you can't measure your AI system, you don't really run it. You're just hoping it holds.
Sources
Primary source: Tech with Mak (@techNmak) on X
- Tweet by @aiwithkhush — @aiwithkhush · community
- Tweet by @s2_underground — @s2_underground · community
- Tweet by @Gemini090222 — @Gemini090222 · community