Buying guides
Deployment is the scarce skill
- On 2 October Anthropic committed $100 million to train 10,000 engineers who deploy Claude inside companies. A model maker is paying to create deployment skill.
- In a September benchmark, the best AI setup passed 23.9% of simulated customer conversations when asked to build and deploy a customer-service agent. A reference built with the answers in hand passed 82.2%.
- The authors trace the gap to deployment work, such as reading the records and asking the client. Switching to a weaker model cost far fewer points.
- Before you ask which model a builder uses, ask who will deploy and maintain the system, and how your own team will take it over.
We think the model is now the easy part of an AI system to buy, and the scarce part is the engineers who can deploy it into real operations and keep it working. That changes the first question a founder should put to any builder. It is not which model, but who will deploy and maintain the system and how your team will take it over.
A model maker is paying to train deployment engineers
On 2 October, Anthropic launched Claude Frontier Academy, a $100 million commitment to train 10,000 engineers it calls Frontier Deployed Engineers by the end of 2027. Anthropic's own claim is that people who can make AI work inside a real business have become "the hardest talent to find."
The program shows what that skill is. Companies nominate their strongest engineers, and each brings a specific Claude project to lead back home. They practice on a simulated company deployment that runs from picking the use case to security review and handover. Then they lead a real use case at their own company and are assessed again. The first cohorts include engineers from Accenture, Bain, Capgemini, Deloitte, McKinsey and Morgan Stanley.
As Anthropic describes it, the program is about choosing the use case, getting through security review and handing over a working system. The lab is paying to build that skill inside its customers and their consultancies.
The supply is thin. Gartner analyst Mukul Saha told Channel Dive there are about 2,000 active forward deployed engineers, or FDEs, the engineers who work inside a client's operations until a system runs there. He put demand at "probably quadruple that."
A benchmark that measures deployment, not coding
A benchmark posted to arXiv on 4 September, which its authors pronounce hyper-tau-bench, measures how far AI is from doing this work itself. It is a preprint and has not been peer reviewed.
The benchmark gives a coding agent what a real engagement starts with: the business's own records, a client who knows the requirements, the live API it must work through, an existing codebase and limits on running cost and on which models it may use. The agent must deliver a working customer-service agent, which is then scored against simulated users it has never seen.
Swapping OpenAI's or Anthropic's top model for its second-tier model cost 4 to 9 points, and the two vendors' strongest setups finished 1.9 points apart. The distance to the reference was about 58 points. In this benchmark, the choice of model moved the score far less than the work the agents skipped.
The failures read like a job description
Most of what went wrong was work other than coding. The developer agents searched the business's records by keyword instead of reading them, rarely asked the client anything and patched the first design that ran instead of trying others. They checked their work with tests they wrote themselves, which shared their own blind spots.
The finding supports our case, in if you cannot grade it, do not automate it, for writing graded examples from real work before any build starts.
Where the builds went wrong
Measured in a preprint
0.3%
of the developer agents' tool calls went to talking to the client
Measured in a preprint
0.16 vs 0.50
average score where requirements lived only with the client, for builds that never asked compared with builds that asked four or more questions
Measured in a preprint
31% to 67%
score in a controlled telecom test after the builder got a one-line design hint, to route by intent and review tool calls
Each configuration built one agent per task, so these are single runs, not averages over repeats.
Reading the records, interviewing the people who hold the requirements, comparing designs and testing against real traffic are what a deployment engineer does. A buyer should ask who will do each of them, because a model upgrade will not.
The case for outcome deals without engineers
Not everyone agrees that deployment needs more engineers. On Salesforce's 26 August earnings call, Marc Benioff, its chief executive, described outcome deals in which the vendor takes a share of the revenue it adds. For those, he said, "you don't need a bunch of FDEs and engineers," because that layer is what the AI is good for. Salesforce's results that day put Agentforce annual recurring revenue above $1.5 billion.
The same company made the case for FDEs nine months earlier. Its blog said in November 2025 that the company had committed to building a team of 1,000 FDEs, and its FDE director warned that without them, thousands of customers could be "stuck in pilot purgatory."
We think both statements can be true. An outcome deal moves deployment work inside the vendor's price. It does not remove it. Someone still measures the baseline the revenue share is paid on, connects the systems and fixes the agent when the process changes. The buyer's question survives the pricing model. Who does that work, and what happens to it when the contract ends?
That last part is where Gartner sees the risk. On 29 September it predicted that by 2028, 70% of enterprises will abandon agentic AI built by vendor forward deployed engineering, "trapped by soaring costs and unable to evolve it on their own." Its advice is to plan the exit at the start and to judge success by whether the enterprise can run and change the system itself.
Questions to ask before anyone builds
These questions come after the wider vetting in how to choose an AI agency, which covers proof, security and ownership. They are about who deploys the system and who keeps it working.
Six questions about deployment and upkeep
Who reads the records before anything is built?
Ask how the builder will learn the process from your documents, transcripts and spreadsheets, and how they will show you what they learned before they build.
Who interviews the people who hold the requirements?
Requirements that live in people's heads come out only under questioning. Ask for a named person and regular sessions with your process owner.
How many designs get compared before one is chosen?
Ask to see the designs that lost and the numbers that decided it.
What is the system tested against, and who writes the tests?
Ask for cases drawn from your real traffic, held back from the build and rerun after every change.
Who maintains it when the model, the process or the data changes?
Ask for a named person, how changes are tested before they go live and who decides when a change ships.
How will your team prove it can run the system without the builder?
Agree on that test at the start and run it before the engagement ends.
Where we land
We expect models to keep getting cheaper and closer to each other. The engineers who can turn one into a running system, and the habit of handing that skill to the client's own team, will stay scarce for longer. Ask who deploys and who maintains before you ask which model.
Our engagements start with one workflow run on your real cases and a scorecard of accuracy and cost per outcome, so the deployment question is answered with numbers before anything scales.
Send the six questions to each builder on your shortlist before the first call, and compare who answers each one with a named person and a test.
Sources
- Anthropic, Claude Frontier Academy: $100M to train 10,000 engineers, 2 October 2026
- Quan Shi, Keshav Dhandhania, Karthik Narasimhan and Victor Barres, τ^τ-Bench: An Environment for End-To-End, Realistic Agent Construction, arXiv preprint, 4 September 2026
- Salesforce, second quarter fiscal 2027 results and earnings call transcript, 26 August 2026
- Salesforce, Today's hottest role: forward deployed engineer, November 2025
- Gartner, Gartner predicts 70% of enterprises will abandon agentic AI built by vendor forward-deployed engineering by 2028, 29 September 2026
- Channel Dive, Are forward deployed engineers flirting with failure?, 30 September 2026