Open Textbook

mysummit.school - AI for Managers Blog

Your AI agent did exactly what you asked. That was the problem.

16 min read

The board of a drilling company has told every division to deliver a successful generative AI project this year. The drilling division reports to a vice-president. By the November board meeting he wants two services that are easy to show: a daily email summary of the wells, and a chat assistant called Ask your well that takes questions about drilling reports in plain language. He has ruled one piece of work out of scope: tagging the causes of downtime. That, he says, is already automated. He regards it as solved and sees no reason to spend an AI project on it.

The company is losing money precisely because of errors in that tagging. Every month it claims compensation from the drilling contractor for downtime. The claim is prepared by an old script. It searches the daily reports for any mention of a repair or a failure and concludes that the contractor caused the downtime, even when the equipment belongs to the service company or the work was planned. The contractor disputes such lines, and a single disputed line is enough to hold back the whole month’s compensation. It gets held back every month.

Carry out the request exactly and the project ships on time and passes every functional test. The compensation still never arrives. To be useful, the engineer would have to open the very system the vice-president ruled out, find the contract clause that governs the dispute, get its meaning confirmed by the cost engineer, and agree the change with someone who is away this week.

This is one scenario from the FDE-Gym study. Thirteen models from seven vendors did the work of forward-deployed engineers: 95 tasks built from real projects across 18 industries. Each result was scored the way a client would score it, by how far the business metric moved. Even the best model, Claude Opus 5.5, reached a little over half of what the experts achieved on average.

Below I go through five problems the agents ran into, and what a manager can do about each one: when writing the brief, when accepting the work and when choosing a model. The agents in the study were deploying systems at a client site, but I suspect the same problems appear when you hand an agent a report, a contract or an analysis inside your own team.

Read more
Your AI agent did exactly what you asked. That was the problem.
An engineer says AI makes them slower. How to check
13 min

An engineer says AI makes them slower. How to check

A manager hands the team a new tool. A month later one of the engineers comes back and says: with AI I am slower than doing it by hand. From there one of two things usually happens. The manager decides the person is resisting and starts pushing. Or the manager decides the tool is the wrong one and shelves the rollout.

Both reactions are guesses. And the awkward part is that the engineer is probably telling the truth, while the tool is probably fine.

Confident, plausible, wrong: checking an AI report
12 min

Confident, plausible, wrong: checking an AI report

I recently read a write-up about a manager who keeps a document with an odd name in his working folder: a correction log. It sits right inside an analytical report – a table headed Claims retracted during the work, with the reason for dropping each one. In the next folder, another report opens with a warning: three claims in earlier versions of this document were wrong and have been withdrawn.

An AI agent wrote both reports. The most valuable part of each is the list of what got taken out.

Coding Agents in 2026: Two Form Factors – Terminal and Desktop
28 min

Coding Agents in 2026: Two Form Factors – Terminal and Desktop

By September 2026, every model vendor had shipped a coding agent: Anthropic, OpenAI, Google, Microsoft, xAI, DeepSeek, Moonshot, Alibaba, Z.ai, Tencent. By mid-year, Claude Code, Cursor, Codex and Antigravity had converged on the same agentic loop: the program reads files, edits them, runs commands, checks the result and reports back. What still differs is packaging, pricing and ecosystem – which is why an alphabetical list of a dozen names tells you nothing.

The split that actually helps is form factor. The terminal is home to agents for engineers: Claude Code, Codex CLI, Antigravity CLI, Grok Build, Kimi Code CLI, Qwen Code. Managers don’t work there, but they pay for these tools – and they need to understand what they are paying for. The desktop is home to agents for office work: Claude Cowork, Copilot Cowork, Kimi Work, Tencent WorkBuddy. These are built for the manager and their team: HR, operations, admin, projects, sales. Sort files, assemble a report, process receipts, clean up spreadsheets.

Microsoft MAI Models in 2026: What You Can Actually Use, and What It Costs
18 min

Microsoft MAI Models in 2026: What You Can Actually Use, and What It Costs

At Build 2026 on June 2, Microsoft unveiled seven in-house models under the MAI (Microsoft AI) brand, covering reasoning, code, images, voice and transcription. A company that spent years as OpenAI’s biggest backer called it a move toward “long-term self-sufficiency.”

Microsoft AI main page

For a manager, two things in this announcement matter. The first is price: an hour of recorded meeting gets transcribed for $0.10, and you don’t need an Azure contract to do it. The second is Frontier Tuning – training a model on your own workflows inside your own compliance boundary. It’s the strongest idea Microsoft has shipped all year, and it comes only with the full Microsoft enterprise stack.

So let’s start with who can buy what, then get to the models and the numbers.

Meta Muse Spark: Meta's First Closed Model Since 2023
12 min

Meta Muse Spark: Meta's First Closed Model Since 2023

Until today, this series had no chapter on Meta. While Llama 4 was the headline, the silence was justified: the model landed to mixed reviews, and there was nothing to tell a manager about it as a working tool.

On July 9, 2026, Meta released Muse Spark 1.1 and, the same day, opened public access to the Meta Model API. The announcement went out on the company blog. The Muse family itself, which replaces Llama, started three months earlier: version 1.0 shipped on April 8, and the current model is built on top of it. In the September LMArena snapshot, muse-spark-1.2 in xHigh mode sits fifth on the text leaderboard, muse-spark-1.1 tenth, and muse-spark is sixth on the image leaderboard. Anyone comparing tools by leaderboard will run into the name. Until now, this series had nothing to say about it.

The scores are only the third most important story here. The first is the licence: in nine months Meta replaced its AI leadership, rebuilt the stack and came back with closed weights. The second is what kind of model it turned out to be: 1.1 leads on tool-calling tasks and trails noticeably on writing code.

AI at work: adoption keeps rising, the returns stay flat
22 min

AI at work: adoption keeps rising, the returns stay flat

Over the past year this blog has worked through more than a dozen large studies of AI at work: Gallup, Stanford, Wharton, BCG, Workday, Epoch AI, Google, Microsoft. Each report tells its own story, and we tested each story on its own terms. Stitched into one timeline, they show something no single report contains: two curves that have spent the year heading in different directions.

The first is adoption. It climbs in a straight line, and predictably: Gallup’s quarterly measurements put daily AI users among working Americans at 15%, up from 8% a year earlier, while Stanford’s AI Index has corporate adoption at 88%. The second curve is return, and it is flat. Workday estimates that 14% of employees get a net gain from AI once the rework is subtracted, and in a year of watching, that number has not moved.

Below are both curves, built from the primary sources we tracked all year, with notes on what can and cannot be compared. Behind them sit four cuts that appear only in the aggregate: the mix of tasks, trust and error, the effect on business, and geography.

Why Teams Use AI for the Wrong Tasks: Gallup Data
15 min

Why Teams Use AI for the Wrong Tasks: Gallup Data

On 20 July, Gallup published its regular quarterly reading on AI adoption inside American companies. The headline is cheerful: 47% of employees say their organisation has integrated AI tools, up six percentage points in a single quarter. Until now, organisational adoption had lagged well behind individual use. Individual use keeps climbing as well: 52% of workers apply AI in their role, and 15% do it every day.

Then comes the table that sits awkwardly with all this optimism. The most common uses are writing and editing text (51% of AI users), search and research (49%), and general help with tasks (39%). Ask the same people where AI actually improved their productivity and the order flips. Coding help and process automation score highest at 77%, followed by presentations (76%) and analytics (75%). Text (68%) and search (65%) end up at the bottom.

So teams pile into exactly the uses that pay back least. The question worth asking is how much weight these numbers can carry, and what a project or product lead should take from them.

What Never to Hand to AI: Why Checking Quietly Stops Working
16 min

What Never to Hand to AI: Why Checking Quietly Stops Working

In 2025, the plaintiffs’ lawyers in a case against Walmart filed a motion citing eight court decisions that did not exist. One of them had used an AI tool, the tool invented the citations, and the rest of the team passed the document on without looking. Everyone, it seems, assumed somebody else had checked.

The hallucination is the least interesting part; there is plenty written about those. What matters is that the document went through several qualified lawyers and not one of them slowed down. The check existed on paper. At no point did it happen.

It is a fairly precise model of what is happening now in teams where AI has taken hold. Speed went up, the output still looked respectable, and the checking function quietly stopped working – in a way that the usual metrics will never show you.

Agent orchestration: the AI skill with the longest shelf life
14 min

Agent orchestration: the AI skill with the longest shelf life

Picture a manager who is quite sure they have AI figured out. Thirty browser tabs are open. In one they spent half an hour pulling apart the risks in a contract; in another they were assembling a project status; in a third they asked for an email to a stakeholder to be rewritten. Every new task starts the same way: explain the project again, who is on the team, what the deadlines are, paste in a chunk of a Jira export, drop in a Slack thread. An answer comes back, gets copied out, the tab closes. Tomorrow, from the top.

They use AI every single day, diligently, and believe they are at the frontier. In reality they are stuck on the first rung, hauling context around by hand like a courier hauling boxes. The tool is powerful. The way they work with it is the most primitive option available.

What separates them from someone who genuinely gets value out of AI is not typing speed. It comes down to one skill that almost nobody teaches explicitly: delegating a task to an agent rather than chatting about it. Oddly enough, that is the longest-lived skill in the whole of AI training. It is also the one most often skipped.