An engineer says AI makes them slower. How to check
AI models in this article
A manager hands the team a new tool. A month later one of the engineers comes back and says: with AI I am slower than doing it by hand. From there one of two things usually happens. The manager decides the person is resisting and starts pushing. Or the manager decides the tool is the wrong one and shelves the rollout.
Both reactions are guesses. And the awkward part is that the engineer is probably telling the truth, while the tool is probably fine.
Alexander Orlov and Vyacheslav Pankratov, founders of the management school Stratoplan, worked through this fork in an episode of Mikhail Omelchenko’s podcast, We bought AI and the team won’t use it (in Russian). They have spent more than sixteen years teaching team leads and directors, with 600–700 people a year coming through their open programmes, so they have heard a fair sample of rollout complaints. Their answer is uncomfortable for both sides: one person’s account cannot settle anything on its own.
Why one employee is not enough
The employee picks up the new tool and decides where to try it. The choice is almost always the same: this I know how to do by hand, that I don’t – let me try the AI on it. Perfectly human logic. Except the test now contains two new variables at once: an unfamiliar tool and an unfamiliar kind of task. It came out slower, and neither of you can say which of the two caused it.
To avoid that trap, the employee would have to work like a careful experimenter: keep a baseline speed for this kind of task in their head, reproduce the conditions, separate the effect of the tool from the effect of novelty. People like that are rare, and you did not hire them for it.
Now the overhead. The podcast deals with the lying-or-not question through an example that contains no AI at all. Take a strong developer who finishes a task in three hours. Give them a junior to help. Eight hours later the task may still be unfinished, and they are five times more exhausted, because explaining it to a junior takes fifteen times longer. You gave them more resource and they got slower. Not one word they told you was a lie.
With AI the arithmetic is the same. Generating text or code got faster; reviewing it, checking it and rewriting it did not. On some tasks the balance still does not favour the tool, and saying so is more honest than explaining it away as resistance.
First, look at how you introduced it
Before you get into the measurement itself, look at the history.
Two scenarios. In the first, a month ago you stood up and said: from now on we work with this. In the second, the employee came to you: I found a tool, I want to try it. A month later both arrive with the same sentence – it is slower – and the two sentences should carry different weight.
The mechanism is simple: responsibility grows out of a voluntary choice. Someone who picked the tool themselves answers for it, and a negative report costs them something, because it means admitting their own initiative was a mistake. When that person says “slower”, it is data. When the person who was handed the tool says it, it may be data, or it may be a work-to-rule: we will use it for a month and you will see for yourself that it is rubbish.
Hence a practical conclusion worth reaching before any investigation. If the tool came from above and the team never agreed that the problem existed, there is not much of a case against the people. You brought them a solution to a problem they did not have. We wrote about choosing the patch where AI genuinely removes pain in where to point AI first (in Russian).
Working out how much truth is in whose words does not require reading minds, if you set the test up so that it answers by itself. Building that test is harder than it sounds: rollout measurements break all the time on measuring the wrong thing and comparing against the wrong baseline.
Telling the tool's overhead apart from the wrong task is a skill, not intuition. Test yourself on 9 real management tasks, free.
No payment required • Get notified on launch
A design that answers instead of guessing
The test works like any technology experiment. You take two or three groups solving roughly the same task. One has the tool, the other does not, and the task is the same. Then you look. If there is a difference, it shows up in the comparison between groups, rather than in one employee’s memory of how they worked in March.
The collusion objection is easy to deal with. Six people colluding against their manager means you hired six people and built a relationship in which they jointly decided to deceive you on this particular point. It happens, but it is a separate problem, and replacing the tool will not solve it.
What to fix before the start:
- One kind of task. A specific recurring piece of work: triaging inbound requests, preparing a status report, reviewing requirements. A month of work has nothing to compare against.
- The same input. Groups draw from one queue, rather than hard cases going to the experienced people.
- A deadline named in advance. Two to three weeks. A deadline you move as you go turns the test into a search for confirmation.
- An outcome metric. How many requests were handled and how many came back for rework. Token spend will not do: it measures load on the tool and says nothing about work done.
- A failure condition. What exactly you will see that lets you say “this doesn’t work here”. Without it, the experiment ends with the rollout extended by another month.
Most measurements break on the last two points. The podcast tells the story of Facebook and several other large companies: committees were set up for the rollout, KPIs introduced and, according to the guests, it went as far as leaderboards for who spent the most tokens. A few months later the annual token budget was gone, and when someone looked at what it had been spent on, the list included ordering pizza through corporate tokens. The employees’ logic was impeccable: you set the engineer an engineering task, spend tokens, and they closed it. In half an hour, so they could get back to work.
One more thing worth knowing in advance: being observed changes the result. On a factory floor people work better once a camera goes up on the wall, even when the camera is a dummy with no wires. So the group you watch will speed up from the attention alone. That is what the control group is for: you watch it exactly as closely, only without the tool, and subtract the effect.
The champion is worth more than the average
While the test runs, a second source of data appears, and it is often worth more than the final numbers.
In any group someone does noticeably better than the rest. Find that person and work out what they do differently: which tasks they hand to the tool, which they keep, how they phrase the request, where they check. At Stratoplan, NotebookLM spread exactly this way – one of the partners started producing decks in it that looked noticeably better than what he made by hand, the second saw the result, asked for a demonstration and began trying it himself.
Notice what happened there. Nobody rolled anything out. A result was shown, a colleague got interested, and then came a voluntary choice and, with it, responsibility. This is the very mechanism that makes a self-brought tool more credible. We have a separate piece on how teams hide their real AI use from managers – the same fork, seen from the other side.
Build the test around your own situation
Designing a test like this is exactly the task where a model helps: break the situation into variables, name what you missed, and produce a table you can take to the team. Below it runs on the two models that are strongest at planning in our benchmark: Kimi K3 and GPT-5.6 Sol. Run it as it is, or swap in your own description.
Look at two places in the answer. First, the list of mixed variables: if the model names something you missed, that is the value of the run. Second, the metrics that are harmful to measure. That is usually where token spend, time in the interface and the share of employees who “tried the tool” turn up – everything that is easy to count and says nothing.
Same situation, different models, different test designs. Working out which answer to act on is what the 9 tasks from a manager's practice are for, free.
No payment required • Get notified on launch
What this test will not tell you
It will not tell you whether the team had a problem in the first place. If the tool speeds up drafting and drafting was never the bottleneck, no control group will fix that – you will get a carefully measured absence of effect. The podcast puts it this way: they brought a saw to people who had nothing to saw. That should have been checked earlier, when the problem was being agreed. What happens when a whole industry skips that check is visible in the annual adoption surveys: the share of people using AI keeps rising while measurable returns stay put.
It will not tell you whether the process itself should change either. Mandating AI in high-volume processes – support, inbox triage, mass servicing – is reasonable enough: a system already exists and you are swapping one node in it. Why a blanket everyone-uses-AI mandate breaks against a team that splits into thirds is a separate article.
With an engineer it does not work that way. You can either add a tool to an unchanged process and get exactly what we started with, or change the work itself: they no longer write code line by line, they set the context and the acceptance criteria. The second is a business-level decision, and it has a price: a strong developer turns into a commissioning editor who reads tonnes of generated text to ship the same amount they would have written by hand. Which parts of managerial work should never go to a model at all is a separate question.
And finally, the test does not replace the conversation. Someone can be slower for reasons that have nothing to do with the tool: overload, burnout, a divorce at home. They honestly linked two events – AI arrived, work got slower – and arrived at a causal story where there was only coincidence. That shows up in a one-to-one and in no table.
What to do this week
If such a complaint is sitting on your desk right now, the sequence is short.
Recall how the tool reached the team: handed down or brought in. Ask exactly which tasks they tried it on, and check whether it was also an unfamiliar kind of work. Find the person in the team who does best and ask them to show how. And before rolling the tool out to the next departments, run a test for two or three weeks, with a control group and a failure condition named in advance. If the rollout is going ahead anyway, we put the order of steps in a step-by-step implementation plan (in Russian).
“The tool doesn’t work” is worth exactly as much as “the employee is sabotaging” until there is a comparison behind it. The difference between a manager who guesses and a manager who knows is one control group.
From guesswork to measurement
The Foundation chapter of the course covers where AI genuinely removes managerial work and where it adds overhead: the hand-to-AI / do-it-together / do-it-yourself matrix, a check for any model answer, and practice on the tasks you handle every week.
Часто задаваемые вопросы
An employee says AI makes them slower. Are they lying?
How do you check whether AI helps the team?
Why can't you measure AI adoption by tokens spent?
What if the employee brought the tool themselves and says it got slower?
Stanislav Belyaev
Engineering Leader at Microsoft18 years leading engineering teams. Founder of mysummit.school. 700+ graduates at Yandex Practicum and Stratoplan.