mysummit.school research

Prompt Engineering: What Actually Works

We tested 10 prompting techniques on 4 models on real manager tasks. In short: the shape of a prompt matters more than its tone. An answer template, a role and self-critique help; capitals and rudeness don’t. And a prompt won’t turn a weaker model into a stronger one. Build your prompt →

What we found

77 comparisons in 100

An answer template beats the naive prompt this often, on average across four models (74–82 on each). A role and self-critique win 71 each.

50 and 44 comparisons in 100

Capitals and an aggressive tone: no better than the naive prompt, no significant effect. Louder is not clearer.

3.44 → 4.27 points of 5, GigaChat-Ultra

The best technique lifted the model’s overall score, yet against GPT-5.4 that answer won only 22 in 100 comparisons. A prompt raises the floor; the model sets the ceiling.

What to try tomorrow: set the shape of the answer and say who it is for. After the answer, ask the model to find its weak spots. Leave capitals and threats out – they don’t help.

What do you think works?

Three changes to the same prompt. Make your prediction, then see what the study found.

The task every version solved

We run an electronics e-commerce store with 45 employees. Last quarter revenue dropped 18%, while website traffic grew 12%. Average order value fell from $120 to $85. Returns increased from 4% to 11%. We raised the ad budget by 30%. What's going on and what should we do?

Example 1 of 3

Answer template

After the task, an answer template was added:

Respond strictly in this format: ## Diagnosis (2–3 sentences: what exactly is wrong) ## Root Causes For each cause: - What is happening - Why it is happening (cause-effect relationship) - What data supports this ## Recommendations (from most urgent to least) For each recommendation: 1. What to do (specific action) 2. Expected result (in numbers, if possible) 3. Timeline 4. Who is responsible ## What I Don't Know What information is missing for a more accurate analysis?

Your prediction

It helps. Against the naive prompt this version won 77 in 100 comparisons, on average across four models. The effect passed the significance test.

Comparisons won against the naive prompt, average of four models

By model
GigaChat-Ultra
76 in 100
GigaChat-2-Max
82 in 100
Alice AI LLM
76 in 100
Qwen3 Max
74 in 100

Example 2 of 3

ALL CAPS

The same task was written in capitals:

WE RUN AN ELECTRONICS E-COMMERCE STORE WITH 45 EMPLOYEES. LAST QUARTER REVENUE DROPPED 18%, WHILE WEBSITE TRAFFIC GREW 12%. AVERAGE ORDER VALUE FELL FROM $120 TO $85. RETURNS INCREASED FROM 4% TO 11%. WE RAISED THE AD BUDGET BY 30%.

Your prediction

It doesn’t help: it won 50 in 100 comparisons, no better than the naive prompt. No significant effect.

Comparisons won against the naive prompt, average of four models

By model
GigaChat-Ultra
47 in 100
GigaChat-2-Max
54 in 100
Alice AI LLM
56 in 100
Qwen3 Max
44 in 100

Example 3 of 3

An aggressive tone

Pressure was added before and after the task:

Look, I need a real answer. … No fluff.

Your prediction

It doesn’t help: it won 44 in 100 comparisons, no better than the naive prompt. No significant effect.

Comparisons won against the naive prompt, average of four models

By model
GigaChat-Ultra
43 in 100
GigaChat-2-Max
46 in 100
Alice AI LLM
43 in 100
Qwen3 Max
44 in 100

Each comparison: two answers to the same task, one with the change and one without; two AI judges picked the better one.

Build a prompt

The study’s task: an online shop whose revenue falls while visits grow. Switch the techniques on and see exactly which sentences each one adds. Copy the result into your chat.

What to add to the prompt

Each technique was compared with the naive prompt separately, so the numbers can’t be added up. This combination is for practice; it was not tested as a whole.

First message

Added: Role and context

You are an experienced business analyst with 15 years in e-commerce consulting. You've worked with Amazon, Shopify merchants, and dozens of mid-sized online stores. Your task is to prepare a McKinsey-level analytical memo for the CEO. The memo must include specific numbers, cause-effect relationships, and prioritized recommendations. Avoid generic phrases like "needs improvement" – write what specifically to do.

The study used a detailed role, as here. For your own task one or two lines are enough: who answers, for whom, and what the result must be. That shorter version was not tested separately.

The task

We run an electronics e-commerce store with 45 employees. Last quarter revenue dropped 18%, while website traffic grew 12%. Average order value fell from $120 to $85. Returns increased from 4% to 11%. We raised the ad budget by 30%. What's going on and what should we do?

Added: Answer template

Respond strictly in this format: ## Diagnosis (2–3 sentences: what exactly is wrong) ## Root Causes For each cause: - What is happening - Why it is happening (cause-effect relationship) - What data supports this ## Recommendations (from most urgent to least) For each recommendation: 1. What to do (specific action) 2. Expected result (in numbers, if possible) 3. Timeline 4. Who is responsible ## What I Don't Know What information is missing for a more accurate analysis?

Second message – send it after the first answer

Added: Self-critique as a second message

Re-read your answer critically. Find: 1) Where were you vague? 2) Where could there be logic errors? 3) What did you miss? Then give an improved version fixing all issues found.

Translated example; the study ran these prompts in Russian.

Role, answer template and self-critique are the first chapter of the course

Practise each habit on your own work tasks, with feedback on every assignment. The first lessons are free.

Start free

What works on your model

All ten techniques on the four study models. Pick a model to highlight its column and see where to start.

Start with the answer template and the role: both beat the naive prompt on all four models.

GigaChat-Ultra: «A worked example answer» won most often – 82 in 100 comparisons. The best overall score (4.27 of 5) came from another technique, «Answer template»: wins and overall score are two different measurements. It needs a ready model answer to show; the answer template needs no preparation and won 76 in 100 here.

GigaChat-2-Max: «A worked example answer» won most often – 89 in 100 comparisons. It also gave the best overall score, 3.63 of 5. It needs a ready model answer to show; the answer template needs no preparation and won 82 in 100 here.

Alice AI LLM: «XML tags and Markdown» won most often – 85 in 100 comparisons. The best overall score (4.46 of 5) came from another technique, «Answer template»: wins and overall score are two different measurements.

Qwen3 Max: «Step-by-step reasoning» won most often – 78 in 100 comparisons. The best overall score (4.49 of 5) came from another technique, «Answer template»: wins and overall score are two different measurements.

Comparisons won in 100 against the naive prompt. «Significant» refers to the technique as a whole, not to each model. Order for reading, not a ranking: the three habits first.
TechniqueAverageGigaChat-UltraGigaChat-2-MaxAlice AI LLMQwen3 Max
Answer templatesignificant · effort 2 of 5 77 76827674
Role and contextsignificant · effort 2 of 5 71 75757560
Self-critique as a second messagesignificant · effort 3 of 5 71 71746575
A worked example answersignificant · effort 4 of 5 78 82897565
Step-by-step reasoningsignificant · effort 3 of 5 71 68796078
XML tags and Markdownsignificant · effort 4 of 5 73 53788574
Refining over several turnsnot significant · effort 3 of 5 65 61726168
Splitting into sub-tasksnot significant · effort 4 of 5 58 79584449
ALL CAPSnot significant · effort 1 of 5 50 47545644
An aggressive tonenot significant · effort 1 of 5 44 43464344
  • Answer templatesignificant · effort 2 of 5 77 in 100 76 in 10082 in 10076 in 10074 in 100
    All models
    Average
    77 in 100
    GigaChat-Ultra
    76 in 100
    GigaChat-2-Max
    82 in 100
    Alice AI LLM
    76 in 100
    Qwen3 Max
    74 in 100
  • Role and contextsignificant · effort 2 of 5 71 in 100 75 in 10075 in 10075 in 10060 in 100
    All models
    Average
    71 in 100
    GigaChat-Ultra
    75 in 100
    GigaChat-2-Max
    75 in 100
    Alice AI LLM
    75 in 100
    Qwen3 Max
    60 in 100
  • Self-critique as a second messagesignificant · effort 3 of 5 71 in 100 71 in 10074 in 10065 in 10075 in 100
    All models
    Average
    71 in 100
    GigaChat-Ultra
    71 in 100
    GigaChat-2-Max
    74 in 100
    Alice AI LLM
    65 in 100
    Qwen3 Max
    75 in 100
  • A worked example answersignificant · effort 4 of 5 78 in 100 82 in 10089 in 10075 in 10065 in 100
    All models
    Average
    78 in 100
    GigaChat-Ultra
    82 in 100
    GigaChat-2-Max
    89 in 100
    Alice AI LLM
    75 in 100
    Qwen3 Max
    65 in 100
  • Step-by-step reasoningsignificant · effort 3 of 5 71 in 100 68 in 10079 in 10060 in 10078 in 100
    All models
    Average
    71 in 100
    GigaChat-Ultra
    68 in 100
    GigaChat-2-Max
    79 in 100
    Alice AI LLM
    60 in 100
    Qwen3 Max
    78 in 100
  • XML tags and Markdownsignificant · effort 4 of 5 73 in 100 53 in 10078 in 10085 in 10074 in 100
    All models
    Average
    73 in 100
    GigaChat-Ultra
    53 in 100
    GigaChat-2-Max
    78 in 100
    Alice AI LLM
    85 in 100
    Qwen3 Max
    74 in 100
  • Refining over several turnsnot significant · effort 3 of 5 65 in 100 61 in 10072 in 10061 in 10068 in 100
    All models
    Average
    65 in 100
    GigaChat-Ultra
    61 in 100
    GigaChat-2-Max
    72 in 100
    Alice AI LLM
    61 in 100
    Qwen3 Max
    68 in 100
  • Splitting into sub-tasksnot significant · effort 4 of 5 58 in 100 79 in 10058 in 10044 in 10049 in 100
    All models
    Average
    58 in 100
    GigaChat-Ultra
    79 in 100
    GigaChat-2-Max
    58 in 100
    Alice AI LLM
    44 in 100
    Qwen3 Max
    49 in 100
  • ALL CAPSnot significant · effort 1 of 5 50 in 100 47 in 10054 in 10056 in 10044 in 100
    All models
    Average
    50 in 100
    GigaChat-Ultra
    47 in 100
    GigaChat-2-Max
    54 in 100
    Alice AI LLM
    56 in 100
    Qwen3 Max
    44 in 100
  • An aggressive tonenot significant · effort 1 of 5 44 in 100 43 in 10046 in 10043 in 10044 in 100
    All models
    Average
    44 in 100
    GigaChat-Ultra
    43 in 100
    GigaChat-2-Max
    46 in 100
    Alice AI LLM
    43 in 100
    Qwen3 Max
    44 in 100

Comparisons won in 100 against the naive prompt. «Significant» refers to the technique as a whole, not to each model.

The model’s ceiling

How far the best technique lifted each model’s overall score, and how its best answer did against premium models.

GigaChat-Ultra naive prompt 3.44 → with «Answer template» 4.27

By five criteria
Accuracy
3.58 3.97
Actionability
3.20 4.10
Completeness
3.32 4.37
Honesty
3.05 4.77
Clarity
4.43 4.67

GigaChat-2-Max naive prompt 3.02 → with «A worked example answer» 3.63

By five criteria
Accuracy
3.10 3.55
Actionability
2.80 3.30
Completeness
2.90 3.50
Honesty
2.85 3.95
Clarity
4.10 4.30

Alice AI LLM naive prompt 3.85 → with «Answer template» 4.46

By five criteria
Accuracy
3.75 4.15
Actionability
3.77 4.32
Completeness
3.72 4.60
Honesty
3.67 4.77
Clarity
4.88 5.00

Qwen3 Max naive prompt 3.93 → with «Answer template» 4.49

By five criteria
Accuracy
3.97 4.05
Actionability
3.60 4.48
Completeness
3.77 4.60
Honesty
3.98 4.88
Clarity
4.88 5.00
  • naive prompt
  • best technique
  • Premium models in the same evaluation: GPT-5.4 – 4.38, Claude Sonnet 4.6 – 4.52, Kimi K2.5 – 4.57

Overall score, out of 5. The study did not record which prompt the premium models were given, so their band is a reference point, not a target that prompting can reach.

Comparisons won in 100 by each model’s best answer against a premium model
ModelGPT-5.4Claude Sonnet 4.6Kimi K2.5
GigaChat-Ultra22 in 1006 in 10026 in 100
GigaChat-2-Max4 in 1000 in 10014 in 100
Alice AI LLM36 in 10028 in 10031 in 100
Qwen3 Max33 in 10039 in 10041 in 100

Answers from current models

Want to see the difference yourself? Run the naive prompt and the one with an answer template on two of today’s models.

A new run, not a replay of the experiment. The study (7–18 April 2026) used GigaChat-Ultra, GigaChat-2-Max, Alice AI LLM and Qwen3 Max; here two current models answer. Qwen 3.6 Plus is not Qwen3 Max.

Each button sends two requests to paid models. Answers usually take 10–40 seconds; nothing runs until you click.

With the answer template

Show the prompt

We run an electronics e-commerce store with 45 employees. Last quarter revenue dropped 18%, while website traffic grew 12%. Average order value fell from $120 to $85. Returns increased from 4% to 11%. We raised the ad budget by 30%. What's going on and what should we do? Respond strictly in this format: ## Diagnosis (2–3 sentences: what exactly is wrong) ## Root Causes For each cause: - What is happening - Why it is happening (cause-effect relationship) - What data supports this ## Recommendations (from most urgent to least) For each recommendation: 1. What to do (specific action) 2. Expected result (in numbers, if possible) 3. Timeline 4. Who is responsible ## What I Don't Know What information is missing for a more accurate analysis?

The naive prompt

Show the prompt

We run an electronics e-commerce store with 45 employees. Last quarter revenue dropped 18%, while website traffic grew 12%. Average order value fell from $120 to $85. Returns increased from 4% to 11%. We raised the ad budget by 30%. What's going on and what should we do?

How we tested it

Each of the 10 techniques was applied to 6 manager work tasks on 4 models, 3 times per combination – 1,717 runs in total. Each answer with a technique was compared with the answer to the naive prompt; two AI judges, Claude Opus 4.6 and Gemini 3.1 Pro, picked the better one (weights 70 and 30).

Limits
  • A snapshot of April 2026: the models have been updated since.
  • The judges agreed with each other in 66–73% of comparisons, so a gap of a few points is no basis for conclusions.
  • 4 of the 10 techniques did not pass the significance test.
  • The tasks and prompts were in Russian; the English prompts here are translations.
Details for specialists

Pre-registered comparisons in three passes: 2,880 pairwise comparisons with the naive prompt; 564 scores of answers on five criteria (accuracy, actionability, completeness, honesty, clarity); 864 comparisons of the best answers with premium models (GPT-5.4, Claude Sonnet 4.6, Kimi K2.5). Significance: Fisher’s exact test with FDR correction, q < 0.05. The study’s own technique names: Structured Output, Few-shot (Reference), Role + Context, Self-Critique, Chain-of-Thought, XML + Markdown, Iterative Refinement, Decomposition, ALL CAPS, Aggressive Tone. Analysis date: 4 May 2026.

Questions

Should I ask the model to reason step by step?

It depends on the model: step-by-step reasoning won GigaChat-2-Max – 79 in 100, Qwen3 Max – 78 in 100, GigaChat-Ultra – 68 in 100, Alice AI LLM – 60 in 100 comparisons against the naive prompt. Start with an answer template instead – it beat the naive prompt on all four models.

Is it better to split a task into several messages?

Usually not. Splitting into sub-tasks gave no significant gain, and on 2 of 4 models it won fewer than half of the comparisons (Alice AI LLM – 44 in 100, Qwen3 Max – 49 in 100). Of the techniques that use several messages, only self-critique helped consistently: 71 in 100 on average.

What matters more: the technique or the model?

The model sets the ceiling. The best technique lifted GigaChat-Ultra to 4.27 of 5, yet that answer won only 22 in 100 comparisons against GPT-5.4. Techniques cost nothing and are worth using – the answer template won 77 in 100 on average – but if you need premium-level answers, start by choosing the model.

Put the habits to work

Set the shape of the answer. Say who it is for. After the answer, ask the model to check its weak spots.

More of our research

On the blog

The first 9 lessons – free