# AI in engineering: which tools teams adopted when | Miyagami > An interactive whitepaper by Bo Wesdorp on four years of AI in engineering: from Copilot to chat, context files, agent harnesses and loops, and what each changed. Source: https://www.miyagami.com/resources/whitepapers/how-ai-changed-engineering Whitepaper · 14 pages · by Bo Wesdorp # How AI changed engineering Since 2022 engineering teams have adopted AI tools in six waves, from autocomplete in the editor to agents that run on their own. This page follows that journey: which tools arrived when, how each one changed the daily work, and what teams that got the most out of them did differently. Follow the journey 1. 1 Completion 2. 2 Chat 3. 3 Context 4. 4 Harness 5. 5 Loop 6. 6 Verification 7. → What to do Introduction Bo Wesdorp **Bo Wesdorp** Co-founder and CTO, Miyagami I've rebuilt the way our engineers work four times since 2022. Each time something arrived that removed a constraint, and within a few months it was obvious the work hadn't disappeared. It had moved. Most teams I speak to bought the tools in that order and changed nothing else. They have Copilot seats from 2023 and agent budgets from this year, running against a review process, a definition of done and a headcount plan that all date from when writing code was the expensive part. On our own builds 80 to 90 percent of production code now comes out of a model. That on its own made us no faster. Rebuilding everything around it is what took ten engineers down to four. **80 to 90%** of production code on our builds comes out of a model **10 to 4** engineers on a project that needed ten in 2024 **6** waves of AI tools in four years The adoption journey ## Four years of AI tools in engineering Most teams bought these tools in this order and changed nothing else around them. Pick a wave to see what teams adopted, how the work changed and what it left for the next wave. The stack builds up as you go. 2022 to 2023Completion *GitHub Copilot* *T**Tabnine* *Amazon CodeWhisperer* *JetBrains full-line completion* *K**Kite* 2023 to 2024Chat *ChatGPT* *Claude* *GitHub Copilot Chat* *Cursor* 2024 to 2025Context *Model Context Protocol* *CLAUDE.md* *A**AGENTS.md* *.cursorrules* *A**Aider* Early 2026Harness *Claude Code* *OpenAI Codex* *LangChain Deep Agents* *L**Langfuse* *Braintrust* Mid 2026Loop *Claude Code /loop* *Codex goals* *GitHub Copilot coding agent* *O**OpenClaw* *GitHub Actions* NowVerification *CodeRabbit* *Greptile* *Qodo* *Graphite* *S**Semgrep* Now · Wave 6 ### Verification What teams adopted : Teams add AI review (CodeRabbit, Greptile, Qodo, Graphite), scanning for what the agent reached (Semgrep, Snyk, Socket) and tools that show the review queue (Faros, LinearB). The teams moving fastest also change the order of the work. How the work changed : The spec and the acceptance tests are written before the agent starts, so most changes are checked automatically and people read only the risky ones. Left for the next wave : Writing the spec and the tests, once, before the agent runs Read the chapter 01 2022 to 2023 · Completion Wave 1 of 6 ## Completion took typing off the job What teams adopted Teams bought GitHub Copilot seats. Completion sat inside the editor and asked nobody to change how they worked, which is why it spread so fast. How the work changed Engineers stopped typing boilerplate and started accepting suggestions with Tab. GitHub Copilot put a model in the editor and finished the line you were typing. It took boilerplate off the job, and nobody misses that work. It also made the wrong approach exactly as fast to write as the right one. The constraint moved from how fast someone could type to whether they knew what to type. Where the focus is Knowing the right approach before the suggestion appears. A fast wrong answer is still a wrong answer. Solved : Boilerplate and the next line of code Left for the next wave : Deciding what to type The tools of this wave [**GitHub Copilot**](https://github.com/features/copilot) [*T* **Tabnine**](https://www.tabnine.com/) [**Amazon CodeWhisperer** now Amazon Q Developer](https://aws.amazon.com/q/developer/) [**JetBrains full-line completion**](https://www.jetbrains.com/help/idea/full-line-code-completion.html) *K* **Kite** shut down in 2022, before the models were good enough Do we test for this? Partly We test what completion can't do for you Nobody is scored on typing speed or on how well they use autocomplete. The fundamentals assessment runs with every AI tool switched off, because an engineer who can't reason about a system without suggestions can't tell when a suggestion is wrong. [Technical understandingFundamentals scorecard](https://www.miyagami.com/how-we-vet#fundamentals)[DebuggingFundamentals scorecard](https://www.miyagami.com/how-we-vet#fundamentals) Hire for this [Full-stack engineers](https://www.miyagami.com/hire/full-stack-engineers) [Frontend engineers](https://www.miyagami.com/hire/frontend-engineers) 02 2023 to 2024 · Chat Wave 2 of 6 ## Chat turned a description into a block of code What teams adopted Engineers kept ChatGPT or Claude open next to the editor and pasted code back and forth. GitHub Copilot Chat brought it into the IDE, and Cursor ended the era by letting the model read the whole repository. How the work changed The work started with describing the problem in words instead of writing the code. ChatGPT and Claude turned a problem description into a block of code. For a mid-level engineer this was the biggest single jump of the whole timeline. Neither could see your system. A year of prompt engineering followed, which was a real skill applied to the wrong layer. Chat also taught the engineers who read the explanation and sped up the ones who didn't. Where the focus is Reading the explanation, not only the code, and noticing when the answer was written for a system that isn't yours. Solved : Turning a described problem into working code Left for the next wave : Getting the model to see your system [**19% slower** Experienced developers in a 2025 trial were 19 percent slower with AI tools, while believing they were 20 percent faster. METR, July 2025](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/) The tools of this wave [**ChatGPT**](https://chatgpt.com/) [**Claude**](https://claude.ai/) [**GitHub Copilot Chat**](https://github.com/features/copilot) [**Cursor** ended the era by letting the model read the whole repository](https://cursor.com/) Do we test for this? Yes We test whether they can explain a change they didn't type In the AI-native assessment the engineer has their own tools running, and we ask why the agent chose its approach. "The agent did it" isn't an explanation we accept, and an engineer who re-prompts the same vague ask instead of fixing the input scores low. [AI output verificationAI-native scorecard](https://www.miyagami.com/how-we-vet#ai-native)[Prompt and context engineeringAI-native scorecard](https://www.miyagami.com/how-we-vet#ai-native) Hire for this [AI-native engineers](https://www.miyagami.com/hire/ai-native-engineers) [Full-stack engineers](https://www.miyagami.com/hire/full-stack-engineers) 03 2024 to 2025 · Context Wave 3 of 6 ## Context made what you feed the model the product What teams adopted The Model Context Protocol arrived in late 2024. Teams added CLAUDE.md, AGENTS.md and .cursorrules files to their repositories, and Aider, Cline and Windsurf brought agentic editing into the editor. How the work changed Engineers started writing for the model: context files in the repository and live connections to tickets, schemas and error logs. This is where the industry stopped treating the model as the product and started treating what you feed it as the product. It is also the layer most teams are still on, whatever they believe about themselves. What nobody warns you about is decay. The file that was accurate in March describes a system that stopped existing in September, and a stale context file is worse than none: it makes the agent confidently wrong in a way that passes review. We treat updating it as part of the definition of done. **Context files** CLAUDE.md, AGENTS.md and .cursorrules hold what a new engineer would need to know, written down where the tool reads it on every run. **Model Context Protocol** Published by Anthropic in late 2024, it is one standard for giving a model access to things that aren't files: the ticket tracker, the schema, the error log. Where the focus is Keeping context files current. A stale CLAUDE.md produces confident work that passes review. Solved : The model can read your repository, your tickets and your schema Left for the next wave : Deciding what the model is allowed to do The tools of this wave [**Model Context Protocol**](https://modelcontextprotocol.io/) [**CLAUDE.md**](https://code.claude.com/docs/en/memory) [*A* **AGENTS.md**](https://agents.md/) [**.cursorrules**](https://cursor.com/docs/context/rules) [*A* **Aider** mapped repositories larger than the context window](https://aider.chat/) [**Cline**](https://cline.bot/) [**Windsurf**](https://windsurf.com/) Do we test for this? Yes We score the context they set up, and what they leave behind Prompt and context engineering is one of the six AI-native criteria: does the setup produce usable output in one or two passes? We also look for leverage, such as a tighter CLAUDE.md or a reusable command that makes the next ticket easier. [Prompt and context engineeringAI-native scorecard](https://www.miyagami.com/how-we-vet#ai-native)[Spec-driven developmentAI-native scorecard](https://www.miyagami.com/how-we-vet#ai-native) Hire for this [AI-native engineers](https://www.miyagami.com/hire/ai-native-engineers) [LLM integration engineers](https://www.miyagami.com/hire/llm-integration-engineers) 04 Early 2026 · Harness Wave 4 of 6 ## The harness: an agent is a model plus everything around it What teams adopted Claude Code with hooks, skills and subagents, OpenAI Codex and LangChain Deep Agents turned the model into an agent. Langfuse and Braintrust showed what the agent actually did. How the work changed The agent ran the tests, read the failure, fixed it and opened the pull request. Engineers configured what it was allowed to reach. The vocabulary arrived in February 2026. Mitchell Hashimoto used the phrase harness engineering, and within weeks it was a discipline built on one line: an agent is a model plus a harness. The model is the part you don't control and can swap. The harness is everything else: the loop that runs it, the tools it can call, how its context is compacted as a task runs long, the sandbox it executes in, the permission boundaries, the hooks that fire before and after an action, the subagents it can spawn, and the observability that tells you afterwards what it did. That explained why two companies with the same model and the same licence get completely different results. Security changed shape here. A harness is a system holding credentials and tool access with a model deciding what to call, so what you review is no longer only the code. It is also what the agent was permitted to reach. Where the focus is The decisions in the setup. Which tools the agent can call, where the sandbox boundary sits and what a hook blocks now apply to everything the team ships. Solved : An agent that can finish something: run the tests, read the failure, fix it and open the pull request Left for the next wave : Prompting it every turn, and the judgement in the setup The tools of this wave [**Claude Code** hooks, skills and subagents](https://code.claude.com/docs) [**OpenAI Codex**](https://developers.openai.com/codex) [**LangChain Deep Agents**](https://docs.langchain.com/oss/python/deepagents/overview) [*L* **Langfuse** seeing what the agent did](https://langfuse.com/) [**Braintrust**](https://www.braintrust.dev/) Do we test for this? Partly We test orchestration, not a full harness build Tool orchestration is scored in the AI-native assessment: what the engineer hands to the agent, what they keep, and whether they run work in parallel worktrees. Designing a team's sandbox, permissions and hooks doesn't fit in sixty minutes, so for platform roles the DevOps assessment adds a generated pipeline change with a permission that's broader than it looks. [Tool orchestrationAI-native scorecard](https://www.miyagami.com/how-we-vet#ai-native)[Context-switch hygieneAI-native scorecard](https://www.miyagami.com/how-we-vet#ai-native) Hire for this [AI-native engineers](https://www.miyagami.com/hire/ai-native-engineers) [DevOps engineers](https://www.miyagami.com/hire/devops-engineers) [AI engineers](https://www.miyagami.com/hire/ai-engineers) 05 Mid 2026 · Loop Wave 5 of 6 ## The loop runs agents without someone approving every turn What teams adopted Claude Code shipped /loop in March 2026, Codex added goals, and the GitHub Copilot coding agent and OpenClaw picked up work from tickets. Existing CI became the trigger and usually the verifier. How the work changed Agents run on a schedule or an event without someone approving every turn, and engineers design the check that tells them to stop. We had agents that could finish real tasks, and senior engineers sitting there approving every turn. A harness equips one run. A loop composes many, triggered on a schedule or an event and bounded by something a machine can check. Claude Code shipped /loop in March 2026 and Codex added goals in April. Your existing CI is usually the trigger and the verifier. Where the focus is The verifier and the stop condition. Both are designed before the loop runs. Solved : A senior engineer typing "yes, continue" into four windows Left for the next wave : Verifying what comes back A loop that works - The verifier is designed before the loop - The stop condition is something a machine can check - It runs on a narrow, well-tested surface first - Someone reads the morning's output A loop that burns money - The agent wrote the tests it is being judged against - The stop condition is "until it looks done" - It was pointed at the whole codebase on day one - Nobody can say what it changed last night The tools of this wave [**Claude Code /loop**](https://code.claude.com/docs/en/scheduled-tasks) [**Codex goals**](https://developers.openai.com/cookbook/examples/codex/using_goals_in_codex) [**GitHub Copilot coding agent**](https://docs.github.com/en/copilot/concepts/agents/coding-agent) [*O* **OpenClaw**](https://github.com/openclaw/openclaw) [**GitHub Actions** CI as trigger and verifier](https://github.com/features/actions) Do we test for this? No We don't test loops, we test what they depend on An unattended loop needs hours, a real CI and a codebase the engineer knows, so a sixty-minute assessment can't show it honestly. What we can see is the groundwork a loop needs: whether the engineer cuts a vague ticket into pieces a machine can check, and whether they verify output instead of trusting it because it compiles. [Spec-driven developmentAI-native scorecard](https://www.miyagami.com/how-we-vet#ai-native)[AI output verificationAI-native scorecard](https://www.miyagami.com/how-we-vet#ai-native) Hire for this [AI-native engineers](https://www.miyagami.com/hire/ai-native-engineers) [DevOps engineers](https://www.miyagami.com/hire/devops-engineers) 06 Now · Verification Wave 6 of 6 ## Verification is the bottleneck, and spec and test-driven development solves it What teams adopted Teams add AI review (CodeRabbit, Greptile, Qodo, Graphite), scanning for what the agent reached (Semgrep, Snyk, Socket) and tools that show the review queue (Faros, LinearB). The teams moving fastest also change the order of the work. How the work changed The spec and the acceptance tests are written before the agent starts, so most changes are checked automatically and people read only the risky ones. Five bottlenecks were dissolved by the next release. This one isn't. A Harvard working paper found that firms adopting coding agents added about 30 percent more code, finished no more work, and took 49 percent longer to get it through review. AI review tools did not close the gap. When every change waits for a senior to read it, your seniors become the queue. Spec and test-driven development takes them out of it: the spec and the acceptance tests are written before the agent generates anything, so every run is checked automatically against something the agent doesn't control, and people only read what the tests can't judge. Where the focus is Write the spec and the acceptance tests first. Then the tests do most of the verifying and your seniors read only what they can't judge. Solved : Spec and test-driven development: the checks exist before the code Left for the next wave : Writing the spec and the tests, once, before the agent runs [**+30%** more code at firms that adopted coding agents, with no statistically significant rise in finished work Chen and Stratton, Harvard, 2026](https://fion.ac/jellyfish.pdf) [**+49%** longer from pull request to merge after agents arrived Chen and Stratton, Harvard, 2026](https://fion.ac/jellyfish.pdf) [**+31%** more pull requests merged with no review at all, human or agentic, under high AI adoption Faros AI, 2026](https://www.faros.ai/research/ai-acceleration-whiplash) [**5.3x** longer that agent-authored pull requests wait before a reviewer picks them up LinearB, 2026](https://linearb.io/library/engineering-benchmarks-and-research) Review is your constraint - More pull requests merging with no review at all - Agent-authored pull requests waiting much longer before pickup - Your seniors are the queue, and new rules keep adding to it - Agents abandon their own pull requests when feedback isn't a failing check You have moved past it - The spec and the tests exist before the code is generated - The verifier is independent of whatever wrote the change - Review depth is routed by what a change can break - The queue is empty at the end of the week The tools of this wave Review [**CodeRabbit**](https://www.coderabbit.ai/) [**Greptile**](https://www.greptile.com/) [**Qodo**](https://www.qodo.ai/) [**Graphite**](https://graphite.dev/) What the agent reached [*S* **Semgrep**](https://semgrep.dev/) [**Snyk**](https://snyk.io/) [**Socket**](https://socket.dev/) Seeing the queue [*F* **Faros**](https://www.faros.ai/) [*L* **LinearB**](https://linearb.io/) Do we test for this? Yes Verification carries the most weight in our assessments AI output verification is the criterion we weight most heavily for AI-native engineers: catching hallucinated APIs, subtly wrong logic and security gaps before they ship. Spec-driven development is scored in the same session: we look at whether the engineer writes down what the change must do and how it will be checked before the agent starts. We also watch for judgement by risk, and QA engineers verify a feature they didn't design, with generated tests that all pass and cover the wrong thing. [AI output verificationAI-native scorecard](https://www.miyagami.com/how-we-vet#ai-native)[Spec-driven developmentAI-native scorecard](https://www.miyagami.com/how-we-vet#ai-native)[DebuggingFundamentals scorecard](https://www.miyagami.com/how-we-vet#fundamentals) Hire for this [AI-native engineers](https://www.miyagami.com/hire/ai-native-engineers) [QA engineers](https://www.miyagami.com/hire/qa-engineers) Where your team is ## Six practices of the teams furthest along Tick the ones your team already does. Four or more and your team has adopted the way of working, not just the tools. **0** of 6 in place Tick what your team already does. 1. 1 **Spec before code** Write what the change must do, what it must not break and which modules it touches, then generate against it. A vague ticket handed to an agent produces confident work built on a guess. 2. 2 **Tests before the feature** If the same model writes the code and the tests in one session, a green pipeline means the model was internally consistent. That is not the same as correct. 3. 3 **An independent verifier** A type check, a performance budget, a contract test owned by another service. Something the thing being verified does not control. 4. 4 **Review by blast radius** A copy change and a schema migration should not queue behind each other. Decide which categories always get a full human read and let the rest run lighter. 5. 5 **A named owner per repository** With the context file in their remit and the authority to say no. A repository taking agent-speed input with no owner bloats in weeks. 6. 6 **Measure verification load** Percentage of code written by AI tells you nothing. How much change your team can read, run and vouch for in a week is your actual ceiling. Engineering Leader Benchmark Q4 2026 ## See which layer your team is on The benchmark asks how your team works with agents, how you verify and how you plan. You see your score in eight minutes, next to teams of your size once enough have answered. [Take the benchmark](https://www.miyagami.com/resources/benchmarks/engineering-leader-q4-2026) The Engineering Leader Benchmark Q4 2026 report FAQs ## Questions about the whitepaper Our other whitepaper describes the two assessments every engineer takes. [The Next 10X Engineer](https://www.miyagami.com/resources/whitepapers/the-next-10x-engineer) What is the current bottleneck in AI-assisted software engineering? Verification. Coding agents increased the volume of code, but review time rose with it: a Harvard working paper covering 718 firms found 30 percent more code, no significant rise in finished work and 49 percent longer from pull request to merge. In what order did engineering teams adopt AI tools? In six waves: code completion such as GitHub Copilot (2022 to 2023), chat such as ChatGPT and Claude (2023 to 2024), context through the Model Context Protocol and files like CLAUDE.md and AGENTS.md (2024 to 2025), agent harnesses such as Claude Code and OpenAI Codex (early 2026), loops that run agents without approval on every turn (mid 2026), and now review, scanning and spec and test-driven development to verify what agents produce. Which AI tools should an engineering team adopt first? Tools matter less than the order of the work around them. Most teams already have completion and chat. The step that pays off next is context: a maintained CLAUDE.md or AGENTS.md and connections to tickets and schemas. Agents and loops only pay off once a team writes the spec and the tests before the agent starts. What is harness engineering? Harness engineering is designing everything around the model: the loop that runs it, the tools it can call, context compaction, the sandbox, permissions, hooks, subagents and observability. Mitchell Hashimoto named it in February 2026, summed up as agent = model + harness. What is the difference between context engineering and harness engineering? Context engineering decides what the model sees, through files such as CLAUDE.md and AGENTS.md and protocols such as MCP. Harness engineering covers the whole runtime around the model, including what it is allowed to do and how its work is checked. Does Miyagami test engineers for these skills? For most of them. Verification, context engineering, tool orchestration and explaining generated code are scored in our AI-native assessment, and the fundamentals assessment runs with AI tools off. We don't test unattended loops, because a sixty-minute session can't show them honestly. Let's talk ## Get a shortlist within three days You share the roles and the stack in a short form or a thirty-minute call. Within three working days you get named senior engineers to review, each with both scorecards, and you can hire within five. [Get a shortlist](https://www.miyagami.com/shortlist) Reviewed on **Clutch** 42 reviews [ISO 27001 Certified](https://trust.miyagami.com/) Book thirty minutes with Dale The calendar is provided by HubSpot, which sets its own cookies. Load it here, or book on HubSpot's page. [Open booking page](https://sales.miyagami.com/meetings/dwesdorp)