AI for Software Engineering
Code models, repository context, patch generation, test-driven agents and SWE benchmarks.
12concepts
101flashcards
87minutes of reading
- 01 Automated Program Repair and Fault Localisation How spectrum-based fault localisation ranks suspicious code from test coverage, why generate-and-validate repair overfits weak test suites, and how LLM pipelines such as Agentless rebuilt the same localise-repair-validate loop.
- 02 Measuring AI-Assisted Developer Productivity Why credible studies of the same tools report everything from a 19 percent slowdown to a 56 percent speedup, what each study design can and cannot establish, and how to read an effect size before repeating it.
- 03 The Verification Cost of Generated Code Generation got cheap and checking did not, so the net speedup of an AI-assisted change is governed by how much of the work was verification in the first place, which is why the same tool helps on a greenfield script and hurts in a mature repository.
- 04 AI-Assisted Code and Maintainability Signals The measured side effects of assistant-heavy development, covering duplication, the collapse of refactoring signals, rising complexity and delivery instability, and which of these signals are strong enough to act on.
- 05 Code Generation Benchmarks and pass@k How HumanEval turned code evaluation into execution, why the unbiased pass@k estimator exists and how it is derived, and why saturation and contamination pushed the field toward time-stamped benchmarks.
- 06 Code Review by Model Why review is a better fit for current models than authoring, the precision problem that determines whether it is used, and the classes of finding worth reporting.
- 07 Fill-in-the-Middle and Completion Models Why a left-to-right model cannot do the most common editing task, the training transformation that fixes it, and the latency budget that shapes everything about inline completion.
- 08 LLM-Assisted Code Migration at Scale The best-documented industrial win for code models is not feature development but mechanical migration, where change location is searchable, correctness is machine-checkable, and the work was previously too tedious to fund.
- 09 Repository Context and the Retrieval Problem Why a model with a large context window still cannot see a codebase, what makes code retrieval different from document retrieval, and the signals that actually locate relevant code.
- 10 Reviewing Machine-Authored Changes When generation gets cheap, human review becomes the binding constraint on delivery, and the queue behaves the way queues do: larger diffs, longer waits, and a reviewer whose attention is the scarce resource.
- 11 SWE-bench and Agentic Coding Evaluation What resolving a real GitHub issue measures that a function-completion benchmark does not, the contamination and scoping problems that complicate it, and how to read a reported score.
- 12 Test-Driven Agents and Verification Loops Why an executable check transforms an agent's reliability, how to structure the loop so it converges, and the failure where the agent optimises the test rather than the code.