Agents: Programming with feedback-driven LLMs
David Crawshaw shares a good perspective on how LLMs transform into powerful programming assistants when given tools to interact with their environment. While definitions of “AI agents” vary widely, David aligns with Simon Willison and Solomon Hykes’ view of agents as systems that repeatedly interact with their environment; what Solomon calls “wrecking its environment in a loop”.
Via Pointer.io’s Essential Reading For Engineering Leaders newsletter
Unlike standalone language models that operate in isolation, agents can execute commands, read files, and iterate based on feedback; this fundamentally changes their capabilities.
The core insight: Agents are essentially feedback loops where an LLM can:
- Run test suites and interpret results
- Execute bash commands to explore codebases
- Apply patches and validate changes
- Access documentation through web searches
“Agents are LLMs with environmental feedback. Just as humans thrive in environments with feedback, LLMs go from nice demos to useful programmers when given a surprisingly small core set of tools.”
A practical example: When an agent runs your test suite and encounters failures, it can:
- Analyse error messages
- Modify the problematic code
- Re-run tests
- Repeat until passing
This transforms coding from a “whiteboard exercise” into an interactive process where the LLM can validate its own work. As David notes:
“The result, compared to an LLM generating code without these tools available is significantly different. API use is greatly improved because the agent can web search for documentation and
curlthe docs into its context window.”
While agents currently work slower than traditional coding (minutes vs seconds, meaning they are still relatively expensive), they enable completing “dreary tasks” that might otherwise take days, freeing developers for higher-value work.
Containerized agent environments
The future likely involves containerized agent environments, isolated execution spaces that solve two key challenges:
- Safety: Prevents agents from accessing sensitive credentials or systems
- Parallelism: Allows multiple agents to work simultaneously without conflicts
These sandboxed environments let developers safely run agents while maintaining their normal workflow, with changes extracted as diffs or branches when complete.
