It is widely touted that “the future of AI is agentic.” However, there is an intricate trade-off in going fully agentic: the more autonomous the agents in an application, the less reliable the application will be.
To address the issue, AI companies either train more capable models to support agentic workflows natively or create specialised application frameworks such as LangGraph and Magentic One to help downstream AI application developers. Of course, you can use both.
2024 saw a few key features emerge, often considered the backbones and benchmarks of a practical agentic solution.
State & memory management & persistence
Memory can be used to remember a single conversation (a thread) or for information across multiple conversations. For example, information about a specific user, their preferences and past interactions with the AI system can all be used to make future conversations between the user and AI more natural. Similar to conversions between real humans, it will feel very uncomfortable if someone forgets everything about you immediately after you talk to them.
Implementation details differ with different frameworks, but the idea is the same. For example, in LangGraph, you can use Checkpointer
to remember a single thread and the Store
interface for long-term memory.
Practitioners have tried to map the memory types of human brains to that of AI agents: semantic, episodic, and procedural. For AI agents, semantic memory often refers to facts about a specific user. Episodic memory refers to the agents’ past actions. Besides using a permanent store, few-shot prompting is another convenient way to provide LLMs with episodic memories. Procedural memory refers to the agent’s system prompts or out-of-box model capabilities. Memories can be updated on the hot path or in the background, with a trade-off between performance and simplicity.
Human-in-the-loop
The ability to intervene is essential for many real-world AI use cases, especially when AI safety, accountability, and regulatory requirements are involved. All popular agentic systems allow users to provide feedback, approve/refuse steps, and update the application state to influence AI agents’ behaviour.
Controllability
Fine-grained control over agents’ behaviour is key to developing high-performing agentic systems. Different tools have different opinions on this topic. Some empower engineers to design their agentic workflow from scratch, while some prescribe the orchestration part of the system.
Other considerations
LLMs are often better prompt authors than humans.
Using LLMs to construct or rewrite prompts for complex tasks proves hugely effective. In addition, combined with long-term memory, we can ask LLMs to continuously improve the system or application prompts (self-improvement) based on human-in-the-loop feedback.
Not all problems require an agentic solution. A simple LLM application with well-designed and optimised prompts can achieve a lot. Although powerful, agentic systems have the drawbacks of increased costs and degraded speed. It may be overkill in many use cases.