Unit testing LLM Agents

A Python unit-test framework - for making sure that Agents do as they're expected.

LLM Agents are powerful constructs - capable of asking questions, using tools, reasoning iteratively while fixing their mistakes or assumptions. When creating such agents for a specific purpose, we want to make sure that purpose is achieved. However, because these beings exist in a conversational, non-deterministic environment, doing proper unit tests becomes tricky. In this talk I will present a method, as well as a Python framework, for tackling this problem.