Tuning guardrails can take time. To accelerate this, we combined the power of Claude Code, Coralogix Telemetry and the CX CLI, to build a loop that iterates on guardrails. To test our loop, we built a fictional chat interface for an iGaming operator, that took bets, gave odds on markets and more, but with a catch: this agent was not permitted to make predictions about where the market was going. With our goal in place, we got to work designing our loop.

The application under test

We built a simple, iGaming operator chat interface. It was powered by a simple AWS Bedrock model, and instrumented using OpenTelemetry. This simple interface captured messages, ran them through Coralogix Guardrails, and created our test bed. This was our production workload.

The instrumentation writes the span and attribute structure according to the OpenTelemetry GenAI semantic conventions. Browser sessions arrive through Real User Monitoring, so a full conversation is tracked as a simple OpenTelemetry trace, from front end application to all backend services and infrastructure.

Everything the application does lands in one telemetry lake in Coralogix, enabling CX CLI and Claude to access this data at will. This is essential for autonomous agents making informed decisions that are grounded in reality.

How the loop runs

Claude Code ran the whole cycle, grading its own policy against live traffic and rewriting it until the tests came back clean.

Our loop ran simply. First, the running application is invoked using Playwright scripts. It was important that we use this as a user would, and not simply bypass the UI. This way the loop can also take into account whether the interface is performing correctly too. A known prompt that should violate the prompt is pushed through (or indeed, an innocent one for a positive test case), and the result is recorded. A positive score typically indicates that the guardrail fires, and a score of 0 indicates the guardrail didn’t block the prompt.

Why you can afford to run it a hundred times

Reading observability data into a model consumes most of a loop’s budget. The Coralogix CLI (cx) aggregates server-side, so the command returns the answer instead of the rows. This transforms thousands of JSON documents into a few hundred tokens, enabling models to ask dozens or hundreds of questions, and still preserve a healthy context window and keep costs down. This, combined with the -agents format in CX CLI, based on TOON, means that each iteration of the loop cost hardly any tokens, but delivered high quality insights.

How did the loop perform?

As the loop iterated, it expanded the set of examples and instructions within the guardrail configuration.

It did this by invoking the CX CLI on each loop to apply the amendments, keeping examples that worked and removing examples that didn’t. It also generalised rules that it put in. In some previous iterations, it would put a certain prompt in verbatim, but after iterating on the next example, Claude decided to generalise a certain instruction to make it work for a broader set of answers. Of course, this translated into a chat interface that guarded its inputs and outputs much more accurately.

Telemetry is the secret sauce of loop engineering

For any loop to survive and iterate closer to a true solution, it must have a connection to reality. Telemetry is the most accurate data in any organisation, and Coralogix makes that data available through a Telemetry lake, that can ingest data, and make it available, at no cost per query. This is the essence of effective, agentic iteration, and it was all done from the agent, without opening a single interface.