We are building an autonomous research agent that turns a vague question into a structured plan, gathers evidence across multiple calls, and synthesizes a markdown report. This is the practical future of LLMs: not monolithic chat, but small, orchestrated reasoning loops that leverage long context and tool use. Because Oxlo.ai charges a flat rate per request instead of per token (see pricing), running multi-step agent workflows like this stays predictable even when prompts grow.
What you'll need
- Python 3.10 or newer
- An Oxlo.ai API key from https://portal.oxlo.ai
- The OpenAI SDK:
pip install openai
Step 1: Initialize the Oxlo.ai client
We point the OpenAI SDK at Oxlo.ai. If you want to experiment later, Oxlo.ai also offers reasoning specialists such as DeepSeek R1 671B MoE and Kimi K2.6, but Llama 3.3 70B is a solid general-purpose default for this pipeline.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")Step 2: Define the agent system prompt
The system prompt forces the model to stay in character and emit structured output. We keep it strict so downstream parsing stays reliable.
SYSTEM_PROMPT = """You are a research agent. Your job is to help a user investigate a complex topic.
When asked to plan, return exactly one sub-question per line, no bullets, no numbers.
When asked to answer a sub-question, return a concise, factual paragraph with citations if possible.
When asked to synthesize, return a markdown report with an H1 title, an executive summary, and detailed sections."""Step 3: Generate a research plan
We send the user query to the model and ask for a list of sub-questions. We split the response on newlines to get discrete tasks.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a research agent. Your job is to help a user investigate a complex topic.
When asked to plan, return exactly one sub-question per line, no bullets, no numbers.
When asked to answer a sub-question, return a concise, factual paragraph with citations if possible.
When asked to synthesize, return a markdown report with an H1 title, an executive summary, and detailed sections."""
def generate_plan(user_query: str) -> list[str]:
planning_prompt = (
f"User question: {user_query}\n\n"
"Generate exactly 3 focused sub-questions that will help answer the user question. "
"Return one per line, no numbering."
)
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": planning_prompt},
],
)
raw = response.choices[0].message.content.strip()
return [line.strip() for line in raw.splitlines() if line.strip()]
# Example
plan = generate_plan("What are the trade-offs between retrieval-augmented generation and long-context LLMs?")
print(plan)Step 4: Gather evidence for each sub-question
We loop over the plan and call the model once per sub-question. On Oxlo.ai, each call costs the same flat amount regardless of prompt length, so expanding context here does not explode the bill.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a research agent. Your job is to help a user investigate a complex topic.
When asked to plan, return exactly one sub-question per line, no bullets, no numbers.
When asked to answer a sub-question, return a concise, factual paragraph with citations if possible.
When asked to synthesize, return a markdown report with an H1 title, an executive summary, and detailed sections."""
def gather_evidence(sub_questions: list[str]) -> dict[str, str]:
evidence = {}
for idx, question in enumerate(sub_questions, 1):
answer_prompt = f"Sub-question {idx}: {question}\n\nAnswer concisely."
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": answer_prompt},
],
)
evidence[question] = response.choices[0].message.content.strip()
return evidence
# Assuming 'plan' from Step 3
answers = gather_evidence(plan)
for q, a in answers.items():
print(f"Q: {q}\nA: {a}\n")Step 5: Synthesize the final report
Finally, we feed the collected evidence back into the model with a synthesis prompt. This demonstrates the long-context strength of modern LLMs: condensing multiple reasoning steps into a coherent deliverable.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a research agent. Your job is to help a user investigate a complex topic.
When asked to plan, return exactly one sub-question per line, no bullets, no numbers.
When asked to answer a sub-question, return a concise, factual paragraph with citations if possible.
When asked to synthesize, return a markdown report with an H1 title, an executive summary, and detailed sections."""
def synthesize(user_query: str, evidence: dict[str, str]) -> str:
evidence_block = "\n\n".join([f"Sub-question: {q}\nAnswer: {a}" for q, a in evidence.items()])
synthesis_prompt = (
f"Original question: {user_query}\n\n"
f"Evidence collected:\n\n{evidence_block}\n\n"
"Synthesize the above into a final markdown report."
)
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": synthesis_prompt},
],
)
return response.choices[0].message.content.strip()
# Assuming 'query' and 'answers' from previous steps
report = synthesize("What are the trade-offs between retrieval-augmented generation and long-context LLMs?", answers)
print(report)Run it
Here is the complete script. I run it on the topic above. Because Oxlo.ai has no cold starts on popular models, the multi-turn pipeline executes immediately.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a research agent. Your job is to help a user investigate a complex topic.
When asked to plan, return exactly one sub-question per line, no bullets, no numbers.
When asked to answer a sub-question, return a concise, factual paragraph with citations if possible.
When asked to synthesize, return a markdown report with an H1 title, an executive summary, and detailed sections."""
def generate_plan(user_query: str) -> list[str]:
planning_prompt = (
f"User question: {user_query}\n\n"
"Generate exactly 3 focused sub-questions that will help answer the user question. "
"Return one per line, no numbering."
)
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": planning_prompt},
],
)
raw = response.choices[0].message.content.strip()
return [line.strip() for line in raw.splitlines() if line.strip()]
def gather_evidence(sub_questions: list[str]) -> dict[str, str]:
evidence = {}
for idx, question in enumerate(sub_questions, 1):
answer_prompt = f"Sub-question {idx}: {question}\n\nAnswer concisely."
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": answer_prompt},
],
)
evidence[question] = response.choices[0].message.content.strip()
return evidence
def synthesize(user_query: str, evidence: dict[str, str]) -> str:
evidence_block = "\n\n".join([f"Sub-question: {q}\nAnswer: {a}" for q, a in evidence.items()])
synthesis_prompt = (
f"Original question: {user_query}\n\n"
f"Evidence collected:\n\n{evidence_block}\n\n"
"Synthesize the above into a final markdown report."
)
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": synthesis_prompt},
],
)
return response.choices[0].message.content.strip()
if __name__ == "__main__":
query = "What are the trade-offs between retrieval-augmented generation and long-context LLMs?"
plan = generate_plan(query)
answers = gather_evidence(plan)
report = synthesize(query, answers)
print(report)Example output:
# Trade-offs Between Retrieval-Augmented Generation and Long-Context LLMs
## Executive Summary
Retrieval-augmented generation (RAG) and long-context LLMs both aim to ground model outputs in external knowledge, but they differ in cost structure, latency, and accuracy dynamics.
## Detailed Analysis
### Cost and Infrastructure
RAG requires vector databases, embedding pipelines, and chunking strategies. Long-context models eliminate much of that infrastructure but demand larger GPU memory and longer inference times per request.
### Accuracy and Hallucination
RAG pinpoints specific source snippets, which reduces hallucination for fact-heavy queries. Long-context models can lose signal in the middle of a huge prompt unless trained with strong attention mechanisms.
### Latency
RAG adds a retrieval round-trip. Long-context models process everything in a single forward pass, though total time can still be high for 100K+ token windows.
## Conclusion
Hybrid architectures are emerging: use RAG for initial filtering, then feed a smaller, relevant corpus into a long-context model for synthesis.Next steps
Swap Llama 3.3 70B for Kimi K2.6 or DeepSeek V3.2 if you want stronger reasoning in the synthesis step. You can also replace the simulated evidence loop with real tool calls using Oxlo.ai's function calling support, feeding live search results or database rows into the same pipeline.