Disclosure: I learn topics like this through LLM dialogue. The prompts are mine, the depth comes from the model, the verification comes back to me, and I publish the result so others don't have to start from zero.

Four files, one wrapper import, and every local Ollama chat turn lands in Langfuse with session_id, user_id, tags, token counts, and reconstructed stream chunks — no custom OTLP exporter, no monkey-patching of the Ollama client, no manual span management.

The trick is that Ollama already speaks OpenAI's chat-completion dialect, and Langfuse ships a drop-in OpenAI subclass that auto-traces. The interesting work is in three places: how you propagate context under Langfuse v4's OTel model, how you reconstruct streams into a single trace, and how you keep the whole thing testable without mocking the world.

1. The wrapper that does the heavy lifting

from langfuse.openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama",  # required by openai-python, ignored by Ollama
)

resp = client.chat.completions.create(
    model="llama3.1",
    messages=[{"role": "user", "content": "Explain MiCA Article 16"}],
)

That is the entire integration surface. The subclass intercepts .create() calls, opens a Langfuse generation, attaches input/output, counts tokens, measures latency, and closes the span — for both streaming and non-streaming responses. Anything you would do with openai.OpenAI, you do with langfuse.openai.OpenAI, pointed at http://localhost:11434/v1.

This beats native OTLP for local LLMs for three reasons:

• Ollama exposes no OTLP endpoint of its own; you would otherwise instrument the HTTP client manually.
• The wrapper captures token usage from the response payload, not from middleware estimation.
• Stream chunks are reassembled inside the wrapper, not in your application code.

2. Langfuse v4 OTel context: what goes where

The v4 SDK is built on OpenTelemetry. This changes how you attach trace-level metadata. Pre-v4, you passed session_id, user_id, and tags as kwargs to .create(). In v4, those fields live in the OTel context, and you set them with propagate_attributes():

from langfuse.openai import OpenAI
from langfuse import propagate_attributes

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

with propagate_attributes(
    session_id="sess-uuid-here",
    user_id="alice",
    tags=["llama3.1", "defi-research"],
):
    resp = client.chat.completions.create(
        model="llama3.1",
        messages=messages,
        name="ollama-chat",  # this one IS a Langfuse-specific kwarg
    )

The rule of thumb:

Passing session_id directly to .create() in v4 either gets dropped or surfaces as a generation-level metadata key — not a trace-level session. This is the single most common migration footgun.

3. Stream reconstruction in a single trace

Streaming responses present an obvious instrumentation problem: each chunk is a separate yield, but you want one trace with the full reconstructed output. The wrapper handles this transparently — but only if you actually drain the iterator. Lazy consumption breaks the trace boundary.

def chat_stream(client, model, messages, **kwargs):
    with propagate_attributes(
        session_id=kwargs.pop("session_id"),
        user_id=kwargs.pop("user_id"),
        tags=kwargs.pop("tags", []),
    ):
        stream = client.chat.completions.create(
            model=model,
            messages=messages,
            stream=True,
            name=kwargs.pop("trace_name", "ollama-stream"),
            **kwargs,
        )
        for chunk in stream:
            delta = chunk.choices[0].delta.content or ""
            yield delta

Two non-obvious requirements:

• The propagate_attributes context must wrap the entire stream consumption, not just the .create() call. Exiting the context before the iterator drains causes attribute loss on later chunks.
• Do not wrap the generator in list(...) for "convenience" inside the context — that defeats streaming. If the caller needs the full string, accumulate downstream.

4. Lazy imports + dependency injection

The module-level import problem: from langfuse.openai import OpenAI triggers SDK initialization, which validates credentials and opens an OTel exporter. That is fine in production, fatal in tests.

Solution — defer the import to call time, and let the function accept an injected client:

def chat_complete(messages, model, *, client=None, **kwargs):
    if client is None:
        from langfuse.openai import OpenAI  # lazy
        client = OpenAI(
            base_url=os.environ["OLLAMA_BASE_URL"] + "/v1",
            api_key="ollama",
        )
    return client.chat.completions.create(
        model=model, messages=messages, **kwargs
    )

Tests pass a fake client object with a .chat.completions.create() shape. No unittest.mock.patch, no MagicMock chains, no module-level monkey-patching:

class FakeChatCompletions:
    def create(self, **kwargs):
        return SimpleNamespace(
            choices=[SimpleNamespace(message=SimpleNamespace(content="ok"))]
        )

class FakeClient:
    chat = SimpleNamespace(completions=FakeChatCompletions())

def test_chat_complete_returns_content():
    resp = chat_complete([{"role": "user", "content": "hi"}], "llama3.1", client=FakeClient())
    assert resp.choices[0].message.content == "ok"

This is faster than mock-based tests (no import-time side effects to suppress) and survives SDK upgrades that rename internals.

5. What ends up in Langfuse

For every .create() call inside a propagate_attributes block, the wrapper emits:

Token counts depend on Ollama returning a usage block — newer Ollama versions do, older ones return zeros. If tokens read as 0, upgrade Ollama before debugging the wrapper.

6. CLI surface for batch tracing

For scripted audit runs (replaying a fixture set, A/B-ing prompts across models, etc.), the wrapper composes cleanly with argparse:

python trace_cli.py \
  --model llama3.1 \
  --prompt "Summarize ERC-4626" \
  --user-id alice \
  --trace-name "defi-research" \
  --tags "defi,erc4626" \
  --temperature 0.5

Each invocation gets a fresh session_id (UUID) by default; pass --session-id to group multiple invocations into one Langfuse session. This is the pattern for batch evaluation runs where you want every prompt in a fixture file to show up under a single session for aggregate scoring.

7. Self-hosted Langfuse switch

One env var difference between cloud and self-hosted:

LANGFUSE_BASE_URL=http://localhost:3000   # self-hosted
# LANGFUSE_BASE_URL=https://cloud.langfuse.com  # EU cloud
# LANGFUSE_BASE_URL=https://us.cloud.langfuse.com  # US cloud

The wrapper reads this from env at SDK init. If you swap the URL mid-process, the existing client keeps the old endpoint — instantiate a new OpenAI(...) after the swap.

Action list

1. Replace any direct openai.OpenAI import with langfuse.openai.OpenAI for any OpenAI-compatible endpoint, including Ollama, vLLM, LiteLLM, and TGI.
2. Move session_id, user_id, and tags out of .create() kwargs and into a propagate_attributes() block — anything left on .create() in v4 is silently downgraded to metadata.
3. Wrap the entire stream consumption in the context manager, not just the .create() call.
4. Defer the SDK import to function bodies and accept an injected client argument; tests get faster and survive SDK refactors.
5. Verify Ollama is on a version that returns a populated usage block before debugging zero-token traces.
6. For batch eval runs, share a single session_id across invocations so aggregate scoring groups correctly in the Langfuse UI.

Screenshots