Trace every stage of an AI pipeline with self-hosted Langfuse

Level
intermediate
Time to complete
60 min
Prerequisites
4

What you will buildBuild the n8n node-inspector experience for your own LLM code, on your own box, for zero license cost. About 60 minutes.

What You Will Build

You will build a stage inspector for a multi-stage LLM pipeline: open any run, click any stage, and read the exact input on the left and the exact output on the right. It is the n8n Node-Detail-View experience - but for your own code, running on your own box, with no trace content ever leaving your infrastructure, and for zero license cost.

The engine is Langfuse, self-hosted. By the end you will have a running Langfuse instance, each stage of a pipeline instrumented as a named step you can click into, and a clear sketch of the one screen Langfuse does not give you - the before-and-after field diff - plus exactly how to build it on the data Langfuse already captures.

All tool facts here are stated as the September 2026 research behind this piece found them, with the dates in the frontmatter sources. Licenses and features move; check them when you read this.

The mental model: a run is a trace, each stage is a named observation in that trace, and an LLM call is a generation that records prompt in, output out, and token cost.

Prerequisites

  • Docker and Docker Compose installed, with enough disk for a database and object storage.
  • A multi-stage LLM pipeline you can edit - any language with a Langfuse SDK or an OpenTelemetry exporter works; the examples here are TypeScript.
  • Node 18 or later if you follow the TypeScript examples, and your pipeline's model provider SDK already working.
  • A short, written list of your pipeline's stages and, for each, the handful of variables that decide quality. The worked example uses an email-reply pipeline: fetch_thread, summarize_crux, retrieve_context, draft_reply, classify.

Tip

Write the stage list and the quality variables down before you touch code. The instrumentation is only as useful as the names you give things: a stage called draft_reply with inputs named crux and retrieved_context reads like a story in the trace tree; a stage called step3 with a blob of JSON does not.

Mental Model

Three words carry the whole tool. A run of your pipeline is a trace. Each stage inside that run is an observation - a named span with its own input and output. An LLM call is a special observation called a generation, which additionally records the prompt that went in, the completion that came out, and the token usage and cost.

That is the entire mapping. When you open a trace, Langfuse draws the observations as a vertical, collapsible tree on the left - the analogue of n8n's list of nodes. Click any observation and a detail panel opens on the right with its Input, its Output, a metadata strip (model, latency, tokens, cost), and a Scores tab for any evaluations or human annotations attached to it. Model your pipeline so each stage is one named observation, and the tree reads like your stage list, and clicking draft_reply shows the exact prompt in and the draft out.

Steps

Work through these in order. The first two stand up Langfuse; the rest instrument a pipeline, open a run, and sketch the diff.

Self-host Langfuse with Docker Compose

Clone the Langfuse repository and start it with the bundled compose file. This runs the whole stack - web app, worker, database, cache, and object storage - on your machine.

git clone https://github.com/langfuse/langfuse.git
cd langfuse
docker compose up -d

Wait for the containers to report healthy, then open the web UI at the address the compose file prints (by default http://localhost:3000). You know this worked when the Langfuse sign-in page loads locally.

Self-host matters here for one concrete reason: a trace of an LLM pipeline contains the content the pipeline processed, and for anyone handling other people's data that content must not leave the box. Langfuse is MIT-licensed for its product features (as of the June 2025 relicensing noted in the September 2026 research - tracing, prompts, evals, playground, and annotation queues are all in the MIT scope; only a few enterprise items like SCIM and extended audit logs sit behind a commercial key), and its own self-hosting documentation states a self-hosted instance sends no data to Langfuse Cloud.

Security

Treat this instance as holding your most sensitive content, because it does. Keep it on a private network, put authentication in front of it, and never point your pipeline at a Langfuse Cloud key when you mean to keep content on-box.

Create a project and copy your keys

In the local UI, create an organization and a project, then open the project settings and create a pair of API keys - a public key and a secret key. Put them in your pipeline's environment, pointed at your local instance.

LANGFUSE_HOST=http://localhost:3000
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...

You know this worked when the keys exist in the project settings and your environment variables match them exactly.

Sources

  1. Langfuse self-hosting: a self-hosted instance sends no data to Langfuse Cloud (accessed 2026-09-29)langfuse.com
  2. Langfuse MIT license scope; /ee commercial features (accessed 2026-09-29)github.com
  3. OpenTelemetry default 256-char attribute limit (2026-01-30)oneuptime.com
  4. OpenLLMetry: Apache-2.0 OpenTelemetry instrumentation for LLMs (accessed 2026-09-29)github.com