Five hours to figure out the JSONL was always on disk

I opened the RAG Factory task window with a one-line ask. Six task windows, delta RAG each, remind me which skill to trigger. What followed was five hours of trust breaking down between me, the skill I built, and the task windows themselves. I want to walk through what happened, because the failure mode is not a bug in any single piece. It is the assumptions each piece carries about the others. If you run Cowork task windows for anything more than throwaway chats, you are going to hit this. Here is what I got wrong, what I finally got right, and the protocol I now use to make sure the next delta RAG takes minutes instead of hours.

What I was actually trying to do

For context, my personal knowledge system is a set of RAG corpora, one per Cowork task window and one per Claude.ai chat. Each corpus lives at D:\Programming\Cowork-sessions\<title>\ and contains the raw JSONL, a rendered docx and txt, a capture.json envelope, a _markup\ folder with the extracted structure (architecture, decisions, timeline, glossary, open items, and so on) and an _extracts\ folder with reformatted deliverables. There is a bookmark file, _markup\RAG_STATE.json, that records the last processed turn number. When I run a delta RAG, the pipeline reads new turns since that bookmark, feeds them into a three-pass Claude Code job, and appends only the new material to the markup.

I built a skill called run-delta-rag so I would never again have to remember the wrapper paths or the mode detection. Trigger phrase, one line, done. Six task windows was a Sunday afternoon. Not a heavy ask.

The prompt that failed

I typed one prompt into the RAG Factory task window: run delta RAG for these six task windows, and remind me which skill I built for this. That is all it should have taken. The skill should have detected each window's mode, dispatched to the right wrapper, and reported back six results.

Instead, everything downstream of that prompt turned into a nested debugging thread on the fundamentals: what mode is a task window in, where is its transcript, why does the bookmark fail, why does the current session UUID not match the previous session UUID, why does the packaged root not show what I expect, why does the sandbox not know its own session ID. The skill was fine. My mental model of the underlying infrastructure was not.

The first wrong turn: assuming rotation

The first window I tried was 1nicorn-backend2. The bookmark said last processed turn n=1226 on Aug 1. Delta refused. Integrity check failed. Turn hashes did not match. The window assumed the standard explanation: Cowork spawned a new session under the same title on compaction, and the previous compactions were lost because this window ran in the cloud sandbox. It called it a session rotation problem, staged the current post-compaction session as 1nicorn-backend2b, restored the Aug 1 canonical from a zip, and concluded that two intermediate compactions were unrecoverable.

I disagreed, and I turned out to be right. Session rotation for local windows is not what the window thought it was. It was pattern-matching on the last similar failure, and it stopped looking for evidence that would contradict the story. That is the first best-practice violation. If your integrity check fails, do not immediately reach for the explanation that lets you file the case and move on. Verify what state the underlying file is actually in.

The bigger wrong turn: assuming cloud

Then came sens-platform. This one hurt because I had delta-RAG'd it four days earlier without incident. The bookmark was clean at n=353. The Cowork-sessions folder had the JSONL, the capture, the docx, the txt. Everything looked fine on disk. I asked the window to run a delta.

It ran the same environment hunts it had run for 1nicorn-backend2. It filtered for JSONLs modified in the last fourteen days that contained any keyword like Sensfix, Keycloak, sensplatform, devsens, Port Manatee, gatewayh, Agunsa, pg-uc1, GKE, sens-platform. It ran that grep against every JSONL under the packaged MSIX root at C:\Users\<you>\AppData\Local\Packages\Claude_pzs8sxrjxfjjc\LocalCache\Roaming\Claude\local-agent-mode-sessions\.

The hunt returned zero hits.

The window read that as a definitive verdict. If none of the on-disk JSONLs contained a Sensfix keyword, the transcript must live only in Anthropic's cloud sandbox. It told me sens-platform had gone cloud, and it proposed the manual seam capture workaround. It told me to open the live sens-platform window and paste in the cloud pull handover.

Both conclusions were wrong. Sens-platform had never gone cloud. The grep was missing content that was very much on disk. And by handing off to another Claude in another window, I set that Claude up to compound the mistake.

The nightmare compounds

The sens-platform Claude, working from my flawed premise, went its own direction. It ran a hunt against $env:LOCALAPPDATA\Claude and $env:APPDATA\Claude. Both returned zero JSONLs. Reasonable conclusion from that Claude's point of view: local Cowork sessions do not persist a copyable transcript at all. The file reader can show the transcript to the model, but there is nothing on disk for PowerShell to copy.

That Claude then proposed two paths forward. First, close the sens-platform window and hope the app flushes the transcript to disk on close, then re-run the discovery scan. Second, do a reader-dump reconstruction: read the transcript in chunks through the file tool, strip the reader's line-number prefixes, write parts under D:\Programming, concatenate them. The Claude flagged, correctly, that the reader can truncate very long lines that contain embedded images or big tool outputs. Truncated lines produce invalid JSON. Invalid JSON silently poisons your RAG index. The Claude was doing the honest thing by naming the risk. But every mitigation on offer was a workaround for a problem that did not exist.

I pushed back. I had seen at least one compaction happen on this window. I wanted to know, before doing anything else, whether every compaction spawns a new JSONL file. I wanted to know whether the JSONL UUID we now had matched the JSONL UUID we had previously RAG'd. I wanted answers, in two lines each, so we could stop chasing shadows.

The turnaround: ask the codename

The right move was one I should have made in turn one, not turn twenty. I asked the sens-platform Claude to run a tiny command in its own sandbox and paste the raw output back.

bash: echo "SESSION_ID=$SESSION_ID"; echo "SANDBOX=$(hostname)"; env | grep -iE 'session|sandbox|cowork' | head -20

The output was almost boring, and that was the point.

SESSION_ID=
SANDBOX=claude
PWD=/sessions/nice-zen-maxwell
HOME=/sessions/nice-zen-maxwell

SESSION_ID was empty. There was no cowork-prefixed variable. But PWD contained a codename: nice-zen-maxwell. In Cowork, every sandbox is spawned under /sessions/<CODENAME>/. The codename is the identifier the sandbox knows itself as. The Claude Desktop app knows what session UUID that codename maps to. That mapping had to exist somewhere on disk, because the app needs it every time it decides which VM to route a bash call to.

The reveal in the logs

I searched the packaged root for any file mentioning the string nice-zen-maxwell. Ten hits came back. Six were in the packaged local-agent-mode-sessions tree, three were in the app logs folder, one was in a specific session metadata file.

The single decisive hit was in main.log, line 509, dated Aug 19 at 18:20:41:

Starting local session local_5e488235-d87b-460f-94a0-a35fcece024e in /home/nice-zen-maxwell.

That one line settled everything.

Sens-platform was a local session, not cloud. It had always been local. It was still local. The session UUID was 5e488235-d87b-460f-94a0-a35fcece024e. Its live JSONL was at \.claude\projects\...\0c21032e-0305-48ea-bc70-741f52016197.jsonl, 31.35 MB, mtime tonight, currently being appended to by the app. My earlier keyword hunt had failed not because sens-platform was cloud, but because Select-String was not reliably matching giant multi-line JSON blobs containing tool payloads. The audit.jsonl in the same folder was 34.93 MB. Select-String's default behavior against a file that size, where individual JSON lines can be multi-megabyte, was silently unreliable. It said zero hits. It meant, at best, zero hits I could confirm.

What compaction actually does

Once the correct JSONL was in view, I had one more question. Did every compaction spawn a new file? I ran a grep for the sessionId field inside the current live JSONL and the sessionId field inside the Aug 19 on-disk copy that had been the input to the previous delta RAG.

Both files contained exactly one sessionId. Same UUID in both.

That answered the question. For local Cowork sessions, compaction does not rotate the JSONL. It does not spawn a new file. It appends in place. Older turns can have summary markers inserted between them, but nothing is removed. The current 31 MB file is a strict superset of the Aug 19 28 MB file. Every turn the old file had, the new file still has, plus four days of new turns after them. My bookmark at n=353 could pick up cleanly from turn 354 onward.

Cloud is a different failure mode entirely

1nicorn-backend2 was cloud from the beginning. The 32 MB JSONL on my disk had been produced by a one-shot cloud pull handover on Aug 1. When the cloud sandbox compacted the session, it never wrote back to my disk. The Aug 1 file froze at n=1226. Future turns lived only in the cloud VM that spawned each user turn. When those VMs were reclaimed, that content evaporated. Two intermediate compactions were genuinely lost.

That is a completely separate failure mode from anything compaction does to a local session. Local compaction is safe. Cloud compaction, without a live pull, loses data. The confusion I created earlier came from applying the cloud failure model to a local session. The evidence for local mode was one line of the main log. I did not look at the log until turn fifteen or so of the debugging thread. That is on me.

The best-practice protocol

Here is the flow I now run every single time I delta RAG a Cowork task window, no exceptions. The whole thing takes maybe two minutes. I built the run-delta-rag skill to do this automatically, but I still want to be able to verify by hand when the skill throws an unexpected result.

Step one. Open the target task window. Ask it to run one line in its sandbox:

echo "SESSION_ID=$SESSION_ID"; echo "PWD=$PWD"; env | grep -iE 'session|sandbox|cowork' | head -20

You are looking for the codename in the PWD. That is the sandbox identifier.

Step two. On the host machine, grep the app's main.log for that codename. This is the definitive mapping:

$log = "$env:LOCALAPPDATA\Packages\Claude_pzs8sxrjxfjjc\LocalCache\Roaming\Claude\logs\main.log"
Select-String -Path $log -Pattern "Starting local session .* in /home/nice-zen-maxwell" -SimpleMatch

Replace nice-zen-maxwell with your window's actual codename. If you get a hit, the window is local. The hit line tells you the session UUID.

Step three. If step two returned a hit, the transcript is at:

%LOCALAPPDATA%\Packages\Claude_pzs8sxrjxfjjc\LocalCache\Roaming\Claude\local-agent-mode-sessions\<ORG_UUID>\<ORG2_UUID>\local_<SESSION_UUID>\.claude\projects\<encoded-path>\<JSONL_UUID>.jsonl

Fire your local wrapper against it. The wrapper does the rest.

Step four. If step two returned nothing, the window is cloud. Do not close it. Do not run reader-dump reconstruction. Do not assume the transcript is unrecoverable. Instead, paste the cloud pull handover into the live window and have it dump its own JSONL to your disk. Then run the cloud wrapper against the freshly dumped file.

Step five. Once your wrapper finishes, verify two things: the log tail contains SESSION COMPLETE, and the bookmark in _markup\RAG_STATE.json advanced.

If you follow this five-step protocol, you will never again waste hours chasing a failure mode that does not apply to your situation. My run-delta-rag skill executes this pipeline programmatically. When it detects local mode from the log grep, it dispatches to the local wrapper. When it detects cloud mode, it asks whether a fresh pull has been done and dispatches to the cloud wrapper accordingly.

What this changes about my RAG workflow

Three lessons I am carrying forward.

Lesson one. Do not trust Select-String on giant JSON files without verifying the match logic. If the file has multi-megabyte lines with embedded content, the string match may silently miss. Use grep from a real Linux environment, or use jq to extract the specific field you care about, or write your own line-by-line reader in Python that treats each JSON line as a discrete object. Do not draw conclusions from a Select-String that says zero without a control test.

Lesson two. Ask the sandbox to identify itself before asking the file system to identify it. The sandbox knows its codename with one environment lookup. The mapping from codename to session UUID lives in a log the app writes every session. That is a two-step chain with no interpretation. Every other approach is inference on top of inference.

Lesson three. When a Claude in one window is working from a premise you gave it, and that premise turns out to be wrong, do not just quietly correct it in your head. Send an explicit stand-down message to that other window with the corrected premise. Otherwise you have two Claudes drifting apart on the same problem, each convinced the other has the wrong information. In this case, the sens-platform Claude was proposing dangerous workarounds because it trusted my initial framing. When I finally told it the truth, it stopped immediately. The cost of the stand-down message was one paragraph. The cost of not sending it earlier was the reader-dump-reconstruction proposal that would have taken hours to execute and could have poisoned the RAG index.

Signing off

Delta RAG on six task windows should be a two-minute action. It became a five-hour action because I trusted a keyword hunt that was silently broken, applied the wrong failure model, handed off a bad premise to a downstream Claude, and did not look at the app log until I had exhausted every wrong hypothesis. Every one of those failures had a cheap counter-move I could have made in turn one. The protocol above is the counter-move. If you run Cowork task windows for anything you care about, use it.