LLMs and their harness usually work together to help reach the goal of the user’s prompt, but ocassionally under the right conditions the result will be a spectactular failure that makes you wonder whether you broke AI’s brain. I’ll call this Model Collapse and I wonder if this has happened to you?
In one Tarsk run, GLM-5.3-flash started a coding task, and in the end spewed out 1.65 million additional input tokens (training data), a whole bunch of chinese text and unrelated TV schema and “Hi Moto — Sad, but yes, this time it’s all I have.”
The run looked healthy at first. The model started implementation, set todos, and followed the expected workflow. The output later lost the task, invented control text, and stopped doing useful work.
The context meter showed 66,000 tokens, or 51% of the window. That number did not force failure by itself, but it left less room for mistakes.
The model set todos and began to inspect the code. Then it stopped making forward progress.
The Failure Pattern
- It re-read a file 26 times (
browser-tab.tsx). - After two English sentences, it lost the task.
- It invented fake control text such as
<linux_note>,Hi Moto,</summary>, andRead tool only. - It “corrected” that fake prompt in Chinese.
- The Chinese text leaked into the visible reply.
- It ended with an unrelated JSON Schema for TV listings, with fields such as
tvName,X-Files, andtvDayOfWk.
Why?
A chat model does one job. It predicts the next token from the prompt, the tool history, and patterns from training.
When the run grows long, the model must keep more weak signals in play. A repeated tool loop can then become the strongest local pattern in the prompt.
In this case, the repeated reads mattered. Each extra pass over the same file added more text and tool history, which increased the chance that the model would follow the wrong pattern.
Hallucination does not only mean a wrong fact. It also means fake instructions, fake tags, fake memory, and fake structure.
The invented tags show that kind of failure. The model did not receive <linux_note> or Read tool only as real control text. The model invented them and treated them as real.
Why Chinese?
GLM-5.3-flash is bilingual. That gives the model two language tracks that can both look valid during generation.
If the model stays on task, that is not a problem. If the model loses the task, either language can become the next continuation.
Training data can push that drift further. A long run can expose fragments of prompt formats, annotation styles, or question-and-answer patterns that resemble old training examples.
That is also how the TV schema appeared. The schema did not solve the task. The model surfaced a stored format that matched its broken local pattern better than the coding job.
The Context Meter
Each model has a maximum amount of context and for GLM 5.3 Flash this is 128k. As the context fills the chance of model collapse increases. Tarsk will usually handle this through compaction techniques (eg microcompaction) or full compaction. In this case though the model got into a runaway loop after around 50% context usage. The lesson here is to proactively click New Chat when you have completed a task. Only continue a conversation when then context of the conversation is needed, and recognise that the longer you chat back and forth: not only is the expense of tokens costing you but the chance of the model entering the “dumb zone” (where it makes mistakes or does a poor implementation) or having a model collapse is increased.
Taming the Toddler
A model needs a harness like Tarsk for a reason, to keep it on the rails and keep it working in a sane, more predictable way. This makes harness engineering a discipline that is really interesting and challenging, and in many ways it’s like baby sitting a toddler in their terrible twos!