Latency, tokens per second and token share

A chat that takes two minutes gives you nothing to work with. You cannot tell whether the model thought for a long time, whether one tool crawled, or whether the agent called twenty tools in a row. Tarsk records all three now, one click from the composer.

Open the numbers

The context button in the composer opens a popover titled Context. It shows how full the window is, splits the prompt into six categories, and lets you open the two panels behind the totals.

The categories read System prompt, Tool definitions, Rules, Skills, Subagent definitions and Conversation. The Conversation row opens Conversation Latency. The Tool definitions row opens a list of every tool the conversation sends, with the size of each one.

Read the latency timeline

Conversation Latency lists one row per recorded event, in the order the run produced them. Each row has an icon: a sparkle for a model response, a brain for a thinking span, a wrench for a tool call. The title names what ran, such as Model response · openai/gpt-5, Thinking or Run, and the right edge shows the time as minutes and seconds.

Hover a tool row and a summary appears: how many calls, the total time, and the share of the conversation. A failed call carries a red x labelled “Tool call failed”, and its time still counts.

A Summary table sits under the list with one row per model, thinking span and tool. Columns read Name, Latency, Calls and %. Rows sort by total time, so the biggest cost of the conversation reads first, and the shares add up to exactly 100.

Tokens per second per model

Settings, then Models, has a TPS column. The value comes from the last finished response: output tokens divided by the time that response took. Under 100 it shows one decimal, above it a whole number. The provider’s model list shows the same observation in short form, 42tps, with 42 tokens per second on hover.

Tarsk keeps one observation per model, stamped with the time it arrived. A single slow response moves the number, so read it as a recent sample of throughput on your account. A model with no recorded response falls back to the catalog average.

What the tool definitions cost

The tool definitions dialog puts the total in its title: Tool definitions · 12.3K tokens. Each row shows a tool name, its description, a Parameters disclosure with the JSON schema, and its share of the total in the form 1.2K Tokens (5%).

Tools from an MCP server group under the server name with a subtotal, ordered by weight, and built-in tools collect at the end. With no MCP server connected the list stays flat.

The estimates are rough, about four characters per token, used when the provider reports no usage. Read the percentage as a guide to which tools are heavy in the prompt.

Use the numbers

Start with the Summary table after a slow run. It answers the first question you have.

• One tool owns most of the time. That tool is the cost. Check whether it can do less work, or turn it off for this project.
• Thinking spans dominate. The model spends its time reasoning, and a faster model or a lower reasoning effort changes the picture.
• Many short calls fill the list. The overhead is in the number of round trips, so narrow the task or trim the tool set.
• Tool definitions take a large slice of the context bar. Open the dialog and look at the top rows with a high percentage. Those are the ones to switch off.

What the numbers do not cover

• Recording starts with a run. A conversation from before the feature shows “No model, thinking, or tool timings recorded for this conversation yet. Run the agent to collect latency data.”
• Tokens per second is a last observation, so treat it as a sample of your own traffic.
• Tool definition tokens are estimates, not provider usage.
• The latency dialog covers one conversation at a time. Open another chat and you get that chat’s timeline.

Try it

Run a chat that felt slow, then open Context and press Conversation. Read the Summary table, find the top row, and decide what to change. Then open Tool definitions and check whether the prompt carries tools the task never used.