Skip to content
RouteSpear

The router is not losing your traffic. It is declining to discuss it.

Everything a traceroute-style tool knows about an intermediate hop, it learned by asking that hop to talk about itself, out of a budget the hop's vendor rations deliberately. This makes per-hop loss ambiguous in a way that end-to-end loss simply is not.

A TTL-limited probe does not observe a router. It asks the router to notice that a packet expired and to generate an ICMP message saying so. That work happens on the control plane, on a CPU that has more important things to do, and essentially every router ships with a limiter capping how often it will bother.

So "hop 7 lost 30% of its probes" has two completely different readings. Hop 7 dropped the traffic. Or hop 7 forwarded every single packet and declined to comment on a third of them.

One of those is an outage. The other is a correctly configured router. Most tools render them identically, in red, which is how teams end up escalating to a carrier about a hop that was never broken, and how they learn to ignore the middle of the path entirely, which is worse.

Three ways of asking, so one silence proves nothing

A single channel cannot resolve the ambiguity, because the thing being measured is the same thing that decides whether to answer. RouteSpear gathers evidence that fails independently.

01

The hop itself

TTL-limited probes, which ask this specific router to generate a message about itself. This is the channel everyone measures, and the one that is rationed.

02

The hop, asked differently

Direct echo probes to the same address. A separate request type, metered by a separate limiter on the same box, so the two rarely fail together for the same reason.

03

Everything past it

The hops downstream on the same flow, and the destination as an anchor. If they answered, the packets that reached them went through the hop in question.

The third channel is the strong one, and it is the one most tools never consult. If hop 9 and hop 12 and the destination all answered, then packets went through hop 7 to get there. Whatever hop 7 says about its own probe loss, it forwarded the traffic.

Thirty percent at hop seven

What the window shows

Hop 7 probe loss
31%
Hop 8 probe loss
0%
Hop 9 probe loss
1%
Destination
Healthy, answering

Verdict: likely rate limited

Hop 7 crosses the degradation threshold on its own numbers. Before that becomes anything you are told about, the hops beyond it are checked: at least one downstream hop on the same flow has enough samples and is comfortably healthy.

That settles it. The traffic went through. The verdict is recorded on the hop, and the portal does not colour it as loss, because it is not loss, and a red hop that is not broken trains people to distrust the whole display.

Flip one number and the conclusion flips with it. Had hop 8 and hop 9 also shown 31%, no downstream evidence would exculpate hop 7, the verdict would be trusted, and you would be looking at the first hop of a real problem with a clear place to point.

The reasoning is attached to the result as machine-readable reason codes, not just a colour. You can see which downstream hops were considered and what they showed.

Five answers, including two kinds of 'I don't know'

Every hop, every window, gets one of these. Two of them are admissions, and having them is what stops the other three from being padded out with guesses.

Trusted
Corroborated by the other evidence available. Read the loss and latency figures at face value.
Likely rate limited
This hop looks degraded, but hops beyond it on the same route are healthy. Traffic demonstrably passed through it, so what is missing is its replies rather than your packets.
Policer divergence
The two independent ways of asking this router about itself disagree substantially. Two rate limiters are giving different answers, so the pessimistic one should not be believed on its own.
Insufficient data
Too few samples in the window, or the route changed part-way through it, so the statistics describe two different things stitched together. No verdict is offered.
No anchor
The destination itself could not be measured this window, so there is no end-to-end ground truth to check the intermediate hops against.

One clarification we insist on, because it would be easy and wrong to claim otherwise: the direct echo channel is not a window onto the data plane. It is a second control-plane channel that happens to be metered separately. When the two disagree, the conclusion is "two rate limiters disagree, distrust the pessimistic one", not "the data plane is fine".

And every measurement records how it was taken

Confidence is about the hop. Fidelity is about the instrument: what the agent was able to do at the moment it measured, carried on the record itself rather than inferred later from a fleet inventory.

Transport
Which probe mechanism produced this: raw ICMP, UDP with error queue, TCP, or an OS helper API. They hit different rate limiters and are not directly comparable.
Flow control
Whether the agent had full, partial or no control over the flow identifier. This is what tells you whether a fan-out width means anything.
Timing resolution
The smallest interval the clock on that host could distinguish. A sub-millisecond link delay claimed by a machine with a 15.6 millisecond tick is not a measurement.
Degraded, and why
A single flag with a human-readable reason when the agent had to fall back, a transport it could not open, a diversity it could not achieve.

Where this earns its keep

Six months from now somebody will compare a latency figure from a Linux agent in a data centre against one from a laptop on Windows and conclude something about the network.

The two numbers came from different transports, hitting different rate limiters, on clocks with different resolutions. The record says so. The comparison can be disallowed, or at least caveated, because the information needed to catch it is attached to the data rather than lost at collection time.

This is the part that cannot be retrofitted. A measurement stored without its provenance never gets it back.

Where we return nothing instead of something plausible

Per-link delay that does not survive the arithmetic

The obvious way to get the delay of one link is to subtract one hop’s round trip from the next. On real networks that goes negative routinely, because the return path from hop seven need not be the return path from hop eight. RouteSpear compares distributions across many samples on the same flow, and where the result is negative or smaller than the clock can resolve, it is published as indeterminate rather than as a number.

Statistics from a window that changed underneath

If the route moved mid-window, the samples before and after describe different paths. Rather than averaging across the seam, the window is marked insufficient and the baseline restarts.

Loss on a hop with no corroboration

When neither downstream hops nor the destination could be measured, there is nothing to check the hop against. That is reported as having no anchor, which is a different statement from the hop being fine.

A fan-out width the transport could not observe

Rather than reporting a width of one and letting you read it as "no parallel paths", the result carries the caveat that this transport cannot see fan-out and names the one that can.

And it knows not to make the problem worse

When a hop starts looking unhealthy, more samples would settle the question faster. So the agent can temporarily probe that hop, and its immediate neighbours, at several times the normal rate for a short burst, within a budget that is a fraction of the agent's overall ceiling and can never exceed it.

Except on hops whose verdict is likely rate limited or policer divergence. Those are the worst possible targets: probing a rate-limited router harder produces more apparent loss, which looks like escalation, which would justify probing harder still. The detector refuses them explicitly.

Which is the same idea as the rest of this page. The measurement is designed around the knowledge that the thing being measured has opinions about being measured.

Next: what happens when the network breaks