← itcant.fail
falsehood verified reproduced

The magic bytes don't lie

T u e Jan 22 14:32:44 MET 1991 offset 4 3 bytes compared 29 bytes intended matched. confidently.

The belief that content sniffing is a property of the content: feed the same bytes in and you get the same type out, so a sniffed type can be trusted to route behaviour. A signature is only as specific as the rule that encodes it, and a rule that silently matches on a prefix will claim files it has no business claiming. The caller sees a confident answer, never a near miss.

Never let a sniffed type decide behaviour when a declared type is available, and when you must sniff, treat a match as a hypothesis: log what matched, and fail closed on an ambiguous one rather than acting on the first rule that fires.

Scores

Editorial judgements on a 0 to 10 scale, argued in the notes below and nowhere else. They order a list. They are not evidence and they never appear in a claim.

wtf
9
denial
7
damage
4

Shown by 1

Notes

THE SHAPE. A magic rule is a byte offset, a comparison and a label. Every part of its specificity lives in the comparison, and nothing in the output records how much of it actually matched. So a rule that was meant to compare twenty-nine bytes and ends up comparing three returns the same shape of confident answer as one that matched in full, and every caller downstream treats it the same way.

WHY THE CALLER CANNOT SEE IT. `file` prints a label, not a confidence. A near miss and an exact hit are indistinguishable at the call site, which is why this survives code review: the sniffing call looks correct, and the defect is in data that nobody reads.

SCORES, opinions, argued only here. wtf 9: the observable symptom, a program that works on six days out of seven, is as close to impossible as this catalog gets. denial 7: lower than the light-speed one, because most engineers will accept that sniffing is heuristic if asked directly; what they do not expect is that a heuristic can be day-dependent. damage 4: a printing path for one suite on one day a week, widely enough deployed to reach stable releases of a major distribution.

REPRODUCED, with the limits stated in the experiment: the magic-rule defect reproduces exactly, the end-to-end print failure does not, because the byte layout of the 2008 spool file is not established here.

Sources 2

file-formats · content-sniffing · parsing · escaping