Skip to Content

Photo handling

How it works, in a paragraph. A photo is checked twice, in two places, for two different reasons. The browser converts HEIC and rejects what it can measure cheaply, so the visitor learns about a bad photo before spending an upload. The server then distrusts all of that and checks again from the bytes, because a browser check is a courtesy to an honest visitor and no obstacle at all to anyone else. The last step re-encodes the image, and that step is what removes the metadata phones attach.

The order is the design

Each step is cheaper than the next and rejects input the next would have had to handle:

  1. Convert first, validate second. A HEIC and the JPEG it becomes are different files — the JPEG is typically several times larger — so checking the HEIC’s size and then uploading the JPEG could hand the API a file it will refuse. Everything after conversion runs on the bytes actually being sent.
  2. Sniff the real format, not the declared one. A filename and a MIME type are claims; both cost nothing to forge.
  3. Cap the dimensions from the header, before a full decode. A forty-megapixel JPEG is a few hundred kilobytes on the wire and gigabytes decoded — a denial of service that passes a byte-size check comfortably.
  4. Re-encode. See below.

HEIC cannot be decoded on the server

iPhones shoot HEIC by default, so this is most of the expected traffic, and it was the largest single launch risk on this feature.

The image library on the server bundles HEIF container support with an AV1 decoder and no HEVC one. iPhone HEIC is HEVC-coded. The failure is therefore late and misleading rather than early and clear: the header parses, reporting the true dimensions and the codec, and then the pixel decode dies reading past the end of the file. Measured on real device photos, from a buffer and from disk, with every combination of decode options — identical failure every time.

So the conversion happens in the browser, and that is the better place regardless of what the server could do. The phone has hardware HEVC support and converts in milliseconds. A software HEVC decoder on the server would be seconds of CPU per photo spent before the daily allowance is reserved — an unmetered CPU cost on an endpoint whose whole purpose is to be cheap and bounded.

Three layers, cheapest first:

LayerDoes
The file picker’s accepted typesiOS transcodes to JPEG when the picker is not offered HEIC — most iPhone traffic, no dependency at all
A converter loaded on demandHandles a HEIC that arrives anyway, from Android or a synced desktop library. Loaded on the first HEIC only, because it carries a decoder measured in megabytes
The server refuses HEVC-HEIF, with its own reasonThe last line, so a client that skips both gets an answer it can act on rather than a decoder error

The accepted-types list is a hint, not a gate — drag-and-drop ignores it — which is why the HEIC check runs before the type check rather than after it. A HEIC that slips past the picker must reach the converter, not be rejected as an unknown type.

The container is checked by codec, not by brand. mif1 is a generic marker that both HEIC and AVIF files carry, so the brand genuinely cannot tell you what the payload is. Deciding on it would reject real AVIF uploads, which decode perfectly, while still letting mif1-branded HEVC through to fail later — exactly backwards.

The re-encode is the privacy step

Nothing here relies on a library that strips metadata. The output is built from pixels rather than edited from the input, so every ancillary block is dropped by construction — EXIF, the ICC profile, and the GPS coordinates phones write by default.

One piece of metadata must be honoured before it is thrown away: orientation. Phones record portrait photos as landscape plus a rotate flag. Strip the flag without applying it first and every portrait photo silently arrives sideways — which is both wrong and very hard to notice in review.

The long edge is capped well below what a phone produces. Beyond that the extra pixels cost provider tokens and add nothing: the question is a coverage pattern across a scalp, not fine texture.

What the visitor is told

Every refusal maps to one short reason the capture step renders — type, size, dimensions, heic. Reasons rather than prose, so the interface is not matching on a message a translation could change. heic is separate from type because the photo is fine and the failure is ours; the message says “save it as a JPG” rather than implying they chose a bad file.

Every failure is a client-side rejection or a 400. That matters more than it looks: an earlier version let a decode error escape as a 500, which presented a visitor with a server fault over input that was entirely theirs to fix.

Last updated on