A file manager on object storage: sharing, comments, thumbnails and a transcode worker

Dropbox-shaped files for member organisations on S3-compatible storage — streamed uploads to 6 GB, WebP thumbnails, HEVC→H.264 in a systemd worker.

2026 · storageapi

upload cap

6 GB

up from 500 MB, streamed

reduction

~136×

76.3 MB down to 560 KB

transcoded clip

1.4 MB

from a 274 MB 8K original

backend tests

300

behind sharing and comments

Context

MemberHub is one Django monolith giving a member organisation nineteen tools; Files is the plainest: per-organisation storage with nested folders, drag-and-drop upload, starring, a trash can, an activity feed and public share links.

The constraint is where the bytes live. Files are objects in an S3-compatible bucket, and the application in front of them is a single synchronous process operated by the person who wrote it — no task queue, no worker fleet, no second deployment. Everything added since has had to fit inside that: two image derivatives per upload, a video transcode step, shares restricted to invited people, comment threads on files, and a per-file ceiling that went from 500 MB to 6 GB.

One more piece of context: Files was the only tool that stored object keys and signed URLs at read time. Eight others, built by copying a pattern, stored the signed URL itself.

What I built

Four models, each carrying an organisation foreign key so tenancy is a property of every row: a folder with a self-referential parent, a file with its object key and metadata, a share link with a 43-character URL-safe token and an optional password, and an activity record covering fifteen actions.

Object keys are namespaced per organisation and randomly generated; the human filename lives only on the database row. Rename is therefore a single-column update with no storage traffic and move just repoints a foreign key — only copy touches the bucket. Deletion stamps a timestamp that every listing filters on; deleting a folder stamps its descendants, and restoring an orphan reparents it to the root.

Upload is where the machinery is. The browser posts the file through the reverse proxy to the application, which streams it into the bucket as a multipart transfer. Two WebP derivatives are generated inside the same request — a 400px thumbnail and a 1600px medium, never upscaled, never charged to the organisation's quota — and video is probed at the same moment, a codec that is not H.264 marking the row pending.

1. Uploadstreamed through the proxy into the bucket as a multipart transfer
2. Derivativesa 400px thumbnail and a 1600px medium, in the same request
3. Probea codec that is not H.264 marks the row pending
4. Transcodea timer claims one pending row every five minutes
5. Ready1080p H.264, preferred by previews

A worker fired by a timer every five minutes claims one pending row at a time, under a row lock that skips locked rows, transcodes to 1080p H.264 with the index at the front of the file, streams the result back and flips the row to ready. Previews prefer the transcoded copy and fall back to the original. The first complaint after it shipped was a video that played in one browser and not another — the bucket's cross-origin rules, not the codec. One browser hides a broken CORS configuration; the other exposes it.

Decisions

Store the key, sign at read time. The database never holds a download URL. Every response mints a fresh signed URL, and the signing helper refuses any key outside a prefix allowlist. Once the server will sign whatever key it is handed you have built a signing oracle, and the allowlist is the other half of that design.

Derivatives synchronously, transcode asynchronously. Two image resizes fit inside the request that uploaded the file. A video transcode does not, so it became a command and a timer, one file per tick, with no new dependency. The honest wart: a video upload can still spend tens of seconds inside the request making stills, an inherited choice nobody has ruled on.

A restriction is a stored flag, never a derived one. A share is open, or restricted to a named list of invited addresses who each get a personal magic link. If "restricted" meant "has recipients", removing the last one would silently republish a private album to anyone holding the link.

Lifting a revocation re-proves identity; it must never restore entitlement. Two reviewers found, from opposite directions, that clearing a revocation left the invitation intact — so a removed recipient could come back stronger than a staff member. The row is demoted instead, and the emailed magic link dies with it.

What went wrong

Eight tools stored a credential as an identifier. Cover images across the platform displayed fine for a week and then silently broke. Each field held a full signed URL, and a signed URL expires; the default was seven days, the maximum. The durable fact was the fifty-character object key; once one tool persisted the ephemeral thing instead, the next seven inherited it for free. The fix inverted the storage contract while keeping every field name and API shape frozen — no frontend changed — with input accepting either a bare key or a full signed URL. That also neutralised a second bug: edit forms echo fields back on save, so any update re-persisted whatever URL the client was holding. A third problem surfaced only in review: a migration had narrowed those columns to 512 characters, while signed URLs run 600 to 900 — it would have hard-failed for any tenant with an existing upload.

Half a pair of derivatives. The same defect appeared three times, in the upload path and again in the backfill command: a loop writes the thumbnail and then the medium, and a successful first write followed by a failing second leaves a real object in the bucket with an empty key on the row. The fix is per-key error handling, so whichever tier uploaded gets recorded. The same review caught something worse: the test guarding that release's highest-consequence rule passed during the red run, before the implementation existed, because the fixture made the wrong answer unobservable. A test that cannot fail is not coverage.

One percent-encoded character defeated the upload limit. Raising the ceiling to 6 GB meant letting the reverse proxy buffer multi-gigabyte bodies, so the large-body allowance and a three-slot concurrency limit were scoped to the upload endpoints — keyed, at first, on the raw request line, while the server matches locations on the decoded path. Encoding one character of the word "upload" reached the real upload view with the large body allowed and the limit unapplied, as did other real endpoints under the same prefix, anonymously. Two independent reviewers found it; the allowance and the limit now live only in locations matched on the normalised path. The lesson: a happy-path check on a security control is not verification.

Outcome

The thumbnails did what they were for: one folder of 22 photographs went from 76.3 MB of originals to 560 KB of thumbnails, about 136 times smaller for a grid of 80-pixel tiles, and the backfill reported 40 updated and 0 failed. The transcode worker turned a 274 MB, 22.86-second 8K clip into a 1.4 MB 1080p file of 22.87 seconds in roughly 200 seconds of background work.

The cap is 6 GB, proven end-to-end with a 1.2 GB upload landing in the bucket and every limit around it re-verified with a table of bypass attempts. Sharing and comments shipped behind 300 backend tests and browser harnesses on the upload-error path.

What is still owed is written down rather than hidden: the hard gate at upload time is the file count, not the storage quota, and every browse mints a fresh signed URL, so browsers re-download thumbnails on each revisit. Fixable by rounding the expiry to a fixed window; carved out deliberately for now.