pornboxdBETA
Devlog

Two new studios, a manual backfill tool, and one ugly URL fix

Today was catalogue-expansion day. Two more studios both publish JSON catalogues, so neither needed an Apify scraper, they plugged into the same writer

By Mortimer CockburnApril 12, 20261 min read

Today was catalogue-expansion day. Two more studios both publish JSON catalogues, so neither needed an Apify scraper, they plugged into the same writer pipeline the first JSON-feed studio uses, just with a different feed-shape mapper (about forty lines of code each). The feed auto-detects based on the first item's field names, so adding a third feed shape later means writing one more mapper, not redesigning the import path. A new nats_platform column on the studios table lets multiple studios share the same /nats postback URL but still attribute conversions to the right account, different networks get different platforms.

The manual-backfill path for the missing dates came in as a Tampermonkey userscript instead of an automated scraper. You open a catalogue page in your browser, paste the userscript, a floating panel extracts every video's metadata from the already-rendered Vue.js HTML, and clicks "Send to API" to POST the batch to a new POST /admin/videos/bulk-update endpoint. Since the script runs in your browser with your real session cookies and residential IP, there's no Cloudflare challenge to fight, it's the same HTTP requests a normal visitor makes, just capturing the data. Roughly 100 videos per page, a few seconds to extract, idempotent (only fills NULL fields, safe to re-run). The admin panel also got a proper /admin/videos page with filters (missing date, studio, search) and an inline release-date editor, one click edits a date, the row flashes green on save, and auto-removes from "Missing date" view.

The ugly URL fix: two studios each had a video with slug outdoors, and the old routing couldn't tell them apart. Video URLs are now namespaced under the studio, /videos/studio-a/outdoors vs /videos/studio-b/outdoors. A UNIQUE (studio_id, slug) constraint on the videos table enforces it. On the image pipeline side: added fetch timeouts and a per-run timeout with an admin "Terminate" button, because a few stuck image downloads were tying up the image processor indefinitely.