The photo stream of my old Astro site needed alt text, titles and camera data for every picture. The script scripts/vision.ts wrote them into a JSON sidecar next to each JPG. This page collects three posts from March 2026: where the script runs, how it gets through rate limits, and its move from OpenAI to Claude.
What the script does
For every JPG without a .json sidecar it:
- reads the EXIF data with
exiftool(camera, lens, aperture, ISO, focal length, shutter speed, GPS); - sends the image to a vision model for alt text, title ideas and tags;
- writes both into a JSON file next to the image.
The sidecar drives the photo pages: alt text for accessibility, titles for the detail page, EXIF for the metadata panel.
Run it where the originals are
At first the VPS generated the sidecars itself. The deploy worked, but no sidecars arrived. The script read the photos in the repository, while the deploy synced a different folder: the photo library in my Obsidian vault on the Mac. And rsync cannot upload files that were never created in the folder it syncs.
The fix was to treat the VPS as a target only. The Mac has the originals, the API key and exiftool, so it generates the sidecars, and the server receives finished files. vision.ts got an optional source folder, and the deploy script runs it first:
pnpm run vision -- "$PHOTO_SRC"
rsync -az --delete \
--include='*/' \
--include='*.md' --include='*.mdx' \
--include='*.jpg' --include='*.jpeg' --include='*.JPG' --include='*.JPEG' \
--include='*.json' \
--exclude='.DS_Store' --exclude='*' \
"$PHOTO_SRC" "$VPS:$REMOTE_PHOTOS/"
After that the server pulls the repository and rebuilds the container with podman-compose. The VPS no longer needs an API key, exiftool or Node for metadata.1
Rate limits on large batches
The first run over a whole trip's photos died on 429 Too Many Requests, with $3.16 of credit left. A positive balance does not help against the tokens-per-minute limit of the account. The script already had bounded concurrency and retries with exponential backoff, so the job was to choose values that keep it moving:
pnpm vision -- --concurrency=1 --retries=10 --backoff-ms=2000
Start there, watch the log for 429 retries, and go to --concurrency=2 only after a clean run. Above 2, on an account without a higher limit tier, the script spends more time in backoff than it saves. For more throughput, raise the tier. Still open at the time: downscaling the images before upload (1024 px on the long edge is enough to describe a scene), which saves tokens and time.2
From OpenAI to Claude
The move to the Claude API mostly changed the request; EXIF, concurrency, batching and the command-line flags stayed the same.
- Package and key:
@anthropic-ai/sdkinstead ofopenai,ANTHROPIC_API_KEYinstead ofOPENAI_API_KEY. - Client:
new Anthropic({ maxRetries: 0 }), because the script has its own backoff loop and two retry layers waste quota. - Structured output: OpenAI used
response_formatwith ajson_schemaand aJSON.parseafterwards. Claude gets a toolvision_responsewith aninput_schema(title_ideas,description,tags) andtool_choice: { type: "tool", name: "vision_response" }. The answer arrives astoolUseBlock.input, an object, without parsing. - Image block: OpenAI takes a data URL in
image_url; Anthropic takes asourceblock withtype: "base64"and its ownmedia_type. - Rate limits: instead of checking a raw
statusand parsing the error text,error instanceof Anthropic.RateLimitError, with the wait time from theretry-afterheader of the exception.
function extractRetryAfterMs(error: unknown): number | null {
if (!(error instanceof Anthropic.RateLimitError)) return null;
const retryAfter = error.headers?.get("retry-after");
if (retryAfter) {
const seconds = Number.parseFloat(retryAfter);
if (Number.isFinite(seconds) && seconds > 0) return Math.ceil(seconds * 1000);
}
return null;
}
The model was claude-opus-4-6. For a batch that runs before a deploy, quality mattered more than speed.3
In current versions of the SDK, error.headers is a fetch Headers object, so the value has to be read with .get("retry-after"). An index access like headers["retry-after"] returns undefined, and the script then falls back to its own backoff without notice.4
Takeaways
- Preprocessing has to run against the folder the deploy syncs, not a copy in the repository.
- Work that needs keys and tools belongs on the machine with the originals; the server only needs what it serves.
- A 429 is usually the throughput limit, not the balance.
-
Generating Photo Sidecars Locally Before a VPS Deploy, 14 March 2026. ↩
-
Vision Rate-Limit Runbook for Photo Sidecars, 15 March 2026. ↩
-
Migrating the Vision Script from OpenAI to Claude, 19 March 2026. ↩
-
Anthropic TypeScript SDK,
APIErrorinsrc/core/error.ts, as of November 2026. ↩
- Topic
-
- AI / Language Models What I use language models for: agents, local models, RAG and small programs in which a model handles one task.
- Uses
-
- Claude API Anthropic's programming interface to the Claude models. Billed separately from a claude.ai subscription, with keys from the Claude Console.
- Mentioned in
-
- Astro Justified Gallery Layout My npm package @altner/astro-justified-gallery-layout: a small justified gallery for Astro with EXIF, LQIP previews, a virtual variant and a lightbox.
- Building the Photo Stream The photo stream of my old Astro site: JSON sidecars next to each JPG, import.meta.glob, a justified grid and batches revealed on scroll.
- My Old Astro Website My old site at adrian-altner.de: Astro with the Node adapter, typed content collections, IndieWeb features and photos as files with JSON sidecars.
- Syndicating Photos to Flickr A script that makes Flickr match the photo tree of my old site: upload, delete, move between albums, and write the Flickr ID and stats back into the sidecars.
- Tags