Website thumbnails and link previews from code: capture, crop, cache

Directories, bookmark managers, link-in-bio pages, "sites using X" galleries and internal dashboards all need the same small thing: a picture of a web page, at a size that fits a card. Writing that yourself looks like a one-line Playwright call until you have a few thousand URLs, a few of which are dead, slow or behind a cookie banner.

This post builds a small thumbnail service in Python: one API call per five URLs, three cropped sizes per page (including a 1200x630 image for link previews), a cache so you do not re-capture pages needlessly, and sane handling for URLs that fail. We ran the code on 6 November 2026 against real sites. One caveat up front, repeated at the end: the tests called the public Pageframe Screenshots Actor, which is the engine behind our screenshot API, not the /v1/screenshot endpoint itself. The only difference in the script is where the image is downloaded from, and we mark it below.

Thumbnail versus link preview: pick the right shot

Two different jobs get called "thumbnail":

• A card thumbnail is a small picture inside your own UI, usually 16:9, 320 to 640 pixels wide.
• A link preview is the og:image other sites show when someone pastes your URL into Slack, X or LinkedIn. The common recommendation is 1200x630 (roughly 1.91:1) at a modest file size.

Both are best taken above the fold: a viewport-sized capture, not a full-page one. A full-page capture of a long landing page is a 1280x9000 image that becomes unreadable lint when scaled to 320 pixels wide. So the plan is to capture one 1280x720 viewport, then derive every size from that single image. One capture, three crops, one bill.

The capture call

The screenshot endpoint takes up to five URLs per request and returns one result per URL, each with a status. The options that matter here:

curl https://siftwright.com/v1/screenshot \
  -H "Authorization: Bearer $SIFTWRIGHT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com", "https://news.ycombinator.com"],
    "format": "jpeg",
    "fullPage": false,
    "viewportWidth": 1280,
    "viewportHeight": 720,
    "blockCookieBanners": true
  }'

fullPage: false is the important one. format: "jpeg" keeps the source image small; for thumbnails you will recompress anyway. blockCookieBanners (on by default) makes a best-effort attempt to dismiss consent dialogs, so your card shows the page and not a "We value your privacy" overlay. It is best effort: unusual banners can survive.

For a successful URL the result has status: "ok" and a signed fileUrl. The link downloads without your key and stays valid for 7 days, so download and store the image rather than hot-linking it. For a failure you get status: "error" and a message, and the capture is not counted against your plan.

The script

Install Pillow (pip install pillow) and set SIFTWRIGHT_API_KEY. The whole thing:

import hashlib, io, json, os, time, urllib.request
from pathlib import Path
from PIL import Image

API = "https://siftwright.com/v1/screenshot"
SIZES = {"sm": (320, 180), "md": (640, 360), "og": (1200, 630)}
CACHE = Path("thumb-cache")
TTL = 24 * 3600            # re-capture after a day
RETRY_AFTER = 3600         # don't hammer a URL that just failed

def capture(urls, headers=None, **opts):
    """One API call for up to 5 URLs. Returns the results list."""
    body = {"urls": urls, "format": "jpeg", "fullPage": False,
            "viewportWidth": 1280, "viewportHeight": 720, **opts}
    req = urllib.request.Request(API, json.dumps(body).encode(),
        {"Authorization": f"Bearer {os.environ['SIFTWRIGHT_API_KEY']}",
         "Content-Type": "application/json", **(headers or {})})
    with urllib.request.urlopen(req, timeout=300) as r:
        return json.load(r)["results"]

def download(url, headers=None):
    req = urllib.request.Request(url, headers=headers or {})
    with urllib.request.urlopen(req, timeout=60) as r:
        return r.read()

def cover(img, size):
    """Scale to fill the box, then crop from the top: page headers carry the brand."""
    w, h = size
    s = max(w / img.width, h / img.height)
    img = img.resize((round(img.width * s), round(img.height * s)), Image.LANCZOS)
    return img.crop((0, 0, w, h))

def key(url):
    return hashlib.sha256(url.encode()).hexdigest()[:16]

def fresh(url):
    meta = CACHE / f"{key(url)}.json"
    if not meta.exists():
        return False
    m = json.loads(meta.read_text())
    age = time.time() - m["at"]
    return age < (TTL if m["ok"] else RETRY_AFTER)

def thumbnails(urls, capture_fn=capture, download_fn=lambda res: download(res["fileUrl"])):
    CACHE.mkdir(exist_ok=True)
    todo = [u for u in dict.fromkeys(urls) if not fresh(u)]   # dedupe, skip cache hits
    for i in range(0, len(todo), 5):                          # the API takes 5 URLs per call
        for res in capture_fn(todo[i:i + 5]):
            k = key(res["url"])
            meta = {"url": res["url"], "at": time.time(), "ok": res["status"] == "ok"}
            if meta["ok"]:
                img = Image.open(io.BytesIO(download_fn(res))).convert("RGB")
                for name, size in SIZES.items():
                    cover(img, size).save(CACHE / f"{k}-{name}.jpg", quality=80, optimize=True)
            else:
                meta["error"] = res.get("error", res["status"])[:120]
            (CACHE / f"{k}.json").write_text(json.dumps(meta))
    return {u: (CACHE / f"{key(u)}-md.jpg") if (CACHE / f"{key(u)}-md.jpg").exists() else None
            for u in urls}

if __name__ == "__main__":
    import sys
    for url, path in thumbnails(sys.argv[1:]).items():
        print(url, "->", path or "failed")

A few design choices worth explaining:

• Crop from the top, not the centre. A site's logo and headline sit at the top. Centre-cropping a 16:9 capture into a different aspect ratio chops the brand off.
• Deduplicate before calling. dict.fromkeys removes repeats while preserving order. If your input is scraped from many pages, repeats are common, and every duplicate would be a paid capture.
• Cache successes and failures differently. A good thumbnail is reused for a day. A failure is remembered for an hour, so a dead domain in a 10,000-row list does not trigger a retry on every run.
• The key is a hash of the URL. Normalise URLs first (lowercase host, strip tracking parameters) if you want example.com/?utm_source=x and example.com/ to share a thumbnail.

What we measured

We ran the script on eight inputs: seven distinct sites, one repeated, and one deliberately dead domain (no-such-domain-siftwright-test.invalid). Here is what happened on the first run:

File sizes for the Hacker News front page, at JPEG quality 80: 8 KB at 320x180, 29 KB at 640x360 and 92 KB at 1200x630. The other sites were between 8 KB and 32 KB at the middle size; sparse pages compress much better than dense ones. We also confirmed a plain HTTP 404 (httpbin.org/status/404) is reported as an error ("Page responded with HTTP 404") and not captured, which is what you want for a gallery: a 404 page makes a bad thumbnail.

In the same test, github.com came back as an 8 KB image at the middle size. That was a mostly plain page at the capture time, which is a reminder that thumbnails show whatever the page looks like at that moment, including interstitials, empty states and A/B variants. Spot-check a sample of your output before shipping a gallery of 10,000.

Using the image as a link preview

Once you have the 1200x630 file on your own storage or CDN, the meta tags are short:

<meta property="og:image" content="https://cdn.example.com/thumbs/3f2a9c1d4b7e8a60-og.jpg">
<meta property="og:image:width" content="1200">
<meta property="og:image:height" content="630">
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:image" content="https://cdn.example.com/thumbs/3f2a9c1d4b7e8a60-og.jpg">

Use an absolute HTTPS URL. If your pages are rendered per URL (a directory entry per site, say), generate the tag from the same key(url) the script uses. Platforms cache previews aggressively, so changing the image later may not update an already shared link.

A thumbnail of someone else's site as a preview for your directory page is common practice, but check the usage terms of what you display, and do not capture pages behind logins or pages you are not entitled to republish.

Run it as a nightly job

For a directory that adds a few hundred sites a day, a cron entry is enough. Pipe URLs from a file, and the cache makes reruns cheap:

xargs -n 5000 python thumbs.py < new-urls.txt

Cache-friendly habits that keep the bill predictable:

1. Capture on first sight, refresh on a schedule. Weekly is plenty for most directories. Do not recapture on every page view.
2. Queue captures; never capture in a request handler. A capture takes from a fraction of a second to a few seconds. Show a placeholder and fill the thumbnail in when the job finishes.
3. Store the failure reason. The cache metadata above keeps error, so you can show "site unreachable" in an admin view and drop persistent failures from the list.
4. Respect rate limits. The API allows 60 requests per minute per key. At five URLs per call that is up to 300 pages a minute, far above what a nightly job needs.

Cost

On the API, plans start at $29 per month for 10,000 successful results, so a thumbnail costs about $0.003 on a full Starter plan, and less on larger tiers. Failed captures do not count. If your volume is occasional, the Pageframe Actor is pay-per-result at $1.50 per 1,000 captures with no subscription. Our pricing page has a calculator, and the earlier post on pay-per-result versus subscription explains when each one wins.

If you only need a handful of images, skip the code: the free website screenshot tool does single captures in the browser.

Limits to know about

• Public pages only. There is no login or interaction step. For pages that need clicks or cookies, run Playwright yourself. Our screenshot API guide covers when that is the better choice.
• Some sites block automated browsers. Expect a small share of URLs to return errors or challenge pages. The script's failure cache keeps that from becoming a retry storm.
• Thumbnails go stale. A redesign is not reflected until the cache expires.
• What we did not test. The /v1/screenshot call with a key was not re-run for this post. We ran the same engine through the public Actor, whose results use the same fields (url, status, fileUrl, error) and whose image links need your Apify token for the download instead of being pre-signed. That one difference is why thumbnails() takes capture_fn and download_fn parameters. Everything else, including batching, cropping, caching and failure handling, ran as shown.

Get a key and try it

The screenshot endpoint, the Actor and the free tool all use the same renderer. If you want thumbnails for a list of URLs today, start from the Siftwright API plans and the docs, paste the script above and point it at your list. Questions or edge cases we should cover next: support@siftwright.com.