Fetch the pages your ingestion job reads without the SSRF risk

Where it goes wrong

Ingestion runs are long, unattended and repetitive, so one unsafe fetch is repeated across a whole corpus. When extraction output looks wrong, the first question is which page was actually retrieved — and a bare fetch() answers none of it.

What Backfire handles

• HTML, JSON, XML and PDF bytes all arrive through one envelope, with the encoding declared.
• Blocked and unresolvable targets return a code your job can skip instead of retrying blindly.
• Enforced timeouts stop one slow host from stalling an entire ingestion run.
• Each fetch is logged against your key, so a bad batch can be traced after the fact.

The request

import os, requests

def fetch(path):
    envelope = requests.post(
        "https://backfireapi.base44.app/functions/forward",
        headers={"Authorization": "Bearer " + os.environ["BACKFIRE_KEY"]},
        json={"url": path, "method": "GET", "timeout_ms": 12000},
        timeout=30,
    ).json()

    if not envelope["ok"]:
        return {"skip": envelope["error"]["code"]}

    body = envelope["body"]
    if body["encoding"] == "text":
        return {"text": body["text"]}
    return {"skip": body["encoding"]}

All use cases