Brainby arc-labs/docs
Sdk pythonJobs resource
Python SDK

Jobs resource

Poll admin-plane jobs (reembed, reextract, hard_delete) via recall.jobs.get — job lifecycle, polling patterns, and backoff.

Jobs is a JWT-only resource. API keys are rejected with HTTP 403 on every jobs endpoint. Use the same control-plane JWT that authenticates recall.namespaces.

get()

GET/v1/jobs/:idJWTstable
def get(self, job_id: str) -> dict[str, Any]: ...

Fetch a single admin job by ID. Returns the public AdminJob projection — no internal fields like cursor or heartbeat_at are exposed.

Returns

The AdminJob shape (see Types for the full TypedDict):

{
  "id": "j_01HEX…",
  "namespaceId": "ns_…",
  "orgId": "org_…",
  "kind": "reembed",
  "status": "running",
  "totalEstimated": 12450,
  "processed": 4280,
  "failedItems": 0,
  "error": null,
  "createdBy": "u_…",
  "createdAt": "2026-05-08T12:00:00Z",
  "startedAt": "2026-05-08T12:00:03Z",
  "finishedAt": null
}
job = recall.jobs.get('j_01HEX…')
print(f"{job['kind']}: {job['status']} {job['processed']}/{job['totalEstimated']}")
job = await recall.jobs.get('j_01HEX…')

Job lifecycle

The status field walks through five states:

OptionTypeDefault / Env
queuedstr

The job is enqueued but no worker has claimed it yet. startedAt is null. Always the initial state from JobAcceptedResponse.

runningstr

A worker has claimed the job and is processing batches. startedAt is set; processed increments per heartbeat.

donestr

Terminal — every batch processed successfully. finishedAt is set.

failedstr

Terminal — the worker exhausted its retry budget on a batch. error carries a human-readable failure message.

cancelledstr

Terminal — operator cancelled the job. The worker honors cancellation at the next batch boundary.

The transition graph is queued → running → {done|failed}, with {queued, running} → cancelled as the operator-driven path.

Polling pattern

The simplest poll loop:

import time

job = recall.namespaces.reembed('ns_01HEX…')
job_id = job['jobId']

while True:
    status = recall.jobs.get(job_id)
    if status['status'] in {'done', 'failed', 'cancelled'}:
        break
    time.sleep(5)

print('Final:', status['status'], 'processed', status['processed'])
import asyncio

job = await recall.namespaces.reembed('ns_01HEX…')
job_id = job['jobId']

while True:
    status = await recall.jobs.get(job_id)
    if status['status'] in {'done', 'failed', 'cancelled'}:
        break
    await asyncio.sleep(5)

print('Final:', status['status'])

Backoff polling

For long jobs, exponential backoff reduces request load on the control plane without sacrificing responsiveness near the end:

import time

def poll(jobs, job_id: str, *, max_delay: float = 60.0) -> dict:
    delay = 1.0
    while True:
        status = jobs.get(job_id)
        if status['status'] in {'done', 'failed', 'cancelled'}:
            return status
        time.sleep(delay)
        delay = min(delay * 1.5, max_delay)

Start at 1 second, scale up by 1.5× per poll, cap at 60 seconds. For a ten-minute reembed this issues roughly 15-20 requests instead of 120.

Async backoff polling

import asyncio

async def poll(jobs, job_id: str, *, max_delay: float = 60.0) -> dict:
    delay = 1.0
    while True:
        status = await jobs.get(job_id)
        if status['status'] in {'done', 'failed', 'cancelled'}:
            return status
        await asyncio.sleep(delay)
        delay = min(delay * 1.5, max_delay)

Progress reporting

processed and totalEstimated together give you a progress fraction:

def progress(status: dict) -> float:
    total = status.get('totalEstimated') or 0
    return status.get('processed', 0) / total if total else 0.0

The totalEstimated count is captured at enqueue time. Concurrent writes to the namespace can change the underlying row count, so the fraction is approximate near the end. Once status == 'done', processed is the true final count.

Failure paths

When status == 'failed', inspect error for the failure cause:

status = recall.jobs.get(job_id)
if status['status'] == 'failed':
    print(f"Job failed: {status['error']}")
    print(f"  processed: {status['processed']}/{status['totalEstimated']}")
    print(f"  failed_items: {status['failedItems']}")

failedItems counts items the worker tried but couldn't complete (e.g. embedding-API errors on individual rows). The job continues past individual failures up to the worker's per-batch failure threshold; only when the threshold is exceeded does the job transition to failed.

failedItems is informational — the job's error message is the authoritative failure reason.

Cross-org safety

get() returns 404 Not Found for jobs in other organisations, never the actual job row. This means a stolen jobId from one tenant cannot leak details about another. The 404 surface is the standard RecallNotFoundError.

Job kinds

The kind field is one of three values, set at enqueue time:

OptionTypeDefault / Env
reembedstr

Re-runs the embedder for matching memories. Enqueued via recall.namespaces.reembed(...).

reextractstr

Re-resolves entity-name surface forms. Enqueued via recall.namespaces.reextract(...).

hard_deletestr

Hard-deletes soft-deleted rows older than retention. Enqueued via the retention CLI (no SDK method).

The set is fixed by a database CHECK constraint on admin_jobs.kind. New kinds require both a schema bump and a Postgres migration.

Errors

CodeStatusRetry
RecallAuthError401/403fatal

Missing JWT or API-key auth attempted on the admin plane.

RecallNotFoundError404fatal

Job not found, or out of org.

RecallServerError5xxfatal

Control plane unreachable. Auto-retried.

Was this page helpful?

On this page