Consume Events by Topic#

Monitoring a single run works well when you are following one OpRun. But if your service submits many runs and just needs to know when each one finishes, polling every run individually does not scale.

For that case, Gyoza groups events into a topic feed: every run that declares a topic publishes its events to a shared stream that you can tail from a single connection, with a cursor so you never re-read what you have already seen.

This is a pure pull model over the REST API: your service connects to the gyoza server, no broker or fixed callback URL required.

How it works#

Consuming events reliably is two layers, and it helps to keep them separate in your head:

1. The feed (fast path) — a cursor-based stream of every event from the runs that share a topic. Use it to follow progress and completion in near real-time.

2. Reconciliation (the durable guarantee) — listing runs by topic and state. A run’s state is the source of truth and never disappears, so a periodic reconciliation pass guarantees you never miss a finished run, even if the feed dropped an event.

Note

The feed is best-effort: under heavy concurrency an event may arrive late or be skipped. That is fine — the reconciliation pass is what guarantees you will always learn that a run finished. Do not rely on the feed alone for that guarantee.

Set a topic on your runs#

Events are only published to a feed when the run declares an event_delivery.topic. There are two ways to set it.

Per OpDefinition — every run created from the definition inherits the topic. Set it when registering the definition (see Deploy a Gyoza Op):

{
  "id": "classify-op",
  "version": "1.0.0",
  "event_delivery": {"topic": "classify-batch"}
}

Per ad-hoc run — set the topic directly when creating a run with POST /runs:

curl -X POST http://gyoza-server:5555/runs \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $GYOZA_API_KEY" \
  -d '{
    "image": "myregistry/classify-op:1.0.0",
    "inputs": {"image_path": "/remote/path/photo.jpg"},
    "event_delivery": {"topic": "classify-batch"}
  }'

Every run that shares classify-batch will land in the same feed, regardless of how many you submit.

Tail the feed#

Read the feed with the topic and an optional cursor:

GET /events?topic={topic}&after={cursor}&limit={n}

topic: Required. The event delivery topic to read.
after: Optional. Cursor from a previous page. Omit it to start from the beginning of the feed.
limit: Optional. Maximum number of events to return (1–500, default 100).

Start without a cursor to get the first page:

curl "http://gyoza-server:5555/events?topic=classify-batch" \
  -H "X-API-Key: $GYOZA_API_KEY"

The response is a page of events, oldest first, each tagged with the run_id it belongs to:

{
  "events": [
    {
      "cursor": "6650a1f0c3a2b1d4e5f60718",
      "run_id": "a1b2c3d4-...",
      "attempt": 1,
      "type": "PROGRESS",
      "msg": 42,
      "t": "2026-03-02T20:00:05Z",
      "state": "RUNNING",
      "payload": {}
    },
    {
      "cursor": "6650a1f5c3a2b1d4e5f60719",
      "run_id": "e5f6a7b8-...",
      "attempt": 1,
      "type": "COMPLETED",
      "msg": "done",
      "t": "2026-03-02T20:00:12Z",
      "state": "COMPLETED",
      "payload": {}
    }
  ],
  "next_cursor": "6650a1f5c3a2b1d4e5f60719",
  "has_more": false
}

Persist next_cursor and pass it as after on the next call to resume exactly where you left off:

curl "http://gyoza-server:5555/events?topic=classify-batch&after=6650a1f5c3a2b1d4e5f60719" \
  -H "X-API-Key: $GYOZA_API_KEY"

A typical consumer loops like this:

cursor = load_saved_cursor()          # or None on first run
loop:
  page = GET /events?topic=...&after=cursor
  for event in page.events:
    if event.type in (COMPLETED, FAILED, CANCELLED):
      handle_finished(event.run_id, event.type)
    elif event.type == PROGRESS:
      update_progress(event.run_id, event.msg)
  cursor = page.next_cursor
  save_cursor(cursor)                 # survive restarts
  sleep(a_few_seconds)

Because the cursor is persisted, a restart of your service just resumes the feed from the last acknowledged event — nothing is replayed and nothing in between is lost.

Never miss a completion#

The feed is fast but best-effort. To guarantee you always learn that a run finished, reconcile against run state by listing runs for your topic:

GET /runs?topic={topic}&state={state}

For example, to fetch every completed run of the topic:

curl "http://gyoza-server:5555/runs?topic=classify-batch&state=COMPLETED" \
  -H "X-API-Key: $GYOZA_API_KEY"

Run this periodically (and on startup) for the terminal states you care about — COMPLETED, FAILED, CANCELLED — and cross off any run you were still tracking as in-flight. Because a terminal state is permanent and stored on the run itself, this pass can never leave you believing a finished run is still running.

Tip

Treat the two endpoints as complementary: the feed keeps you up to date in near real-time, and the reconciliation sweep is the safety net that closes any gap. Together they let an external service track hundreds of runs from a single connection without losing a completion.