Gyoza Worker#

A worker is what actually runs your OpRuns. The server never executes containers itself, instead it delegates all execution to workers. You can connect as many workers as you need to a single server.

Workers talk to the server through heartbeats, each worker periodically tells the server what resources it has available, and the server uses that information to decide which pending runs to send its way. When a run comes in, the worker spins up the container, runs it, and reports the result back.

Resource management#

On startup, each worker detects how much CPU, RAM, and GPU VRAM it has and includes that in every heartbeat. The server tracks what each worker is running and computes its free resources from there, it only assigns a run if the worker has enough room for it. Each run holds its declared resources for its entire duration, so the worker never gets overloaded.

If a worker stops sending heartbeats, the server eventually evicts it from the pool and reassigns any work it was holding to other workers. After the connection is reestablished, the worker will be re-added to the pool and start receiving new runs.