Python runtime

The opt-in CPython lane — off-tick, behind the python build tag

View as Markdown

Sulfur’s second plugin runtime embeds CPython 3.14 (via gopy, cgo). It exists for work that doesn’t belong on the tick: heavy aggregation, logging pipelines, external IO, ML — anything where you want real Python and its ecosystem, and where “eventually, off the tick” is the right latency contract.

The build tag

All cgo code lives behind the python build tag. The exported package surface is byte-identical across both builds, so the same server code compiles either way:

$# DEFAULT build — pure-Go static binary, CGO_ENABLED=0.
$# runtime="python" plugins are skipped gracefully at load (with a log line).
$go build ./...
$
$# OPT-IN build — embeds CPython (needs libpython 3.14 + CGO_ENABLED=1).
$go build -tags python ./...

The default binary’s import graph carries zero cgo — the pure-Go static build is preserved. The plugin host holds the Python runtime strictly by interface; nothing in the host or server packages imports gopy.

Writing a Python plugin

Same manifest, same register API — only the runtime selector and the file extension change:

1name = "heavylogger"
2version = "0.1.0"
3entrypoint = "main.py"
4runtime = "python"
5capabilities = []

Module-level state (like counts above) persists across hook calls — Python module globals are not frozen the way Starlark’s are.

The hot-path / off-tick split

This is the load-bearing difference between the two runtimes:

StarlarkPython
Hook dispatchInline on the tick goroutineSubmitted to a bounded off-tick worker pool
Latency contractSynchronous with the eventEventually, shortly after the event
Overload behaviorStep budget bounds each callA saturated pool drops the dispatch — the tick never blocks
ConcurrencyFresh isolated thread per callSerialized by the CPython GIL on pool workers

When a discrete event fires, the host offers it to every loaded Python plugin by submitting the hook (with the event’s payload marshalled to plain scalars) to the plugin pool and returning immediately. A Python hook never runs on the tick goroutine. Hooks the plugin never registered are cheap no-ops.

Because a saturated pool drops dispatches rather than queueing unboundedly, Python hooks must tolerate missed events under load. Use them for statistics, logging, and side channels — not for gameplay-critical logic. Gameplay logic belongs in the Starlark lane.

World access: the request bridge

A Python plugin gets four world builtins injected alongside register:

BuiltinCapabilityBehavior
set_block(x, y, z, state)world.writeConstructs a block-change request and enqueues it; the tick owner applies it.
spawn(x, y, z)entities.writeRequests a simple entity spawn, applied by the owner.
log(msg)noneRequests a log line attributed to the plugin.
block_at(x, y, z)world.readBlocks on an owner snapshot and returns the copied (state_id, ok) scalar.

Writes are asynchronous request producers — the owning tick goroutine applies them through the same seams the Starlark handles use, with the plugin’s manifest capabilities enforced on apply. No live tick-owned reference ever crosses into Python; only plain scalars and the request/apply indirection. Unknown capability strings still abort the load, exactly as in the Starlark lane.

Semantics summary

  • The plugin’s .py runs once at load; top-level register(event, fn) calls capture hooks. Re-registering the same event overwrites (last registration wins).
  • CPython is initialized once per process; every hook call acquires the GIL on its pinned worker goroutine for the duration of the call.
  • Unloading a Python plugin releases its captured callables and module globals.
  • On a server without the Python lane (default build), the plugin is skipped at load — a missing optional runtime is never a server-down condition.