Bunnyland RL administration

The RL add-on contributes a server plugin and a static admin dashboard. The server plugin adds RLControllerComponent, offline training jobs, saved model artifacts, model assignment, and /admin/rl/* API routes. The web bundle adds the /rl/ dashboard.

Use this guide to install the add-on, train models, assign controllers, and inspect saved artifacts.

Server plugin

The Python package exposes bunnyland_rl.bunnyland_plugins() and registers plugin id bunnyland.rl.

When loaded, it contributes:

RLControllerComponent for assigned RL controller state;
runtime controller registration that emits normal Bunnyland ToolCalls;
an admin-zoned HTTP contribution for service status, offline training jobs, saved models, bounded weight inspection, and controller assignment.

Consult the server OpenAPI document for concrete operations and payload schemas.

Load it with the stock Bunnyland server:

bunnyland serve --module bunnyland_rl

It can be loaded alongside the 3D add-on:

bunnyland serve --module bunnyland_3d --module bunnyland_rl

The plugin is default_enabled=True, so no separate --plugin flag is required once the module is importable. If a container or supervisor overrides the server command, keep --module bunnyland_rl in the final arguments.

Storage and tracking

Training jobs run offline arena copies seeded from the live world state. They do not submit commands to the live world.

Completed jobs write reloadable model metadata under BUNNYLAND_RL_DIR, which defaults to data/rl. Neural-network weights are stored separately as safetensors files under BUNNYLAND_RL_DIR/weights/.

Set BUNNYLAND_RL_WANDB=1 or normal WANDB_* settings to enable Weights & Biases tracking. Install the tracking extra when you want the optional wandb dependency available.

Dashboard

The web image copies the dashboard into /usr/share/nginx/html/rl.

Open /rl/ on the hosted Bunnyland origin and set the server field to /api. Sign in through the normal same-origin login flow; the dashboard uses the secure HttpOnly session cookie and does not store bearer credentials in browser storage. Browser password login is deliberately refused for cross-origin API URLs.

The dashboard can:

list playable characters;
list available base controller behaviors;
start offline training jobs;
show job status, reward and loss curves, action histograms, trust weights, checkpoint paths, and W&B links;
cancel active jobs;
list saved models;
preview safetensors weight layers;
assign a saved model to a character.

The /admin/rl/* routes require world:admin; serving the dashboard does not grant that scope.

Start a training job

Pick a character and a base behavior. The base behavior is used as the behavior overlay mode while the model learns. Choose a policy network, lens set, episode count, update count, and seed.

The default lens set is:

Lens	Purpose
`room_text`	Text about the current room.
`perception_text`	Text from the character's perception.
`stats_vector`	Numeric character stats.

Additional lenses may be available, such as components_vector and room_grid. Larger lens sets can expose more state but also increase training cost and model complexity.

Start small. Short jobs are useful for checking plumbing, artifact writes, and dashboard rendering before spending time on longer runs.

Assign a model

After a job completes, select a saved model and assign it to a character. Assignment writes RLControllerComponent to that character with the selected model id and runtime settings.

Watch the character in a client or inspector after assignment. Confirm it acts at the expected pace, emits normal queued commands, and can be claimed by a human when needed.

Build the images

Build the server image from the add-on repo:

docker build -f Dockerfile.server \
  --build-arg BUNNYLAND_SERVER_IMAGE=ghcr.io/thalismind/bunnyland-server:main \
  -t bunnyland-rl-server .

Build the web image with the sibling shared UI package as a build context:

docker build -f Dockerfile.web \
  --build-context bunnyland-ui-web=../bunnyland-ui-web \
  --build-arg BUNNYLAND_WEB_IMAGE=ghcr.io/thalismind/bunnyland-web:main \
  -t bunnyland-rl-web .

Local checks

Run all add-on checks from the repo root:

scripts/check

Run only the server plugin checks:

BUNNYLAND_SERVER_PATH=../bunnyland-server scripts/test-server

Run only the web checks:

scripts/test-web

Verify a deployment

After deployment:

Open /rl/ and connect to /api.
Confirm characters, jobs, and models load.
Start a short training job against a test character.
Confirm the job reaches a terminal state or can be cancelled.
Confirm a completed job writes JSON metadata and safetensors weights under BUNNYLAND_RL_DIR.
Assign a model to a test character.
Claim and release that character from a player client to confirm handoff still works.

Operational notes

RL is controller infrastructure, not a privilege layer. Model output still goes through normal Bunnyland action validation, queueing, costs, and rejection paths.

Keep saved model directories backed up if you want models to survive server replacement. Do not commit generated model artifacts, checkpoints, safetensors files, node_modules, or dashboard build output unless the repo's ignore rules explicitly call for them.