Activity
01
Groundwork
5/5DoneWork out what to build, and give tuning its own place on the hub.
Updated
1. Groundwork
Steps 5 of 5
- done: Survey of existing run viewers and node tools
- done: Proposal for nodes and a live stream
- done: A separate space for tuning on the hub
- done: One job queue
- done: Three-seed test run
Done when Three seeds run in order.
Updates
The first test passed: three seeds ran in order.
Proposed after Claviger's first tuning runs on the hub. One seed misled an early report, so configs now run in threes.
02
First stream
5/5DoneGet a real tuning run onto the public site.
Updated
2. First stream
Steps 5 of 5
- done: Wrapper for any training command
- done: Runs reach the public site
- done: Live view takes run data
- done: A way to take a run back down
- done: Real item counts on each run
Done when A real run shows live on the bench and can be taken down.
Updates
The first run reached the public site and was taken down: its card showed more training items than it used. Runs now report their real count.
Wrapper built: it reads a training command's output and writes the numbers to a file, with no change to the code.
03
Instrument bench
6/6DoneMake the bench worth reading, live or quiet.
Updated
3. Instrument bench
Recordings page with 14 runs on one loss chart and a list of runs by date Done when Bench, Recordings and run pages are live in both themes.
Steps 6 of 6
- done: Three designs tried, Instrument picked
- done: Bench, Recordings and run pages
- done: Phases, end time, learning rate, speed and peak memory
- done: Power, temperature and per-GPU readings
- done: Light and dark, following the device
- done: One screen on desktop
Updates
The bench became one screen on desktop. The last three runs so far finished the same day.
A real Claviger run proved the whole path, with GPU power, temperature and memory on the public page in both themes.
The Instrument design went live, with Bench, Recordings and run pages.
Settled
- What the site isA general public view of what's on the bench.
- LookInstrument: a full-screen panel of readouts in a film-interface style.
- ThemeFollows each device, light or dark.
1 more
- Main readoutSegment digits, not plain type.
04
Bench refinements
1/5In progressTidy the bench and rework the run page.
- Pick the new visual direction for the front pageWaiting on Severian
1 to decide
Updated
4. Bench refinements
The live bench, quiet between runs, showing the last Claviger run Done when Smoothing and the new run page are live.
To decide
- Very small lossesShow 0.0000 as it is, or as a less-than mark.
Steps 1 of 5
- done: Smoothing on by default, built
- Put smoothing live
- Remove the footer
- Bench strip on the new front page
- Run page reworked as a dashboard
Updates
The Bench, where these runs are shown, now lives at citadel.severian.dev/bench; each run's own page keeps its address. A finished run with no eval now says "no eval" rather than "no eval yet", and a run's elapsed times no longer print over each other: when a stretch is too short to name in the bar, the times are listed under it.
Past training runs can now be published to the Bench in batches. The 41 earlier runs of Claviger and the Familiar deciders went public this way. The training machine's job runner also gained a watchdog: a training job that stops printing for its limit is stopped and recorded as failed, so one stuck job can no longer hold the queue for hours.
New training runs now go up on the public Bench by default and show live as they train, with GPU readings. The first was a run from the next Claviger round, on Nov 18.
1 older
Smoothing built: on by default, and every line but eval accuracy follows it. Not live yet.
05
Drop-on node
2/5Not startedStream runs from any machine used for tuning, not only the hub.
5. Drop-on node
Steps 2 of 5
- done: GPU readings with NVIDIA and AMD backends
- done: Hardware label on each run
- Node that installs as a package or a container image
- Test on one rented machine
- Runs from a node reach the bench
Done when A rented GPU machine streams to the bench after a short install.
Settled
- HardwareNodes will run on rented machines too, so readings can't assume one kind of GPU.
06
Agent access
0/2LaterLet an agent run a tuning job on a node.
6. Agent access
Steps 0 of 2
- Run a tune from an agent
- Recipes for common runs
Done when An agent runs a tune on a rented machine.
07
Open release
0/3LaterPublish the training wrapper as a small open tool.
- OK for the public repo and the first releaseWaiting on Severian
1 to decide
7. Open release
To decide
- NamesPalaemon is a working name, and the wrapper's name isn't final either.
Steps 0 of 3
- Final name
- Public repo
- First signed release
Done when The wrapper has a public repo and a signed release.
Settled
- Open partOnly the training wrapper goes public, under Apache-2.0. The rest stays in citadel.
By the numbers
- Runs on the bench
- 74
- Projects
- 13
- Configs
- 6
- Best eval accuracy
- 0.8385
- Last run
- Nov 20
- Public delay
- 2 min
Still open
- lowOther hardwareAMD's own sensor tools and multi-GPU runs are untested on real machines.
- lowBrowsersSafari and real phones are untested.
Parked
- ExtrasAgent notes beside curves, approved synthetic samples, and cost from provider APIs. Each only if there's demand.
Every update
The Bench, where these runs are shown, now lives at citadel.severian.dev/bench; each run's own page keeps its address. A finished run with no eval now says "no eval" rather than "no eval yet", and a run's elapsed times no longer print over each other: when a stretch is too short to name in the bar, the times are listed under it.
4 Bench refinementsPast training runs can now be published to the Bench in batches. The 41 earlier runs of Claviger and the Familiar deciders went public this way. The training machine's job runner also gained a watchdog: a training job that stops printing for its limit is stopped and recorded as failed, so one stuck job can no longer hold the queue for hours.
4 Bench refinementsNew training runs now go up on the public Bench by default and show live as they train, with GPU readings. The first was a run from the next Claviger round, on Nov 18.
4 Bench refinements- 14 days quiet
Smoothing built: on by default, and every line but eval accuracy follows it. Not live yet.
4 Bench refinementsThe bench became one screen on desktop. The last three runs so far finished the same day.
3 Instrument benchA real Claviger run proved the whole path, with GPU power, temperature and memory on the public page in both themes.
3 Instrument benchThe Instrument design went live, with Bench, Recordings and run pages.
3 Instrument benchThe first run reached the public site and was taken down: its card showed more training items than it used. Runs now report their real count.
2 First streamWrapper built: it reads a training command's output and writes the numbers to a file, with no change to the code.
2 First streamThe first test passed: three seeds ran in order.
1 GroundworkProposed after Claviger's first tuning runs on the hub. One seed misled an early report, so configs now run in threes.
1 Groundwork