CITADEL
StandbyNothing is live right now.
Eval accuracylast measured Wed 18 Nov, 03:00 UTC
%
0.9180459 of 500 eval, 41 wrong
▲ +75.40 pts across 4 evals▲ +0.80 pts on the eval before
0%25%50%75%100%0%25%50%75%100%0%25%50%75%100%

Familiar decider: ModernBERT-base (backfilled)

finishedRadeon 8060S

One of the small models that decide whether Familiar, a desktop companion, reacts to something happening on the PC, and how. Backfilled: recorded after the fact from the run's own logs and records. Times inside a run are estimated, and there are no GPU sensor readings.

started
Wed 18 Nov, 02:41 UTC
ran for
14:43
config
learning rate 0.00005, batch 16, seed 2 of 3

run5/5step1125/1125epoch3/3config3/3seed2/3lr0.00005batch16modelModernBERT-base 149Mrev8949b90items6000+500

Eval measures

Elapsed

294.6 s

epoch interval

294.6 sto eval 2294.6 sto eval 3294.6 sto eval 4

294.6 s to eval 2, 294.6 s to eval 3, 294.6 s to eval 4

1125 of 1125 steps, 3 of 3 epochs, 100%

0.79 s per step, 76.4 steps a minute

Eval accuracy by eval

+0.0080

▲ on the eval before

eval 1 0.1640, eval 4 0.9180

Against the other runs

1 of 5

by last eval accuracy

best so far 0.9180, run 5

▲ +0.0240 against run 4

Set-up

6000 train, 500 eval

synthetic, learning rate 0.00005, batch size 16, epochs 3, max length 512, precision bf16-mixed, loss ce, optimizer adamw, scheduler linear, seed 2

Seed spread, observed

0.0300
0.850.95

3 finished seeds

Input

  • Runs128 runs
  • Agentsnone showing
  • Treesnone showing
  • Projectsnone showing

Check

Every entry is checked on its way out; some are held for review.

Not shown on any run: data samples, predictions, raw logs, file paths, machine names

Delay

2 min, set

Watching the agents work, about 2 min behind.

One way

Nothing here reaches the agents.

This page

This page updates by itself.

visits counted with Google Analytics · choose

Runsheight is eval accuracyall recordings ▸ | finished× failed■ stopped○ lost● live newer ▸

5 of 5 finished, 0.9180, config 3 of 3 (learning rate 0.00005, batch 16), seed 2 of 3, Wed 18 Nov, 02:41 UTC

▸
Runs as a table
StartedStatusEval accuracyStepsConfig
Wed 18 Nov, 02:41 UTC| finished0.91801125 of 1125learning rate 0.00005, batch 16, seed 2 of 3
Wed 18 Nov, 02:22 UTC| finished0.89401125 of 1125learning rate 0.00005, batch 16, seed 1 of 3
Tue 17 Nov, 23:55 UTC| finished0.88801125 of 1125learning rate 0.00005, batch 16, seed 0 of 3
Tue 17 Nov, 23:36 UTC| finished0.86401125 of 1125learning rate 0.00002, batch 16, seed 0
Tue 17 Nov, 23:17 UTC| finished0.85801125 of 1125learning rate 0.0001, batch 16, seed 0

Plate

Model
answerdotai/ModernBERT-base, 149M parameters revision 8949b90
Data
Familiar decisions, shared draw, synthetic6000 train500 eval
Config
learning rate 0.00005, batch size 16, epochs 3, max length 512, precision bf16-mixed, loss ce, optimizer adamw, scheduler linear, seed 2
Hardware
Radeon 8060S, ROCm

Log

Since startMark
0.0 sphase starting
0.0 seval 1
294.6 seval 2
294.6 sepoch 1
589.2 seval 3
589.2 sepoch 2
883.8 seval 4
883.8 sepoch 3
10 h 11 min laterwent public

Seeds

SeedStatusEval accuracy
seed 0| finished0.8880
seed 1| finished0.8940
seed 2| finished0.9180

observed spread 0.0300, 3 finished seeds

Readings

StepSecondsEval accuracy
00.00.1640
375294.60.7940
750589.20.9100
1125883.80.9180

Worked out

ReadoutValueWorked out from
Run complete100%1125 steps over 1125
Step time0.79 s294.6 s between epoch marks, over 375 steps an epoch. The reading interval holds the eval, so it is left out
Steps per minute76.460 s over the step time of 0.79 s
Epoch interval294.6 sMean of 2 gaps between epoch marks
Eval accuracy, as shown91.8%0.9180 times 100, to one decimal. Changes on that plate are percentage points
Eval accuracy since first eval+0.75400.9180 minus 0.1640
Eval change+0.0080Eval 4 at 0.9180 minus eval 3 at 0.9100
Wrong answers41500 eval items minus 459 correct. 459 over 500 is the only count that gives 0.9180
Rank1 of 5By last eval accuracy, across this project's 5 runs with an eval
Best so far0.9180Highest last eval accuracy, run 5
Against the run before+0.02400.9180 minus 0.8940 on run 4
Seed spread, observed0.0300High 0.9180 minus low 0.8880 across 3 finished seeds
Items per second20.41 step over 0.79 s, times batch size 16