CITADEL
StandbyNothing is live right now.
Eval accuracylast measured Sat 21 Nov, 12:19 UTC
%
0.8449376 of 445 eval, 69 wrong
▲ +3.59 pts across 3 evals▲ +2.24 pts on the eval before
0%25%50%75%100%0%25%50%75%100%0%25%50%75%100%

Familiar decider: ModernBERT-base

finishedRadeon 8060S

One of the small models that decide whether Familiar, a desktop companion, reacts to something happening on the PC, and how. Every candidate trains on the same 6,000 decision points from synthetic timelines. The encoders also get one run on the full pool.

started
Sat 21 Nov, 12:02 UTC
ran for
12:05
config
learning rate 0.00002, batch 16, seed 0

run3/6step1032/1032epoch3/3config3/5seed0lr0.00002batch16modelModernBERT-base 149Mrev8949b90items5503+445

Loss train 0.2515 ▼ −2.6515 lowest 0.0209 at step 810 run before, run 2 ▼ −0.0618 step 1032
Timebase
Range
Smoothing
Reference
104 readings, 5 min/div5 min/div2 min/div
0123400:0005:0010:00phase starting, 0.0 s to 20.6 sphase training, 20.6 s to 12:05epoch 1, 04:15epoch 2, 08:10epoch 3, 11:58trainingepoch 1epoch 2−2.6515since first2.9030Lowest last train loss across this project's 6 runs, run 6lowest last 0.1056, run 60.25150123400:0005:0010:00phase starting, 0.0 s to 20.6 sphase training, 20.6 s to 12:05epoch 1, 04:15epoch 2, 08:10epoch 3, 11:58trainingepoch 1epoch 22.9030Lowest last train loss across this project's 6 runs, run 6lowest last 0.1056, run 60.2515▼ −2.65150123400:0002:0004:0006:0008:0010:0012:00phase starting, 0.0 s to 20.6 sphase training, 20.6 s to 12:05epoch 1, 04:15epoch 2, 08:10epoch 3, 11:58trainingepoch 1epoch 22.9030Lowest last train loss across this project's 6 runs, run 6lowest last 0.1056, run 60.2515▼ −2.6515

Eval measures

Elapsed

231.5 s

epoch interval

1032 of 1032 steps, 3 of 3 epochs, 100%

0.67 s per step, 89.8 steps a minute

Eval accuracy by eval

+0.0224

▲ on the eval before

eval 1 0.8090, eval 3 0.8449

Against the other runs

4 of 5

by last eval accuracy

best so far 0.9101, run 6

▲ +0.0045 against run 2

Set-up

5503 train, 445 eval

synthetic, learning rate 0.00002, batch size 16, epochs 3, max length 512, precision bf16-mixed, loss ce, optimizer adamw, scheduler linear, seed 0

Learning rate

0.00e+0

▼ −3.28e-6 since first

On one axis

phase starting, 0.0 s to 20.6 sphase training, 20.6 s to 12:05E1E2E3traininglowest last 0.1056Train loss0.2515▼ −2.65153.000.002.90300.2515last ▼ −2.6515best 0.9101, run 6Last-value range, 5 runs, 0.8404 to 0.9101Eval accuracy0.8449▲ +0.03590.950.800.80900.8449last ▲ +0.0359Learning rate0.00e+0▼ −3.28e-62.0e-50.0e+03.28e-6last 0.00e+0Power, whole chip, CPU and GPU115 W▼ −1 W123 W113 W116 Wlast 115 WHottest, one board zone94 °C▲ +5 °C96 °C88 °C89 °Clast 94 °C00:0002:0004:0006:0008:0010:0012:00
GPUpower: whole chip, CPU and GPUtemperature: one board zone

GPU 0

Wwhole chip, CPU and GPU

115 W whole chip, CPU and GPU

use87%
memory, whole GPU1.1 GB

peak 12.5 GB, whole GPU

94 °C one board zone

history 0 to 121 W

Input

  • Runs128 runs
  • Agentsnone showing
  • Treesnone showing
  • Projectsnone showing

Check

Every entry is checked on its way out; some are held for review.

Not shown on any run: data samples, predictions, raw logs, file paths, machine names

Delay

2 min, set

Watching the agents work, about 2 min behind.

One way

Nothing here reaches the agents.

This page

This page updates by itself.

visits counted with Google Analytics · choose

Runsheight is eval accuracyall recordings ▸ | finished× failed■ stopped○ lost● live

3 of 6 finished, 0.8449, config 3 of 5 (learning rate 0.00002, batch 16), seed 0, Sat 21 Nov, 12:02 UTC

Runs as a table
StartedStatusEval accuracyStepsConfig
Sat 21 Nov, 18:44 UTC| finished0.91011032 of 1032batch 16, seed 2 of 2
Sat 21 Nov, 18:26 UTC| finished0.85391032 of 1032batch 16, seed 1 of 2
Sat 21 Nov, 12:19 UTC| finished0.88311032 of 1032learning rate 0.00005, batch 16, seed 0
Sat 21 Nov, 12:02 UTC| finished0.84491032 of 1032learning rate 0.00002, batch 16, seed 0
Sat 21 Nov, 11:45 UTC| finished0.84041032 of 1032learning rate 0.0001, batch 16, seed 0
Fri 20 Nov, 07:59 UTC× failed-2180 of 7274batch 16, seed 0

Plate

Model
answerdotai/ModernBERT-base, 149M parameters revision 8949b90
Data
Familiar decisions, shared draw, plain-case intents, synthetic5503 train445 eval
Config
learning rate 0.00002, batch size 16, epochs 3, max length 512, precision bf16-mixed, loss ce, optimizer adamw, scheduler linear, seed 0
Hardware
Radeon 8060S, ROCm

Log

Since startMark
0.0 sphase starting
3.7 swent public
20.6 sphase training
250.3 seval 1
255.3 sepoch 1
488.3 seval 2
490.7 sepoch 2
718.3 sepoch 3
725.7 seval 3

Readings

106 rows as a table

rows 1 to 100 of 106

StepSecondsTrain lossEval accuracyLearning rate
1020.62.9030-3.28e-6
2027.22.4205-6.56e-6
3033.82.1267-9.84e-6
4040.51.4660-1.31e-5
5047.21.1843-1.64e-5
6053.91.5256-1.97e-5
7060.61.3253-1.98e-5
8067.21.5171-1.96e-5
9073.81.6583-1.94e-5
10080.51.4030-1.92e-5
11087.12.1051-1.90e-5
12093.71.3288-1.88e-5
130100.41.4571-1.86e-5
140107.01.6501-1.84e-5
150113.61.9682-1.82e-5
160120.21.2417-1.80e-5
170126.92.1900-1.78e-5
180133.71.5109-1.75e-5
190140.41.1887-1.73e-5
200147.11.3831-1.71e-5
210153.81.3372-1.69e-5
220160.41.2503-1.67e-5
230167.11.5682-1.65e-5
240173.80.6447-1.63e-5
250180.50.7850-1.61e-5
260187.11.0122-1.59e-5
270193.70.9356-1.57e-5
280200.30.9827-1.55e-5
290206.90.7438-1.53e-5
300213.60.4810-1.51e-5
310220.31.1472-1.49e-5
320227.00.6575-1.47e-5
330233.70.4245-1.45e-5
340240.40.4184-1.43e-5
344250.3-0.8090-
350255.30.4491-1.40e-5
360262.00.3684-1.38e-5
370268.71.1837-1.36e-5
380275.30.6397-1.34e-5
390282.10.6008-1.32e-5
400288.70.6120-1.30e-5
410295.40.5153-1.28e-5
420302.00.0795-1.26e-5
430308.70.6116-1.24e-5
440315.30.4201-1.22e-5
450322.00.6052-1.20e-5
460328.70.4546-1.18e-5
470335.30.2500-1.16e-5
480342.00.3625-1.14e-5
490348.70.3714-1.12e-5
500355.40.2653-1.10e-5
510362.10.8237-1.08e-5
520368.80.3892-1.05e-5
530375.50.5043-1.03e-5
540382.11.1962-1.01e-5
550388.90.7983-9.93e-6
560395.50.8052-9.72e-6
570402.20.4369-9.52e-6
580408.80.2823-9.31e-6
590415.50.4255-9.10e-6
600422.21.1210-8.90e-6
610428.90.1877-8.69e-6
620435.60.1595-8.49e-6
630442.30.6320-8.28e-6
640449.00.4605-8.07e-6
650455.60.4484-7.87e-6
660462.20.3366-7.66e-6
670468.90.5278-7.46e-6
680475.60.4664-7.25e-6
688488.3-0.8225-
690490.70.4325-7.04e-6
700497.20.4582-6.84e-6
710503.90.3557-6.63e-6
720510.50.1910-6.43e-6
730517.20.3186-6.22e-6
740523.90.1399-6.01e-6
750530.60.1790-5.81e-6
760537.30.6353-5.60e-6
770544.00.3020-5.40e-6
780550.60.4241-5.19e-6
790557.30.2746-4.98e-6
800564.00.1161-4.78e-6
810570.70.0209-4.57e-6
820577.40.0813-4.37e-6
830584.00.2834-4.16e-6
840590.60.1552-3.95e-6
850597.30.1951-3.75e-6
860603.90.1073-3.54e-6
870610.50.2887-3.34e-6
880617.20.1812-3.13e-6
890623.90.2043-2.92e-6
900630.60.8547-2.72e-6
910637.30.0669-2.51e-6
920643.90.3963-2.31e-6
930650.50.1537-2.10e-6
940657.20.0983-1.89e-6
950663.80.5157-1.69e-6
960670.40.0671-1.48e-6
970677.10.5091-1.28e-6
980683.70.0935-1.07e-6

GPU readings

106 rows as a table
StepGPUUse %Power W, whole chip, CPU and GPUMemory GB, whole GPUTemp C, one board zone
20066116.18.588.8
70066120.68.588.8
120068120.510.690.8
1700100118.710.690.8
220062118.412.591.8
260065120.112.591.8
310065121.412.591.8
350066119.612.594.8
4000100115.712.591.8
4500100115.112.591.8
500065120.412.591.8
5400100115.312.591.8
590067120.112.591.8
6400100116.112.591.8
680096117.912.592.8
730072120.612.591.8
7800100114.912.591.8
820065119.812.591.8
870062120.012.591.8
9200100116.312.591.8
9700100115.712.590.8
1020066116.312.591.8
1032087115.41.193.8

Worked out

ReadoutValueWorked out from
Run complete100%1032 steps over 1032
Step time0.67 s673.8 s over 1008 steps, between readings. Startup and intervals that hold an eval are left out
Steps per minute89.860 s over the step time of 0.67 s
Epoch interval231.5 sMean of 2 gaps between epoch marks
Train loss since first reading−2.65150.2515 at step 1032 minus 2.9030 at step 10
Lowest train loss0.0209Lowest of 104 readings, at step 810
Eval accuracy, as shown84.5%0.8449 times 100, to one decimal. Changes on that plate are percentage points
Eval accuracy since first eval+0.03590.8449 minus 0.8090
Eval change+0.0224Eval 3 at 0.8449 minus eval 2 at 0.8225
Wrong answers69445 eval items minus 376 correct. 376 over 445 is the only count that gives 0.8449
Rank4 of 5By last eval accuracy, across this project's 5 runs with an eval
Best so far0.9101Highest last eval accuracy, run 6
Against the run before+0.00450.8449 minus 0.8404 on run 2
Lowest last train loss0.1056Lowest last train loss across this project's 6 runs, run 6
Items per second23.91 step over 0.67 s, times batch size 16
Train loss against run 2−0.06180.2515 minus 0.3133 on run 2, both at step 1032
Learning rate since first reading−3.28e-60.00e+0 minus 3.28e-6
Peak memory per GPU, whole GPU12.5 GBHighest sensor reading of any GPU, GPU 0, counting every process on it