CITADEL
StandbyNothing is live right now.
Eval accuracylast measured Tue 10 Nov, 10:55 UTC
%
0.7923103 of 130 eval, 27 wrong
▲ +16.92 pts across 2 evals▲ +16.92 pts on the eval before
0%25%50%75%100%0%25%50%75%100%0%25%50%75%100%

Claviger: Kev-4B (LoRA) (backfilled)

finishedRadeon 8060S

A small model that reads one Discord message and decides whether an agent should reply, and which one. Backfilled: recorded after the fact from the run's own logs and records. Times inside a run are estimated, and there are no GPU sensor readings.

started
Tue 10 Nov, 10:30 UTC
ran for
20:22
config
learning rate 0.00002, batch 1, seed 2 of 3

run3/3step120/120epoch2/2seed2/3lr0.00002batch1modelkev-4b 4Brev6cfce5citems479+130

Loss train 0.0102 ▼ −1.8309 eval 0.6818 lowest 0.0055 at step 89 run before, run 2 ▼ −0.0228 step 120
Timebase
Range
Smoothing
Reference
120 readings, 10 min/div5 min/div5 min/div
00.511.5200:0010:0020:00phase starting, 0.0 s to 10.2 sphase training, 10.2 s to 20:22epoch 1, 10.2 sepoch 2, 10:21trainingepoch 2−1.8309since first1.8411Lowest last train loss across this project's 3 runs, run 3lowest last 0.0102, run 30.010200.511.5200:0005:0010:0015:0020:00phase starting, 0.0 s to 10.2 sphase training, 10.2 s to 20:22epoch 1, 10.2 sepoch 2, 10:21trainingepoch 21.8411Lowest last train loss across this project's 3 runs, run 3lowest last 0.0102, run 3eval 0.68180.0102▼ −1.830900.511.5200:0005:0010:0015:0020:00phase starting, 0.0 s to 10.2 sphase training, 10.2 s to 20:22epoch 1, 10.2 sepoch 2, 10:21trainingepoch 21.8411Lowest last train loss across this project's 3 runs, run 3lowest last 0.0102, run 3eval 0.68180.0102▼ −1.8309

Eval measures

Eval losslower is better
0.6818▼ −0.1211 since first
02 →
ECEcalibration error, lower is better
0.1465▲ +0.0607 since first
01

Elapsed

611.3 s

epoch interval

10.2 s to epoch 1, 611.3 s to epoch 2, 601.2 s to eval 2

120 of 120 steps, 2 of 2 epochs, 100%

10.2 s per step, 5.9 steps a minute

Eval accuracy by eval

+0.1692

▲ on the eval before

eval 1 0.6231, eval 2 0.7923

Against the other runs

2 of 3

by last eval accuracy

best so far 0.8308, run 1

▲ +0.0077 against run 2

Train minus eval

−0.6716

step 120

train 0.0102, eval 0.6818

Set-up

479 train, 130 eval

synthetic, learning rate 0.00002, batch size 1, grad accum 8, epochs 2, optimizer adamw, seed 2

Seed spread, observed

0.0462
0.750.85

3 finished seeds

Learning rate

4.31e-9

▼ −1.18e-6 since first

On one axis

phase starting, 0.0 s to 10.2 sphase training, 10.2 s to 20:22E2traininglowest last 0.0102Train loss0.0102▼ −1.83092.000.001.84110.0102last ▼ −1.8309best 0.8308, run 1Last-value range, 3 runs, 0.7846 to 0.8308last-value range, n 3Eval accuracy0.7923▲ +0.16920.900.600.62310.7923last ▲ +0.1692Learning rate4.31e-9▼ −1.18e-62.0e-50.0e+01.19e-64.31e-9last ▼ −1.18e-600:0005:0010:0015:0020:00

Input

  • Runs128 runs
  • Agentsnone showing
  • Treesnone showing
  • Projectsnone showing

Check

Every entry is checked on its way out; some are held for review.

Not shown on any run: data samples, predictions, raw logs, file paths, machine names

Delay

2 min, set

Watching the agents work, about 2 min behind.

One way

Nothing here reaches the agents.

This page

This page updates by itself.

visits counted with Google Analytics · choose

Runsheight is eval accuracyall recordings ▸ | finished× failed■ stopped○ lost● live newer ▸

3 of 3 finished, 0.7923, config 1 of 1 (learning rate 0.00002, batch 1), seed 2 of 3, Tue 10 Nov, 10:30 UTC

▸
Runs as a table
StartedStatusEval accuracyStepsConfig
Tue 10 Nov, 10:30 UTC| finished0.7923120 of 120learning rate 0.00002, batch 1, seed 2 of 3
Tue 10 Nov, 10:04 UTC| finished0.7846120 of 120learning rate 0.00002, batch 1, seed 1 of 3
Tue 10 Nov, 09:38 UTC| finished0.8308120 of 120learning rate 0.00002, batch 1, seed 0 of 3

Plate

Model
jaredpalmer/kev-4b, 4B parameters revision 6cfce5c
Data
Synthetic Discord triage set, mixed rosters, synthetic479 train130 eval
Config
learning rate 0.00002, batch size 1, grad accum 8, epochs 2, optimizer adamw, seed 2
Hardware
Radeon 8060S, ROCm

Log

Since startMark
0.0 sphase starting
0.0 seval 1
10.2 sphase training
10.2 sepoch 1
621.5 sepoch 2
1222.7 seval 2
194 h 22 min laterwent public

Seeds

SeedStatusEval accuracy
seed 0| finished0.8308
seed 1| finished0.7846
seed 2| finished0.7923

observed spread 0.0462, 3 finished seeds

Readings

121 rows as a table

rows 1 to 100 of 121

StepSecondsTrain lossEval lossEval accuracyECELearning rate
00.0-0.80290.62310.0858-
110.21.8411---1.19e-6
220.40.5103---2.32e-6
330.61.3727---4.11e-6
440.80.7915---6.41e-6
550.91.1342---9.03e-6
661.11.2229---1.18e-5
771.30.5660---1.44e-5
881.51.0893---1.67e-5
991.70.7343---1.85e-5
10101.91.0545---1.96e-5
11112.10.7497---2.00e-5
12122.30.6038---2.00e-5
13132.50.6743---2.00e-5
14142.60.6475---2.00e-5
15152.80.5208---1.99e-5
16163.00.6870---1.99e-5
17173.20.7009---1.98e-5
18183.40.4620---1.98e-5
19193.60.5074---1.97e-5
20203.80.2733---1.97e-5
21214.00.5836---1.96e-5
22224.20.3481---1.95e-5
23234.40.1507---1.94e-5
24244.50.8859---1.93e-5
25254.70.2574---1.92e-5
26264.90.6306---1.91e-5
27275.10.8119---1.89e-5
28285.30.5173---1.88e-5
29295.50.4083---1.87e-5
30305.70.8250---1.85e-5
31315.91.0978---1.84e-5
32326.10.6771---1.82e-5
33336.20.3443---1.80e-5
34346.40.7883---1.78e-5
35356.60.6727---1.77e-5
36366.80.2805---1.75e-5
37377.00.1471---1.73e-5
38387.20.4494---1.71e-5
39397.41.1599---1.69e-5
40407.60.2281---1.66e-5
41417.80.3961---1.64e-5
42427.90.2327---1.62e-5
43438.10.5082---1.60e-5
44448.30.3025---1.57e-5
45458.50.5629---1.55e-5
46468.70.2928---1.52e-5
47478.90.6659---1.50e-5
48489.10.2845---1.47e-5
49499.30.3637---1.45e-5
50509.50.1612---1.42e-5
51519.60.1556---1.40e-5
52529.80.4147---1.37e-5
53540.00.1649---1.34e-5
54550.20.1135---1.31e-5
55560.40.4960---1.29e-5
56570.60.5954---1.26e-5
57580.80.2303---1.23e-5
58591.00.7549---1.20e-5
59601.20.2242---1.17e-5
60611.40.5285---1.14e-5
61621.50.2098---1.12e-5
62631.70.3028---1.09e-5
63641.90.1853---1.06e-5
64652.10.2770---1.03e-5
65662.30.8060---1.00e-5
66672.50.4593---9.71e-6
67682.70.0622---9.42e-6
68692.90.0426---9.13e-6
69703.10.0505---8.84e-6
70713.20.3088---8.55e-6
71723.40.6009---8.26e-6
72733.60.0120---7.98e-6
73743.80.0778---7.69e-6
74754.00.2700---7.41e-6
75764.20.4170---7.13e-6
76774.40.1039---6.85e-6
77784.60.0224---6.58e-6
78794.80.4994---6.31e-6
79804.90.0129---6.04e-6
80815.10.0112---5.77e-6
81825.30.0953---5.51e-6
82835.50.0280---5.25e-6
83845.70.3726---5.00e-6
84855.90.1896---4.75e-6
85866.10.0365---4.50e-6
86876.30.0383---4.26e-6
87886.50.0931---4.03e-6
88896.60.0133---3.80e-6
89906.80.0055---3.57e-6
90917.00.7928---3.35e-6
91927.20.7090---3.14e-6
92937.40.9449---2.93e-6
93947.60.0180---2.73e-6
94957.80.3705---2.53e-6
95968.00.3049---2.34e-6
96978.20.7982---2.16e-6
97988.30.5635---1.98e-6
98998.51.8298---1.81e-6
991008.70.4061---1.65e-6

Worked out

ReadoutValueWorked out from
Run complete100%120 steps over 120
Step time10.2 s1212.5 s over 119 steps, between readings. Startup and intervals that hold an eval are left out
Steps per minute5.960 s over the step time of 10.19 s
Epoch interval611.3 sThe one gap between epoch marks
Train loss since first reading−1.83090.0102 at step 120 minus 1.8411 at step 1
Lowest train loss0.0055Lowest of 120 readings, at step 89
Eval accuracy, as shown79.2%0.7923 times 100, to one decimal. Changes on that plate are percentage points
Eval accuracy since first eval+0.16920.7923 minus 0.6231
Eval loss since first eval−0.12110.6818 minus 0.8029
ECE since first eval+0.06070.1465 minus 0.0858
Eval change+0.1692Eval 2 at 0.7923 minus eval 1 at 0.6231
Train minus eval loss−0.67160.0102 minus 0.6818, both at step 120
Wrong answers27130 eval items minus 103 correct. 103 over 130 is the only count that gives 0.7923
Rank2 of 3By last eval accuracy, across this project's 3 runs with an eval
Best so far0.8308Highest last eval accuracy, run 1
Against the run before+0.00770.7923 minus 0.7846 on run 2
Lowest last train loss0.0102Lowest last train loss across this project's 3 runs, run 3
Seed spread, observed0.0462High 0.8308 minus low 0.7846 across 3 finished seeds
Items per second0.11 step over 10.19 s, times batch size 1
Train loss against run 2−0.02280.0102 minus 0.0329 on run 2, both at step 120
Learning rate since first reading−1.18e-64.31e-9 minus 1.19e-6