CITADEL
StandbyNothing is live right now.
Eval accuracylast measured Thu 19 Nov, 07:46 UTC
%
0.9300465 of 500 eval, 35 wrong
▲ +8.00 pts across 3 evals▲ +3.00 pts on the eval before
0%25%50%75%100%0%25%50%75%100%0%25%50%75%100%

Familiar decider: ettin-encoder-400m

finishedRadeon 8060S

One of the small models that decide whether Familiar, a desktop companion, reacts to something happening on the PC, and how. Every candidate trains on the same 6,000 decision points from synthetic timelines. The encoders also get one run on the full pool.

started
Thu 19 Nov, 05:50 UTC
ran for
103:54
config
batch 16, seed 2 of 2

run5/11step1125/1125epoch3/3config4/9seed2/2batch16modelettin-encoder-400m 400Mrev7662476items6000+500

Loss train 0.0580 ▼ −1.5178 lowest 0.0055 at step 970 run before, run 4 ▲ +0.0509 step 1125
Timebase
Range
Smoothing
Reference
113 readings, 30 min/div30 min/div30 min/div
012300:0030:0060:0090:00phase starting, 0.0 s to 87.6 sphase training, 87.6 s to 103:54epoch 1, 36:10epoch 2, 69:11epoch 3, 103:34trainingepoch 1epoch 2−1.5178since first1.5758Lowest last train loss across this project's 11 runs, run 10lowest last 0.0002, run 100.0580012300:0030:0060:0090:00phase starting, 0.0 s to 87.6 sphase training, 87.6 s to 103:54epoch 1, 36:10epoch 2, 69:11epoch 3, 103:34trainingepoch 1epoch 21.5758Lowest last train loss across this project's 11 runs, run 10lowest last 0.0002, run 100.0580▼ −1.5178012300:0030:0060:0090:00phase starting, 0.0 s to 87.6 sphase training, 87.6 s to 103:54epoch 1, 36:10epoch 2, 69:11epoch 3, 103:34trainingepoch 1epoch 21.5758Lowest last train loss across this project's 11 runs, run 10lowest last 0.0002, run 100.0580▼ −1.5178

Eval measures

Elapsed

2022.0 s

epoch interval

2140.4 sto eval 12011.3 sto epoch 22062.7 sto epoch 3

2140.4 s to eval 1, 2011.3 s to epoch 2, 2062.7 s to epoch 3

1125 of 1125 steps, 3 of 3 epochs, 100%

5.50 s per step, 10.9 steps a minute

Eval accuracy by eval

+0.0300

▲ on the eval before

eval 1 0.8500, eval 3 0.9300

Against the other runs

2 of 11

by last eval accuracy

best so far 0.9640, run 6

▲ +0.0100 against run 4

Set-up

6000 train, 500 eval

synthetic, batch size 16, epochs 3, max length 512, precision bf16-mixed, loss ce, optimizer adamw, scheduler linear, seed 2

Seed spread, observed

0.0100
0.900.95

2 finished seeds

Learning rate

0.00e+0

▼ −7.46e-6 since first

On one axis

phase starting, 0.0 s to 87.6 sphase training, 87.6 s to 103:54E1E2E3traininglowest last 0.0002Train loss0.0580▼ −1.51783.000.001.57580.0580last ▼ −1.5178best 0.9640, run 6Last-value range, 11 runs, 0.8854 to 0.9640Eval accuracy0.9300▲ +0.08001.000.850.85000.9300last ▲ +0.0800Learning rate0.00e+0▼ −7.46e-66.0e-50.0e+07.46e-6last 0.00e+0Power, whole chip, CPU and GPU53 W▼ −0 W56 W50 W53 Wlast 53 WHottest, one board zone70 °C▲ +1 °C75 °C65 °C69 °Clast 70 °C00:0030:0060:0090:00
GPUpower: whole chip, CPU and GPUtemperature: one board zone

GPU 0

Wwhole chip, CPU and GPU

53 W whole chip, CPU and GPU

use44%
memory, whole GPU2.6 GB

peak 26.4 GB, whole GPU

70 °C one board zone

history 0 to 56 W

Input

  • Runs128 runs
  • Agentsnone showing
  • Treesnone showing
  • Projectsnone showing

Check

Every entry is checked on its way out; some are held for review.

Not shown on any run: data samples, predictions, raw logs, file paths, machine names

Delay

2 min, set

Watching the agents work, about 2 min behind.

One way

Nothing here reaches the agents.

This page

This page updates by itself.

visits counted with Google Analytics · choose

Runsheight is eval accuracyall recordings ▸ | finished× failed■ stopped○ lost● live
1.000.851.000.85finished, eval accuracy 0.9100, config 1 of 9 (learning rate 0.0001, batch 16), seed 0, Wed 18 Nov, 18:39 UTCfinished, eval accuracy 0.9120, config 2 of 9 (learning rate 0.00002, batch 16), seed 0, Wed 18 Nov, 20:29 UTCfinished, eval accuracy 0.9240, config 3 of 9 (learning rate 0.00005, batch 16), seed 0, Wed 18 Nov, 22:27 UTCfinished, eval accuracy 0.9200, config 4 of 9 (batch 16), seed 1 of 2, Thu 19 Nov, 04:02 UTCfinished, eval accuracy 0.9300, config 4 of 9 (batch 16), seed 2 of 2, Thu 19 Nov, 05:50 UTCfinished, eval accuracy 0.9640, config 5 of 9 (batch 16), seed 0, Sat 21 Nov, 05:11 UTCfinished, eval accuracy 0.8854, config 6 of 9 (learning rate 0.0001, batch 16), seed 0, Sat 21 Nov, 14:33 UTCfinished, eval accuracy 0.9034, config 7 of 9 (learning rate 0.00002, batch 16), seed 0, Sat 21 Nov, 15:12 UTCfinished, eval accuracy 0.8854, config 8 of 9 (learning rate 0.00005, batch 16), seed 0, Sat 21 Nov, 15:51 UTCfinished, eval accuracy 0.8899, config 9 of 9 (batch 16), seed 1 of 2, Sat 21 Nov, 20:19 UTCfinished, eval accuracy 0.8989, config 9 of 9 (batch 16), seed 2 of 2, Sat 21 Nov, 20:58 UTC1.000.85finished, eval accuracy 0.9100, config 1 of 9 (learning rate 0.0001, batch 16), seed 0, Wed 18 Nov, 18:39 UTCfinished, eval accuracy 0.9120, config 2 of 9 (learning rate 0.00002, batch 16), seed 0, Wed 18 Nov, 20:29 UTCfinished, eval accuracy 0.9240, config 3 of 9 (learning rate 0.00005, batch 16), seed 0, Wed 18 Nov, 22:27 UTCfinished, eval accuracy 0.9200, config 4 of 9 (batch 16), seed 1 of 2, Thu 19 Nov, 04:02 UTCfinished, eval accuracy 0.9300, config 4 of 9 (batch 16), seed 2 of 2, Thu 19 Nov, 05:50 UTCfinished, eval accuracy 0.9640, config 5 of 9 (batch 16), seed 0, Sat 21 Nov, 05:11 UTCfinished, eval accuracy 0.8854, config 6 of 9 (learning rate 0.0001, batch 16), seed 0, Sat 21 Nov, 14:33 UTCfinished, eval accuracy 0.9034, config 7 of 9 (learning rate 0.00002, batch 16), seed 0, Sat 21 Nov, 15:12 UTCfinished, eval accuracy 0.8854, config 8 of 9 (learning rate 0.00005, batch 16), seed 0, Sat 21 Nov, 15:51 UTCfinished, eval accuracy 0.8899, config 9 of 9 (batch 16), seed 1 of 2, Sat 21 Nov, 20:19 UTCfinished, eval accuracy 0.8989, config 9 of 9 (batch 16), seed 2 of 2, Sat 21 Nov, 20:58 UTC

5 of 11 finished, 0.9300, config 4 of 9 (batch 16), seed 2 of 2, Thu 19 Nov, 05:50 UTC

Runs as a table
StartedStatusEval accuracyStepsConfig
Sat 21 Nov, 20:58 UTC| finished0.89891032 of 1032batch 16, seed 2 of 2
Sat 21 Nov, 20:19 UTC| finished0.88991032 of 1032batch 16, seed 1 of 2
Sat 21 Nov, 15:51 UTC| finished0.88541032 of 1032learning rate 0.00005, batch 16, seed 0
Sat 21 Nov, 15:12 UTC| finished0.90341032 of 1032learning rate 0.00002, batch 16, seed 0
Sat 21 Nov, 14:33 UTC| finished0.88541032 of 1032learning rate 0.0001, batch 16, seed 0
Sat 21 Nov, 05:11 UTC| finished0.96407274 of 7274batch 16, seed 0
Thu 19 Nov, 05:50 UTC| finished0.93001125 of 1125batch 16, seed 2 of 2
Thu 19 Nov, 04:02 UTC| finished0.92001125 of 1125batch 16, seed 1 of 2
Wed 18 Nov, 22:27 UTC| finished0.92401125 of 1125learning rate 0.00005, batch 16, seed 0
Wed 18 Nov, 20:29 UTC| finished0.91201125 of 1125learning rate 0.00002, batch 16, seed 0
Wed 18 Nov, 18:39 UTC| finished0.91001125 of 1125learning rate 0.0001, batch 16, seed 0

Plate

Model
jhu-clsp/ettin-encoder-400m, 400M parameters revision 7662476
Data
Familiar decisions, shared draw, synthetic6000 train500 eval
Config
batch size 16, epochs 3, max length 512, precision bf16-mixed, loss ce, optimizer adamw, scheduler linear, seed 2
Hardware
Radeon 8060S, ROCm

Log

Since startMark
0.0 sphase starting
14.7 swent public
87.6 sphase training
2140.4 seval 1
2170.5 sepoch 1
4151.8 sepoch 2
4172.3 seval 2
6214.5 sepoch 3
6235.0 seval 3

Seeds

SeedStatusEval accuracy
seed 1| finished0.9200
seed 2| finished0.9300

observed spread 0.0100, 2 finished seeds

Readings

114 rows as a table

rows 1 to 100 of 114

StepSecondsTrain lossEval accuracyLearning rate
1087.61.5758-7.46e-6
20137.11.4777-1.49e-5
30190.51.2301-2.24e-5
40241.60.8516-2.99e-5
50294.51.7176-3.73e-5
60354.21.8259-4.48e-5
70397.01.6463-4.99e-5
80456.91.0865-4.94e-5
90527.11.1217-4.89e-5
100605.31.1675-4.84e-5
110656.01.0293-4.80e-5
120726.61.4923-4.75e-5
130778.20.8478-4.70e-5
140847.00.9068-4.66e-5
150899.10.5990-4.61e-5
160958.51.0340-4.56e-5
1701027.01.1475-4.51e-5
1801062.80.6209-4.47e-5
1901131.70.4532-4.42e-5
2001174.31.3597-4.37e-5
2101245.70.9435-4.32e-5
2201280.11.0105-4.28e-5
2301322.70.5118-4.23e-5
2401356.90.5670-4.18e-5
2501425.00.2386-4.14e-5
2601467.50.2221-4.09e-5
2701510.70.1618-4.04e-5
2801545.40.8980-3.99e-5
2901606.00.2372-3.95e-5
3001675.10.5193-3.90e-5
3101744.80.1448-3.85e-5
3201824.60.3170-3.80e-5
3301894.00.5880-3.76e-5
3401938.20.8877-3.71e-5
3501989.40.6363-3.66e-5
3602041.00.4562-3.62e-5
3702094.00.3742-3.57e-5
3752140.4-0.8500-
3802170.50.1238-3.52e-5
3902248.00.1373-3.47e-5
4002308.40.2021-3.43e-5
4102352.30.1898-3.38e-5
4202370.01.0823-3.33e-5
4302430.21.2274-3.28e-5
4402498.30.3598-3.24e-5
4502550.50.5308-3.19e-5
4602636.90.3415-3.14e-5
4702722.50.1681-3.10e-5
4802765.50.5138-3.05e-5
4902834.40.6021-3.00e-5
5002879.71.0530-2.95e-5
5102933.20.0973-2.91e-5
5202959.90.5844-2.86e-5
5303035.60.5126-2.81e-5
5403070.80.5848-2.76e-5
5503122.70.5192-2.72e-5
5603156.90.1006-2.67e-5
5703182.70.7743-2.62e-5
5803241.30.1890-2.58e-5
5903283.60.4514-2.53e-5
6003326.20.1832-2.48e-5
6103378.60.4159-2.43e-5
6203439.30.1294-2.39e-5
6303492.00.3166-2.34e-5
6403560.60.8620-2.29e-5
6503614.50.0680-2.24e-5
6603692.10.2878-2.20e-5
6703727.70.0136-2.15e-5
6803778.70.3268-2.10e-5
6903822.30.6844-2.06e-5
7003857.00.0537-2.01e-5
7103917.40.4565-1.96e-5
7203976.90.1290-1.91e-5
7304028.00.0333-1.87e-5
7404090.50.2216-1.82e-5
7504172.30.20050.90001.77e-5
7604231.90.2871-1.72e-5
7704291.30.1111-1.68e-5
7804354.10.3683-1.63e-5
7904414.20.0643-1.58e-5
8004465.80.0319-1.54e-5
8104500.80.4443-1.49e-5
8204543.70.5532-1.44e-5
8304588.70.7201-1.39e-5
8404639.90.2510-1.35e-5
8504693.10.4972-1.30e-5
8604736.00.3760-1.25e-5
8704796.00.1911-1.21e-5
8804832.80.0254-1.16e-5
8904901.50.1039-1.11e-5
9004962.10.1999-1.06e-5
9105006.30.1682-1.02e-5
9205059.10.2988-9.69e-6
9305119.50.2572-9.22e-6
9405199.10.0762-8.74e-6
9505242.10.0270-8.27e-6
9605302.10.7813-7.80e-6
9705345.10.0055-7.33e-6
9805390.00.0078-6.85e-6
9905458.60.0607-6.38e-6

GPU readings

114 rows as a table

rows 1 to 100 of 114

StepGPUUse %Power W, whole chip, CPU and GPUMemory GB, whole GPUTemp C, one board zone
100053.122.868.8
200053.022.867.8
300252.822.871.8
400054.922.869.8
500053.322.867.8
600052.922.866.8
700054.022.868.8
800053.322.867.8
900053.322.870.8
1000052.522.868.8
1100054.326.469.8
1200054.626.469.8
1300053.326.468.8
1400053.426.467.8
1500053.226.468.8
1600053.426.468.8
1700054.626.472.8
1800053.026.468.8
1900054.626.466.8
2000053.526.469.8
2100053.426.467.8
2200053.126.468.8
2300053.226.467.8
2400051.526.468.8
2500452.626.468.8
2600053.126.468.8
2700054.326.466.8
2800054.726.470.8
2900053.526.467.8
3000054.126.468.8
3100052.026.472.8
3200053.126.468.8
3300053.426.469.8
3400053.326.468.8
3500051.426.471.8
3600152.926.469.8
3700253.226.466.8
3750052.626.467.8
38002352.826.470.8
3900053.126.468.8
4000551.226.470.8
4100052.326.466.8
4200053.026.468.8
4300052.726.469.8
4400053.026.467.8
4500054.026.468.8
4600053.426.468.8
4700054.726.468.8
4800052.826.469.8
4900054.726.467.8
5000053.226.467.8
5100052.626.466.8
5200052.626.468.8
5300053.126.467.8
5400053.026.468.8
5500052.926.470.8
5600052.526.466.8
5700052.926.468.8
5800052.226.466.8
5900053.526.467.8
6000053.326.467.8
6100051.426.467.8
6200053.426.467.8
6300052.926.468.8
6400053.026.468.8
6500052.526.468.8
6600053.026.466.8
6700053.526.469.8
6800052.626.468.8
6900053.326.465.8
7000052.726.468.8
7100053.026.469.8
7200053.126.469.8
7300053.326.468.8
7400051.726.468.8
7500053.426.467.8
7600053.326.468.8
77002252.126.468.8
7800053.226.467.8
7900054.626.467.8
8000053.526.468.8
8100852.526.468.8
8200053.026.466.8
8300552.626.468.8
8400053.626.468.8
8500053.926.468.8
8600053.426.467.8
8700053.126.467.8
8800055.626.469.8
8900053.426.468.8
9000052.126.467.8
9100052.726.468.8
9200053.026.467.8
9300053.326.468.8
9400053.026.465.8
9500053.126.467.8
9600053.426.467.8
9700053.126.468.8
9800051.026.468.8
9900054.026.467.8

Worked out

ReadoutValueWorked out from
Run complete100%1125 steps over 1125
Step time5.50 s6020.1 s over 1095 steps, between readings. Startup and intervals that hold an eval are left out
Steps per minute10.960 s over the step time of 5.50 s
Epoch interval2022.0 sMean of 2 gaps between epoch marks
Train loss since first reading−1.51780.0580 at step 1125 minus 1.5758 at step 10
Lowest train loss0.0055Lowest of 113 readings, at step 970
Eval accuracy, as shown93.0%0.9300 times 100, to one decimal. Changes on that plate are percentage points
Eval accuracy since first eval+0.08000.9300 minus 0.8500
Eval change+0.0300Eval 3 at 0.9300 minus eval 2 at 0.9000
Wrong answers35500 eval items minus 465 correct. 465 over 500 is the only count that gives 0.9300
Rank2 of 11By last eval accuracy, across this project's 11 runs with an eval
Best so far0.9640Highest last eval accuracy, run 6
Against the run before+0.01000.9300 minus 0.9200 on run 4
Lowest last train loss0.0002Lowest last train loss across this project's 11 runs, run 10
Seed spread, observed0.0100High 0.9300 minus low 0.9200 across 2 finished seeds
Items per second2.91 step over 5.50 s, times batch size 16
Train loss against run 4+0.05090.0580 minus 0.0070 on run 4, both at step 1125
Learning rate since first reading−7.46e-60.00e+0 minus 7.46e-6
Peak memory per GPU, whole GPU26.4 GBHighest sensor reading of any GPU, GPU 0, counting every process on it