CITADEL
StandbyNothing is live right now.
Eval accuracylast measured Sun 22 Nov, 12:47 UTC
%
0.8831393 of 445 eval, 52 wrong
▲ +6.29 pts across 3 evals▲ +1.12 pts on the eval before
0%25%50%75%100%0%25%50%75%100%0%25%50%75%100%

Familiar decider: ettin-encoder-68m

finishedRadeon 8060S

One of the small models that decide whether Familiar, a desktop companion, reacts to something on the PC, and how. Trained on the same 6,000 synthetic decision points as every candidate, two ways: the reaction itself, and the plain-case intent that rules adjust for her traits.

started
Sun 22 Nov, 12:38 UTC
ran for
06:35
config
batch 16, seed 1 of 2

run9/10step1032/1032epoch3/3config8/8seed1/2batch16modelettin-encoder-68m 68Mrevac19ae4items5503+445

Loss train 0.0119 ▼ −1.7614 lowest 0.0101 at step 790 run before, run 8 ▼ −0.0491 step 1030
Timebase
Range
Smoothing
Reference
104 readings, 2 min/div2 min/div1 min/div
012300:0002:0004:0006:00phase starting, 0.0 s to 14.5 sphase training, 14.5 s to 06:35epoch 1, 02:21epoch 2, 04:28epoch 3, 06:30trainingepoch 1epoch 2−1.7614since first1.7733Lowest last train loss across this project's 10 runs, run 9lowest last 0.0119, run 90.0119012300:0002:0004:0006:00phase starting, 0.0 s to 14.5 sphase training, 14.5 s to 06:35epoch 1, 02:21epoch 2, 04:28epoch 3, 06:30trainingepoch 1epoch 21.7733Lowest last train loss across this project's 10 runs, run 9lowest last 0.0119, run 90.0119▼ −1.7614012300:0001:0002:0003:0004:0005:0006:00phase starting, 0.0 s to 14.5 sphase training, 14.5 s to 06:35epoch 1, 02:21epoch 2, 04:28epoch 3, 06:30trainingepoch 1epoch 21.7733Lowest last train loss across this project's 10 runs, run 9lowest last 0.0119, run 90.0119▼ −1.7614

Eval measures

Elapsed

125.0 s

epoch interval

1032 of 1032 steps, 3 of 3 epochs, 100%

0.36 s per step, 164.7 steps a minute

Eval accuracy by eval

+0.0112

▲ on the eval before

eval 1 0.8202, eval 3 0.8831

Against the other runs

6 of 10

by last eval accuracy

best so far 0.9120, run 8

▼ −0.0289 against run 8

Set-up

5503 train, 445 eval

synthetic, batch size 16, epochs 3, max length 512, precision bf16-mixed, loss ce, optimizer adamw, scheduler linear, seed 1

Seed spread, observed

0.0247
0.850.90

2 finished seeds

Learning rate

0.00e+0

▼ −3.28e-6 since first

On one axis

phase starting, 0.0 s to 14.5 sphase training, 14.5 s to 06:35E1E2E3traininglowest last 0.0119Train loss0.0119▼ −1.76143.000.001.77330.0119last ▼ −1.7614best 0.9120, run 8Last-value range, 10 runs, 0.8584 to 0.9120Eval accuracy0.8831▲ +0.06290.950.800.82020.8831last ▲ +0.0629Learning rate0.00e+0▼ −3.28e-62.0e-50.0e+03.28e-6last 0.00e+0Power, whole chip, CPU and GPU116 W▼ −1 W122 W114 W117 Wlast 116 WHottest, one board zone92 °C▲ +6 °C92 °C84 °C86 °Clast 92 °C00:0001:0002:0003:0004:0005:0006:00
GPUpower: whole chip, CPU and GPUtemperature: one board zone

GPU 0

Wwhole chip, CPU and GPU

116 W whole chip, CPU and GPU

use86%
memory, whole GPU0.8 GB

peak 7.9 GB, whole GPU

92 °C one board zone

history 0 to 121 W

Input

  • Runs128 runs
  • Agentsnone showing
  • Treesnone showing
  • Projectsnone showing

Check

Every entry is checked on its way out; some are held for review.

Not shown on any run: data samples, predictions, raw logs, file paths, machine names

Delay

2 min, set

Watching the agents work, about 2 min behind.

One way

Nothing here reaches the agents.

This page

This page updates by itself.

visits counted with Google Analytics · choose

Runsheight is eval accuracyall recordings ▸ | finished× failed■ stopped○ lost● live
0.950.850.950.85finished, eval accuracy 0.8780, config 1 of 8 (learning rate 0.0001, batch 16), seed 0, Sun 22 Nov, 09:36 UTCfinished, eval accuracy 0.8800, config 2 of 8 (learning rate 0.00002, batch 16), seed 0, Sun 22 Nov, 09:46 UTCfinished, eval accuracy 0.8860, config 3 of 8 (learning rate 0.00005, batch 16), seed 0, Sun 22 Nov, 09:56 UTCfinished, eval accuracy 0.8831, config 4 of 8 (learning rate 0.0001, batch 16), seed 0, Sun 22 Nov, 10:05 UTCfinished, eval accuracy 0.9034, config 5 of 8 (learning rate 0.00002, batch 16), seed 0, Sun 22 Nov, 10:14 UTCfinished, eval accuracy 0.8809, config 6 of 8 (learning rate 0.00005, batch 16), seed 0, Sun 22 Nov, 10:24 UTCfinished, eval accuracy 0.9060, config 7 of 8 (batch 16), seed 1 of 2, Sun 22 Nov, 12:18 UTCfinished, eval accuracy 0.9120, config 7 of 8 (batch 16), seed 2 of 2, Sun 22 Nov, 12:28 UTCfinished, eval accuracy 0.8831, config 8 of 8 (batch 16), seed 1 of 2, Sun 22 Nov, 12:38 UTCfinished, eval accuracy 0.8584, config 8 of 8 (batch 16), seed 2 of 2, Sun 22 Nov, 12:47 UTC0.950.85finished, eval accuracy 0.8780, config 1 of 8 (learning rate 0.0001, batch 16), seed 0, Sun 22 Nov, 09:36 UTCfinished, eval accuracy 0.8800, config 2 of 8 (learning rate 0.00002, batch 16), seed 0, Sun 22 Nov, 09:46 UTCfinished, eval accuracy 0.8860, config 3 of 8 (learning rate 0.00005, batch 16), seed 0, Sun 22 Nov, 09:56 UTCfinished, eval accuracy 0.8831, config 4 of 8 (learning rate 0.0001, batch 16), seed 0, Sun 22 Nov, 10:05 UTCfinished, eval accuracy 0.9034, config 5 of 8 (learning rate 0.00002, batch 16), seed 0, Sun 22 Nov, 10:14 UTCfinished, eval accuracy 0.8809, config 6 of 8 (learning rate 0.00005, batch 16), seed 0, Sun 22 Nov, 10:24 UTCfinished, eval accuracy 0.9060, config 7 of 8 (batch 16), seed 1 of 2, Sun 22 Nov, 12:18 UTCfinished, eval accuracy 0.9120, config 7 of 8 (batch 16), seed 2 of 2, Sun 22 Nov, 12:28 UTCfinished, eval accuracy 0.8831, config 8 of 8 (batch 16), seed 1 of 2, Sun 22 Nov, 12:38 UTCfinished, eval accuracy 0.8584, config 8 of 8 (batch 16), seed 2 of 2, Sun 22 Nov, 12:47 UTC

9 of 10 finished, 0.8831, config 8 of 8 (batch 16), seed 1 of 2, Sun 22 Nov, 12:38 UTC

Runs as a table
StartedStatusEval accuracyStepsConfig
Sun 22 Nov, 12:47 UTC| finished0.85841032 of 1032batch 16, seed 2 of 2
Sun 22 Nov, 12:38 UTC| finished0.88311032 of 1032batch 16, seed 1 of 2
Sun 22 Nov, 12:28 UTC| finished0.91201125 of 1125batch 16, seed 2 of 2
Sun 22 Nov, 12:18 UTC| finished0.90601125 of 1125batch 16, seed 1 of 2
Sun 22 Nov, 10:24 UTC| finished0.88091032 of 1032learning rate 0.00005, batch 16, seed 0
Sun 22 Nov, 10:14 UTC| finished0.90341032 of 1032learning rate 0.00002, batch 16, seed 0
Sun 22 Nov, 10:05 UTC| finished0.88311032 of 1032learning rate 0.0001, batch 16, seed 0
Sun 22 Nov, 09:56 UTC| finished0.88601125 of 1125learning rate 0.00005, batch 16, seed 0
Sun 22 Nov, 09:46 UTC| finished0.88001125 of 1125learning rate 0.00002, batch 16, seed 0
Sun 22 Nov, 09:36 UTC| finished0.87801125 of 1125learning rate 0.0001, batch 16, seed 0

Plate

Model
jhu-clsp/ettin-encoder-68m, 68M parameters revision ac19ae4
Data
Familiar decisions, shared draw, plain-case intents, synthetic5503 train445 eval
Config
batch size 16, epochs 3, max length 512, precision bf16-mixed, loss ce, optimizer adamw, scheduler linear, seed 1
Hardware
Radeon 8060S, ROCm

Log

Since startMark
0.0 sphase starting
7.6 swent public
14.5 sphase training
138.5 seval 1
141.1 sepoch 1
266.8 seval 2
268.0 sepoch 2
391.0 sepoch 3
395.2 seval 3

Seeds

SeedStatusEval accuracy
seed 1| finished0.8831
seed 2| finished0.8584

observed spread 0.0247, 2 finished seeds

Readings

106 rows as a table

rows 1 to 100 of 106

StepSecondsTrain lossEval accuracyLearning rate
1014.51.7733-3.28e-6
2018.01.8855-6.56e-6
3021.51.2126-9.84e-6
4025.21.4423-1.31e-5
5028.81.4765-1.64e-5
6032.31.2886-1.97e-5
7036.01.7685-1.98e-5
8039.52.0039-1.96e-5
9043.10.8104-1.94e-5
10046.60.9939-1.92e-5
11050.21.7047-1.90e-5
12053.71.8475-1.88e-5
13057.41.0308-1.86e-5
14061.01.4251-1.84e-5
15064.61.4010-1.82e-5
16068.20.8572-1.80e-5
17071.80.6088-1.78e-5
18075.31.3890-1.75e-5
19079.00.9736-1.73e-5
20082.61.0156-1.71e-5
21086.21.0634-1.69e-5
22089.80.7418-1.67e-5
23093.30.5628-1.65e-5
24096.90.8735-1.63e-5
250100.50.7586-1.61e-5
260104.10.5771-1.59e-5
270107.70.8305-1.57e-5
280111.30.7952-1.55e-5
290114.90.2729-1.53e-5
300118.60.4145-1.51e-5
310122.10.9778-1.49e-5
320125.70.0569-1.47e-5
330129.30.8237-1.45e-5
340132.90.4141-1.43e-5
344138.5-0.8202-
350141.10.3977-1.40e-5
360144.70.3097-1.38e-5
370148.30.5463-1.36e-5
380151.80.1395-1.34e-5
390155.50.0448-1.32e-5
400159.00.1927-1.30e-5
410162.60.1794-1.28e-5
420166.30.5364-1.26e-5
430169.80.4210-1.24e-5
440173.50.1610-1.22e-5
450177.10.7730-1.20e-5
460180.60.4462-1.18e-5
470184.20.4577-1.16e-5
480187.81.1458-1.14e-5
490191.40.6231-1.12e-5
500195.00.3779-1.10e-5
510198.60.2813-1.08e-5
520202.20.0879-1.05e-5
530205.80.9308-1.03e-5
540209.40.7783-1.01e-5
550213.00.5573-9.93e-6
560216.60.5378-9.72e-6
570220.20.1831-9.52e-6
580223.80.1894-9.31e-6
590227.40.6540-9.10e-6
600231.00.0680-8.90e-6
610234.60.6652-8.69e-6
620238.30.1441-8.49e-6
630241.90.0555-8.28e-6
640245.50.1663-8.07e-6
650249.00.2756-7.87e-6
660252.60.3301-7.66e-6
670256.20.2056-7.46e-6
680259.80.3081-7.25e-6
688266.8-0.8719-
690268.00.5751-7.04e-6
700271.50.0522-6.84e-6
710275.10.0313-6.63e-6
720278.70.1480-6.43e-6
730282.20.2081-6.22e-6
740285.80.2541-6.01e-6
750289.40.1846-5.81e-6
760293.10.0454-5.60e-6
770296.70.2746-5.40e-6
780300.30.0654-5.19e-6
790303.90.0101-4.98e-6
800307.50.3602-4.78e-6
810311.10.6626-4.57e-6
820314.70.3578-4.37e-6
830318.20.1416-4.16e-6
840321.80.0436-3.95e-6
850325.40.0386-3.75e-6
860329.00.1682-3.54e-6
870332.60.5286-3.34e-6
880336.20.1601-3.13e-6
890339.80.0107-2.92e-6
900343.40.2553-2.72e-6
910347.00.1524-2.51e-6
920350.60.1456-2.31e-6
930354.30.0571-2.10e-6
940357.90.0803-1.89e-6
950361.50.0828-1.69e-6
960365.10.3059-1.48e-6
970368.70.3880-1.28e-6
980372.30.2685-1.07e-6

GPU readings

106 rows as a table
StepGPUUse %Power W, whole chip, CPU and GPUMemory GB, whole GPUTemp C, one board zone
500100116.96.785.8
1400100116.57.985.8
2300100116.87.986.8
3200100117.37.987.8
4000100117.67.987.8
490068120.57.987.8
580067119.87.988.8
670066120.37.988.8
7500100116.87.988.8
8400100116.57.988.8
9300100117.17.988.8
10200100117.37.988.8
1032086115.90.891.8

Worked out

ReadoutValueWorked out from
Run complete100%1032 steps over 1032
Step time0.36 s368.0 s over 1010 steps, between readings. Startup and intervals that hold an eval are left out
Steps per minute164.760 s over the step time of 0.36 s
Epoch interval125.0 sMean of 2 gaps between epoch marks
Train loss since first reading−1.76140.0119 at step 1032 minus 1.7733 at step 10
Lowest train loss0.0101Lowest of 104 readings, at step 790
Eval accuracy, as shown88.3%0.8831 times 100, to one decimal. Changes on that plate are percentage points
Eval accuracy since first eval+0.06290.8831 minus 0.8202
Eval change+0.0112Eval 3 at 0.8831 minus eval 2 at 0.8719
Wrong answers52445 eval items minus 393 correct. 393 over 445 is the only count that gives 0.8831
Rank6 of 10By last eval accuracy, across this project's 10 runs with an eval
Best so far0.9120Highest last eval accuracy, run 8
Against the run before−0.02890.8831 minus 0.9120 on run 8
Lowest last train loss0.0119Lowest last train loss across this project's 10 runs, run 9
Seed spread, observed0.0247High 0.8831 minus low 0.8584 across 2 finished seeds
Items per second43.91 step over 0.36 s, times batch size 16
Train loss against run 8−0.04910.0708 minus 0.1198 on run 8, both at step 1030
Learning rate since first reading−3.28e-60.00e+0 minus 3.28e-6
Peak memory per GPU, whole GPU7.9 GBHighest sensor reading of any GPU, GPU 0, counting every process on it