Kiln

Methodology

How benchmarks are run

Every leaderboard number is reproducible with a documented command, fixed seed, and dataset citation. Detection metrics are verified Colab GPU runs — not synthetic placeholders.

Core principles

  • Fixed seed (42) for reproducible splits and training
  • Real public datasets with citations in DATASETS.md
  • Committed JSON leaderboards synced to the live site
  • CI smoke tests on every push (pytest, ruff, tabular + Keras)

Per-track commands

Tabular

Primary: accuracy / RMSE

Titanic, Breast Cancer, Wine Quality

kiln-benchmark --track tabular --seed 42

Unsupervised

Primary: silhouette / accuracy

Iris, Digits

kiln-benchmark --track unsupervised --seed 42

Vision

Primary: accuracy

Fashion-MNIST

kiln-benchmark --track vision --epochs 5 --seed 42

Detection

Primary: mAP@50

Hard Hat Workers (Roboflow)

kiln-benchmark --track detection --seed 42

Detection verification (YOLO)

Hard Hat Workers mAP values are from verified GPU training via Colab. Reproduce locally after exporting the Roboflow YOLOv8 dataset to ~/.cache/kiln/datasets/hardhat/.

Models
YOLOv8n vs YOLOv8s
Epochs
10
Image size
640
Seed
42
Environment
Google Colab T4 GPU
Verified
2026-07-01
Notebook
04_roboflow_yolo_hardhat.ipynb
Open YOLO notebook in Colab

Full details in METHODOLOGY.md. View live results on Benchmarks.