mirror of
https://github.com/Crosstalk-Solutions/project-nomad.git
synced 2026-07-28 19:24:39 +02:00
0891d176e5
Three changes that together let ARM hardware appear on the leaderboard honestly. Shipping them separately would leave ARM half-supported either way: without the image a Pi cannot submit at all, and without the architecture field it submits but is indistinguishable from x86. 1. PIN THE OFFICIAL MULTI-ARCH SYSBENCH IMAGE severalnines/sysbench publishes amd64 only, so ARM hosts could not run the System Benchmark at all — not a graceful failure, the container simply cannot execute. Apple Silicon could only run it under Rosetta emulation, which distorts the measurement it is taking, and that is what drove a community macOS fork to substitute a different benchmark and submit incomparable numbers. Swaps to ghcr.io/crosstalk-solutions/nomad-sysbench (Debian 12 + sysbench 1.0.20+ds-5, built for linux/amd64 + linux/arm64). One digest covers both architectures; verified that pulling the pinned manifest-list digest resolves to arm64 on a Raspberry Pi 5 and amd64 on x86, and that RepoDigests reports the same manifest-list digest on both — so a single allowlist entry serves both. No rescoring: 1.0.17 -> 1.0.20 measured 1.25% apart on identical hardware with identical flags (7170.18 vs 7259.56 events/sec), inside run-to-run noise and ~0.3% on a composite. Both digests are allowlisted server-side, so the fleet can cross over gradually. 2. REPORT THE DIGEST ACTUALLY RESOLVED The submission previously sent SYSBENCH_DIGEST, the constant the client was compiled with. The leaderboard validates that field, but a constant attests to how a client was BUILT rather than what it RAN, so any build inherits a valid value simply by carrying the same source. Now reads it back from the image. Uses RepoDigests (the manifest digest we pulled by), never Id — Id is the config digest, differs per architecture, and would never match the allowlist. Falls back to the constant if inspection yields nothing usable, so a benchmark never fails over provenance metadata. Still forgeable, and always will be with an open-source client. It moves the bar from "no effort" to "deliberate", which is the distinction that matters when judging whether a submission is a mistake or a choice. 3. RECORD CPU ARCHITECTURE AND OS The leaderboard is a single board across instruction sets by design, with disclosure as the fairness mechanism. Without an architecture field an ARM result sits unlabelled beside x86 — exactly what the disclosure exists to prevent. All three fields come from the Docker daemon, reusing the docker.info() call _detectRunEnvironment already makes. That is deliberate: inside the admin container os.arch() and si.osInfo() describe the CONTAINER, not the host being benchmarked. cpu_architecture Architecture x86_64 -> amd64, aarch64 -> arm64 os_version OSVersion '24.04' (already structured, no parsing) os_name OperatingSystem 'Ubuntu 24.04.4 LTS' minus the version run_environment is kept rather than replaced: "which distro" and "is this virtualised" are different questions, and WSL2 is a real performance factor. String handling lives in app/utils/platform_metadata.ts with unit tests, matching the amd_hsa_override convention, so it is testable without a Docker daemon. Unknown architectures pass through verbatim rather than being guessed at, and os_name falls back to the full description whenever the version is missing or absent from it — an over-long name is harmless, a wrong one is not. Columns are nullable and the submission fields optional, so results recorded before this shipped remain submittable. Closes #1156 Refs #1151
69 lines
2.7 KiB
TypeScript
69 lines
2.7 KiB
TypeScript
/**
|
|
* Pure helpers for turning the Docker daemon's platform strings into the fields
|
|
* the benchmark submission carries.
|
|
*
|
|
* These read the HOST's platform, which is the entire point: inside the admin
|
|
* container `os.arch()` and `si.osInfo()` describe the container, not the
|
|
* machine being benchmarked. `BenchmarkService` delegates to these so the string
|
|
* handling is unit-testable without a Docker daemon.
|
|
*
|
|
* Observed daemon output across the test fleet:
|
|
*
|
|
* Architecture 'x86_64' | 'aarch64'
|
|
* OSVersion '24.04' | '26.04'
|
|
* OperatingSystem 'Ubuntu 24.04.4 LTS' | 'Ubuntu 26.04 LTS'
|
|
*/
|
|
|
|
/**
|
|
* Canonicalise the daemon's architecture string to the OCI platform names used
|
|
* everywhere else in the project (image manifests, install docs, the
|
|
* leaderboard).
|
|
*
|
|
* Docker reports `x86_64` / `aarch64`; images and the board talk in `amd64` /
|
|
* `arm64`. A fixed two-way map rather than a general normalisation table: these
|
|
* are the only architectures NOMAD targets, and anything unrecognised passes
|
|
* through verbatim rather than being guessed at, so an unexpected platform shows
|
|
* up honestly instead of mislabelled.
|
|
*/
|
|
export function normalizeArchitecture(raw: string): string {
|
|
const map: Record<string, string> = {
|
|
x86_64: 'amd64',
|
|
amd64: 'amd64',
|
|
aarch64: 'arm64',
|
|
arm64: 'arm64',
|
|
}
|
|
const key = raw.trim().toLowerCase()
|
|
return map[key] ?? raw.trim()
|
|
}
|
|
|
|
/**
|
|
* Split the distro name out of the daemon's free-form OperatingSystem string.
|
|
*
|
|
* `OperatingSystem` is a description ('Ubuntu 24.04.4 LTS') while `OSVersion` is
|
|
* structured ('24.04'). Taking the text before the version yields the name
|
|
* without hand-maintaining a list of distributions:
|
|
*
|
|
* 'Ubuntu 24.04.4 LTS' + '24.04' -> 'Ubuntu'
|
|
* 'Ubuntu 26.04 LTS' + '26.04' -> 'Ubuntu'
|
|
* 'Debian GNU/Linux 12 (bookworm)' + '12' -> 'Debian GNU/Linux'
|
|
*
|
|
* Falls back to the full description whenever the version is missing, empty, or
|
|
* doesn't appear in the string. An over-long name is harmless; a wrong one is
|
|
* not, and silently truncating an unfamiliar distro would be worse than leaving
|
|
* it verbose.
|
|
*/
|
|
export function deriveOsName(operatingSystem: string, osVersion: string | null): string {
|
|
const description = operatingSystem.trim()
|
|
if (!osVersion) return description
|
|
|
|
const version = osVersion.trim()
|
|
if (version === '') return description
|
|
|
|
const idx = description.indexOf(version)
|
|
// idx === 0 means the string starts with the version and has no name to take.
|
|
if (idx <= 0) return description
|
|
|
|
const name = description.slice(0, idx).trim()
|
|
return name.length > 0 ? name : description
|
|
}
|