mirror of
https://github.com/Crosstalk-Solutions/project-nomad.git
synced 2026-07-28 19:24:39 +02:00
0891d176e5
Three changes that together let ARM hardware appear on the leaderboard honestly. Shipping them separately would leave ARM half-supported either way: without the image a Pi cannot submit at all, and without the architecture field it submits but is indistinguishable from x86. 1. PIN THE OFFICIAL MULTI-ARCH SYSBENCH IMAGE severalnines/sysbench publishes amd64 only, so ARM hosts could not run the System Benchmark at all — not a graceful failure, the container simply cannot execute. Apple Silicon could only run it under Rosetta emulation, which distorts the measurement it is taking, and that is what drove a community macOS fork to substitute a different benchmark and submit incomparable numbers. Swaps to ghcr.io/crosstalk-solutions/nomad-sysbench (Debian 12 + sysbench 1.0.20+ds-5, built for linux/amd64 + linux/arm64). One digest covers both architectures; verified that pulling the pinned manifest-list digest resolves to arm64 on a Raspberry Pi 5 and amd64 on x86, and that RepoDigests reports the same manifest-list digest on both — so a single allowlist entry serves both. No rescoring: 1.0.17 -> 1.0.20 measured 1.25% apart on identical hardware with identical flags (7170.18 vs 7259.56 events/sec), inside run-to-run noise and ~0.3% on a composite. Both digests are allowlisted server-side, so the fleet can cross over gradually. 2. REPORT THE DIGEST ACTUALLY RESOLVED The submission previously sent SYSBENCH_DIGEST, the constant the client was compiled with. The leaderboard validates that field, but a constant attests to how a client was BUILT rather than what it RAN, so any build inherits a valid value simply by carrying the same source. Now reads it back from the image. Uses RepoDigests (the manifest digest we pulled by), never Id — Id is the config digest, differs per architecture, and would never match the allowlist. Falls back to the constant if inspection yields nothing usable, so a benchmark never fails over provenance metadata. Still forgeable, and always will be with an open-source client. It moves the bar from "no effort" to "deliberate", which is the distinction that matters when judging whether a submission is a mistake or a choice. 3. RECORD CPU ARCHITECTURE AND OS The leaderboard is a single board across instruction sets by design, with disclosure as the fairness mechanism. Without an architecture field an ARM result sits unlabelled beside x86 — exactly what the disclosure exists to prevent. All three fields come from the Docker daemon, reusing the docker.info() call _detectRunEnvironment already makes. That is deliberate: inside the admin container os.arch() and si.osInfo() describe the CONTAINER, not the host being benchmarked. cpu_architecture Architecture x86_64 -> amd64, aarch64 -> arm64 os_version OSVersion '24.04' (already structured, no parsing) os_name OperatingSystem 'Ubuntu 24.04.4 LTS' minus the version run_environment is kept rather than replaced: "which distro" and "is this virtualised" are different questions, and WSL2 is a real performance factor. String handling lives in app/utils/platform_metadata.ts with unit tests, matching the amd_hsa_override convention, so it is testable without a Docker daemon. Unknown architectures pass through verbatim rather than being guessed at, and os_name falls back to the full description whenever the version is missing or absent from it — an over-long name is harmless, a wrong one is not. Columns are nullable and the submission fields optional, so results recorded before this shipped remain submittable. Closes #1156 Refs #1151 |
||
|---|---|---|
| .. | ||
| benchmark_result.ts | ||
| benchmark_setting.ts | ||
| chat_message.ts | ||
| chat_session.ts | ||
| collection_manifest.ts | ||
| custom_library_source.ts | ||
| drug_label.ts | ||
| installed_resource.ts | ||
| kb_ingest_state.ts | ||
| kb_ratio_registry.ts | ||
| kv_store.ts | ||
| map_marker.ts | ||
| service.ts | ||
| wikipedia_selection.ts | ||