mirror of
https://github.com/Crosstalk-Solutions/project-nomad.git
synced 2026-07-28 19:24:39 +02:00
0891d176e5
Three changes that together let ARM hardware appear on the leaderboard honestly. Shipping them separately would leave ARM half-supported either way: without the image a Pi cannot submit at all, and without the architecture field it submits but is indistinguishable from x86. 1. PIN THE OFFICIAL MULTI-ARCH SYSBENCH IMAGE severalnines/sysbench publishes amd64 only, so ARM hosts could not run the System Benchmark at all — not a graceful failure, the container simply cannot execute. Apple Silicon could only run it under Rosetta emulation, which distorts the measurement it is taking, and that is what drove a community macOS fork to substitute a different benchmark and submit incomparable numbers. Swaps to ghcr.io/crosstalk-solutions/nomad-sysbench (Debian 12 + sysbench 1.0.20+ds-5, built for linux/amd64 + linux/arm64). One digest covers both architectures; verified that pulling the pinned manifest-list digest resolves to arm64 on a Raspberry Pi 5 and amd64 on x86, and that RepoDigests reports the same manifest-list digest on both — so a single allowlist entry serves both. No rescoring: 1.0.17 -> 1.0.20 measured 1.25% apart on identical hardware with identical flags (7170.18 vs 7259.56 events/sec), inside run-to-run noise and ~0.3% on a composite. Both digests are allowlisted server-side, so the fleet can cross over gradually. 2. REPORT THE DIGEST ACTUALLY RESOLVED The submission previously sent SYSBENCH_DIGEST, the constant the client was compiled with. The leaderboard validates that field, but a constant attests to how a client was BUILT rather than what it RAN, so any build inherits a valid value simply by carrying the same source. Now reads it back from the image. Uses RepoDigests (the manifest digest we pulled by), never Id — Id is the config digest, differs per architecture, and would never match the allowlist. Falls back to the constant if inspection yields nothing usable, so a benchmark never fails over provenance metadata. Still forgeable, and always will be with an open-source client. It moves the bar from "no effort" to "deliberate", which is the distinction that matters when judging whether a submission is a mistake or a choice. 3. RECORD CPU ARCHITECTURE AND OS The leaderboard is a single board across instruction sets by design, with disclosure as the fairness mechanism. Without an architecture field an ARM result sits unlabelled beside x86 — exactly what the disclosure exists to prevent. All three fields come from the Docker daemon, reusing the docker.info() call _detectRunEnvironment already makes. That is deliberate: inside the admin container os.arch() and si.osInfo() describe the CONTAINER, not the host being benchmarked. cpu_architecture Architecture x86_64 -> amd64, aarch64 -> arm64 os_version OSVersion '24.04' (already structured, no parsing) os_name OperatingSystem 'Ubuntu 24.04.4 LTS' minus the version run_environment is kept rather than replaced: "which distro" and "is this virtualised" are different questions, and WSL2 is a real performance factor. String handling lives in app/utils/platform_metadata.ts with unit tests, matching the amd_hsa_override convention, so it is testable without a Docker daemon. Unknown architectures pass through verbatim rather than being guessed at, and os_name falls back to the full description whenever the version is missing or absent from it — an over-long name is harmless, a wrong one is not. Columns are nullable and the submission fields optional, so results recorded before this shipped remain submittable. Closes #1156 Refs #1151 |
||
|---|---|---|
| .. | ||
| 1751086751801_create_services_table.ts | ||
| 1763499145832_update_services_table.ts | ||
| 1764912210741_create_curated_collections_table.ts | ||
| 1764912270123_create_curated_collection_resources_table.ts | ||
| 1768170944482_update_services_add_installation_statuses_table.ts | ||
| 1768453747522_update_services_add_icon.ts | ||
| 1769097600001_create_benchmark_results_table.ts | ||
| 1769097600002_create_benchmark_settings_table.ts | ||
| 1769300000001_add_powered_by_and_display_order_to_services.ts | ||
| 1769300000002_update_services_friendly_names.ts | ||
| 1769324448000_add_builder_tag_to_benchmark_results.ts | ||
| 1769400000001_create_installed_tiers_table.ts | ||
| 1769400000002_create_kv_store_table.ts | ||
| 1769500000001_create_wikipedia_selection_table.ts | ||
| 1769646771604_create_create_chat_sessions_table.ts | ||
| 1769646798266_create_create_chat_messages_table.ts | ||
| 1769700000001_create_zim_file_metadata_table.ts | ||
| 1770269324176_add_unique_constraint_to_curated_collection_resources_table.ts | ||
| 1770273423670_drop_installed_tiers_table.ts | ||
| 1770849108030_create_create_collection_manifests_table.ts | ||
| 1770849119787_create_create_installed_resources_table.ts | ||
| 1770850092871_create_drop_legacy_curated_tables_table.ts | ||
| 1771000000001_add_update_fields_to_services.ts | ||
| 1771000000002_pin_latest_service_images.ts | ||
| 1771100000001_migrate_kiwix_to_library_mode.ts | ||
| 1771200000001_create_map_markers_table.ts | ||
| 1772000000001_add_supply_depot_fields_to_services.ts | ||
| 1772000000002_add_user_modified_to_services.ts | ||
| 1772000000003_add_app_auto_update_fields_to_services.ts | ||
| 1775100000001_create_custom_library_sources_table.ts | ||
| 1776000000001_create_kb_ingest_state_table.ts | ||
| 1776100000001_create_kb_ratio_registry_table.ts | ||
| 1776200000001_add_content_auto_update_fields_to_installed_resources.ts | ||
| 1776200000001_add_custom_url_to_services.ts | ||
| 1776300000001_deprecate_legacy_kolibri.ts | ||
| 1776400000001_add_benchmark_harness_metadata.ts | ||
| 1776400000001_add_collection_to_kb_ingest_state.ts | ||
| 1776400000001_sunset_meshtastic_daemon.ts | ||
| 1776400000002_add_benchmark_score_v2.ts | ||
| 1776400000003_add_benchmark_platform_metadata.ts | ||
| 1778600000004_create_drug_labels_table.ts | ||
| 1778600000006_add_indications_fulltext_index.ts | ||
| 1778700000006_add_ingested_at_default_drug_labels.ts | ||