mirror of
https://github.com/Crosstalk-Solutions/project-nomad.git
synced 2026-10-17 05:14:19 +02:00
363a99818c
Phase 4 of the NOMAD Score v2 effort (stacked on #1089). Captures the raw channel values the leaderboard scores from, computes the uncapped v2 score in-app (byte-matching the leaderboard's server-side recompute), submits the v2 payload, and surfaces it in the benchmark UI. - Capture raws: per-channel sysbench values + O_DIRECT disk (W4), single/multi thread CPU + memory, W6 consistency companions, and run-environment metadata (run_environment, storage_path_type, gpu_compute_detected, #1016). - AI hardening: reference model llama3.1:8b, num_predict=256, VRAM eviction before the run + unload own model after, pre-flight disk check (~6.5GB, only when the model is uncached), median-of-3 run (W7). - Score: _calculateNomadScoreV2 (uncapped, reference->1000) byte-matches the leaderboard recompute. nomad_score_v2 stays null unless a full run with AI. - Payload: submitToRepository sends the v2 raws + score alongside legacy v1. - UI: v2 headline score, legacy v1 as secondary, raws/env in details. - Migration: 13 nullable columns; double (not knex float(8,2)) so millions-scale raws keep full precision and don't break the leaderboard byte-match. Reference constant REFERENCE_SCORES_V2.ai_tokens_per_second is still the stale 40.5 placeholder (TODO in source); locking it to the measured 13.2 is a follow-up gated on the AMD iGPU fix (#1074). Live prod v2 submit is separately blocked by the leaderboard TTFT floor (tracked in the leaderboard follow-up). Verified on NOMAD3 (RTX 5060) dev env: migration applies, raws recorded, v2 score computed, backend + benchmark.tsx typecheck clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| 1751086751801_create_services_table.ts | ||
| 1763499145832_update_services_table.ts | ||
| 1764912210741_create_curated_collections_table.ts | ||
| 1764912270123_create_curated_collection_resources_table.ts | ||
| 1768170944482_update_services_add_installation_statuses_table.ts | ||
| 1768453747522_update_services_add_icon.ts | ||
| 1769097600001_create_benchmark_results_table.ts | ||
| 1769097600002_create_benchmark_settings_table.ts | ||
| 1769300000001_add_powered_by_and_display_order_to_services.ts | ||
| 1769300000002_update_services_friendly_names.ts | ||
| 1769324448000_add_builder_tag_to_benchmark_results.ts | ||
| 1769400000001_create_installed_tiers_table.ts | ||
| 1769400000002_create_kv_store_table.ts | ||
| 1769500000001_create_wikipedia_selection_table.ts | ||
| 1769646771604_create_create_chat_sessions_table.ts | ||
| 1769646798266_create_create_chat_messages_table.ts | ||
| 1769700000001_create_zim_file_metadata_table.ts | ||
| 1770269324176_add_unique_constraint_to_curated_collection_resources_table.ts | ||
| 1770273423670_drop_installed_tiers_table.ts | ||
| 1770849108030_create_create_collection_manifests_table.ts | ||
| 1770849119787_create_create_installed_resources_table.ts | ||
| 1770850092871_create_drop_legacy_curated_tables_table.ts | ||
| 1771000000001_add_update_fields_to_services.ts | ||
| 1771000000002_pin_latest_service_images.ts | ||
| 1771100000001_migrate_kiwix_to_library_mode.ts | ||
| 1771200000001_create_map_markers_table.ts | ||
| 1772000000001_add_supply_depot_fields_to_services.ts | ||
| 1772000000002_add_user_modified_to_services.ts | ||
| 1772000000003_add_app_auto_update_fields_to_services.ts | ||
| 1775100000001_create_custom_library_sources_table.ts | ||
| 1776000000001_create_kb_ingest_state_table.ts | ||
| 1776100000001_create_kb_ratio_registry_table.ts | ||
| 1776200000001_add_content_auto_update_fields_to_installed_resources.ts | ||
| 1776200000001_add_custom_url_to_services.ts | ||
| 1776300000001_deprecate_legacy_kolibri.ts | ||
| 1776400000001_add_benchmark_harness_metadata.ts | ||
| 1776400000001_add_collection_to_kb_ingest_state.ts | ||
| 1776400000001_sunset_meshtastic_daemon.ts | ||
| 1776400000002_add_benchmark_score_v2.ts | ||