project-nomad/admin/database/migrations
John Cortright 9fde7f7747 feat: show source document and date as citations under RAG chat answers
Closes #1179.

Surfaces the documents an answer was drawn from as a "Sources" list under the
assistant message, with the archive date where one is known, and persists it so
reopening a past conversation still shows what backed each answer.

Offline there is no second opinion to check an answer against, so provenance is
the only signal the user has that a claim came from a real document rather than
from the model. This is the other half of retrieval being allowed to decline:
one says when nothing relevant was found, this says what was found.

Citations are built from the chunks actually injected into the prompt, not from
everything retrieval returned. A chunk dropped by the budget planner never
reached the model, so citing it would credit the answer to a document it was
not based on -- the exact failure the list exists to prevent.

The archive title and date have been written into the vector payload by the ZIM
ingestion path all along and were simply never read back, so this works on
already-embedded content with no re-embed.

Rebased onto the RagPipelineService refactor, which landed after this was
written: the citation list now comes off the pipeline trace rather than the
inline retrieval block that used to live in the controller. Entries with
neither a path nor a title are skipped rather than shown as "Unknown source" --
a row the user cannot act on makes an answer look sourced when it is not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
2026-09-29 15:54:41 -07:00
..
1751086751801_create_services_table.ts feat: openwebui+ollama and zim management 2025-07-09 09:08:21 -07:00
1763499145832_update_services_table.ts feat(Services): friendly names and descriptions 2025-11-18 14:02:22 -08:00
1764912210741_create_curated_collections_table.ts feat: curated zim collections 2025-12-05 15:47:22 -08:00
1764912270123_create_curated_collection_resources_table.ts feat: curated zim collections 2025-12-05 15:47:22 -08:00
1768170944482_update_services_add_installation_statuses_table.ts fix(admin): improve duplicate install request handling 2026-01-13 06:58:05 -08:00
1768453747522_update_services_add_icon.ts feat: improved app cards and custom icons 2026-01-15 15:54:59 -08:00
1769097600001_create_benchmark_results_table.ts feat: Add system benchmark feature with NOMAD Score 2026-01-22 21:48:12 -08:00
1769097600002_create_benchmark_settings_table.ts feat: Add system benchmark feature with NOMAD Score 2026-01-22 21:48:12 -08:00
1769300000001_add_powered_by_and_display_order_to_services.ts feat: Use friendly app names on Dashboard with open source attribution 2026-01-22 16:43:32 -08:00
1769300000002_update_services_friendly_names.ts fix: service name defs and ollama ui location 2026-02-01 05:46:23 +00:00
1769324448000_add_builder_tag_to_benchmark_results.ts fix(migrations): Fix timestamp ordering for builder_tag migration 2026-01-28 13:43:21 -08:00
1769400000001_create_installed_tiers_table.ts feat: Improve curated collections UX with persistent tier selection 2026-01-24 15:33:50 -08:00
1769400000002_create_kv_store_table.ts feat(RAG): [wip] RAG capabilities 2026-01-31 20:39:49 -08:00
1769500000001_create_wikipedia_selection_table.ts feat: Add dedicated Wikipedia Selector with smart package management 2026-01-31 21:00:51 -08:00
1769646771604_create_create_chat_sessions_table.ts feat: [wip] native AI chat interface 2026-01-31 20:39:49 -08:00
1769646798266_create_create_chat_messages_table.ts feat: [wip] native AI chat interface 2026-01-31 20:39:49 -08:00
1769700000001_create_zim_file_metadata_table.ts feat(ContentManager): Display friendly names instead of filenames 2026-02-03 23:14:28 -08:00
1770269324176_add_unique_constraint_to_curated_collection_resources_table.ts fix(Curated Collections): ensure resources are not duplicated on fetch-latest 2026-02-04 21:35:18 -08:00
1770273423670_drop_installed_tiers_table.ts fix: rework content tier system to dynamically determine install status 2026-02-04 22:58:21 -08:00
1770849108030_create_create_collection_manifests_table.ts feat: curated content system overhaul 2026-02-11 15:44:46 -08:00
1770849119787_create_create_installed_resources_table.ts feat: curated content system overhaul 2026-02-11 15:44:46 -08:00
1770850092871_create_drop_legacy_curated_tables_table.ts feat: curated content system overhaul 2026-02-11 15:44:46 -08:00
1771000000001_add_update_fields_to_services.ts feat: support for updating services 2026-03-11 14:08:09 -07:00
1771000000002_pin_latest_service_images.ts feat: support for updating services 2026-03-11 14:08:09 -07:00
1771100000001_migrate_kiwix_to_library_mode.ts feat(Kiwix): migrate to Kiwix library mode for improved stability (#622) 2026-04-03 14:26:50 -07:00
1771200000001_create_map_markers_table.ts feat(maps): add scale bar and location markers (#636) 2026-04-03 14:26:50 -07:00
1772000000001_add_supply_depot_fields_to_services.ts feat: supply depot 2026-06-23 04:46:45 +00:00
1772000000002_add_user_modified_to_services.ts feat: edit curated apps + fix dropdown clip + stale _old rollback 2026-06-23 04:46:46 +00:00
1772000000003_add_app_auto_update_fields_to_services.ts feat(supply-depot): opt-in automatic updates for installed apps 2026-06-23 04:46:52 +00:00
1775100000001_create_custom_library_sources_table.ts feat(Content): custom ZIM library sources with pre-seeded mirrors (#593) 2026-05-20 10:16:00 -07:00
1776000000001_create_kb_ingest_state_table.ts feat(KB): per-file ingest state machine (Phase 1 of RFC #883) (#888) 2026-05-20 10:16:00 -07:00
1776100000001_create_kb_ratio_registry_table.ts fix(KB): align chunks_per_mb column type with TS contract 2026-05-20 10:16:00 -07:00
1776200000001_add_content_auto_update_fields_to_installed_resources.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
1776200000001_add_custom_url_to_services.ts feat(supply-depot): add custom launch URLs for apps 2026-06-23 04:46:56 +00:00
1776300000001_deprecate_legacy_kolibri.ts feat: replace legacy Kolibri image default with latest v19 image (#1019) 2026-06-23 04:47:07 +00:00
1776400000001_add_benchmark_harness_metadata.ts feat(benchmark): harness hardening — fail loudly + pin sysbench + record provenance (#1089) 2026-08-04 03:37:41 +00:00
1776400000001_add_collection_to_kb_ingest_state.ts feat(rag): add subject/collection organization to knowledge base (#1063) 2026-08-04 03:37:42 +00:00
1776400000001_sunset_meshtastic_daemon.ts chore(catalog): sunset orphaned Meshtastic Daemon card (#1049) 2026-08-04 03:37:31 +00:00
1776400000002_add_benchmark_score_v2.ts feat(benchmark): Score v2 app client — raws, uncapped score, v2 payload + UI 2026-08-04 03:38:30 +00:00
1776400000003_add_benchmark_platform_metadata.ts feat(benchmark): official multi-arch sysbench, resolved digest, platform metadata (#1158) 2026-08-04 03:39:53 +00:00
1778600000004_create_drug_labels_table.ts Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040) 2026-08-04 03:38:33 +00:00
1778600000006_add_indications_fulltext_index.ts Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040) 2026-08-04 03:38:33 +00:00
1778700000006_add_ingested_at_default_drug_labels.ts Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040) 2026-08-04 03:38:33 +00:00
1785468975052_create_add_sources_to_chat_messages_table.ts feat: show source document and date as citations under RAG chat answers 2026-09-29 15:54:41 -07:00