perf python: Port export-to-postgresql to perf module

Port export-to-postgresql.py to a standalone script in
tools/perf/python/ using the perf module and libpq via ctypes.

Improvements compared to the legacy script:
- Remove the dependency on PySide/QtSql for database creation and DDL
  execution by driving libpq directly via ctypes (PQconnectdb, PQexec,
  PQputCopyData, PQputCopyEnd) and streaming binary PostgreSQL COPY
  files in PostgresExporter, enabling export on headless servers without
  Qt installed.
- Harden database connection and SQL identifier handling by rejecting
  URI ('://') and connection-parameter ('=') injection strings in
  setup_db() and connect(), escaping single quotes and backslashes in
  connection strings, and quoting SQL identifiers (quote_ident).
- Support Intel PT and instruction trace export via
  perf.session(itrace=...) and perf.call_return callbacks, reconstructing
  relational call_paths and calls tables (including id=0 placeholder
  rows and callfk/returnfk foreign keys) and unpacking synthesized PT
  payloads (ptwrite, cbr, mwait, pwre, exstop, pwrx).
- Use a dedicated context_switch ID counter so context-switch exports
  do not advance sample database IDs out of sync with call_return
  references.

Update Documentation/db-export.txt and add a shell test
(test_export_to_postgresql_python.sh) to verify the standalone exporter.

Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This commit is contained in:
Ian Rogers 2026-09-25 23:20:10 -07:00 committed by Arnaldo Carvalho de Melo
parent 62d350135e
commit b1f968c965
3 changed files with 1451 additions and 1 deletions

View File

@ -8,7 +8,7 @@ perf tool's python scripting engine:
supports scripts: supports scripts:
tools/perf/python/export-to-sqlite.py tools/perf/python/export-to-sqlite.py
tools/perf/scripts/python/export-to-postgresql.py tools/perf/python/export-to-postgresql.py
which export data to a SQLite3 or PostgreSQL database. which export data to a SQLite3 or PostgreSQL database.

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,122 @@
#!/bin/bash
# SPDX-License-Identifier: GPL-2.0
# export-to-postgresql python test (exclusive)
set -e
shelldir=$(dirname "$0")
# shellcheck source=lib/setup_python.sh
. "${shelldir}"/lib/setup_python.sh
if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
echo "Skipping test, perf python module not found"
exit 2
fi
# If we don't have psql, we can't test
if ! command -v psql >/dev/null 2>&1; then
echo "Skipping test, psql not found"
exit 2
fi
# Check if we can connect to postgres and create a database
if ! psql -c "SELECT 1" postgres >/dev/null 2>&1; then
echo "Skipping test, cannot connect to postgres"
exit 2
fi
script_dir="$(dirname "$0")/../../python"
script_path="${script_dir}/export-to-postgresql.py"
if [ ! -f "$script_path" ]; then
echo "Skipping test, export-to-postgresql.py not found at $script_path"
exit 2
fi
err=0
temp_dir=""
temp_data=""
temp_db=""
cleanup() {
[ -n "${temp_dir}" ] && rm -rf "${temp_dir}"
psql -c "DROP DATABASE IF EXISTS ${temp_db}" postgres >/dev/null 2>&1 || true
}
trap 'cleanup' EXIT
trap 'cleanup; exit 1' TERM INT
temp_dir=$(mktemp -d /tmp/perf.pg.XXXXXX)
temp_data="${temp_dir}/perf.data"
temp_db="perf_test_db_$$"
test_file_mode() {
echo "Testing export-to-postgresql.py..."
# Verify that invalid URI / key-value database names are rejected
if "$PYTHON" "$script_path" -i /dev/null -o "dbname=foo host=evil" >/dev/null 2>&1; then
echo "Connection parameter injection check failed."
err=1
fi
# Generate events with callchains and context switches
if ! perf record -g --switch-events -o "${temp_data}" \
-- perf test -w noploop >/dev/null 2>&1 && \
! perf record -g -o "${temp_data}" -- perf test -w noploop >/dev/null 2>&1; then
echo "Skipping test, perf record failed"
exit 2
fi
# Ensure clean db start
psql -c "DROP DATABASE IF EXISTS ${temp_db}" postgres >/dev/null 2>&1 || true
# Run the script
if ! "$PYTHON" "$script_path" -i "${temp_data}" -o "${temp_db}" >/dev/null; then
echo "File mode test failed."
err=1
else
# Check DB
if ! psql -d "${temp_db}" -t -c 'SELECT COUNT(*) FROM samples WHERE id > 0;' | \
grep -q '[1-9]' || \
! psql -d "${temp_db}" -t -c 'SELECT COUNT(*) FROM call_paths WHERE id > 0;' | \
grep -q '[1-9]'; then
echo "PostgreSQL validation failed."
err=1
else
echo "File mode test passed."
fi
fi
}
test_intel_pt() {
echo "Testing export-to-postgresql.py with intel_pt..."
psql -c "DROP DATABASE IF EXISTS ${temp_db}" postgres >/dev/null 2>&1 || true
rm -f "${temp_data}"
# Generate some intel_pt events; use a subshell that waits for uname
if ! perf record -B -N --no-bpf-event -e intel_pt//u -o "${temp_data}" \
-- sh -c "uname; true" >/dev/null 2>&1; then
echo "Skipping intel_pt test, intel_pt not available."
return 0
fi
# Run the script with --itrace cr to synthesize call_returns
if ! "$PYTHON" "$script_path" -i "${temp_data}" -o "${temp_db}" --itrace cr >/dev/null; then
echo "intel_pt file mode test failed."
err=1
else
# Check DB for calls
if ! psql -d "${temp_db}" -t -c 'SELECT COUNT(*) FROM calls WHERE id > 0;' | \
grep -q '[1-9]'; then
echo "PostgreSQL intel_pt validation failed (no calls found)."
err=1
else
echo "intel_pt test passed (cr validated)."
fi
fi
}
test_file_mode
test_intel_pt
exit $err