feat: Search annotations in pipeline run API

[yuechao-qin]

TL;DR

Implemented annotation-based filtering for Pipeline Run API and proper pagination handling.

What changed?

API GET /api/pipeline_runs/

Functionality

• Implemented support for filter_query which handles annotation searches in pipeline runs.
• Added new database index for annotation-based queries. Covering index (key, pipeline_run_id, value).
• Enhanced pagination to preserve both legacy filter and new filter_query across pages.

Other

• Created new filter_query_sql.py module with comprehensive filter query processing:
  • PageToken class for encoding/decoding pagination state
  • Support for annotation-based predicates: key_exists, value_equals, value_contains, value_in
  • Logical operators: and, or, not with proper nesting
  • Integration with existing legacy filter system
• Refactored pipeline run listing to use the new filter query system while maintaining backward compatibility

How to test?

uv run pytest tests/test_api_server_sql.py tests/test_filter_query_sql.py

• Test basic annotation filtering: {"and": [{"key_exists": {"key": "team"}}]}
• Test value-based filtering: {"and": [{"value_equals": {"key": "env", "value": "prod"}}]}
• Test logical combinations with nested and/or/not predicates
• Verify mutual exclusivity validation between filter and filter_query parameters
• Test pagination preservation of filter_query state across multiple pages
• Confirm legacy filter functionality remains unchanged

Why make this change?

Annotation search support is the backbone for all the upcoming search features (create_by, pipeline name, status) for Pipeline Runs. The other searches will be search against the annotations table.

[yuechao-qin]

• feat: Add include_sql to Search Pipeline Run API #135
• feat: API Server - Pipeline Runs - Changed the pagination token to a cursor #131
• feat: Search within date range in pipeline run API #130
• feat: Search pipeline name in pipeline run API #129
• refactor: Convert legacy filter to be filter_query #128
• feat: Search created by user in pipeline run API #108
• feat: Search annotations in pipeline run API #107 👈 (View in Graphite)
• feat: Create Pydantic models for Search Pipeline Run API #106
• refactor: Search pipeline run refactoring and baseline unit tests #105
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[Ark-kun]

Approved with comments.

[Ark-kun]

Approved. But please resolve the comments.

[yuechao-qin]

Merge activity

• Mar 6, 8:42 PM UTC: A user started a stack merge that includes this pull request via Graphite.
• Mar 6, 8:45 PM UTC: Graphite rebased this pull request as part of a merge.
• Mar 6, 8:46 PM UTC: @yuechao-qin merged this pull request with Graphite.