Skip to content

Update search parameters to better leverage DB indexes#64963

Merged
pierrejeambrun merged 3 commits into
apache:mainfrom
astronomer:feature/api-prefix-pattern-search
Apr 24, 2026
Merged

Update search parameters to better leverage DB indexes#64963
pierrejeambrun merged 3 commits into
apache:mainfrom
astronomer:feature/api-prefix-pattern-search

Conversation

@pierrejeambrun
Copy link
Copy Markdown
Member

@pierrejeambrun pierrejeambrun commented Apr 9, 2026

dag_id and run_id are significantly faster now when using the endpoints because they now leverage indexes.

task_id is significantly faster only when a dag_id_pattern or when the dag_id path param are specified. This is explained in the documentation. It's not possible to leverage any existing db index without narrowing the dag_id selection. (users can still add their own indexes if they need to as explained in the doc)


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: [Cursor] following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.
@boring-cyborg boring-cyborg Bot added area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. labels Apr 9, 2026
@pierrejeambrun pierrejeambrun force-pushed the feature/api-prefix-pattern-search branch from 9390e4f to 16c5739 Compare April 9, 2026 13:06
@pierrejeambrun pierrejeambrun marked this pull request as draft April 9, 2026 13:08
@kaxil kaxil requested a review from Copilot April 10, 2026 19:55
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR changes API “pattern” filters to use index-friendly prefix matching and introduces cursor-based (keyset) pagination for task-instance listings, updating backend models, OpenAPI artifacts, and the UI accordingly.

Changes:

  • Replace substring ILIKE '%term%' searches with prefix range filtering (and | OR support) to better leverage B-tree indexes.
  • Add cursor-based pagination to the Task Instances endpoint (alongside existing offset pagination) and propagate new response shapes through generated clients and UI.
  • Update/extend unit tests and add a significant newsfragment describing the behavior change.

Reviewed changes

Copilot reviewed 35 out of 35 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
airflow-ctl/src/airflowctl/api/datamodels/generated.py Updates generated client models to reflect new cursor/offset TI collection responses.
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_variables.py Adjusts variables pattern tests for new prefix semantics.
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_task_instances.py Adds cursor pagination tests and updates TI response assertions for new pagination discriminator.
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_connections.py Adjusts connections pattern test to match new prefix behavior.
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_assets.py Updates assets pattern tests to align with prefix matching behavior.
airflow-core/tests/unit/api_fastapi/common/test_parameters.py Updates parameter tests for new prefix-range SQL generation and task display name param behavior.
airflow-core/tests/unit/api_fastapi/common/test_cursors.py Adds unit tests for cursor encode/decode and keyset filtering helpers.
airflow-core/src/airflow/ui/src/queries/useClearTaskInstances.ts Updates UI query types to new offset TI collection response type.
airflow-core/src/airflow/ui/src/queries/useClearDagRunDryRun.ts Updates generic default type to new offset TI collection response type.
airflow-core/src/airflow/ui/src/queries/useBulkMarkAsDryRun.ts Updates dry-run helper types/constants to offset TI collection response type.
airflow-core/src/airflow/ui/src/queries/useBulkClearDryRun.ts Updates bulk clear dry-run helper types/constants to offset TI collection response type.
airflow-core/src/airflow/ui/src/pages/TaskInstances/TaskInstances.tsx Switches TI list UI to cursor pagination and wires next/previous cursor controls.
airflow-core/src/airflow/ui/src/pages/Task/Overview/Overview.tsx Makes overview resilient to cursor responses by safely reading total_entries.
airflow-core/src/airflow/ui/src/pages/Dag/Overview/Overview.tsx Makes DAG overview resilient to cursor responses by safely reading total_entries.
airflow-core/src/airflow/ui/src/pages/Dag/Overview/FailedLogs.tsx Broadens TI response type to the new union response.
airflow-core/src/airflow/ui/src/components/DataTable/index.ts Re-exports cursor pagination prop type.
airflow-core/src/airflow/ui/src/components/DataTable/DataTable.tsx Adds cursor-pagination UI controls and disables offset pagination when active.
airflow-core/src/airflow/ui/src/components/ActionAccordion/ActionAccordion.tsx Updates affected-tasks response type to the new offset TI collection response.
airflow-core/src/airflow/ui/openapi-gen/requests/types.gen.ts Regenerates OpenAPI TS types for prefix matching + cursor/offset pagination models.
airflow-core/src/airflow/ui/openapi-gen/requests/services.gen.ts Regenerates OpenAPI TS services docs/signatures for updated query params/responses.
airflow-core/src/airflow/ui/openapi-gen/requests/schemas.gen.ts Regenerates OpenAPI TS schemas for new TI pagination responses.
airflow-core/src/airflow/ui/openapi-gen/queries/suspense.ts Regenerates query hooks docs/types for updated params and responses.
airflow-core/src/airflow/ui/openapi-gen/queries/queries.ts Regenerates query hooks docs/types for updated params and responses.
airflow-core/src/airflow/ui/openapi-gen/queries/prefetch.ts Regenerates prefetch helpers for updated params and responses.
airflow-core/src/airflow/ui/openapi-gen/queries/ensureQueryData.ts Regenerates ensureQueryData helpers for updated params and responses.
airflow-core/src/airflow/ui/openapi-gen/queries/common.ts Updates query keys to include cursor param.
airflow-core/src/airflow/api_fastapi/core_api/routes/public/task_instances.py Implements cursor pagination path + returns new offset/cursor response models.
airflow-core/src/airflow/api_fastapi/core_api/routes/public/dag_run.py Updates clear DAG run response type to new offset TI collection response.
airflow-core/src/airflow/api_fastapi/core_api/openapi/v2-rest-api-generated.yaml Updates OpenAPI spec for prefix matching docs and TI cursor/offset response union.
airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml Updates private UI OpenAPI documentation for prefix matching.
airflow-core/src/airflow/api_fastapi/core_api/datamodels/task_instances.py Introduces cursor/offset TI response models and discriminator union alias.
airflow-core/src/airflow/api_fastapi/core_api/datamodels/common.py Adds base response models for offset vs cursor pagination.
airflow-core/src/airflow/api_fastapi/common/parameters.py Implements prefix-range params, task display name pattern rewrite, and SortParam resolved-column caching.
airflow-core/src/airflow/api_fastapi/common/cursors.py Adds reusable cursor encode/decode and keyset WHERE-clause helper.
airflow-core/newsfragments/64963.significant.rst Documents the behavioral change from substring to prefix pattern matching.
Comment thread airflow-core/src/airflow/api_fastapi/core_api/routes/public/task_instances.py Outdated
Comment thread airflow-core/src/airflow/api_fastapi/common/parameters.py
Comment thread airflow-core/src/airflow/api_fastapi/core_api/datamodels/common.py Outdated
Comment thread airflow-core/src/airflow/ui/src/components/DataTable/DataTable.tsx Outdated
Comment thread airflow-core/src/airflow/api_fastapi/core_api/routes/public/task_instances.py Outdated
@pierrejeambrun pierrejeambrun force-pushed the feature/api-prefix-pattern-search branch from 16c5739 to df9d9f1 Compare April 21, 2026 13:14
@pierrejeambrun pierrejeambrun marked this pull request as ready for review April 21, 2026 16:07
Copy link
Copy Markdown
Contributor

@bbovenzi bbovenzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm like the prefixPattern for a more efficient search while keeping the old contains search for backward-compatibility and to let users still search by any substring.

We just need to rebase.

@pierrejeambrun pierrejeambrun force-pushed the feature/api-prefix-pattern-search branch from ae7573c to 6d673a0 Compare April 22, 2026 13:54
@bbovenzi
Copy link
Copy Markdown
Contributor

Approval except for the need for a rebase and fixing CI tests

@pierrejeambrun pierrejeambrun force-pushed the feature/api-prefix-pattern-search branch from 2a76351 to a610b03 Compare April 24, 2026 09:06
Search query parameters on list endpoints now use prefix matching (LIKE 'value%') instead of substring search, which allows the database to use indexes more effectively.

This is a breaking change for clients that relied on matching arbitrary substrings in names and identifiers. Regenerate OpenAPI specs and UI client types accordingly.
Keep the existing *_pattern substring filters unchanged and add
parallel *_prefix_pattern variants that use a B-tree-friendly
range comparison, so large deployments can filter list endpoints
without paying for a full table scan.

Drop custom TaskDisplayName substring class; replace feature newsfragment with significant

Full-match on ``coalesce(...)`` can't use an index anyway, so the
custom ``_TaskDisplayNamePatternParam`` class adds nothing over a
plain ``_SearchParam``: use ``search_param_factory`` directly for
the substring variant and keep the custom prefix variant since the
split on ``IS NULL`` is what makes the B-tree index apply.

Swap 64963.feature for 64963.significant to flag the user-visible
UI change (search-as-you-type is now prefix-based, not substring).

Fix CI

Fix CI
@pierrejeambrun pierrejeambrun force-pushed the feature/api-prefix-pattern-search branch from a610b03 to 85acf80 Compare April 24, 2026 09:14
The fixtures had names like search_production_<ts> but the UI search now
hits the prefix-match API (connection_id_prefix_pattern). Move the
environment word to the front (production_search_<ts>, staging_search_<ts>,
development_search_<ts>) so "production" is a valid prefix.
@pierrejeambrun pierrejeambrun added this to the Airflow 3.2.2 milestone Apr 24, 2026
@pierrejeambrun pierrejeambrun self-assigned this Apr 24, 2026
@pierrejeambrun pierrejeambrun added the backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch label Apr 24, 2026
@pierrejeambrun pierrejeambrun merged commit 7336ed4 into apache:main Apr 24, 2026
141 checks passed
@pierrejeambrun pierrejeambrun deleted the feature/api-prefix-pattern-search branch April 24, 2026 14:10
@github-actions
Copy link
Copy Markdown
Contributor

Backport failed to create: v3-2-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-2-test Commit Link

You can attempt to backport this manually by running:

cherry_picker 7336ed4 v3-2-test

This should apply the commit to the v3-2-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

@dheerajturaga
Copy link
Copy Markdown
Member

@pierrejeambrun, @bbovenzi This PR regressed DAG search in the UI. it now only matches DAG IDs that start with the search term, where previously substring search worked. My DAG IDs follow patterns like , substring is how users actually find DAGs. This is a big problem imo, Can we switch the UI back to dag_id_pattern / dag_display_name_pattern ?

@pierrejeambrun
Copy link
Copy Markdown
Member Author

The problem is that, the former version isn't scalable and cause delays / unusable UI on really big airflow installation so we need to have this option.

What we could imagine is a configurable UI option to opt back in to subtring search ?

pierrejeambrun added a commit that referenced this pull request Apr 28, 2026
* API: Use prefix matching for REST list search parameters

Search query parameters on list endpoints now use prefix matching (LIKE 'value%') instead of substring search, which allows the database to use indexes more effectively.

This is a breaking change for clients that relied on matching arbitrary substrings in names and identifiers. Regenerate OpenAPI specs and UI client types accordingly.

* API: Add index-friendly *_prefix_pattern query params (non-breaking)

Keep the existing *_pattern substring filters unchanged and add
parallel *_prefix_pattern variants that use a B-tree-friendly
range comparison, so large deployments can filter list endpoints
without paying for a full table scan.

Drop custom TaskDisplayName substring class; replace feature newsfragment with significant

Full-match on ``coalesce(...)`` can't use an index anyway, so the
custom ``_TaskDisplayNamePatternParam`` class adds nothing over a
plain ``_SearchParam``: use ``search_param_factory`` directly for
the substring variant and keep the custom prefix variant since the
split on ``IS NULL`` is what makes the B-tree index apply.

Swap 64963.feature for 64963.significant to flag the user-visible
UI change (search-as-you-type is now prefix-based, not substring).

Fix CI

Fix CI

* UI: Fix connections search e2e fixtures for prefix-match API

The fixtures had names like search_production_<ts> but the UI search now
hits the prefix-match API (connection_id_prefix_pattern). Move the
environment word to the front (production_search_<ts>, staging_search_<ts>,
development_search_<ts>) so "production" is a valid prefix.

(cherry picked from commit 7336ed4)
@pierrejeambrun
Copy link
Copy Markdown
Member Author

#66015 Follow up PR where you can opt in for full substring match

pierrejeambrun added a commit to astronomer/airflow that referenced this pull request Apr 28, 2026
Move the user-visible note from `64963.significant.rst` to a new
`66015.significant.rst` (this PR's number) and split it into two
paragraphs:

- The first paragraph references apache#64963 — the change from full-match
  `*_pattern` to index-friendly `*_prefix_pattern`.
- The second paragraph references apache#66015 — the new per-searchbar
  "Match anywhere" toggle that lets users opt back into substring
  search.
pierrejeambrun added a commit to astronomer/airflow that referenced this pull request Apr 28, 2026
Follow-up to apache#64963. The default search now uses the new prefix patterns
(``*_prefix_pattern``) for index-friendly lookups. Users who relied on
the prior substring behavior (``*_pattern``) can opt back in per
searchbar via a small regex-icon toggle.

Both surfaces support it:

* ``SearchBar`` — page-level inputs on Pools, Connections, Variables,
  Assets, Asset events, and Dags lists.
* ``FilterBar`` text pills — when ``supportsAdvancedSearch`` is set on
  the FilterConfig, the pill renders the toggle inline while editing
  and shows a regex indicator next to the value when collapsed. Wired
  on TaskInstances, DagRuns, Events, XCom, and HITL.

URL is intentionally untouched: shared links default to fast prefix
mode; toggle state is per-searchbar in ``localStorage``
(``advanced_search-<id>``). The toggle uses ``onMouseDown`` +
``preventDefault`` so clicking it inside a FilterPill does not collapse
the pill.
pierrejeambrun added a commit to astronomer/airflow that referenced this pull request Apr 28, 2026
Move the user-visible note from `64963.significant.rst` to a new
`66015.significant.rst` (this PR's number) and split it into two
paragraphs:

- The first paragraph references apache#64963 — the change from full-match
  `*_pattern` to index-friendly `*_prefix_pattern`.
- The second paragraph references apache#66015 — the new per-searchbar
  "Match anywhere" toggle that lets users opt back into substring
  search.
bbovenzi pushed a commit that referenced this pull request Apr 29, 2026
…66015)

* UI: Add per-searchbar 'Match anywhere' toggle for substring search

Follow-up to #64963. The default search now uses the new prefix patterns
(``*_prefix_pattern``) for index-friendly lookups. Users who relied on
the prior substring behavior (``*_pattern``) can opt back in per
searchbar via a small regex-icon toggle.

Both surfaces support it:

* ``SearchBar`` — page-level inputs on Pools, Connections, Variables,
  Assets, Asset events, and Dags lists.
* ``FilterBar`` text pills — when ``supportsAdvancedSearch`` is set on
  the FilterConfig, the pill renders the toggle inline while editing
  and shows a regex indicator next to the value when collapsed. Wired
  on TaskInstances, DagRuns, Events, XCom, and HITL.

URL is intentionally untouched: shared links default to fast prefix
mode; toggle state is per-searchbar in ``localStorage``
(``advanced_search-<id>``). The toggle uses ``onMouseDown`` +
``preventDefault`` so clicking it inside a FilterPill does not collapse
the pill.

* Move toggle inside FilterPill input, drop warning palette

Address review feedback:

- Move the advanced-search toggle from a sibling next to the InputWithAddon
  to a real ``endAddon`` slot inside the same rounded box. The toggle now
  sits flush on the right of the pill input, mirroring the "Pool:" label on
  the left.
- Drop the ``colorPalette="warning"`` (orange) for the on state. Use the
  brand palette in both states; outline = off, solid = on. The UI doesn't
  use the warning palette much elsewhere, so this is more consistent.
- Move the toggle's tooltip strings (``search.advanced.{title,description}``)
  from ``dags.json`` to ``common.json`` since the toggle renders on
  admin-namespace pages too. Drop the ``toggleAriaLabel`` translation key —
  aria-labels stay as plain English.
- Update ``64963.significant.rst`` to point users at this opt-in toggle as
  the way back to the previous substring search behavior.

* Style FilterPill toggle as flush rounded addon

Make the toggle inside the FilterPill match the look of the "Pool:" label
on the left:

- Render as a Box-as-button with ``borderRightRadius="full"`` and
  ``alignSelf="stretch"`` so it fills the full pill height instead of
  being a tiny crushed icon centered in white space.
- Off state matches the label palette (gray.muted bg). On state uses the
  brand palette (solid bg, contrast fg).
- New ``variant="addon" | "standalone"`` on ``AdvancedSearchToggle``.
  ``standalone`` keeps the previous IconButton look used next to the
  ``SearchBar``; ``addon`` is the rounded right-side fill used inside
  ``InputWithAddon``.
- Drop the ``pr={1}`` wrapper around ``InputWithAddon``'s ``endAddon`` so
  the addon abuts the pill's rounded edge.

* Rename newsfragment to 66015 and split into two parts

Move the user-visible note from `64963.significant.rst` to a new
`66015.significant.rst` (this PR's number) and split it into two
paragraphs:

- The first paragraph references #64963 — the change from full-match
  `*_pattern` to index-friendly `*_prefix_pattern`.
- The second paragraph references #66015 — the new per-searchbar
  "Match anywhere" toggle that lets users opt back into substring
  search.

* Trim newsfragment, drop localStorage/URL detail
seruman pushed a commit to seruman/airflow that referenced this pull request Apr 30, 2026
…pache#66015)

* UI: Add per-searchbar 'Match anywhere' toggle for substring search

Follow-up to apache#64963. The default search now uses the new prefix patterns
(``*_prefix_pattern``) for index-friendly lookups. Users who relied on
the prior substring behavior (``*_pattern``) can opt back in per
searchbar via a small regex-icon toggle.

Both surfaces support it:

* ``SearchBar`` — page-level inputs on Pools, Connections, Variables,
  Assets, Asset events, and Dags lists.
* ``FilterBar`` text pills — when ``supportsAdvancedSearch`` is set on
  the FilterConfig, the pill renders the toggle inline while editing
  and shows a regex indicator next to the value when collapsed. Wired
  on TaskInstances, DagRuns, Events, XCom, and HITL.

URL is intentionally untouched: shared links default to fast prefix
mode; toggle state is per-searchbar in ``localStorage``
(``advanced_search-<id>``). The toggle uses ``onMouseDown`` +
``preventDefault`` so clicking it inside a FilterPill does not collapse
the pill.

* Move toggle inside FilterPill input, drop warning palette

Address review feedback:

- Move the advanced-search toggle from a sibling next to the InputWithAddon
  to a real ``endAddon`` slot inside the same rounded box. The toggle now
  sits flush on the right of the pill input, mirroring the "Pool:" label on
  the left.
- Drop the ``colorPalette="warning"`` (orange) for the on state. Use the
  brand palette in both states; outline = off, solid = on. The UI doesn't
  use the warning palette much elsewhere, so this is more consistent.
- Move the toggle's tooltip strings (``search.advanced.{title,description}``)
  from ``dags.json`` to ``common.json`` since the toggle renders on
  admin-namespace pages too. Drop the ``toggleAriaLabel`` translation key —
  aria-labels stay as plain English.
- Update ``64963.significant.rst`` to point users at this opt-in toggle as
  the way back to the previous substring search behavior.

* Style FilterPill toggle as flush rounded addon

Make the toggle inside the FilterPill match the look of the "Pool:" label
on the left:

- Render as a Box-as-button with ``borderRightRadius="full"`` and
  ``alignSelf="stretch"`` so it fills the full pill height instead of
  being a tiny crushed icon centered in white space.
- Off state matches the label palette (gray.muted bg). On state uses the
  brand palette (solid bg, contrast fg).
- New ``variant="addon" | "standalone"`` on ``AdvancedSearchToggle``.
  ``standalone`` keeps the previous IconButton look used next to the
  ``SearchBar``; ``addon`` is the rounded right-side fill used inside
  ``InputWithAddon``.
- Drop the ``pr={1}`` wrapper around ``InputWithAddon``'s ``endAddon`` so
  the addon abuts the pill's rounded edge.

* Rename newsfragment to 66015 and split into two parts

Move the user-visible note from `64963.significant.rst` to a new
`66015.significant.rst` (this PR's number) and split it into two
paragraphs:

- The first paragraph references apache#64963 — the change from full-match
  `*_pattern` to index-friendly `*_prefix_pattern`.
- The second paragraph references apache#66015 — the new per-searchbar
  "Match anywhere" toggle that lets users opt back into substring
  search.

* Trim newsfragment, drop localStorage/URL detail
pierrejeambrun added a commit that referenced this pull request Apr 30, 2026
…ng search (#66015) (#66106)

* UI: Add per-searchbar 'Match anywhere' toggle for substring search

Follow-up to #64963. The default search now uses the new prefix patterns
(``*_prefix_pattern``) for index-friendly lookups. Users who relied on
the prior substring behavior (``*_pattern``) can opt back in per
searchbar via a small regex-icon toggle.

Both surfaces support it:

* ``SearchBar`` — page-level inputs on Pools, Connections, Variables,
  Assets, Asset events, and Dags lists.
* ``FilterBar`` text pills — when ``supportsAdvancedSearch`` is set on
  the FilterConfig, the pill renders the toggle inline while editing
  and shows a regex indicator next to the value when collapsed. Wired
  on TaskInstances, DagRuns, Events, XCom, and HITL.

URL is intentionally untouched: shared links default to fast prefix
mode; toggle state is per-searchbar in ``localStorage``
(``advanced_search-<id>``). The toggle uses ``onMouseDown`` +
``preventDefault`` so clicking it inside a FilterPill does not collapse
the pill.

* Move toggle inside FilterPill input, drop warning palette

Address review feedback:

- Move the advanced-search toggle from a sibling next to the InputWithAddon
  to a real ``endAddon`` slot inside the same rounded box. The toggle now
  sits flush on the right of the pill input, mirroring the "Pool:" label on
  the left.
- Drop the ``colorPalette="warning"`` (orange) for the on state. Use the
  brand palette in both states; outline = off, solid = on. The UI doesn't
  use the warning palette much elsewhere, so this is more consistent.
- Move the toggle's tooltip strings (``search.advanced.{title,description}``)
  from ``dags.json`` to ``common.json`` since the toggle renders on
  admin-namespace pages too. Drop the ``toggleAriaLabel`` translation key —
  aria-labels stay as plain English.
- Update ``64963.significant.rst`` to point users at this opt-in toggle as
  the way back to the previous substring search behavior.

* Style FilterPill toggle as flush rounded addon

Make the toggle inside the FilterPill match the look of the "Pool:" label
on the left:

- Render as a Box-as-button with ``borderRightRadius="full"`` and
  ``alignSelf="stretch"`` so it fills the full pill height instead of
  being a tiny crushed icon centered in white space.
- Off state matches the label palette (gray.muted bg). On state uses the
  brand palette (solid bg, contrast fg).
- New ``variant="addon" | "standalone"`` on ``AdvancedSearchToggle``.
  ``standalone`` keeps the previous IconButton look used next to the
  ``SearchBar``; ``addon`` is the rounded right-side fill used inside
  ``InputWithAddon``.
- Drop the ``pr={1}`` wrapper around ``InputWithAddon``'s ``endAddon`` so
  the addon abuts the pill's rounded edge.

* Rename newsfragment to 66015 and split into two parts

Move the user-visible note from `64963.significant.rst` to a new
`66015.significant.rst` (this PR's number) and split it into two
paragraphs:

- The first paragraph references #64963 — the change from full-match
  `*_pattern` to index-friendly `*_prefix_pattern`.
- The second paragraph references #66015 — the new per-searchbar
  "Match anywhere" toggle that lets users opt back into substring
  search.

* Trim newsfragment, drop localStorage/URL detail
(cherry picked from commit d748835)

Co-authored-by: Pierre Jeambrun <pierrejbrun@gmail.com>
@bbovenzi
Copy link
Copy Markdown
Contributor

@dheerajturaga We're trying to balance UX and performance. That new PR helps, but let us know how else we can improve it.

vatsrahul1001 pushed a commit that referenced this pull request May 20, 2026
* API: Use prefix matching for REST list search parameters

Search query parameters on list endpoints now use prefix matching (LIKE 'value%') instead of substring search, which allows the database to use indexes more effectively.

This is a breaking change for clients that relied on matching arbitrary substrings in names and identifiers. Regenerate OpenAPI specs and UI client types accordingly.

* API: Add index-friendly *_prefix_pattern query params (non-breaking)

Keep the existing *_pattern substring filters unchanged and add
parallel *_prefix_pattern variants that use a B-tree-friendly
range comparison, so large deployments can filter list endpoints
without paying for a full table scan.

Drop custom TaskDisplayName substring class; replace feature newsfragment with significant

Full-match on ``coalesce(...)`` can't use an index anyway, so the
custom ``_TaskDisplayNamePatternParam`` class adds nothing over a
plain ``_SearchParam``: use ``search_param_factory`` directly for
the substring variant and keep the custom prefix variant since the
split on ``IS NULL`` is what makes the B-tree index apply.

Swap 64963.feature for 64963.significant to flag the user-visible
UI change (search-as-you-type is now prefix-based, not substring).

Fix CI

Fix CI

* UI: Fix connections search e2e fixtures for prefix-match API

The fixtures had names like search_production_<ts> but the UI search now
hits the prefix-match API (connection_id_prefix_pattern). Move the
environment word to the front (production_search_<ts>, staging_search_<ts>,
development_search_<ts>) so "production" is a valid prefix.

(cherry picked from commit 7336ed4)
vatsrahul1001 pushed a commit that referenced this pull request May 20, 2026
…ng search (#66015) (#66106)

* UI: Add per-searchbar 'Match anywhere' toggle for substring search

Follow-up to #64963. The default search now uses the new prefix patterns
(``*_prefix_pattern``) for index-friendly lookups. Users who relied on
the prior substring behavior (``*_pattern``) can opt back in per
searchbar via a small regex-icon toggle.

Both surfaces support it:

* ``SearchBar`` — page-level inputs on Pools, Connections, Variables,
  Assets, Asset events, and Dags lists.
* ``FilterBar`` text pills — when ``supportsAdvancedSearch`` is set on
  the FilterConfig, the pill renders the toggle inline while editing
  and shows a regex indicator next to the value when collapsed. Wired
  on TaskInstances, DagRuns, Events, XCom, and HITL.

URL is intentionally untouched: shared links default to fast prefix
mode; toggle state is per-searchbar in ``localStorage``
(``advanced_search-<id>``). The toggle uses ``onMouseDown`` +
``preventDefault`` so clicking it inside a FilterPill does not collapse
the pill.

* Move toggle inside FilterPill input, drop warning palette

Address review feedback:

- Move the advanced-search toggle from a sibling next to the InputWithAddon
  to a real ``endAddon`` slot inside the same rounded box. The toggle now
  sits flush on the right of the pill input, mirroring the "Pool:" label on
  the left.
- Drop the ``colorPalette="warning"`` (orange) for the on state. Use the
  brand palette in both states; outline = off, solid = on. The UI doesn't
  use the warning palette much elsewhere, so this is more consistent.
- Move the toggle's tooltip strings (``search.advanced.{title,description}``)
  from ``dags.json`` to ``common.json`` since the toggle renders on
  admin-namespace pages too. Drop the ``toggleAriaLabel`` translation key —
  aria-labels stay as plain English.
- Update ``64963.significant.rst`` to point users at this opt-in toggle as
  the way back to the previous substring search behavior.

* Style FilterPill toggle as flush rounded addon

Make the toggle inside the FilterPill match the look of the "Pool:" label
on the left:

- Render as a Box-as-button with ``borderRightRadius="full"`` and
  ``alignSelf="stretch"`` so it fills the full pill height instead of
  being a tiny crushed icon centered in white space.
- Off state matches the label palette (gray.muted bg). On state uses the
  brand palette (solid bg, contrast fg).
- New ``variant="addon" | "standalone"`` on ``AdvancedSearchToggle``.
  ``standalone`` keeps the previous IconButton look used next to the
  ``SearchBar``; ``addon`` is the rounded right-side fill used inside
  ``InputWithAddon``.
- Drop the ``pr={1}`` wrapper around ``InputWithAddon``'s ``endAddon`` so
  the addon abuts the pill's rounded edge.

* Rename newsfragment to 66015 and split into two parts

Move the user-visible note from `64963.significant.rst` to a new
`66015.significant.rst` (this PR's number) and split it into two
paragraphs:

- The first paragraph references #64963 — the change from full-match
  `*_pattern` to index-friendly `*_prefix_pattern`.
- The second paragraph references #66015 — the new per-searchbar
  "Match anywhere" toggle that lets users opt back into substring
  search.

* Trim newsfragment, drop localStorage/URL detail
(cherry picked from commit d748835)

Co-authored-by: Pierre Jeambrun <pierrejbrun@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch

4 participants