.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. .. SPDX-License-Identifier: Apache-2.0 .. .. Licensed under the Apache License, Version 2.0 (the "License"); .. you may not use this file except in compliance with the License. .. You may obtain a copy of the License at .. .. http://www.apache.org/licenses/LICENSE-2.0 .. .. Unless required by applicable law or agreed to in writing, software .. distributed under the License is distributed on an "AS IS" BASIS, .. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. .. See the License for the specific language governing permissions and .. limitations under the License. Search ====== **Retrieval APIs: the entry point to everything else in Wheel. Each mode defines a slice of the corpus; several can be composed in one query.** Wheel provides multiple composable search modes that can be combined within a single query. These modes let users define slices using language, visual content, motion, classifier outputs, metadata, and structured scene constraints. **Start here** whatever you are ultimately trying to do. Annotation, curation, and evaluation all take the current search result as their input, so defining the right slice is the first step of every workflow. **If you do not yet know what to search for**, begin with :doc:`cluster_search`, the one mode that needs no query: it surfaces the natural structure of a dataset and shows you what is there to ask about. Combining Modes --------------- Search modes fall into two categories: * **Ranked modes** assign each candidate clip a similarity, relevance, or probability score. Results are sorted in descending order by the active ranked mode that controls ordering. * **Filter-only modes** restrict the eligible pool without producing a score of their own. Only clips that satisfy every active filter remain in the result set. Filter-only modes apply first to define the candidate pool. Ranked modes then score and order the remaining clips. If no ranked mode is active, results are returned in arbitrary order. When multiple ranked modes are active, Wheel uses a fixed priority order to determine which score controls final sorting: Numeric metric -> Classifier -> Caption embedding -> Semantic text -> Semantic clip -> Visual (text) -> Visual (image) -> Trajectory shape -> Cluster distance For example, combining semantic text search with a perception-based filter returns only clips that satisfy the specified spatial constraint, ranked by semantic similarity. Replacing the semantic search with a classifier ranks the same filtered pool by classifier score instead. Search Modes at a Glance ------------------------ .. list-table:: :header-rows: 1 :widths: 30 55 15 * - Search type - What it retrieves - Behavior * - Caption keyword - Clips whose captions contain matching words or expressions - Filter * - Caption embedding - Clips whose captions are semantically similar to a textual query (Qwen3 embeddings) - Ranked * - Semantic text-to-video - Clips visually and semantically similar to a text description (CosmosEmbed1) - Ranked * - Semantic video-to-video - Clips visually and semantically similar to a reference clip (CosmosEmbed1) - Ranked * - Visual - Clips with frames that match a text description or uploaded image (CLIP); ranked by best-matching frame - Ranked * - Trajectory shape - Clips with an ego-trajectory shape geometrically similar to a reference clip - Ranked * - Trajectory pattern - Clips matching a named ego-trajectory pattern or custom motion expression - Filter * - Perception-based - Clips where specific objects appear at a given proximity or direction - Filter * - Classifier - Clips scored above a threshold by a trained scenario-specific classifier - Ranked * - Cluster - Clips belonging to a precomputed cluster, ranked by centroid proximity - Ranked * - Metadata - Clips matching annotation labels, data source, geography, or numeric metrics - Filter / Ranked * \* Metadata is filter-only except when a numeric metric sort is configured, in which case results are ranked by that metric value. Not every mode is available on every deployment. Each one depends on artifacts built during preprocessing, and a mode whose artifacts are missing is hidden rather than returning nothing. :ref:`what-each-search-mode-needs` lists what each requires. Examples -------- **Trajectory shape + semantic video-to-video** .. figure:: /_static/images/trajectory_semantic_search.png :alt: Two active search chips, trajectory shape and semantic video, with each result card carrying both scores and its ego path plot :width: 90% :align: center Trajectory shape search combined with semantic video-to-video retrieval. Find a reference clip with an interesting manoeuvre, then run trajectory shape search to retrieve clips with a geometrically similar path. Add a semantic video-to-video search using the same reference clip to further rank the results by visual similarity. The result is a pool of clips that both drive a similar path and look like the reference scene. **Trajectory pattern + metadata** .. figure:: /_static/images/trajectory_pattern_metadata.png :alt: The Country field set to GR and the Trajectory Pattern dropdown set to Hard Braking, with the matching clips listed below :width: 90% :align: center Trajectory pattern filter (``Hard Braking``) combined with metadata constraints (country Greece). Use a trajectory pattern filter such as ``hard_braking`` or a custom expression to restrict results to clips with a specific motion characteristic. Combine with metadata filters to scope the search to a particular dataset, country, or annotation label. Because both are filter-only modes, the result is an unranked set of clips satisfying all constraints simultaneously, which can then be annotated or exported directly. .. toctree:: :maxdepth: 1 caption_search caption_embed_search semantic_search visual_search trajectory_search perception_search classifier_search cluster_search metadata_search