.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
.. SPDX-License-Identifier: Apache-2.0
..
.. Licensed under the Apache License, Version 2.0 (the "License");
.. you may not use this file except in compliance with the License.
.. You may obtain a copy of the License at
..
.. http://www.apache.org/licenses/LICENSE-2.0
..
.. Unless required by applicable law or agreed to in writing, software
.. distributed under the License is distributed on an "AS IS" BASIS,
.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
.. See the License for the specific language governing permissions and
.. limitations under the License.

Perception-based Search
=======================

**Search mode: geometric filtering on where detected road users sit relative
to the ego vehicle.**

Perception-based search filters clips based on the spatial relationship between
the ego vehicle and detected road users. It answers questions like "clips
where a pedestrian was within 3 metres directly ahead" or "clips with more
than 5 vehicles present simultaneously" without relying on captions or
manual annotations. Detections are produced by a 3D object detector run
over every clip offline, so filters evaluate in milliseconds regardless of
dataset size. This is a filter-only mode: it restricts the eligible clip
pool without producing a ranking score of its own.

**Reach for it when** the question is spatial and quantitative: a pedestrian
within 3 metres directly ahead, more than five vehicles at once, a cyclist
approaching from the front-left. These are conditions captions rarely state
precisely and embeddings cannot measure.

**Reach for something else when** your dataset has no precomputed detections,
or the question is about how a scene looks rather than where objects are, where
:doc:`semantic_search` and :doc:`visual_search` apply.

.. figure:: /_static/images/world_model_selection_menu.png
   :alt: The Perception-based Search panel with object label Bike w/ Rider, a six-sector diagram around the ego car with the three front sectors selected, and distance and duration sliders
   :width: 90%
   :align: center

   The perception-based search panel for configuring object class, angle sectors, count, and range filters.

.. figure:: /_static/images/world_model_search.png
   :alt: An active perception-based search chip summarising the Bike w/ Rider filter, with clips showing riders ahead of the ego vehicle

   Perception-based search: clips containing a bike with rider within 10 m of
   the ego vehicle, in the front or front-side sectors, for at least 2.5 s.


How It Works
------------

A 3D object detector is run over every clip offline. Per-frame detections
are aggregated into per-clip statistics (object counts, distances, and
angular positions) and held in memory, so filters evaluate in milliseconds
regardless of dataset size.

Object Classes
--------------

The **Object Label** dropdown lists the object classes that can be
filtered. Pick exactly one:

.. list-table::
   :header-rows: 1
   :widths: 30 70

   * - Label (UI)
     - Underlying class
   * - Car
     - ``VEHICLE_CAR``
   * - Truck
     - ``VEHICLE_TRUCK``
   * - Bus
     - ``VEHICLE_BUS``
   * - Bike w/ Rider
     - ``BIKE_WITH_RIDER``
   * - Pedestrian
     - ``PEDESTRIAN_UNKNOWN``

Angle Ranges
------------

The space around the ego vehicle is divided into six sectors. One or more
sectors can be selected; the filter matches clips where the object appears
in any of the selected sectors.

.. list-table::
   :header-rows: 1
   :widths: 20 80

   * - Sector
     - Description
   * - ``FRONT``
     - Directly ahead
   * - ``FRONT_RIGHT``
     - Front-right quadrant
   * - ``BACK_RIGHT``
     - Rear-right quadrant
   * - ``BACK``
     - Directly behind
   * - ``BACK_LEFT``
     - Rear-left quadrant
   * - ``FRONT_LEFT``
     - Front-left quadrant

Count Filter
------------

Filters by the total number of detections of a given object class across
all frames in the clip. A minimum and/or maximum count can be set. Useful
for finding clips with high pedestrian density, or clips with no other
vehicles present at all (max_count = 0).

Range Filter
------------

Filters by object presence within a combined spatial and temporal window.
The object must appear in one of the selected angular sectors, within a
specified distance from the ego vehicle, for at least a minimum continuous
duration. All three constraints must be satisfied simultaneously.

For example: "a pedestrian within 5 metres in the FRONT or FRONT_LEFT
sector for at least 2 seconds" translates to selecting
``PEDESTRIAN_UNKNOWN``, sectors ``FRONT, FRONT_LEFT``, max distance 5 m,
min time 2 s.