Caption Search

Search mode: keyword and phrase matching over the text captions attached to each clip.

Caption search matches the query against the captions associated with each clip. All captions for a clip are searched together, so a match on any caption surfaces the clip.

Reach for it when you can predict the words a caption will use, such as a specific object, sign, manoeuvre, or road condition, and you want exact, fast matching across the whole corpus with results you can verify by eye.

Reach for something else when the phrasing varies from clip to clip (Caption Embedding Search), the concept may never have been captioned at all (Semantic Search), or the thing you want is visible in a single frame rather than described in words (Visual Search).

Caption search results for crosswalk. The active query appears as a removable chip above the results.

Captions are indexed upfront using Full-Text Search (FTS). At search time the index is looked up directly, making this fast even across hundreds of millions of captions.

When no other ranked mode is active, results are returned in BM25 order: rare terms that appear frequently in a clip’s captions score more highly than common terms. When combined with a ranked mode such as semantic or classifier search, caption search acts as a filter and does not contribute a ranking score of its own.

Query Behavior

All queries are normalised before being sent to the index:

• Quoted phrases: wrapping terms in double quotes, e.g. "heavy rain", matches clips whose captions contain that exact phrase in that order.

• Unquoted multi-word input: consecutive unquoted terms are automatically grouped into a quoted phrase. A query like heavy rain is treated as "heavy rain", not as two independent terms. Hyphens are replaced with spaces before this step, so stop-sign is treated as "stop sign".

• Single terms: a bare word like crosswalk matches clips whose captions contain that exact token.

Matched query terms are highlighted directly in the captions of returned clips, making it easy to verify why each result was retrieved.

Boolean operators AND, OR, and NOT are supported. For example, crosswalk AND rain returns clips whose captions mention both, while rain NOT highway excludes clips mentioning highways.

Retrieved results of caption search with query rain NOT highway, i.e. find clips whose captions contain the word rain but not the word highway.

Query Rewriting

Query rewriting expands crosswalk into five related phrases, each shown as a checkable tag beneath the search field.

Clicking Rewrite sends the query to a language model that generates semantically related short phrases, the kind of phrasing that would appear in a video description rather than a dictionary definition. Each variant appears as a checkable tag in the UI; unchecking a tag removes it from the search. Results from all active variants are merged so each clip appears at most once.