Tutorials

Short video walkthroughs of key SIL-Wheel workflows.

Search, Annotation, and Evaluation Workflow

This tutorial demonstrates a complete end-to-end Wheel workflow. Starting from a dataset selection, it composes a semantic search for “a pedestrian crossing a crosswalk” with a trajectory filter that discards clips where the ego vehicle is static, retaining only scenarios where the vehicle is in motion. The resulting slice is auto labeled and the semantic search is broadened by applying the query rewriter. The tutorial then opens the Policy Leaderboard on this targeted slice to inspect model scores and review per clip results. Finally, it progressively relaxes the filters, first removing the ego motion constraint and then the semantic query, showing how each step changes the leaderboard scores and illustrating the tight connection between data curation decisions and model evaluation outcomes in Wheel.

Composing semantic search, trajectory filtering, and auto-labeling to build a targeted data slice, evaluating a policy model on it, and observing how progressively relaxing the filters shifts leaderboard scores.

Classifier Bootstrapping Workflow

This tutorial demonstrates the classifier bootstrapping workflow for curating a dataset of children crossing the street. The process begins with a semantic text to video search for “children crossing the street”. While many of the retrieved results are correct, the query also surfaces a large number of clips showing adult pedestrians crossing, since the two scenarios share similar visual structure. To refine the results, valid clips are manually annotated and video to video search is used to find visually similar examples.

Once a small set of positive clips has been collected, a classifier is trained on top of the clip embeddings. Applying it to the dataset and keeping only results with a probability threshold above 0.95 yields a high confidence set, which is reviewed and further annotated. However, the classifier still retrieves too many pedestrian crossing clips. To address this, a second classifier is trained using the label VRU crossing pedestrian as explicit negative examples, rather than relying on randomly sampled negatives. This gives the classifier a sharper decision boundary around the target scenario, and the quality of the retrieved results visibly improves. The high confidence results are then auto labeled. This iterative loop of retrieval, annotation, classifier training, and review continues until a sufficiently large and clean dataset has been curated.

Iterative classifier bootstrapping: from semantic search to targeted annotation, classifier training with explicit negatives, and auto labeling to scale a children crossing dataset.

Clustering and Topic Modeling

This tutorial demonstrates the clustering and topic modeling workflow. It walks through launching a clustering run from the Clustering Tools panel, browsing the resulting UMAP projection to explore how clusters relate to each other, and inspecting per-cluster topic keywords and descriptions that are automatically extracted from clip captions.

Launching a clustering run, browsing the UMAP projection, and inspecting per-cluster topic keywords and descriptions.

Agent Setup and Search Demo

This tutorial walks through installing the SIL-Wheel Agent in Cursor, configuring credentials, and running natural-language searches against a live Wheel instance. The agent composes caption, semantic, and trajectory queries from plain English and returns ranked clips with clickable Wheel URLs.

Installing the SIL-Wheel Agent in Cursor and running natural-language searches that compose multiple filters into a single query.