NVIDIA Tel-Aviv Lab

NVIDIA Tel-Aviv Lab

Introduction

Welcome to the homepage of the NVIDIA Research Israel AI Lab. Our research spans algorithms, theory and applications of deep learning, with a focus on computer vision and reinforcement learning. We are particularly interested in perception, action, and reasoning (PAR), and their intersection. The lab is led by Professor Gal Chechik. See recent projects and current team members.

News

September 2023

July 2023

April 2022

  • The workshop “3D Perception for Autonomous Driving” is accepted to ECCV 2022. Link

Feb 2022

  • The paper “Equivariant Subgraph Aggregation Networks” by Beatrice Bevilacqua, Fabrizio Frasca, Derek Lim, Balasubramaniam Srinivasan, Chen Cai, Gopinath Balamurugan, Michael M. Bronstein, Haggai Maron, was accepted as a spotlight presentation to ICLR 2022 , PDF.

Dec 2021

  • Shie Mannor was elected as an IEEE Fellow.

Nov 2021

  • Gal presented our work on perception reasoning and action at GTC 2021, Video.
  • Shie presented our work on accelerated tree search with off-plicycorrection at GTC 2021, Video.
  • Gal presented work on federated learning VIDEO

Oct 2021

  • We coorganize the Israeli Geometric deep learning day 2021, Schedule

May 2021

  • The paper “Known unknowns: Learning novel concepts using reasoning-by-elimination” by Harsh Agrawal, Eli A. Meirom, Yuval Atzmon, Shie Mannor, and Gal Chechik was accepted as an oral to UAI 2021 PDF

Apr 2021

  • We coorganize the Israeli Reinforcement Learning day, Schedule

Dec 2020

  • The paper “A causal view of compositional zero-shot recognition” by Y. Atzmon,F. Kreuk, Uri Shalit, G. Chechik, was presented as a NeurIPS spotlight, PDF.

Oct 2020

  • NVIDIA teaches machines to communicate: YNET (Hebrew)

July 2020

May 2020

  • NVIDIA efforts around COVID-19: YNET(Hebrew).

Members

Avatar

Assaf Hallak

Avatar

Chen Tessler

Avatar

Eli Meirom

Avatar

Gal Chechik

Avatar

Gal Dalal

Avatar

Haggai Maron

Avatar

Rinon Gal

Avatar

Shie Mannor

Avatar

Yoni Kasten

Avatar

Yuval Atzmon

Current interns

Avatar

Ido Greenberg

Avatar

Ofir Nabati

Highlighted Projects

Quickly discover relevant content by filtering publications.

Train Hard, Fight Easy: Robust Meta Reinforcement Learning

NeurIPS 2023

We introduce RoML - a meta-algorithm that takes any meta-learning baseline algorithm and generates a robust version of it. A test task corresponding to high body mass, which is typically more difficult to control.
PDF Code Poster Video
Train Hard, Fight Easy: Robust Meta Reinforcement Learning

SoftTreeMax: Exponential Variance Reduction in Policy Gradient via Tree Search

Submitted for review

Despite the popularity of policy gradient methods, they are known to suffer from large variance and high sample complexity. To mitigate this, we introduce SoftTreeMax – a generalization of softmax that takes planning into account.
PDF Code
SoftTreeMax: Exponential Variance Reduction in Policy Gradient via Tree Search

Expressive Sign Equivariant Networks for Spectral Geometric Learning

NeurIPS 2023

Recent work has shown the utility of developing machine learning models that respect the symmetries of eigenvectors. These works promote sign invariance, since for any eigenvector the negation is also an eigenvector. In this work, we demonstrate that sign equivariance is useful for applications such as building orthogonally equivariant models and link prediction. To obtain these benefits, we develop novel sign equivariant neural network architectures. These models are based on our analytic characterization of the sign equivariant polynomials and thus inherit provable expressiveness properties.
PDF
Expressive Sign Equivariant Networks for Spectral Geometric Learning

Norm-guided latent space exploration for text-to-image generation

NeurIPS 2023

Text-to-image diffusion models show great potential in synthesizing a large variety of concepts in new compositions and scenarios. …
PDF
Norm-guided latent space exploration for text-to-image generation

Equivariant Architectures for Learning in Deep Weight Spaces

ICML 2023 (Oral presentation)

Many machine learning tasks involve processing eigenvectors derived from data. Especially valuable are Laplacian eigenvectors, which …
PDF
Equivariant Architectures for Learning in Deep Weight Spaces
See all publications

Publications

Quickly discover relevant content by filtering publications.
*

Individualized Dosing Dynamics via Neural Eigen Decomposition

NeurIPS 2023

Abstract Dosing models often use differential equations to model biological dynamics. Neural differential equations in particular can learn to predict the derivative of a process, which permits predictions at irregular points of time.
PDF Code
Individualized Dosing Dynamics via Neural Eigen Decomposition

Optimization or Architecture: How to Hack Kalman Filtering

NeurIPS 2023

Since the KF assumptions are often violated, noise estimation is not a proxy to MSE optimization. Instead, our method (OKF) optimizes the MSE directly. In particular, neural network models should be tested against OKF rather than the non-optimized KF – in contrast to the common practice in the literature.
PDF Code Poster Video
Optimization or Architecture: How to Hack Kalman Filtering

Train Hard, Fight Easy: Robust Meta Reinforcement Learning

NeurIPS 2023

We introduce RoML - a meta-algorithm that takes any meta-learning baseline algorithm and generates a robust version of it. A test task corresponding to high body mass, which is typically more difficult to control.
PDF Code Poster Video
Train Hard, Fight Easy: Robust Meta Reinforcement Learning

SoftTreeMax: Exponential Variance Reduction in Policy Gradient via Tree Search

Submitted for review

Despite the popularity of policy gradient methods, they are known to suffer from large variance and high sample complexity. To mitigate this, we introduce SoftTreeMax – a generalization of softmax that takes planning into account.
PDF Code
SoftTreeMax: Exponential Variance Reduction in Policy Gradient via Tree Search

Expressive Sign Equivariant Networks for Spectral Geometric Learning

NeurIPS 2023

Recent work has shown the utility of developing machine learning models that respect the symmetries of eigenvectors. These works promote sign invariance, since for any eigenvector the negation is also an eigenvector. In this work, we demonstrate that sign equivariance is useful for applications such as building orthogonally equivariant models and link prediction. To obtain these benefits, we develop novel sign equivariant neural network architectures. These models are based on our analytic characterization of the sign equivariant polynomials and thus inherit provable expressiveness properties.
PDF
Expressive Sign Equivariant Networks for Spectral Geometric Learning

Norm-guided latent space exploration for text-to-image generation

NeurIPS 2023

Text-to-image diffusion models show great potential in synthesizing a large variety of concepts in new compositions and scenarios. …
PDF
Norm-guided latent space exploration for text-to-image generation

Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Models

SIGGRAPH Asia 2023

Abstract Text-to-image (T2I) personalization allows users to guide the creative image generation process by combining their own visual concepts in natural language prompts. Recently, encoder-based techniques have emerged as a new effective approach for T2I personalization, reducing the need for multiple images and long training times.
PDF Project
Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Models

Equivariant Architectures for Learning in Deep Weight Spaces

ICML 2023 (Oral presentation)

Many machine learning tasks involve processing eigenvectors derived from data. Especially valuable are Laplacian eigenvectors, which …
PDF
Equivariant Architectures for Learning in Deep Weight Spaces

Equivariant Polynomials for Graph Neural Networks

ICML 2023 (Oral presentation)

Graph Neural Networks (GNN) are inherently limited in their expressive power. Recent seminal works (Xu et al., 2019; Morris et al., …
PDF
Equivariant Polynomials for Graph Neural Networks

Graph Positional Encoding via Random Feature Propagation

ICML 2023

Two main families of node feature augmentation schemes have been explored for enhancing GNNs: random features and spectral positional …
PDF
Graph Positional Encoding via Random Feature Propagation

Learning to Initiate and Reason in Event-Driven Cascading Processes

ICML 2023

We describe “Cascade”, a new counterfactual reasoning setup. An agent is provided a semantic instruction and the results of a played out dynamical system. Its goal is to intervene in the dynamic environment, triggering a cascade of events that will lead to a different and counterfactual outcome.
PDF
Learning to Initiate and Reason in Event-Driven Cascading Processes

Point-Cloud Completion with Pretrained Text-to-image Diffusion Models

NeurIPS 2023

Abstract Point-cloud data collected in real-world applications are often incomplete, because objects are being observed from specific viewpoints, which only capture one perspective. Data can also be incomplete due to occlusion and low-resolution sampling.
PDF Project
Point-Cloud Completion with Pretrained Text-to-image Diffusion Models

Key-Locked Rank One Editing for Text-to-Image Personalization

SIGGRAPH 2023

Summary: We present Perfusion, a new text-to-image personalization method. With only a 100KB model size, trained for roughly 4 minutes, Perfusion can creatively portray personalized objects. It allows significant changes in their appearance, while maintaining their identity, using a novel mechanism we call “Key-Locking”.
PDF Project
Key-Locked Rank One Editing for Text-to-Image Personalization

CALM: Conditional Adversarial Latent Models for Directable Virtual Characters

SIGGRAPH 2023

In this work, we present Conditional Adversarial Latent Models (CALM), an approach for generating diverse and directable behaviors for …
PDF Code Project Video
CALM: Conditional Adversarial Latent Models for Directable Virtual Characters

Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models

SIGGRAPH 2023

Summary: We use an encoder to personalize a text-to-image model to new concepts with a single image and 5-15 tuning steps. Abstract: Text-to-image personalization aims to teach a pre-trained diffusion model to reason about novel, user provided concepts, embedding them into new scenes guided by natural language prompts.
PDF Project
Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models

Planning and Learning with Adaptive Lookahead

AAAI 2023

Some of the most powerful reinforcement learning frameworks use planning for action selection. Interestingly, their planning horizon is …
PDF
Planning and Learning with Adaptive Lookahead

Never Worse, Mostly Better: Stable Policy Improvement in Deep Reinforcement Learning

AAMAS 2023

In recent years, there has been significant progress in applying deep reinforcement learning (RL) for solving challenging problems …
PDF
Never Worse, Mostly Better: Stable Policy Improvement in Deep Reinforcement Learning

A Simple and Universal Rotation Equivariant Point-cloud Network

TAG in Machine Learning workshop, ICML 2022

Equivariance to permutations and rigid motions is an important inductive bias for various 3D learning problems. Recently it has been …
PDF
A Simple and Universal Rotation Equivariant Point-cloud Network

Sign and Basis Invariant Networks for Spectral Graph Representation Learning

ICLR 2023 (Spotlight presentation)

Many machine learning tasks involve processing eigenvectors derived from data. Especially valuable are Laplacian eigenvectors, which …
PDF Code
Sign and Basis Invariant Networks for Spectral Graph Representation Learning

Understanding and Extending Subgraph GNNs by Rethinking Their Symmetries

NeurIPS 2022 (Oral presentation)

Subgraph GNNs are a recent class of expressive Graph Neural Networks (GNNs) which model graphs as collections of subgraphs. So far, the …
PDF Code Video
Understanding and Extending Subgraph GNNs by Rethinking Their Symmetries

Equivariant Subgraph Aggregation Networks

ICLR 2022 (Spotlight presentation)

Message-passing neural networks (MPNNs) are the leading architecture for deep learning on graph-structured data, in large part due to …
PDF Code Video
Equivariant Subgraph Aggregation Networks

Federated Learning with Heterogeneous Architectures using Graph HyperNetworks

Arxiv, 2022

Standard Federated Learning (FL) techniques are limited to clients with identical network architectures. This restricts potential use-cases like cross-platform training or inter-organizational collaboration when both data privacy and architectural proprietary are required.
PDF
Federated Learning with Heterogeneous Architectures using Graph HyperNetworks

Multi-Task Learning as a Bargaining Game

ICML 2022

In Multi-task learning (MTL), a joint model is trained to simultaneously make predictions for several tasks. Joint training reduces computation costs and improves data efficiency; however, since the gradients of these different tasks may conflict, training a joint model for MTL often yields lower performance than its corresponding single-task counterparts.
PDF Video
Multi-Task Learning as a Bargaining Game

Optimizing Tensor Network Contraction Using Reinforcement Learning

ICML 2022

Quantum Computing (QC) stands to revolutionize computing, but is currently still limited. To develop and test quantum algorithms today, quantum circuits are often simulated on classical computers. Simulating a complex quantum circuit requires computing the contraction of a large network of tensors.
PDF Code
Optimizing Tensor Network Contraction Using Reinforcement Learning

An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion

ICLR 2023

Abstract Text-to-image models offer unprecedented freedom to guide creation through natural language. Yet, it is unclear how such freedom can be exercised to generate images of specific unique concepts, modify their appearance, or compose them in new roles and novel scenes.
PDF Code Project Video
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion

Reinforcement Learning with a Terminator

Arxiv preprint

We present the problem of reinforcement learning with exogenous termination. We define the Termination Markov Decision Process …
PDF
Reinforcement Learning with a Terminator

"This is my unicorn, Fluffy": Personalizing frozen vision-language representations

ECCV 2022

Abstract Large Vision & Language models pretrained on web-scale data provide representations that are invaluable for numerous V&L problems. However, it is unclear how they can be used for reasoning about user-specific visual concepts in unstructured language.
PDF Code Dataset Video
"This is my unicorn, Fluffy": Personalizing frozen vision-language representations

Reinforcement Learning for Datacenter Congestion Control

IAAI 2022

We approach the task of network congestion control in datacenters using Reinforcement Learning (RL). Successful congestion control …
PDF
Reinforcement Learning for Datacenter Congestion Control

Improve Agents without Retraining: Parallel Tree Search with Off-Policy Correction

NeurIPS 2021

Tree Search (TS) is crucial to some of the most influential successes in reinforcement learning. Here, we tackle two major challenges …
PDF
Improve Agents without Retraining: Parallel Tree Search with Off-Policy Correction

Perception and Reasoning

Understanding of a complex scene goes way beyond top-down perception. When people operate in a natural scene, they can detect and recognize objects and relations using context, they can predict how objects and people will move next, and even reason why they behave as they do.
Perception and Reasoning

StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators

SIGGRAPH 2022

Can a generative model be trained to produce images from a specific domain, guided by a text prompt only, without seeing any image? In other words: can an image generator be trained blindly?
PDF Code Video

From Local Structures to Size Generalization in Graph Neural Networks

ICML 2021

Graph neural networks (GNNs) can process graphs of different sizes, but their ability to generalize across sizes, specifically from …
PDF
From Local Structures to Size Generalization in Graph Neural Networks

How to Stop Epidemics: Controlling Graph Dynamics with Reinforcement Learning and Graph Neural Networks

ICML 2021

We consider the problem of monitoring and controlling a partially-observed dynamic process that spreads over a graph. This problem …
PDF
How to Stop Epidemics: Controlling Graph Dynamics with Reinforcement Learning and Graph Neural Networks

Compositional Video Synthesis with Action Graphs

ICML 2021

Video Abstract Videos of actions are complex signals, containing rich compositional structure. Current video generation models are limited in their ability to generate such videos. To address this challenge, we introduce a generative model (AG2Vid) that can be conditioned on an Action Graph, a structure that naturally represents the dynamics of actions and interactions between objects.
PDF

GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning

ICML 2021

Video Abstract Gaussian processes (GPs) are non-parametric, flexible, models that work well in many tasks. Combining GPs with deep learning methods via deep kernel learning is especially compelling due to the strong expressive power induced by the network.
PDF

Known unknowns: Learning novel concepts using exploratory reasoning-by-elimination

UAI 2021

Video Abstract Cite the paper If you use the contents of this project, please cite our paper. @article{hagrawal2021unknown, title={Known unknowns: Learning novel concepts using exploratory reasoning-by-elimination}, author={Harsh Agrawal, Eli Meirom, Yuval Atzmon, Shie Mannor, Gal Chechik}, journal={Uncertainty in artificial intelligence}, year={2021} }
PDF Poster

Personalized Federated Learning using Hypernetworks

ICML 2021

Video Abstract Personalized federated learning is tasked with training machine learning models for multiple clients, each with its own data distribution. The goal is to train personalized models in a collaborative way while accounting for data disparities across clients and reducing communication costs.
PDF Code
Personalized Federated Learning using Hypernetworks

Acting in Delayed Environments with Non-Stationary Markov Policies

ICLR 2021

The standard Markov Decision Process (MDP) formulation hinges on the assumption that an action is executed immediately after it was …
PDF Code
Acting in Delayed Environments with Non-Stationary Markov Policies

A causal view of compositional zero-shot recognition

NeurIPS 2020

Video Abstract People easily recognize new visual categories that are new combinations of known components. This compositional generalization capacity is critical for learning in real-world domains like vision and language because the long tail of new combinations dominates the distribution.
PDF Code Dataset Video
A causal view of compositional zero-shot recognition

Auxiliary Learning by Implicit Differentiation

ICLR 2021

Video Abstract Training with multiple auxiliary tasks is a common practice used in deep learning for improving the performance on the main task of interest. Two main challenges arise in this multi-task learning setting: (i) Designing useful auxiliary tasks; and (ii) Combining auxiliary tasks into a single coherent loss.
PDF Code

Learning the Pareto Front with Hypernetworks

ICLR 2021

Video Abstract Multi-objective optimization problems are prevalent in machine learning. These problems have a set of optimal solutions, called the Pareto front, where each point on the front represents a different trade-off between possibly conflicting objectives.
PDF Code
Learning the Pareto Front with Hypernetworks

Self-Supervised Learning for Domain Adaptation on Point-Clouds

WACV 2021

Video Abstract Self-supervised learning (SSL) is a technique for learning useful representations from unlabeled data. It has been applied effectively to domain adaptation (DA) on images and videos. It is still unknown if and how it can be leveraged for domain adaptation in 3D perception problems.
PDF Code Video

On the Universality of Rotation Equivariant Point Cloud Networks

ICLR 2021

Learning from unordered sets is a fundamental learning setup, recently attracting increasing attention. Research in this area has …
PDF
On the Universality of Rotation Equivariant Point Cloud Networks

On Learning Sets of Symmetric Elements

ICML 2020

Learning from unordered sets is a fundamental learning setup, recently attracting increasing attention. Research in this area has …
PDF Slides Video
On Learning Sets of Symmetric Elements

Contact

Our lab is located in Tel-Aviv.

  • Yitzhak Sadeh St 6, Tel-Aviv,
  • Monday-Friday 9:00 to 18:00