Optimization or Architecture: How to Hack Kalman Filtering
NeurIPS 2023
Since the KF assumptions are often violated, noise estimation is not a proxy to MSE optimization. Instead, our method (OKF) optimizes the MSE directly. In particular, neural network models should be tested against OKF rather than the non-optimized KF – in contrast to the common practice in the literature.
Individualized Dosing Dynamics via Neural Eigen Decomposition
NeurIPS 2023
Abstract Dosing models often use differential equations to model biological dynamics. Neural differential equations in particular can learn to predict the derivative of a process, which permits predictions at irregular points of time.
SoftTreeMax: Exponential Variance Reduction in Policy Gradient via Tree Search
Submitted for review
Despite the popularity of policy gradient methods, they are known to suffer from large variance and high sample complexity. To mitigate this, we introduce SoftTreeMax – a generalization of softmax that takes planning into account.
Expressive Sign Equivariant Networks for Spectral Geometric Learning
NeurIPS 2023
Recent work has shown the utility of developing machine learning models that respect the symmetries of eigenvectors. These works promote sign invariance, since for any eigenvector the negation is also an eigenvector. In this work, we demonstrate that sign equivariance is useful for applications such as building orthogonally equivariant models and link prediction. To obtain these benefits, we develop novel sign equivariant neural network architectures. These models are based on our analytic characterization of the sign equivariant polynomials and thus inherit provable expressiveness properties.
Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Models
SIGGRAPH Asia 2023
Abstract Text-to-image (T2I) personalization allows users to guide the creative image generation process by combining their own visual concepts in natural language prompts. Recently, encoder-based techniques have emerged as a new effective approach for T2I personalization, reducing the need for multiple images and long training times.
Learning to Initiate and Reason in Event-Driven Cascading Processes
ICML 2023
We describe “Cascade”, a new counterfactual reasoning setup. An agent is provided a semantic instruction and the results of a played out dynamical system. Its goal is to intervene in the dynamic environment, triggering a cascade of events that will lead to a different and counterfactual outcome.
Point-Cloud Completion with Pretrained Text-to-image Diffusion Models
NeurIPS 2023
Abstract Point-cloud data collected in real-world applications are often incomplete, because objects are being observed from specific viewpoints, which only capture one perspective. Data can also be incomplete due to occlusion and low-resolution sampling.
Key-Locked Rank One Editing for Text-to-Image Personalization
SIGGRAPH 2023
Summary: We present Perfusion, a new text-to-image personalization method. With only a 100KB model size, trained for roughly 4 minutes, Perfusion can creatively portray personalized objects. It allows significant changes in their appearance, while maintaining their identity, using a novel mechanism we call “Key-Locking”.
Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models
SIGGRAPH 2023
Summary: We use an encoder to personalize a text-to-image model to new concepts with a single image and 5-15 tuning steps.
Abstract: Text-to-image personalization aims to teach a pre-trained diffusion model to reason about novel, user provided concepts, embedding them into new scenes guided by natural language prompts.
Optimizing Tensor Network Contraction Using Reinforcement Learning
ICML 2022
Quantum Computing (QC) stands to revolutionize computing, but is currently still limited. To develop and test quantum algorithms today, quantum circuits are often simulated on classical computers. Simulating a complex quantum circuit requires computing the contraction of a large network of tensors.
Multi-Task Learning as a Bargaining Game
ICML 2022
In Multi-task learning (MTL), a joint model is trained to simultaneously make predictions for several tasks. Joint training reduces computation costs and improves data efficiency; however, since the gradients of these different tasks may conflict, training a joint model for MTL often yields lower performance than its corresponding single-task counterparts.
Federated Learning with Heterogeneous Architectures using Graph HyperNetworks
Arxiv, 2022
Standard Federated Learning (FL) techniques are limited to clients with identical network architectures. This restricts potential use-cases like cross-platform training or inter-organizational collaboration when both data privacy and architectural proprietary are required.
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
ICLR 2023
Abstract Text-to-image models offer unprecedented freedom to guide creation through natural language. Yet, it is unclear how such freedom can be exercised to generate images of specific unique concepts, modify their appearance, or compose them in new roles and novel scenes.
"This is my unicorn, Fluffy": Personalizing frozen vision-language representations
ECCV 2022
Abstract Large Vision & Language models pretrained on web-scale data provide representations that are invaluable for numerous V&L problems. However, it is unclear how they can be used for reasoning about user-specific visual concepts in unstructured language.
Perception and Reasoning
Understanding of a complex scene goes way beyond top-down perception. When people operate in a natural scene, they can detect and recognize objects and relations using context, they can predict how objects and people will move next, and even reason why they behave as they do.
Known unknowns: Learning novel concepts using exploratory reasoning-by-elimination
UAI 2021
Video Abstract Cite the paper If you use the contents of this project, please cite our paper. @article{hagrawal2021unknown, title={Known unknowns: Learning novel concepts using exploratory reasoning-by-elimination}, author={Harsh Agrawal, Eli Meirom, Yuval Atzmon, Shie Mannor, Gal Chechik}, journal={Uncertainty in artificial intelligence}, year={2021} }
GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning
ICML 2021
Video Abstract Gaussian processes (GPs) are non-parametric, flexible, models that work well in many tasks. Combining GPs with deep learning methods via deep kernel learning is especially compelling due to the strong expressive power induced by the network.
Compositional Video Synthesis with Action Graphs
ICML 2021
Video Abstract Videos of actions are complex signals, containing rich compositional structure. Current video generation models are limited in their ability to generate such videos. To address this challenge, we introduce a generative model (AG2Vid) that can be conditioned on an Action Graph, a structure that naturally represents the dynamics of actions and interactions between objects.
Personalized Federated Learning using Hypernetworks
ICML 2021
Video Abstract Personalized federated learning is tasked with training machine learning models for multiple clients, each with its own data distribution. The goal is to train personalized models in a collaborative way while accounting for data disparities across clients and reducing communication costs.
Self-Supervised Learning for Domain Adaptation on Point-Clouds
WACV 2021
Video Abstract Self-supervised learning (SSL) is a technique for learning useful representations from unlabeled data. It has been applied effectively to domain adaptation (DA) on images and videos. It is still unknown if and how it can be leveraged for domain adaptation in 3D perception problems.
Learning the Pareto Front with Hypernetworks
ICLR 2021
Video Abstract Multi-objective optimization problems are prevalent in machine learning. These problems have a set of optimal solutions, called the Pareto front, where each point on the front represents a different trade-off between possibly conflicting objectives.
Auxiliary Learning by Implicit Differentiation
ICLR 2021
Video Abstract Training with multiple auxiliary tasks is a common practice used in deep learning for improving the performance on the main task of interest. Two main challenges arise in this multi-task learning setting: (i) Designing useful auxiliary tasks; and (ii) Combining auxiliary tasks into a single coherent loss.
A causal view of compositional zero-shot recognition
NeurIPS 2020
Video Abstract People easily recognize new visual categories that are new combinations of known components. This compositional generalization capacity is critical for learning in real-world domains like vision and language because the long tail of new combinations dominates the distribution.
