Interactive AI Material Generation and Editing in NVIDIA Omniverse

We present an AI-based tool for interactive material generation within the NVIDIA Omniverse environment. Our approach leverages a State-of-the-art Latent Diffusion model with some notable modifications to adapt it to the task of material generation. Specifically, we employ circular-padded convolution layers in place of standard convolution layers. This unique adaptation ensures the production of seamless tiling textures, as the circular padding facilitates seamless blending at image edges.

FactorSim: Generative Simulation via Factorized Representation

Generating simulations to train intelligent agents in game-playing and robotics from natural language input, from user input or task documentation, remains an open-ended challenge. Existing approaches focus on parts of this challenge, such as generating reward functions or task hyperparameters. Unlike previous work, we introduce FACTORSIM that generates full simulations in code from language input that can be used to train agents.

Hanrong Ye

Hanrong Ye is currently a research scientist at Nvidia Research, conducting research on multi-task, multi-media, and multi-modality models for machine understanding and generation. Personal Website: Link

Xuan Li

Xuan Li is a Research Scientist at NVIDIA Research. He received his PhD from UCLA, advised by Prof. Chenfanfu Jiang. His research focuses on 3D generation and reconstruction, with a particular emphasis on utilizing physics-based simulation.

For more details about his research, please visit https://xuan-li.github.io.

Proteina: Scaling Flow-based Protein Structure Generative Models

Recently, diffusion- and flow-based generative models of protein structures have emerged as a powerful tool for de novo protein design. Here, we develop Proteina, a new large-scale flow-based protein backbone generator that utilizes hierarchical fold class labels for conditioning and relies on a tailored scalable transformer architecture with up to 5x as many parameters as previous models. To meaningfully quantify performance, we introduce a new set of metrics that directly measure the distributional similarity of generated proteins with reference sets, complementing existing metrics.