1. [Publications](/publications)
2. Diffusion-Reward Adversarial Imitation Learning
 
 # Diffusion-Reward Adversarial Imitation Learning

  ![](/sites/default/files/styles/wide/public/publications/drail_model_fig.png?itok=UsIBPbAo)

 Imitation learning aims to learn a policy from observing expert demonstrations without access to reward signals from environments. Generative adversarial imitation learning (GAIL) formulates imitation learning as adversarial learning, employing a generator policy learning to imitate expert behaviors and discriminator learning to distinguish the expert demonstrations from agent trajectories. Despite its encouraging results, GAIL training is often brittle and unstable. Inspired by the recent dominance of diffusion models in generative modeling, this work proposes Diffusion-Reward Adversarial Imitation Learning (DRAIL), which integrates a diffusion model into GAIL, aiming to yield more precise and smoother rewards for policy learning. Specifically, we propose a diffusion discriminative classifier to construct an enhanced discriminator; then, we design diffusion rewards based on the classifier's output for policy learning. We conduct extensive experiments in navigation, manipulation, and locomotion, verifying DRAIL's effectiveness compared to prior imitation learning methods. Moreover, additional experimental results demonstrate the generalizability and data efficiency of DRAIL. Visualized learned reward functions of GAIL and DRAIL suggest that DRAIL can produce more precise and smoother rewards. Code available at [this https URL](https://github.com/NVlabs/DRAIL).



 ## Authors



Chun-Mao Lai (National Taiwan University)

Hsiang-Chun Wang (National Taiwan University)

Ping-Chun Hsieh (National Yang Ming Chiao Tung University)

[Frank Wang](/person/frank-wang)

[Min-Hung Chen](/person/min-hung-chen)

Shao-Hua Sun (National Taiwan University)

 

 

 ## Publication Date



Monday, December 9, 2024

 

 ## Published in



[Neural Information Processing Systems (NeurIPS)](https://neurips.cc/Conferences/2024)

 

 ## Research Area



[Artificial Intelligence and Machine Learning ](/research-area/machine-learning-artificial-intelligence)

[Generative AI](/research-area/generative-ai)

[Robotics](/research-area/robotics)

 

 

 ## External Links



[Project Website](https://nturobotlearninglab.github.io/DRAIL/)

[arXiv](https://arxiv.org/abs/2405.16194)

[Code](https://github.com/NVlabs/DRAIL)