Research

Our Experiments

Self-learned vehicle control using PPO
This work tackles the completion of an obstacle maze by a self-driving vehicle in a realistic physics environment. We tested the adaptability of our algorithm by learning both the controls of a car and a drone. The agents are trained using the PPO (policy gradient) algorithm and using curriculum learning for faster training.
This project was part of a contest and we achieved the faster maze completion times against heuristic approaches.
NN Surgery in Deep RL
In this work we experiment on weight transplantation after slight ANN modifications. Networks can be modified by increasing or reducing the number of input/outputs as well as the number of hidden layers and units. We show that when modifying the network structure, weight transplant achieves faster and better results than training from scratch.

Stay tunned for new experiments such as a RL-trained RocketLeague bot and a Terrarium where animals were trained to survive while following a simple food chain.


Paper reviews

Uncertainty Estimation

Weight Uncertainty in Neural Networks
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, Daan Wierstra, 2015
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Balaji Lakshminarayanan, Alexander Pritzel, Charles Blundell, 2017
Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifiers
A. Vyas, N. Jammalamadaka, X. Zhu, D. Das, B. Kaul, T. L. Willke, 2018
A Simple Baseline for Bayesian Uncertainty in Deep Learning
Wesley J. Maddox, Timur Garipov, Pavel Izmailov, Dmitry Vetrov, Andrew Gordon Wilson, 2019
Ensemble Distribution Distillation
Andrey Malinin, Bruno Mlodozeniec, Mark Gales, 2019


ANN Explainability

Understanding Black-box Predictions via Influence Functions
Pang Wei Koh, Percy Liang, 2017
Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batra, 2019


Generative Models

Glow: Generative Flow with Invertible 1x1 Convolutions
Diederik P. Kingma, Prafulla Dhariwal, 2018
LMConv: Locally Masked Convolutions for Autoregressive Models
Ajay Jain, Pieter Abbeel, Deepak Pathak, 2020


Beyond standard labels

Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
Zhilu Zhang, Mert R. Sabuncu


Reinforcement Learning

Soft Actor-Critic (SAC)
T. Haarnoja, A. Zhou, P. Abbeel, S. Levine, 2018
Hierarchical Reinforcement Learning with MAXQ Value Function Decomposition
Thomas G. Dietterich, 2000
The Option-Critic Architecture
Pierre-Luc Bacon, Jean Harb, Doina Precup, 2016
Neural Network Surgery with Sets
Jonathan Raiman, Susan Zhang, Christy Dennison, 2019