About

The UT Robot Learning Reading Group meets to discuss the latest papers in robot learning. We follow the latest robotics and embodied AI papers, spanning computer vision, reinforcement learning, neuro-symbolic AI, foundation models, and control. We survey papers from a broad set of conferences, including (but not limited to) CoRL, RSS, and CVPR.

This semester (Spring ‘24), the reading group will focus on the intersection of Robot Learning and Foundation Models. We will discuss one paper every two weeks, preparing 30-minute presentations with 30 minutes of Q&A, and optional discussion time (30 minutes).

The reading group is co-ordinated by,

Want to join us?

Please subscribe to the emailing list. If you have trouble joining it, please reach out to Rutav Shah (rutavms@utexas.edu).

Upcoming Reading Group

You can view this spreadsheet for a tentative schedule of future reading group sessions.

Reading list

Spring 2024

[2024.04.22]
Gaussian Splatting SLAM (Visual Odometry)
Presenter: Amanda Adkins
[Slides]

[2024.04.08]
SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning
Presenter: Mingyo Seo
[Slides]

[2024.3.25]
AutoRT: Embodied Foundation Models for Large-Scale Orchestration of Robotic Agents
Presenter: Arpit Bahety
[Slides]

[2024.3.18]
Vison-Language Foundation Models as Effective Robot Imitators
Presenter: Zichao Hu
[Slides]

[2024.2.26]
OK-Robot: An open, modular framework for zero-shot, language-conditioned pick-and-drop tasks in arbitrary homes
Presenter: Shivin Dass
[Slides]

Fall 2023

[2023.12.11]
Sequential Modeling Enables Scalable Learning for Large Vision Models
Presenter: Yue Zhao
[Slides]

[2023.11.27]
Code Llama: Open Foundation Models for Code
Presenter: Jierui Li
[Slides]

[2023.11.06]
Generative Agents: Interactive Simulacra of Human Behavior
Presenter: Caroline Wang
[Slides]

[2023.10.23]
Code as Policies: Language Model Programs for Embodied Control
Presenter: Zichao Hu
[Slides]

[2023.10.09]
Voyager: An Open-Ended Embodied Agent with Large Language Models
Presenter: Yuqi Xie
[Slides]

[2023.09.25]
GNM: A General Navigation Model to Drive Any Robot
Presenter: Arthur Zhang
[Slides]

[2023.09.11]
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models
Presenter: Albert Yu
[Slides]

Spring 2023

[2023.04.21]
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
Presenter: Mingyo Seo
[Slides]

[2023.03.24]
RT-1: Robotics Transformer for Real-World Control at Scale
Presenter: Rutav Shah
[Slides]

[2023.03.02]
Behavior Transformers: Cloning $k$ modes with one stone
Presenter: Soroush Nasiriany
[Slides]

[2023.02.11]
Synthesizing Physical Character-Scene Interactions
Presenter: Hanwen Jiang
[Slides]

Fall 2022

[2022.12.09]
Runtime Monitoring for Safe Learning and Deployment
Presenter: Huihan Liu
[Slides]

[2022.11.18]
Diffusion Models
Presenter: Zhenyu Jiang
[Slides]

[2022.10.21]
Survey of Goal-Conditioned and Upside-Down RL
Presenter: Jake Grigsby
[Slides]

[2022.09.30]
Recent Advances in Visual Language Models
Presenter: Yue Zhao
[Slides]

[2022.09.09]
Domain Adaptation and Robot Learning
Presenter: Rutav Shah
[Slides]

Spring 2022

[2022.04.08]
Foundation Models for Robotics
Presenter: Soroush Nasiriany
[Slides]

[2022.03.31]
Legged Locomotion
Presenter: Mingyo Seo
[Slides]

[2022.03.10]
Transformers
Presenter: Yifeng Zhu
[Slides]

[2022.02.17]
Hand-Object Interaction
Presenter: Hanwen Jiang
[Slides]

Fall 2021

[2021.12.01]
Autonomous Driving
Presenter: Jerry Lin
[Slides]

[2021.11.03]
Offline Reinforcement Learning for Robotics
Presenter: Soroush Nasiriany
[Slides]

[2021.10.13]
Human-in-the-Loop Robot Learning
Presenter: Huihan Liu
[Slides]

[2021.09.17]
Implicit Neural Representations
Presenter: Zhenyu Jiang
[Slides]

Spring 2021

[2021.05.06]
Robustness via Cross-Domain Ensembles
Teresa Yeo*, Oğuzhan Fatih Kar*, Amir Zamir
arXiv 2021
Presenter: Jeffrey Zhang
[Paper] [Project] [Slides]

[2021.04.29]
Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation
Bryan Chen*, Alexander Sax*, Gene Lewis, Iro Armeni, Silvio Savarese, Amir Zamir, Jitendra Malik, Lerrel Pinto
CoRL 2020
Presenter: Soroush Nasiriany
[Paper] [Project] [Slides]

[2021.04.22]
Learning Generalizable Robotic Reward Functions from “In-The-Wild” Human Videos
Annie S. Chen, Suraj Nair, Chelsea Finn
arXiv 2021
Presenter: Sagnik Majumder
[Paper] [Project] [Slides]

[2021.04.15]
3D Shape Reconstruction from Vision and Touch
Edward J. Smith, Roberto Calandra, Adriana Romero, Georgia Gkioxari, David Meger, Jitendra Malik, Michal Drozdzal
NeurIPS 2020
Presenter: Priyanka Mandikal
[Paper] [Slides]

[2021.04.08]
Unsupervised Monocular Depth Learning in Dynamic Scenes
Hanhan Li, Ariel Gordon, Hang Zhao, Vincent Casser, Anelia Angelova
CoRL 2020
Presenter: Amanda Adkins
[Paper] [Slides]

[2021.04.01]
Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classification
Benjamin Eysenbach, Sergey Levine, Ruslan Salakhutdinov
arXiv 2021
Presenter: Dian Chen
[Paper] [Project] [Slides]

[2021.03.25]
Point Transformer
Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip Torr, Vladlen Koltun
arXiv 2020
Presenter: Zhenyu Jiang
[Paper] [Slides]

[2021.03.11]
Stein Variational Model Predictive Control
Alexander Lambert, Adam Fishman, Dieter Fox, Byron Boots, Fabio Ramos
CoRL 2020
Presenter: Mingyo Seo
[Paper] [Video] [Slides]

[2021.03.04]
Semantic Visual Navigation by Watching YouTube Videos
Matthew Chang, Arjun Gupta, Saurabh Gupta
NeurIPS 2020
Presenter: Jerry Lin
[Paper] [Project] [Slides]

[2021.02.25]
Parrot: Data-Driven Behavioral Priors for Reinforcement Learning
Avi Singh*, Huihan Liu*, Gaoyue Zhou, Albert Yu, Nicholas Rhinehart, Sergey Levine
ICLR 2021
Presenter: Mohan Kumar Sriram
[Paper] [Project] [Slides]

[2021.02.11]
Constructing Symbolic Representations for High-Level Planning
George Konidaris, Leslie Pack Kaelbling, Tomás Lozano-Pérez
AAAI 2014
Presenter: Yifeng Zhu
[Paper] [Video] [Slides]

[2021.02.04]
Where2Act: From Pixels to Actions for Articulated 3D Objects
Kaichun Mo, Leonidas Guibas, Mustafa Mukadam, Abhinav Gupta, Shubham Tulsiani
arXiv 2021
Presenter: Zhenyu Jiang
[Paper] [Project] [Slides]

[2021.01.28]
RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Real
Kanishka Rao, Chris Harris, Alex Irpan, Sergey Levine, Julian Ibarz, Mohi Khansari
CVPR 2020
Presenter: Soroush Nasiriany
[Paper] [Video] [Slides]

Fall 2020

[2020.12.03]
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Ben Mildenhall*, Pratul P. Srinivasan*, Matthew Tancik*, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng
ECCV 2020
Presenter: Zhenyu Jiang
[Paper] [Project] [Slides]

[2020.11.19]
Learning Invariant Representations for Reinforcement Learning without Reconstruction
Amy Zhang*, Rowan McAllister*, Roberto Calandra, Yarin Gal, Sergey Levine
ICLR 2021
Presenter: Jerry Lin
[Paper] [Project]

[2020.11.12]
Accelerating Reinforcement Learning with Learned Skill Priors
Karl Pertsch, Youngwoon Lee, Joseph J. Lim
CoRL 2020
Presenter: Soroush Nasiriany
[Paper] [Project] [Slides]

[2020.11.05]
Learning an Optimal Sampling Distribution for Efficient Motion Planning
Richard Cheng, Krishna Shankar, Joel W. Burdick
IROS 2020
Presenter: Mingyo Seo
[Paper] [Slides]