Learning to Synthesize Programs as Interpretable and Generalizable Policies
Dweep Trivedi*, Jesse Zhang*, Shao-Hua Sun*, Joseph J. Lim (* - indicates equal contribution)
NeurIPS 2021 [PDF] [Project]
Generalizable Imitation Learning from Observation via Inferring Goal Proximity
Youngwoon Lee*, Andrew Szot*, Shao-Hua Sun, Joseph J. Lim
NeurIPS 2021 [PDF to come]
Demonstration-Guided Reinforcement Learning with Learned Skills
Karl Pertsch, Youngwoon Lee, Yue Wu, Joseph J. Lim
CoRL 2021 [PDF] [Project] [Code]
Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularization
Youngwoon Lee, Joseph J. Lim, Anima Anandkumar, Yuke Zhu
CoRL 2021 [PDF]
Distilling Motion-Planner Augmented Policies into Visual Control Policies for Robot Manipulation
I-Chun Arthur Liu*, Shagun Uppal*, Gaurav S. Sukhatme, Joseph J. Lim, Peter Englert, Youngwoon Lee
CoRL 2021 [PDF]
Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Grounding
Grace Zhang, Linghan Zhong, Youngwoon Lee, Joseph J. Lim
Message Passing Adaptive Resonance Theory for Online Active Semi-supervised Learning
Taehyeong Kim, Injune Hwang, Hyundo Lee, Hyunseo Kim, Won-Seok Choi, Joseph J. Lim, Byoung-Tak Zhang
ICML 2021 [PDF]
IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Tasks
Youngwoon Lee, Edward S. Hu, Joseph J. Lim
ICRA 2021 [PDF] [Project] [Code]


Accelerating Reinforcement Learning with Learned Skill Priors
Karl Pertsch, Youngwoon Lee, Joseph J. Lim
CoRL 2020 (Plenary Talk, Best Paper Presentation Award) [PDF] [Project] [Code] [Talk]
Motion Planner Augmented Reinforcement Learning for Robot Manipulation in Obstructed Environments
Jun Yamada*, Youngwoon Lee*, Gautam Salhotra, Karl Pertsch, Max Pflueger, Gaurav S. Sukhatme, Joseph J. Lim, Peter Englert
CoRL 2020 [PDF] [Project] [Code]
Generalization to New Actions in Reinforcement Learning
Ayush Jain*, Andrew Szot*, Joseph J. Lim
Keyframing the Future: Keyframe Discovery for Visual Prediction and Planning
Karl Pertsch*, Oleh Rybkin*, Jingyun Yang, Shenghao Zhou, Kosta Derpanis, Joseph J. Lim, Kostas Daniilidis, Andrew Jaegle (* - indicates equal contribution)
L4DC 2020 [PDF] [Project]
Program Guided Agent
Shao-Hua Sun, Te-Lin Wu, Joseph J. Lim
ICLR 2020, Addis Ababa, Ethiopia (Spotlight) [PDF] [Project]
Learning to Coordinate Manipulation Skills via Skill Behavior Diversification
Youngwoon Lee, Jingyun Yang, Joseph J. Lim
ICLR 2020, Addis Ababa, Ethiopia [PDF] [Project] [Code]


To Follow or not to Follow: Selective Imitation Learning from Observations
Youngwoon Lee, Edward S. Hu, Zhengyu Yang, Joseph J. Lim
CoRL 2019, Osaka, Japan [PDF] [Project] [Spotlight]
Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation
Risto Vuorioi*, Shao-Hua Sun*, Hexiang Hu, Joseph J. Lim (* - indicates equal contribution)
NeurIPS 2019, Vancouver, Canada (Spotlight) [PDF] [Project] [Code] [Spotlight]
Composing Complex Skills by Learning Transition Policies
Youngwoon Lee*, Shao-Hua Sun*, Sriram Somasundaram, Edward Hu, Joseph J. Lim (* - indicates equal contribution)
ICLR 2019, New Orleans, Louisiana [PDF] [Project] [Code]
Scaling Simulation-to-Real Transfer by Learning Composable Robot Skills
Ryan Julian*, Eric Heiden*, Zhanpeng He, Hejia Zhang, Stefan Schaal, Joseph J. Lim, Gaurav Sukhatme, Karol Hausman (* - indicates equal contribution)
ISER 2018, Buenos Aires, Argentina (IJRR 2019) [PDF]


Multi-view to Novel view: Synthesizing Novel Views with Self-Learned Confidence
Shao-Hua Sun, Minyoung Huh, Yuan-Hong Liao, Ning Zhang, Joseph J. Lim
ECCV 2018, Munich, Germany. [PDF] [Project] [Code]
Neural Program Synthesis from Diverse Demonstration Videos
Shao-Hua Sun*, Hyeonwoo Noh*, Sriram Somasundaram, Joseph J. Lim (* - indicates equal contribution)
ICML 2018, Stockholm, Sweden. [PDF] [Project] [Code]
Demo2Vec: Reasoning Object Affordances From Online Videos
Kuan Fang*, Te-Lin Wu*, Daniel Yang, Silvio Savarese, Joseph J. Lim (* - indicates equal contribution)
CVPR 2018, Salt Lake City, Utah. [PDF] [Project] [Data]
Learning and Using the Arrow of Time
Donglai Wei, Joseph J. Lim, Andrew Zisserman, William Freeman
CVPR 2018, Salt Lake City, Utah. [PDF]
Universal Agent for Disentangling Environments and Tasks
Jiayuan Mao, Honghua Dong, Joseph J. Lim
ICLR 2018, Vancouver, Canada [PDF]


Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Nets
Karol Hausman, Yevgen Chebotar, Stefan Schaal, Gaurav Sukhatme, Joseph J. Lim
NIPS 2017, Long Beach, California [PDF]
Unsupervised Visual-Linguistic Reference Resolution in Instructional Videos
De-An Huang, Joseph J. Lim, Li Fei-Fei, Juan Carlos Niebles
CVPR 2017, Honolulu, Hawaii [PDF]
Knowledge Acquisition for Visual Question Answering via Iterative Querying
Yuke Zhu, Joseph J. Lim, Li Fei-Fei
CVPR 2017, Honolulu, Hawaii [PDF]
Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J. Lim, Abhinav Gupta, Li Fei-Fei, Ali Farhadi
ICRA 2017, Singapore [PDF] [arXiv]


High-Fidelity Facial and Speech Animation for VR HMDs
Kyle Olszewski, Joseph J. Lim, Shunsuke Saito, Hao Li
SIGGRAPH Asia 2016, Macau, China [PDF]
Single Image 3D Interpreter Network
Jiajun Wu*, Tianfan Xue*, Joseph J. Lim, Yuandong Tian, Joshua B. Tenenbaum, Antonio Torralba, William T. Freeman (* - indicates equal contribution)
ECCV 2016, Amsterdam, Netherlands [PDF] [arXiv]
Physics 101: Learning Physical Object Properties from Unlabeled Videos
Jiajun Wu, Joseph J. Lim, Hongyi Zhang, Joshua B. Tenenbaum, William T. Freeman
BMVC 2016, York, UK [PDF]

Before 2016

Galileo: Perceiving Physical Object Properties by Integrating a Physics Engine with Deep Learning
Jiajun Wu*, Ilker Yildirim*, Joseph J. Lim, William T. Freeman, Joshua B. Tenenbaum. (* - indicates equal contribution)
NIPS 2015, Montreal, Canada [PDF]
Discovering States and Transformations in Image Collections
Phillip Isola*, Joseph J. Lim*, Edward H. Adelson. (* - indicates equal contribution)
CVPR 2015, Boston, Massachusetts [PDF]
FPM: Fine pose Parts-based Model with 3D CAD models
Joseph J. Lim, Aditya Khosla, Antonio Torralba.
ECCV 2014, Zurich, Switzerland [PDF]
Looking Beyond the Visible Scene
Aditya Khosla*, Byoungkwon An*, Joseph J. Lim*, Antonio Torralba. (* - indicates equal contribution)
ECCV 2014, Zurich, Switzerland [PDF]
Parsing IKEA Objects: Fine Pose Estimation
Joseph J. Lim, Hamed Pirsiavash, Antonio Torralba.
ICCV 2013, Sydney, Australia [PDF]
Sketch Tokens: A Learned Mid-level Representation for Contour and Object Detection
Joseph J. Lim, C. Lawrence Zitnick, and Piotr Dollar.
CVPR 2013, Portland, Oregon [PDF]
Transfer Learning by Borrowing Examples for Multiclass Object Detection
Joseph J. Lim, Ruslan Salakhutdinov, and Antonio Torralba
NIPS 2011, Granada, Spain [PDF]
Experimental Evaluation of Support Vector Machine-based and Correlation-based Approaches to Automatic Particle Selection
Pablo Arbelaez, Bong-Gyoon Han, Dieter Typke, Joseph J. Lim, Robert M. Glaeser, Jitendra Malik.
Journal of Structural Biology 2011. doi:10.1016/j.jsb.2011.05.017 [PDF]
Exploiting Hierarchical Context on a Large Database of Object Categories
Myung Jin Choi, Joseph J. Lim, Antonio Torralba, and Alan S. Willsky.
CVPR 2010, San Francisco, California [PDF]
Context by Region Ancestry
Joseph J. Lim, Pablo Arbelaez, Chunhui Gu, Jitendra Malik.
ICCV 2009, Kyoto, Japan [PDF]
Recognition using Regions
Chunhui Gu, Joseph J. Lim, Pablo Arbelaez, Jitendra Malik.
CVPR 2009, Miami, Florida [PDF]
A Distributed Message Passing Algorithm for Sensor Localization
Max Welling and Joseph J. Lim.
ICANN 2007, Porto, Portugal. [PDF]