Prof. Kelly’s Canada Research Chair (Tier II) in Collaborative Robotics has been renewed for a second five-year term. The Laboratory is grateful to NSERC and the Government of Canada for supporting this important program and our research work! A short press release is available from University of Toronto Engineering News.

Members of the STARS Laboratory will be at IROS 2024 in Abu Dhabi! We will present two papers on work led by Oliver Limoyo:

O. Limoyo, J. Li, D. Rivkin, J. Kelly, and G. Dudek, “Photobot: Reference-guided interactive photog- raphy via natural language,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Abu Dhabi, United Arab Emirates, 2024. [Online]. Available: https://arxiv.org/abs/2401.11061.

O. Limoyo, A. Konar, T. Ablett, J. Kelly, F. R. Hogan, and G. Dudek, “Working backwards: Learning to place by picking,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Abu Dhabi, United Arab Emirates), 2024. [Online]. Available: https://arxiv.org/abs/2312.02352.

Prof. Kelly will also present at the “From Geometry to General Autonomy of Robotic Systems” workshop on Tuesday. See you in the UAE!

Paper presentation times:

• Wednesday, October 16, 09:00-10:00 PhotoBot: Reference-Guided Interactive Photography Via Natural Language
• Thursday, October 17, 17:45-18:00, Working Backwards: Learning to Place by Picking

Our own Prof. Kelly received the IEEE Robotics & Automation Magazine’s Outstanding Associated Editor Award for 2024. The award was presented at ICRA 2024 in Yokohama, Japan. Congratulations to all the awardees!

We’re celebrating 10 years of research in the STARS Laboratory in 2023-2024! The lab was founded and began operating in June of 2013. We’d like to thank all of the lab members and research partners for making the last decade an exiting one! Robotics continues to grow rapidly as a field, and we’re looking forward to the next 10 years!

Thanks to Phys.org for running a feature all about the recent preprint article by Olivier Lamarre (STARS) and Shantanu Malhotra (NASA JPL)! The article describes how to define recovery policies to ensure that a solar-powered rover remains safe (i.e., limits the risk of complete battery depletion) while travelling through permanently shadowed regions (at, e.g., the lunar south pole). The full article is under review; read the preprint on arXiv here: https://arxiv.org/abs/2307.16786

The University of Toronto Faculty of Applied Science and Engineering has released their annual list of Grads to Watch (2023), and we’re delighted that STARS Laboratory alumnus Flip Marić is part of the cohort! Filip’s PhD work focused on geometric methods for robotic manipulation; he was a Joint Educational Placement (JEP) student, co-supervised by Ivan Petrović in the LAMoR group at the University of Zagreb. Filip is now a research scientist at the Samsung AI Centre in Montreal. The full story is available here. Congratulations Filip!

Our paper by lead author Andrej Janda on self-supervised pre-training for 3D semantic segmentation won the Dyson Best Paper Award at the CoRL’22 Pre-Training Robot Learning Workshop! The paper describes an approach for pre-training 3D segmentation networks using 2D image data; our contrastive learning approach enables self-supervision and 2D-to-3D feature transfer. Better segmentation with far less (or no) manual labelling!

Congratulations to Andrej, Brandon, and Edwin! The short paper is available on arXiv: https://arxiv.org/abs/2211.11801. A big thank you to the workshop organizers and to Dyson for sponsoring the workshop award!

Congratulations to our own Olivier Lamarre for being featured in Season 2, Episode 4 of U of T’s Groundbreakers series on future innovators! Oliver talks about planetary rover navigation and prospects for discovering life beyond Earth, as well as his time at NASA JPL. The full video is available here.

Our paper entitled “Learning Sequential Latent Variable Models from Multimodal Time Series Data” was a Best Paper Finalist at the 17th International Conference on Intelligent Autonomous Systems (IAS’17), held in Zagreb, Croatia, in June! The paper studies how to best combine multimodal data (vision, haptic, and proprioceptive) to improve the performance of manipulation tasks; we learn the latent dynamics of a specific task in an fully self-supervised manner. Congratulations to Oliver Limoyo and Trevor Ablett! The preprint is available on arXiv: https://arxiv.org/abs/2204.10419.

We’re just a week away from ICRA 2022! Our laboratory will present four papers at this year’s ICRA conference in Philadelphia. Two papers discuss our recent work on a distance-geometric formulation of inverse kinematics – by unifying the problem domain and co-domain, we avoid dealing with joint angle variables and are able to leverage key results from low-rank matrix completion theory and Riemannian optimization:

F. Marić, M. Giamou, A. W. Hall, S. Khoubyarian, I. Petrović, and J. Kelly, “Riemannian Optimization for Distance-Geometric Inverse Kinematics,” IEEE Transactions on Robotics, 2021. (Early Access) [Online]. Available: https://arxiv.org/abs/2108.13720.

M. Giamou, F. Marić, D. M. Rosen, V. Peretroukhin, N. Roy, I. Petrović, and J. Kelly, “Convex Iteration for Distance-Geometric Inverse Kinematics,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 1952– 1959, Apr. 2022. [Online]. Available: https://arxiv.org/abs/2109.03374.

Our third paper investigates ways to efficiently recover the inertial parameters of manipulated objects, ensuring physically-plausible and accurate results even when operating in the typical cobot motion regime (i.e., slowly and safely):

P. Nadeau, M. Giamou, and J. Kelly, “Fast Object Inertial Parameter Identification for Collaborative Robots,” in Proceedings of the IEEE International Conference on Robotics and Automation, Philadelphia, Pennsylvania, USA, May 23–27, 2022. [Online]. Available: https://arxiv.org/abs/2203.00830.

Our fourth paper explores the use of low cost, slim, and lightweight barometric tactile sensors for slip detection. We take a data-driven, learning-based approach by training a TCN to reliably detect slip events from the raw barometer data stream. Cheap and useful for a wide range of applications:

A. Grover, P. Nadeau, C. Grebe, and J. Kelly, “Learning to Detect Slip with Barometric Tactile Sensors and a Temporal Convolutional Neural Network,” in Proceedings of the IEEE International Conference on Robotics and Automation, Philadelphia, Pennsylvania, USA, May 23–27, 2022. [Online]. Available: https://arxiv.org/abs/2202.09549.