Full day Workshop on

Learning and control for autonomous manipulation systems:
the role of dimensionality reduction 

Invited Speakers

Jan Babic – Jožef Stefan Institute

Jan Babic (https://www.ijs.si/~jbabic) is a Senior Researcher at Jozef Stefan Institute, Slovenia and an Assistant Professor at Faculty of Electrical Engineering, University of Ljubljana, Slovenia. He received his Ph.D. from Faculty of Electrical Engineering, University in Ljubljana examining the role of biarticular muscles in human locomotion. During the years 2006/2007 he was a visiting researcher at ATR Computational Neuroscience Laboratories in Japan. In November 2014 he was a visiting professor at The Institute for Intelligent Systems and Robotics, University of Pierre and Marie Curie in France. His current research is particularly concerned with the understanding how human brain controls movement of the body with the application of this knowledge in robotic tasks that involve physical interaction with humans, such as human – robot collaboration and assistive devices. A main focus of his research is to understand how the central nervous system process sensory information and transfer them to motor commands. He is especially interested in robustness and adaptations of the movements to the changing environment. Currently he is involved in three larger European projects; in Horizon 2020 SPEXOR as the coordinator, and in FP7 CoDyCo and Horizon 2020 AnDy as the principal investigator.

Title: Complementary sensorimotor control during physical human-robot collaboration

 Abstract: In the past many studies on human motor control investigated how individuals perform arm manipulation tasks. In robotics, these studies were frequently used as a basis to control robots in scenarios of  human-robot collaboration. Nevertheless, a deeper understanding of how  multiple human subjects collaborate between each other is needed to further enhance the performance of robots while they collaborate with the humans. In this talk I will present our latest experimental study  where we investigated how pairs of humans perform and co-adapt to each other while they have to physically collaborate to carry out a joint task and how we utilized such models to control the robots in human-robot cooperative setups.


  Antonio Bicchi – Centro E. Piaggio

Antonio Bicchi is Professor of Robotics at the University of Pisa, and Senior Scientist at the Italian Institute of Technology in Genoa. He graduated from the University of Bologna in 1988 and was a postdoc scholar at M.I.T.  Artificial Intelligence lab in 1988–1990.  He teaches  Control Systems and Robotics in the Department of Information Engineering  (DII) of the University of Pisa. He leads the Robotics Group at the  Research Center "E. Piaggio'' of the University of Pisa since 1990, where he was Director from 2003 to 2012. He is the Head of the SoftRobotics Lab for Human Cooperation and Rehabilitation at IIT in Genoa. Since 2013 he serves ad Adjunct Professor at the School of Biological and Health Systems Engineering of Arizona State University. His main research interests are in Robotics, Haptics, and Control Systems in general. He has published more than 400 papers on international journals, books, and refereed conferences.  He is Editor-in-Chief of the IEEE Robotics and Automation Letters, which he started in 2015. He has organized and chaired the first WorldHaptics Conference (2005). He is Editor of the book series ``Springer Briefs on Control, Automation and Robotics,'' and has served in the Int.l J. Robotics Research, the IEEE Trans. on Robotics and Automation, IEEE Trans. Automation Science and Engineering, and IEEE RAS Magazine. He was  Program Chair of the IEEE Int.. Conf. Robotics and Automation (ICRA'15), and General Chair of  the Int. Symposium on Robotics Research (ISRR' 2015) and Hybrid Systems: Computation and Control (HSCC 2007). He was Editor in Chief of the Conference Editorial Board for the IEEE Robotics and Automation Society (RAS), Vice President for Publications (2013-2014), for Membership (2006-2007), and as Distinguished Lecturer (2004-2006) of IEEE RAS. He served as the President of the Italian Association or Researchers in Automatic Control (2012-2013). He is the recipient of several awards and honors. In 2012, he was awarded with an Advanced Grant from the European Research Council for his research on human and robot hands. Antonio Bicchi is a Fellow of IEEE since 2005.

Title: Designing and Using Hands with More Synergies 

Abstract: To deal with the complexity of grasping and manipulation with human and robot hands, many researchers have been using hierarchically organized representations of reduced dimensions, e.g. principal components of observed hand postures ordered by their statistical relevance to a set of tasks, aka postural synergies. Ideally, one would like to have tools to exploit this structure to design and control hands of increasing complexity, matching the needs of more advanced tasks. This could be in principle a solution to the minimalistic robotic problem, i.e. to find the simplest possible system that solves a given task. However, much remains to be done in this direction. In this talk I will discuss a few preliminary steps taken toward designing and controlling hands with more synergies.


Aude Billard – École Polytechnique Fédérale de Lausanne (EPFL)

Professor Aude Billard is head of the Learning Algorithms and Systems Laboratory (LASA) at the School of Engineering at the EPFL. She received a M.Sc. in Physics from EPFL (1995), a MSc. in Knowledge-based Systems (1996) and a Ph.D. in Artificial Intelligence (1998) from the University of Edinburgh. Her research spans the fields of machine learning and robotics with a particular emphasis on learning from sparse data and performing fast and robust retrieval. Her work finds application to robotics, human-robot / human-computer interaction and computational neuroscience. She was the recipient of the Intel Corporation Teaching award, the Swiss National Science Foundation career award in 2002, the Outstanding Young Person in Science and Innovation from the Swiss Chamber of Commerce and the IEEE-RAS Best Reviewer Award. Her research was nominated and received best paper awards at IEEE TRO, ICRA, IROS, NIPS, Humanoids, ROMAN and was featured in premier venues (BBC, IEEE Spectrum, Wired).

Title: Learning the dimensions that do not matter is important to offer robustness in control

Abstract: Most tasks entail redundancy.  Redundancy is advantageous in that it offers multiple ways to solve the task, a flexibility required to adapt to perturbations. Task redundancy is conveyed through variability in control strategies. When manipulating objects with one or several arms, the dimension of the control can be very high. Machine learning techniques provide tools to embed a representation of this high-dimensional and non-linear manifold of this space of feasible motion strategies that can ease control.  Identifying task-relative constraints and learning how these constraints vary during task completion can also be used to model changes in impedance during manipulation. Task-relative constraints correspond to control directions in which the task shows no variability. This translates into parameters of impedance, with low stiffness along directions with high variability and, conversely, with high stiffness in directions with low variability, i.e. directions highly constrained


Tamar Flash – The Weizmann Institute of Science

Tamar Flash is a professor at the Department of Computer Science and Applied Mathematics, Weizmann Institute of Science, Israel. She earned her BSc and MSc degrees in Physics from the Tel-Aviv University, Israel. She received her Ph.D. in Medical Physics from the Massachusetts Institute of Technology (1983) where she enrolled in the Harvard-MIT Division of Health Science and Technology. She continued with her postdoctoral training at MIT, at the Department of Brain and Cognitive Science and the Artificial Intelligence Laboratory (1983-1985). In 1985 she joined the Department of Computer Science and Applied Mathematics at the Weizmann Institute of Science where she established a research group, focusing on motor control and robotics and also served as the department head (2004-2007). She was a visiting professor at MIT, the College de France, Berkeley University and a fellow at the Radcliffe Institute for Advanced Studies, Harvard University. The focus of her research is on computational neuroscience, human motor control, movement disorders, the control of hyper-redundant flexible biological and robotic systems and humanoid robots.

Title: Principles underlying dimension reduction and compositionality in human movements and robotic implementations

Abstract: In my talk I will discuss several recent research directions we have taken to explore the different principles underlying dimension reduction and compositionality in the control and construction of complex human upper limb and gait movements.   Investigating motor compositionality we have explored the nature of motor primitives underlying the construction of complex movements at different levels of the motor hierarchy.  In particular we have focused on seeking and inferring what motion primitives are possibly used in the generation of hand and center of mass (COM) trajectories during human upper limb and locomotion movements, respectively. I will also discuss   the topic of motor coordination and the mapping between end-effector and joint motions both during arm and leg movements. The mathematical models we have used in these studies combine geometrical approaches with optimization models aimed at inferring motion invariants and to unravel motor coordination and timing strategies. The usefulness of these approaches for humanoid robot research will be demonstrated.


Ken Goldberg – University of California, Berkeley

Ken Goldberg is an artist, inventor, and UC Berkeley Professor. He is Chair of the Industrial Engineering and Operations Research Department, with secondary appointments in EECS, Art Practice, the School of Information, and Radiation Oncology at the UCSF Medical School. Ken is Director of the CITRIS "People and Robots" Initiative and the UC Berkeley AUTOLAB where he and his students pursue research in geometric algorithms and machine learning for robotics and automation in surgery, manufacturing, and other applications. Ken developed the first provably complete algorithms for part feeding and part fixturing and the first robot on the Internet. Despite agonizingly slow progress, Ken persists in trying to make robots less clumsy. He has over 200 peer-reviewed publications and eight U.S. Patents. He co-founded and served as Editor-in-Chief of the IEEE Transactions on Automation Science and Engineering. Ken's artwork has appeared in 70 exhibits including the Whitney Biennial and films he has co-written have been selected for Sundance and nominated for an Emmy Award. Ken was awarded the NSF PECASE (Presidential Faculty Fellowship) from President Bill Clinton in 1995, elected IEEE Fellow in 2005 and selected by the IEEE Robotics and Automation Society for the George Saridis Leadership Award in 2016. He lives in the Bay Area and is madly in love with his wife, filmmaker and Webby Awards founder Tiffany Shlain, and their two daughters. He is fiercely protective of his family, his students, and his frequent-flier miles.

Title: The New Wave in Robot Grasping 

Abstract:  Despite 50 years of research, robots are still remarkably clumsy.  I will present what I see as three "waves" in methodology.  The first wave, which is still dominant, uses analytic methods based on screw theory and assumes exact knowledge of pose, shape, and other properties (see the 2016 Handbook of Robotics).  The relatively recent Second Wave is purely empirical: purely data driven approaches which learn grasp strategies from many examples using techniques such as reinforcement learning and hyperparametric function approximation (Deep Learning).  The "New" wave will be hybrid methods that combine analytic models to bootstrap and tune empirical models, where data and code is exchanged via the Cloud using emerging advances in cloud computing, big data, deep learning, and the internet of things.  This talk will present an overview and new results from my lab for applications in home decluttering, warehouse order fulfillment, and robot-assisted surgery.



 Matthew Howard – Kings College London  


Dr Matthew Howard is a lecturer at the Centre for Robotics Research, Dept. Informatics, King's College London. Prior to joining King's in 2013, he held a Japan Society for Promotion of Science fellowship at the Department of Mechanoinformatics at the University of Tokyo and was a research fellow at the University of Edinburgh from 2009-2012. He also obtained his PhD in 2009 at Edinburgh with award of an EPSRC CASE award sponsored by Honda Research. His research interests span the fields of robotics and autonomous systems, statistical machine learning and adaptive control. His current work focuses on
robotic skill learning and fast (re)programming by demonstration for soft robotic devices, design and control of variable impedance devices and EMG-based robot control and teleimpedance, and textile-based wearable sensor design.

Title: : Learnt Redundancy Resolution and Constraints in Grasping



  Dongheui Lee – Technical University of Munich

Dongheui Lee is an assistant professor at the Institute of Automatic Control Engineering, Department of Electrical and Computer Engineering, Technical University of Munich, Germany since October 2009. She is the head of Dynamic Human Robot Interaction for Automation System Lab and Carl-von-Linde Fellow at TUM Institute for Advanced Study. She received her B.S. and M.S degrees at the department of mechanical engineering, Kyunghee University, Korea, in 2001 and 2003, respectively. She worked as a research scientist at the Advanced Robotics Research Center, Korea Institute of Science and Technology (KIST) in Korea from 2001 to 2004. In 2007, she received her PhD degree at the department of Mechano‐Informatics, the University of Tokyo, Japan. After receiving PhD degree she joined the center of Information and Robot Technology at the University of Tokyo as a project assistant professor. In 2015 she was awarded for a Helmholtz professorship prize. Her research interests include human motion understanding, human robot interaction, machine learning in robotics, and mobile robot navigation. She is Coordinator of euRobotics Topic Group on physical Human Robot Interaction and the co-coordinator of TUM Center of Competence Robotics, Autonomy and Interaction.

Title: Challenges in Kinesthetic and Teleoperation Teaching of Manipulation Skills  

Abstract: In this talk, I will present our recent activities on manipulation skill learning at the Dynamic Human Robot Interaction Lab at TUM. New action descriptors have been developed in order to represent skills more effectively.  We have investigated efficient teaching approaches for a human to transfer his/her manipulation skills to a humanoid robot. Different teaching modalities face different challenges. I will discuss some challenges in kinesthetic teaching and teleoperation teaching and present our recently developed control and learning algorithms in order to tackle these issues in a manipulation skill learning context. 


Sergey Levine – UC Berkeley

Sergey Levine received a BS and MS in Computer Science from Stanford University in 2009, and a Ph.D. in Computer Science from Stanford University in 2014. He joined the faculty of the Department of Electrical Engineering and Computer Sciences at UC Berkeley in fall 2016. His work focuses on machine learning for decision making and control, with an emphasis on deep learning and reinforcement learning algorithms. Applications of his work include autonomous robots and vehicles, as well as computer vision and graphics. His research includes developing algorithms for end-to-end training of deep neural network policies that combine perception and control, scalable algorithms for inverse reinforcement learning, deep reinforcement learning algorithms, and more.

Title: Deep Robotic Learning and Dexterous Manipulation

Abstract: Deep learning has produced widespread improvements in passive perception: from speech recognition to computer vision, tasks that involve detecting, classifying, or localizing in passive sensory input data benefit tremendously from the representational capacity of deep neural networks. However, does this extend to real-world decision making and control, at the degree of fidelity necessary for dexterous manipulation? In this talk, I will discuss a few recently developed algorithms that could be suitable for acquiring complex dexterous manipulation skills in challenging real-world settings. I will present experimental results for vision-based control with conventional parallel-jaw grippers for tasks such as grasping and basic object manipulation, as well as some of our experiments with deep learning for control of dexterous 5-finger hands.


Jan Peters – TU Darmstadt and Max-Planck Institute for Intelligent Systems 

Jan Peters is a full professor (W3) for Intelligent Autonomous Systems at the Computer Science Department of the Technische Universitaet Darmstadt and at the 
same time a senior research scientist and group leader at the Max-Planck Institute for Intelligent Systems, where he heads the interdepartmental Robot Learning Group. Jan Peters has received the Dick Volz Best 2007 US PhD Thesis Runner-Up Award, the Robotics: Science & Systems - Early Career Spotlight, the INNS Young Investigator Award, and the IEEE Robotics & Automation Society's Early Career Award. 
Jan Peters has studied Computer Science, Electrical, Mechanical and Control  Engineering at TU Munich and FernUni Hagen in Germany, at the National University of Singapore (NUS) and the University of Southern California (USC). He has received four Master's degrees in these disciplines as well as a Computer Science PhD from USC. Jan Peters has performed research in Germany at DLR, TU Munich and the Max Planck Institute for Biological Cybernetics (in addition to the institutions above), in Japan at the Advanced Telecommunication Research Center (ATR), at USC and at both NUS and Siemens Advanced Engineering in Singapore.

Title: TBA