Robotics

This research activity, carried out at the Ampère laboratory, deals with the fields of Automation and Robotics: in the broad sense, control of mechatronic systems.
It has both methodological and applied characters, which explains my commitment to the development of experimental platforms within the Fluid Power Test Center of Ampere laboratory. The topics I work on are:

  • haptic interfaces, mainly in medical robotics,
  • teleoperation and bilateral control,
  • gesture recognition and learning.

All in one: Research Projects about Robotics

Period Project Name Description
2017-2021 Green Shield: Pesticide Free Robotized Pest Control in Agriculture This project aims at designing and prototyping a mobile robot to detect and kill pests in fields.
2015-2017 PERISim
( IDEFI SAMSEI )
This project aims at designing and prototyping an needle insertion training simulator for epidural anaesthesia.
2015-2017 LAPAROSim
( IDEFI SAMSEI )
This project aimed at designing and prototyping a training simulator concerning basic gestures in Laparoscopy.
2013-2017 SoHappy
(PEPS CNRS INSIS)
In collaboration with PRISME institute, this project aims at prototyping remote ultrasonography haptic probes using pneumatic actuators.
2012-2015 INTELO
(FEDER Région Rhône Alpes with European funds)

This project aimed at designing and prototyping a mobile robot to inspect the underneath of railway and road bridges.

2013-2016 SAGA
(ANR Modèles Numériques)

This project consisted of enhancing and extending BirthSIM simulator.
2011-2013 Decortiquemax
(PEPS CNRS INSIS)
This project aimed at studying a pneumatic actuating for a needle insertion robot compliant with MRI scanners.
2008-2011 PROSIT Teleoperation work-package
(ANR)
As a collaborator of LIRMM laboratory, I participated to the study of the teleoperation over Internet and a satellite connection of the remote ultrasonography robot developed by the partners during this project.

Haptic Systems for Hands-on Training

2015-... PERISIM: Simulator of Epidural Needle Insertion

Research Project
Objective: Study, Design and Prototyping of an Epidural Anaesthesia Simulator

Keywords: Haptics, Hands-on Training, Simulation, Anaesthetics, Medical Robotics

Students: Pierre-Jean ALES-ROUX (MSc 2015), Thibaut SENAC (MSc 2016) and (PhD)

In collaboration with: Richard MOREAU

Period: 2015-2019

Financed by IDEFI SAMSEI

Publication list: click here

PhD report: click here

2016-2020 Sparte

Research Project
Objective: Study, Design and Prototyping of a training simulator for needle insertion under ultrasonography in Rheumatology

Keywords: Haptics, Hands-on Training, Simulation, Rheumatology, Medical Robotics, Needle Insertion, Ultrasonography

In collaboration with: Richard MOREAU

Financed by IDEFI SAMSEI

Period: 2015-2019

Dual User Haptic Training

Joint trainer and trainee haptic simulation (for learning by doing)

Introduction

For surgical gestures that are more difficult to acquire, the classic training method, in the field (in the operating room), consists, for a trainee, in operating on a patient, the hands guided by those of a trainer (nicknamed “four-handed” method). However, this does not currently find its equivalent in computer based simulators where the trainees are alone with their tools immersed in their environment.

It presents however some disadvantages: in particular, it is difficult for the two people to dose, for one and to estimate
for the other, the efforts to be made when the four hands are joined two by two because the effort is shared in such a random way
between the two people. It is a real obstacle for training the gesture because this dimension is distorted.

In the synthesis [Coles 2010] concerning educational simulators in the medical field, it appears that the current simulators are mainly based on real or virtual environments where the trainee is alone, which makes any possibility of guiding him in his actions difficult. Yet, as in practical training in four hands, especially for complex gestures, it is important that the trainer can intervene; to guide the trainees, to evaluate them immediately, or to correct potentially dangerous trajectories.

It is also necessary, for more flexibility in the training, to keep the possibility for the trainer to intervene in the simulation as he classically intervenes in classical practical training. It is not possible to program all scenarios in advance in the simulator. In use, the most recurrent can gradually be integrated into the simulator, but there will always be special cases where the intervention
of the trainer will be necessary: ​​to unblock the trainees, to advise them, ... Hence the interest of proposing simulators integrating the trainer into the simulation.

The da Vinci Si dual-console system robot [g71] offers a training mode for two concurrent users. However, only one user at a time has access to the instruments and neither user has no haptic feedback.

In dual user systems, several haptic interfaces are connected to a robot (real or virtual) thanks to a software. The parameter α determines the dominance of each user over the slave. When α = 1 (resp. α = 0, it is user 1 (resp. 2) who has complete control of the slave. When 0

Contributions

First, we designed the bases of a new educational simulator, gesture training, usable to two users (trainer and learner) based on a novel controller, managing energy exchanges between sub-systems (Energy Shared Control - ESC) [Liu 2015].The energy approach (modeling by Hamiltonian ports) offers the advantage of proving intrinsically that the system is passive whatever the evolution of α (which is not the case time-invariant linear dual-user models where α is a parameter and is therefore supposed to be constant).
Whatever the level of authority granted to a user, the latter perceives an effort feedback in accordance with the interaction efforts tool-environment, even if it is not the user at the origin of this interaction; thus the person who observes the movement feels the same efforts as the person actually handling the tool. This property was not seen ever in the scientific literature.

We validated it experimentally using axis 1 (vertical) of two Omni haptic interfaces and one virtual interface (simulated under Matlab) . Having noticed that the trainer needs to regain control very quickly in the event of an erroneous or dangerous gesture (like the driving instructor who can brake on his own brake pedal), we have developed the AAA (Adaptive Authority Adjustment) function which, when in evaluation mode, switches control back (changes α) to the trainer as soon as the trajectory of its interface moves away (necessarily voluntarily) from that of the learner.

However, this solution had the disadvantage of requiring two parameters that were difficult to adjust by a non-professional trainer.
We compared the performance of ESC against two recognized architectures offered by Khademian and Hashtrudi-Zaad [Khademian 2011](Complementary Linear Combination (CLC) and Masters Correspondence
with Environment Transfer (MCET)), in simulation [Liu 2016]. This study demonstrated that the performance of ESC in terms of position tracking were equivalent to those of CLC and MCET. However, the force feedback from ESC is intrinsically better for educational applications because CLC and MCET do not allow to realize demonstrations and evaluations involving simultaneous positioning and force feedback to both users.
We then improved ESC for which the environment and users were assumed to be passive (which is debatable particularly concerning the users). We added a passivity controller (Time Domain Passivity Controller: TDPC) to keep the system passive whatever the behavior of the slave environment and of the users and independently of α and the parameters of the IPC controllers. [Liu 2016]

At the end of Fei Liu's Ph.D., this simulator had only one degree of freedom (one rotation).
Angel Licona's objectives were to extend this simulator in terms of degrees of freedom.

Thus, we tested this architecture with n degrees of freedom by duplicating ESC for each joint. This supposes that the three interfaces have the same kinematics. That is the case for the masters but one can argue for the slave. We experimentally tested this approach with three degrees of freedom. The results are available in [Liu 2019].
We also proposed a new algorithm for AAA (also extended to n degrees of freedom) which now requires only one easily adjustable parameter in continuous by the trainer, in order to leave more or less freedom of movement to the learner.
Experiments were carried out integrating all these developments . They were published in [Liu 2019b].

We have studied the extension of this architecture to m > 2 users in order to meet the needs of collective training during which, for example, the trainer would only have to perform a single demonstration to m − 1 simultaneous learners. All other scenarios are possible as long as a single user is in control on the slave and the other observers. This experimentally validated study was published as part of the IROS 2019 conference [Licona 2019].

We have also studied the use of ESC with haptic interfaces with different kinematics to be able to control a slave robot different from the haptic interfaces, which seems the most interesting configuration in practice. For this, we proposed to use ESC for each dimension in Cartesian space instead of the joint space, hypothesizing that the couplings between these dimensions would be considered as disturbances by each IPC controller and absorbed as such.

Pneumatic Actuation for Haptic Feeback and Teleoperation

Introduction

Haptic systems are designed for the interaction between a virtual tool in a simulation situation computing [Corrêa 2019], to teleoperate a remote robot (carrying a haptic probe, for example Krupa 2014], or a UAV flotilla [Son 2019], ...
For educational purposes, the behavior of such systems must be realistic (also closer than the tool they simulate), but off-the-shelf haptic systems are not always suitable [Kheddar 2004].

For some practical reasons, commercial simulators are equipped with electric actuators which provide feedback imitating, for example, the behavior of a tool touching a human organ in a surgical context.
Today, the haptic control laws applied to electric actuators are well mastered.
However, electric actuators have certain limitations for this type of use:

  • they have a lower power-to-weight ratio than pneumatic actuators;
  • it is difficult for them to provide a high torque at high speed;
  • they need reducers (which limits their reversibility), and;
  • they heat up when low-speed torque is required.

All these limitations reduce their performance when it is necessary to reproduce a variable stiffness quickly.
For several decades, complex mechanisms have been devised with the aim of providing compliance to actuators: these are called “Variable Stiffness Actuators – UAV ”. These actuators independently control their balance position and stiffness. Van Ham et al. present a state of the art on VSAs [Van Ham]. Most of them are designed with two opposing electric motors and passive compliant components. One of the advantages of this approach is that the control of position and stiffness are obtained independently by controlling the position of each motor. The main disadvantages are their cost (two motors per axis) and the limited amplitude of the stiffness due to the use of passive components [Huang 2013].

Shortly before my arrival at the Ampère laboratory, due to a long-standing expertise in action control, pneumatic actuators, the medical robotics team had begun to take an interest in the use of such actuators to render a haptic rendering. This technology is underused at the industrial level because it is considered too complex to control. However, it brings new possibilities in the medical field. In fact the actuators tires have a very interesting structural compliance. Simultaneous pressure control in both chambers of a cylinder opens the way to control of the mechanical stiffness of the piston and therefore to a rendering in effort of better quality than that obtained with electric actuators. At equal and constant pressure at rest in each chamber of a jack, this one, during an excitation, will react like a spring presenting a stiffness during the initial pressure level. With an electrical system, it is necessary to enslave the motor to recreate this natural phenomenon. On the other hand, by adjusting the pressure difference in each chamber, it is possible to check the pneumatic force applied to the piston. In summary, the advantages of pneumatic actuators over electric actuators are:

  • a higher force-to-mass ratio;
  • the non-necessity of a reducer;
  • their ability to be powered by energy readily available in industrial and medical environments and sufficient
    sufficiently clean (provided it is filtered) for biomedical applications;
  • the possibility of being non-magnetic, which allows them to be used in environments such as an MRI.

However, pneumatic actuators have a major drawback: the air is compressible and the behavior of pneumatic actuators is inherently non-linear. Unlike hydraulic actuators, dry friction is important since air is a low viscous fluid.

Contributions

In 2011, when I arrived at the Ampère laboratory, a research project concerning the teleoperation of robots using pneumatic actuators had been started. Indeed, it turned out that many works dealt with the modeling of pneumatic components (actuators, power modulators) but also their control with a view of position or force control [Belgharbi, 1999], but very few concerned their use in teleoperation. In the framework of of Minh Quyen Le's Ph.D., the team had developed a pneumatic haptic interface that could accommodate two types of power modulators: proportional servo valves or solenoid valves. The servo valves deliver an air flow depending on the control voltage and downstream pressure, while the control of solenoid valves is limited to open or close.
In the industrial world, despite their performance, servo valves are much less widespread than solenoid valves because of their price but also the expertise needed to fully exploit them. A modeling had been proposed and a control architecture produced using a linear tangent model of the pneumatic and mechanical chain around an operating point. This model resulted in a transfer function of the third order (integrator + second order) which has been used in a four-channel teleoperation architecture.
Experiments led to interesting results.

However, additional experiments, which I conducted personally upon my arrival, showed that this control architecture did not make it possible to correctly control the pressure levels in the cylinder chambers, resulting in under-performance. Typically, cylinder chambers were often at average pressure levels close to the intake pressure (therefore at the maximum), which prevented to efficiently and quickly generate pneumatic forces as it was necessary to wait for one of the chambers to empty the air, to generate this desired force. Maintaining the chambers at an intermediate pressure would have made it possible to play simultaneously on the pressurization of one chamber and the depressurization of the other and to gain in dynamics.
On the other hand, for our haptic applications, the choice of servo-valves available on the market is limited because the latter are mainly dedicated to more powerful industrial applications and are poorly suited to low forces and small displacements encountered in our case. For all these reasons, the team had simultaneously decided to study the use of solenoid valves. Unlike proportional servo valves, there is a range largest number of off-the-shelf solenoid valves. These components have an on/off type operation which more coarsely modulates the useful air flow. By playing on the rapid switching from closing to the opening (and vice versa) of the solenoid valves, the team sought to control the flow of air sent to the rooms pneumatic actuators (and the exhaust flow from them). It was therefore an approach of control of a hybrid system: switching and non-linear dynamics.

For my part, I participated in the development, optimization and experimental validation of the integration control laws developed by the team for a pneumatic actuator in a teleoperation loop with only one degree of freedom.

Two control approaches have been proposed, tested and compared. This work has been published in two international journal articles [Hodgson 2014a] [Hodgson 2015] and two international communications [Hodgson 2012] [Hodgson 2014b].

Telerobotics

History

This activity started in 1996 :

This research activity drastically slowed down between 2001 and 2011 as I was working on E-Laboratories in ICTT and afterwards in LIESP laboratories, both dealing with e-learning research topics.

Since early 2011, this topic is restarting at AMPERE laboratory and in collaboration with LIRMM robotics team.

Introduction to teleoperation

There are situations when firms or laboratories have to resort to remote manipulation. Such cases appear when dangerous objects have to be handled [1] or/and when the environment is too aggressive for humans. Typical applications belong to the nuclear domain (for instance in the dismantling of a nuclear plant), deep-sea domain (work on underwater structures of oil rigs) and spatial domain (exploration of distant planets).

Teleoperation has the supplementary advantage of giving the possibility of sharing an experiment between several operators located in distinct places. This way, heavy outdoor experimentations could be easily shared between several laboratories and costs could be reduced as much. However, long distance control of a remote system requires the use of different transmission media which causes two main technical problems in teleoperation: limited bandwidth and transmission delays due to the propagation, packetisation and many other events digital links may inflict on data [2]. Moreover bandwidth and delays may vary according to events occurring all along the transmission lines. In acoustic transmission, round-trip delays greater than 10s and bit-rates smaller than 10kbits/s are common.

These technical constraints result in one hand in difficulties for the operator to securely control the remote system and, in the other hand, make classical controls unstable. Many researches have proposed solutions when delays are small or constant (for instance [3]), but when delays go beyond a few seconds and vary a lot as over long distances asynchronous links, solutions not based on teleprogramation [4] are fewer because such delays make master and slave asynchronous and the control unstable.

Work done

We worked on a low-level teleoperation technique when transmissions go through asynchronous computer networks such as local IP networks and the Internet when necessary. This architecture provides constant delays and synchronous data transmission when network delays have jitter. It also features a prediction/estimation function located on the master in order to compensate for delayed feedback. As soon as this low level layer gets into steady state, an higher level teleoperation scheme can be used (Position Error Based, 4 channel, wave transformations, ... ) such as in [3].

This work has been applied on an enhanced mobile manipulator (see [1])

A small piece of bibliography related to this topic

  1. A. LELEVÉ, P. FRAISSE, P. DAUCHEZ & F. PIERROT, "An enhanced mobile manipulator”, Proc. of the 4th World Automation Congress (WAC 2K), Maui, Hawaii, USA, 2000.
  2. R. OBOE, P. FIORINI, "Internet-based telerobotics: problems and approaches", Proc Intl. Conf. on Advanced Robotics (ICAR’97), Monterey (CA - USA), July 1997, pp. 765—770.
  3. G. NIEMEYER & J-J. E. SLOTINE, “Designing Force Reflecting Teleoperators with Large Time Delays to Appear as Virtual Tools”, Proc. of the IEEE Intl. Conf. on Robotics and Automation (ICRA’97), pp. 2212—2218.
  4. P. G. BACKES, K. S. TUO, J. S. NORRIS, G. K. THARP, J. T. SLOSTAD, R. G. BONITZ & K. S. ALI, “Internet-Based Operations for the Mars Polar Lander Mission”, Proc. of the IEEE/ASME Intl. Conf. on Advanced Intelligent Mechatronics, September 1999, pp. 317—322.

Control of a teleoperation system actuated by low-cost pneumatic on/off valves

Taxonomy upgrade extras: 

Networked Robots slides


Slides (in french) from presentation given at 4th ARC meeting of ARC workgroup of GDR Macs on April 1st 2011

AttachmentSize
PDF icon ARC2011-slides.pdf7.08 MB
Image icon robot team coin purse.jpg72.13 KB

Towards Delayed Teleoperation with Pneumatic Master and Slave for MRI

Taxonomy upgrade extras: