Nonlinear Model Predictive Control for Multi-rotor Aerial Vehicles under Visual-perceptive Constraints
1. Description
The objective of this work is to develop a controller that allows an aerial robot to safely and ergonomically hand over an object to a human operator.
The robot is equipped with an onboard sensor, for instance a monocular camera, through which it can perceive the human. The human can move freely in the environment. The goal of the robot is to hand over the tool while taking into consideration the safety and ergonomics of the human operator.
Precisely, a Nonlinear Model Predictive Control (NMPC) algorithm is formulated which accounts for: . Maintaining within the sensor field of view (FoV) the human, in order to guarantee a proper human state estimation. . Avoiding collision between the robot and human, which is of paramount importance in Human-Robot Interaction (HRI). . Maximizing the ergonomics of the human operator during the object transfer phase.
2. Simulations
Simulations are performed withing the Gazebo simulator.
In the following, a screenshot of a simulation is provided, where the robot is in front of the human operator in the process of closing the distance and handing over the object.
On the top right, it is shown what the human sees from his perspective. On the bottom right, instead, the onboard camera video stream is displayed.
As shown there, the human is detected by means AruCo tags and the correct detection is marked with a red dot over the tag. This simple but yet effective method is used as possible implementation of the human detection pipeline. One could imagine that the human detection could take place by means of more sophisticated means and algorithms, for instance based on learning-based methods.
Another snapshot taken during the simulations is provided below. In this other figure, the robot is navigating around the human in order to reach a location in front of his body.
3. Technical details
3.1. Control architecture
In the following, a block diagram of the developed control framework is provided.
The Software collects all the modules related to the control framework, namely:
* a reference generator in charge of providing the desired motion task that the robot has to perform (reach a position in front of the human);
* the NMPC controller which computes the low-level commands for the robot actuators and takes into consideration the motion, visibility, ergonomics and safety tasks;
* the robot and human state estimators which provide an estimation of the state for both agents.
The Hardware group contains the interface to the flight controller mounted on the robot, which receives the motor commands and sends them to the actuators.
Additionally, it groups also the device to detect the human from the onboard sensor (Human Detector), and the interfaces to the other sensing devices providing the measures used in the robot state estimation (onboard IMU and external motion capture system).
4. Related publication
Nonlinear Model Predictive Control for Human-Robot Handover with Application to the Aerial Case
Gianluca Corsini[1], Martin Jacquet[1], Hemjyoti Das[2], Amr Afifi[2], Daniel Sidobre[1], Antonio Franchi[1][2]
G. Corsini, M. Jacquet, H. Das, A. Afifi, D. Sidobre, A. Franchi, "Nonlinear Model Predictive Control for Human-Robot Handover with Application to the Aerial Case", in The 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022), Oct 2022, Kyoto, Japan. doi:10.1109/IROS47612.2022.9981045.
Aerial Robotics; Human-Aerial Robot Interaction; Human-Robot Handover; Nonlinear Model Predictive Control (NMPC)