• provide an alternative for joystick


The framework has been written and tested using Ubuntu 18.04, since it is the OS used by the LAAS-CNRS robotic platform. It should work seamlessly on a recent Linux version, but there is no guaranteet
The installation on a non-Linux OS has to be handled by the user.

The installation assumes the use of a package manager (e.g. apt) to install some dependencies, as well as the Gazebo simulator. Everything provided in this repository or by the LAAS-CNRS robotic platform aims to be installed locally in the repository folder to avoid polluting the user's system.

In order to use our launcher, it is required to use a USB joystick (such a Microsoft XBox joystick for PC or any equivalent one).

I - Software Overview

I.1. Openrobots

Collections of all the open-source software used at LAAS. You can find more details in Openrobots Wiki-Homepage

I-2. Robotpkg

Robotpkg is a packaging system for installing robotics software developed by the robotic community.
We will use robotpkg to install the required modules for the simulations (state estimation, gazebo interface...) as well as third-party dependencies (qpOases).

I-3. GenoM

GenoM is a generator of modules, designed to be middleware independent, i.e. the same module can be compiled for, e.g., ROS, YARP, or Pocolibs, without any modification.
This allows a great code re-usability and to abstracts the user from any specific choice of a middleware.
Originally GenoM has been developed tightly with Pocolibs, then from version 3, aka GenoM3, ROS templates have been provided.

Another specificity of GenoM is the interaction with and between components.
Each component is started independently like a Linux executable (within a roscore, for ROS, or a h2 intance, for Pocolibs), then the connection between ports (or topics) is made using a supervisor, Genomix, either with Matlab or TCL.

I-4. Pocolibs

Pocolibs is a middleware, like ROS.
It aims at being lighter and faster than ROS, when running on a single machine, thanks to the exploitation of shared memory. ROS, on the other hand, uses a network layer for sending messages between nodes, this leads to greater delays and loss of performance.

I-5. TeleKyb

The TeleKyb software platform provides the aerial-robotic oriented software developed at LAAS-CNRS.
In particular, we will use:
  • pom, a UKF-based state estimator merging state feedback for different sources (e.g. mocap + IMU)
  • optitrack, to export the motion capture data to the genom software stack
  • rotorcraft, the low-level interface, with either the simulated or real platform
  • maneuver, a global trajectory planner, providing position and attitude (as quaternions) as well as first and second derivatives. It implements take-off and waypoint-to-waypoint motions.
  • dynamixel, an interface to control the Dynamixel motors. It is used since the gazebo gripper plugin used for the simulation (presented below) adopts the same interface protocol as the Dynamixel motors (precisely Dynamixel Protocol 2.0).
  • joystick, a component to read the joystick inputs.

I-6. Gazebo

To simulate the platform, we use the Gazebo simulator. To interface it with the genom software stack, we use two dedicated components:
  • mrsim-gazebo a plugin to interface the simulated multi-rotor with the genom components. It uses libmrsim, a Multi-Robot SIMulator interface, designed to be a transparent interface w.r.t. the real aerial vehicles used in LAAS-CNRS. It makes the transition between simulation and experiments transparent, from the software point of view.
  • optitrack-gazebo emulates the optitrack network interface to publish the model poses.

The installation procedure for Gazebo can be found at

I-7. TCL

The interaction with the GenoM components is handled using a scripting language, implementing the communication through the genomix HTTP server.
There are two available language interfaces: matlab and tcl.
eltclsh is an in-terminal TCL shell to interact with the components. However, in the following, we provide a TCL-based software that is all-embedded to avoid the use of the inline interaction through eltclsh.

II - Installation procedure

This section is a tutorial on how to install the software architecture to run the simulations.

II-0. Clone the Visual and Physical Control Architecture for Flying End-Effector repository

Clone the repo associated to this project, using the git daemon. Its root will act as the devel folder for the following.

git git://
cd ./visual-physical-control-architecture/

To simplify the installation, we provide a script that exports all the required variables.
In order to run all the installed executables, we need to set up the path to the newly created folders.
/!\ the source has to be called in the repository root since it uses the pwd command to export the paths.


II-1. Set up robotpkg

(Steps taken from

1. Clone the robotpkg lastest release

git clone git://

2. Check that the openrobots/ folder exists in the repository root, and update the environment variables accordingly if you didn't source the file

export ROBOTPKG_BASE=`pwd`/openrobots

3. Install robotpkg

cd robotpkg/bootstrap
./bootstrap --prefix=$ROBOTPKG_BASE

4. Install the required components and their dependencies

The installation can be done 'manually' by navigating to the desired folder in ./robotpkg/ and install with make update; but we will simplify the process using a set.
To do so, we need to edit the config file: $ROBOTPKG_BASE/etc/robotpkg.conf. Add the following at the end of the file:

PKG_OPTIONS.%-genom3 = \
        codels \
        pocolibs-server \

PKGSET.myset = \
    middleware/pocolibs \
    architecture/genom3 \
    architecture/genom3-pocolibs \
    robots/rotorcraft-genom3 \
    localization/pom-genom3 \
    localization/optitrack-genom3 \
    net/genomix \
    supervision/tcl-genomix \
    shell/eltclsh \
    simulation/mrsim-gazebo \
    simulation/libmrsim \
    simulation/optitrack-gazebo \

PREFER.lapack = robotpkg
PREFIX.matlab = <path/to/Matlab>

The last line needs to point to the Matlab root folder in the system (e.g. /opt/Matlab).
It is recommended to use Matlab for the proposed simulations since the syntax is more intuitive and comprehensible for the user to modify them. However, we also provide all the launch files in tcl, as well as the environment to run them (shell/eltclsh in the above list is a custom tcl script shell).
If Matlab is not installed on the system, remove the lines supervision/matlab-genomix \ and PREFIX.matlab = <path/to/Matlab> from the above list.
Also, all the above is meant for using Pocolibs, not ROS. Futur version of this tutorial might come to use the ROS install.

Now return to the robotpkg folder and install all the set:

cd robotpkg
make update-myset

During the installation, some required dependencies need to be installed with the usual package manager (e.g. apt on Ubuntu). When the install stops, install the required packages and rerun the command above.

II-2. Install custom components

List of the components

The src/ folder contains some additional components, in particular:
  • vision-idl: provides the type declarations regarding the camera modules.
  • camgazebo-genom3: reads the data from the gazebo innate camera, via the gazebo API.
  • camviz-genom3: records and/or displays the images from a camera.
  • arucotag-genom3: detects and filters (EKF-based) the ArUco markers/tags.
  • phynt-genom3: handles the physical interaction (wrench observer and admittance filter).
  • uavatt-genom3: is the attitude controller for fully-actuated UAVs.
  • uavpos-genom3: is the position controller for fully-actuated UAVs.
  • visualservoing-genom3: implements the state machine for the pick-n-place experiment and provides the reference trajectory (either based on visual-servoing, or based on waypoints for takeoff/exploration).
  • dynamixel-genom3: reads and sends data to Dynamixel devices (e.g. motors) that adopt Dynamixel protocols.
  • libdynamixel: provides the type and function declarations used by magdynamixel-gazebo.
  • magdynamixel-gazebo: is a gazebo plugin that emulates a magnetic gripper and adopts the Dynamixel Protocol 2.0.

Install the extra components

Since the extra necessary components are not considered 'stable' as the one provided in robotpkg, we rather install them in a devel folder.
Go to the project root, check that the devel folder exists, export the path if you didn't source the Then go to the sources folder:

export DEVEL_BASE=`pwd`/devel
cd src/

For the manual installation, asciidoctor is needed. It can be installed using apt or any package manager.
Each component here has to be installed manually, using autoconf. To do so, proceed as follow:

cd src/<component>/
mkdir build
cd build
../configure --prefix=$DEVEL_BASE --with-templates=pocolibs/client/c,pocolibs/server
make install
  • The component vision-idl has to be installed first since it defines some type headers used by others.
  • The component libdynamixel must be installed before magdynamixel-gazebo since it defines some variable types and the necessary functions for the other module. To install libdynamixel, and later on magdynamixel-gazebo please follow these instructions:
cd src/libdynamixel
autoreconf -if
./configure --prefix=$DEVEL_BASE
make install
cd ../magdynamixel-gazebo
autoreconf -if
./configure --prefix=$DEVEL_BASE
make install

II-3. Set up the environment

In order to run all the installed executables, we need to set up the path to the newly created folders.
All the required variables are exported in the file, so nothing more has to be done here.
However, remember to source the each time that you want to run simulations from a new terminal.

Remember that the have to be sourced from the repository root!

III - Running the simulation

III-1. GInterface

In order to start all the required software, connect the components together and store the parameters, we use a TCL-based interface.
The folder called ginterface contains all the necessary scripts.
For convenience, we provide as many generic scripts as possible.
The next section explains how to set up the GInterface, then how to use it to run the proposed simulation.
We also provide the "mission" script used in the experiment presented in the paper, so that the reader can have a glance at the parameters used in this experiment.

III-2. Set up the GInterface

1. Install dependencies

Before being able to run GInterface, the following packages might be required to be installed with the usual package manager (e.g. apt on Ubuntu): tcllib, rsync, grsync, rpcbind, python-pandas, python-qt4.

2. Configuration

In order to configure it, it is necessary to modify the content of the file user file. This file is located in the GInterface repository inside the folder users.
For convenience, we provide a generic airpharo_user.tcl file that is pre-filled with environment variables exported from the file.

Remember that the have to be sourced from the repository root!

Nothing needs to be modified in this file the sourced has been done correctly.
For who is interested, the variables listed in the user file are defined as below, and must be modified according to your system setup:
  • pc_name: refers to the name of your machine, when running simulations, otherwise it should be set to the name of the aerial platform's machine when running experiments. Since these instructions will cover only how to run simulations, set this variable to the name of your machine.
  • ground_station_hostname: refers to the name of your machine, when accessing it through the network (e.g. through ssh).
  • path_tcl: is the path to tcl-genomix, where the software in robotpkg has been installed. If you followed these instructions it should be set to <repository_root>/openrobots/lib/tcl-genomix.
  • path_rep: is the path to the ginterface folder of this repository in your machine.
  • path_sup: is the path to the ginterface folder in the aerial platform's machine. This path is not required for running the provided simulation, thus it can be left unchanged.
  • path_log: is the path where the logs are saved during experiments (precisely in the machine specified at pc_name).
  • path_log_sim: is the path where the logs are saved during simulations (precisely in the machine specified at ground_station_hostname).
  • path_launch: is the path to the ginterface/launchers folder of this repository in your machine.
  • path_devel: is the path to the devel folder where the extra necessary components have been installed in your machine. If you followed these instructions it should be set to <repository_root>/devel.
  • path_openrobots: is the path where the software in robotpkg has been installed in your machine. If you followed these instructions it should be set to <repository_root>/openrobots.
  • path_gazebo_world: is the path to the folder ginterface/gazebo/worlds of this repository, where the world files for Gazebo are located. If you followed these instructions it should be set to <repository_root>/gazebo/worlds.

3. About the joystick

As explained previously, a joytick is required to run the simulation, because this interface is primarily used to run real experiments on a remote UAV, hence control is made through a joystick.
If the joystick is actually plugged and no joystick found appears in the terminal during the simulation bootstrap, it means that it is not detected by Linux drivers and probably does not comply with the standards, so cannot be used to play the simulation.

III-3. Run the simulation

First of all, remember to source the file in the terminal to export the paths correctly,and connect a USB joystick to your pc, prior to running the simulation.
Navigate to ginterface folder and run the GInterface.tcl script:

$ cd ./ginterface/
$ ./GInterface.tcl

At this point, the window in the next figure should appear.

Click on File (pointed by the red arrow in the figure above), then on Missions and select sim_fiberthex_airpharo, to run the proposed simulation.
After that, Gazebo should be launched, and right after a XTerm console should appear.
If everything worked correctly, you will have the situation depicted in the next figure.

Here is a picture of the buttons numbers on the joystick. Below, we will refer them by these numbers.


UAV1 is Ready to FLY ...

is displayed in the terminal, press and hold down for 3s the central button (button 8) of your joypad until the propellers start to run the message Armed! appears in the Xterm console.
As soon as the central button of the joypad is pressed the message Arming... should appear in the Xterm console and, while holding it down, a countdown should be initialized. The countdown will be reset if the central button is released before its end. If so, you have to restart the arming sequence (by pressing button 0).
If everything is well configured, the propellers should spin.

When the propeller started spinning and the platform slightly took off, press the button 3 to start the pick-n-place mission. The sequence from this point on will be entirely autonomous.
One can move the bricks and the ground tag in the gazebo workspace to see the searching phase before the brick is detected by the onboard camera.

Updated by Gianluca Corsini over 1 year ago · 64 revisions