Teleoperation of a Car-Like Mobile Robot Using MQTT Protocol

(1)

Abstract— This paper presents the design and implementation of a teleoperation system for a real and a simulated car-like mobile robot. It proposes the use of communication protocol MQTT, which uses publisher-subscriber architecture. The teleoperation system has a local station and a remote station. The human operator uses a joystick to command the mobile robot located in the remote station. A human-machine interface and visual feedback are the aids for the human operator to manipulate the car-like. In this case the internet was used as the medium of communication. Finally, the efficiency of the system was analyzed from different remote locations.

Keywords— Teleoperation, car-like mobile robot, MQTT protocol.

I. INTRODUCTION

ELEOPERATIONis the action through which a human operator, who acts in real time, controls the activities of a robot located in a remote location. That is, the human operator closes a high-level control loop with the robot. The most advanced systems provide sensory feedback of the environment, so the operator can make decisions. However, the system is limited in man-robot accuracy and coordination due to its signal processing capability. In addition, the delay in the transmission of the information plays an important role [1].

Teleoperation has presented a high-level development from its beginning the last century in the different applications that exist in several sectors such as industrial, medicine, ground exploration, etc. The demand of this technology nowadays is high [2].

Within the teleoperation, the mobile robots are of extended use. A mobile robot is an automatic machine capable of moving in any given environment. The fundamental of a mobile robot is the displacement in a known or unknown environment, therefore, it is necessary to know the position of the robot in its universe in a precise or relative way according to the applicability and behavior.

As shown in Fig. 1, the teleoperated system consists of a master device and a slave device. The human operator commands the master device from the local station and the slave

device reproduces the actions of the master in the remote station.

Fig. 1. Block diagram of Teleoperation System. [3]

Where h2 is the time delay that information takes to travel from the local station to the remote station, and h₁ is the time delay that the information takes to travel in the opposite direction.

The both delay time depend on the reliability of the communication channel used to transport the data, and on the data traffic that exists in the network. Time delay in teleoperation systems based on network is an important factor in system instability, as it is studied in [4].

The internet of things refers to the interconnection in network of everyday objects endowed with ubiquitous intelligence.

These devices communicate and interact with humans and other devices through integrated systems as described in [5].

To develop this project, it was used the MQTT (Message Queuing Telemetry Transport) protocol which is widely used in the internet of things IoT. MQTT is a broker-based publishing/subscribing instant messaging protocol, is open, lightweight, easy to implement and it doesn’t need a big band weight [6]. The feasibility of implementing the MQTT protocol for teleoperated systems was studied in [7].

MQTT Protocol of IoT for Real Time Bilateral Teleoperation Applied to Car-Like Mobile

Robot

Freddy Panchi*¹, Karla Hernández*², Danilo Chávez*³

*Escuela Politécnica Nacional, Quito, Ecuador, (freddy.panchi, karla.hernandez, danilo.chavez)@epn.edu.ec

T

(2)

II. DESIGNOFTHEGENERALSYSTEM ARCHITECTURE

The proposed teleoperation system consists of a local station, a communication channel and a remote station.

A. Local Station

The human operator uses the joystick to maneuver the remote station. The HMI (MQTT client) shows the information of visual and odometrical feedback of the remote station, so that the human operator can make decisions when maneuvering the remote station using a joystick.

B. Communication Channel

The communication channel uses internet, in this case a computer fulfills the function of data server (MQTT broker).

The function of the server is to manage bidirectional communication between clients in real time.

C. Remote Station

The remote station (MQTT client) is a car-like mobile robot.

This robot is designed in both virtual environment and real environment. The Car-like acquires the data of the data server to fulfill the activities sent by the human operator from the local station. Fig. 2 shows the general architecture of the teleoperation system.

Fig. 2. General Architecture of the Teleoperation System D. Visual Feedback

In the local station the human operator has visual feedback from the remote station. An IP camera is used in the physical car-like, and the software Team Viewer is used in the simulated car-like.

III. CAR-LIKE

The car-like mobile robot is a conventional four-wheel vehicle, two rear wheels for traction and two front wheels for steering. Car-like is based on the Ackerman model. [8]

A. Traction

The robot has a motor coupled to the axis that joins the rear wheels, the drive of this motor provides movement forward or backward to the robot. The motor coupled to the axle must meet the needs of the vehicle, as the enough torque to move the robot and the payload to be loaded on top, as well as the speed necessary to perform the necessary tasks.

B. Ackerman Model

The front wheels are those that provide the direction of the robot for this reason they should not be connected to the same axis since in curving the radius of curvature of the tires are not the same. Also, there are landslides that affect the wear of the tires and it is difficult to make turns, rotate at different speeds.

The Ackerman model is shown in Fig. 3.

Fig. 3. Ackerman Model. [8]

Where:

P is the cut-off point between the projections of the axes of the front wheels and the projection axis of the rear wheels.

b is the distance between rear wheels.

𝜃₀ is the angle of rotation of the right front rim.

𝜃1 is the angle of rotation of the left front rim.

C. Kinematic Model of the Robot

The equations that describe the simplified kinematic model of the mobile robot type car-like are shown below:

𝜃̇ =𝑣(𝑡) tan ∅(𝑡) 𝑙

(1)

𝑥̇ = 𝑣(𝑡) ∗ cos 𝜃(𝑡) (2)

𝑦̇ = 𝑣(𝑡) ∗ sin 𝜃(𝑡) (3)

Where:

𝜃(𝑡) is the orientation of the robot with respect to the x-axis.

∅(𝑡) is the direction of the front wheels.

𝑣(𝑡) is the linear velocity of the robot.

𝑥̇ is the ratio of position change on the x-axis.

𝑦̇ is the ratio of position change on the y-axis

IV. ARCHITECTUREOFCOMMUNICATION The architecture of communication used by MQTT protocol is Publisher/Subscriber, in this architecture any client may have

(3)

the role of publisher or subscriber. The publisher produces several packets of information data called “Topics”. The subscriber is the agent that consumes the “Topics”.[9]

The main characteristic of this communication architecture is how the topics travel from the publisher to the subscriber, the subscriber is not directly addressed to the publisher. The Fig. 4 shows how it works.

Fig. 4. Publisher-Subscriber Architecture

To works over the internet the EPN provides to the broker a public IP address, this is how MQTT clients can connect to the broker everywhere they are located.

V. SIMULATIONUNITY3D

The simulated system was designed in Unity 3D due to it offers a friendly and powerful design environment. The designed car-like is shown in Fig. 5.

Fig. 5. Car-like Model Simulated in Unity 3D

To control the car-like model, it is necessary to include objects called wheel collider on front and back wheels. The wheel collider have properties like rotation to direct the car- like, motor torque to accelerate the car-like, and motor break to stop the car-like. The wheel colliders are controlled from mono developer as necessary. [10]

Other elements of the project are the camera which is programed to follow the car-like, the directional lights that provide natural light to the environment, the terrain that is the base where the car-like moves, and the city where the car-like will navigate.

Unity 3D have libraries for network connection dedicated to MQTT protocol. Using this library is possible to create a MQTT client which is available to publish and subscribe in topics.

The MQTT client of Unity 3D subscribes to the topics where the local station publishes information like the acceleration, the brake, the direction of steering wheel, the sense of the lever forward or backward and an indicator that there is communication.

The Unity 3d MQTT client publish odometrical data of the model of car-like like the x position, y position and the speed to be considered by the local station.

VI. PHYSICALCAR-LIKEMOBILEROBOT The physical car-like implemented is shown in Fig. 6.

Fig. 6. Car-like implemented

To control all the system, it was used an embedded card in this case a Raspberry pi 3. This card is a mini computer that works with Raspbian as Operative system. The Raspberry Pi 3 has an 802.11n Wireless LAN module needed to work on the network, it has 40 GPIO pins necessary to command drivers that in turn command the actuators that has the car-like. [11]

To program the logic part, it was used the Python 2.7 compiler. The python 2.7 compiler is a friendly language which facilitates libraries to be able to work on this project.

The Raspberry pi 3 connects to the internet using its Wi-Fi module, with the help of the libraries paho.mqtt creates a new MQTT client. The MQTT client subscribes to the information for acceleration, direction of the front wheels and direction of forward or backward movement and the indicator of communication. Likewise, the MQTT client of the Raspberry publishes odometrical data about the position in x, the position in y and the speed.

Also, security measures have been adopted in the programming using the information of a proximity sensor prevent the car-like crashes to any obstacle that it encounters.

The hardware elements used to implement the car-like are:

• Driver 298D: is an H-bridge that receives control signals from the Raspberry and acts on micromotor coupled to the rear axle of the car-like to generate traction.

• Servomotor: receives control signals from the Raspberry and moves its axis to direct the front wheels

(4)

• Indicator lights: allow the user to know if there is communication with the control station and to know if the batteries are discharged.

• IP Camera: Serves as visual feedback element for the operator to make decisions, is one of the most important elements in the control loop.

• Batteries: in charge of providing the necessary energy for the robot to operate normally.

VII. TESTANDRESULTS

To evaluate the performance of the protocol working in the internet, it was done tests like Round Trip Time Delay (RTTD), RTTD variance (JITTER) and the lost package rate.

To evaluate the performance of the visual feedback, it was performed a test of time delay.

Also, it was made test of operation of the system to evaluate the performance of the teleoperator and it was taken data like the task execution time and the number of collisions.

To evaluate the teleoperated system it was considered 3 scenes of teleoperación which are the following:

• Ideal test. The operator operating the system directly.

• S1. The EPN and a point in the same network.

• S2. The EPN and a point in the same city (2 km).

• S3. The EPN and a point in another city (70 km).

The MQTT broker is in the EPN, the client 1 (the robot) is in the UAV’s laboratory and the client 2 (operating station) will be in the 3 different proposed scenes.

A. Methodology of test implementation

• Round Trip Time Delay. The test of RTTD consists in publishing a simple data from the operator station to the robot, a time counter starts when the data is published by the operator station, and it is stopped when a message from the robot is received the operator station.

• Lost package rate. From the operator station 150 data are sent to the robot, after all, the number of data sent, and the number of data received by the robot are contrasted and the lost package rate is calculated.

• Visual feedback time delay. Two chronometers are synchronized in the site of operation station and in the site of the robot. The difference between the chronometer from the operation station and the remote station were reflected in the operation station is considered the visual feedback time delay.

• Task execution time. It is the time that takes the operator control the robot from the start to the finish of the circuit.

• Number of collisions. While the operator controls the robot, the number of collision are counted and registered.

• The circuit: The circuit for the real robot is shown in Fig. 7 and the circuit for simulated robot is shown in Fig. 8.

Fig. 7. Circuit for real car-like

The circuit for the real car-like has 5 curves, the width of the road is approximately 1.5 m

Fig. 8. Circuit for simulated car-like

The environment for the simulated car-like is a city, the circuit has 5 curves and the width of the road is approximately 3 times the width of the car-like.

B. Teleoperation for the simulated robot.

Below are shown the contrasted results that were taken experimentally from the teleoperation of the robot in the 3 proposed scenes.

• Round Trip Time Delay and Jitter

In the Fig. 9 is shown the graph of the RTTD in the 3 scenes, it can be noted that the RTTD is not constant, the RTTD is very variable even it has high peaks.

Fig. 9. RTTD in the 3 proposed scenes

In the Table I the values more representative of the RTTD are shown, it can be noted that the average value of the RTTD

(5)

increase 150% from the scene 1 to the scene 2 and increase 460% from scene 1 to the scene 3.

Table I. RTTD representative values RTTD Average

Value (ms)

JITTER Average Value (ms)

RTTD Max (ms)

RTTD Min (ms)

S1 48.45 12 247.85 13.16

S2 72.73 47.37 1500 30.6

S3 226.27 97.1 836.41 110.91

• Lost package rate

In all cases the lost package rate was 0%.

• Visual feedback time delay

The visual feedback is an important parameter that must be considered. The registered values are shown in the figure 10.

In the Table II are shown the most representative values of the visual feedback time delay. These values are much bigger than the values of RTTD. The average value of delay increase to 140% from scene 1 to scene 2 and increase to 170 % from scene 1 to scene 3.

Table II. Visual feedback time delay representative values Average Value (s) Max (s) Min (s)

S1 0.68 1.80 0.219

S2 0.96 3.8 0.49

S3 1.21 2.83 0.32

• Task execution time

In the Table III are shown the results of the time registered for each 5 tests in each scene. It can be noted that the average values increase from the ideal test to each scene.

The time of task execution increase according to the distance in each scene due to the previous parameters like the RTTD and the visual feedback time delay influence in the operability of the system.

Table III. Visual feedback time delay representative values

No. test Ideal Test S1 S2 S3

1 2:57 4:29 7:26 8:09

2 2:40 4:30 7:05 7:46

3 2:37 4:25 7:24 8:16

4 2:31 4:18 7:56 7:01

5 2:47 5:13 7:19 7:30

Average Value

2:42 4:35 7:26 7:44

• Number of collisions

The same as in the previous point, the number of collision increase according to the effect of the physical distance. The number of collision increase due to the time delay of data and visual feedback as it is shown in Table IV.

Table IV. Registry of number of collision for simulated car-like

1 0 0 0 3

2 0 0 0 0

3 0 0 0 1

4 0 0 1 1

5 0 0 0 1

0 0 0.2 1.2

C. Teleoperation for the real robot.

Below are shown the contrasted results that were taken experimentally.

• Round Trip Time Delay and Jitter

In the Fig. 11 is shown the graph of the RTTD in the 3 scenes, it can be noted that the RTTD is not constant, the RTTD is very variable even it has peaks very high.

In the Table V the values more representative of the RTTD are shown, it can be noted that the average value of the RTTD increase 142% from the scene 1 to the scene 2 and increase 205% from scene 1 to the scene 3.

Table V. RTTD representative values RTTD

Average Value (ms)

JITTER Average Value

(ms)

RTTD Max (ms)

RTTD Min (ms)

S1 145.42 95.68 493.87 30.81

S2 207.93 56.27 940.98 141.28

S3 298.52 134.25 1466 124.98

• Lost package rate

In all cases the lost package rate was 0%.

• Visual feedback time delay

The visual feedback is an important parameter that must be considered. The registered values are shown in the Fig. 12.

(6)

In the table VI are shown the most representative values of the visual feedback time delay. These values are much bigger than the values of RTTD. The average value of delay increase to 102% from scene 1 to scene 2 and increase to 200 % from scene 1 to scene 3.

Table VI. Visual feedback time delay representative values for Team viewer Average Value (s) Max (s) Min (s)

S1 0.799 2.81 0.586

S2 0.817 1.149 0.303

S3 1.67 2.80 1.209

• Task execution time

In the table VII are shown the results of the time registered for each 5 tests in each scene. It can be noted that the average values increase from the ideal test to each scene.

The time of task execution increase according to the distance in each scene due to the previous parameters like the RTTD and the visual feedback time delay influence in the operability of the system.

Table VII. Visual feedback time delay representative values for IP CAM

1 0:26 0:55 2:36 2:35

2 0:31 0:57 2:24 2:00

3 0:32 0:55 2:35 1:41

4 0:25 0:54 2:02 2:15

5 0:28 0:52 2:40 1:34

0:28 0:55 2:27 2:01

• Number of collisions

Like in the previous point, the number of collision increase according to the effect of the physical distance. The number of collision increase because the time delay of data and visual feedback as shown in Table VIII

Table VIII. Registry of number of collision for physical car-like

1 0 0 2 1

2 0 0 1 0

3 0 0 0 0

4 0 0 0 1

5 0 0 1 1

0 0 0.8 0.6

It may be noted that the delays increase according to the distance, but the time delay is not too bigger.

In each case exist high peaks of delays because of the characteristics of the Internet but in general the delays are held in low values due to the communication protocol is light and short.

VIII. C^ONCLUSIONS

The physical distance influences the teleoperated system, increasing the delay time, it causes that the operator screws up the control over the system. Nevertheless, the system was controllable enough, and when the operator increases his experience, he could control the system in a better way.

The operation of the MQTT communication protocol has demonstrated, experimentally, that it is quite efficient in Internet networks, proving that a constant sending of data between the local station and the remote station is possible without loss of data packets.

IoT protocols like MQTT become very useful because they work asynchronously in real time and ensure their functioning in hostile communication environments.

It is possible to implement teleoperated systems using a not dedicated communication channel like the internet, the implemented system is monitorable and controllable and his characteristics are overshadowed by the effect of the distance over the communication channel.

R^EFERENCES

[1] J. Vertut, Teleoperation and Robotics: Applications and Technology.

Springer Science & Business Media, 2013.

[2] J. M. Bogado Torres, “Control bilateral de robots teleoperados por convergencia de estados”, phd, E.T.S.I. Industriales (UPM), 2007.

[3] E. Slawiñski, V. Mut, L. Salinas, y S. García, “Teleoperation of a mobile robot with time-varying delay and force feedback”, Robotica, vol. 30, núm. 01, pp. 67–77, ene. 2012.

[4] C. C. Hua y X. P. Liu, “Delay-Dependent Stability Criteria of Teleoperation Systems With Asymmetric Time-Varying Delays”, IEEE Trans. Robot., vol. 26, núm. 5, pp. 925–932, oct. 2010.

[5] F. Xia, L. T. Yang, L. Wang, y A. Vinel, “Internet of Things”, Int. J.

Commun. Syst., vol. 25, núm. 9, pp. 1101–1102, sep. 2012.

[6] K. Tang, Y. Wang, H. Liu, Y. Sheng, X. Wang, y Z. Wei, “Design and Implementation of Push Notification System Based on the MQTT Protocol”, 2013.

[7] B. Vallejo y L. Francisco, “Estudio e implementación en Matlab de un entorno de comunicación basado en protocolos del Internet de las Cosas para clientes de Teleoperación en Robótica”, jul. 2017.

[8] A. O. Baturone, Robótica: manipuladores y robots móviles. Marcombo, 2005.

[9] Mosquitto, “An Open Source MQTT v3.1 Broker”.

[10] unity3d.com, “Unity - Editor”. [En línea]. Disponible en:

https://unity3d.com/es/unity/editor. [Consultado: 18-may-2017].

[11] RASPBERRY PI FOUNDATION, “Raspberry Pi 3 Model B”, Raspberry Pi.