The angle φ*k* can be obtained such as Eq. (8).

$$\begin{aligned} \phi\_k &= \alpha\_k - \beta\_k\\ \alpha\_k &= \operatorname{atan2}(y\_{L(k)}, x\_{L(k)})\\ \beta\_k &= \operatorname{acos}(\frac{d\_k}{\sqrt{\mathbf{x}^2\_{L(k)} + \mathbf{y}^2\_{L(k)}}})\\ \phi\_k &= \operatorname{atan2}(y\_{L(k)}, x\_{L(k)}) - \operatorname{acos}(\frac{d\_k}{\sqrt{\mathbf{x}^2\_{L(k)} + \mathbf{y}^2\_{L(k)}}}) \end{aligned} \tag{8}$$

The proposed curb detection algorithm is experimented to identify the performance of it. The mobile robot was driven along a road with the distance (1m) between the mobile robot and curbs. The tilted angle (φ *L* in Fig. 4) between the mobile robot and the laser range finder is 25°. The values *d* and σ as 50mm and π / 4 are selected in Eq. (4.3). The navigation server sends the data of curb edge points (position data of stars in Fig. 14) to the remote server. The remote server receives the image and curbs edges data from the navigation server and displays the map of curbs and navigation information as shown in Fig. 16. The rectangular and the dots in the map represent the mobile robot and curb edge points, respectively.

Tracking Control for Reliable Outdoor Navigation Using Curb Detection 337

The left and right-side stars and the black triangles in Fig. 15(b), 17 show the detected curbs and the robot path, respectively. The left-side stars in Figures are irregular because the rightside curb has regular shape (height: 13cm), but the left-side of the road is not the curb. It is just the structure with low height (2cm). Even though the mobile robot moves zigzag, the

detected right-side curb path is smooth. The average error of the curb points is 2cm.

Fig. 16. Screen of the remote server.

Fig. 17. Robot path and extracted curb positions.

Fig. 14. Extracted curb edge points on the road.

Fig. 16. Screen of the remote server.

336 Recent Advances in Mobile Robotics

(a) (b)

(a) The mobile robot tracking the curbs (b) Robot path and detected curbs.

Fig. 14. Extracted curb edge points on the road.

Fig. 15. Curb detection experiment.

The left and right-side stars and the black triangles in Fig. 15(b), 17 show the detected curbs and the robot path, respectively. The left-side stars in Figures are irregular because the rightside curb has regular shape (height: 13cm), but the left-side of the road is not the curb. It is just the structure with low height (2cm). Even though the mobile robot moves zigzag, the detected right-side curb path is smooth. The average error of the curb points is 2cm.

Fig. 17. Robot path and extracted curb positions.

Tracking Control for Reliable Outdoor Navigation Using Curb Detection 339

(c)

An ATRV-mini is a nonholonomic mobile robot, equivalent to a four-wheeled differential drive. Kanayama proposed a kinematic control law in Cartesian coordinates for solving the trajectory tracking problems [10]. We add a term related to the distance between the robot

Yr

*r*

between robot and curb.

Xr

XG

φ

Fig. 18. Results for curb detection. (c) Angle

**4. Tracking control using curb position**

and the right-side curb of roads to Kanayama controller.

YG

*r y*

*c y*

Fig. 19. Mobile robot and reference robot.

θ*c*

θ*r*

*y e x e*

*c*

c *x r* O *x*

Xc Yc

*e*θ

Fig. 18 shows the distance between robot and curb, the curb edge angle α and angle φ between robot and curb, respectively. We can know that the proposed curb detection algorithm produces proper results and the distance and angle information calculated from the curb position could be used for reliable and safe navigation.

Fig 18. Results for curb detection. (a) Distance *d* between robot and curb (b) Curb edge angle α

338 Recent Advances in Mobile Robotics

between robot and curb, respectively. We can know that the proposed curb detection algorithm produces proper results and the distance and angle information calculated from

(a)

(b)

Fig 18. Results for curb detection. (a) Distance *d* between robot and curb (b) Curb edge

angle α α

and angle

φ

Fig. 18 shows the distance between robot and curb, the curb edge angle

the curb position could be used for reliable and safe navigation.

Fig. 18. Results for curb detection. (c) Angle φbetween robot and curb.

#### **4. Tracking control using curb position**

An ATRV-mini is a nonholonomic mobile robot, equivalent to a four-wheeled differential drive. Kanayama proposed a kinematic control law in Cartesian coordinates for solving the trajectory tracking problems [10]. We add a term related to the distance between the robot and the right-side curb of roads to Kanayama controller.

Fig. 19. Mobile robot and reference robot.

Tracking Control for Reliable Outdoor Navigation Using Curb Detection 341

We have performed a straight path tracking experiment, which target point is (78m, -22m) and linear velocity is 0.5m/sec. The gains of controller were determined by some simulations and experiments. Fig. 20 shows the reference path and the error of path, position and angle of the robot. Fig. 20(a) shows the reference path and tracking path and the position error of 20cm at the final point, respectively. Even though DGPS position error of 1m has occurred at the point of 12m from X-axis, the mobile robot returned to its

Reference and Tracking path

reference path tracking path

(a)

Commanded linear velocity

0 20 40 60 80 100 120 140 160 -0.1

Time [sec]

(b)

**4.1 Straight path tracking**

reference path by the stable tracking controller.

5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

linear velocity [m/sec]

Fig. 20. (Continued)

10

15

20

25

30

Considering the Fig. 19, in which the mobile robot *c* is represented by the coordinates [ ]*T c c* **p** = *x y* θ of the center of mass, moving over plane with linear velocity υ and angular velocity ω . The matrix equation that describes the kinematic model of a mobile robot is given by

$$
\begin{bmatrix}
\dot{\mathbf{x}}\_c \\
\dot{y}\_c \\
\dot{\theta}
\end{bmatrix} = \begin{bmatrix}
\cos\theta & -\sin\theta \\
\sin\theta & \cos\theta \\
0 & 1
\end{bmatrix} \begin{bmatrix}
\nu \\
\nu \\
\theta
\end{bmatrix} \tag{9}
$$

The posture tracking error vector *pe* is calculated in the reference robot frame , *X Yr r* .

$$\begin{aligned} e\_p &= R(\theta)(p\_r - p) \\ e\_p &= \begin{bmatrix} e\_x \\ e\_y \\ e\_\theta \end{bmatrix} = \begin{bmatrix} \cos\theta & \sin\theta & 0 \\ -\sin\theta & \cos\theta & 0 \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} x\_r - x\_c \\ y\_r - y\_c \\ \theta\_r - \theta\_c \end{bmatrix} \end{aligned} \tag{10}$$

where *R*( ) θ is the rotation matrix. Differentiating the Eq. (10), yields

$$\begin{bmatrix} \dot{e}\_x\\ \dot{e}\_y\\ \dot{e}\_\theta \end{bmatrix} = \begin{bmatrix} \alpha e\_y - \nu + \nu\_r \cos(e\_\theta) \\ -\alpha e\_x + \nu\_r \sin(e\_\theta) \\ \alpha e\_r - \alpha \end{bmatrix} \tag{11}$$

The asymptotic convergence of posture error can be obtained

$$V\_c = \begin{bmatrix} \nu\_c \\ \alpha\_c \end{bmatrix} = \begin{bmatrix} k\_x \ e\_x + \nu\_r \cos(e\_\theta) \\ \alpha\_r + k\_y \ \nu\_r \ e\_y + k\_\theta \ \nu\_r \sin(e\_\theta) \end{bmatrix}' \tag{12}$$

where , *x y k k* and *k*θare positive constants, defined as controller gains.

For reliable and safe navigation, we design a new controller *Vd* by adding the extracted curb distance and angle information to the above Kanayama controller.

$$\begin{aligned} e\_d &= d\_{ref} - d\\ V\_d &= \begin{bmatrix} \nu\_d\\ \alpha\_d \end{bmatrix} = \begin{bmatrix} k\_x \ e\_x + \nu\_r \cos(\varepsilon\_\theta) \\\ \alpha\_r + \nu\_r (k\_y \ e\_y + k\_\theta \sin(\varepsilon\_\theta) + k\_d \ e\_d + k\_\phi \sin\phi) \end{bmatrix}' \end{aligned} \tag{13}$$

where *ref d* is the reference distance from the curb for safety and , *d k k*φ are positive constants as controller gains.

If the robot is too close to the curb, the added terms of the proposed controller try to prohibit the robot colliding with curbs, and vice versa.

We have performed two kinds of experiments to evaluate the performance of the proposed path tracking controller in outdoor environment. One is that the mobile robot has to follow a straight path. The other is that the mobile robot has to follow a curved path.

#### **4.1 Straight path tracking**

340 Recent Advances in Mobile Robotics

Considering the Fig. 19, in which the mobile robot *c* is represented by the coordinates

cos sin sin cos 0 1

θ

θ

The posture tracking error vector *pe* is calculated in the reference robot frame , *X Yr r* .

θ

θ

*y r x y xr*

ω

*c xx r*

ω

*r*

*c r yry r*

 are positive constants, defined as controller gains. For reliable and safe navigation, we design a new controller *Vd* by adding the extracted

⎡ ⎤ ⎡ + ⎤ = = ⎢ ⎥ ⎢ ⎥ + + ⎣ ⎦ ⎢⎣ ⎥⎦

cos( )

⎡ ⎤ ⎡ + ⎤ = = ⎢ ⎥ ⎢ ⎥ + + ++ ⎣ ⎦ ⎢⎣ ⎥⎦

If the robot is too close to the curb, the added terms of the proposed controller try to prohibit

We have performed two kinds of experiments to evaluate the performance of the proposed path tracking controller in outdoor environment. One is that the mobile robot has to follow a

θ

θθ

*d r r yy d d*

*ke e V ke k e ke k*

υ

*V k ek e*

ωυ

=− + ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ − ⎣ ⎦ ⎣ ⎦

ω ω

*e e e ee e*

 υυ

⎡ ⎤ ⎡ − + ⎤ ⎢ ⎥ ⎢ ⎥

 υ

⎡ ⎤ ⎡ ⎤ − ⎢ ⎥ ⎢ ⎥⎡ ⎤ ⎢ ⎥ = ⎢ ⎥⎢ ⎥ ⎣ ⎦ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ⎣ ⎦

cos sin 0 sin cos 0 001

*x r c p y r c*

 θ

 θ

*e x x e e y y*

⎡ ⎤ ⎡ ⎤⎡ ⎤ − ⎢ ⎥ ⎢ ⎥⎢ ⎥ = ⎢ ⎥ = − − ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ − ⎣ ⎦ ⎣ ⎦⎣ ⎦

. The matrix equation that describes the kinematic model of a mobile robot is

 θ

 θ υ

ω

*r c*

θ

θ

cos( ) sin( )

θ

cos( )

θ

 υ

( sin( ) sin )

θ

*ke e*

υ

sin( )

 θ

> φ

 φ

θ

υ

, (10)

, (12)

, (13)

are positive

φ

and angular

(9)

(11)

of the center of mass, moving over plane with linear velocity

*c c x y*

θ

( )( )

*p r*

*e*θ

*e*

υ

ω

curb distance and angle information to the above Kanayama controller.

*d xx r*

where *ref d* is the reference distance from the curb for safety and , *d k k*

straight path. The other is that the mobile robot has to follow a curved path.

ω υ

The asymptotic convergence of posture error can be obtained

*c*

θ

is the rotation matrix.

Differentiating the Eq. (10), yields

θ

*d ref*

*ed d*

= −

υ

ω

*d*

the robot colliding with curbs, and vice versa.

constants as controller gains.

*e R pp*

= −

θ

[ ]*T c c* **p** = *x y*

velocity

given by

where *R*( ) θ

where , *x y k k* and *k*

θ

ω

We have performed a straight path tracking experiment, which target point is (78m, -22m) and linear velocity is 0.5m/sec. The gains of controller were determined by some simulations and experiments. Fig. 20 shows the reference path and the error of path, position and angle of the robot. Fig. 20(a) shows the reference path and tracking path and the position error of 20cm at the final point, respectively. Even though DGPS position error of 1m has occurred at the point of 12m from X-axis, the mobile robot returned to its reference path by the stable tracking controller.

Fig. 20. (Continued)

Tracking Control for Reliable Outdoor Navigation Using Curb Detection 343

angle error

0 20 40 60 80 100 120 140 160 -14

Time [sec]

(f) Fig. 20. Results for straight path tracking. (a) Reference path and tracking path (b) Controller linear velocity (c) Controller angular velocity (d) X-axis position error for robot coordinate

We have performed a curved path tracking experiment, which linear velocity is 0.3m/sec and angular velocity is 0.06rad/s. Fig. 21 shows the reference path and the error of path, position and angle of the robot. Fig. 21(a) shows the reference path and tracking path. The X-axis position error and the angular error are 40cm and 7° at the final point, respectively. The reason that the errors are bigger than the straight path tracking experiment is related to the characteristic of skid steering system of our mobile robot platform. It is necessary to

Reference and Traking path

reference path tracking path

X [m]

(a)

(e) Y-axis position error for robot coordinate (f) Angle error for robot coordinate.

optimize the gains of controller to reduce the errors.

Fig. 21. (Continued)

0

2

4

Y [m]

6

8

10

**4.2 Curved path tracking**

[deg]

Fig. 20. (Continued)

Fig. 20. Results for straight path tracking. (a) Reference path and tracking path (b) Controller linear velocity (c) Controller angular velocity (d) X-axis position error for robot coordinate (e) Y-axis position error for robot coordinate (f) Angle error for robot coordinate.

#### **4.2 Curved path tracking**

342 Recent Advances in Mobile Robotics

Commanded angular velocity

0 20 40 60 80 100 120 140 160 -10

Time [sec]

(c)

X-axis position error

0 20 40 60 80 100 120 140 160 -1.2

(d)

Y-axis position error

0 20 40 60 80 100 120 140 160 -0.4

(e)

Time [sec]

Time [sec]

0 0.2

Fig. 20. (Continued)

[m]

[m]

0

5

angular velocity [deg/sec]

10

15

20

We have performed a curved path tracking experiment, which linear velocity is 0.3m/sec and angular velocity is 0.06rad/s. Fig. 21 shows the reference path and the error of path, position and angle of the robot. Fig. 21(a) shows the reference path and tracking path. The X-axis position error and the angular error are 40cm and 7° at the final point, respectively. The reason that the errors are bigger than the straight path tracking experiment is related to the characteristic of skid steering system of our mobile robot platform. It is necessary to optimize the gains of controller to reduce the errors.

Fig. 21. (Continued)

Tracking Control for Reliable Outdoor Navigation Using Curb Detection 345

Y-axis position error

0 5 10 15 20 25 30 35 40 45 -0.14

Time [sec]

(e)

angle error

0 5 10 15 20 25 30 35 40 45 -6

Time [sec]

(f)

In this chapter, an algorithm is described to detect road curb position through a laser range finder. The curbs map was built and the width of the road was calculated by the detected curbs. The proposed algorithm is to calculate the distance and the angle between the mobile robot and the curb. A new controller is designed by adding the extracted curb distance and angle information to the tracking controller. It helps the mobile robot to navigate safely and reliably. To identify the availability of the proposed algorithm, navigation experiments was

performed in outdoor environment and proper results was obtained.

(a) Reference path and tracking path (b) Controller linear velocity (c) Controller angular velocity (d) X-axis position error for robot coordinate (e) Y-axis position error for robot

> -4 -2 0 2 4

Fig. 21. Results for curved path tracking.

**5. Conclusion**

coordinate (f) Angle error for robot coordinate

[deg]

6 8

[m]

Fig. 21. (Continued)

Fig. 21. Results for curved path tracking.

(a) Reference path and tracking path (b) Controller linear velocity (c) Controller angular velocity (d) X-axis position error for robot coordinate (e) Y-axis position error for robot coordinate (f) Angle error for robot coordinate

#### **5. Conclusion**

344 Recent Advances in Mobile Robotics

Commanded linear velocity

0 5 10 15 20 25 30 35 40 45 0

Time [sec]

(b)

Commanded angular velocity

0 5 10 15 20 25 30 35 40 45 -4

Time [sec]

(c)

X-axis position error

0 5 10 15 20 25 30 35 40 45 -0.5

Time [sec]

(d)

0.05 0.1 0.15

Fig. 21. (Continued)

[m]

angular velocity [deg/sec]

0.2 0.25 0.3 0.35

linear velocity [m/sec]

In this chapter, an algorithm is described to detect road curb position through a laser range finder. The curbs map was built and the width of the road was calculated by the detected curbs. The proposed algorithm is to calculate the distance and the angle between the mobile robot and the curb. A new controller is designed by adding the extracted curb distance and angle information to the tracking controller. It helps the mobile robot to navigate safely and reliably. To identify the availability of the proposed algorithm, navigation experiments was performed in outdoor environment and proper results was obtained.

**Part 4**

**Methods for Control**

#### **6. References**

## **Part 4**

**Methods for Control**

346 Recent Advances in Mobile Robotics

[1] S. Thrun at el. (2000). Probabilistic Algorithms and the Interactive Museum Tour-Guide Robot Minerva, *International Journal of Robotics Research*, Vol.19, No.11, pp. 972-999 [2] Richard Thrapp at el. (2001). Robust localization algorithms for an autonomous campus tour guide, *Proc. of IEEE Int'l Conf. on Robotics and Automation*, pp. 2065-2071 [3] C.C. Wang and C. Thorpe, "Simultaneous Localization and Mapping with Detection and

[4] W. S. Wijesoma, K. R. S. Kodagoda, and Arjuna P. Balasuriya, "Road-Boundary

[5] Seung-Hun Kim at el. (2006). A hybrid autonomous/teleoperated strategy for reliable mobile robot outdoor navigation, *SICE-ICASE Int. Joint Conf.*, pp. 3120-3125 [6] K. Ohno at el. (2003). Outdoor Navigation of a Mobile Robot between Buildings based on

[7] J. Forsberg at el. (March 1995). Mobile robot navigation using the rangeweighted Hough

[8] R. O. Duda. & P. E. Hart. (1972). Use of the Hough transformation to detect lines and

[9] P. V. C. Hough. (December 1962). Methods and means for recognizing complex patterns,

[10] Y. Kanayama at el. (1990). A stable tracking control method for an autonomous mobile

transform, *IEEE Robotics & Automation Magazine*, vol. 2, pp. 18-26

curves in pictures, *Communications of the ACM*, vol. 15, pp. 11-15

robot, *IEEE Int. Conf. on Robotics and Automation*, vol. 1, pp. 384-389

Tracking of Moving Objects", Proc. of IEEE Int'l Conf. on Robotics and

Detection and Tracking Using Ladar Sensing", IEEE Trans. On Robotics and

DGPS and Odometry Data Fusion, *Proc. of IEEE Int'l Conf. on Robotics and*

**6. References**

Automation, 2002, pp. 2918-2924

Automation, Vol.20, No.3, 2004

*Automation*, pp. 1978-1984

in U.S. Patent 3 069 654

**0**

**17**

*Poland*

Krzysztof Okarma and Piotr Lech

*West Pomeranian University of Technology, Szczecin*

**Statistical Video Based Control of Mobile Robots**

A typical approach to the control of mobile robots is based on the analysis of signals from various kinds of sensors (e.g. infra-red or ultrasound). Another type of input data used for motion control of robots can be video signals acquired by the cameras. In such case many algorithms can be utilised e.g. SLAM (Simultaneous Localization and Mapping) and its modifications, being still an active field of research (Kalyan et al., 2010), which require usually the full analysis of the images (or video frames) being the input data (Se et al., 2001; 2002), in some applications using also additional operations such as Principal Component Analysis (Tamimi & Zell, 2004). Similar problems may also occur in the robot localisation and

An interesting alternative can be the application of the fast image analysis techniques based on the statistical experiments. Presented statistical approach is related to the first step of such control systems, which can be considered as the pre-processing technique, reducing the amount of data for further analysis with similar control accuracy. Discussed methods can be utilised for many control algorithms based on the image and video data e.g. the proportional steering of line following robots (Cowan et al., 2003) considered as a representative example in further discussions. Nevertheless, some other applications of the presented approach are also possible e.g. fast object classification based on the shape analysis (Okarma & Lech, 2009) or motion detection, as well as fast reduced-reference image quality estimation (Okarma &

Considering the autonomous mobile robots, one of their most important parameters, corresponding directly to their working properties, is the power consumption related to the maximum possible working time. Limited energy resources are often the main element reducing the practical applicability of many such constructions regardless of many modern energy sources which can be used such as e.g. solar based solutions. The limited capacity of the installed batteries reduces the range and capabilities of these devices and each of the possible attempts to optimise energy consumption usually affects the mobility and efficiency

It is possible to reduce the consumed energy by finding the shortest path to the target (robot's drive) or conducting the optimisation of the amount of processed data by the control algorithm. Since the computational cost of mapping algorithms, searching for the optimal path, finding robot's own location, visual SLAM algorithms etc., based on the image analysis is strongly dependent on the resolution and representation of the analysed image, a significant "smart" decrease of the amount of processed data seems to be an interesting direction of

**1. Introduction**

Lech, 2008b).

of the robots.

map building processes (Hähnel et al., 2003).

**2. Reduction of power consumption by mobile robots**

### **Statistical Video Based Control of Mobile Robots**

Krzysztof Okarma and Piotr Lech

*West Pomeranian University of Technology, Szczecin Poland*

#### **1. Introduction**

A typical approach to the control of mobile robots is based on the analysis of signals from various kinds of sensors (e.g. infra-red or ultrasound). Another type of input data used for motion control of robots can be video signals acquired by the cameras. In such case many algorithms can be utilised e.g. SLAM (Simultaneous Localization and Mapping) and its modifications, being still an active field of research (Kalyan et al., 2010), which require usually the full analysis of the images (or video frames) being the input data (Se et al., 2001; 2002), in some applications using also additional operations such as Principal Component Analysis (Tamimi & Zell, 2004). Similar problems may also occur in the robot localisation and map building processes (Hähnel et al., 2003).

An interesting alternative can be the application of the fast image analysis techniques based on the statistical experiments. Presented statistical approach is related to the first step of such control systems, which can be considered as the pre-processing technique, reducing the amount of data for further analysis with similar control accuracy. Discussed methods can be utilised for many control algorithms based on the image and video data e.g. the proportional steering of line following robots (Cowan et al., 2003) considered as a representative example in further discussions. Nevertheless, some other applications of the presented approach are also possible e.g. fast object classification based on the shape analysis (Okarma & Lech, 2009) or motion detection, as well as fast reduced-reference image quality estimation (Okarma & Lech, 2008b).

#### **2. Reduction of power consumption by mobile robots**

Considering the autonomous mobile robots, one of their most important parameters, corresponding directly to their working properties, is the power consumption related to the maximum possible working time. Limited energy resources are often the main element reducing the practical applicability of many such constructions regardless of many modern energy sources which can be used such as e.g. solar based solutions. The limited capacity of the installed batteries reduces the range and capabilities of these devices and each of the possible attempts to optimise energy consumption usually affects the mobility and efficiency of the robots.

It is possible to reduce the consumed energy by finding the shortest path to the target (robot's drive) or conducting the optimisation of the amount of processed data by the control algorithm. Since the computational cost of mapping algorithms, searching for the optimal path, finding robot's own location, visual SLAM algorithms etc., based on the image analysis is strongly dependent on the resolution and representation of the analysed image, a significant "smart" decrease of the amount of processed data seems to be an interesting direction of

 -

-
- -

 ---

!" #

&"# ' ((

> )-

dependent on the frame rate of the analysed video sequence

dependent on the image resolution

\$-

-

- -

Fig. 2. Illustration of the general vision based control algorithm of the mobile robot

 × 480 2.15 5.022 × 288 2.39 4.519 × 240 2.69 4.268 × 144 2.80 4.168 × 120 2.95 4.369 Table 1. Path length and power consumption of the mobile robot used in the experiments

Resolution in pixels Length of the path [m] Total energy consumption [kJ]

Frame rate [fps] Length of the path [m] Total energy consumption [kJ] 2.39 4.519 2.44 4.432 2.66 4.332 3.23 4.668 3.33 4.798 Table 2. Path length and power consumption of the mobile robot used in the experiments

Statistical Video Based Control of Mobile Robots 351

 -

research in this area. Reducing the computational load of the processor the overall demand for the electrical energy decreases, so it can be assumed that limiting the amount of processed data in the system, energy savings can be obtained, despite using a non-optimal (longer) motion path. For the video based control algorithms such restriction can be achieved simply by reducing the image resolution or reducing the frame rate of the acquired image sequence used for he robot's motion control. Nevertheless, such decrease of the amount of analysed data may lead to some steering errors, as illustrated in Fig. 1, where the optimal shortest path cannot be used due to the presence of some obstacles. Each of the indicated alternative paths has a different length related to the different energy necessary for the robot to reach the target point.

Fig. 1. Illustration of the problem of the optimal path searching in the presence of obstacles

The optimal robot's path in the known environment can be chosen using the machine vision approach with motion detection and robot's motion parameters estimation, using the general algorithm presented in Fig. 2 such that the total energy consumption, related both to the robot's drive and the control part dependent mainly on the processing of the vision data, is minimum. Conducting the experimental verification of the idea for the mobile robot based on a simple netbook platform using the joystick port for controlling the drives, for the testing area of 6 square meters illustrated in Fig. 1, the maximum power consumption is about 5.2 A (42.5% by the drive part and 57.5% by the control part of the system). The length of the optimum straight path, assuming no obstacles on the scene, is equal to 2 meters so all the alternative trajectories are longer.

The experiments conducted in the environment described above has been related to the decrease of the resolution of the image (obtained results are presented in Table 1) and the frame rate for the fixed resolution of 320 × 240 pixels (results shown in Table 2). For the reduction to 8 frames per second the vision based control algorithm could not find any appropriate path.

2 Will-be-set-by-IN-TECH

research in this area. Reducing the computational load of the processor the overall demand for the electrical energy decreases, so it can be assumed that limiting the amount of processed data in the system, energy savings can be obtained, despite using a non-optimal (longer) motion path. For the video based control algorithms such restriction can be achieved simply by reducing the image resolution or reducing the frame rate of the acquired image sequence used for he robot's motion control. Nevertheless, such decrease of the amount of analysed data may lead to some steering errors, as illustrated in Fig. 1, where the optimal shortest path cannot be used due to the presence of some obstacles. Each of the indicated alternative paths has a different length related to the different energy necessary for the robot to reach the target

> --

Fig. 1. Illustration of the problem of the optimal path searching in the presence of obstacles The optimal robot's path in the known environment can be chosen using the machine vision approach with motion detection and robot's motion parameters estimation, using the general algorithm presented in Fig. 2 such that the total energy consumption, related both to the robot's drive and the control part dependent mainly on the processing of the vision data, is minimum. Conducting the experimental verification of the idea for the mobile robot based on a simple netbook platform using the joystick port for controlling the drives, for the testing area of 6 square meters illustrated in Fig. 1, the maximum power consumption is about 5.2 A (42.5% by the drive part and 57.5% by the control part of the system). The length of the optimum straight path, assuming no obstacles on the scene, is equal to 2 meters so all the

The experiments conducted in the environment described above has been related to the decrease of the resolution of the image (obtained results are presented in Table 1) and the frame rate for the fixed resolution of 320 × 240 pixels (results shown in Table 2). For the reduction to 8 frames per second the vision based control algorithm could not find any

point.

alternative trajectories are longer.

appropriate path.

Fig. 2. Illustration of the general vision based control algorithm of the mobile robot

Table 1. Path length and power consumption of the mobile robot used in the experiments dependent on the image resolution

Table 2. Path length and power consumption of the mobile robot used in the experiments dependent on the frame rate of the analysed video sequence

For the infinite number of samples the reduction of the sampling distance takes place and the probability of choosing the point representing the object on the image can be expressed as:

Statistical Video Based Control of Mobile Robots 353

*KN N* = *AOb ASc*

*AOb* ≈ *KN* · *ASc*

Since the total area of the scene is equal to *ASc* = *N* · *kx* · *ky*, the estimated object's area is equal to *AOb* ≈ *KN* · *kx* · *ky*, where *kx* and *ky* are the scale factors for horizontal and vertical

Considering the above analysis, the probability of choosing the point belonging to the object can be used in the proposed algorithm with a reduced number of analysed samples instead of the full image analysis using a statistical experiment based on the Monte Carlo method. This method originates directly from the law of large numbers, because the sequence of successive approximations of the estimated value is convergent to the sought solution and the distance of the actual value after performing the specified number of statistical tests to the solution can

After the binarization the luminance (or colour) samples represented as "ones" or "zeros" corresponding to the allowed values of the specified logical condition, which are stored in one-dimensional vector are chosen randomly. For a single draw a random variable *Xi* of the

1 for black samples

An important element of the method is the proper choice of the logical condition for the binarization allowing a proper separation of the object from the background. Such choice depends on the specific application, for example for the light based vehicles' tracking purposes the chroma keying based on the CIELAB colour model can be appropriate (Mazurek

> *n* ∑ *i*=1

According to the Lindberg-Levy's theorem, the distribution of *Yn* tends to the normal distribution *N*(*my*, *σy*) if *n* → ∞. Since the expected value and variance of *Yn* are equal

*Yn* = 1 *n* · (2)

*N* (3)

0 for white samples, (4)

*Xi* (6)

*my* = *p* (7)

*P*(*Xi* = 1) = *p P*(*Xi* = 0) = *q* (5)

*n* respectively, the distrubution of the random value *Yn* is normal

*p* = lim *N*→∞

coordinates respectively, equivalent to the number of samples per unit.

*Xi* =

so the estimated area of the object is:

be determined using the central limit theorem.

leading to the following probability expressions:

For *n* independent draws the variable *Yn* is obtained:

where *p* + *q* = 1, *E*(*Xi*) = *p*, *V*(*Xi*) = *p* · *q*.

two-way distribution is obtained:

& Okarma, 2006).

to *E*(*Yn*) = *p* and *V*(*Yn*) = *p*·*q*

with the following parameters:

Analysing the results presented in Tables 1 and 2 a significant influence of the amount of the analysed video data on the obtained results and the power consumption can be noticed. The decrease of the resolution as well as the reduction of the video frame rate causes the increase of the path's length but due to the reduction of the processor's computational load the total energy consumption may decrease. Nevertheless, the decrease of the amount of the processed data should be balanced with the control correctness in order to prevent an excessive extension of the robot's path.

Since the simple decrease of the resolution may cause significant control errors as discussed above, a "smart" decrease of the number of analysed pixels, which does not cause large control errors should be used for such purpose much more efficiently. In the proposed statistical approach the main decrease of the amount of data is related to the random draw of a specified number of pixels for further analysis using the modified Monte Carlo method. In a typical version of this method (as often used for the fast object's area estimation) the pixels are chosen randomly by an independent drawing of two coordinates (horizontal and vertical), what complicates the error estimation. For the simplification, the additional mapping of pixels to the one-dimensional vector can be used, what is equivalent to the data transmission e.g. as an uncompressed stream. The additional reduction takes place during the binarization performed for the drawn pixels.

#### **3. The Monte Carlo method for the fast estimation of the objects' geometrical properties**

#### **3.1 The area estimation**

Conducting a statistical experiment based on the Monte Carlo method, an efficient method of estimation of the number of pixels in the image fulfilling a specified condition can be proposed, which can be used for the motion detection purposes as well as for the area estimation of objects together with their basic geometrical features. Instead of counting all the pixels from a high resolution image, the reduction of the number of pixels used for calculations can be achieved by using a pseudo-random generator of uniform distribution. Due to that the performance of the algorithm can be significantly increased. The logical condition which has to be fulfilled can be defined for a greyscale image as well as for the colour one, similarly to the chroma keying commonly used in television. leading to the binary image, which can be further processed. In order to prevent using two independent pseudo-random generators it is assumed that the numbered samples taken from the binary image are stored in the one-dimensional vector, where "1" denotes the black pixels and "0" stands for the white ones representing the background (or reversely). Then a single element from the vector is drawn, which is returned, so each random choice is conducted independently. For simplifying the further theoretical analysis the presence of a single moving dark object (Ob) in the scene (Sc) with a light background can be assumed.

In order to determine the object's geometrical features as well as some motion parameters, such as direction and velocity, the random choice of pixels cannot be performed using the whole image due to the integrating properties of the Monte Carlo method. For preventing possible errors caused by such integration the scene can be divided into smaller and smaller squares, so the *KN* squares from *N* elements of the scene would represent the object. In such case the probability of choosing the point on the object's surface (assuming the generator's uniform distribution) is equal to:

$$p = \frac{K\_N}{N} \tag{1}$$

4 Will-be-set-by-IN-TECH

Analysing the results presented in Tables 1 and 2 a significant influence of the amount of the analysed video data on the obtained results and the power consumption can be noticed. The decrease of the resolution as well as the reduction of the video frame rate causes the increase of the path's length but due to the reduction of the processor's computational load the total energy consumption may decrease. Nevertheless, the decrease of the amount of the processed data should be balanced with the control correctness in order to prevent an excessive extension

Since the simple decrease of the resolution may cause significant control errors as discussed above, a "smart" decrease of the number of analysed pixels, which does not cause large control errors should be used for such purpose much more efficiently. In the proposed statistical approach the main decrease of the amount of data is related to the random draw of a specified number of pixels for further analysis using the modified Monte Carlo method. In a typical version of this method (as often used for the fast object's area estimation) the pixels are chosen randomly by an independent drawing of two coordinates (horizontal and vertical), what complicates the error estimation. For the simplification, the additional mapping of pixels to the one-dimensional vector can be used, what is equivalent to the data transmission e.g. as an uncompressed stream. The additional reduction takes place during the binarization

**3. The Monte Carlo method for the fast estimation of the objects' geometrical**

Conducting a statistical experiment based on the Monte Carlo method, an efficient method of estimation of the number of pixels in the image fulfilling a specified condition can be proposed, which can be used for the motion detection purposes as well as for the area estimation of objects together with their basic geometrical features. Instead of counting all the pixels from a high resolution image, the reduction of the number of pixels used for calculations can be achieved by using a pseudo-random generator of uniform distribution. Due to that the performance of the algorithm can be significantly increased. The logical condition which has to be fulfilled can be defined for a greyscale image as well as for the colour one, similarly to the chroma keying commonly used in television. leading to the binary image, which can be further processed. In order to prevent using two independent pseudo-random generators it is assumed that the numbered samples taken from the binary image are stored in the one-dimensional vector, where "1" denotes the black pixels and "0" stands for the white ones representing the background (or reversely). Then a single element from the vector is drawn, which is returned, so each random choice is conducted independently. For simplifying the further theoretical analysis the presence of a single moving dark object (Ob) in the scene (Sc)

In order to determine the object's geometrical features as well as some motion parameters, such as direction and velocity, the random choice of pixels cannot be performed using the whole image due to the integrating properties of the Monte Carlo method. For preventing possible errors caused by such integration the scene can be divided into smaller and smaller squares, so the *KN* squares from *N* elements of the scene would represent the object. In such case the probability of choosing the point on the object's surface (assuming the generator's

*p* = *KN*

*N* (1)

of the robot's path.

**properties**

**3.1 The area estimation**

performed for the drawn pixels.

with a light background can be assumed.

uniform distribution) is equal to:

For the infinite number of samples the reduction of the sampling distance takes place and the probability of choosing the point representing the object on the image can be expressed as:

$$p = \lim\_{N \to \infty} \frac{K\_N}{N} = \frac{A\_{Ob}}{A\_{Sc}} \tag{2}$$

so the estimated area of the object is:

$$A\_{Ob} \approx K\_N \cdot \frac{A\_{Sc}}{N} \tag{3}$$

Since the total area of the scene is equal to *ASc* = *N* · *kx* · *ky*, the estimated object's area is equal to *AOb* ≈ *KN* · *kx* · *ky*, where *kx* and *ky* are the scale factors for horizontal and vertical coordinates respectively, equivalent to the number of samples per unit.

Considering the above analysis, the probability of choosing the point belonging to the object can be used in the proposed algorithm with a reduced number of analysed samples instead of the full image analysis using a statistical experiment based on the Monte Carlo method. This method originates directly from the law of large numbers, because the sequence of successive approximations of the estimated value is convergent to the sought solution and the distance of the actual value after performing the specified number of statistical tests to the solution can be determined using the central limit theorem.

After the binarization the luminance (or colour) samples represented as "ones" or "zeros" corresponding to the allowed values of the specified logical condition, which are stored in one-dimensional vector are chosen randomly. For a single draw a random variable *Xi* of the two-way distribution is obtained:

$$X\_{i} = \begin{cases} 1 & \text{for black samples} \\ 0 & \text{for white samples} \end{cases} \tag{4}$$

leading to the following probability expressions:

$$P(X\_i = 1) = p \qquad P(X\_i = 0) = q \tag{5}$$

where *p* + *q* = 1, *E*(*Xi*) = *p*, *V*(*Xi*) = *p* · *q*.

An important element of the method is the proper choice of the logical condition for the binarization allowing a proper separation of the object from the background. Such choice depends on the specific application, for example for the light based vehicles' tracking purposes the chroma keying based on the CIELAB colour model can be appropriate (Mazurek & Okarma, 2006).

For *n* independent draws the variable *Yn* is obtained:

$$Y\_n = \frac{1}{n} \cdot \sum\_{i=1}^n X\_i \tag{6}$$

According to the Lindberg-Levy's theorem, the distribution of *Yn* tends to the normal distribution *N*(*my*, *σy*) if *n* → ∞. Since the expected value and variance of *Yn* are equal to *E*(*Yn*) = *p* and *V*(*Yn*) = *p*·*q n* respectively, the distrubution of the random value *Yn* is normal with the following parameters:

$$m\_{\mathcal{Y}} = p \tag{7}$$

Such prepared array, considered as a reduced resolution "greyscale" image, can be directly utilised in some typical control algorithms. Further decrease of the computational cost can be obtained by the additional binarization of this image leading to the ultra-low resolution binary image considered in further discussions, where each of the blocks can be classified as a representative of an object or the background. For the line following robot control purposes the objects is equivalent to the path. The control accuracy of the mobile robots corresponds to the quality loss of the data present in this image. Such quality can be treated as the quality of the binary image assuming the knowledge of the reference image (without any distortions). Unfortunately, most of the image quality assessment methods, even the most recent ones, can be successfully applied only for greyscale or colour images and the specific character of the binary images is not respected by them. For this reason only the methods designed exclusively for the binary images can be used. Such binary image quality assessment methods can be used as the optimisation criteria for the proposed algorithm. In the result the proper choice of the block size, binarization threshold and the relative number of randomly drawn pixels can be

Statistical Video Based Control of Mobile Robots 355

Depending on the logical condition used for the statistical analysis (the construction of the 1-D binary vector), the algorithm can be implemented using various colour spaces (or only some chosen channels), with the possibility of utilising independent logical conditions for

The real-time navigation of an autonomous mobile robot based on the machine vision algorithms requires a fast processing of images acquired by the camera (or cameras) mounted on the robot. In some more sophisticated systems the data fusion based on the additional informations acquired by some other sensors, such as PIR sensors, ultrasound detectors, a radar technique equipment or optical barriers, is also possible. Most of such sensors are responsible for the motion detection, especially if the presence of some moving obstacles in the robot's surrounding is assumed. The vision based sensors can be divided into three main groups: analogue (i.e. comparing the luminance with a given threshold value), digital (obtained by the sampling and quantization of acquired analogue signal) and the digital ones acquired directly from digital video cameras. There are also some restrictions of typical alternative solutions, e.g. passive infrared detectors are sensitive to temperature changes and they are useless for the motion detection of objects with the same temperature as the background. Some more robust solutions, such as ultrasound and radar detectors, are usually active (energy emitting) and require an additional power supply. The application of a vision based motion detection procedure utilising the Monte Carlo method for the mobile robots control purposes is caused mainly by the fact that no additional hardware is necessary, since the images used by the motion detector are acquired by the same camera as in the classification

Assuming the scene with the constant light conditions without any moving objects, it can be characterised by a constant value defined as the number of black (or white) pixels of the respective binary image, which usually represent the object visible on the image obtained from the camera. The algorithm of the motion detection utilises the rapid changes of the number of such pixels (treated as "ones") when an object moves respectively to the light sources. Such change is caused by the dependence of the image pixel's luminance (and colour) on the angle between normal vector to its surface and the direction of the light ray passing the pixel, as well as the influence of the surface's reflection coefficient, shape, roughness etc. Real objects usually have a heterogeneous structure of their surface so the number of analysed points changes dynamically (assuming constant threshold or chroma keying range). For slow

chosen.

procedure.

each channel (chroma keying).

**3.2 The Monte Carlo based motion detection**

$$
\sigma\_{\mathcal{Y}} = \sqrt{\frac{p \cdot q}{n}} \tag{8}
$$

Considering the asymptotic normal distribution *N*(*p*, *p* · *q*/*n*) it can be stated that the central limit theorem is fulfilled for the variable *Yn*. Substituting:

$$\mathcal{U}\_n = \frac{Y\_n - m\_y}{\sigma\_y} \tag{9}$$

obtained normal distribution can be standardized towards the standard normal distribution *N*(0, 1).

In the interval estimation method the following formula is used:

$$p(|\mathcal{U}\_{\mathfrak{n}}| \le \mathfrak{a}) = 1 - \mathfrak{a} \tag{10}$$

Assuming the interval:

$$|\mathcal{U}\_{\mathbb{N}}| \le \mu\_{\mathbb{N}} \tag{11}$$

considering also the formulas (3), (7), (8) and (9), the following expression can be achieved:

$$\left| Y\_{\hbar} - \frac{\mathcal{K}\_{\mathcal{N}}}{N} \right| \le \varepsilon\_{\mathcal{N}} \tag{12}$$

where

$$
\varepsilon\_{\mathfrak{A}} = \frac{\mathfrak{u}\_{\mathfrak{A}}}{\sqrt{n}} \cdot \sqrt{\frac{\mathcal{K}\_N}{N} \cdot \left(1 - \frac{\mathcal{K}\_N}{N}\right)}\tag{13}
$$

The probability estimator *p* (eq. 1), for *k* elements from *n* draws, fulfilling the specified logical condition used for the definition of *Xi* (eq. 4) and representing the number of the drawn pixels representing the object, can be expressed as:

$$
\hat{p} = \frac{k}{n} = \frac{1}{n} \cdot \sum\_{i=1}^{n} X\_i = Y\_n \tag{14}
$$

and the object's area estimator as:

$$
\hat{A}\_{Ob} = \mathfrak{P} \cdot A\_{\mathrm{Sc}} = \frac{\mathrm{k}}{\mathrm{n}} \cdot A\_{\mathrm{Sc}} \tag{15}
$$

Using the equations (12) and (15) the obtained formula describing the interval estimation for the object's area is:

$$\left| \frac{\hat{A}\_{Ob}}{A\_{Sc}} - \frac{\mathcal{K}\_N}{N} \right| \le \varepsilon\_d \tag{16}$$

where *εα* is specified by the equation (13).

It is worth to notice that all the above considerations are correct only for a random number generator with the uniform distribution, which should have as good statistical properties as possible. The discussed algorithm are identical to the method of area estimation of the 2-D object's (expressed in pixels). Nevertheless, the applicability of such approach is limited by the integrating character of the method so an additional modification based on the block approach is necessary, as mentioned earlier.

6 Will-be-set-by-IN-TECH

*Un* = *Yn* − *my σy*

obtained normal distribution can be standardized towards the standard normal distribution

considering also the formulas (3), (7), (8) and (9), the following expression can be achieved:

*Yn* − *KN N*

> *KN N* · 1 − *KN N*

The probability estimator *p* (eq. 1), for *k* elements from *n* draws, fulfilling the specified logical condition used for the definition of *Xi* (eq. 4) and representing the number of the drawn pixels

> *n* ∑ *i*=1

*εα* = *uα* √*n* ·

*p*ˆ = *k*

*A*ˆ

*A*ˆ *Ob ASc*

*n* = 1 *n* ·

*Ob* = *p*ˆ · *ASc* = *k*

Using the equations (12) and (15) the obtained formula describing the interval estimation for

− *KN N*

It is worth to notice that all the above considerations are correct only for a random number generator with the uniform distribution, which should have as good statistical properties as possible. The discussed algorithm are identical to the method of area estimation of the 2-D object's (expressed in pixels). Nevertheless, the applicability of such approach is limited by the integrating character of the method so an additional modification based on the block approach

*p* · *q*

*n* (8)

*p*(|*Un*| ≤ *α*) = 1 − *α* (10)

≤ *εα* (12)

*Xi* = *Yn* (14)

*n* · *ASc* (15)

≤ *εα* (16)

*p* · *q*/*n*) it can be stated that the

(9)

(13)

*σy* =

Considering the asymptotic normal distribution *N*(*p*,

In the interval estimation method the following formula is used:

central limit theorem is fulfilled for the variable *Yn*.

representing the object, can be expressed as:

and the object's area estimator as:

where *εα* is specified by the equation (13).

is necessary, as mentioned earlier.

the object's area is:

Substituting:

Assuming the interval:

*N*(0, 1).

where

Such prepared array, considered as a reduced resolution "greyscale" image, can be directly utilised in some typical control algorithms. Further decrease of the computational cost can be obtained by the additional binarization of this image leading to the ultra-low resolution binary image considered in further discussions, where each of the blocks can be classified as a representative of an object or the background. For the line following robot control purposes the objects is equivalent to the path. The control accuracy of the mobile robots corresponds to the quality loss of the data present in this image. Such quality can be treated as the quality of the binary image assuming the knowledge of the reference image (without any distortions). Unfortunately, most of the image quality assessment methods, even the most recent ones, can be successfully applied only for greyscale or colour images and the specific character of the binary images is not respected by them. For this reason only the methods designed exclusively for the binary images can be used. Such binary image quality assessment methods can be used as the optimisation criteria for the proposed algorithm. In the result the proper choice of the block size, binarization threshold and the relative number of randomly drawn pixels can be chosen.

Depending on the logical condition used for the statistical analysis (the construction of the 1-D binary vector), the algorithm can be implemented using various colour spaces (or only some chosen channels), with the possibility of utilising independent logical conditions for each channel (chroma keying).

#### **3.2 The Monte Carlo based motion detection**

The real-time navigation of an autonomous mobile robot based on the machine vision algorithms requires a fast processing of images acquired by the camera (or cameras) mounted on the robot. In some more sophisticated systems the data fusion based on the additional informations acquired by some other sensors, such as PIR sensors, ultrasound detectors, a radar technique equipment or optical barriers, is also possible. Most of such sensors are responsible for the motion detection, especially if the presence of some moving obstacles in the robot's surrounding is assumed. The vision based sensors can be divided into three main groups: analogue (i.e. comparing the luminance with a given threshold value), digital (obtained by the sampling and quantization of acquired analogue signal) and the digital ones acquired directly from digital video cameras. There are also some restrictions of typical alternative solutions, e.g. passive infrared detectors are sensitive to temperature changes and they are useless for the motion detection of objects with the same temperature as the background. Some more robust solutions, such as ultrasound and radar detectors, are usually active (energy emitting) and require an additional power supply. The application of a vision based motion detection procedure utilising the Monte Carlo method for the mobile robots control purposes is caused mainly by the fact that no additional hardware is necessary, since the images used by the motion detector are acquired by the same camera as in the classification procedure.

Assuming the scene with the constant light conditions without any moving objects, it can be characterised by a constant value defined as the number of black (or white) pixels of the respective binary image, which usually represent the object visible on the image obtained from the camera. The algorithm of the motion detection utilises the rapid changes of the number of such pixels (treated as "ones") when an object moves respectively to the light sources. Such change is caused by the dependence of the image pixel's luminance (and colour) on the angle between normal vector to its surface and the direction of the light ray passing the pixel, as well as the influence of the surface's reflection coefficient, shape, roughness etc. Real objects usually have a heterogeneous structure of their surface so the number of analysed points changes dynamically (assuming constant threshold or chroma keying range). For slow

Object

Detected edges

Elements with the estimated area equal to the size of the elementary block

1 2 3 4 5

Statistical Video Based Control of Mobile Robots 357

191817161514

The simplest approach to the edge detection, which is necessary for further processing, is based on the array *P*. For this purpose the second array *K* with the same size is created and

• zero if the corresponding element's value in the array *P* is equal to zero (background), • zero if the corresponding element's value in the array *P* is equal to the size of the elementary block (all pixels represent the inner part of the object) and none of its

neighbouring blocks (using the 8-directional neighbourhood) has the zero value,

The obtained array *K* is the projection of edges detected from the source image, so the number of its non-zero elements represents the estimated value of object's perimeter expressed in squares of *r* × *r* pixels. For a better estimation the number of square elements can be increased (smaller values of the parameter *r*) and then the previously discussed operation should be repeated. In such case the analysis of the blocks in the array *P* with the zero values (representing the inner parts of the object) is not necessary, so the further analysis can be conducted only for the significantly reduced number of indexed elements classified as the edge in the previous step. Such alternative edge detection method based on the two-step algorithm with the same initial step of the algorithm and another pass of the previous algorithm conducted as the second step with the edge detection calculated using the array *K* (the edge obtained in the first step is treated as the new object without fill) allows more accurate estimation of the edge with slight increase of the computational complexity. All the pixels with the zero values should be replaced from the array *K* and using such corrected image the two curves are obtained as the result (assuming the scale factor of the block greater than *t* = 2 - at least 3 × 3 elements). As the estimate of the edge the middle line between the two obtained curves can be used, providing slightly lower accuracy but also a reduced computational cost. The estimate of the total object's area can also be determined with similarly increased accuracy as the sum of the values in the array *P*. The limit accuracy of the algorithm is determined by the size of the elementary block equal to 1 pixel what is equivalent

Geometrical parameters of the objects, which can be estimated using the modified Monte Carlo method, can be divided into the two major groups: local (such as mean diameter or the

2120

Fig. 3. The idea of the block based edge estimation

its elements have the following values:

• one for the others (representing the edge).

The idea of the edge detection is illustrated in Fig. 3

to the well-known convolution edge detection filters.

2728

changes of the light conditions, especially for outside environment, the binarization threshold can be updated. It it worth to notice that in the proposed approach the necessity of the storage of the single value only (the number of pixels corresponding to the "ones") allows the reduction of the required system's operating memory size.

A typical well known video based algorithm of motion detection is based on the comparison of two neighbouring frames of the video signal but it requires the storage of the images used during the comparison so relatively large amount of memory is needed. Some more robust algorithms of background estimation, directly related to motion detection, usually based on moving average or median filter, are also more computationally demanding and require the analysis of several video frames (Cucchiara et al., 2003; Piccardi, 2004). Even a comparison of all the pixels from two video frames can be treated as a time consuming operation for an autonomous mobile robot with limited computational performance and relatively small amount of operating memory, so in the real-time applications some high performance processing units would be required. Instead of them the statistical Monte Carlo approach described above can be used for the reduction of the computational complexity, where only some randomly chosen pixels are taken into account. Therefore the comparison of full frames is reduced to the comparison of only two values representing the number of "ones" in each frame. Any rapid change of that value is interpreted as the presence of a moving object. A slightly modified method based on the additional division of the image into smaller blocks can be applied for a more robust detection. In the presence of moving objects the changes can be detected only in some of obtained blocks so the integrating character of the Monte Carlo method is eliminated.

The fast motion detection based on the comparison of two values representing the estimated areas of the object in the two consecutive binary frames is based on the following formula:

$$
\hat{A}\_{Ob}(\mathbf{i} + \delta) - \hat{A}\_{Ob}(\mathbf{i}) > \text{threshold} \tag{17}
$$

where *threshold* >> *εα* and *δ* ≥ 1 denotes the shift between both analysed frames.

In the proposed block based approach the application of the formula (17) for each of the *r* × *r* pixels blocks allows proper motion detection, using the array of the Monte Carlo motion detectors, also in the situation when the motion of the mobile robot causes the relative motion of the objects observed by the integrated camera. For the perspectively looking camera, typical for such applications, even if the objects do not change their size on the image, the changes can be observed among the blocks. The objects moving towards the camera always cause the increase of their estimated area in the image used by the motion detector. The estimation of the geometrical parameters of the objects, often necessary for the proper navigation of the controlled robot, can be performed for the frame with the maximum value of the estimated area occupied by the object on the image with eliminated background, assuming that robot is not moving directly towards the object. If the size of the object increases in consecutive frames, such estimation should be performed immediately preventing possible collision with the object.

#### **3.3 The estimation of geometrical parameters**

The extraction of some geometrical parameters, such as perimeters or diameters, using the discussed approach is also possible. For this purpose the analysed binary image has to be divided into *T* × *S* blocks of *r* × *r* pixels each using a square grid. Then the area of each object's fragment in the elementary square elements (blocks) is calculated and the results of such estimation can be stored in the array *P* containing *T* × *S* elements.

8 Will-be-set-by-IN-TECH

changes of the light conditions, especially for outside environment, the binarization threshold can be updated. It it worth to notice that in the proposed approach the necessity of the storage of the single value only (the number of pixels corresponding to the "ones") allows

A typical well known video based algorithm of motion detection is based on the comparison of two neighbouring frames of the video signal but it requires the storage of the images used during the comparison so relatively large amount of memory is needed. Some more robust algorithms of background estimation, directly related to motion detection, usually based on moving average or median filter, are also more computationally demanding and require the analysis of several video frames (Cucchiara et al., 2003; Piccardi, 2004). Even a comparison of all the pixels from two video frames can be treated as a time consuming operation for an autonomous mobile robot with limited computational performance and relatively small amount of operating memory, so in the real-time applications some high performance processing units would be required. Instead of them the statistical Monte Carlo approach described above can be used for the reduction of the computational complexity, where only some randomly chosen pixels are taken into account. Therefore the comparison of full frames is reduced to the comparison of only two values representing the number of "ones" in each frame. Any rapid change of that value is interpreted as the presence of a moving object. A slightly modified method based on the additional division of the image into smaller blocks can be applied for a more robust detection. In the presence of moving objects the changes can be detected only in some of obtained blocks so the integrating character of the Monte Carlo

The fast motion detection based on the comparison of two values representing the estimated areas of the object in the two consecutive binary frames is based on the following formula:

In the proposed block based approach the application of the formula (17) for each of the *r* × *r* pixels blocks allows proper motion detection, using the array of the Monte Carlo motion detectors, also in the situation when the motion of the mobile robot causes the relative motion of the objects observed by the integrated camera. For the perspectively looking camera, typical for such applications, even if the objects do not change their size on the image, the changes can be observed among the blocks. The objects moving towards the camera always cause the increase of their estimated area in the image used by the motion detector. The estimation of the geometrical parameters of the objects, often necessary for the proper navigation of the controlled robot, can be performed for the frame with the maximum value of the estimated area occupied by the object on the image with eliminated background, assuming that robot is not moving directly towards the object. If the size of the object increases in consecutive frames, such estimation should be performed immediately preventing possible collision with

The extraction of some geometrical parameters, such as perimeters or diameters, using the discussed approach is also possible. For this purpose the analysed binary image has to be divided into *T* × *S* blocks of *r* × *r* pixels each using a square grid. Then the area of each object's fragment in the elementary square elements (blocks) is calculated and the results of

such estimation can be stored in the array *P* containing *T* × *S* elements.

*Ob*(*i*) > *threshold* (17)

the reduction of the required system's operating memory size.

*A*ˆ

**3.3 The estimation of geometrical parameters**

*Ob*(*i* + *δ*) − *A*ˆ

where *threshold* >> *εα* and *δ* ≥ 1 denotes the shift between both analysed frames.

method is eliminated.

the object.

Fig. 3. The idea of the block based edge estimation

The simplest approach to the edge detection, which is necessary for further processing, is based on the array *P*. For this purpose the second array *K* with the same size is created and its elements have the following values:

The idea of the edge detection is illustrated in Fig. 3

The obtained array *K* is the projection of edges detected from the source image, so the number of its non-zero elements represents the estimated value of object's perimeter expressed in squares of *r* × *r* pixels. For a better estimation the number of square elements can be increased (smaller values of the parameter *r*) and then the previously discussed operation should be repeated. In such case the analysis of the blocks in the array *P* with the zero values (representing the inner parts of the object) is not necessary, so the further analysis can be conducted only for the significantly reduced number of indexed elements classified as the edge in the previous step. Such alternative edge detection method based on the two-step algorithm with the same initial step of the algorithm and another pass of the previous algorithm conducted as the second step with the edge detection calculated using the array *K* (the edge obtained in the first step is treated as the new object without fill) allows more accurate estimation of the edge with slight increase of the computational complexity. All the pixels with the zero values should be replaced from the array *K* and using such corrected image the two curves are obtained as the result (assuming the scale factor of the block greater than *t* = 2 - at least 3 × 3 elements). As the estimate of the edge the middle line between the two obtained curves can be used, providing slightly lower accuracy but also a reduced computational cost. The estimate of the total object's area can also be determined with similarly increased accuracy as the sum of the values in the array *P*. The limit accuracy of the algorithm is determined by the size of the elementary block equal to 1 pixel what is equivalent to the well-known convolution edge detection filters.

Geometrical parameters of the objects, which can be estimated using the modified Monte Carlo method, can be divided into the two major groups: local (such as mean diameter or the

Fig. 4. The idea of the "cut-off" operation

the image representing the tracked line.

• ignoring the empty lines (without any "ones").

control possibilities of the controlled robot.

**–** the detection of the crossing lines

the control flag is set to 0).

line-crossing or using the speed control

• binarization of the current frame,

• orphan blocks removal,

• control operation:

block

**4.2 Extension of the robot control by vision based prediction**

• filling the gaps in the detected line using the approximation methods,

neighbouring "ones"),

infra-red sensors.

row of the obtained simplified binary image which is equivalent to the typical line of the

Statistical Video Based Control of Mobile Robots 359

Nevertheless, the machine vision algorithm can be disturbed by some artifacts caused by the errors during the preprocessing step (e.g. insufficient filtration of some undesired elements on the image). An optimal situation would take place for a single "one" in the lowest row of

In the simplified version of the algorithm no line crossings can be assumed. The reduction of

• limitation of the line thickness by the proper choice of the binarization threshold (lines containing more than a single "one" can be ignored assuming no line crossings),

• orphan blocks removal (applicable for all blocks with the value "one" without any

The control algorithm of the robot should point towards such a position that only a single block of the bottom image row will be filled (representing the line). In the case when two neighbouring blocks represent the line the choice of the block should be based on the simple analysis of the current trend, assuming that the turn angles of the line match the motion and

The control algorithm for a single frame can be described as follows (Okarma & Lech, 2010):

\* if not turning: moving forward with the maximum speed for the symmetrical

\* else: turning and using the speed control if the maximum value is not in the middle

**–** speed control: velocity should be proportional to the sum of the values in the middle blocks of each horizontal line (if zero, the minimum speed is set before the turning and

the possible disturbances can be achieved using the following additional operations:

average area) and global (e.g. the number of objects in the specified area). The estimation of the relative global parameters is not necessary if only a single object is present on the analysed fragment of the image. The most useful parameters used for image analysis, which can be also used for robot's control purposes, are insensitive to image deformations which can occur during the acquisition, as well as to some typical geometrical transformations. In such sense the usefulness of the simplest parameters such as area and perimeter is limited. Nevertheless, the majority of more advanced parameters (such as moments) can be usually determined using the previously described ones, similarly as some motion parameters e.g. direction or velocity (Okarma & Lech, 2008a).

#### **4. Application for the line following robots**

One of the most important tasks of the control systems designed for mobile robots is line tracking. Its typical optical implementation is based on a line of sensors receiving information about the position of the traced line, usually located underneath the robot. Such line sensor is built from a specified number of cells limiting the resolution of the optical system. Another important parameter of such systems is the distance from the sensors to the centre of steering, responsible for the maximum possible speed of the properly controlled robot. The smoothness of the robot's motion is also dependent on the spacing between the cells forming the line sensor.

A significant disadvantage of such optical systems is relatively low resolution of the tracking system, which can be increases using some other vision based systems with wide possibilities of analysing data acquired from the front Dupuis & Parizeau (2006); Rahman et al. (2005). Nevertheless the analysis of the full acquired image even with low resolution is usually computationally demanding and time consuming, especially in the presence of some obstacles, some line intersections etc., assuming also varying lighting conditions. Another serious drawback of using line of sensors is the short time for the reaction limited by the distance between the sensors and the steering centre (or wheels). The proposed approach, based on the fast image analysis using the Monte Carlo method, preserves the main advantages of the vision systems allowing the reduction of the amount of processed data. Considering its application for the control of the mobile robots, the camera moves relatively to the static scene, differently than in the primary version of the algorithm, but the working properties of the method are similar.

In order to filter the undesired data, usually related to some contaminations which should not be used for the robot control purposes, the binarization threshold should be properly set. In the conducted experiments the additional "cut-off" value has been set as minimum 20% black pixels possibly representing the object within the block in order to avoid the influence of the small artifacts, especially close to the region of interest. Finally, the simplified binary representation of the image is obtained where such artifacts (the elements which in fact do not represent the followed line) have been removed during the "cut-off" operation illustrated in Fig. 4 where the original binary image and the intermediate result obtained by the Monte Carlo method are also presented.

#### **4.1 A simple proportional control algorithm for controlling a line following robot**

The mobile robot control process can be based on the popular proportional control approach. It is typically used for controlling the robots with sensors based on the infra-red receivers grouped into the line containing a specified number of the infra-red cells. In the proposed method the differential steering signals for the motors are obtained using the first (the lowest) 10 Will-be-set-by-IN-TECH

average area) and global (e.g. the number of objects in the specified area). The estimation of the relative global parameters is not necessary if only a single object is present on the analysed fragment of the image. The most useful parameters used for image analysis, which can be also used for robot's control purposes, are insensitive to image deformations which can occur during the acquisition, as well as to some typical geometrical transformations. In such sense the usefulness of the simplest parameters such as area and perimeter is limited. Nevertheless, the majority of more advanced parameters (such as moments) can be usually determined using the previously described ones, similarly as some motion parameters e.g.

One of the most important tasks of the control systems designed for mobile robots is line tracking. Its typical optical implementation is based on a line of sensors receiving information about the position of the traced line, usually located underneath the robot. Such line sensor is built from a specified number of cells limiting the resolution of the optical system. Another important parameter of such systems is the distance from the sensors to the centre of steering, responsible for the maximum possible speed of the properly controlled robot. The smoothness of the robot's motion is also dependent on the spacing between the cells forming the line

A significant disadvantage of such optical systems is relatively low resolution of the tracking system, which can be increases using some other vision based systems with wide possibilities of analysing data acquired from the front Dupuis & Parizeau (2006); Rahman et al. (2005). Nevertheless the analysis of the full acquired image even with low resolution is usually computationally demanding and time consuming, especially in the presence of some obstacles, some line intersections etc., assuming also varying lighting conditions. Another serious drawback of using line of sensors is the short time for the reaction limited by the distance between the sensors and the steering centre (or wheels). The proposed approach, based on the fast image analysis using the Monte Carlo method, preserves the main advantages of the vision systems allowing the reduction of the amount of processed data. Considering its application for the control of the mobile robots, the camera moves relatively to the static scene, differently than in the primary version of the algorithm, but the working

In order to filter the undesired data, usually related to some contaminations which should not be used for the robot control purposes, the binarization threshold should be properly set. In the conducted experiments the additional "cut-off" value has been set as minimum 20% black pixels possibly representing the object within the block in order to avoid the influence of the small artifacts, especially close to the region of interest. Finally, the simplified binary representation of the image is obtained where such artifacts (the elements which in fact do not represent the followed line) have been removed during the "cut-off" operation illustrated in Fig. 4 where the original binary image and the intermediate result obtained by the Monte

The mobile robot control process can be based on the popular proportional control approach. It is typically used for controlling the robots with sensors based on the infra-red receivers grouped into the line containing a specified number of the infra-red cells. In the proposed method the differential steering signals for the motors are obtained using the first (the lowest)

**4.1 A simple proportional control algorithm for controlling a line following robot**

direction or velocity (Okarma & Lech, 2008a).

**4. Application for the line following robots**

properties of the method are similar.

Carlo method are also presented.

sensor.

Fig. 4. The idea of the "cut-off" operation

row of the obtained simplified binary image which is equivalent to the typical line of the infra-red sensors.

Nevertheless, the machine vision algorithm can be disturbed by some artifacts caused by the errors during the preprocessing step (e.g. insufficient filtration of some undesired elements on the image). An optimal situation would take place for a single "one" in the lowest row of the image representing the tracked line.

In the simplified version of the algorithm no line crossings can be assumed. The reduction of the possible disturbances can be achieved using the following additional operations:

#### **4.2 Extension of the robot control by vision based prediction**

The control algorithm of the robot should point towards such a position that only a single block of the bottom image row will be filled (representing the line). In the case when two neighbouring blocks represent the line the choice of the block should be based on the simple analysis of the current trend, assuming that the turn angles of the line match the motion and control possibilities of the controlled robot.

The control algorithm for a single frame can be described as follows (Okarma & Lech, 2010):

	- **–** the detection of the crossing lines
		- \* if not turning: moving forward with the maximum speed for the symmetrical line-crossing or using the speed control
		- \* else: turning and using the speed control if the maximum value is not in the middle block
	- **–** speed control: velocity should be proportional to the sum of the values in the middle blocks of each horizontal line (if zero, the minimum speed is set before the turning and the control flag is set to 0).

**4.3 Application of the Centre of Gravity**

of Mass). It can be determined for a grayscale image as:

previous filtration of any artifacts from the image.

If the robot is located directly over the straight line its central point should be exactly between the two edges obtained using the two-step algorithm of edge detection discussed above. In such situation the sum of the pixel values on its left side should be identical as on its right side. Any significant disturbance can be interpreted as the beginning of a line curve with the assumption that no additional elements, except two line edges, are visible on the image, so the mobile robot should begin turning. An efficient and simple method for controlling the robot which can be used for such purpose is based on the image's Centre of Gravity (Centre

Statistical Video Based Control of Mobile Robots 361

*xc* =

*yc* =

where *Xi*,*j* represents the luminance of the pixel located at position (*i*, *j*).

computation time as has been verified in the conducted experiments.

Monte Carlo method can be additionally verified by the CoG calculations.

Gravity used for determining the current trend (direction of the line).

∑ *i*,*j*

> ∑ *i*,*j Xi*,*j*

∑ *i*,*j*

> ∑ *i*,*j Xi*,*j*

Calculating the coordinates of the Centre of Gravity (CoG), they can be used by the control algorithm as the current target point for the mobile robot. The horizontal coordinate *xc* can be used for controlling the turning angle and the vertical *yc* for the speed control, assuming the

The approach discussed above can also be implemented using the block based Monte Carlo method using the "cut-off" operation for decrease of the influence of noise, leading to the hybrid control method. For each block the local area estimator *A*ˆ is used as the input for the "cut-off" operation leading to the simplified binary representation utilised for steering using the method discussed above. In comparison to the classical Center of Gravity method its implementation using the Monte Carlo approach causes about 8 times reduction of the

One of the main modifications proposed in the hybrid method is the switch between the CoG and Monte Carlo based control in order to increase the robustness of the control. For this purpose the sum of each column of the simplified binary image is calculated and stored in an additional vector. If the vector containing the sums of the values, obtained by counted blocks with "ones", has more than one local minimum (e.g. 2 4 0 0 1 6 0 0 1), some additional objects are present on the image. In such situation the calculation of the Centre of Gravity coordinates would be disturbed by a neighbouring object located close to the line, so the motion control should be based on the previous value. Otherwise, the control values determined by the

A more complicated situation may take place for the presence of artifacts together with some discontinuities of the followed line. In this situation the vector containing the sum of the column in the simplified block representation of the image has more than one local maximum. In such case the additional elimination of the blocks which are not directly connected to the followed line (detected as represented by the maximum value in the vector) should be conducted e.g. using the morphological erosion, before the calculations of the Centre of

*i* · *Xi*,*j*

*j* · *Xi*,*j*

(18)

(19)

Fig. 5. The illustration of the various configurations of the followed line

The main element of the modified control system is the variable (flag), determined using the previous frames, informing the control system about the current state (e.g. 0 for moving forward or turning, 1 for turning with detected crossing lines).

In the case of detected crossing line the control depends on the symmetry. For the symmetrical line-crossing the robot should move forward with the maximum speed or the additional speed control. If the maximum value is not in the middle block the line-crossing is not symmetrical and the robot should turn left or right and decrease it speed. The speed of the robot should be proportional to the sum of the values in the middle blocks of each horizontal line. If that sum is equal to zero, the minimum speed should be set before the turning. In that case the mentioned above flag should be set to 1. The illustration of the various scenarios is presented in Fig. 5.

One of the typical features of the proportionally controlled systems is the oscillating character of the path for the robot reaching the specified line, causing also the characteristic oscillations of the images acquired from the camera. Nevertheless, the conducted experiments have verified the usefulness of the fast image analysis based on the Monte Carlo method for the efficient controlling of the line following robot in a simplified but very efficient way, especially useful for the real-time applications. The application of the proposed solution for the control of the mobile robot, despite of the simplicity of its control algorithm, is comparable with some typical ones based on the classical approach (proportional steering based on the infra-red sensors).

The main advantage of the proposed method is the possibility of using some prediction algorithms allowing the increase of the robot's dynamic properties and fluent changes of its speed depending on the shape of the line in front of the robot. Another relevant feature is its low computational complexity causing the relatively high processing speed, especially important in the real-time embedded systems.

#### **4.3 Application of the Centre of Gravity**

12 Will-be-set-by-IN-TECH

Fig. 5. The illustration of the various configurations of the followed line

forward or turning, 1 for turning with detected crossing lines).

important in the real-time embedded systems.

The main element of the modified control system is the variable (flag), determined using the previous frames, informing the control system about the current state (e.g. 0 for moving

In the case of detected crossing line the control depends on the symmetry. For the symmetrical line-crossing the robot should move forward with the maximum speed or the additional speed control. If the maximum value is not in the middle block the line-crossing is not symmetrical and the robot should turn left or right and decrease it speed. The speed of the robot should be proportional to the sum of the values in the middle blocks of each horizontal line. If that sum is equal to zero, the minimum speed should be set before the turning. In that case the mentioned above flag should be set to 1. The illustration of the various scenarios is presented

One of the typical features of the proportionally controlled systems is the oscillating character of the path for the robot reaching the specified line, causing also the characteristic oscillations of the images acquired from the camera. Nevertheless, the conducted experiments have verified the usefulness of the fast image analysis based on the Monte Carlo method for the efficient controlling of the line following robot in a simplified but very efficient way, especially useful for the real-time applications. The application of the proposed solution for the control of the mobile robot, despite of the simplicity of its control algorithm, is comparable with some typical ones based on the classical approach (proportional steering based on the infra-red

The main advantage of the proposed method is the possibility of using some prediction algorithms allowing the increase of the robot's dynamic properties and fluent changes of its speed depending on the shape of the line in front of the robot. Another relevant feature is its low computational complexity causing the relatively high processing speed, especially

in Fig. 5.

sensors).

If the robot is located directly over the straight line its central point should be exactly between the two edges obtained using the two-step algorithm of edge detection discussed above. In such situation the sum of the pixel values on its left side should be identical as on its right side. Any significant disturbance can be interpreted as the beginning of a line curve with the assumption that no additional elements, except two line edges, are visible on the image, so the mobile robot should begin turning. An efficient and simple method for controlling the robot which can be used for such purpose is based on the image's Centre of Gravity (Centre of Mass). It can be determined for a grayscale image as:

$$\mathbf{x}\_{\mathcal{L}} = \frac{\sum\_{i,j} i \cdot \mathbf{X}\_{i,j}}{\sum\_{i,j} \mathbf{X}\_{i,j}} \tag{18}$$

$$y\_c = \frac{\sum\_{i,j} j \cdot X\_{i,j}}{\sum\_{i,j} X\_{i,j}} \tag{19}$$

where *Xi*,*j* represents the luminance of the pixel located at position (*i*, *j*).

Calculating the coordinates of the Centre of Gravity (CoG), they can be used by the control algorithm as the current target point for the mobile robot. The horizontal coordinate *xc* can be used for controlling the turning angle and the vertical *yc* for the speed control, assuming the previous filtration of any artifacts from the image.

The approach discussed above can also be implemented using the block based Monte Carlo method using the "cut-off" operation for decrease of the influence of noise, leading to the hybrid control method. For each block the local area estimator *A*ˆ is used as the input for the "cut-off" operation leading to the simplified binary representation utilised for steering using the method discussed above. In comparison to the classical Center of Gravity method its implementation using the Monte Carlo approach causes about 8 times reduction of the computation time as has been verified in the conducted experiments.

One of the main modifications proposed in the hybrid method is the switch between the CoG and Monte Carlo based control in order to increase the robustness of the control. For this purpose the sum of each column of the simplified binary image is calculated and stored in an additional vector. If the vector containing the sums of the values, obtained by counted blocks with "ones", has more than one local minimum (e.g. 2 4 0 0 1 6 0 0 1), some additional objects are present on the image. In such situation the calculation of the Centre of Gravity coordinates would be disturbed by a neighbouring object located close to the line, so the motion control should be based on the previous value. Otherwise, the control values determined by the Monte Carlo method can be additionally verified by the CoG calculations.

A more complicated situation may take place for the presence of artifacts together with some discontinuities of the followed line. In this situation the vector containing the sum of the column in the simplified block representation of the image has more than one local maximum. In such case the additional elimination of the blocks which are not directly connected to the followed line (detected as represented by the maximum value in the vector) should be conducted e.g. using the morphological erosion, before the calculations of the Centre of Gravity used for determining the current trend (direction of the line).

line reconstruction can be treated as one of the most relevant advantages of the image based motion control. Such reconstruction can also be performed using the ultra-low resolution binary image as the input data. Another possibility, being analysed in our experiments, is the adaptation of the Centre of Gravity method for the real time steering based on the binary

Statistical Video Based Control of Mobile Robots 363

The additional robustness of the method can be achieved by the adaptive change of the block size used in the modified Monte Carlo method according to the quality assessment of the

Positive results of the simulations performed in Java 3D based Simbad environment allow the proper implementation of the proposed soft computing approach in the experimental video controlled line following robot. Good simulation results, as weel as the low energy consumption by the experimental netbook-based robot, have been also obtained considering the synthetic reduced resolution "grayscale" image constructed using the Monte Carlo method, where the relative areas of the objects in each block have been used instead of the

The planned future work is related to the physical application of the proposed approach for some other types of robot control algorithms based on video analysis, e.g. SLAM. Another direction of the future research should be the development of a dedicated binary image quality assessment method, which could be used for the optimisation of the proposed algorithm. Another interesting direction of the future work is the extension of the proposed method towards direct operations using the colour images. For this purpose a verification of the usefulness of the typical colour models is necessary by means of the chroma keying used

Cowan, N., Shakernia, O., Vidal, R. & Sastry, S. (2003). Vision-based follow-the-leader,

Cucchiara, R., Grana, C., Piccardi, M. & Prati, A. (2003). Detecting moving objects, ghosts,

Dupuis, J.-F. & Parizeau, M. (2006). Evolving a vision-based line-following robot controller,

Geiger, C., Reimann, C., Stocklein, J. & Paelke, V. (2002). JARToolkit - a Java binding

Hähnel, D., Triebel, R., Burgard, W. & Thrun, S. (2003). Map building with mobile robots

Hugues, L. & Bredeche, N. (2006). Simbad: An autonomous robot simulation package for

Kalyan, B., Lee, K. & Wijesoma, W. (2010). FISST-SLAM: Finite set statistical approach

*Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems*, Vol. 2,

and shadows in video streams, *IEEE Transactions on Pattern Analysis and Machine*

*Proceedings of the 3rd Canadian Conference on Computer and Robot Vision*, Quebec City,

for ARToolkit, *Proceedings of the The First IEEE International Workshop on Augmented*

in dynamic environments, *International Conference on Robotics and Automation*, Vol. 2,

education and research, *in* S. Nolfi, G. Baldassarre, R. Calabretta, J. C. T. Hallam, D. Marocco, J.-A. Meyer, O. Miglino & D. Parisi (eds), *SAB*, Vol. 4095 of *Lecture Notes*

to simultaneous localization and mapping, *International Journal of Robotics Research*

binary image and the additional analysis of the line continuity.

luminance level (after normalisation).

for the thresholding in the Monte Carlo method.

Las Vegas, Nevada, pp. 1796–1801.

*Reality Toolkit*, Darmstadt, Germany, p. 5.

*in Computer Science*, Springer, pp. 831–842.

*Intelligence* 25(10): 1337–1342.

Taipei, Taiwan, pp. 1557–1563.

Canada, p. 75.

29(10): 1251–1262.

image analysis.

**7. References**

#### **5. Simulations in the Simbad environment**

In order to verify the working properties of the proposed methods, regardless of some tests conducted using a netbook based prototype, a test application has been created using Java programming language in NetBeans environment allowing the integration of many libraries developed for Java e.g JARToolKit (Geiger et al., 2002) allowing to access the Augmented Reality Toolkit (ARToolKit) functionality via Java and the use of different rendering libraries allows high and low level access. The environment of the mobile robot as well as the robot itself have been modelled using the Simbad simulator (Hugues & Bredeche, 2006), which is available for this platform.

A significant advantage of the simulator is its 3D environment, due to the utilisation of Sun Java3D technology. Nevertheless, an important restrictions of Simbad, in opposite to some other similar products, are the limited working properties for the controlling the real robots die to its internal structure related to the modelling of physical phenomena. However, this results in a simplified and transparent programming source code, so the implementation of the presented algorithms does not cause any considerable problems. The only restriction in the initialisation part of the robot controller is the directing the camera towards the "ceiling", caused by the inability to put the followed line on the ground. Instead of it the line has been located over the robot and the lines in the front of the robot are shown in the top part of the image. The only negative consequence is the reverse transformation of the left and right directions between the robot and the camera. The proportional controlled differential drive robots used in the experiments are typically implemented in Simbad and conducted simulations have confirmed the working properties of the presented steering methods for all control modes with different simulation speed values. The test application used for the simulations is illustrated in Fig . 6.

Fig. 6. Illustration of the test application

#### **6. Conclusions and future work**

Obtained results allowed the full control of the line following robot (speed, direction and "virtual" line reconstruction). Due to the ability of path prediction, in comparison to the typical line following robots based on the line of sensors mounted under the robot, the "virtual" 14 Will-be-set-by-IN-TECH

In order to verify the working properties of the proposed methods, regardless of some tests conducted using a netbook based prototype, a test application has been created using Java programming language in NetBeans environment allowing the integration of many libraries developed for Java e.g JARToolKit (Geiger et al., 2002) allowing to access the Augmented Reality Toolkit (ARToolKit) functionality via Java and the use of different rendering libraries allows high and low level access. The environment of the mobile robot as well as the robot itself have been modelled using the Simbad simulator (Hugues & Bredeche, 2006), which is

A significant advantage of the simulator is its 3D environment, due to the utilisation of Sun Java3D technology. Nevertheless, an important restrictions of Simbad, in opposite to some other similar products, are the limited working properties for the controlling the real robots die to its internal structure related to the modelling of physical phenomena. However, this results in a simplified and transparent programming source code, so the implementation of the presented algorithms does not cause any considerable problems. The only restriction in the initialisation part of the robot controller is the directing the camera towards the "ceiling", caused by the inability to put the followed line on the ground. Instead of it the line has been located over the robot and the lines in the front of the robot are shown in the top part of the image. The only negative consequence is the reverse transformation of the left and right directions between the robot and the camera. The proportional controlled differential drive robots used in the experiments are typically implemented in Simbad and conducted simulations have confirmed the working properties of the presented steering methods for all control modes with different simulation speed values. The test application used for the

Obtained results allowed the full control of the line following robot (speed, direction and "virtual" line reconstruction). Due to the ability of path prediction, in comparison to the typical line following robots based on the line of sensors mounted under the robot, the "virtual"

**5. Simulations in the Simbad environment**

available for this platform.

simulations is illustrated in Fig . 6.

Fig. 6. Illustration of the test application

**6. Conclusions and future work**

line reconstruction can be treated as one of the most relevant advantages of the image based motion control. Such reconstruction can also be performed using the ultra-low resolution binary image as the input data. Another possibility, being analysed in our experiments, is the adaptation of the Centre of Gravity method for the real time steering based on the binary image analysis.

The additional robustness of the method can be achieved by the adaptive change of the block size used in the modified Monte Carlo method according to the quality assessment of the binary image and the additional analysis of the line continuity.

Positive results of the simulations performed in Java 3D based Simbad environment allow the proper implementation of the proposed soft computing approach in the experimental video controlled line following robot. Good simulation results, as weel as the low energy consumption by the experimental netbook-based robot, have been also obtained considering the synthetic reduced resolution "grayscale" image constructed using the Monte Carlo method, where the relative areas of the objects in each block have been used instead of the luminance level (after normalisation).

The planned future work is related to the physical application of the proposed approach for some other types of robot control algorithms based on video analysis, e.g. SLAM. Another direction of the future research should be the development of a dedicated binary image quality assessment method, which could be used for the optimisation of the proposed algorithm.

Another interesting direction of the future work is the extension of the proposed method towards direct operations using the colour images. For this purpose a verification of the usefulness of the typical colour models is necessary by means of the chroma keying used for the thresholding in the Monte Carlo method.

#### **7. References**

**18**

*1Chile 2Colombia*

**An Embedded Type-2 Fuzzy Controller**

*Department of Electrical Engineering, University of Chile, Santiago*

*2Laboratory for Automation, Microelectronics and Computational Intelligence (LAMIC), Faculty of Engineering, Universidad Distrital Francisco José de Caldas, Bogotá D.C.*

Fuzzy logic systems (FLS) have been used in different applications with satisfactory performance (Wang, 1997). The human perception cannot be modelled by traditional mathematical techniques, thus, the introduction of fuzzy set (FS) theory in this modelling has been suitable (John & Coupland, 2007). When real-world applications are treated, many sources of uncertainty often appear. Several natures of uncertainties would influence the performance of a system. It is independent from what kind of methodology is used to

Type-1 Fuzzy logic systems (T1-FLS) have limited capabilities to directly handle data uncertainties (Mendel, 2007). Once a type-1 membership function (MF) has been defined, uncertainty disappears because a T1-MF is totally precise (Hagras, 2007). Type-2 fuzzy logic systems (T2-FLS) make possible to model and handle uncertainties. These are rule based systems in which linguistic variables are described by means of Type-2 fuzzy sets (T2-FSs) that include a footprint of uncertainty (FOU) (Mendel, 2001). It provides a measure of dispersion to capture more about uncertainties (Mendel, 2007). While T2-FSs have non-crisp

A representation of the inference model for T2-FLS is depicted in Figure 1 (Mendel, 2007). It begins with fuzzification, which maps crisp points into T2-FSs. Next, inference engine computes the rule base by making logical combinations of antecedent T2-FS, whose results are implicated with consequent T2-FS to form an aggregated output type-2 fuzzy set. Afterwards, Type-Reduction (TR) takes all output sets and performs a centroid calculation of this combined type-2 fuzzy set, which leads to a type-1 fuzzy set called *type-reduced set*. That reduced set is finally defuzzyfied in order to obtain a crisp output (Mendel, 2001; Karnik & Mendel 2001). The computational complexity of this model is reduced if interval type-2 fuzzy sets are used (Mendel, 2001), it is convenient in the context of hardware implementation in order to make softer the computational effort and sped up the inference

Type-2 fuzzy hardware is a topic of special interest, since the application of T2-FLS to particular fields that demand mobile electronic solutions would be necessary. Some recent

MFs, T1-FSs have crisp membership grades (MGs) (John & Coupland, 2007).

**1. Introduction**

handle it (Mendel, 2001).

time.

**for a Mobile Robot Application**

Leonardo Leottau1 and Miguel Melgarejo2 *1Advanced Mining Technology Center (AMTC),*

## **An Embedded Type-2 Fuzzy Controller for a Mobile Robot Application**

Leonardo Leottau1 and Miguel Melgarejo2

*1Advanced Mining Technology Center (AMTC), Department of Electrical Engineering, University of Chile, Santiago 2Laboratory for Automation, Microelectronics and Computational Intelligence (LAMIC), Faculty of Engineering, Universidad Distrital Francisco José de Caldas, Bogotá D.C. 1Chile*

*2Colombia*

#### **1. Introduction**

16 Will-be-set-by-IN-TECH

364 Recent Advances in Mobile Robotics

Mazurek, P. & Okarma, K. (2006). Car-by-light tracking approach based on log-likelihood

Okarma, K. & Lech, P. (2008a). Monte Carlo based algorithm for fast preliminary video

Okarma, K. & Lech, P. (2009). Application of Monte Carlo preliminary image analysis and

Okarma, K. & Lech, P. (2010). A fast image analysis technique for the line tracking robots, *in*

Rahman, M., Rahman, M., Haque, A. & Islam, M. (2005). Architecture of the vision system of

Se, S., Lowe, D. & Little, J. (2002). Mobile robot localization and mapping with uncertainty

Tamimi, H. & Zell, A. (2004). Vision based localization of mobile robots using kernel

*International Multitopic Conference, IEEE INMIC 2005*, Karachi, pp. 1–8. Se, S., Lowe, D. & Little, J. (2001). Local and global localization for mobile robots using visual

*(2)*, Vol. 6114 of *Lecture Notes in Computer Science*, Springer, pp. 329–336. Piccardi, M. (2004). Background subtraction techniques: a review, *Proceedings of the*

*(1)*, Vol. 5101 of *Lecture Notes in Computer Science*, Springer, pp. 790–799. Okarma, K. & Lech, P. (2008b). A statistical reduced-reference approach to digital image

Vol. 5337 of *Lecture Notes in Computer Science*, Springer, pp. 43–54.

*and Vision: International Journal* 18(4): 439–452.

*Systems*, Vol. 1, Maui, Hawaii, pp. 414–420.

*Robots and Systems*, Sendai, Japan, pp. 1896–1901.

Zakopane, Poland, pp. 15–20.

Netherlands, pp. 3099–3104.

21: 735–758.

track-before-detect algorithm, *Proceedings of the 10th International Conference "Computer Systems Aided Science Industry and Transport" TRANSCOMP*, Vol. 2,

analysis, *in* M. Bubak, G. D. van Albada, J. Dongarra & P. M. A. Sloot (eds), *ICCS*

quality assessment, *in* L. Bolc, J. L. Kulikowski & K. W. Wojciechowski (eds), *ICCVG*,

classification method for automatic reservation of parking space, *Machine Graphics*

L. Rutkowski, R. Scherer, R. Tadeusiewicz, L. A. Zadeh & J. M. Zurada (eds), *ICAISC*

*IEEE International Conference on Systems, Man & Cybernetics*, Vol. 4, The Hague,

a line following mobile robot operating in static environment, *Proceedings of the 9th*

landmarks, *Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and*

using scale-invariant visual landmarks, *International Journal of Robotics Research*

approaches, *Proceedings of the 2004 IEEE/RSJ International Conference on Intelligent*

Fuzzy logic systems (FLS) have been used in different applications with satisfactory performance (Wang, 1997). The human perception cannot be modelled by traditional mathematical techniques, thus, the introduction of fuzzy set (FS) theory in this modelling has been suitable (John & Coupland, 2007). When real-world applications are treated, many sources of uncertainty often appear. Several natures of uncertainties would influence the performance of a system. It is independent from what kind of methodology is used to handle it (Mendel, 2001).

Type-1 Fuzzy logic systems (T1-FLS) have limited capabilities to directly handle data uncertainties (Mendel, 2007). Once a type-1 membership function (MF) has been defined, uncertainty disappears because a T1-MF is totally precise (Hagras, 2007). Type-2 fuzzy logic systems (T2-FLS) make possible to model and handle uncertainties. These are rule based systems in which linguistic variables are described by means of Type-2 fuzzy sets (T2-FSs) that include a footprint of uncertainty (FOU) (Mendel, 2001). It provides a measure of dispersion to capture more about uncertainties (Mendel, 2007). While T2-FSs have non-crisp MFs, T1-FSs have crisp membership grades (MGs) (John & Coupland, 2007).

A representation of the inference model for T2-FLS is depicted in Figure 1 (Mendel, 2007). It begins with fuzzification, which maps crisp points into T2-FSs. Next, inference engine computes the rule base by making logical combinations of antecedent T2-FS, whose results are implicated with consequent T2-FS to form an aggregated output type-2 fuzzy set. Afterwards, Type-Reduction (TR) takes all output sets and performs a centroid calculation of this combined type-2 fuzzy set, which leads to a type-1 fuzzy set called *type-reduced set*. That reduced set is finally defuzzyfied in order to obtain a crisp output (Mendel, 2001; Karnik & Mendel 2001). The computational complexity of this model is reduced if interval type-2 fuzzy sets are used (Mendel, 2001), it is convenient in the context of hardware implementation in order to make softer the computational effort and sped up the inference time.

Type-2 fuzzy hardware is a topic of special interest, since the application of T2-FLS to particular fields that demand mobile electronic solutions would be necessary. Some recent

An Embedded Type-2 Fuzzy Controller for a Mobile Robot Application 367

A FLC is a control strategy whose decisions are made by using a fuzzy inference system. Particularly, an IT2-FLC uses IT2-FSs to represent the inputs and/or outputs of the T2-FLS and these sets are completely characterized by their Footprints of Uncertainty (Mendel, 2001). So, this kind of FLC can model the uncertainty by means of the FOUs. In this way, uncertainties in sensors, actuators, operational changes, environmental changes, noise can be considered (Hagras, 2007; Mendel 2001). A FOU is described by its upper and lower MFs. Each MF is composed by as many membership grades (MGs) as discretization levels are considered in antecedents and consequents universes of discourse (Baturone et al., 2000). This work considers two alternatives for computing the MGs of IT2-FSs: memory based approach and function computing method, which are involved in fuzzyfier and inference engine stages. These two alternatives are the most commonly used in fuzzy hardware implementations (Baturone et al., 2000). Moreover, centroids and center of sets methods are considered for TR stage. As TR algorithm, the Enhanced Karnik-Mendel algorithm is used (Wu & Mendel, 2009). For the sake of clarity, figure 1 depicts the stage structure of an IT2-

**2. Hardware Implementation of an IT2-FLC for robot mobile applications**

FLC. These mentioned methods and alternatives are briefly introduced below:

several machine cycles depending on the complexity of MF (Baturone et al., 2000).

This method carries out a direct computation of MF by using numeric algorithms that avoid constructing and storing look up tables (Leottau & Melgarejo, 2010a). It reduces memory usage and facilitates the implementation of MF. However, its execution could require

This approach stores the MGs of every input value into a memory. This strategy is executed considerably fast, because it uses the input value as the pointer to the memory and to

The centroid TR combines all the rule-output T2-FSs using union and then finds the centroid of this T2-FS (Karnik & Mendel, 2001). Therefore, this method presents high accuracy but low computational performance (Mendel, 2001). Cent TR is chosen in order to obtain the best accurate output while the hardware platform is forced to the highest computational effort.

In center-of-sets TR, each type-2 consequent set is replaced by its centroid. Then, the weighted average of these centroids is found, being the weight of each centroid the degree of firing of it corresponding rule (Mendel, 2001). CoS TR is chosen because its convenient trade-off between accuracy and computational cost. It is more accurate than Heights method

The EKM algorithm is an iterative procedure to obtain the generalized centroid of an T2-FS. It uses statistically defined values as initialization points to reduce the amount of iterations that are necessary, in this way it converges monotonically and super-exponentially fast (Wu & Mendel, 2009). In addition, as it has been mentioned in (Leottau & Melgarejo, 2010a), EKM is the fastest algorithm reported up to that date for finding the generalized centroid of

*Function Computing Approach (FCA):*

*Memory Based Approach (MBA):*

*Centroid Type-Reduction (Cent):*

*Center-of-sets type-reduction (CoS):*

*Enhanced Karnik Mendel Algorithm (EKM):*

a inferred IT2-FS over DSC technology.

directly retrieve the MG (Leottau & Melgarejo, 2010a).

(Mendel, 2001) and computationally less expensive than Cent TR.

Fig. 1. Type-2 fuzzy system.

applications of T2-FLS have been developed in fields like robotics, communication and control systems among others (Castro et al., 2007; Hagras, 2007; Melgarejo & Peña, 2004, 2007; Torres & Saez, 2008). It is worth to think about the possibility of embedding T2-FLS handling these applications in order to achieve better communication speeds in smaller areas (Leottau & Melgarejo, 2010a).

Control and robotics are one of the most widely used application fields of fuzzy logic. The advantages of type-2 fuzzy logic controllers (T2-FLCs) over type-1 fuzzy logic controllers (T1-FLC) have also been demonstrated and documented in (Castro et al., 2007; Figueroa et al., 2005; Hagras 2004, 2007; Martinez et al., 2008; Torres & Saez, 2008). Although this kind of works presents improvements in the T2-FLCs performance, it is necessary to propose methodologies where these advances can be reflected in design processes as (Hagras 2008, Leottau & Melgarejo, 2010b; Melin & Castillo, 2003; Wu & Tan, 2004) and hardware implementation approaches over embedded devices as (Hagras, 2008; Leottau & Melgarejo, 2010a; Melgarejo & Peña, 2004, 2007). In this way T2-FLS would become widely used in different applicative contexts.

This work presents the implementation of an Interval Type-2 Fuzzy Logic Controller (IT2- FLC) for tracking the trajectory of a mobile robot application. The FLC is design according to the approach proposed in (Leottau & Melgarejo, 2010b), which involves some of the T1-FLC and T2-FLC properties. A hardware implementation of the designed T2-FLC is carried out over a digital signal controller (DSC) embedded platform. Different tests are performed for evaluating the performance of the IT2-FLC with respect to a T1-FLC. Simulation and emulation (i.e. with the embedded controller) results evidence that the IT2-FLC is robust to type reducer changes and exhibits better performance than its T1-FLC counterpart when noise is added to inputs and outputs. The IT2-FLC outperforms the T1-FLC in all tested cases, taking the interval time square error (ITSE) as performance index.

The chapter is organized as follows: Section 2 presents an overview of hardware implementation of IT2-FLCs. Section 3 describes the design approach followed for developing the IT2-FLC and section 4 describes its hardware implementation. Section 5 presents test, obtained results and discussion. Finally, conclusions and future work are presented in Section 6.

### **2. Hardware Implementation of an IT2-FLC for robot mobile applications**

A FLC is a control strategy whose decisions are made by using a fuzzy inference system. Particularly, an IT2-FLC uses IT2-FSs to represent the inputs and/or outputs of the T2-FLS and these sets are completely characterized by their Footprints of Uncertainty (Mendel, 2001). So, this kind of FLC can model the uncertainty by means of the FOUs. In this way, uncertainties in sensors, actuators, operational changes, environmental changes, noise can be considered (Hagras, 2007; Mendel 2001). A FOU is described by its upper and lower MFs. Each MF is composed by as many membership grades (MGs) as discretization levels are considered in antecedents and consequents universes of discourse (Baturone et al., 2000). This work considers two alternatives for computing the MGs of IT2-FSs: memory based approach and function computing method, which are involved in fuzzyfier and inference engine stages. These two alternatives are the most commonly used in fuzzy hardware implementations (Baturone et al., 2000). Moreover, centroids and center of sets methods are considered for TR stage. As TR algorithm, the Enhanced Karnik-Mendel algorithm is used (Wu & Mendel, 2009). For the sake of clarity, figure 1 depicts the stage structure of an IT2- FLC. These mentioned methods and alternatives are briefly introduced below:

#### *Function Computing Approach (FCA):*

366 Recent Advances in Mobile Robotics

applications of T2-FLS have been developed in fields like robotics, communication and control systems among others (Castro et al., 2007; Hagras, 2007; Melgarejo & Peña, 2004, 2007; Torres & Saez, 2008). It is worth to think about the possibility of embedding T2-FLS handling these applications in order to achieve better communication speeds in smaller

Control and robotics are one of the most widely used application fields of fuzzy logic. The advantages of type-2 fuzzy logic controllers (T2-FLCs) over type-1 fuzzy logic controllers (T1-FLC) have also been demonstrated and documented in (Castro et al., 2007; Figueroa et al., 2005; Hagras 2004, 2007; Martinez et al., 2008; Torres & Saez, 2008). Although this kind of works presents improvements in the T2-FLCs performance, it is necessary to propose methodologies where these advances can be reflected in design processes as (Hagras 2008, Leottau & Melgarejo, 2010b; Melin & Castillo, 2003; Wu & Tan, 2004) and hardware implementation approaches over embedded devices as (Hagras, 2008; Leottau & Melgarejo, 2010a; Melgarejo & Peña, 2004, 2007). In this way T2-FLS would become widely used in

This work presents the implementation of an Interval Type-2 Fuzzy Logic Controller (IT2- FLC) for tracking the trajectory of a mobile robot application. The FLC is design according to the approach proposed in (Leottau & Melgarejo, 2010b), which involves some of the T1-FLC and T2-FLC properties. A hardware implementation of the designed T2-FLC is carried out over a digital signal controller (DSC) embedded platform. Different tests are performed for evaluating the performance of the IT2-FLC with respect to a T1-FLC. Simulation and emulation (i.e. with the embedded controller) results evidence that the IT2-FLC is robust to type reducer changes and exhibits better performance than its T1-FLC counterpart when noise is added to inputs and outputs. The IT2-FLC outperforms the T1-FLC in all tested

The chapter is organized as follows: Section 2 presents an overview of hardware implementation of IT2-FLCs. Section 3 describes the design approach followed for developing the IT2-FLC and section 4 describes its hardware implementation. Section 5 presents test, obtained results and discussion. Finally, conclusions and future work are

cases, taking the interval time square error (ITSE) as performance index.

Fig. 1. Type-2 fuzzy system.

areas (Leottau & Melgarejo, 2010a).

different applicative contexts.

presented in Section 6.

This method carries out a direct computation of MF by using numeric algorithms that avoid constructing and storing look up tables (Leottau & Melgarejo, 2010a). It reduces memory usage and facilitates the implementation of MF. However, its execution could require several machine cycles depending on the complexity of MF (Baturone et al., 2000).

#### *Memory Based Approach (MBA):*

This approach stores the MGs of every input value into a memory. This strategy is executed considerably fast, because it uses the input value as the pointer to the memory and to directly retrieve the MG (Leottau & Melgarejo, 2010a).

#### *Centroid Type-Reduction (Cent):*

The centroid TR combines all the rule-output T2-FSs using union and then finds the centroid of this T2-FS (Karnik & Mendel, 2001). Therefore, this method presents high accuracy but low computational performance (Mendel, 2001). Cent TR is chosen in order to obtain the best accurate output while the hardware platform is forced to the highest computational effort.

#### *Center-of-sets type-reduction (CoS):*

In center-of-sets TR, each type-2 consequent set is replaced by its centroid. Then, the weighted average of these centroids is found, being the weight of each centroid the degree of firing of it corresponding rule (Mendel, 2001). CoS TR is chosen because its convenient trade-off between accuracy and computational cost. It is more accurate than Heights method (Mendel, 2001) and computationally less expensive than Cent TR.

#### *Enhanced Karnik Mendel Algorithm (EKM):*

The EKM algorithm is an iterative procedure to obtain the generalized centroid of an T2-FS. It uses statistically defined values as initialization points to reduce the amount of iterations that are necessary, in this way it converges monotonically and super-exponentially fast (Wu & Mendel, 2009). In addition, as it has been mentioned in (Leottau & Melgarejo, 2010a), EKM is the fastest algorithm reported up to that date for finding the generalized centroid of a inferred IT2-FS over DSC technology.

$$\mathbf{x}(t+1) = \mathbf{x}(t) + \cos[\varphi(t) + \theta(t)] + \sin[\theta(t)]\sin[\varphi(t)] \tag{1}$$

$$y(t+1) = y(t) + \sin\{\varphi(t) + \theta(t)\} - \sin[\theta(t)]\cos[\varphi(t)] \tag{2}$$

$$
\varphi(t+1) = \varphi(t) - \sin^{-1}[2\sin(\theta(t))/b] \tag{3}
$$

$$\begin{aligned} &\text{Rule } 0 \colon \text{if } Ex^- \text{ and } E\mathfrak{g}^- \text{ and } \Delta\mathfrak{g}^- \text{ then } \mathfrak{g}^+\\ &\text{Rule } 1 \colon \text{if } Ex^- \text{ and } E\mathfrak{g}^- \text{ and } \Delta\mathfrak{g}^+ \text{ then } \mathfrak{g}^-\\ &\text{Rule } 2 \colon \text{if } Ex^- \text{ and } E\mathfrak{g}^+ \text{ and } \Delta\mathfrak{g}^- \text{ then } \mathfrak{g}^-\\ &\text{Rule } 3 \colon \text{if } Ex^- \text{ and } E\mathfrak{g}^+ \text{ and } \Delta\mathfrak{g}^+ \text{ then } \mathfrak{g}^-\\ &\text{Rule } 4 \colon \text{if } Ex^+ \text{ and } E\mathfrak{g}^- \text{ and } \Delta\mathfrak{g}^- \text{ then } \mathfrak{g}^{++}\\ &\text{Rule } 5 \colon \text{if } Ex^+ \text{ and } E\mathfrak{g}^- \text{ and } \Delta\mathfrak{g}^+ \text{ then } \mathfrak{g}^+\\ &\text{Rule } 6 \colon \text{if } Ex^+ \text{ and } E\mathfrak{g}^+ \text{ and } \Delta\mathfrak{g}^- \text{ then } \mathfrak{g}^+\\ &\text{Rule } 7 \colon \text{if } Ex^+ \text{ and } E\mathfrak{g}^+ \text{ and } \Delta\mathfrak{g}^+ \text{ then } \mathfrak{g}^- \end{aligned} \tag{4}$$

An Embedded Type-2 Fuzzy Controller for a Mobile Robot Application 371

is carried out using Microsoft Excel® solver tool that uses the Generalized Reduced Gradient Method as non linear optimization algorithm (Frontline Systems, 2010). A screen

of used Excel® spreadsheet is shown in Figure 5.

Fig. 4. Reducing the dimensionality of an IT2-FS.

Fig. 5. A screen of used Excel® spreadsheet for MF parameters' tuning.

Once the T1-FLC parameters are tuned, returning to T2-FLS scheme using the same uncertainties defined in step 3.3 is proposed in (Leottau & Melgarejo, 2010b), but in this case, locating the FOUs over their embedded T1-FSs obtained in previously tuning. So, By

**3.5 Returning to Type-2**

The sources of uncertainty mentioned above are included in the final design. But, as it is mentioned in step 3.2 of this procedure, FOUs in this part of design, makes reference to linguistics uncertainties. This is the main advantage of use IT2-FS here, because the initial tuning would be based in some knowledge of the control process.

As it is proposed in (Leottau & Melgarejo, 2010b), it is used the middle of universe as center of antecedent sets location and the 10% of universe size as a measure for uncertainty. As universe size of ܧ߮οܧ isʹߨ, the FOU wide of this IT2-FSs is initially defined as 2ߨ ή ͳͲΨ ൌ ͲǤʹߨ and center of sets are located at zero. Please, see Figure 7.a for more clarity.

Fig. 3. Simulink model for the truck system.

The whole system is simulated using Matlab®, fuzzy Toolbox and Simulink. In addition, the IT2-FLC is designed and simulated with the IT2 Fuzzy Toolbox by the Tijuana Institute of Technology and Baja California Autonomous University (Castro et al., 2007). The Simulink model for the truck system is depicted in Figure 3.

As a figure of merit, the integral time square error (ITSE) is used (Dorf & Bishop, 1998). The objective of this example application is not to achieve an optimal performance in the controller. Thus, tuning parameters is carried out using the prove-error method, looking that the mobile robot follows a defined trajectory (see Figure 6).

#### **3.4 The reduction of dimensionality**

The reduction of dimensionality consists of converting the IT2-FLC into a T1-FLC. It is possible compressing the FOU in the IT2-FSs up to transform it in a T1-FS as is shown in Figure 4. The main objective of this procedure is making easier the tuning of fuzzy sets parameters (Leottau & Melgarejo, 2010b). The amount of parameters used for modelling a T2-FS is greater than for a T1-FS. In this way, tuning these parameters in a T1-FLC is easier and faster (Mendel, 2001).

In this application, the embedded T1-FS that are located in the middle of each FOU is used as a tuning start point. Once the IT2-FSs are converted to T1-FSs, the MF parameters' tuning 370 Recent Advances in Mobile Robotics

• Uncertainties in control outputs. Actuators characteristics could change with time. The mobile robots mechanics often are susceptible to wear or tear (Hagras, 2007). • Uncertainties associated with changing operational conditions (Mendel, 2007).