Abstract
This work presents a methodology for the development of Teleoperated Robotic Systems through the Internet. Initially, it is presented a bibliographical review of the Telerobotic systems that uses Internet as way of control. The methodology is implemented and tested through the development of two systems. The first is a manipulator with two degrees of freedom commanded remotely through the Internet denominated RobWebCam (<A HREF="http://www.graco.unb.br/robwebcam">http://www.graco.unb.br/robwebcam</A>). The second is a system which teleoperates an ABB (Asea Brown Boveri) Industrial Robot of six degrees of freedom denominated RobWebLink (<A HREF="http://webrobot.graco.unb.br/">http://webrobot.graco.unb.br</A>). RobWebCam is composed of a manipulator with two degrees of freedom, a video camera, Internet, computers and communication driver between the manipulator and the Unix system; and RobWebLink composed of the same components plus the Industrial Robot. With the use of this technology, it is possible to move far distant positioning objects minimizing transport costs, materials and people; acting in real time in the process that is wanted to be controller. This work demonstrates that the teleoperating via Internet of robotic systems and other equipments is viable, in spite of using rate transmission data with low bandwidth. Possible applications include remote surveillance, control and remote diagnosis and maintenance of machines and equipments.
Telerobotic; internet; teleoperating; robotics; mechatronics
Telerobotics: Methodology for the Development of a Through-the-Internet Robotic Teleoperated System
A. J. Álvares
L. S. J. Romariz Jr.
GRACO Automation and Control Group
Mechanical Engineering Department
University of Brasilia
70910-900 Brasília, DF. Brazil
This work presents a methodology for the development of Teleoperated Robotic Systems through the Internet. Initially, it is presented a bibliographical review of the Telerobotic systems that uses Internet as way of control. The methodology is implemented and tested through the development of two systems. The first is a manipulator with two degrees of freedom commanded remotely through the Internet denominated RobWebCam (http://www.graco.unb.br/robwebcam). The second is a system which teleoperates an ABB (Asea Brown Boveri) Industrial Robot of six degrees of freedom denominated RobWebLink (http://webrobot.graco.unb.br). RobWebCam is composed of a manipulator with two degrees of freedom, a video camera, Internet, computers and communication driver between the manipulator and the Unix system; and RobWebLink composed of the same components plus the Industrial Robot. With the use of this technology, it is possible to move far distant positioning objects minimizing transport costs, materials and people; acting in real time in the process that is wanted to be controller. This work demonstrates that the teleoperating via Internet of robotic systems and other equipments is viable, in spite of using rate transmission data with low bandwidth. Possible applications include remote surveillance, control and remote diagnosis and maintenance of machines and equipments.
Keywords: Telerobotic, internet, teleoperating, robotics, mechatronics
Introduction
TeleRobotic using Internet as communication link is a new research field that grows in the area of Teleoperation, having many research groups acting in this promising area of Telematic (Monteiro et al., 1997), mainly in function of the low access costs to Internet. One of the first applications of TeleRobotic through Internet was the system developed by Taylor & Trevelyan (1995) in the University of Western Australia in 1994. This system is constituted by an Industrial Robot being controlled through Interface WWW (World Wide Web), allowing the manipulation of objects with the use of a claw. This system can be accessed through the address http://telerobot.mech.uwa.edu.au. The Figure 1 presents the interface with the user of this system.
Starting from this system, countless others were developed all over the world by several research groups. A list of some applications can be found in http://dir.yahoo.com/Computers_and_Internet/Internet/Interesting_Devices_Connected_to_the_Net/Robots. Among these, the teleoperation of Movable Robots (Klaus et al., 1997 e Hirzinger et al., 1997), WebVideo and Manipulator (Álvares & Romariz, 1998 e Wolf & Freitzhein, 1997), Manipulators, Televigilance (Almeida et al., 1995) can be found.
TeleRobotic can be defined as being an area of Telematic and Robotics aimed at teleoperating of robotic systems using a communication link (Klauss et al., 1997). A new terminology is being used in the case of Communication Internet's Net as link of telecommunications. In this case, it is designated systems World Wide Web Robots, Web Robots or simply Internet Robots. This terminology is valid for applications that use Industrial Robots, Manipulators, Pan-Tilt, Machines of Numerical Control and other similar industrial equipments.
It is desirable to control and monitor remote machines, centralizing its supervision, making possible sharing personnel and material resources and minimizing costs. Another advantage is associated to the location of the application that can be in the same industrial plant or even in other country or continent, making it possible to connect to the services available through Comunication Internet's Net.
The use of the Internet as link of telecommunications, in Telerobotic, attains the decrease of communication costs, turning to be achievable applications returning to technical support, to maintenance and repairs, and the control of machines and robots in places where previously was economically unviable. Typical applications of teleoperation in the past included hostile atmospheres (desert, out space, deep waters, among other) and tedious works (inspection of pipelines, train lines, etc.).
Now, with communication links through Internet, Modem and/or Radio can settle down a cheap connection to any part of the world to a Computer Server of Industrial Tasks that can dispatch a series of services for the application in subject.
Industrial plants, therefore, can be monitored and supervised remotely at an extremely low cost (Monteiro et al., 1997). Typical tasks such as bad-operation, telediagnoses and telemaintenence can be executed directly from the supplier office or even from another branch located thousands of kilometers away.
Methodology: Teleoperated Systems Through Internet
Teleoperated Systems remotely need data or images from the control object, as well as, the transmission of commands through a communication link, that is based upon connection through Communication Net (Wolf et al., 1997)or through the Protocol Internet (IP).
The proposed methodology, implemented and tested for TeleRobotic using Internet as communication link, is based on the Architecture Client/Server using the Protocol HTTP (Hypertext Transfer Protocol) through a conventional WWW Server (CERN, NCSA or APACHE) that disposes a multimedia interface. This can be accessed through a Client WWW (browser) like Netscape, Arena or Internet Explorer (Eckel & Hare, 1995). The proposed architecture of the system is presented in the figure 2.
As examples of teleoperated equipments are: manipulators with several degrees of freedom, video cameras, pan-tilt, industrial robots, welding machines, numeric command machines, programmable logical controllers, toys, among others.
These equipments are connected to WWW server, usually, through a serial interface, parallel, owner or even, through Local Net of Ethernet Communication. The WWW Browser is connected to WWW Server via the protocol TCP/IP (Transport Control Protocol/Internet Protocol) being able to use dialed connection (115 kbps) or a dedicated line T1 or T3 with 1,4 Mbps and 45 Mbps, respectively.
The client interacts with WWW Server using the HTML programming language (Hypertext Markup Language). The sent or requested data by the client use the mechanism of petition/answer of the HTTP Protocol, disposed for the server. The asked/sent data for the user are codified in URI/URL (Universal Resource Identifier/Uniform Resource Locator) and sending to HTTP Server.
WWW Environment
The server extracts the information of URI, more specifically of the URL; it processes them and an answer HTTP comes back. An URL is a subset of an URI, being the address of a file accessed via Internet (http://www.whatis.com/url.htm).
The HTTP Server uses a resource called CGI (Common Gateway Interface). Through this extension of the (Eckel & Hare,1995), it is possible to execute a program in "C" or Perl, or in other language or script, to accomplish a certain task. For instance, a program can be especially compiled to be used in a WWW Server, that controls the step motor actuator through the connection of a motor power driver to the parallel interface of the server (Álvares & Romariz, 1998). The figure 3, presents the mechanism CGI, where URI appears for the executable program. The figure 4 presents a part of the code source of this program in "C" language to control the step motor actuator.
WWW Server: WebCam Server and Server of Industrial Teleservices (WebRobot)
HTTP Server (WWW) should, preferentially, be based on platform Unix that makes possible larger robustness, flexibility, modulate and even the need of machines with smaller processing capacity, when compared to the Windows NT environment. The operating system Linux in platform PC (Personal Computer) is an extremely attractive option satisfying all the requirements of the teleservices applications for robot teleoperation.
In this context, HTTP Server should dispose two types of basic services, which are shown in the figure 5 through two modules:
. WebCam: Visualization of the teleoperated object, through video and/or image;
. WebRobot: Disposing teleoperated object's remote control functions.
Webcam server: visualization of the teleoperated object
This module is responsible for the capture of images through cameras CCD (Charge Coupled Device) using the technology WebCam (Álvares & Romariz, 1998) or WebVideo (Wolf & Froitzheim, 1997). Static images can be acquired in several formats as GIF (Graphics Interchanging Format) and JPEG (Joint Photographic Experts Group). The favorite format, due to the compression of obtained data is JPEG. Dynamic images, in the format of digital video, can be used with or without data compression. Among the used patterns are: MPEG (Moving Picture Experts Group), UYVY, Real Video, M-JPEG (sequence of images JPEG), CellB (Cell) and CuSeeme (Sun, 1994), (Connectix, 1996). The video system (capture board and software) should capture, digitalize and compress a video sign NTSC or PAL non modulated (composed or S-video). The compressed video then can be stored in disk or transmitted through net for the customer in a certain compression pattern.
In the case of using sequence of images JPEG, the compression rate is very low when compared with MPEG or Real Video. To main advantage is that the video images can be sent using HTTP Server's mechanism server-push directly to the WWW client, like Netscape, without the need to use a special software or a plug in to receive the video format (Otsuka, 1996).
To use the compression, it is necessary a specific software (plug in) that should be installed by the client and adjusted to the environment to receive images in the specified format, usually, MPEG or Real Video (Melchiors, 1996).
In relation to the image capture hardware can be used: camera of videos (NTSC or PAL) connected to a board for frame grabber (Sun, 1994); or cameras with board for image capture with CCD using the parallel interface as video entrance for the personal microcomputer, as in the case of QuickCam (Connectix, 1996). The disadvantage of the use of this hardware is associated to the smallest quality and the low speed of obtained capture images, due to the fact that the parallel interface restrictions speed.
Webrobot server: functions of teleoperated object's remote control
In order to teleoperate the robotic system is necessary to have as pre-requirement some capacity of remote communication using a protocol of open communication through an interface serial, parallel or even of an Ethernet, allowing the connection to a Microcomputer or a Workstation. Using this capacity is possible that any industrial equipment can be teleoperated through Internet.
Starting from this approach, a robotic server, denominated WebRobot, connected to the industrial equipment through the equipment communication interface should be available. This server can be a HTTP server, in a similar way to the described for WebCam (figure 5).
The access mechanism to the teleoperated object functions is also based on programs CGI and HTML. For each function disposed by the robot communication protocol there is a program CGI that is accessed in WebRobot server by the client using a WWW browser. The same equipment can be used for dispose both services: WebCam and WebRobot. On the other hand, it also can be used two or more servers to accomplish the functions of WebCam and WebRobot.
A second approach is to use a single WWW Server connected to one or more PC servers through sockets TCP/IP. These PC servers are not necessarily servers WWW. They are specific servers for robot control and for image capture, and WWW Server, pondering all the actions, accomplishes the whole treatment of the obtained information. The figure 6 presents this approach. In this configuration all the requested information go by WWW Server that establishes the communication through Communication Local Net (Intranet) through sockets TCP/IP using programs CGI.
Restrictions Of The Protocol Internet (TCP/IP)
The Internet presents a width of heterogeneous band with variable rates of transmission that vary from 10 kbps (connection wireless) to more than 10 Mbps in local net, depending on the Internet connection and the traffic.
With this limitation of bandwidth, applications in real time for video capture present serious restrictions. To overcome these restrictions, it is necessary to use data compression and connection to a great speed Internet. Typical rates of video transmission with compression need 20 Kbps (Real Video), and without compression, of 100 Kbps (sequence of images JPEG) with 5 frames/sec (Melchiors,1996).
Another limitation is the delay inherent to the protocol TCP, because the packages sent are not necessarily in the same order of the packages received by the client, what is not desirable for applications in real time.
This restriction can be solved adding some degree of autonomy to for the application, in the case a Robot, in such a way to outline emergency situations, as well as, dangerous situations. This autonomy level is obtained locally in Web Robot Server, which should monitor these limit condition (Hirzinger et al., 1997).
In spite of the datagrams with control information sent for the Robot by the client through Protocol TCP/IP be small, of the order from 2 to 3 Kbytes, the TCP Protocol doesn't guarantee applications in real time. An alternative for the future is to use the RTP Protocol (Real-Time Transport Protocol) for applications in real time (Otsuka, 1996).
Interface With The User (GUI Graphical User Interface)
The graphic interface with the user should be based on the HTML programming languages, JavaScript and JAVA (Otsuka, 1996). The use of JAVA allows an independence of hardware architecture, because the applet written in JAVA (application JAVA) it can be executed in any platform with a WWW browser that has support for this language.
The program in JAVA/HTML allows the client to have a friendly interface with the teleoperated system accepting commands and running the CGI programs necessary to execute the disposed functions and presenting the necessary information for the user, which is received by the server (figures 7 and 8).
The visual feedback is made through WebCam Server that presents a friendly interface beside the client that receives the images in compressed format (MPEG or Real Video) or in a sequence of images JPEG.
The figures 7 and 8 present some examples of interfaces with users, that were developed in teleoperation projects executed in GRACO. The figure 7 shows the RobWebLink system interface (http://webrobot.graco.unb.br) and the figure 8, the RobWebCam system (http://www.graco.unb.br/webcam).
Another approach is the use of VRML (Virtual Reality Modeling Language). VRML is a description language of interactive simulations with several participants communicating via Internet (Otsuka, 1996). The objective of the development of this language is to become a standard for the description of interactive simulations in WWW, such as; the HTML language is the standard for description of web pages. Its main characteristic for telerobotic applications is associated to the capacity of operating in connections with small bandwidth, what turns it a candidate in potential to be used.
Validation Of The Methodology - Developed Applications
Two applications were developed using this methodology in GRACO: RobWebLink (figure 7) and RobWebCam (figure 8).
RobWebCam System (http://www.graco.unb.br/robwebcam)
The RobWebCam system is constituted by a manipulator with 2 DOF (Degree-of-Freedom) that supports a video camera, and it is moved by step motors. These motors are controlled through a circuit board printed, which contains the driver power and the module of electric feeding of the equipment. The board communicates with the manipulator's server (WebRobot) through the parallel interface. This server, WebRobot, contains the control programs. The camera (SunVideo) possesses its own power and it is interlinked to server WebCam through a cable of video signal connected to the video capture board server. In Server WebCam are allocated the driver (programs in cgi) to receive the images and the WWW pages used for teleoperating the system. The client, via browser (figure 8), receives the images and the commands of this server through Internet. This architecture is described thoroughly in Álvares & Romariz (1998). The system can be accessed through the address http://www.graco.unb.br/robwebcam.
RobWebLink System (http://webrobot.graco.unb.br)
Starting from the success of the RobWebCam architecture, the implementation carried on the development of a second system denominated RobWebLink (figure 7). This system allows the IRB 2000 Asea Brown Boveri Industrial Robot's remote control with six degrees of freedom using Internet as way of control. The IRB 2000 controller has incorporated a remote control system through the serial RS-232 interface based on 42 functions, using an owner communication protocol.
Starting from this capacity of remote communication, it was developed a WWW server (WebRobot) connected to the robot via serial interface, disposing the remote control robot, through the 42 functions, in communication net using the protocol TCP/IP.
The remote control operation is monitored through image capture in real time, using the RobWebCam system (WebCam Server), besides information and status of the Robot sent by its controller. The system of developed teleoperation is available in the address http://webrobot.graco.unb.br and it can be remotely operated allowing the communication between WWW server and Robot's Controller.
The speed of data transmission requested by the service for commanding Robot functions is low, not committing the teleoperated system, notwithstanding inherent Internet latency. However, this latency makes the graphic feedback, through video on-line, be viable when it is used transmission speeds above 64 Kbps. It is also recommended the use of data compression. The system is in operation since May of 1998.
For safety reasons, access is only allowed to properly registered users. So, WebRobot Server requests password, (user properly registered), as well as, due to the Robot's replay and WWW Server that can be degraded in function of communication delay between WWW server and the client.
On the other hand, services that need of a small bandwith such as: initialization of the Robot, starting a program, to call and to turn off the robot, download and upload of programs, among others, are fully satisfied. The evolution of the system is foreseen by applying the same methodology to control through Internet a Welding Flexible Cell.
Conclusions
The work locates in the teleoperation field, presenting a methodology for the development of Robotic teleoperated systems via Internet. The methodology was tested through the development of two systems that are available in http://www.graco.unb.br.
The RobWebCam system was validated proving its functionality being in operation since September of 1997. Some optimization possibilities were observed, drawing from analyses of the system evolution (Álvares & Romariz, 1998). The built manipulator presents low noise in the accomplished displacements and it possesses reduced dimensions when compared to the other systems of camera positioning available in the market. Besides, the costs were quite reduced.
As characteristic application areas of the system are had: surveillance of establishments, visualization of work environment, visualization of commercial environment for propaganda, visualization of streets or tourist points, among others. One of the great difficulties to the teleoperation technology through Internet, is the large time of waiting in function of the width of existent band in the network, in other words, it is still considerable the interval of time among the sent command, the positioning effectiveness and the visualization of the new monitored situation, what commits applications in real time that request larger reliability and safety, as in the case of the RobWebLink system.
The RobWebLink system presented good results concerning to implement functions directly related to the Robot's control through its communication protocol, for requesting files from 1 to 2 Kbytes. However, the feedback through video demands larger bandwidth for applications in real time. An option would be, the use of Virtual Reality instead of using video sign, through VRML VRML (Otsuka, 1996).
The technology of TeleRobotic via Internet is still incipient and is having a great development by virtue of the great progresses obtained in the data compression technology and in the great pulse that is giving to the Web TV technology and Video Conference VRML (Otsuka, 1996). Certainly, these technologies associated to largest Internet bandwidth will make possible applications in real time in near future, with fewer restrictions than it is had nowadays.
Presented at COBEM 99 - 15th Brazilian Congress of Mechanical Engineering, 22-26 November 1999, São Paulo. SP. Brazil. Technical Editor: José Roberto F. Arruda
- Almeida, R., Almeida, F. and Carvalho, R., 1995, "Sistema de Televigilância", ISR Project Report.
- Álvares, A. J. & Romariz, L. J., 1998, "Desenvolvimento de um Manipulador com Dois Graus de Liberdade Controlado Remotamente Via Internet", V Congresso de Engenharia Mecânica Norte e Nordeste, Fortaleza, 27-30 de Outubro, pp. 529-536.
- Connectix, 1996, "QuickCam Color - Guia do Usuário".
- Eckel, G. & Hare, C., 1995, "Building a Linux Internet Server", New Riders Publishing, Indianapolis.
- Hirzinger, G. et al, 1997, "Teleoperating Space Robots - Impact for the Design of Industrial Robots", IEEE International Symposium on Industrial Electronics, Portugal, July 12-16, pp. 250-256.
- Melchiors, C., 1996, "Sistemas Interpessoais de Videoconferência (mbone)", http://www.penta.ufrgs.br/~cristina/mbone/ti/indiceti.htm
- Monteiro F. et al, 1997, "Teleoperating a Mobile Robot - a Solution Based on Java Language", IEEE International Symposium on Industrial Electronics, Portugal, July 12-16, pp. 263-267.
- Otsuka, J., 1996, "Fatores Determinantes na Efetividade de Ferramentas de Comunicação Mediada por Computador no Ensino à Distância", http://penta.ufrgs.br:80/pesquisa/joice/joice_ti.html
- Schilling, K., Roth, H. and Lieb, R., 1997, "Teleoperations of Rovers - From Mars to Education", IEEE International Symposium on Industrial Electronics, Portugal, July 12-16, pp. 257-262.
- Sun, 1994, "Sun Video User's Guide".
- Taylor, K. & Trevelyan, J., 1995, "A Telerobot on the World Wide Web", National Conference of the Australian Robot Association, Melbourne, July 5-7, http://telerobot.mech.uwa.edu.au
- Wolf, H. & Froitzheim, K., 1997, "Webvideo - A Tool for WWW Based Teleoperation", IEEE International Symposium on Industrial Electronics, Portugal, July 12-16, pp. 268-273.
Publication Dates
-
Publication in this collection
21 Aug 2002 -
Date of issue
May 2002
