An introduction to Robot Web Services
Robot Web Services is designed after the "architectural style" REST [1]. Similar to "object oriented programming" principles, REST is a style with good and bad practices, and there is no "REST protocol specification". For more information, search in the internet with the keywords "REST" or "REST API". Robot Web Services uses HTTP as the application protocol. The cornerstones in HTTP are URL's and Verbs. An URL identifies a web page (for example, www.google.com) or an IO-signal (for example, 127.0.0.1/rw/iosystem/signals/DO1). In REST, a URL identifies a resource. The representation of the application data sent from the robot controller can be either in XML or JSON format. Robot Web Services do not return web pages, so formatting information is not available on how the data should be displayed in a web browser contained in the application data representations.
A URL can contain query parameters which are identified with the character '?'. For example, most of the resources in Robot Web Services supports the following query parameters:
- json=1 Returns JSON representation. The default is always XML
- debug=1 Returns more information to help debugging.
The URL "127.0.0.1/rw/iosystem?json=1" will request the representation of this resource in a JSON format. A HTTP Verb defines which method shall be executed on a resource. Each resource can support one or more HTTP verb. HTTP has a number of predefined verbs. The most important verbs are:
- GET: Retrieve a resource
- PUT: Create or update a resource
- POST: Update a resource
- DELETE: Delete a resource
GET does not change the state of the resource (idempotent) whereas DELETE, PUT and POST will change the state of the resource. The Robot Web Services expose a set of web APIs that can be consumed by any HTTP aware client using any programming language. The APIs return data either as XML or JSON which can be parsed using standard XML/JSON parsers. A simple way to check the available Robot Web Service is to use a Web browser and type the URL of the robot controller in the address field. Other "REST clients" tools can be used, such as cURL from http://curl.haxx.se/. In this document, each API description has a curl sample call, which is tested on windows only. To use the REST APIs, the client application should make a HTTP request and parse the response.
Clients are not required to poll for the state changes on resources, state changes are sent as events to the client. The Robot Web Services supports the Websockets protocol, WebSockets [2]. The clients should first subscribe the changes and then the changes are sent via Websockets to subscribers.
Pre-requisites
- Knowledge of Hypertext Transfer Protocol (HTTP)
- Knowledge of XML or JSON
- Programming library which can initiate HTTP requests and parse the response.
- A client such as a standard browser
Services
The Robot Web Services consists of a number of services and each service may have additional services or one or more resources. A subset of available services and resources are:
- Fileservice: Provides remote access to files and directories. Handles the transferring, create, remove and renaming of files and directories (similar to FTP service).
- Subscription service: Handles subscriptions of resources and sends events when the subscribed resources are updated. Subscription service uses "WebSockets" [2] for events. Websockets are defined in RFC 6455.
- Ctrl service: Handles robot controller global functionality, such as access to the controller clock, controller identification, performs restart etc.
- Users service: Handles registration of connected clients.
- RW: Handles RobotWare services, such as IO, RAPID, E-log, CFG, etc.
Mime Type
Mime type identifies the type of representation contained in a HTTP request or response using the "Content-Type" HTTP header. The client or the server can inspect the Content-Type header to determine what type of message (html,xml,avi movie,etc) it has received. The receiver can use this information to decide how to handle the message, whether to render the html or launch the movie player.
Robot Web Service currently use following content types:
- application/xhtml+xml For XML representation
- application/json For JSON representation
- application/x-www-form-urlencoded For "form data"
Authentication
All clients using Robot Web Services must logon with a user name and password. User names and passwords are defined in the robot controller User Authorization System (UAS). The Robot Web Services uses digest as the default authentication method, since it does not send the name and password in clear text. When an application connects and performs the first (unauthenticated) HTTP request, the HTTP server on the robot controller responds with a HTTP error "Unauthorized (401)". The requester must then send its credentials to be granted with a connection. When the connection is granted, the server also sends a cookie (ABBCX) to the client application. The ABBCX cookie is used by Robot Web Services to maintain a session (see section Session.). Client applications must use a HTTP client that supports digest authentication and can handle the cookies in order to access the Robot Web Services. The details are visible to the developer when using a browser or HTTP client library that supports digest authentication and cookies.
Form data
The client applications can update a resource by sending encoded data to the server in the URL, also known as "Form data". The content-type shall be of type application/x-www-form-urlencoded . The RFC 3986 defines on how the reserved characters will be encoded. Most libraries have a built in support for character encoding. The cUrl uses the content-type www-form-urlencoded as default for POST and PUT. An example of a cUrl call to set (POST) an IO-signal may look like:
curl –digest -u "Default User":robotics -d "lvalue=1" -X POST "http://locahost/rw/iosystem/signals/Virtual/DUNIT/sig1?action=set"
The Form data is comprised of name/value pairs where the name is separated from the value by a '=' character and each name/value pair is separated by an '&' character.
Representation
The Robot Web Services supports two types of representation XML and JSON. Currently form-data and subscription events are not supported in json format. The XML is the default representation. All resources in Robot Web Services follow the same pattern. When returning XML, the XML is in accordance with the XHTML syntax, which uses the following XHTML elements:
Element description:
| Element | Description |
|---|---|
| <?xml version="1.0" encoding="UTF-8"?> | XML declaration, resources are encoded as UTF-8 |
| <html xmlns="http://www.w3.org/1999/xhtml"> | Namespace is XHTML |
| <head> | All XHTML documents are divided into a header and body |
| <title> | Mandatory element in XHTML |
| <base> | The base URL for all relative URL's in the document |
| <body> | All XHTML documents are divided into a header and body |
| <div class="status"> | Defines the start of the error status |
| <span class="code" | The Robot controller error code |
| <div class="state"> | Defines the start of the resource state |
| <ul> | The state is defined in an unsorted list with zero or more list items |
| <il class="xyz-li"> | A list item of type xyz |
| <a href="abc/def" class="xyz" rel="abc"></a> | A list item can include zero or more links to other resources |
| <span class="nn">abc</span> | A list item can include zero or more properties |
| Class link attribute | Identifies the type of resource the link references to |
| Rel link attribute | Identifies the semantic of a link |
Resources can also be represented as JSON by using the query parameter json=1. The JSON is a bit more compact than XML and returns less overhead, but the amount of information is still the same. Robot Web Services JSON representation follows the HAL specification [5]. Resources are returned as Unicode UTF-8. The controller supports the Latin1 for the majority of text strings. Robot Web Services converts characters between Latin1 and UTF-8.
Class and rel attributes
When client requests for a resource, it returns the list of sub resources, such as
curl -digest -u "Default User":robotics "http://locahost/rw/iosystem/signals"
returns an XML response with more than one list item (xx-li).
The XML response may include something like:
<li class="ios-signal-li">
<a href="signal/s01" class="ios-signal" rel="self"></a>
<span class="value">1</span>
</li>
This example shows the use of the attributes class and rel. The following table provides more details about these attributes:
| Attribute | Description | List item | Link elements | Properties |
|---|---|---|---|---|
| class | Defines the type | optional | required | required |
| rel | Defines the semantics | n/a | required | n/a |
The class defines a type and the rel defines the semantics. The rel attributes are mandatory for links, the class attribute is optional. The class attribute is mandatory for link elements. A client program may use both class and rel depending on the needs.
Common rel attributes are:
- Self: Refers to a resource equivalent to the containing element
- Action: Refers to an action resource.
- First: Refers to first resource (page) in a list of resources
- Next: Refers to next resource (page) in a list of resources
- Prev: Referes to previous resource (page) in a list of resources
- Last: Referes to last resource (page) in a list of resources
A service may define additional rel attributes such as parent, devices, network etc.
The class attributes are unique for each service. The pattern "service-abc" is used, where service is an abbreviation of the service name. The class attributes are appended with "-li" when returning a list of resources. The "-li" element may include the whole resource or just a part of it. For most of the "-li" items, there is a link included, with rel=self, to the actual resource. A large list of items may need to be retrieved using multiple requests. If a list returns a link with a rel=next attribute, then there are more items to retrieve, and the client will use the below link to get more items.
The following request can be used to get the next signals:
curl -digest -u "Default User":robotics "http://locahost/rw/iosystem/signals?start=200&limit=100"
Limit is optional, each resource has a maximum limited number of resources that it can return in a list.
Below is a sample representation of JSON:
Error handling
Robot Web Services returns HTTP error codes according to RFC 2616, [3]. The client applications can get a more detailed description of the internal error codes used and will return the resource representation of the resource. Normally, no internal error code is returned in the body for successful HTTP return codes (20X). For example, a request of a non-existing IO-signal will return the HTTP error 400 (Bad Request). The XML returned body will look something like:
The internal status code -1073445879 means SYS_CTRL_E_INVALIDARG, "An argument specified by the client is not valid for this type of operation."
The service /rw/retcode returns a list of all the defined internal error codes with their description. It is also possible to get a specific error description by retrieving /rw/retcode?code={error-code}. A status messageg can be used to get more information about an error via "stack trace" in the robot controller. If something does not work as expected, the "stack trace" will help the help ABB engineers to search for the root cause.
Subscriptions
It is possible to subscribe on changes on various resources. Events are sent on a Websocket connection (the Websocket protocol is specified in RFC 6455). Other requests to the subscription server are done on the HTTP connection. A single client is allowed only with three connections; two connections for HTTP requests and other one for Websocket connection to receive events. A single client can create a maximum of two subscriptions (groups). The subscription service supports a maximum of 1000 unique resources for all clients. If a client has subscribed to 1000 resources, another client or the same client can only subscribe to these 1000 resources and not any other resource. The limit of 1000 resources is applicable only for low and medium priority subscription. For high priority subscription, the limit is 64 resources.
A client should first create a subscription by specifying the resources of interest via HTTP POST request. Once the subscription is created, the client can establish a Websocket connection to receive updates from the subscribed resources. The client must send the http-session and the ABBCX cookie as headers, while creating the Websocket connection. The client must also note that, the Websocket subprotocol should be configured as robapi2_subscription. Unsubscribing or updating the subscription resource list is done via HTTP request. HTTP DELETE method is used for unsubscribing and HTTP PUT method is used to update the resources. The priority for each resource can be specified during subscription. The resource priorities can be either 2, 1, or 0. The priority 2 means "High", priority 1 means "Medium", and priority 0 means “Low”. A “High” priority subscription is applicable only for persistent RAPID variables and IO signals.
Events for "High" priority subscriptions are sent to the client as soon as they occur. For "Medium" priority subscriptions, events are sent with a maximum delay of 200ms. For "Low" priority subscription, events are sent with a maximum delay of 5 seconds. To subscribe to resources on the controller, the client application needs to perform HTTP POST to "/subscription" URI with the HTTP request body containing the resources to be subscribed. The HTTP request body is in standard form data format and consists of three name-value pairs per resource.
- 1st pair: resources={identifier} e.g. resources=1
- 2nd pair: {identifier}={subscription-resource} e.g. 1=/rw/iosystem/signals/Virtual1/Board1/do1;state
- 3rd pair: {identifier}-p={0|1} e.g. 1-p=1
In the above pairs per resources, the {identifier} can be of any value, but it is recommended to use integer values such as 1, 2,3 etc.
The complete request body of the previous example to subscribe on a single resource "/rw/iosystem/signals/Virtual1/Board1/do1;state" and associate priority "1" to that resource is shown below:
Similarly, the payload for subscribing to two resources are:
To setup a subscription and start listening for events are as follows:
- Subscribe to resources - Response to this HTTP request is a list of initial events for the subscribed resources along with the location header. The value of the location header is the Websocket address.
- Retrieve the Websocket address from the location header.
- Configure the Websocket subprotocol as robapi2_subscription.
- Add the http-session and the ABBCX cookie to the Websocket request's header.
- Setup Websocket connection using the value returned from the location header.
- Receive and parse the events.
A pseudo-code for the above steps are:
The following XML is an example of a single event returned by the controller:
Asynchronous Operations
Some operations such as backup, compress etc. are long running operations, these operations take a long time to complete. To ensure that a client is not forced to wait for the request until the operation is completed, these long running operations are handled asynchronously. When a client triggers a long running operation, the operation immediately returns a status response (202 Accepted, see [Section 6.3.3 of RFC 7231](https://tools.ietf.org/html/rfc7231#section-6.3.3)) along with aLocationheader. TheLocationheader includes a progress URI with a unique identifier, these identifier triggers the long running operation(/progress/{id}). The client can poll for status of the long running operation by issuing a GET on the progress URI. The client can also use the URI in theLocation` header to subscribe for status change events instead of polling. In case subscription the status change events are sent via Websockets. Subscription to these events work exactly like regular subscription see Subscriptions for more details.
The following shows an example of handling asynchronous operation. The backup operation is a long running operation and hence is handled asynchronously.
Backup request
Response to Backup request
Once the above response is received, the client can poll for status change by using the following polling request:
Polling Request
Polling Response
Or subscribe to status change events by issuing the following subscription request:
Subscription Request
and listen on Websockets for status change events.
Note: A GET on /progress URI without an identifier returns the statuses of all long running operations in progress.
Session
A session is maintained using cookies (-http-session- and ABBCX). At logon the cookie is sent to the connecting application (see Authentication.). A single session can accommodate a maximum of three connections, that is two HTTP connections and one Websocket connection. Note: The HTTP/1.1 specification recommends the number of connections from an application to two connections per server. The first connection is established when the authentication is successful. To establish a second and third connection belonging to the same session, the client must send the same cookies (-http-session- and ABBCX) that it received during the authentication process. If the client tries to establish more than three connections per session (i.e. more than two HTTP connection or more than one Websocket connection), the connection is rejected and reported as "Service Unavailable" (503). A total number of 70 sessions are currently supported. If the total number of session exceeds 70, then creating a new session will return "Service Unavailable" (503). Also, a total of 5 sessions or 15 connections are allowed per IP address. When a session is broken or a client performs logout, any resources such as subscriptions, mastership etc. is released by Robot Web Services. A client can initiate an explicit logout by invoking a HTTP GET request to /logout URL.
Ref link https://docs.microsoft.com/en-us/dotnet/framework/network-programming/managing-connections
Detecting connection lost
The Websocket connection in Robot Web Services supports a custom ping/pong protocol to enable a client to detect if the Websocket connection to the controller is lost. The ping/pong protocol requires a client to send a "ping" message on the Websocket connection in frequent intervals (recommended every 30 seconds). The server always responds with a "pong" message to indicate the connection is alive. The client can detect if the Websocket connection is broken, when it fails to send the "ping" message or doesn't receive a "pong" message from the server within 1 second.
HTTP Connection
Detecting if a HTTP connection is alive requires the client to request a resource in frequent intervals (recommended 30 seconds). It must be noted that if a client has created a subscription and if an HTTP connection time out occurs then the client is automatically logged out and all subscription related information is removed. The client is then required to reconnect and re-establish the subscriptions.
Cookie Management
After successful authentication, the server sends a cookie back to the client. This cookie must be re-sent with all subsequent requests as well as when the client tries to establish a new connection belonging to the same session. Note that in some client libraries, the cookie management is done transparently. That is after authentication, the subsequent requests or new connection will automatically reuse the cookie received during the authentication process. Some client libraries require the developer to retrieve the cookie via an authenticated request and explicitly set the cookie on subsequent requests or new connections. Failing to send the cookie in the subsequent request will result in "Service Unavailable" (503) error being returned by the controller. An example of explicit cookie handling using the "requests" in a Python module is shown below:
Character encoding
URL encoding
URLs can only contain ASCII characters. When URLs contain characters outside the ASCII character sets, those URLs must be properly encoded. The unsafe characters should be percent encoded, see RFC3986 [6]. Example: The URL
http://127.0.0.1/rw/åäöUnit#21
must be encoded as
http://127.0.0.1/rw/%C3%A5%C3%A4%C3%B6Unit%2321
This is mostly handled internally by the HTTP library.
Resource encoding
Resources are encoded as UTF-8 and on the controller as ISO Latin1, as used for most of the text strings. This conversion is transparent for the client, the XML or JSON libraries will handle this. To emphasize, the Robot Web Services API use UTF-8 but there is no support for other characters on the controller outside the ISO Latin1 character encoding. A character which cannot be encoded from UTF-8 to Latin1 will be set to '?'.
Web pages and Web applications
In ~/HOME/docs the web application (like web pages,JavaScript etc.) can be stored. It is then possible to access the web page via the URL http://{ip}/docs/{web page}.
Virtual Controller (VC)
The VC (and RC) default socket port is 80, but can be changed to any number. To change the port number edit the appweb.conf file. When running multiple VCs on the same PC, then at least one of the VC port numbers must be changed.
Microsoft .Net and Robot Web Services
The .Net on Windows 7, does not have built-in support for WebSockets, there are other C# libraries which can be used, such as "webSocket-sharp" at "https://github.com/sta/websocket-sharp".
The RestSharp is another simple REST and HTTP API for C# which is easy to use, can be found at http://restsharp.org.
Guidelines to use HTTP error codes
Before trying to process the resource representation, a client should first check the returned HTTP status code to determine if the request was successful or not, and then handle the representation appropriately. Skip elements you don't understand, this makes it possible for the client to be compatible with new Robot Web Services releases. New XML elements or attributes may be added in future releases of Robot Web Services. Not that XML elements or attributes released by Robot Web Services should not be removed to provide backward compatibility
Discovery
Robot Web Services uses Bonjour for discovery. Bonjour is an open protocol for zero-configuration networking over an IP network. Discovering Robot Web Services requires a Bonjour client software to browse and look up the services. Several standard software components such as Safari, Bonjour Browser, and command line tools can be used to discover the services. A custom client for discovery can also be built using publicly available Bonjour client libraries. An example of discovery and lookup using Bonjour command line tools for Windows is shown below:
dns-sd -B _http._tcp,rws
Timestamp A/R Flags if Domain Service Type Instance Name
12:09:48.363 Add 3 11 local. _http._tcp. RobotWebServices_ABB_Testrack
12:09:48.364 Add 2 11 local. _http._tcp. RobotWebServices_Sys_1035
In the above example, the first entry is for a RC and the second is for a VC. Once the instance name is discovered, we can resolve the instance name to get the hostname or the IP address of RC/VC:
Resolve the above RC instance
dns-sd -L "RobotWebServices_ABB_Testrack" _http._tcp
12:11:10.857 RobotWebServices_ABB_Testrack._http._tcp.local. can be reached at ABB_Testrack.local.:80 (interface 11)
Resolve the above VC instance
dns-sd -L "RobotWebServices_Sys_1035" _http._tcp
12:12:27.451 RobotWebServices_Sys_1035._http._tcp.local. can be reached at IN-L-KBXI011934.local.:80 (interface 11)
Using the above information, the client can connect and use the service.
Notes:
- dns-sd is a software available from the Apple’s web site.
- VC discovery requires the Bonjour daemon to be running on the PC on which VC is running.
- RobotWare doesn't support IPv6. If the hostname of the VC or RC resolves to an IPv6 address, a 400 status code is reported with "IPv6 Address not supported" in the message body. A client then needs to send the IPv4 address instead. Use dns-sd to get the IPv4 address:
First lookup the name using the following commands:
dns-sd -L "RobotWebServices_Sys_1035" _http._tcp
Lookup RobotWebServices_Sys_1035._http._tcp.local 8:23:25.300 RobotWebServices_Sys_1035._http._tcp.local. can be reached at IN-L-KBXI011934.local.:8888 (interface 12)
Then use the looked up name as mentioned below:
dns-sd –Q IN-L-KBXI011934.local.
Timestamp A/R Flags if Name T C Rdata 15:19:26.307 Add 2 12 IN-L-KBXI011934.local. 1 1 10.140.188.52
use the above mentioned IP address to access the VC or RC.
Debugging
For debugging purpose, a standard web browser or browser plugins such as REST client can be used to inspect the responses from Robot Web Services. Most resources also support the debug=1 query parameter, which returns the additional information in the representation to enable easy viewing using the browser.
Tools which we have found helpful are:
- RESTClient: A simple to use test client, can be downloaded from https://code.google.com/p/rest-client/.
- Fiddler: It monitors the traffic between client and server, can be downloaded from http://fiddler2.com/.
- Wireshark: It monitors the network traffic, can be downloaded from http://www.wireshark.org/.
System Level Specification
In RWS, some of the key system level specifications are as below:
RequestTimeout
- RequestTimeout is 25 minutes.
- If RWS takes more than 25 minutes to server client request, then client session will be removed after serving the request.
InactivityTimeout
- InactivityTimeout is 5 minutes.
- If there is a gap of 5 minutes in between two consecutive requests, then client session will be removed.
API performance
- Except few APIs (like load RAPID program, search RAPID symbol, upload file etc.) RWS generally takes less then 50ms to serve one API request. So it is recommended to make API request at an interval of 50ms or maximum 20 requests per second.
Maximum data packages
- RWS supports less then 100KB data as input parameter, except file operation. Maximum 800MB of file can be uploaded using RWS.
Maximum sessions
- Maximum session which can be created in controller is 70.
- Maximum 19 sessions can be created if all the sessions have subscripon with web-socket connection.
Maximum HTTP connections per session
- Maximum number of HTTP connection per session is 2.
- If client makes more than 2 connections per session, then the 3rd connection will be closed gracefully.
Maximum webSocket connections per session
- Maximum number of websocket connection allowed per sesion is 1.
Maximum connections per IP
- Maximum number of conections per IP allowed is 15.
References:
- http://en.wikipedia.org/wiki/Representational_state_transfer
- http://en.wikipedia.org/wiki/WebSocket
- http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html.
- http://embedthis.com/products/appweb/doc/guide/appweb/users/configuration.html
- http://stateless.co/hal_specification.html
- http://www.ietf.org/rfc/rfc3986.txt