External Source Integration Guide

Vantiq supports the integration of external data sources into the automation service. A source may supply data or be a recipient of notifications produced by Vantiq or both supply data and be a recipient of notifications. At the present time Vantiq supports following source types:

A source implements one or more of the following capabilities:

Source

Sources are defined by creating a source resource that represents the definition of the source and submitting the definition to Vantiq for registration. Once registered, the source will immediately begin reading and processing messages if a stream is configured and will also immediately be available for query and publish operations.

The specific details of the ArsSource properties required are described in the separate documentation for each source class:

The general representation of a source resource is defined in Resource Reference Guide: Source. In summary, a source contains the following properties:

Source Operations

As described above sources can support up to 3 operations, all of which operate on data in the form of “messages”. The operations and message formats supported are specific to each source implementation and described in the source specific documentation. Here we describe the general capabilities.

Content Types

Vantiq supports 3 distinct content types for data sent and received by sources:

Each source implementation is responsible for transforming data sent and received between the external form supported by the source and one of these 3 content types supported by the Vantiq system. For example, an Email source may accept a JSON document which describes the elements of an Email message and transform that document into the actual SMTP message.

Streaming Data

A stream of information is read from the external system using either an asynchronous model or a synchronous (polling) model depending on the capabilities of the external system. The process stream method must run continuously and may not terminate until the source is shut down. For example, the MQTT source type asynchronously reads messages when they are published to the corresponding MQTT topic while the REMOTE source periodically polls the specified REST endpoint for new data.

When a message arrives on a streaming source an event is generated which will be delivered to any subscribed rules for processing. The event may also be delivered to any web socket clients which have subscribed to the event’s id.

The ultimate result of processing is usually, but not always, a change to the state of the automation model that may trigger additional rules.

If load balancing is required on the MQTT or AMQP stream it is best if the processing rule is stateless and not use internal state. Stateful data can be placed in the automation model so that the change can dispatch rules across the server cluster. It is also practical to partition the work across servers by assigning messages to different servers.

Query

A query can be presented to sources that support queries to obtain data from the external system. A query is executed by invoking the query function from the body of a rule. The query method is provided with the source type specific object that defines the query to execute. For example, the REMOTE source type that supports REST endpoints is invoked with an object that defines the URL, headers and body for the REST request that obtains the data from the remote endpoint.

Notify

A source can send data or notifications to the external system or, perhaps more appropriately, via the external system since the ultimate receiver of the notification may be a person. Examples of notify method uses:

The notify function is invoked from a rule to send data or notifications via a source.