Introductory Tutorial

Tutorial Overview

This tutorial guides a new developer through simple, yet complete, lessons in the use of the VANTIQ system. It uses an example from the Internet of Things (IoT): processing engine sensor data to detect and diagnose an overheating engine. The lessons include the definition of data types to record sensor data then adding an app to process that data and to correlate sensor data to provide overheating alerts. The tutorial also includes lessons on how to generate simulated sensor data and to monitor the results generated by that data in real-time.

All lessons make use of Modelo, VANTIQ’s web-based developer toolset. Modelo has two modes of operation, Development and Operations, each of which can have multiple projects. The lessons in this tutorial use Development mode, which is selected at the top of the Modelo page. In addition, when following the lessons, please make sure you are logged in as the administrator of the VANTIQ namespace to which you are assigned.

1: Creating an Engine Monitoring Project

The first task in building our engine monitoring system is to create a project in VANTIQ Modelo to assemble all the system components.

Use the Projects button, select Create New Project and title the project “EngineMonitor”:

        EMProject

The rest of the lessons take place inside this Modelo project.

2: Creating Data Types

The simulated engine for this example includes two sensors: temperature and speed. These two sensors each produce a stream of data which is stored in the VANTIQ database. The first task is to define the data types for that sensor data.

Use the Add button to select Type…:

        AddType

Then use the New Type button to create the temperature sensor type:

        EngTempDef

The EngineTemp type contains two properties:

Both systemId and temperature are required to make sure the sensor sends well-formed data. Once these two user properties are defined, use the Save button to save the EngineTemp type.

Use the Add button to select Type… again then the New Type button in the same manner to create a speed sensor type, EngineSpeed, with two properties:

Both speed and systemId should be marked as required properties.

Use the Add button to select Type… then the New Type button a third time to create an overall system status type, SystemStatus, which is used to correlate the most recent properties of engine temperature and speed:

systemId must be designated as a Natural Key because we only want one SystemStatus record for each systemId value (i.e. only one SystemStatus for each simulated engine). Use the Natural Key link on the Editing Type page to designate Natural Keys. Further, because systemId is a Natural Key, it must also be added as an Index for the type. Use the + button next to the No Indexes are defined label on the Editing Type page to add systemId as an Index. When adding systemId as an Index, check the Is Index unique? box to ensure that only one SystemStatus record is created for each systemId value.

        CreateSysStat

Use the Add button to select Type… then the New Type button a fourth time to create a type, SystemHUD, which could represent a heads-up display to alert the user to an engine overheating condition:

systemId must be designated as a Natural Key because we only want one SystemHUD record for each systemId value (i.e. only one SystemHUD for each simulated engine). Further, because systemId is a Natural Key, it must also be added as an Index for the type. Use the + button next to the No Indexes are defined label on the Editing Type page to add systemId as an Index. When adding systemId as an Index, check the Is Index unique? box to ensure that only one SystemHUD record is created for each systemId value.

The systemId property is a unique identifier that is present in all four types and allows the sensors to be associated with one system (e.g. car or industrial machine).

Once the four types have been created, the Project Resource Graph will look similar to this:

        TypesGraph

Use the Save button at the top, left of Modelo to save the project.

3: Simulating Sensor Data

Now that you’ve created the data types, use Modelo to simulate a running system. This lesson uses Modelo’s Data Generators feature to simulate engine temperature and speed data streams.

Use the Show button to select Data Generators.

        DataGenTab

Use the Create New icon button (small plus sign in a circle at the top, right of the Data Generators pane) to create our engine temperature sensor generator:

        TempStream

Select the EngineTemp Data Type since you want to create a stream of EngineTemp sensor entries. Enter 100 as the Number of Inserts and 1000 as the Time between inserts. These properties cause the data generator to create 100 sensor readings, each separated by one second (1000ms). Enter Auto0123456789 as the systemId property.

Enter %R200:215:1% as the temperature. This instructs the data generator to step through a range of properties, starting at 200 and ending at 215, one degree at a time. Make sure you also select Step from the Variation menu. (For more information about the options available for data generation, please read the Data Entry Options section at the bottom of the Data Generation definition page.) Use the Save button to save the generator and return to the project.

Use the New Data Generator button again to create our engine speed sensor generator:

        SpeedStream

Select the EngineSpeed Data Type since you want to create a stream of EngineSpeed sensor entries. Enter 100 as the Number of Inserts and 1000 as the Time between inserts. These properties cause the data generator to create 100 sensor readings, each separated by one second (1000ms). Enter Auto0123456789 as the systemId property.

Enter %R35:55:5% as the speed. This instructs the data generator to step through a range of properties, starting at 35 and ending at 55, five miles per hour at a time. Make sure you also select Step from the Variation menu. Use the Save button to save the generator and return to the project.

You now have two data generators ready to stream simulated sensor data to the VANTIQ database. Use the Save button at the top, left of Modelo to save the project.

4: Creating an App

VANTIQ apps are constructed using the App Builder visual interface to represent the receipt and transformation of data in the VANTIQ system. In our example, the app will process the temperature and speed data, then merge that data into a SystemStatus record and update a SystemHUD record.

Use the Add button to select App…:

        AddApp

Then use the New App button to create an empty app. When the App pane is added, you will see a graphical editor with a single colored rectangle. Start by giving the app a name, EngineMonitor, in the right hand section titled App:

        NewApp

Next, click the colored rectangle to configure the first Event Stream task in the app. Every app has one or more Event Streams, which define the sources of inbound data that flow into the app. Event Streams can be triggered by inserts or updates on types, messages arriving from sources, or publishes to topics. In this example, we will have two Event Streams, one for inserts on the EngineTemp type, and another for inserts on the EngineSpeed type.

With the first Event Stream selected, you should see the box is hatched, indicating it is selected and the property sheet to the right applies to just this task. Give the stream a name, TempReading, and then click on the Click to Edit link to enter the Event Stream’s Configuration parameters. Every task needs to be configured, and each comes with a set of optional and required configuration properties.

        TempReadingConfig

Configure the TempReading task by selecting types as the inboundResource, and EngineTemp from the inboundResourceId dropdown menu. Additionally, since we only care about inserts on the EngineTemp type, we can select INSERT from the op dropdown menu.

Use the OK button to save the configuration parameters, then use the Save Changes icon button (down arrow at the top, right of the App: EngineMonitor pane) to save the first iteration of the EngineMonitor app. Use the Save button at the top, left of Modelo to save the project.

5: Testing Tasks

With one task implemented, we can test that our app is properly triggering by attaching a logger to the output of TempReading. To add a new task, right-click on the TempReading rectangle and select Link New Task from the menu.

        LinkLogger

Select LogStream from the Activity Pattern dropdown list, give it a name LogTemp, and click OK. For a complete list of the supported Activity Patterns and an explanation of their configuration parameters, see here.

        TempLogger

Once the LogTemp task appears in the app, select it and click the Click to Edit link to enter the LogStream’s Configuration parameters. For LogStream, the only option is the level with which to log the events, so specify info in the dropdown and click OK.

        LogTempConfig

With the LogTemp task fully configured, save the app again:

        PartialApp

To test the partially built app, we will look for log messages generated by the LogTemp task. Use the Debug button to select Live Log Messages, which displays the Log Messages pane. The Log Messages pane provide near real-time display of log messages generated by the app.

        TestApp1

To begin the test, use the Run icon button (small triangle in a square at the top, right of the Data Generators pane). As simulated temperature and speed sensor readings are generated, new entries appear in the Log Messages pane to indicate the LogTemp task is executed.

        TestApp3

Another helpful feature in the App Builder is Debug Mode, which is activated by using the Toggle Debug On icon button (small bug at the top, right of the App: EngineMonitor pane). Try toggling Debug Mode on, then running the data generators again. Badges appear over the TempReading and LogTemp tasks indicating each execution of the task.

        TestApp2

Use the Save button at the top, left of Modelo to save the project.

6: Updating SystemStatus

Now that the first event stream is working, it’s time to add the second. Right-click the TempReading task rectangle and select Add Event Stream from the menu. A new colored rectangle named NewStream appears in the App: EngineMonitor pane. Select it and give it a name, SpeedReading.

        SpeedReading

With the SpeedReading Event Stream selected, click on the Click to Edit link to enter the Event Stream’s Configuration parameters. Configure the SpeedReading task by selecting types as the inboundResource, and EngineSpeed from the inboundResourceId dropdown menu. Additionally, since we only care about inserts on the EngineTemp type, we can select INSERT from the op dropdown menu.

Next attach another LogStream task to SpeedReading named LogSpeed and configure it with a log level of info as well.

Use the Save Changes icon button (down arrow at the top, right of the App: EngineMonitor pane) to save the second iteration of the EngineMonitor app.

        PartialApp2

To test, use the Run icon button (small triangle in a square at the top, right of the Data Generators pane). As simulated temperature and speed sensor readings are generated, new entries appear in the Log Messages pane and badge counts are incremented to indicate both LogStream tasks are executed.

Now that both streams of data are reliably received, we need to merge them together in the SystemStatus type. Before doing this, we can clean up the app by removing the loggers once we’ve tested the Event Streams. Right click on each logger and select Delete Task, then save the app.

        PartialApp3

To merge the two streams, we need to add a Merge task to the app. Right click the TempReading task, and select Link New Task, just as you did before when adding a logger.

In the popup, select Merge in the Activity Pattern dropdown list, give it a name, UpdateSystemStatus, and click OK. Notice that the newly added node is a diamond and not a rectangle. This indicates that the merge task is a “decision” node (see the App Builder Reference Guide for more info).

Connect SpeedReading to the UpdateSystemStatus node by right clicking on the SpeedReading rectangle then selecting Link Existing Task. In the popup, select UpdateSystemStatus then click OK. The graph will redraw to place the Merge task between the two streams.

Select the Merge task and open its configuration popup. Notice that merge task has no options by default. Close the popup without specifying any options then add a logger named LogStatus to UpdateSystemStatus, configuring it with a log level of info, as with the previous LogStream tasks.

        PartialApp4

Save the app then run the data generators again to exercise the app. Notice that now the merge and logger tasks execute every time either event stream receives an event. Merge tasks take input independently from two parent tasks, which is useful for combining multiple streams of data into a single stream.

At this point we’ve merged the TempReading and SpeedReading streams, but we haven’t used the output to update the SystemStatus type. To do this, click the UpdateSystemStatus merge task and open the configuration popup. We want to save the output of the merge task in the SystemStatus type, so specify types as the outboundResource and SystemStatus as the outboundResourceId. Check the upsert box, which indicates we want to update existing SystemStatus records instead of creating new ones if a record already exists with the same natural key (systemId).

        MergeConfig

Once you’ve configured the merge task, save the app and data generators again. Use the Show button to select Find Records, select SystemStatus as the Type, then click Run Query. This displays a Find Record Results pane which should have a single record for systemId equal to Auto01234567890 and values for both the speed and temperature properties:

        FindSystemStatus

When new data comes in for an existing systemId, you’ll see those changes reflected in the corresponding SystemStatus record. Play around with this a bit and get a feel for the relationship between the EngineSpeed/EngineTemp types and SystemStatus.

Use the Save button at the top, left of Modelo to save the project.

7: Updating SystemHUD

To update the SystemHUD record for the simulated engine, we have to respond to each update to the engine’s SystemStatus record. This is accomplished by adding a new Transformation task to the app.

Right-click the UpdateSystemStatus task diamond and select Link New Task from the menu. Select the Transformation Activity Pattern and name it UpdateSystemHUD. A new colored rectangle named UpdateSystemHUD appears in the App: EngineMonitor pane.

Select the UpdateSystemHUD task and open its configuration popup. The Transformation task has one required parameter, transformation. Click the link next to the transformation property to display the Edit union parameter ‘transformation’ dialog:

        transformationConfig

The Union Name is Visual Transformation by default and we want to use the default since the data transformation is fairly simple. Use the + button next to the Add a Transformation label to add two transformations:

    event.systemId
    ((event.temperature >= 210) && (event.speed >= 45)) ?
    "Your engine is overheating, please reduce your speed." :
    (((event.temperature >= 210) && (event.speed < 45)) ?
    "Your engine is overheating: check for a malfunctioning fan or a coolant leak." : "")

Recall the output of the UpdateSystemStatus task was a record of type SystemStatus, which has three properties: systemId, speed, and temperature. Since the UpdateSystemHUD task follows the UpdateSystemStatus task, this means the UpdateSystemHUD task has as its input the properties associated with a SystemStatus type. This means we can reference each of its properties by prefixing them with event., e.g. event.temperature.

The purpose of the UpdateSystemHUD Transformation task is to update the SystemHUD record for the simulated engine, which means the transformation needs to account for the two SystemHUD type properties: systemId and alertMsg.

Setting the systemId is easy: it’s simply the value of the systemId property from the input SystemStatus type, so the Transformation Expression is event.systemId.

Setting the alertMsg property is more complicated since we want it to contain a text diagnosis based on the speed and temperature of the engine. The Transformation Expression checks the values of event.speed and event.temperature so the first part of the expression checks for a temperature >= 210 degrees Fahrenheit and a speed >= 45MPH and, if true, updates the heads-up display message (alertMsg) to contain “Your engine is overheating, please reduce your speed.” Further, the Transformation Expression checks for a temperature >= 210 degrees Fahrenheit and a speed < 45MPH and, if true, updates the heads-up display message (alertMsg) to contain “Your engine is overheating: check for a malfunctioning fan or a coolant leak.” Finally, if neither of the previous two conditions is true, the alertMsg property is assigned an empty string, indicating no engine problems.

We also want to save the output of the transformation task in the SystemHUD type, so specify types as the outboundResource and SystemHUD as the outboundResourceId. Check the upsert box, which indicates we want to update existing SystemHUD records instead of creating new ones if a record already exists with the same natural key (systemId).

        TransformConfig

With these four tasks (TempReading, SpeedReading, UpdateSystemStatus and UpdateSystemHUD), the EngineMonitor app is now complete:

        CompleteApp

Use the Save button at the top, left of Modelo to save the project.

8: Monitoring A Running System

This lesson uses Modelo’s Subscriptions feature to view changes in VANTIQ database properties to test whether the simulated data streams are processed correctly.

Use the Add button to select Advanced>Subscription…, then use the New Subscription button to create our heads-up display (HUD) simulator subscription:

        HudSub

Select the SystemHUD Data Type and the update Operation since you want to monitor updates to any SystemHUD entries in the VANTIQ database. Remember that the UpdateSystemHUD task in Lesson 7 updated an engine system’s SystemHUD entry whenever SystemStatus records were updated.

Enable both alertMsg and systemId so that both properties are displayed whenever a SystemHUD entry is updated. Use the Save button to save the subscription. The SystemHUD subscription is added to the Project Resource Graph:

        HudSubGraph

Click the Subscription: SystemHUD oval in the Project Resource Graph to display the Subscription: SystemHUD pane:

        RunningSubscription

When the Subscription:SystemHUD pane opens, it causes Modelo to register the subscription with the VANTIQ system. Whenever a systemHUD entry is updated (you defined one of those for systemId Auto0123456789), messages are displayed in that pane.

You now have a subscription running and ready to display the output of the simulated HUD. The upcoming Lesson 10 uses the Data Generators created in Lesson 3 to produce simulated sensor data and trigger the EngineMonitor app. You will not see any simulated data displayed in the Subscriptions pane until Lesson 10.

Use the Save button at the top, left of Modelo to save the project.

9: Visualizing a Running System

This lesson uses Modelo’s Client Builder feature to create a visual client for the engine simulation.

Use the Add button to select Client…, then use the New Client button to display the Client Builder:

        NewCB

Rename the client by using the Properties button, then changing the default name (“MyClient”) to “EngineSimulation” then use the OK button to save the name change.

Drag and drop one Line chart widget, one Column chart widget, two Gauge widgets, one Pie chart widget and six Label widgets to the canvas area below the widget palette to create a Client Builder layout similar to this:

        CB

We’ll get to configuring each widget later so don’t worry about the exact appearance of the widgets yet.

Now that the Client Builder widgets appear in the canvas, we need to create two Data Streams to feed live data for display in the widgets. A Data Stream defines from where a client widget receives its data. Several widgets want to display speed or temperature values, which are found in the SystemStatus record updated from Lesson 6. The Pie chart widget wants to display the heads-up display alert message, which is found in the SystemHUD record updated from Lesson 7.

Use the Data Streams button, then the New Data Stream button in the Client Builder to define a new Data Stream. The Create New Data Stream dialog is displayed:

        VisEngine

Name the Data Stream “SystemStatus”, select On Data Changed, select SystemStatus from the type menu, and check the Update checkbox. Use the Save button to save the new data stream. Client widgets that want to display speed or temperature values will use the SystemStatus Data Stream since it has access to those two properties from type SystemStatus.

Create a second Data Stream named “SystemHUD”, which is also On Data Changed, uses SystemHUD from the type menu, and checks the Update checkbox.

There are five display widgets and six label widgets in the diagram above. What follows are the properties for each of those widgets. Any property not mentioned is the default value for that widget. To display the property sheet for any widget, click on the widget. For example, here is the property sheet for the Speed gauge:

        SpeedProperties

For the Engine Simulation label widget (top, center):

 Text: Engine Simulation
 Font Color: Dark Gray

For the remaining label widgets (above each display widget):

 Text: the corresponding title for each display widget
 Font Size: 16

For the Temperature line chart widget (top, left):

 Data Stream: SystemStatus
 X-Axis Property: Timestamp
 Y-Axis Properties: temperature

For the Speed bar chart widget (bottom, left):

 Data Stream: SystemStatus
 X-Axis Property: Timestamp
 Y-Axis Properties: speed

For the Speed gauge widget (top, left):

 Data Stream: SystemStatus
 Data Stream Property: speed
 Yellow Minimum: 45
 Red Minimum: 55
 Maximum: 65

For the Temperature gauge widget (top, right):

 Data Stream: SystemStatus
 Data Stream Property: temperatures
 Minimum: 180
 Yellow Minimum: 200
 Red Minimum: 210
 Maximum: 220

For the Alert Status pie chart widget (bottom, right):

 Data Stream: SystemHUD
 Data Stream Property: alertMsg

When selecting the Data Stream Property value, the menu choices may be restricted to those properties appropriate to the type of the widget. For example, when configuring a bar or line chart, only numeric properties will be available for selection. Other display elements, such as pie charts or lists, accept either numeric or string properties.

When you have finished adding and configuring your client widgets, use the Save button to save the client and return to Modelo. The next lesson uses the data generators created in Lesson 3 to produce simulated sensor data and run your client. You will not see any simulated data displayed in the client until the next lesson.

Use the Save button at the top, left of Modelo to save the project.

10: Running the Simulation

The last lesson of this tutorial is to start the data generators and observe the output from the VANTIQ system in real-time.

To begin the test, use the Run icon button of the Data Generators pane (small triangle in a square at the top, right of the pane).

        DataGenTab

As the generators insert simulated engine sensor data in the VANTIQ database, the EngineMonitor app runs, which triggers the UpdateSystemHUD task. The resulting updates to the SystemHUD record will cause the VANTIQ server to send messages to the SystemHUD subscription. Those message results are displayed in the Subscription: SystemHUD pane:

        RunningSubscription

As the simulated engine temperature sensor runs through its range of configured temperatures and the engine speed sensor runs through its range of configured speeds, the engine will overheat at various points in time and the HUD alert field is updated accordingly. When the temperature sensor settles below the 210 degree overheating value, the HUD alert field is cleared. These various alert states are displayed in the Subscription: SystemHUD pane.

Click the Client: EngineSimulation oval in the Project Resource Graph to display the Client Builder, then use the Run icon button of the Client: EngineSimulation pane (small triangle in a square at the top, right of the pane).

        RunningClient

The data generators will run out of data points in 100 seconds and the Data Generators pane will end the generation run.