Natural Language Processing Guide

Introduction

In the (near) past, users often struggle(d) with learning how to work with applications. New systems are focusing on more people-literate systems. That is, the applications are responsible for providing an easy to understand interaction for their users.

VANTIQ now supports providing natural language interfaces. This style of interface can be made available via a chatbot, thus manifesting in a chatroom in a collaboration, or via any other connections provided via the chatbot.

VANTIQ’s natural language processing is provided in concert with an external source that provides the parsing and interpretation of the natural language utterances. Incorporating natural language processing into a VANTIQ application will require setting up and integrating this source into the VANTIQ system. Customers will need to define a LUIS application for use by the VANTIQ system.

Note: The processing of natural language is separate from the conversion of speech to text (or vice versa). This guide focuses on the interpretation of natural language text; obtaining the natural language text from speech is a separate exercise.

Overview

Concepts

To talk about building a natural language interface, there are a few concepts and terms to define.

Examples

The following are some example utterances and their interpretations

Note that the interpretations above return information about the language specified. They do NOT ensure, for example, that the resource ‘people’ exists, nor that it has a property ‘age’. That would happen during the execution phase.

Architecture and Flow

The act of interpretation is, as noted, performed by an external (remote) source, Interpreter Source. The interpreter source must be a Language Understanding Intelligent Service (LUIS). LUIS is part of the general group of Cognitive Services.

Once your LUIS application is defined and the assocaited service defined, it can be added to VANTIQ as a Remote Source.

Often, a chatbot will be required as well. Information about defining a chatbot can be found here.

The overall system view is shown below.

VANTIQ Architecture Diagram

Utterances typically arrive via a chabot. These utterances are then passed to the intepreter source (provided by LUIS) for interpretation. The interpretation contains the intents and entities that describe, in a machine readable manner, what actions are required. This interpretation is returned to the VANTIQ system, and the interpretation is acted upon or executed, and the results, if any, can be returned to the user.

Within the VANTIQ application, the flow of information is dependent upon the application. Typically, the interpretation is determined, and the result examined. Assuming that a reasonable interpretation is returned, the execution of that interpretation is then invoked. In subsequent sections, this process will be shown for a rule-based system and for a collaboration conversation.

Application Design

Language Design

A LUIS application contains a set of intents, utterances, and entities that it can process. The LUIS application, including an application specific Natural Language Subset (NLS) for your Intent Set, is developed using the LUIS console. If you wish to use the VANTIQ language entities or intents, you should start by importing the VANTIQ NLS. More information is available in the VANTIQ NLS section.

The VANTIQ NLS contains an English culture (LUIS term for a specific natural language e.g. English, French, etc.) for the system intent set as well as the smalltalk intent set. These are described in more detail below.

Conventions

In order to more easily distinguish custom elements from VANTIQ elements, the following conventions are used.

System Entities

There are a number of entities that are defined as part of the VANTIQ NLS. These may, of course, be used as part of the custom application. These entities are described below. It is important to note the following. Natural language processing is imprecise. Consequently, the entities returned can vary from exactly what would be expected. Thus, it is important to have some flexibility in the interpretation to handle these inconsistencies.

The VANTIQ Intent Set also makes use of the builtin.age, builtin.dateTimeV2.*, and builtin.number from the LUIS provided builtins.

System Intents

The VANTIQ Intent Set includes the following intents. These are given with some examples from each intent.

See Example

Smalltalk Intents

The VANTIQ Intent Set also includes a set of smalltalk intents. These are basically chatter that users often try out. They perform no real function other than providing some comfort and/or entertainment.

Custom Intents

Custom intents are the intents that are specific to the VANTIQ application. The system intents and smalltalk intents are provided by VANTIQ, providing answers about the VANTIQ system or general chatter, respectively. For example, suppose there is an application for health care. In such an application, there might be a type patients. To find out who the patients are, one could use the VANTIQ intent system.list and say list patients.. However, in some environments, it may be more natural to ask about who is sick. Providing a natural language interface allows the application to become more people-literate, to understand things from the perspective of its users rather than having to train the users to ask the correct questions.

Interpreter Source

Finally, the interpreter must be defined as an Interpreter Source. This is a VANTIQ Source that is set up to call a Microsoft LUIS application.

To define such an application, use the LUIS Console as outlined in the associated documentation. If you are including the VANTIQ NLS, create the application in LUIS. See the VANTIQ Natural Language Subset section for more details.

The VANTIQ system will interact with this source using the POST method.

To create the source, find the URL from the LUIS console, providing that as the Server URI property. The Response Type property should be application/json. The Headers property should have one header Ocp-Apim-Subscription-Key, with a value corresponding to the subscription key of your LUIS application. If you have published your application in a staging (as opposed to production) endpoint slot (as is often the case during testing), set the Query Parameters property to {"staging" : "true"}. Please set the polling properties Polling Interval property to 0. All other properties can be left unset.

An example is shown below.

LUIS Source Example

Working with Natural Language in VANTIQ

Generally speaking, natural language processing in VANTIQ involves two phases:

This section describes how those are accomplished in various scenarios.

Processing

When a natural language text is obtained, it must be interpreted to determine what function is requested. This section outlines the basic processing and how to accomplish it.

Prepare Text for Interpretation

Different interactions can produce text that is formatted as something other than plain text. For example, Slack uses an XML-esque encoding of various characters, and these must be removed for the LUIS service to interpret things.

To do this, use the NaturalLanguageUtils.prepareText() procedure. This procedure takes three parameters:

The procedure returns the prepared text.

var preparedText = NaturalLanguageUtils.prepareText(message.channelId, true, message.text)

Interpret Text

To interpret the text, make use of the interpreter source, and use the NaturalLanguageCore.interpretNLQuery() procedure. This procedures takes two parameters.

This procedure will return an Object that contains the following elements.

Generally, the caller should check errorMsg for content. If content is present there, the response field may not be complete, may be missing, or may contain an invalid interpretation.

var sourceName = "LuisService"  // Using the name used in the example above.  
var interpretation = NaturalLanguageCore.interpretNLQuery(preparedText, sourceName)
var resultsToReturn
if (interpretation.errorMsg != null) {
    // return the error to the caller
    resultsToReturn = interpretation.errorMsg
} else {
    // Process the returned interpretation
}

Execute Interpretation

To execute the interpretation, the application may wish to make some decisions. For example, some applications may not wish to show the VANTIQ system.* intents through. However, they may want to allow the system.smalltalk intents. So, the execution phase may first wish to filter based on intent.

To execute only system.smalltalk intents, use the NaturalLanguageCore.executeSmalltalkIntent() procedure. To execute system.smalltalk and other system.* intents, use the NaturalLanguageCore.executeSystemIntent() procedure (i.e. this will also execute smalltalk intents). Both of these procedures take a single parameter:

To execute custom intents, the application should provide suitable execution code. It will execute the intent based upon the intent and entities found in the response object.

if (interpretation.response.intent.startsWith("system.smalltalk")) {
    // Handle smalltalk
    resultsToReturn = executeSmalltalkIntent(interpretation.response)
} else if (interpretation.response.intent.startsWith("system.")) {
    // Assuming system intents should be processed...
    resultstoReturn = executeSystemIntent(interpretation.response)
} else {
    // Handle any custom intents
    resultsToReturn = ...
}

Prepare Response

This is done the same way (using the same procedure) as was shown for preparation for interpretation. The difference is that incoming is set to false. (Note that if results are being returned using NaturalLanguageCore.publishResponse(), the response will be prepared by that procedure. In that case, this prepareText() need not be called.)

var outputText = NaturalLanguageUtils.prepareText(message.channelId, false, resultsToReturn)

Processing Summary

Putting these code snippets together, a (mostly) complete processing of an natural language message is as follows. Here, it is assumed that this procedure is called with a channel and text to be interpreted.

// NLExample.processMessage()
// 
// @param   channel     The channel over which this communication is happening
// @param   text        The message text to be acted upon
// @returns                 The text to return to the user


NLExample.processMessage(channel String, text String, prepareResponse Boolean)

// Strip formatting, encoding, etc.
var preparedText = NaturalLanguageUtils.prepareText(channel, true, text)

// Determine interpretation using the LuisService
var sourceName = "LuisService"  // Using the name used in the example above.  
var interpretation = NaturalLanguageCore.interpretNLQuery(preparedText, sourceName)
var resultsToReturn
if (interpretation.errorMsg != null) {
    // return the error to the caller
    resultsToReturn = interpretation.errorMsg
} else {

    // Process the returned interpretation
    if (interpretation.response.intent.startsWith("system.smalltalk")) {
        // Handle smalltalk
        resultsToReturn = executeSmalltalkIntent(interpretation.response)
    } else if (interpretation.response.intent.startsWith("system.")) {
        // Assuming system intents should be processed...
        resultstoReturn = executeSystemIntent(interpretation.response)
    } else {
        // Handle any custom intents
        resultsToReturn = ...
    }
}

var outputText
if (prepareResponse) {
    // Prepare the returned text for the channel
    outputText = NaturalLanguageUtils.prepareText(message.channelId, false, resultsToReturn)
} else {
    outputText = resultsToReturn
}
return outputText

Natural Language via Rules

VANTIQ Rules can be used to process natural language requests. A simple rule that does so is shown below.

This rule makes use of another VANTIQ-supplied procedure, NaturalLanguageCore.publishResponse(). This procedure handles the preparing of the output text as well as the publication back to the caller. For more details on this (and other) available services, please see the Vail Resources section, more specifically VAIL Procedures.

RULE ConverseViaRules
WHEN MESSAGE ARRIVES FROM someChatbot AS message

// Let's use the procedure just developed to perform the interpretation & execution

var ruleResponse = NLExample.processMessage(message.channelId, message.text, false)

// Here, let's reuse the same message to respond to the caller.
// However, it must replace the text with the response.

message.text = ruleResponse

// Here, use VANTIQ procedure to handle the publication.
// publishResponse() will perform the output prepare for us, so we 
// suppressed that call above.

NaturalLanguageCore.publishResponse(message, message.text, "someChatbot")

Collaborations

(For details on collaborations, please see the collaboration guide. This section presumes that pre-requisite.)

It is worth noting here that VANTIQ’s direct interaction with a chatroom (via the VANTIQ Mobile App) requires the chatbot to have a Direct Line Secret Key. When you set up the chatbot, make sure that you add that channel to your chatbot. Information about adding the Direct Line channel to the bot can be found in the Bot Framework documentation.

Natural language processing in collaborations will generally take place in the context of a conversation activity. The conversation activity type creates a chatroom. To add natural language processing, activities of the appropriate activity types are added to the collaboration. There are three (3) activity types of interest here.

A very simple collaboration performing only this conversation is shown here.

Collaboration Visualization

For a more detailed example, please see the natural language tutorial.

VANTIQ Resources

VAIL Services and Procedures Reference

IntentSpecification

The intentSpecification object structure is used throughout the natural language processing system to pass information about intents. The properties of the object are as follows.

Service: NaturalLanguageCore

interpretNLQuery()

Generally, the caller should check errorMsg for content. If content is present there, the response field may not be complete, may be missing, or may contain an invalid interpretation.

executeSmalltalkIntent()
executeSystemIntent()

publishResponse()

Service: NaturalLanguageUtils

prepareText()

VANTIQ System Natural Language Subset

The VANTIQ NLS can be found here.

Importing the VANTIQ NLS

The general instructions for importing into LUIS can be found in the LUIS Documentation.

Of specific interest here is that the LUIS API does not support partial or additive imports. That is, there is, at present, no option to import the VANTIQ NLS and, subsequently, import an additional set of custom intents. Instead, the LUIS user must start with the VANTIQ NLS, then add the custom intents using the LUIS console. When that work is complete, tested, trained, etc., the LUIS application can be published and used as an interpreter source.

However, if the VANTIQ NLS were to change (e.g. add new utterances, intents, entities), it is not currently possible to import only those changes. At present, the LUIS application developer should keep the various forms of things, and manually merge any changes required.