Packaging Agents

Getting Started

The XMPro Package Manager is a Windows 10 desktop application that enables you to package a new Agent or update details for an existing Agent. See the Agent article for more information on Agents.
This application takes you through the process of specifying all the properties your Agent requires, adding or changing the controls for each of the user settings, and uploading the DLL files of the Agent code. It will provide you, upon completion, with a file that can be uploaded to Data Stream Designer after which you can build Data Streams using the Agent.
You can download the software from the Microsoft Windows 10 Store or by clicking here.
After installing the XMPro Package Manager, launch the application from the Microsoft Store or search for “XMPro Package Manager” in the Start menu and then click on “XMPro Package Manager”.

New / Import

On the first screen of the application, you will have the option to choose to either create a new Agent package or import and update an existing one.
Use the arrows at the bottom of the page section to move forward or backward in the application.

Details

The Details form allows you to specify or edit the properties of an Agent. Each of these properties are listed and explained below.

Name

The name of the Agent is what the Agent will be known as once it is uploaded to the Data Stream Designer platform, for example, “Jupyter Notebook”.

Description

The description is usually a brief explanation of what the Agent does, for example, “This Action Agent allows you to create and load Jupyter Notebook files“.

Version

The version of the Agent is defined within this field. Any real number is acceptable, e.g. 1.02.
If you are making a change to an existing version, make sure you increment the version number as Data Stream Designer will not allow you to upload two of the same Agents with the same version.

Virtual

Agents can be classified as being either Virtual or Non-Virtual.
An Agent is Virtual if it is not specific to a certain environment and can be configured remotely. Non-Virtual Agents have to be in their respective environments to be able to function correctly, e.g. the SQL Server Agent, which has to connect to SQL Server via the intranet.

Help URL

The Help URL is a URL that will navigate a user to the help document for this Agent. This document will be opened in the Help pane in the Data Stream Designer when requested by the user.

Entry Point

The entry point is the namespace and class name of the actual Agent’s DLL file.
For example, if an Agent with the class name “ActionAgent” is located in the XMPro.JupyterNotebookAgents namespace, the Entry Endpoint for it would be “XMPro.JupyterNotebookAgents.ActionAgent”.

Isolated Loading

When loading Agents to use in a Stream Host, all the libraries are put in a separate Load Context. Enabling the Isolated Loading option helps keep Agent files separate and reduced the risk of libraries clashing or conflicting together. In most cases, this option should be enabled.

Icon File

The icon that you upload in this field will be used to represent your Agent. To change an existing image or upload a new one, click on the Browse button, navigate to where you’ve stored the file via the Explorer and select the new image file.
It is recommended that you upload either a JPG or PNG file with a size of 64×64 pixels to accommodate for retina displays.

Require Input Map

The Require Input Map feature allows you to specify that your Agent will be receiving events in a certain defined structure. This enables the arrows leading to your Agent to be made configurable and will allow the user to map the inputs of your Agent to incoming attributes.
If left unchecked, parent outputs will be published to this Agent as they are.

Add On-Error Endpoint?

This checkbox allows you to add an additional Output Endpoint, called an Error Endpoint. Error Endpoints will output error information when your stream is running and something goes wrong, making debugging easier. You can also define actions that will be executed after error data is sent to the next Agent in the stream by your Agent, for example when a certain record is not valid.

Endpoints

The Endpoints form allows you to specify any input or Output Endpoints.

Input Endpoints

These Endpoints represent entry points to the Agent, which will allow the Agent to receive data or input from another Agent. To add a new Input Endpoint, type the name of your Endpoint in the text field and click on the Add button. The new Endpoint will appear in the drop-down list below the text field. Please note that you may need to hover over the list and scroll down to see it. As per the image above, an Input Endpoint named “Input” has been added.
The name of the Input Endpoints has to match what has been defined in the Agent’s code.
To remove an existing Endpoint, scroll down in the list until you see it, select the Endpoint and click on the Remove button.

Output Endpoints

Output Endpoints represent exit points from an Agent and allow you to connect your Agent to another Agent, making it possible to pass data from your Agent to another Agent. Output Endpoints can be added in a similar way to the Input Endpoints. To add an Output Endpoint, type the value in the text field provided and click on Add. The list will contain the Endpoint after you clicked on the button. Please note that you may need to hover over the list underneath the text field and scroll down before being able to see the new value. In the image above, “Output” has been added as an example Output Endpoint.
The name of the Output Endpoints should match what has been defined in the Agent’s code.
To remove an existing Endpoint, scroll down in the list until you see it, select the Endpoint and click on the Remove button.

References

After completing the Endpoints form, you will be presented with the References screen where you will be asked to upload the DLL file(s) that were generated when you built the project containing your Agent. You are only required to upload your Agent’s DLL file; there is no need to upload the XMIoT.Framework.dll file as this DLL is automatically included.
To upload a file, click on the Browse button next to the DLL File(s) field and navigate to where the files are located and select them. When you’ve selected all the files needed, click on the Add button to add them to the Selected File(s) list. Please note that only files in the Selected File(s) list will be included in the package. To remove a DLL from the list, click on the trash-icon next to the DLL name in the Selected File(s) list.
The XMPro Package Manager can package DLL(s) created in .NET or Java.

Settings

Depending on what your Agent does, it might require that the user provide certain information, such as a server URL, username, or password. For each of these information fields (or settings), you need to specify which control should be used and what each control represents, for example, the Jupyter Notebook Agent will require the user to add a server URL. The user should provide this value using a text-box control. Thus, you need to create a control with a type of “TextBox” and a caption that reads “Server URL” in the XMPro Package Manager application.
The following controls are available to be used to capture user input:
  • FileUpload
  • Filter
  • Grid
  • Group
  • HTML Editor
  • ScriptBox
  • TextBox
  • Title
  • TokenBox
Each control can have a number of properties that have to be set. However, not all properties apply to all controls, for example, you don’t need to add options for a text-box control as it is only intended to be available for a drop-down control to allow the user to select an option from the drop-down menu. The table below contains a list of all the available properties, along with their description, and an indication to which controls each property will be applicable to.
Property Name
Control Type
Description
Allow Custom Text
Drop-Down
Allows the user to type custom text in the drop-down field if checked.
Allow Custom Tokens
Token Box
Allows the user to add custom tokens if checked.
Caption
All
Text that will be displayed with the group or setting. The caption is usually one or two words, describing the value that should be provided by the user, for example, “Server URL”.
Default Value
Title
The default value of the title.
Font Size
Script Box
Size of the font in the Script Box.
Help Text
All, excluding Groups
If you need to provide the user with any additional information about the purpose of the setting or helpful instructions, specify it in this field.
Key
All
Uniquely identifies the group or setting.
Keywords
Script Box
Define your variables or other custom keywords here, so that they will be available in the editor’s IntelliSense.
Options (Drop Down)
Drop-Down
Use the Options-area to add values to the drop-down menu by specifying the Text and Value fields and then clicking Save. You may also choose an option to be used as the default option by checking the “Set as Default Value” box.
Options (HTML Editor)
HTML Editor
Allows you to specify placeholders that can be mapped to input fields in the input received by the Agent.
Postback
All
If checked, will cause the form to do a postback to retrieve values from the server when the field loses focus (when the user clicks out of the field).
Required
All, excluding Groups
The control will be validated to make sure that a value has been specified if this box is checked.
ScriptBox Height
Script Box
Height of Script Box.
ScriptBox Mode
Script Box
Language in which script has to be written.
ScriptBox Theme
Script Box
The theme of the Script Box. Themes available include the following:
  • Ambiance
  • Chaos
  • Chrome
  • Clouds
  • Clouds_midnight
  • Cobalt
  • Cromson_editor
  • Dawn
  • Dreamweaver
ScriptBox Width
Script Box
Width of Script Box.
Secure
All
The value of the control will be treated as a secure value if this box is checked (encrypted and not displayed on the form in plain text). An example of a secure value is a SQL Server password.
Show Grid Lines
Grid
The grid lines of the grid will be shown if checked.
Show Header
Grid
The header of the grid will be displayed, if checked.

Adding Settings

All settings that are added need to belong to a group. Thus, you have to create a group first. You will then be able to add controls for settings to the group. To do this, follow the steps below:
  1. 1.
    Click on the plus-icon (top right, next to the Settings header). A form section will open, allowing you to specify a group for the settings.
  2. 2.
    Specify a unique value that can be used as the key for the group.
  3. 3.
    Add the caption you would like to use.
  4. 4.
    Click Save.
  5. 5.
    Next, we are going to add a setting. Click on the plus-icon next to the group you’ve created.
  6. 6.
    Choose the type of control you would like to use.
  7. 7.
    Add a unique key for your control. Please note that this key needs to correspond to what you defined in your code.
  8. 8.
    Add a caption for your control.
  9. 9.
    If needed, add a default value.
  10. 10.
    If required, add help text.
  11. 11.
    Select the options that apply from the list of check-boxes.
  12. 12.
    Click Save.

Output

The second last step is to provide the location on the file system where you would like the package to be created. Click on the Browse button and navigate to the folder where you would like the package to be exported.

Review: Details

Lastly, you can navigate through the pages to review all the details that you’ve specified so far to make sure the values are correct. If you are satisfied, continue with the wizard. Click on the Save buttons below, and your package will be created as a file with a “.xmp” extension.

Data Stream Designer: Uploading the package

You need to have the correct permissions set against your user to be able to edit and upload Agents. This is a role not typically given to all users.