How do I create a user defined data receiver?

Creating a user defined data receiver

Note on interpolation syntax

Note on names

Device Types tab

To create a user defined device type

To edit or delete user defined device types

Data Sources tab

To select the data source and start creating a data receiver

General tab

To define the data receiver general settings 

Steps tab

To create a step for the data receiver

Variables tab

To add a variable

Attributes tab

To specify the Index attribute (required for Object Creation steps)

To add a new attribute to a step

To edit or delete an attribute:

Summary tab

Creating a user defined data receiver

You can create a User Defined Data Receiver by following a step-by-step process from the User Defined Data Receiver page.

To access the User Defined Data Receiver page:

1. Click the Main Menu and then Administration.
2. On the Administration page, click User Defined Data Receiver under Polling.

The process of defining a User Defined Data Receiver is through a set of tabs. The step-by-step process goes to each tab in order, but you can navigate to any tab in any order. Each tab is completed by clicking Next.

A data receiver needs at least one step. Upon completing a step, you can add another step. 

Note, alternatively, you can create, edit, and delete User Defined Data Receivers with the Entuity RESTful API functionality.

Note on interpolation syntax:

As you navigate through the tabs to create a User Defined Data Receiver, certain fields accept a particular syntax to insert specific data values.

These fields are as follows:

• Step Filter – Steps tab
• Headers – Steps tab
• Source Path field – Variable tab and Attribute tab

These fields can accept the following syntax formats:   

• ${vars::myVarName} 

• ${sw::polledDeviceName} 

• ${src::items[@name=='wibble']} 

• ${ctx::id} 

• unscoped variables, for example: ${dataSourceId} or ${acceptedTimeSec} 

Where: 

As a special case, when you edit a Source Path field (on the Variables and Attributes tabs), you can optionally exclude the ${} format from the field.

For example:  

• items[@name=='wibble'] 

If you want to refer to, for example, the value of a variable inside the value part of a Source Path filter (in the previous example, the text, 'wibble'), you can do so, but you must use the full ${} syntax for both, and the embedded value must be escaped as follows: 

• ${src::[@objectId==\${vars::serviceId\}].serviceName} 

In this scenario, both the $ and the closing } inside the outer ${...} must be escaped with a backslash \. 

Note on names:

All names (for example, data receiver name, device type name, attribute name, variable name) support only alphanumeric characters and underscores. No other characters are supported. 

The maximum character length is dependent on the field. Most interpolated fields have a limit of 1024 characters. 

Fields can't be empty and can't contain only white spaces. The fields must contain some characters. 

Device Types tab

From the Device Types tab, you can optionally create and manage user defined device types. These user defined device types can be selected in the Context Type field on the General tab when you create a User Defined Data Receiver. 

Through this functionality, you can distinguish between User Defined Data Receivers by the device type to which the receivers apply. You can create dashboards that only appear for objects of the specific type, and can prevent all the attributes on those receivers appearing for every custom device, even when they are empty. 

You can attach attributes directly to a user defined device type. Attributes attached to a user defined device type are different from attributes added to a User Defined Data Receiver (on the General and Attributes tabs) in the following ways: 

• The attributes are directly on the device itself, not on an associated object of the device. 

• The attributes are populated during discovery of that object, using the statement language method provided. This means that the attributes do not retrieve their value through messages received, but instead have a fixed value. 

• The attributes can be overridden from the Object Attributes dashlet and/or Attributes dashboard. This means that the attributes are useful for storing extra device-specific information that might be needed by a subsequently defined User Defined Data Receiver (for example, when the User Defined Data Receiver needs values supplied by the user, like account numbers or hostnames, and which are not provided in the payload). 

Device Types tab lists the user defined device types that you have created, providing the following information.

To create a user defined device type:

1. From the Device Types tab of the User Defined Data Receiver page, click the Create Device Type button at the top of the page, or from the Overflow Menu.
   
   
   
   The Create User Defined Device Type form is displayed.
   
   
    
2. From the Parent Type dropdown menu, select the parent type of this user defined device type.
   You can select either User Defined Device or an existing device type that you have already created, thus extending that existing type.
3. In the Type Name field, enter a name for the device type, with the prefix udt_. If the name does not contain the prefix, you will not be able to save this user defined device type. The name can not include spaces and must be unique. This name does not get displayed in the UI.
4. In the Display Name field, enter the name to display in the UI for the device type.
   You can include spaces in the display name. Entuity recommends entering a unique name in this field.  
5. Optionally enter a description for the device type in the Description field. 
6. From the Visibility dropdown menu, select the visibility level of this device type in the Entuity UI. See the device type table for information about each Visibility menu option. 
7. Status Attribute - you can specify a status attribute if you want to use that attribute to drive the object status of the new device type. Specifying an attribute status is useful because, by default, custom devices have no status information. By defining a status attribute, you can give the custom device type a status that is, for example, derived from a value within the message payload that the receiver will be processing. 
   Note, the list of attributes from which you can select for this dropdown menu contain only pre-existing attributes that have a statusMap defined.  
   Note, you must first create the device type, then create the User Defined Data Receiver, and then return to the device type and select the status attribute from one that you have defined in the receiver. 
8. In the Attributes field, click Add Attribute if you want to add an attribute to the user defined device type. 
   Attributes that you create here will be editable from the Object Attributes dashlet and the Attributes dashboard. Clicking Add Attribute opens the Create Attribute form:
   • Name - Specify the name of the attribute. This must be unique. 
   • Display Name - enter the name to display in the UI for the attribute. 
   • Data Type - For example: string, integer, float 
   • Display Format - For example: traffic, percent, timestamp 
   • Visibility - Whether the attribute is short listed, visible (default), advanced, or hidden. For information about the visibility levels, see the description in the device type table.
   • Graphable - Whether the attribute can be used in a chart. 
   • Enumerable - Whether the attribute is suitable for a pie chart category. 
   • Searchable - Whether the Entuity search functionality will look at its value. 
   • Allow in dashboard filters - Whether the attribute can be used when specifying filters on dashboards. 
   • Allow in UD Poller filters - Whether the attribute can be used when filtering on user defined REST pollers or data receivers. 
   • Define status mapping - Map the values of the attribute to status categories. Click the field to specify the mappings, for example, 1 = OK, 2 = Down, 3 = Unknown, and so on. These mappings are used if this attribute is then chosen to determine the status icon for the device type, but are otherwise not used. The available statuses follow: 
     • OK 
     • Admin Down 
     • System Uninitialized 
     • Unknown 
     • Degraded 
     • Down 
   • Collector Method - Specify an expression to be used in the collector that discovers the attribute, using StormWorks simple statement language. For example, rand(101). 
   • Test Expression Context - Specify the context by which the method is tested: 
     • None - If you add attributes to a user defined device type that has not yet been saved, you can test the expression without a context. 
     • Sample Device - If selected, the attribute will be tested against a sample context of the right type, if it can be found (for example, the context of the parent type). 
     • Selected Device - If selected, the Select Evaluation Context field will be displayed. Click this field to open a list of devices from which you can select a device to test against. 
   • Test Method - After you enter a Collector Method, click Test Method to test the collector. 
     Note, if you try to reference other attributes within the collector on this new user defined device type, the Test Method button will not work until you save the device type and its attributes. 
     Click Done in the top right of the Create Attribute form to save your changes, and return to the previous form. Otherwise, click Cancel. 
9. Click Create on the Create User Defined Device Type form.
   The new device type is now created and can be used when you create a User Defined Data Receiver.

To edit or delete user defined device types:

1. Click the Device Types tab at the top of the User Defined Data Receiver page.
   The existing user defined device types are displayed.
2. Click the device types that you want to select for editing or deletion. 
3. Do one of the following:
   • Click the Edit Device Type or Delete Device Type buttons at the top of the page.
   • Select the Edit Device Type or Delete Device Type menu options from the Overflow Menu.
   • Right-click the user defined device types, and select the edit or delete menu options from the Context Menu.  
4. If you choose to edit the user defined device types, make your edits on the Update User Defined Device Type form that is displayed, and click Done.
   Note, the Update User Defined Device Type form is the same as the Create User Defined Device Type form, except the Parent Type and Type Name fields can't be edited on the Update User Defined Device Type form. 
5. If you choose to delete the user defined device types, click Yes to confirm your deletion at the prompt, or click No to cancel the deletion.
   
    

Note, if you try to delete a user defined device type that is being used by a User Defined Data Receiver, or if the device type is the parent of another custom device type, an error message will be displayed at the top of the page. In this case, before you can delete the device type, you must delete the custom devices that are using the device type, or you must edit the custom devices to use a different device type. Alternatively, you can 'force delete' the device type, which means that any custom devices using this device type are 'demoted' to regular custom devices, and consequently lose any attributes from this now-deleted user defined device type. 

Data Sources tab

The Data Sources tab lists all the known data sources that have sent data to Entuity through the Message Broker. Each entry represents a specific record type from a device, so the same device might be displayed multiple times, if the device is configured to send more than one record type. From the Data Sources tab, you can choose the data source you want to use with the data receiver. 

For information about configuring devices to send data to Entuity, see the "Device configuration" sections in Example workflows for creating User Defined Data Receivers.

To select the data source and start creating the data receiver:

1. On the User Defined Data Receiver page, click the Data Sources tab. 
   
   
    
2. Once a Data Source is selected, click Create Receiver button at the top of the page or via the Overflow Menu.
3. This will take you to the General tab where you can start creating the new receiver. 

General tab

From the General tab, you can define the general settings of the User Defined Data Receiver. The Details column on the left of the General tab is where you specify the details of the new User Defined Data Receiver. The Example Payload column on the right displays the example payload from the selected data source the receiver will be created for. 

To define the data receiver general settings:

1. In the Receiver Name field, enter the name of the data receiver, without any spaces. 
   The name must be unique, and won't be displayed in the UI.
2. In the Data Source field, the data source ID is displayed that the receiver is being defined for.
3. In the Matching Device field, a device name is displayed if Entuity is able to match the data source with an existing device that is managed by Entuity. 
4. From the Record Types dropdown menu, optionally choose one or more record types to be processed by the data receiver. 
   A particular data receiver is used to process a message that matches based solely on its record type, only if there is no data receiver matching its exact Data Source ID. 
   Note, specifying a record type is the only mechanism where the same receiver can be used for updating more than one device. For example, if the record type is defined as redfish_power_powersupplies at creation, when another message is received sharing the same record type, but from a different data source, the same data receiver can be used to process the message. 
5. In the Context Type field, optionally click the arrow to the right of the field to open an Explorer window from which you can specify the device type for the receiver. By default, the Context Type field value is Device, meaning any device type. 
   You can click Show more at the bottom of the Explorer window to select from non-device types or select a user defined device type you created (See To create a user defined device type).  
   Note, if you add a data receiver to a context type that is neither a device nor an associated device, each instance of it will be charged an Object license.
6. From the Receiver Type dropdown menu, choose the type of receiver you want to create: 
   • Attributes Only – The data receiver will populate one or more attributes added to a single associated component attached to the target device. Each time a new message is received, the value of these attributes is updated.
   • Sub-object Creation – The data receiver will create one or more sub-components associated with the device each time a new message is received, and will populate attributes on them, as specified. Any defined attributes will exist on each child object.
7. In the Display Name field, enter the data receiver name to be displayed in the Entuity UI. 
   The display name can include spaces.
8. In the Description field, optionally enter a description for the data receiver.
9. From the Visibility dropdown menu, choose the visibility level of the receiver's association is throughout the Entuity UI: 
   • Shortlisted – The attributes or sub-objects created by the receiver are included in cases when a shortlist of suggestions is provided.
   • Visible – (default) The attributes or sub-objects created by the receiver are shown in most areas. If they are not shown, then when you click more… throughout the UI, they will then be displayed. This is usually in areas where only Shortlisted items are shown by default, such as listing types as contexts for dashboards/dashlets.
   • Advanced – The attributes or sub-objects created by the receiver will only appear in, for example, the Object Attributes dashlet, when the Show Advanced option is selected from the Overflow Menu.
   • Hidden – The attributes or sub-objects created by the receiver are never shown in the UI.
10. From the Retention Period dropdown menu, choose the retention period to define how much history is maintained on attributes populated by this data receiver (by default, 1 week).
11. From the Media Type dropdown menu, you can accept the media type that is inferred by the example payload, or you can override the media type by selecting JSON, XML, TEXT, or AUTO.
12. Data Adaptor – Provides the option to apply a data adaptor to the incoming data that will transform the data from one format to another through processing.
    • CSV Adaptor – Turns CSV input into valid JSON.
    • JSON Sanitiser Adaptor – Sanitizes JSON that may not be parsed correctly in Entuity so it transforms it into another form of valid JSON that does not contain the characters that have the potential of having this issue.
    • JSONLines Adaptor – Processes JSONLines input into valid JSON with additional adaptor parameters that can be configured such as sanitize Property Names, ignore invalid JSON Entries or throw exceptions for invalid JSON. 
13. Click Next to go to the Steps tab.

Steps tab

The Steps tab displays the steps defined for the User Defined Data Receiver. You must have at least one step per User Defined Data Receiver.

The first column of the Steps tab shows the currently defined steps for the data receiver, and provides the ability to add or remove a step. The second column of the Steps tab shows the step details, where you define the individual step. For most User Defined Data Receivers, only one step is required. However, Entuity supports creating multiple steps, as needed. 

There are three types of steps that you can create for a data receiver: 

• Object Creation step - (Applicable to the Sub-Object Creation data receiver type.) Defines the type of sub-objects to be created by the data receiver.
• Attribute Population step - (Applicable to the Attributes Only data receiver type.) Defines how attributes are populated by the data receiver.
• Set Up step - (Applicable to both the Sub-Object Creation and Attributes Only data receiver types.) Populates variables that you can then access in later steps. 

To create a step for the data receiver:

1. Click the type of step that you want to create (New Setup Step, New Object Creation Step, or New Attribute Population Step), depending on the receiver type.
   The step parameters are displayed in the second column. If you select an existing step from the first column, the second column shows the workflow of the currently selected step.
2. In the Step Name field, enter a name for the step.
3. In the Step Execution field, select whether to enable Unconditional  (default) or Conditional execution of poller steps. 
   If you select Conditional, the Step Filter field is displayed, in which you can enter an interpolated expression. This uses the same interpolation as elsewhere in User Defined Data Receiver variables and attributes. For information about the syntax of the Step Filter field, see Note on interpolation syntax.
4. Component Type  (Object Creation step only) – The StormWorks type for this currently selected Sub-object. Opens a form where you can enter the name, display name, description, visibility, and object keep time of the object that will be created. 
   Note, the Object Keep Time determines how long objects created by the data receiver(s) are kept before being marked stale and being removed if the objects aren't in subsequent data messages as the objects are not created by a regular polling cycle.
5. Parent Object Creation  (Object Creation step only) – This field is for selecting a previous step in the receiver when creating child objects that will be under the newly created parent object. 
6. Optionally add another step, as necessary or click Next which will take you to the Variables tab. 

Note, the following tabs will depend on, and relate to, the specific step that you select on the Steps tab. 

Variables tab

From the Variables tab, you can store variables that you want to pass from a previous step to the next, but that you do not want to store in the attributes. 

Note, this tab is applicable only to data receivers that have Set Up steps. For more information about step types, see Steps tab.

To add a variable:

1. From the Variables tab, click Add and the Add Variable form is displayed.
2. Enter the name of the variable in the Name field.
3. In the Source Path field, you can click Data Explorer to open the Data Explorer window, from which you can navigate to, and select, the variable source path that you want. Alternatively, you can search for other data in the Source Path field through the interpolation mechanism by entering ${. For information about the interpolation syntax to use for the Source Path field, see Note on interpolation syntax.
4. Select one of the following options in the Scope field for variables that have persistent values:
   1. Session (default) – This matches the previous behavior. Such variables last for the duration of the groovy script for the current message receival or “session”.
   2. Context Object – This stores the value of the variable in memory so that it can be referenced in future message receivals. The variable value is stored separately for each context object (device) against which the data receiver is run.
   3. Receiver – This stores the value of the variable in memory so that it can be referenced in future message receivals. The variable value is shared across all invocations of the data receiver.
5. Click Done to save your selections, otherwise click Cancel. Once saved, the new variable is now displayed in the table.
6. Once you have added all the variables desired click Next to go to the Attributes tab. 

If you want to use the variable in a particular field, you can access the variable with the following format: 

${vars::myVarName}

The UI provides a list of variables when you enter ${.

For example:

${src::nameOfMySource}

Attributes tab

From the Attributes tab, you can specify the attributes to be populated by the User Defined Data Receiver.

Note, this tab is applicable only to data receivers that have Attribute Population and Object Creation steps. The Attributes tab is not available to data receivers that have only the Set Up step type. For more information about step types, see Steps tab.

Attribute considerations:

• When defining an Attribute Population step, you can add as many attributes to the receiver as needed. 

• When defining an Object Creation step, two predefined attributes are applicable – the Index and Name for the new sub-objects. You must specify the Index attribute because this is needed for giving each object a unique key that identifies the object. Additional attributes can also be defined in the step.

To specify the Index attribute (required for Object Creation steps):

1.  From the Attributes tab, click the Index attribute in the table, and then click Edit.
   
    
   
   A form is displayed for editing the Index attribute.
   
   
    
2. At the Parent Source Path field, specify the list of data within the source data that represents the sub-objects.
3. In the Source Path field, specify the field that will provide the unique key. 
   For information about the interpolation syntax to use for the Source Path field, see Note on interpolation syntax.
4. Select the Single Item Path checkbox to tell the data receiver to expect the selected source path to identify as a single object, rather than a list of objects.
   Note, in most cases, you must select the Single Item Path checkbox so that the values are parsed and correctly populate the attributes. 

To add a new attribute to a step:

1. From the Attributes tab, click  Add at the top of the page or from the Overflow Menu.
   
    
   The Add Attribute form will then open on the right side of the page.
2. Enter the name of the attribute in the Name field.
3. From the Source Path  field, specify the source path if you want a particular item based on one of its attribute values. 
   You can click Data Explorer  to open the Data Explorer window, from which you can navigate to, and select, the source path that you want. 
4. In the Display Name field, specify if you want to change the name of the attribute from its data name.
5. From the Data Type  dropdown menu, choose the data type of the attribute. For example: string, integer, float 
6. In the Display Format field, specify how the attribute is to be displayed. For example: traffic, percent, timestamp 
7. From the Visibility dropdown menu, choose the visibility level of the attribute throughout the Entuity UI:
   • Shortlisted – The attribute is included in cases when a shortlist of suggestions is provided.
   • Visible – (default) The attribute is shown in most areas. If it is not shown, then when you click more… throughout the UI, it will be displayed. This is usually in areas where only Shortlisted items are shown by default, such as listing types as contexts for dashboards/dashlets.
   • Advanced – The attribute will only appear in, for example, the Object Attributes dashlet when the Show Advanced option is selected from the Overflow Menu.
   • Hidden – The attribute is never shown in the UI.
8. Select the Graphable toggle if you want the attribute to be used in a chart. 
9. Select the Enumerable toggle if you want the attribute to be suitable for a pie chart category. 
   If selected, the Display Transform  field is displayed. The Transform field lets you specify on which attribute to convert raw data into more meaningful or standardized values used by Entuity and the mapping to use for the transform. The transform cannot be saved unless a mapping is defined. 
   Note, a new transform cannot have the same name as an existing transform.
10. Searchable - whether the Entuity search functionality will look at its value.
11. Use this attribute for object display name - whether the attribute can be used as the display name for the object that it is on. Only one attribute can be used for this.
12. Use this attribute for object status - whether the attribute can be used to drive the status icon for the object on which it appears.

13. Event Mode - specify the basis on which events are raised, either None, Threshold, or Status. 
    If Threshold is selected, you can add event mapping: 

    1. Click the Threshold field, then click the Add Threshold button, and fill out the following fields: 
       • Name - specify a name for the threshold. By default, this is derived from ud_, the attribute name and the threshold level. This name cannot contain a hyphen ' - '. 
       • Display Name - specify a display name to be displayed on the Thresholds Summary dashlet/Thresholds dashboard. 
       • Description - specify a description for the threshold. 
       • Group Name - specify the group name you want to use for this threshold. The group name is used on the Thresholds Summary dashlet/Thresholds dashboard to group together different thresholds, for example different severity level thresholds set against the same attribute. 
       • Display Units - specify the measurement unit of the attribute. 
       • Minimum Value - specify the minimum value of the threshold range. 
       • Maximum Value - specify the maximum value of the threshold range. 
       • Default Value - specify the default value of the threshold range. 
       • Enabled - whether to enable the threshold. 
    2. Click Done to save your threshold.
    3. In the Mapped Value field, specify the threshold value from one of following: Critical, High, Warn, or Low. 
    4. Click Done to save the threshold. 

    
    If Status is selected, you can add event mapping: 

    1. Specify the Status Value and then the equivalent Status Mapping from one of the following: Up, Down, Disabled, Other.
    2. Click Done to save your status mapping. 

To edit or delete an attribute:

1. From the User Defined Data Receiver page, click a receiver in the table, and then click the Attributes tab at the top of the page.
   The existing attributes are displayed.
2. Select the attribute that you want to edit or delete, then click the Edit or Remove button at the top of the page. Alternatively, select the desired option via the Overflow Menu or right-click Context Menu.
3. If you choose to edit an attribute, make your edits on the Edit Attributes form that is displayed, and click Done.
   Note, the Edit Attribute form has the same fields as the Add Attribute form. 
4. When you have removed or edited the attributes that you want, click Next to go to the Summary tab. 

Summary tab

The Summary tab displays the details of all previous tabs on a single page. 

To add another step to the data receiver, click New Step. This returns you to the Steps tab, from which you can add a new step or edit an existing step. 

Click Save on the Summary tab to save the data receiver. By default, the data receiver is enabled. If the data receiver already exists and is disabled, you are prompted to enable the data receiver. After the data receiver is saved, the data receiver is displayed in the table on the Receivers tab of the User Defined Data Receiver page.