Archived - How do I add a user defined REST API poller? (Entuity v21.0 P02 and below)

To add an enhanced user defined REST API poller

Note on interpolation syntax

Note on names and character limits

1. Pollers tab

2. General tab

3. Step tab

4. Parameterize tab

5. Variables tab

6. Attributes tab

7. Summary tab

Example workflows for adding a user defined REST poller

To add an enhanced user defined REST API poller:

To add an enhanced user defined REST API poller, you will need to follow a step-by-step process from the User Defined REST Polling page:

1. Click the Main Menu and then Administration.
2. On the Administration page, click User Defined REST Polling.

The process of defining a REST API poller is across a number of tabs. Once you have completed at least two tabs (each tab is completed (validated and saved) by clicking Next), you can then navigate between the tabs in any order.

A poller needs at least one step. Once you have completed one step, you can add another.

Note, you can also add, edit and delete user defined REST pollers via the Entuity RESTful API functionality. Please see this article for further help and information on Entuity's RESTful API functionality.

Note on interpolation syntax:

When adding a user defined REST API poller, in certain fields you can use particular syntax to insert specific data values. These fields are as follows:

• Host - General tab
• Port - General tab
• Headers - Step tab
• parameter replacement value - Parameterize tab
• Source Path field - Variable tab and Attribute tab
• the 'Body' of a RESTful API POST request. It is useful e.g. if sending authentication details through a non-standard method, sent as a POST for security reasons possibly needing to embed e.g. an API key. Or, if the API uses POST requests as standard and requires, in the Body, information about the IDs you are requesting (which might be different for each component).

These fields can accept the following formats:

• ${vars::myVarName}
• ${header::Content-Type}
• ${sw::polledDeviceName}
• ${src::items[@name=='wibble']}
• ${ctx::id}
• unscoped variables, e.g. ${apikey} or ${username}

Where:

• vars:: -> allows you to insert the name of a variable defined in a previous step.
• header:: -> allows you to insert the value of a header returned in a previous response.
• sw:: -> allows you to access existing StormWorks attributes from the context of the device.
• src:: --> allows you to explicitly enter a source path, like the path you can get from the Data Explorer, e.g. item[@deviceName=='Hogwarts'].macAddr, item[@deviceId==1234].macAddr
• ctx:: --> useful for sub component pollers, has only two variables: ‘Index’ and ‘Type’. This allows the poller to reference an object which it is in the process of making but is not yet added to the StormWorks data model. E.g., when creating a sub component under another sub component, you may need to insert the id (‘Index’) of the parent sub component into the URL to retrieve the data for the sub- sub component.
• unscoped variables are variables provided automatically by the Entuity Collection Engine process, and may not always be populated - this will depend on e.g. the credentials that are defined on the defined against which it is running, etc.

As a special case, when you edit a Source Path field (when adding a Variable or Attribute), this field can (but does not necessarily have to) exclude the ${ format: 

• items[@name=='wibble']

If you wish to refer to e.g. the value of a variable inside the value part of a source path filter (in the above example, the text 'wibble') you can do so, but you will need to use the full ${} syntax for both and the embedded one will have to be escaped as follows:

• ${src::[@objectId==\${vars::serviceId\}].serviceName}

i.e. the $ and the closing } inside the outer ${...} need to be escaped with a backslash \

Note on names:

All names (e.g. poller name, attribute name, transform name, threshold name) can only support alphanumeric characters and underscores. No other characters are supported.

The maximum character length is dependent on the field in question. Most interpolated fields (e.g. the 'Host' and 'Port' fields) have a limit of 1024 characters, but there is no limit to the character limit for header values.

Fields cannot be empty and cannot contain only whitespaces - they must have some other characters.

A new transform cannot have the same name as an existing transform.

1. Pollers tab:

1. On the User Defined REST Polling page, click the Pollers tab.
2. This tab lists the pollers you have created. Click Create Poller.
   
   
   
   
3. This will take you to the General tab, wherein you will start creating the new poller.

For further help and information on the functionality enabled from the Pollers tab, please see this article.

2. General tab:

The first column is the Details column, where you specify the details of the poller and associations.

1. Poller Name - specify the poller name. This does not appear in the UI. There cannot be any spaces in the poller name, and it must be unique from that of other pollers.
2. Context Type - click this field to open an explorer, from which you can specify the device type. You can click 'Show more' at the bottom of the explorer to select from non-device types. If you add a poller to a type that is neither a device nor an associated device, each instance of it will be charged an associated device license.
3. Poller Type - specify the type of poller to create from the dropdown options:
   • Attributes - this poller will create a single associated object for each context object to which it applies. Any defined attributes will exist on that associated object.
   • Sub-Components - this poller will create an unspecified number of child objects for each context object to which it applies. There is no specific limit on how many child objects a particular parent object (device or associated device) may have, and is dependent on the data returned by the polling query. The number of child objects can be different for each parent object, and may vary over time as the data changes. Any defined attributes will exist on each child object. An example would be polling a cloud controller and then adding sub component objects that represent the virtual devices on that cloud controller.
4. Display Name - specify the display name that will be given to the associated object of the device. This will appear in the UI, and can have spaces in the name. It does not need to be unique, but is best if it is.
5. Description - specify a description for the association.
6. Visibility - specify how visible this association is from the dropdown options:
   • Shortlisted - these are included in cases when a shortlist of suggestions is provided.
   • Visible - (default), these are always shown.
   • Advanced - these will only appear in e.g. the Object Attributes dashlet when the 'Show Advanced' option is selected from the Overflow Menu.
   • Hidden - these are never shown.
7. Polling Period - specify the polling period from the dropdown options, by default 5 minutes.
8. Retention Period - specify for how long the polled data is retained from the dropdown options, by default 1 week.

The second column is the Filter column, where you specify the details of the poller's filter(s).

1. There are two parts to creating a filter:
   
   1. If you want to create a common filter based on specific devices, click Add Device. This will open an explorer from which you can select a specific device. For example, you might select a device called 'Hogwarts', which means that the polling will only apply to that particular device name. Once you have added a device, it will appear in the list below. You can remove a device by selecting it and clicking Remove Device.
      
      Note, from Entuity v21.0 P01 upwards, the list of devices you can choose from is filtered based on the choice made in the Context Type field. Before this patch, the list of devices is not filtered based on this choice, so be sure to select an applicable device.
      
      Note, the checkbox next to each device in this list is only important if you intend to show the advanced filter or remove that device. The device does not need to be otherwise ticked for it to be valid for your filter.
      
      
      
      If you want to create a filter with more specific parameters, tick the device and then click the Show Advanced Filter switch. This Advanced Filter is instead of a device filter, and if you specify an advanced filter, any devices previously selected will be disregarded. In the Operator Type field, specify either Logically ANDed Filters or Logically ORed Filters, and then click Add Filter to open the Add New Filter form, where you can specify the Filter Attributes, Filter Type, and Values. Once you have specified the advanced filter, it will be displayed below. To edit the filter, click the filter name, which will open the Edit Filter form.
      
      
      
      
      
      
2. Once you have created your filter or filters, click Next in the top right.
3. This will then take you to the Step tab.

3. Step tab:

The first column displays your currently defined steps, and provides the options from where you can add or remove a step. The second column displays the step details, where you define the individual step.

There are 3 types of step that you can create:

• Setup - applicable to both types of poller (Attributes and Sub-Components). This can be used to e.g. perform required authentication and store the results in a variable for access by later steps. A setup step is executed during both the discovery and polling phases.
• Discovery - applicable only to Sub-Component pollers.  One or more Discovery steps is required for a Sub-Component poller. This defines how new Sub-Components are created, providing a unique key for each one and optionally a name. This step is required to know which sub-component type will contain the polling attributes. Each Discovery step will need a corresponding Polling step to be defined in order to populate any attributes beyond the name of the component.
• Polling - applicable to both types of poller. This defines where you will poll a particular endpoint.

1. Click the type of step that you would like to create (New Setup Step, New Discovery Step, or New Polling Step). 
   
   
   
   
2. The step parameters will appear in the second column. If you select an existing step from the first column, the second column will update to display the workflow of the currently selected step.
   
   
   
   
   1. Step Name - enter a name for the step.
   2. Data Collection - (Entuity v20.0 P04 upwards) - switch on to specify that the step collects data via an API, or switch off to prevent data collection.
      • If switched off, the other options for this step below are removed, the Parameterize tab is not shown, and Data Explorer buttons are no longer applicable to this step.
      • If switched off, you will instead need to specify existing data for this step. (e.g. retrieving a variable from a previous step or StormWorks attributes).
   3. Component Type (Discovery step only) - the StormWorks type for this currently selected Sub-Component. Opens a form wherein you can specify the name and display name of the new type of component that will be created.
   4. Parent Discovery Step (Discovery step only) - this is to specify the parent component type (StormWorks type) of this currently selected discovery step's component type (StormWorks type), but only from the ones that have been created by other discovery steps in your poller.
   5. Discovery Component type (Polling step only) - specify the discovered components to which this is applicable from the dropdown field.
   6. Data Scope (Polling step only) - whether an API request is required for each sub-component (Request PerSub-Component), or if a single API request will get all data for every sub-component (Single Request).
   7. Credential Mode - specify the type of credential to use for the device from the dropdown options, either Device Credential, Fixed Credential or No Credential. This defaults to 'Device Credential', which means that at runtime, it will use whichever user defined polling credentials are defined on the devices it is running against. User defined credentials can be defined when adding or modifying an asset. 
   8. Specify the Credential to use, or create a new credential. This field does not appear if you selected No Credential above. For help and information on creating credentials in Entuity, please see this article.
   9. Connection Method - specify whether to use a GET or POST call to the device. If you are using some form of authentication, you will need to select POST, and when you select this the Post Media Type and Body fields will appear below. In the Body field, specify the text that will be sent to the server, the format of which will be determined by the author of the REST API to which you are connecting to. Examples of body text might include posting a username-password, or an API key in order to authenticate.
   10. Connection Url - specify the URL that Entuity will use to connect to the device. The protocol is also specified here.
   11. Host - if you are using a URL that is not related to the device that you are running it on, you will need to override the Host, otherwise it will write the device name into the Connection URL.
   12. Port - defaults to the port specified when the device was added. You can override it by specifying a fixed port.
   13. Headers - click to open the Headers form, wherein you can add a text value in Name and Value. In both of these fields you can enter placeholders to be interpolated - please see the notes on interpolation above for further. Headers can be used for authentication information (such as an authentication key) or for specifying content type, but also can be used for any arbitrary data that specific REST APIs want Entuity to send in a header.
       
       
       For example, you might have a Meraki cloud controller. If you hard code the key, it will be the same for every device that the poller runs against. Alternatively (and if the device has credentials that provide an API key), if you set the value to the API key that is in the device credentials using the following interpolated value in the Value field:
       
       ${apikey}
       
       
       
       
   14. Click Test to open a popout dialog box that tests the specified API and displays an error messages if necessary.
3. Once the step's definitions are as you intend, add another step as necessary or click Next. Clicking Next will take you to the Parameterize tab.

Note, the following tabs will depend on and relate to the specific step that you have selected under the Steps tab. For example, if you have selected Step 1 under the Steps tab.

4. Parameterize tab:

From the Parameterize tab, you can specify any parameters required for your poller. If you are running the poller against multiple devices and you are using one URL for different types of information on each device, then you will need to parameterize the URL in order to poll for information from each individual device.

Path Components:

The first table on this tab is the Path Components table, which lists the components specified in the Connection URL of the Step tab. For example, if you specified 'http://localhost/api/inventory/1', then the device with the deviceId of 1 will appear in this table.

To parameterize this component:

1. Select it in the table and click Edit Component at the top of the page, or via the Overflow Menu or right-click Context Menu.
   
   
   
   
2. The Edit Component form will open. Enter your replacement parameter in the Replacement field. If you are using an attribute, you will need to know the StormWorks name of that attribute, which you can find via the Entuity StormWorks Data Dictionary (you will need to open this in a separate tab/window). If you have specified a variable in a previous step, this will be shortlisted as an option when you enter '${' in the field.
3. Click Done to save your changes, otherwise click Cancel.
4. In the Path Components table, the replacement paremeter will then appear in the Replacement column next to the selected component.

Path Parameters:

The second table on this tab is the Path Parameters table. This lists the path parameters specified in the Step tab. You can edit these parameters by clicking Edit Parameter at the top of the page (or via the Overflow Menu or right-click Context Menu), which opens a form requesting you enter a Parameter Name and its Replacement. You can also add further parameters as you need by clicking Add Parameter from the same places.

Once you have specified your required parameters, click Next. This will take you to the Variables tab.

5. Variables tab:

From the Variables tab, you can store variables that you wish to pass from a previous step to the next, but you do not want to store in the attributes. Examples of this include:

• data that you collect in one step and then use it to build e.g. the URL in the next step.
• a unique identifier in one step baed on e.g. the mac address, and then you might use this unique identifier to make a second REST call in the following step to get more specific data on that object.

To add a variable:

1. Click Add to open the Add Variable form.
   
   
   
   
2. You can manually enter the Name and Source Path, or click Data Explorer to open the Data Explorer tree, from which you can navigate to and select the variable that you wish to add.
   
   
   
   
3. Once you have made your changes, click Done in the top right corner, otherwise click Cancel.
4. The variable will then appear in the table. Once you have added all the variables that you want to add, click Next. This will take you to the Attributes tab.

When you want to use the variable in a particular field, variables are accessed in the following format:

${vars::myVarName}

The UI will present a list of variables once you type ' ${ ', e.g. ${header::nameOfMyHeader}

6. Attributes tab:

From the Attributes tab, you can specify the attributes that will be polled. This tab is only applicable to the Polling and Discovery steps, and is not available for the Setup step.

• When defining a Polling step, you can add as many attributes as you may need for an Attributes poller.
• When defining a Discovery step, only two attributes are applicable - the Index and the Name for the new sub-components. It is mandatory to specify the Index attribute, because this is necessary for giving each component a unique key that identifies it.
  
  
  • To specify the Index attribute, select it from the table and click Edit, which will open a form in which you can specify the following fields:
    
    
    
    
    
    
  • Parent Source Path - specify the list of data within the source data that represents the sub-components.
  • Source Path - specify the field that will provide the unique key.

To add a new attribute to be polled to a Polling step, click Add at the top of the page or via the Overflow Menu.

This will open the Add Attribute form on the right of the page, wherein you can specify the following:

• Name
• Source Path - specify the source path if you want a particular item based on one of its attribute values. You can click the Data Explorer button at the top of the form, from which you can navigate to and select the source path.
• Display Name - specify if you want to change the name of the attribute from its data name.
• Data Type - e.g. string, integer, float
• Display Format - e.g. traffic, percent, timestamp
• Visibility - whether the attribute is shortlisted, visible, advanced, hidden (see 2. General tab above).
• Graphable - whether the attribute can be used in a chart.
• Enumerable - whether the attribute is suitable for a pie chart category.
  • If set to On, the Transform field will appear below, from where you can specify on which attribute you would like to convert raw data into more meaningful or standardized values used by Entuity, and the mapping to use for this. The transform cannot be saved unless a mapping is defined. Note, a new transform cannot have the same name as an existing transform.
• Searchable - whether the Entuity search functionality should look at its value.
• Use this attribute for object display name - whether the attribute can be used as the display name for the object that it is on. Only one attribute can be used for this.
• Use this attribute for object status - whether the attribute can be used to drive the status icon for the object on which it appears.
• Event Mode - specify the basis on which events are raised, either None, Threshold or Status.

Once you have added an attribute, you can edit that attribute by selecting it and clicking Edit at the top of the page, or via the Overflow Menu or right-click Context Menu. This will open the Edit Attribute form, which contains the same fields as when adding an attribute.

To remove an attribute, select it and click Remove at the top of the page, or via the Overflow Menu or right-click Context Menu.

Once you have selected and edited attributes as you need, click Next. This will take you to the Summary tab.

7. Summary tab:

The Summary tab displays the details of all previous tabs in a single page.

To add another step to this poller, click New Step, which will take you to the Step tab again, where you can add or edit another step as you wish.

Click Save to save your poller. By default, the poller will be enabled. If the poller was previously disabled, you will be prompted as to whether you want to enable it. Once the poller is saved, it will appear in the table displayed under the Pollers tab.

Example workflows for adding a user defined REST poller:

Please see this article for examples of how to add a user defined REST poller.