How do I check or retrieve configuration files for a device? – Entuity

You can undertake device configuration retrieval for a device that is managed by Entuity, regardless of its management level (for example, Full, Ping Only, Basic, and so on), and if the device has a retrieval script file associated with either its device sysOid or vendor sysOid. Full configuration management is functionality is only available if there are also mapped exclusion and policy files.

Entuity retrieves and interprets device configuration files through three types of configuration:

retrieval task.
exclude differences.
policy rules.

Entuity associates device and vendor sysOids to the appropriate retrieval script, excluded differences, and policy rule files.

For example:

cisco-generic-exclusions.cfg(.1.3.6.1.4.1.9)
.
.
cisco-generic-policies.cfg(.1.3.6.1.4.1.9)
hp-generic-policies.cfg(.1.3.6.1.4.1.11)

Entuity includes retrieval tasks for Cisco, Dell, HP, Huawei, and Juniper devices. Devices that support configuration retrieval are then discovered as part of Entuity's standard discovery process. Entuity first attempts to match on a vendor sysOID. If that fails, then a configuration file is not associated with the device. You can amend the default association through Entuity, but you must be careful to avoid making an invalid association.

You can use the Entuity UI or RESTful API to automate the management of device configuration retrieval.

Pattern matching and policy files:

Before using configuration management to monitor network device configurations, you must amend pattern matching behavior and policy to meet your requirements so that you can identify device configuration change and policy violations. You can set up pattern matching and policy through:

generic exclusions files (Configuration Retrieval Excluded Differences File)
- these specify text patterns that configuration management can safely ignore when trying to identify important configuration changes, for example, timestamp changes.
- Entuity includes example generic exclusions files, such as cisco-generic-exclusions.cfg.
policy files (Configuration Retrieval Policy Rules File)
- these specify configuration lines to which good and bad configurations must conform.
- Entuity supplies example generic policy files for Cisco, HP and Juniper devices, the content of which you can amend to meet your requirements: cisco-generic-policies.cfg, hp-generic-policies.cfg.

Note, when amending exclusion and policy files, rename the files to ensure your changes are not overwritten during your next Entuity upgrade.

Managing policy files:

You can manage policy rules files and exclude files from the Policy Management page. To access the Policy Management page, click Administration from the Main Menu, and then click Policy Management.

For information on managing policy files, see this article.

Single configuration files vs dual (startup/running) configuration files:

Entuity supports devices that retrieve either a single configuration file or two ('startup' and 'running') configuration files.

This is an editable attribute, meaning you can specify whether a device retrieves one or two configuration files. You can specify this from the Attributes dashboard, under the attribute name 'Configuration Retrieval Number of Configurations'. By default, the following device types have two configuration files:

Aruba
Cisco
Dell
F5
H3C
HP
Siemens

All other device types have one configuration file, by default.

Throughout the UI, the configuration files are referred to in the following manner:

Single configuration file:

Configuration

Dual configuration files:

Startup configuration
Running configuration

To identify device configuration change:

When you retrieve device configuration information, Entuity compares the retrieved file (which is stored in the tftpHome directory in entuity.cfg) with the previously retrieved file (which is stored in the Archivedir directory in entuity.cfg). Entuity configuration management includes exclusion files to avoid flagging trivial changes in the configuration file (for example, timestamp changes), because these would not be meaningful. You can specify default patterns that configuration management must ignore through these files.

The pattern matching file is parsed when saved to the device. It takes one discovery cycle for changes to be included to Entuity. The patterns are stored in the Entuity database, although updates must be amended in the pattern matching file.

Trivial changes

Different configuration changes made to any line, or group of lines, that match one of the ignore patterns are not considered a change to the configuration file. These changes are considered as trivial.
Where the only difference between a newly retrieved configuration file and the last archived file are trivial, the newly retrieved file is treated as thought it is an exact copy of the archived one and is discarded. Entuity does not raise a change event.

Pattern matching rules are global, and applied to both network device startup configurations and running configurations, as well as singular configurations.

Device configuration changes are displayed in the device's Device Configurations dashlet table, which is in the device's Configuration Monitor dashboard.

To identify policy violations:

Entuity configuration management checks configuration files for policy violations, which are specified by policy mandated statement files. Policy files can be specified on a device from the device's Configuration Monitor dashboard, under its Settings (Configuration Monitor Settings) dashlet.

Policy files specify configuration lines to which good and bad configurations must conform.
Entuity supplies example generic policy files for Cisco, HP and Juniper devices, the content of which you can amend to meet your requirements.

Policy violations are identified through two sections in the file. When a pattern is violated, Entuity raises a policy violation event that includes the section name:

Include patterns, which identifies patterns that must be included in a configuration file.
Exclude patterns, which identifies patterns that must be excluded from a configuration file.

You can amend policy files through a text editor, external to Entuity.

Policy violations

When configuration management identifies policy violations, it can invoke policy violation events. Each policy definition includes a policy name, and when violated the policy name is included to the event.

When the Include section is violated, Entuity raises a CM Configuration Missing Policy Mandated Statement event on the first match of a particular violation.
When the Exclude section is violated, Entuity raises a CM Configuration Includes Policy Exclusions event.

Policy violation events have configurable expiry times, and corresponding clearing events that are raised when the configuration is found to have been edited to fix the violation.

To prepare for checking or retrieving configuration files for a device:

There are some stages you need to complete before you can check/retrieve the latest configuration (singular or running/startup) for a device:

Click Main Menu and then Asset Management.
From the Managed Assets tab of the Asset Management page, navigate to, and select, the device you want to check.
Click Modify Asset at the top of the tab or from the right-click Context Menu.
The Modify Asset form is displayed.
Under the Config Management section:
1. Specify the device to be Enabled for config management.
2. Click the Credential field to open the Credential form.
3. Specify the Access Method from the dropdown field, either SSH or Telnet.
4. Specify the Access Port on the device.
5. Enter the Username and Password that Entuity can use to access the device itself.
  You might require a second password for admin or 'enable' rights.
6. Click Done to save your changes.
  Note, these credentials are not automatically passed when logging in. You must use Expect methods inside your login groovy script to access them.
Navigate to the Configuration Monitor dashboard of the selected device.
In the Configuration Monitor Settings dashlet, click Edit Settings.
The Configuration Monitor Settings form is displayed.
Specify the parameters for configuration file retrieval, relating to the three types of configuration retrieval detailed later in this article.
Click Done to save your changes and return to the Configuration Monitor dashboard.

To check or retrieve configuration files for a device:

Note, this functionality requires the Configuration Monitor Check Config permission. For more information on user permissions in Entuity, see this article.

Once you have prepared the device for configuration retrieval, Entuity provides three mechanisms through which you can retrieve device configuration:

manual retrieval.
scheduled retrieval.
change-based retrieval.

Manual retrieval:

See the important note regarding the archiving of configuration files for information regarding the archiving of manually-retrieved files.

Navigate to the Configuration Monitor dashboard of the device you want to check.
Click Check Configuration Now at the top of the dashlet (or from the dashlet Overflow Menu or device's Context Menu).
Entuity then retrieves the latest (running/startup) configuration from the device, and if it is different from the currently recorded version, Entuity updates the device's Device Configurations dashlet table with the new file.

If the number of configuration files in the table exceeds the specified Number of Archives listed in the device's Configuration Monitor Settings dashlet, the table removes the oldest of these configuration files from the table.

Scheduled retrieval:

By default, Entuity does not activate scheduled retrieval of device configuration.

Navigate to the Configuration Monitor dashboard of the device you want to check.
In the Configuration Monitor Settings dashlet, click Edit Settings to open the Configuration Monitor Settings form.
Turn on the slider in the Nightly Retrieval field.
When enabled, Entuity retrieves configuration files daily at 02:00. Entuity creates a queue and then, at one-minute intervals, retrieves the configuration files from each monitored device, in turn.
Click Done to save your changes.

Change-based retrieval:

You can configure Entuity to check for changes in the (startup or running) configuration files' timestamps. A change in the timestamp indicates a change in the device configuration. After identifying a timestamp change:

Entuity waits until 2 consecutive polls return unchanged timestamps.
Entuity then waits a set period, as defined through lcm.SNMPTriggerHoldOffTime in entuity.cfg.
After the hold time elapses, Entuity checks the latest poll of the device. If the timestamp remains unchanged (which indicates updates to the configuration are complete), Entuity then initiates a configuration retrieval.

To enable change-based retrieval:

Navigate to the Configuration Monitor dashboard of the device you want to check.
In the Configuration Monitor Settings dashlet, click Edit Settings to open the Configuration Monitor Settings form.
Turn on the slider in the Change Based Retrieval field.
Click Done to save your changes.

Results of the configuration retrieval:

If the configuration retrieval fails, Entuity raises the CM Startup Configuration Retrieval Failed event or CM Running Configuration Retrieval Failed event.

In the Configuration Monitor Status dashlet, the Last Attempted field displays the time of the failed retrieval attempt and Last Retrieval Outcome is set to Failed.
You can view the task history to see its diagnostic data and understand the details of retrieval's failure.
You can also check for other events that are raised against the device, for example, Network Outage, to identify whether retrieval failure is a symptom of a more widespread problem or whether it is the real issue.
You might also want to identify whether this event is raised against one or more devices:
- if it is raised against one device, then it may be an issue specific to the device, for example, the device is down. However, if you have initiated a manual retrieval, it may be a more widespread issue that is yet to show itself.
- if it is raised against many devices:
  - check the transfer servers - although configuration monitor is configured to work with the specified server, it does not check that the server is running when attempting a retrieval. If the server is not running, the retrieval will fail. You can configure transfer servers (including OpenTFTPserver) to run in logging mode. You can then use the transfer server log files to identify the success of file uploads and troubleshoot any issues.
  - check that the specified transfer and archive folders exist and that they permit your transport servers to write to them.
  - check CLI credential sets are still valid.

If the configuration retrieval succeeds and identifies a configuration change, Entuity raises an appropriate event:

If the configuration retrieval succeeds and does not identify a configuration change, Entuity updates the Last Attempted field to display the time of the successful retrieval attempt, and Last Retrieval Outcome is set to Succeeded.

If the device configuration was previously identified as not being saved but is now saved, then Entuity raises a CM Previously Unsaved Configuration Saved event.

You can also refer to the configuration monitor reports:

Configuration Monitor Settings report - summarizes the current configuration monitor settings of devices.
Device Configuration Status report - details the last attempt at configuration retrieval.
Device Configuration Summary report - summarizes the device configuration of the reporting period.

Archiving retrieved configuration files:

When the retrieval successfully completes, Entuity only archives the files if they include a non-trivial change when compared to the previously archived files, or a specified policy violation. Newly archived files appear as a new row in the device’s Web UI Archived Configuration Files table.

See the important note regarding the archiving of configuration files below for information regarding the archiving of manually-retrieved files.

By default, Entuity retains a maximum of four archived configuration versions, including the current configuration. You can amend this number in the # of Archives field in the Configuration Monitor Settings form, setting it to a value between one and 10.

The following table details the configuration structure:

Attribute	Description
$ARCHIVEDIR	directory chosen for the archives during configure.
$DEVICE_ID	numeric StormWorks identifier.
$CONFIG_TYPE	either running or startup.
$CONFIG_FILE	the file itself.

A configuration file has the format:

<$DEVICE_ID>-<$CONFIG_TYPE>-<$TIMESTAMP>

where $TIMESTAMP is the time that configuration was triggered.

If a device signals an error condition during an attempted configuration check, Entuity raises CM Startup Configuration Retrieval Failed and CM Running Configuration Retrieval Failed incidents and events. The details string identifies the device and the configuration file. If the retrieval failure is a symptom of other problems with the network or the device itself, other components of Entuity alert the user to the probable cause.

To upload archived configurations from a specified directory on a device:

Applicable to Entuity v22.0 upwards

You can specify an archived configuration to be parsed from a chosen device, that can then be used in the groovy script specified in a custom step of a custom configuration management task.

This is achieved by specifying a parameter with a new Data Type in the custom configuration management task. The new parameter type allows you to select an archived config when running the task.

For example, you might want to write a script to put an archive config onto a temporary directory on the device. This new parameter type allows you to do this.

Create a new (custom) configuration management task.
Under the New Parameter section, specify the Data Type as Archived Configuration.
You can leave the Default Value field as empty.
In this example, the Name of the new parameter is 'Test upload archive config'.
Save your parameter and custom configuration management task.
In the Entuity UI, navigate to, and select, the device from which you want to upload archived configuration.
From the right-click Context Menu, click the custom configuration management task (in this example, 'Test upload archive config').

The task's Parameters form is displayed, listing the parameters that you have specified for this task.
Click the Archived Configuration parameter that you have added (in this example, the parameter is named 'Test upload archive config').
The available archived configs are displayed in the dropdown fields.
Select the archived config you want to parse into the script specified in the task's step, and then click Save.

Important note regarding archiving of configuration files:

A file will only be archived if the configuration task is run as a configuration monitoring task, or, in the following ways:

if a configuration task is scheduled for nightly retrieval.
using Check Configuration Now (either from the Device Configurations dashlet of the Configuration Monitoring dashboard, or the right-click Context Menu).

However, if the configuration task is run by selecting the specific configuration task itself from the right-click Context Menu > Configuration Management > [configuration retrieval task name], for example,

then the task will only be moved to the tftpHome directory in entuity.cfg, and will not be processed and moved to Archivedir under entuity.cfg. Additionally, this retrieval will not be displayed in the Device Configuration dashlet of the Configuration Monitor dashboard.

To view device configuration information by View:

You can see individual device configuration information by View from the Device Configurations Summary dashlet. This can be accessed by either:

Configuration Monitor dashboard for a View.
by adding a Device Configurations Summary dashlet to a custom dashboard and then accessing that custom dashboard in the context of your desired View.

Example of excluded differences from pattern matching files:

The following extract includes two ignore pattern lines. The first ignores a changing timestamp, and the second ignores a clock change:

#--------------------------------------------------------------------
# Ignore patterns
#--------------------------------------------------------------------
# Ignore timestamps
! Last configuration change at.*
#Ignore ntp clock change
ntp clock-period.*

In pattern matching files:

lines starting with a hash, #, are considered as comments and are ignored.
patterns that span several lines should use \n (escape n) to signal a newline.
lines ending with a dot asterisk, .*, include the wildcard character. This is used to allow matching on the parts of the line that vary from retrieval to retrieval. Matches must be otherwise exact. For example, the pattern:
service timestamps
matches only the first of the following three lines

service timestamps
service timestamps debug uptime
service timestamps log datetime

This pattern:
service timestamps.*
matches each of the three lines.

All pattern matching is against the original text, never against the transformed text. Where a line matches one or more patterns, the line is handled the same as though it matched only one.

Examples of policy mandated statement files:

Example of policy include files:

[PolicyMustInclude logging]
IncludePattern=^logging.*

[PolicyMustInclude logging_buffered]
IncludePattern=^logging buffered.*

[PolicyMustInclude snmp_server]
IncludePattern=^snmp-server.*

#[PolicyMustInclude interface_FastEthernet_no_ip_proxy-arp]
#IncludePattern=^interface FastEthernet.*no ip proxy-arp.*

#[PolicyMustInclude interface_FastEthernet_no_ip_unreachables]
#IncludePattern=^interface FastEthernet.*no ip unreachables.*

#[PolicyMustInclude interface_FastEthernet_no ip_redirects]
#IncludePattern=^interface FastEthernet.*no ip redirects.*

#[PolicyMustInclude interface_FastEthernet_no mop_enabled]
#IncludePattern=^interface FastEthernet.*no mop enabled.*

where:

^logging.*, checks that logging is enabled.
^logging buffered.*, a second check for enabled logging, by checking the router has the buffer enabled.
^snmp-server.*, checks SNMP server is enabled.
^interface FastEthernet.*no ip proxy-arp.*, checks proxy ARP is disabled on the device. Proxy ARP may have security and performance overhead:
- increasing the amount of ARP traffic on your segment.
- hosts need larger ARP tables to handle IP-to-MAC address mappings.
- a machine can claim to be another in order to intercept packets, an act called
  "spoofing."
^interface FastEthernet.*no ip unreachables.*, checks the router configuration prevents sending of ICMP unreachable message, the information within which can be used for DNS ping attacks.
^interface FastEthernet.*no ip redirects.*, checks routers do not support IP redirects. IP redirects allow the sender to bypass the router and forward future packets directly to the destination (or a router closer to the destination).
^interface FastEthernet.*no mop enabled.*, checks maintenance operation protocol is disabled, reducing network traffic.

Example of policy exclude files:

[PolicyMustExclude no_logging]
ExcludePattern=^no logging.*

[PolicyMustExclude no_snmp-server]
ExcludePattern=^no snmp-server.*

[PolicyMustExclude snmp-server_community_public]
ExcludePattern=^snmp-server community public.*

[PolicyMustExclude snmp-server_community_private]
ExcludePattern=^snmp-server community private.*

[PolicyMustExclude service_tcp-small-servers]
ExcludePattern=^service tcp-small-servers.*

[PolicyMustExclude service_udp-small-servers]
ExcludePattern=^service udp-small-servers.*

[PolicyMustExclude ip_finger]
ExcludePattern=^ip finger.*

[PolicyMustExclude ip_ident]
ExcludePattern=^ip ident.*

[PolicyMustExclude tftp-server]
ExcludePattern=^tftp-server.*

# The following might sometimes be desirable
[PolicyMustExclude ip_http_server]
ExcludePattern=^ip http server.*

[PolicyMustExclude service_config]
ExcludePattern=^service config.*

[PolicyMustExclude boot_network]
ExcludePattern=^boot network.*

[PolicyMustExclude interface_ip_mask_reply]
ExcludePattern=^interface.*ip mask reply.*

[PolicyMustExclude interface_ip_directed-broadcast]
ExcludePattern=^interface.*ip directed-broadcast.*