Managing Validation Engines

Use the Validation button on the Admin menu to access options to manage validation engines and their component rules. You can add, view or edit, delete or copy estimation engines via this interface.

The CygNet Measurement system supports the configuration of multiple validation engines, which check that measurement data conforms to specification. Set up multiple validation engines to ensure that different validation handling processes can be applied to specified Nodes and circumstances. The validation process will quickly identify data of suspect quality, by verifying the data against discrete values, historical values, and/or station totals, and will then make the results available for user analysis and closing. Data records failing the validation process will generate exceptions, which can be viewed and handled via the Exceptions Control.

You must have proper security authorization to perform these actions. See FMS Security (and ACCESS security event) for information about configuring security access to validation engines and rules.

Validation Engines and Rules

Multiple validation rules can be grouped together into a validation engine with well-defined applicability triggers. Within each engine, you can configure rules and related parameters accordingly. Each component validation rule can be customized and configured by an authorized user to activate for specified times (e.g. day(s) of the week, or a time window), and applied using a preferred time schedule. If a device resolves to multiple validation engines, then all the component rules will be used according to their individual settings.

Validation engines can be configured on a group or device level. Because an engine can be defined and assigned to a group Node, the engine and its component rules apply indirectly to the multiple devices within the group. However, you can modify the group-level rules for a device by also applying different validation engines to specific device Nodes that belong to the group. When the system needs to validate data for a device, it will use the rules from as many engines as are configured for the given device. For example, if the device belongs to five groups, each with configured validation engines containing associated rules, plus has an engine configured directly for the device, then FMS will validate all six sets of rules. If multiple engines are evaluated that contain the same validation rule, then that rule will be tested by each of the validation engines containing it.

Gas records are automatically assigned a data quality flag from the validation engine as data comes into the system. The purpose of the flag is to indicate the level of confidence in the validity of the data. Each validation rule can have an associated data quality level set, to be assigned if the data fails the rule. If multiple validation rules are failed, then the data quality of the affected records will be set to the lowest of the configured values.

Exceptions are generated for the records that are affected by the invalid data. Each time a record fails a validation rule, a related exception will be generated. Any given record can generate one or more exceptions.

Managing Validation Engines

Validation engines are configured via the Admin menu in FMS Explorer. Click Validation on the Admin ribbon to access the Validation Engines dialog box.

The Validation Engines dialog box lists all defined validation engines in the measurement system. Each row displays a configured validation engine name, details about the Node to which it applies, the active begin and end times, and whether or not the engine is currently enabled. Results can be filtered by the values of any column. You can also access options to add, view or edit, delete, or copy validation engines and rules from this dialog box.

The properties for the Validation Engines dialog box are described below.

Creating Validation Engines

Use the Add button to add a new validation engine in your system via the Create New Validation Engine dialog box.

The Create New Validation Engine dialog box accommodates configuration of all validation rules and properties necessary to define a new validation engine. Each row will display a configured validation rule name, details about the category and type of rule, the severity level to assign to data records that fail the rule, and the activation schedule to assign to the rule. The list can be filtered by the values of any column. You can access options to add a rule, delete a rule, or schedule activation of a rule from this dialog box. Configure all of the rules and properties that will comprise the new validation engine via this dialog box.

The properties in this dialog box are similar to the properties in the Validation Engine Properties dialog box (with the exception of the historization navigator), and are described in the Validation Engine Properties table below.

To Create a New Validation Engine

1. On the Validation Engines list, click Add to access the Create New Validation Engine dialog box.

   

2. On the Create New Validation Engine dialog box, define the engine and its Node assignment. See Validation Engine Properties for more information.

1. Enter a Name for the new engine.
2. Click to select/clear the Enabled check box.
3. Select the Node to which the validation engine will be assigned.

3. Configure new validation rules for the engine. See Configuring Validation Rules for more information.
4. When all rules and properties for the new validation engine are configured, click Create.

Viewing or Editing Validation Engines

Use the Edit button to modify or view the properties for an existing validation engine, or to view its historized records, via the Validation Engine Properties dialog box.

The Validation Engine Properties dialog box displays all of the validation rules and properties comprising an existing validation engine. Each row displays a configured validation rule name, details about the category and type of rule, the severity level assigned to failure of the rule, and its activation schedule. Results can be sorted by the values of any column. Click to select a validation rule to view or edit its properties. You can also access options to add a rule, delete a rule, view historical rule details, or schedule activation of a rule from this dialog box. View or edit all of the rules and properties comprising an existing validation engine via this dialog box.

The properties in this dialog box are described in the Validation Engine Properties table below, and are similar to the properties in the Create New Validation Engine dialog box.

To View or Edit a Validation Engine

1. On the Validation Engines list, click to select an existing engine and then click Edit to access the Validation Engine Properties dialog box.

   

2. On the Validation Engine Properties dialog box, you can view or edit properties defining the engine and its Node assignment. See Validation Engine Properties for more information.

1. Optionally edit the engine Name.
2. Optionally click to select/clear the Enabled check box.
3. Optionally reassign the Node to which the validation engine applies.

3. Optionally create or edit validation rules belonging to the engine. See Configuring Validation Rules for more information.
4. Click Apply and repeat the process for additional engines, or click OK when all desired edits have been made.

Validation Engine Properties

The properties for the Validation Engine Properties dialog box, and for the similar Create New Validation Engine dialog box, are described below.

Deleting Validation Engines

Use the Delete button to either delete a Validation Engine from a point in time, or to completely purge a Validation Engine from the system. You can only take this action for one validation engine at a time.

To Delete a Validation Engine

1. On the Validation Engines list, click to select an existing engine name and then click Delete to access the Delete Validation Engine dialog box.

   

2. On the Delete Validation Engine dialog box, select whether to delete the validation engine from a Point in Time, or to completely purge the validation engine from the system, and then click the radio button corresponding to your selection.

   • Delete from a Point in Time removes (effectively deactivates) the selected Validation Engine record from a specified Point in Time (in Contract time, (C)), and historizes the change. The Validation Engine still exists in the system, and its properties can still be edited. Selecting this option causes the action button at the bottom of the dialog box to read Delete.

     If you select to delete from a Point in Time, also specify the PIT using the date/time selector provided. See Using the Home Menu for more information about PIT.

   • Purge from system removes the selected Validation Engine and its data records from the system entirely. Selecting this option causes the action button at the bottom of the dialog box to read Purge.

   Note: Be certain of your selection, as there is NO "undo" for this action.

3. Click Delete or Purge.
4. A message will appear asking you to confirm that you wish to delete/purge the selected Validation Engine from your system. Select Yes or No.

Copying Validation Engines

Use the Copy button to make a new validation engine exactly like an existing engine, and then alter it to meet a new purpose.

You will be prompted to save a record of the action, requiring you to enter effective dates, audit category, and comments. (You will be prompted to enter this information again once you save any edits you make to the copied validation engine.) See Save FMS Record for more information.

The new validation engine will contain all the properties of its source. Click the *copy* name and then click Edit to assign a new engine name, Node, and/or edit the engine properties, as desired. See Viewing or Editing Validation Engines for more information.

To Copy a Validation Engine

1. On the Validation Engines list, click an existing engine name and then click Copy to duplicate the existing validation engine.
2. The Save FMS Record dialog box will appear. Configure the date and audit information requested to historize the creation of the new engine in your system, and then click Save. See Save FMS Record for more information.

3. The duplicate validation engine will be created at the top of the list, and the new engine name will be the existing engine name, prefixed with *copy*. The new engine will initially contain all the properties of its source engine.

   

4. Click to select the newly created *copy* validation engine in the list, and then click Edit to access the Validation Engine Properties dialog box to edit the default properties (name, status, Node assignment, rules) as desired. See Viewing or Editing Validation Engines for more information.

Historization Navigator

The Validation Engine Properties dialog box contains a historization navigator. Use the historization navigator to access sequential historized records for the selected validation engine, where applicable.

The following fields are displayed in the historization navigator.

Save FMS Record

After creating or editing a validation engine or rule, you'll be asked to save your changes via the Save FMS Record dialog box. Configure the date and audit information requested to historize the changes in your system.

More:

Configuring Validation Rules

FMS Validation Rules