Data Storage

CygNet Software uses Microsoft’s Extensible Storage Engine (ESE) data storage technology for its Value History Service database technology. ESE is used for all CygNet’s database services and offers the following advantages to the VHS:

• Robustness — ESE is extremely robust with ACID compliance (Atomicity, Consistency, Isolation, and Durability), improved performance in database startup times, improved data retrieval speed, and a powerful crash recovery mechanism.
• Capacity — The ESE-based datastore supports very large database sizes. The maximum size of a VHS datastore is 5000 GB (5 TB).
• Extensibility — An ESE-based datastore will support future data structure extensions.
• Reliability — ESE has been the underlying data storage technology of several core Windows components including Microsoft Exchange Server, Active Directory, and many other Microsoft Windows components since the release of Windows 2000.

VHS File Synchronization

The VHS data files consist of the main data storage file (history.edb) and the transaction log files (\txlogs directory and files), which is by default in the service’s data directory. The directory location for the transaction log files is set with the service configuration keyword VHS_TRANSACTION_LOG_DIR. The default value for this keyword is \txlogs and is set in the service’s configuration file (Vhs.cfg). However, if you prefer to store the transaction log files in another location, you may do so.

These files work together to store all historical data. See How ESE Works and ESE File Types for more information about the database engine operation.

VHS Legacy Data Storage

Older versions of CygNet used a legacy data storage technology for its VHS. The ESE data storage technology for the VHS was introduced in CygNet v8.1.1. CygNet v8.1.3 was the last release that supported the VHS legacy datastore.

The following information applies to a VHS legacy datastore.

The ESE-based VHS requires more disk space than the VHS legacy datastore due to its large number of transaction log files. Because of this, ensure that you have enough available disk space for your ESE-based VHS; we recommend at least twice the disk space of the old datastore.

Legacy Data Files

The three legacy data files (HISTORY.DAT, BLOCKNDX.DAT, and NAMES.DAT) are a matching set of service files. There is information within the files that interrelate with each other, so mixing data files from other VHS services would not be an acceptable practice. For example, data within the NAMES.DAT file is referenced in the HISTORY.DAT file.

The VHS marks the data files set with an identifier when it starts up and when it shuts down. This allows it to synchronize the file set so that it can detect if a particular data file does not belong with the others. If this situation occurs, the VHS will not start and will log that the data files do not match. This allows the VHS to enforce the rule that the set of data files must originate together from the same source, and that a single data file cannot be brought into the set from another service. This prevents the possibility of starting up the service in an unstable condition.

If for some reason an administrator feels that a set of files should go together despite the VHS reporting they are stamped differently, the VHS Check Utility can be used to reset the data file versions so that they can be used together. If the utility detects that the files do not match, an error message will be issued. You can then choose to resynchronize the files. This should not be a common occurrence and should be exercised with extreme caution. Although the VHS will start once the files have been resynchronized, if the data files are in fact not from the same origin then it is not guaranteed that all history data in the mismatching files will be preserved.