ARS Redundancy

When more than one Address Resolution Service (ARS) is in operation in a CygNet domain (which is generally the case in a multi-host system) the ARS databases become redundant. Redundancy is ensured through ARS-to-ARS communication on service changes and frequent database comparisons. Any data discrepancies are noted, and the ARSs are resolved based on the timestamps of database entries and ownership.

ARS-to-ARS Communication

ARS-to-ARS communication is established through broadcasting an "I_AM_HERE" message. When another ARS gets the broadcast, it records the service information (service name, operational status, and address) in its database and the computer registry. If the broadcast is not sufficient to establish communication between ARSs due to network architecture, the utility ConnectARS can be used to connect the services.

Once communication is established, each ARS sends out the I_AM_HERE message every few minutes to let the other ARS’s know it is still running. This message is broadcast and sent directly to the addresses of the other ARSs in the registry.

ARS Timestamps

All changes to an ARS database are timestamped. Each time a database change occurs within an ARS, that ARS notifies the other ARSs and forwards the change. (Periodic updates are sent approximately every 10 minutes.) If one of the other ARSs contains discrepant data, the entry with the most recent timestamp is considered the most current data and all ARSs are updated accordingly.

In certain situations, the receiving ARS may have a newer timestamp for a service status. This occurs when the clock of the ARS sending the update is behind the clock of the ARS receiving the update. This is why it is important that one ARS on a multi-host domain be configured as the License "Master."

Example

Two hosts are at 15:00 Greenwich Mean Time (GMT). SITE1 sends a service status message to SITE2. Then, the clock of SITE1 is set back to 14:00 GMT. At 14:05, the SITE1 sends another service status message to SITE2. This time, SITE2 ARS ignores the message because it has an update that is more recent (15:00 GMT) than the timestamp of the current message (14:00 GMT).

When the services are running and the clock is adjusted, this is not a problem because both ARSs have the registered address of each service and can continue to communicate with the services (in particular, send health checks).

If a service is not running and the clock is adjusted, this results in a synchronization issue because the ARS rejects the "old" message regarding the service status change. As such, the service with the newer message continues to show the service out of service and has no registered address for the service. An ARS only perform health checks on services it has listed with an operational status of OK.

In the example above if the services were stopped when the clock was adjusted and then restarted, the SITE2 ARS would show the SITE1 services (except the ARS) out of service even though they are running on the SITE1 host. This is because the SITE2 ARS rejects the "old" update message from SITE1. Conversely, the SITE1 ARS would show all services from both sites as running. (The SITE1 ARS would be shown as running on SITE2 because ARS status updates are handled differently than other service types.)

If this type of situation occurs, you can remedy it immediately by stopping the services, resetting the clock back to original date/time, then restarting the services. Be aware that this may result in overlapping or gaps in data. Or, if you do not want to reset the clock, wait until the time period elapses and then restart the services. In the example above, you would have to wait one hour.

Host Time and the ARS

The host server clock must be set to accurately reflect the time settings of the field. This includes selection of the proper time zone, and if appropriate, automatic adjustment for daylight saving time. See Daylight Saving Time for more information about daylight saving time and CygNet.

Host time is particularly important to the ARS on a multi-host domain. The ARS uses it to keep in sync with the other ARSs.

One of the purposes of the License Master ARS is to alleviate time discrepancies between hosts. This is done by synchronizing the clocks of all ARSs on a domain. The time is sent in GMT. Each host server sets its clock accordingly and displays the time based on its time zone/daylight saving time settings. If your company uses a time synchronization utility, disable this utility on all host servers except the License Master. Otherwise, the utility and License Master will be continually resetting the time of the non-License Master host servers.

The ACCEPT_TIME_SYNC keyword in the ARS configuration file (Ars.cfg) is used to manage time synchronization between the computer time of an ARS and the computer time of the License Master. By default the time difference is periodically evaluated and the computer time of the ARS is synchronized with the computer time of the License Master, regardless of the time delta between the two computers. Set the keyword to "NO" to synchronize the times only when the computer times are significantly different (10 seconds or more).

Ownership

When a service is added to an ARS, that ARS is set as its owner. Ownership is another item used to ensure ARS redundancy. A service can be deleted only from its owner. Ownership can be changed from one ARS to another.

Service Startup Domains

The following information about domains apply when starting services:

Once an RSM is running on a domain, all services that it starts will be on the same domain.

Back to top