Troubleshooting the Archiver

If you are experiencing problems with your Archiver, learn about the symptoms, potential causes, and solutions to help you troubleshoot the issue.

Symptoms

Here are the common issues you might experience with the role. To help you solve the issue, click the symptoms that you are currently experiencing.

Things to consider

Archiver role is offline

If the Archiver role is offline (icon is red), check whether the server that is hosting the Archiver role is online and operational. To help you troubleshoot this issue, learn about its possible causes and their respective solutions.

Host server is offline

Description of cause: The server hosting the Archiver role is offline (server icon is red in Config Tool). To check the status of the server, open Config Tool, and then click System > Roles > [Your role] > Resources.

Solution: In Windows Services, ensure that the Genetec™ Server service is running on that server. If the service is running, make sure that the expansion server has connected successfully to the Directory. If the server fails to start, troubleshoot the server issue.

Incorrect NIC priority

Description of cause: The priority for the network interface cards (NIC) is incorrect. The order in which the NICs appear might be incorrect in either the network settings in your Windows Control Panel or in the Network View task in Config Tool.

To check the NIC priority, open Config Tool, and then click Network view > [Select your server from the Networks list] > Properties.

Solution: In both your Windows Control Panel and Config Tool, make sure that NIC used for the Genetec™ Service appears at the top of the list of network interface cards.

Archiver ports are closed

Description of cause: The ports used by the Archiver role might be either closed or in use by other applications.

You can use Telnet Client to confirm whether communication between servers is open. To check the ports using Telnet Client, open the command prompt as administrator and enter the following command: telnet <IP address> <port number>. For example, if your IP address is 172.10.10.10 and you want to check port 4502, enter: telnet 172.10.10.10 4502. To enable Telnet Client, in Windows, click Control Panel > Programs and Features > Turn Windows features on or off, and then select Telnet Client from the list of features.

Solution: Make sure that the Archiver role's ports are open and not in use by other applications. For a list of required ports used in Security Center, click here.

Antivirus or firewalls are blocking communication

Description of cause: Your antivirus or Windows firewall is obstructing communication between servers.

Solution: Disable or add exceptions to your antivirus or Windows firewall.

Host server has low disk space

Description of cause: The server hosting the Archiver role is running out of disk space on its C drive. Low disk space can have negative effects on your server's performance and cause processes to falter or fail.

Solution: Free up disk space on the server's C drive and restart the Genetec™ Server Service. To save disk space, you can change where the Archiver logs are saved.

High CPU or memory usage on host server

Description of cause: The CPU or memory of the server hosting the Archiver role is overloaded. Using the Task Manager, check whether the machine is running within the Security Center specifications. To check CPU and memory usage, open the command prompt as administrator and enter the following command: taskmgr.

Solution: Do the following:

Make sure the server is set up according to the Security Center specifications.
Investigate processes and applications that use the most CPU or consume excessive amounts of physical memory (RAM). Consult the vendors of the respective applications or softwares, if required.
Close all unwanted application that might affect CPU and RAM usage.

Archiver role is corrupted

Description of cause: If the Archiver role is offline but the Genetec™ Server Service is running on the host server, the Archiver role might be corrupted.

To determine if the Archiver role is corrupted, check for configuration (config) files in the following folder of your installation drive: ProgramFiles(x86)\Genetec Security Center [5.x]\Configuration Files. If the role is corrupted, the General Settings or Archiver.gconfig configuration files are missing. Another indication that the Archiver role is corrupted is if Config Tool or Security Desk produce exception handling errors when started, prompting you to use their respective installation files to repair the application.

Solution: Create a new Archiver role on the same server and use the same database. If the role connects, move the video units from the old Archiver role to the new one, using the Move Unit tool in Config Tool. Once the video units have been moved, delete the old Archiver role.

System process or application is affecting Archiver role

Description of cause: A Windows process or application is negatively impacting the Archiver role. If this is the case, there might be Archiver-related errors in the Windows event logs or dump files that have been created.

To check the Windows logs, open the command prompt as administrator and enter the following command: eventvwr.msc. Make sure to check application, system, and Genetec event logs. To check whether dump files have been created, open C:\Program Data\Genetec Security Center [5.x], and then open the Dumps or Logs folders.

Solution: Contact technical support and provide the event logs and dump files.

Archiver database is not connected

If the Archiver database is not connected, check whether you have sufficient SysAdmin rights. To help you troubleshoot this issue, learn about its possible causes and their respective solutions.

Missing SysAdmin rights

Description of cause: The Genetec™ Service logon user does not have SysAdmin rights on the Archiver database in SQL. To check this, open SQL Management Studio, click Security > Logins > [Right-click on the user] > Properties > Server roles, and then select public and sysadmin.

Solution: In SQL Server Management Studio, add the SysAdmin server role to the user who is running the Genetec™ Server service.

Archiver database is full

Description of cause: The Archiver database has reached its size limit. In SQL Server 2008 Express and later, the size limit is 10 GB.

Solution: Upgrade SQL Server Express to the Standard or Enterprise editions. As a workaround, you can do the following:

Create a new database
Reduce the retention period of the recordings
Contact tech support to help you determine why the database has reached its storage limit

Database transaction log is too big

Description of cause: The transaction log (LDF file) of the SQL database is too big.

Solution: In SQL Management Studio, do the following:

Switch the recovery model of the database to Simple.
Shrink the transaction log file.
Set a maximum size limit for the transaction log file.

Remote database ports are not open

Description of cause: The remote database ports, TCP 1433 and UDP 1434, are not open or they are being used by other applications and causing conflicts.

Solution: Open the ports and make sure that they are reserved for the SQL Server.

Cannot add network drives for Archiver storage

If you cannot add network drives for Archiver storage, make sure that you have the required read/write access to the storage disk.

Cause

The user running the Genetec™ Server service does not have read/write access to the storage disk.

To check the read/write access on the storage disk, in Windows, right-click the recording drive, click Properties > Security, and assign full access privileges to the user.

Solution

For both the storage disk and its subfolders, grant read/write access to the user running the Genetec™ Server service.

Archiver role cannot write to disk

If the Archiver role has stopped recording video and the logs indicate that the role cannot record to any drive, there might be an issue with the disk or the role configuration. To help you troubleshoot this issue, learn about its possible causes and their respective solutions.

Storage drives not accessible

Description of cause: The storage drives that are selected in the Resources tab of the Archiver role are inaccessible (icons are red) and Security Center cannot communicate with the drives.

Solution:

If the drives are local to the Archiver server, do the following:
1. Log on to the server using the same credentials that the Genetec™ Server service uses (verified by running services.msc from the Windows Start menu).
2. In Windows Explorer, find the folder where the archived footage is stored.
  Enable Show hidden files, folders and drives in case the folder is hidden.
3. Try creating a simple text file.
  If you cannot create the text file, you do not have the required write privileges. To get the required privileges, contact your administrator.
If the drives are hosted elsewhere on the network, do the following:
1. Log on to the affected Archiver server using the same credentials that the Genetec™ Server service uses (verified by running services.msc from the Windows Start menu).
2. Open Windows Explorer and try accessing the remote drives.
  - If you cannot access one ore more of the remote drives, either your access is denied because you are not in the list of permitted users for those drives, or there is a communication failure between the Archiver and the remote drives. This failure is most likely caused by a network issue. Try investigating and repairing the issue.
  - If you can access the remote drives, do the following:
    1. In Windows Explorer, find the folder where the archived footage is stored.
      Enable Show hidden files, folders and drives in case the folder is hidden.
    2. Try creating a simple text file.
      If you cannot create the text file, you do not have the required write privileges. To get the required privileges, contact your administrator.

Recording disk full

Description of cause: All disks allotted for archiving are full and the Archiver is unable to free disk space by deleting existing video files. This event can occur when another application has used up all the disk space reserved for Security Center, or when the Delete oldest files when disks full option is not selected in the Server Admin. When this happens, archiving is stopped. The Archiver re-evaluates the disk space every 30 seconds.

You can verify the statistics of the drives by checking the free space compared to the minimum free space in the Resources tab of the Archiver role. You can also check for disk full or disk 80% events in the Archiver logs (C:\ArchiverLogs by default).

Solution: Do the following:

Ensure that the Archiver role is set to delete oldest files first when the disk is full.
In the Camera default settings tab of the Archiver role, reduce the Automatic Cleanup threshold so that older archives are deleted automatically.
Lower the retention period of the recordings.
In the Resources tab of the Archiver role, reduce the Min. free space space value. Make sure that the minimum free space value is at least 0.2% of the value of the total size.
Increase your storage capacity by adding more storage drives to the server.

Recording disk full due to orphan files

Description of cause: If the drives are full in Windows Explorer, there might be orphan files present. Orphan files are video files that exist on the drive but are not detected by Security Center. As a result, orphan files cannot be deleted when either of the following options are enabled:

Automatic cleanup (Camera default settings tab of Archiver role)
Delete oldest files when disks are full (Resources tab of Archiver role).

Solution: Check whether orphan files are present. If you find orphan files, delete them to free up disk space or re-integrate them into the database. For more information, see Finding orphan files on your system.

Missing read/write access

Description of cause: The user running the Genetec™ Server service does not have read/write access to the storage disk.

To check the read/write access on the storage disk, in Windows, right-click the recording drive, click Properties > Security, and assign full access privileges to the user.

Solution: For both the storage disk and its subfolders, grant read/write access to the user running the Genetec™ Server service.

Hardware or software issues

Description of cause: The disk on the server hosting the Archiver role is experiencing hardware or software issues.

Solution: Do the following:

In Windows, check if there are any disk errors in the Device Manager.
In Windows Event Viewer, check if there are any errors or unusual messages in the event logs.
Run a diagnostic check, using your storage's dedicated diagnostic tool.

Archiver performance is slow

If your Archiver performance is slow, check that your server specifications meet the Security Center requirements. To help you troubleshoot this issue, learn about its possible causes and their respective solutions.

Insufficient server specifications

Description of cause: The server specifications do not meet the Security Center requirements, thus causing performance issues on the server.

Solution: Make sure your Archiver server specifications meet the Security Center requirements.

Server's memory is being used up

Description of cause: The SQL Server Service is using most of the server's physical memory (RAM). By default, the SQL Server Service is set to use all of the RAM available on the server. As a result, not much memory is left for the OS and other applications that are running on the server.

Solution: Limit the amount of RAM that the SQL Server Service can use.

Hard disk drives are slow

Description of cause: The hard disk drives of the server that the Archiver role is running on and recording to are slow.

Solution: In Windows, go to Resource Monitor > Disk to view the write speed of the server's hard disk drive. If required, contact your storage provider to upgrade your hard disk.

Outdated drivers

Description of cause: The network card (NIC) drivers and hard disk drivers are outdated. To check the NIC drivers, open Device Manager in Windows, and then click Network Adapters > Properties. To check the hard disk drivers, open Device Manager in Windows, and then click Disk drives > Properties.

Solution: Update your network card drivers and hard disk drivers to their latest versions.

Archiver is handling motion detection

Description of cause: The Archiver is handling motion detection on a large number of cameras.

Solution: Switch the motion detection settings of your video units to Unit instead Archiver. If this is not doable, you can reduce the number of video units that detect motion and set them to record continuously instead.

BIOS configuration is not optimal

Description of cause: The BIOS settings are not set to Performance. Adjusting your server's BIOS settings can help increase the server's overall performance.

Solution: Set the server BIOS settings to Performance.

Creating a new Archiver cache in Security Center

If the Archiver is yellow in Security Center and you receive an Archiver agent failed to start on the server error message, or if you are experiencing strange camera behavior, your Archiver cache might be corrupted. You can create a new Archiver cache to try and resolve these issues.

What you should know

In Security Center, all camera settings are stored in the Directory database. The settings are sent from the Directory to the Archivers in the form of .cache files, and then the Archivers apply the settings on the cameras they control. If an Archiver cache becomes corrupted, you might experience strange camera behavior, or the Archiver might be yellow in Security Center. The cache can get corrupted if you replace a video unit, upgrade your system, or add a video unit in Security Center from a VMS.

If you receive an Archiver agent failed to start on the server error message in Security Center, the following exception is shown in your Windows event logs:

--------  Exception ----------------
System.ApplicationException: System.Exception:UIProxyAdapter creation failed. --->
System.Runtime.InteropServices.SEHException: External component has thrown an  exception.at Genetec.Archiver.Entities.GxUIProxyAdapterClass.Initialize(IGxUIProxyAdapterEvents
EventListener, IEntityBuilder EntityBuilder, String cacheFolder) at  Genetec.Archiver.OmnicastArchiver.Run()
--- End of inner exception stack  trace ---
at Genetec.Archiver.OmnicastArchiver.Run()

Procedure

Deactivate the Archiver role in Config Tool, as follows:

IMPORTANT: Perform this step at a non-crucial time, since all the units connected to the Archiver will temporarily go offline.
1. In the Config Tool Video task, select the Archiver role.
2. At the bottom of the Video task, click Deactivate role ().
3. In the confirmation dialog box that opens, click Continue.
  The Archiver and all video units controlled by the role become red.
In Windows on the Archiver Server, navigate to the following location: C:\Program Files (x86)\Genetec Security Center 5.x\Archiver\.
Rename the Cache folder to keep it as a backup for analysis purposes, or if you need to revert to the old settings.
Cache.OLD.

IMPORTANT: If the Archiver role is configured for failover, you must rename the Cache folder on all standby servers.
Reactivate the Archiver role in Config Tool, as follows:
1. In the Config Tool Video task, select the Archiver role.
2. At the bottom of the Video task, click Activate role ().

Results

Once the Archiver role is online again, a new Cache folder is created in C:\Program Files (x86)\Genetec Security Center 5.x\Archiver\.

Video units offline in Security Center

When a camera is red in the area view, it means the video unit is offline or has lost communication with the Archiver. To troubleshoot the issue, learn about the possible causes and their respective solutions.

What you should know

When a unit goes offline in Security Center, it typically coincides with a Unit lost event in Security Desk. This can be caused by an unstable network connection, or issues with the unit.

Procedure

Verify that you can ping the unit:
1. In the Config Tool Video task, select the red video unit.
2. At the bottom of the Video task, click Unit > Ping ().
If there is no reply, the unit is offline (broken, unplugged, and so on), or there is a problem with your network.
Make sure you can connect to the unit, and then click Unit > Unit’s web page.

Tip: You can also determine if you have the correct credentials for the unit.
Restart the unit:
1. In the Config Tool Video task, select the red video unit.
2. At the bottom of the Video task, click Unit > Reboot ().
Verify that the unit is supported by Security Center, and that it is running the certified firmware.
For a list of video units supported by Security Center, see our Supported Device List.
Restart the Archiver role controlling the unit:

IMPORTANT: Perform this step at a non-crucial time, because all the units connected to the Archiver will go offline temporarily.
1. In the Config Tool Video task, select the Archiver role.
2. At the bottom of the Video task, click Maintenance > Deactivate role ().
3. In the confirmation dialog box, click Continue.
  The Archiver role and all video units controlled by the role turn red.
4. At the bottom of the Video task, click Maintenance > Activate role ().

After you finish

If the video unit is still offline, contact Genetec™ Technical Assistance Center.

Cannot watch live video in Security Desk

If you cannot view live video in Security Desk, you can troubleshoot the issue.

What you should know

There are several possible causes for a missing video error:

The network is slow.
Port connection has issues.
The video stream was dropped while it was being redirected to Security Desk.

Procedure

Wait to see if the camera connects.
If the problem persists for more than 10 seconds, click Show diagnosis in the tile, or press Ctrl+Shift+D.
Information about the video stream is displayed. The current step is highlighted:

Initializing

The media player is preparing the required resources to display the video stream.

Connecting to Media Router

The media player is establishing connection with the Media Router to obtain the network location of the stream.

Connecting to Archiver and redirector

The media player is establishing connection with the Archiver and the Redirector to request video.

Requesting live stream

The connection is established between the Archiver and the Media Player. The Media Player is now requesting the live stream.

Analyzing the stream

The stream was requested and received by the client workstation. The media player analyzes the stream to detect the video format and the presence of key frames. After the stream is validated, the video is decoded.

Tip: Click the Help link for a list of things you can do to troubleshoot the issue.
Confirm that the unit is online.
If the unit is red in the Video task in Config Tool, then troubleshoot why the video unit is offline.
Verify that you can ping the unit:
1. In the Config Tool Video task, select the red video unit.
2. At the bottom of the Video task, click Unit > Ping ().
If there is no reply, the unit is offline (broken, unplugged, and so on), or there is a problem with your network.
Make sure you can connect to the unit, and then click Unit > Unit’s web page.

Tip: You can also determine if you have the correct credentials for the unit.
Verify that the unit is supported by Security Center, and that it is running the certified firmware.
For a list of video units supported by Security Center, see our Supported Device List.
Change the video unit’s connection type to the Archiver:
1. In the Config Tool Video task, select the red camera.
2. Click the Video tab.
3. From the Connection type drop-down list in the Network settings section, select a different connection type.
4. Click Apply.
Try viewing playback video from the camera:
1. In the Security Desk Archives task, select the camera.
2. Select the most recent video archive available and click Generate report.
3. After the report is generated, try to view the video from the archive.
  - If you can view the video, continue with the next troubleshooting step.
  - If you cannot view any video, contact Genetec™ Technical Assistance Center.
If you have an expansion server on your system running the Archiver role, try to view video from the expansion server:
1. Open Security Desk on the expansion server.
2. In the Monitoring task, drag the camera from the area view to a tile in the canvas.
  - If you can view video, it might be a problem with the redirection from the Media Router to your Security Desk. Continue with the next troubleshooting step.
  - If you cannot view any video, contact Genetec™ Technical Assistance Center.
Make sure the correct ports are open on your network so that there is no firewall blocking the video stream.
Verify that each network on your system is configured properly:
1. From the Config Tool home page, open the Network view task.
2. Select a network, click the Properties tab, and make sure all the settings are correct (IP prefix, subnet mask, routes, network capabilities, and so on).
3. If required, change the network settings and click Apply.
For more information, see About networks.
Force Security Desk to use a different connection type:
1. From the Security Desk home page, click Options > General.
2. In the Network options section, next to the Network option, select Specific.
3. From the drop-down list, select a different network and click Save.
4. Restart Security Desk.
5. If changing the network connection does not work, repeat the steps to test using other networks.
If you still cannot view video, click Show video stream status in the tile, and then troubleshoot the video stream.
If the issue persists, contact Genetec™ Technical Assistance Center.

Cannot watch playback video in Security Desk

If you cannot view playback video or video archives in Security Desk, you can troubleshoot the issue.

Procedure

Try viewing live video from the same camera by dragging the camera from the area view to a tile in the canvas in the Security Desk Monitoring task.
- If you can view live video, continue with the next troubleshooting step.
- If you cannot view any video, then it is probably a network issue. See Video units offline in Security Center.
Try viewing playback video from the Archives task:
1. In the Security Desk Archives task, select your camera.
2. Search for video archives at different dates and times and click Generate report.
3. After the report is generated, try to view video from the archives.
4. Repeat the steps with other cameras that are connected to the same Archiver.
- If you can view the video from some of the video archives, continue with the next troubleshooting step.
- If you cannot view any video, skip the next troubleshooting step.
Verify that the unit is supported by Security Center, and that it is running the certified firmware.
For a list of video units supported by Security Center, see our Supported Device List.
Try viewing playback video from the Archives task on another Security Desk, and on the server where the Archiver role is running.
- If you can view video, it might be a problem with the redirection from the Media Router to your Security Desk. Continue with the next troubleshooting step.
- If you cannot view any video, contact Genetec™ Technical Assistance.
Make sure the correct ports are open on your network so that there is no firewall blocking the video stream.
If you still cannot view playback video, contact Genetec™ Technical Assistance Center.

Changing settings for Archiver logs in Security Center

If the server hosting the Archiver role has low disk space, you can change the retention period and save location of the Archiver logs to free up disk space and prevent issues with the server performance.

What you should know

The Archiver.gconfig file has two sections: ArchiverRole and ArchiverAgent. The settings in the file apply to all the Archiver roles or Archiver agents that are hosted on the server. The Archiver log settings are modified in the ArchiverAgent section of the file.

CAUTION:

Only modify a .gconfig file if you are sure the changes are valid. Incorrect code in a .gconfig file can cause issues on your system or cause your system to go offline.

Procedure

On the Archiver server, do one of the following:
- Back up the Archiver.gconfig file by copying it to another folder.
  The file is located by default in C:\Program Files (x86)\Genetec Security Center 5.x\ConfigurationFiles on a 64-bit computer and in C:\Program Files\Genetec Security Center 5.x\ConfigurationFiles on a 32-bit computer.
- If the Archiver.gconfig file does not exist, generate it as follows:
  1. Log on to Server Admin and select the server hosting the Archiver role.
  2. Click Actions > Console > Commands > Archiver Agent commands > GenerateConfigFile.
Open the Archiver.gconfig file and change the following options in the ArchiverAgent section:
- Change the ArchiverLogPath value from "C:\ArchiverLogs\" to the desired file path
```
ArchiverLogPath="D:\ArchiverLogs\"
```
- Change the logDaysToKeep value from 90 days to the desired retention period.
```
logDaysToKeep="60"
```
NOTE: If the Archiver.gconfig file already existed on the computer but the options are not listed, it is due to one of the following reasons:
- You have an outdated version of the file. Generate the file using step 1.
- The file was previously generated for the Archiver role (Actions > Console > Commands > Archiver Role commands > GenerateConfigFile), and you are missing the ArchiverAgent section of the file. Generate the file using step 1.
- If you are using Archiver failover, the Archiver role might be running on a different server. Generate the file on the other server and copy it to this server.
Save the file and restart the Archiver role.