Introduction:
TNG is a delta tracking and synchronization option available for Windows hosts being migrated and being protected (using DR) with RMM by installing a delta tracker on the source. This option is typically used with servers that have a large amount of data on them that changes rapidly. Using this option can reduce the time it takes to do a delta sync. It does not reduce the time it takes to do an initial sync.
Enabling TNG on a wave item:
TNG can be enabled by checking the TNG option under the Sync Options tab while editing a Wave Item.
Checking TNG Status on a Host:
The status of the TNG delta tracker and daemon can be checked under the summary tab of the wave-item.
The RMM CLI command "rw host show" will show which hosts are running TNG and which are not
[root@rmm-02 opc]# rw host show
Friendly Name Host State TNG State
--------------------- ------------ ---------
host-small-src DISCOVERED NO_TNG
host-large-src DISCOVERED RUNNING
ZFS Storage with TNG
Any sync with the TNG option turned on requires storage in the RMM's ZFS pool. If Staged syncs (Stage 1 sync and Stage 2 sync) are being used, then a zfs pool should have already been set up when the RMM was installed. If Host Syncs/Direct Syncs are being used, it is possible the RMM was installed without a zfs pool being configured. In that case a zfs pool will need to be configured using the "rwadm zfs configure" command as described in Section 5.1.1 of the Product Operations (CLI) Guide.
The amount of ZFS space used by TNG is given by the formula:
(0.15 * Used) * (MaxParallel /TotalServ)
Where
Used = Total Used Space of the Production Servers that will be using the TNG option
MaxParallel = The maximum number of servers that will be running syncs using the TNG option at any given time
TotalServ = The total number of servers using the TNG option being synced by this RMM
If MaxParallel is unknown at this time, then assume that MaxParallel = TotalServ for the purpose of this calculation.
TNG Logging:
The TNG delta tracker produces logs to track the changes occurring on the filesystem.
Logs are stored by default at the following location on a Windows host:
C:\windows\RackwareDb\<volume GUID>\*.log
* Logs for each volume are stored inside individual GUID titled folders under RackwareDb.
<volume GUID> is the mountvol output with no \Volume prefix.
Figure: Sample output for mountvol command
Understanding the TNG option:
The TNG synchronization option is made up of two parts:
1. TNG Service / Daemon
2. File System delta tracker
To check if the TNG service is running, execute the "sc query rwchangesvc" command on the Windows host. If the output shows that the STATE of the service is RUNNING, then the TNG service is running..
The filesystem delta tracker is responsible for tracking all changes to the filesystems and logging those changes to the log files. These log files are merged into the rwfstrack.db (a SQLite database locally present on the source) for each volume. The rwfstrack.db file is located in the same directory as the log files (C:\Windows\RackwareDb\<volume guid> by default).
Trimming the TNG database files
As changes occur over time, the rwfstrack.db files will continue to grow until the next TNG sync is performed. To prevent the files from taking up "too much" space on the drive they reside (which is the C drive, by default), the TNG service will "trim" the rwfstrack.db files if they become too large. When the database files are trimmed their contents will be lost and the next delta sync will be a full sync, and thus will take much longer to complete.
The definition of "too large" is that the sum of the space taken up by all of the rwfstrack.db files and all of the log files exceeds
MIN ((0.1 * Total_space), (0.5 * Free_space))
where "Total space" is the total space on the drive on which the RackwareDb directory resides, and "Free space" is the amount of free space on the drive on which the RackwareDb directory resides.
When the TNG service needs to trim the database files to prevent them from becoming too large, an event stating "TNG Log limit reached" will be logged by the Rackware Change Tracking Service in the Windows Application Event Log
When trimming happens, the next sync will be a full sync for all drives on the host. If trimming happens frequently, the benefits of TNG (i.e. shorter sync times) will not occur.
There are 3 ways to prevent trimming from happening:
1) Add more space to the disk. This will allow the files in the RackwareDb subdirectories to grow more before they need to be trimmed
2) Sync more frequently. If TNG syncs are done more frequently, the database files will have less time to grow, and thus it is less likely they will need to be trimmed.
3) Change the location of the RackwareDb files. The RackwareDb files are on the C drive by default, and typically the C drive is smaller and has less free space than other drives. Move the RackwareDb files to an existing drive, or add a new drive that is just used to hold the RackwareDb files. This new drive can be excluded from any syncs being performed.
https://rackware.freshdesk.com/a/solutions/articles/5000870974 explains how to move the RackwareDb from the C drive to another drive.
TNG Terminology:
First TNG Sync: This is the first sync that has the TNG option set; this sync this will install the TNG delta tracker and TNG service/daemon on the source the first time and run a Full Sync job.
TNG Sync: After the First TNG Sync, subsequent delta syncs are TNG syncs, which use the delta tracker logs and rwfstrack.db to get the changes from the source, pack them into a tgz file on RMM and then apply them on the image (for staged syncs) or the target host (for host syncs). A TNG sync will take less time to complete than a Full Sync.
Full Sync: A Full Sync is a standard rsync sync, which does not use any of the TNG enhancements. If the TNG option is not set on the sync, a full sync will be performed. If the source/origin host is rebooted, the first sync after the reboot will be a full sync on all the drives of the server. If the TNG database files need to be trimmed, the next sync will be a full sync of all the drives of the server.
Note - there are certain error conditions that could cause a full sync to be performed on one drive while while a TNG sync is performed on the other drives.
No-Reboot Syncs:
The TNG option is applicable for Stage 1 syncs and host/direct syncs. Stage 2 syncs are never performed with the TNG option, even if the TNG option is set on the GUI.
A host/direct sync with the TNG option will keep the target OS in the RackWare microkernel, even if the "no reboot" option is not selected. This allows RMM to eliminate steps that prepare the target to boot into Windows. In order to boot the target into OS, please issue the rw host release <target_friendlyname> command. This will cause the Prepare steps to be run and will then boot the target into Windows.