Skip to main content
Version: 2.6

Create a recurring migration

Recurring migrations let you continuously scan for changing data if your source doesn't support or isn’t configured to support notification-based live changes. Existing data on the source will be moved to the target, then the migration scan will be repeated to discover new changes.

As recurring migrations repeatedly rescan, so they don't necessitate additional rescanning. Therefore, a Rescan is not permitted for recurring migrations.

Data Migrator supports recurring migrations for the following filesystems:

Find all migration types and target file systems supported here.

tip

Dangling files.

Recurring migrations repeatedly scan source and copy to target, files subsequently deleted from source will therefore remain present (dangling) at target. See Target Match for more options when dealing with dangling files.

Create a recurring migration with the UI​

To create a recurring migration with the UI, select Recurring Migration under Migration Type when you create a migration.

Set an optional Rescan Limit by selecting the minimum number of minutes or hours from the start of the first scan to the start of the next scan.

tip

When using exclusions with a recurring migration, the count of number of files excluded reported in the migration status show a cumulative total for each recurrence of the migration.

Create a recurring migration with the CLI​

Create a recurring migration with the CLI by using the migration add command with the --recurring-migration, --recurring-period, and --scan-only parameters.

  • --recurring-migration Enter this parameter to create a recurring migration.
  • --recurring-period Define the minimum number of minutes or hours from the start of the first scan to the start of the next scan. For example, 4H (four hours) or 10M (ten minutes). If not specified, the default is 0, and the scan will immediately recur after the last scan completes.
  • --scan-only Enter this parameter to create a recurring migration.

Example

Create a recurring migration with a rescan limit of 12 hours
migration add --path /repl1 --target mytarget –-migration-id myNewMigration --recurring-migration --recurring-period 12H --scan-only

Rescan limit​

Recurring migrations scan continuously by default, which can cause performance degradation on the source.

Set a rescan limit to reduce risks to performance by selecting the minimum number of minutes or hours that must pass from the start of the first scan to the start of the next scan.

Only one scan can run at once for a migration.

Examples

  • If you set a rescan limit of 12 hours and your scan takes 1 hour, the next one won't start for another 11 hours.
  • If you set a rescan limit of 24 hours and your scan takes 25 hours, the next one will start immediately.

If a limit isn't specified, it defaults to 0, and the scan will immediately recur after the last scan completes.

note

You can amend the rescan limit at any time for an existing migration.

Check the current rescan limit for a migration by selecting the migration from your Migrations panel, then selecting the Recurring Scan tab.

info

Contact Support for guidance on adjusting this setting.

Recurring scan metrics​

Go to the Scan Progress panel of the Recurring Scan tab of a migration to see metrics on recurring scans:

  • The number of scans in the last 24 hours
  • The number of scans in the last 7 days
  • The average time taken for all scans
  • The number of times the source has been scanned, including the initial scan

The Last 10 Scans table provides detailed metrics on the start timestamps and duration of the latest 10 scans.