Manage migrations

Manage your migrations with Data Migrator using either the UI or CLI.

caution

Don't write to target filesystem paths when a migration is in progress. This could interfere with Data Migrator functionality and lead to undetermined behavior.

Use different filesystem paths when writing to the target filesystem directly (and not through Data Migrator).

Manage migrations with the UI

On the Dashboard, the Migrations panel displays a list of migrations with their priority and status.

You can filter the list by priority or by status.

You can also search for migrations by migration name and path name.

note

If you enter a string that is part of either a migration name and a path name, or both, the search returns all migrations that contain the string.

Select the ellipsis at the end of the migration listed to go to:

• Settings to change the priority of a migration or Assign management group.
• Check status to stop, resume, or delete a migration, for example, or view details on errors and retries.
• Verify migration to check that the source and target filesystems match.

Select the migration to go directly to the migration's Status

Status

Select a migration from the Dashboard, Migrations panel to view its State, to stop, resume, or delete a migration and view summary information.

Data migration states

Data migrations can be in one of the following states:

State	Description
NONSCHEDULED	A migration in a Non-scheduled or Ready state is defined but not started. Create a migration in this state by not entering the --auto-start parameter on creation.
SCHEDULED	A migration in a Scheduled state is defined and configured to start, but is waiting to start due to the running migration limit.
RUNNING	A migration in a Running or In Progress state is scanning through source data and transferring data to the target, as well as responding to change notifications from the source if applicable. Note: Prioritized migrations without a running migration limit may appear in a Running or In Progress state before Data Migrator starts migrating data. When it starts migrating data, you’ll see progress on the Migration Status bar.
LIVE	A migration in a Live state has scanned through source data, continues to respond to change notifications from the source, and makes changes to the target as required.
STOPPING	A migration in a Stopping state is waiting for all the active action threads to complete before transitioning to the STOPPED state
STOPPED	A migration in a Stopped state has been stopped manually by a user or because of some other application exception.
COMPLETED	A non-live migration in a Completed state has scanned through all source data, finished transferring applicable data to the target, and doesn't migrate further changes to source data. See One time migrations.
RECURRENCE_SCHEDULED	A recurring migration enters the Recurrence scheduled state once it is scheduled to run. See Recurring migration.
WAITING_FOR_NEXT_RECURRENCE	A recurring migration in a Waiting for next recurrence state has finished its scan and remains in this state until the next scan is initiated. See Recurring migration.

Start a migration

To start a migration and initiate data transfer, select Start.

Stop a migration

Once a migration has started, you can select Stop to suspend transferring files indefinitely.

Resume a migration

Once a migration is stopped, you can select Resume to continue where it paused transferring files.

info

See Stopping and resuming Live data migrations for information on limits for resuming stopped Live migrations.

Delete a migration

note

You must stop a migration before you can delete it.

To delete a migration:

1. Select the migration from your Migrations panel.
2. On the Migration Status page, select Stop.
3. Select Delete.

Reset a migration

Reset a migration to set it to the state it was in before it started. This deletes the migration and creates a new migration in its place with the same settings as the previous.

When you stop a migration, a Reset button becomes available next to the Resume button. Select it to open the Reset Migration panel. Configure the following:

note

You can only reset stopped or failed live or one-time migrations. You can't reset recurring migrations or migrations in a READY, COMPLETED, or IN PROGRESS state.

Migrations must be in a STOPPED or FAILED state to appear in the list of migrations available to reset. See Bulk actions.

• Target Match: Select Enable Target Match to enable Target Match on this migration.
• Reset Path Mappings: Select this box to erase the migration's path mapping configuration and automatically use your most recent path mapping configuration for Data Migrator. Clear it to continue using the same path mapping configuration from before the reset.
• Skip or Overwrite Settings: See Skip or Overwrite Settings.

Select Reset to confirm your selection and reset the migration.

info

You'll still be able to view migration errors in the Migration Errors Report after a Reset.

Data summary

• Total Amount of Data Migrated: The total number of files and bytes migrated for this migration.
• Pre-existing Data Discovered: The number of files and bytes initially discovered on the file system for this migration.
• Pre-existing Data Migrated: For live migrations, the number of files and bytes initially discovered on the file system that have been migrated by this migration.
• Data Added Since Migration Started: For live migrations, the number of files and bytes added to this migration since it started.
• Total paths removed by Target Match: For migrations with Target Match enabled. This shows the number of removal actions taken.
  • files: Number of files removed from the base migration path. (Not a total count of files removed contained in subdirectories.)
  • directories: Number of directories removed from the replication path. (Directories may contain numerous files.)
• Files Excluded Since Migration Started: The number of files and bytes excluded by this migration.

note

The number of files shown in 'Total paths removed by Target Match' in the UI and the value of filesRemovedTargetMatchScan from the migration stats CLI command show the number of files removed from the base migration path and not a total number of files removed contained in subfolders. See the following Knowledge base article to learn more.

Settings

Migration Priority

See Prioritize migrations to configure migration processing and prioritization.

Assign management group

1. In the Dashboard, select a migration.
2. Select Settings.
3. Under Management Group, select a management group.
4. Select Submit to apply the change.

See Migration Management delegation for more information on Management Groups and delegation of migration management.

Migration errors

Check the status of a live migration by selecting the migration from the Dashboard, under Migration Errors you'll find the number of Operations Failed for this migration.

Select Migration Errors from the Migration menu to view the Migration Errors Summary

The Migration Errors Summary panel provides a Migration Errors Report that lists failed migration operations. The report table lists each failed operation with columns for the Timestamp, Operation type, and Path and Failure, which provides the path to the failed element, along with a description of the failure.

View the current status and results of any verification scans configured to check that all the source data and changes are available on the target and are consistent between the source and target. For more information, see Verify migrations.

Filter and search for failed operations

You can filter or search for failed operations by using the Operation type dropdown on the Migration Errors Report panel. The filter contains the following operations:

Operation	Description
Create	Create operations set up a new data migration. For more information, see Create a migration.
Migration (Transfer file)	Migration operations handle the transfer of data to be migrated.
Unlink (Delete path)	Unlink operations remove files under migration.
Rescan (retry)	Rescan operations restart an earlier source filesystem scan.
Rename	Rename operation handles name changes in files or directories under replication.
Metadata (Change permissions)	Metadata operations change file metadata, such as file ownership.
Marker (Internal to Data Migrator)	Marker operations are part of Data Migrator's internal migration mechanics. Marker file operation failures may be evidence of a serious problem with your Data Migrator deployment. Contact Support with details of the errors.

Advanced search

Select Advanced search ^ to display Filter Rules for additional search options. Use the following entry fields to create an advanced search rule:

• Field: Select Path to search all available migration paths or Failure to search only failed migration paths.

• Operator: Select Contains or Does not contain to apply to your search.

• Enter Value: Enter the value you want your search to match, according to your chosen operator. If you select the "Does not contain" operator, your search will match with paths or failure paths that don't include the value.

Select + Add Rule to add additional search criteria. Select Apply to view the search results. You can continue to add rules to your current search, but you must select Apply to update the search results.

note

Historical failure information may be garbage collected, so it may not be possible to show all failures. In this case, a warning appears:

"We're unable to show detailed results of failed operations. We only have detailed results from the most recent [number of available failures] results"

Migration verification

View the Verification Status and Total Inconsistencies recorded for any migration verification operations on this migration. See migration verification to learn more about the migration verification feature.

Migration metrics

Use metrics to view details about your migrations to better understand progress, including their bandwidth usage, migration events, and remaining scanned paths.

1. In the Dashboard, select a migration.
2. You will be taken to the Status of the migration tab by default.

Under Migration Metrics, observe the following:

• Events yet to be processed: this graph displays the size of the event manager queue across a time period selected in the Filter by dropdown list.
• Events yet to be migrated: this graph shows the size of the scheduled actions queue across a time period selected in the Filter by dropdown list.
• Paths to be scanned: this graph displays the size of the queue for paths yet to be scanned across a time period selected in the Filter by dropdown list.

Assign exclusions to an existing migration

Adding exclusions to an existing migration will change the future actions performed for that migration, but will not affect previously migrated content.

1. In the Dashboard, select a migration.
2. Under the sidebar that appears, select Exclusions.
3. Select Add new exclusion and the appropriate exclusion template from the dropdown list.

Remove exclusions from an existing migration

1. In the Dashboard, select a migration.
2. Under the sidebar that appears, select Exclusions.
3. Remove any of the exclusions one at a time.

Rescan a migration

Data Migrator tracks updates on directory paths on the source filesystem. However, if a subdirectory path requires rescanning due to missed or time-delayed updates, perform a Rescan on a path with the UI or use the migration pending-region add CLI command. The entire migration can be scanned again by making the root of the migration the path to be rescanned.

info

Rescan can only be performed with Live or One-time Migrations. Recurring migrations repeatedly rescan, so they don't necessitate additional rescanning. Therefore, a Rescan is not permitted for recurring migrations.

Add paths for rescanning in the UI

To add a path for rescanning in the UI:

• Select the migration you want to add a path for rescanning to in the Dashboard.
• Under the sidebar that appears, select Rescan.
• Enter the path to the directory in Add rescan directories.
• Select which action policy to use (Overwrite or Skip If Size Match).
• Select +Add.

Add paths for rescanning in the CLI

To add a path for rescanning to a migration, run the command migration pending-region add.

Example

migration pending-region add --migration-id myFirstMigration --path /dir1/userA --action-policy com.wandisco.livemigrator2.migration.SkipIfSizeMatchActionPolicy

Manage migrations with the CLI

Command	Action
migration stop	Stop a migration
migration stop all	Stop all migrations
migration reset	Reset a migration
migration resume	Resume a stopped migration
migration delete	Delete a migration
migration exclusion add	Add an exclusion to a migration
migration exclusion delete	Remove an exclusion from a migration
migration list	List running and active migrations
migration start	Start a migration
migration show	Get migration details
migration pending-region add	Add a path for rescanning to a migration
status	Get migration status