Define metadata rules

When you've connected your metastores, define metadata rules to determine the scope of metadata you want to migrate. Use these metadata rules when creating a metadata migration.

info

Ensure migrations exist for the data (databases and tables) you want to migrate, this is not optional for transactional tables which will not be populated on the target unless the migration for the data exists first.

You need both the data and associated metadata before you can successfully run queries on migrated databases.

note

Hivemigrator automatically applies default exclusions for any metadata rule. See Default metadata exclusions to learn more.

Define metadata rules with the UI

Create a metadata rule

Define the scope of a metadata migration with metadata rules.

To create a metadata rule:

1. From the Dashboard, select an instance under Instances.
2. In the Migrations menu, select Metadata Rules.
3. Select Add new rule.
4. Enter a name for the metadata rule.
5. Select a Rule Type. See Rule Types for more information on available types.
   • Name Patterns: Supply Database Pattern and Table Pattern as Hive DDL patterns that match the database and table names.
6. Select Create to add the metadata rule.

View metadata rules

To view a list of existing metadata rules:

1. From the Dashboard, select an instance under Instances.
2. In the Migrations menu, select Metadata Rules.
3. View all rules under Metadata Rules or select a rule to view its properties.

Delete metadata rules

To delete a metadata rule not currently associated with a metadata migration:

1. From the Dashboard, select an instance under Instances.
2. In the Migrations menu, select Metadata Rules.
3. Under Metadata Rules, select the metadata rule.
4. Select Delete to remove the rule.

Edit metadata rules

To edit an existing metadata rule not currently associated with a metadata migration:

1. From the Dashboard, select an instance under Instances.
2. In the Migrations menu, select Metadata Rules.
3. Under Metadata Rules, select the metadata rule.
4. Select the field and adjust the value as needed.
5. Select Save to apply the changes to the rule.

note

Editing or deleting a rule is disabled when it's currently associated with a metadata migration. The Rule Type cannot be changed.

Rule Types

Type	Description
Name Patterns	Supply Database Pattern and Table Pattern as Hive DDL patterns that match the database and table names.

Define metadata rules with the CLI

Add metadata rules

Define which databases and tables you want to migrate by creating metadata rules.

Create a database pattern and a table pattern.

note

Wildcards support
When creating patterns for your migration, note the following:

• If using Hive 1, 2, or 3: Only use patterns with the wildcards * and |.

  For example, using --database-pattern test* will match any database with "test" at the beginning of its name, such as test01, test02, test03.

• If using Hive 4: Use any wildcards based on Hive's Data Definition Language (DDL).

Important See the following Known issue article showing important changes in interpretation of the underscore character in different versions of Data Migrator metadata rule patterns.

Command	Action
hive rule add, hive rule create	Create a metadata rule that can be used for a migration

Configure existing metadata rules

Command	Action
hive rule configure	Configure an existing metadata rule

Manage metadata rules

Command	Action
hive rule delete	Delete a metadata rule
hive rule list	List all metadata rules
hive rule show	Show the configuration of a metadata rule

Next steps

Once you have defined the metadata rules, create a metadata migration.