Skip to main content
Version: 2.4.3

Network requirements

Configure your network to allow connectivity between your source and target environments, and to access the UI.

There are three components that use network ports for TCP connections: Data Migrator, Hive Migrator, and the UI.

Data Migrator port requirements

Ensure the following ports are open on your on-premises network:

PortDescriptionDirection
18080Data Migrator REST APIInbound
443Data transfer (HTTPS)Outbound

Additionally ensure the following port is open on the NameNode of any target filesystems you will use with Data Migrator:

PortDescriptionDirection
8020Namenode port (fs.defaultFS)Inbound

Enable one of these ports on all data nodes of any target filesystems you will use with Data Migrator:

PortDescriptionDirection
9866DFS datanode addressInbound
1004Secure DFS datanode addressInbound
note

The port numbers 9866 and 1004 are default values and you should check the values in dfs.datanode.address and dfs.datanode.https.address properties for your cluster.

Open the following port between the host running Data Migrator and all hosts running data transfer agents:

PortDescriptionDirection
1433Data transfer agentInbound

For more information (such as port changes), see General configuration for Data Migrator.

Refer to the following diagram when configuring ports for Data Migrator and Hive Migrator:

Data Migrator and Hive Migrator ports

Enable one of these ports depending on whether your connection uses HTTP or HTTPS:

PortDescriptionDirection
6780Hive Migrator REST API (HTTP)Inbound
6781Hive Migrator REST API (HTTPS)Inbound

The following port must be open on the target server where the Hive Migrator remote server is installed:

PortDescriptionDirection
5052Hive Migrator Remote AgentInbound
note

You may view the remote agent port in the hdfs-site.xml file on your cluster.

Hive Migrator remote agents port requirements

The port used to communicate with a remote agent is defined in the remote agent's configuration. This port will be open for inbound connections on the remote agent side and Hive Migrator will connect to it as a client.

UI port requirements

Ensure the following ports are open on your platform to allow bi-directional communication for the UI.

PortDescriptionDirection
8081UIInbound
443Data transfer (HTTPS)Outbound

For more information (such as port changes), see General configuration for the UI.

Architecture diagram

This diagram gives you a simplified view of how Data Migrator and Hive Migrator fit together.

Data Migrator Architecture

  • Command Line Interface (CLI) allows for terminal-based control of Data Migrator and Hive Migrator. Learn more about CLI.

  • Remote agents are optional - access a metastore directly or through a remote agent. Learn more about remote agents.

  • Data transfer agents are optional - for scaling data transfer from the source data. For small throughputs, transfer data directly using Data Migrator. Learn more about data transfer agents.

  • Registration and consumption reporting goes through the UI, which sends product usage data to the gateway service. Learn more about the Data Migrator Gateway

Troubleshooting

If you are having problems with networking, you can find solutions in the troubleshooting guide.

Next steps

Install Data Migrator after setting up your network.