Network requirements
Configure your network to allow connectivity between your source and target environments, and to access the UI.
There are three components that use network ports for TCP connections: Data Migrator, Hive Migrator, and the UI.
Data Migrator port requirements
Ensure the following ports are open on your on-premises network:
| Port | Description | Direction |
|---|---|---|
| 18080 | Data Migrator REST API | Inbound |
| 443 | Data transfer (HTTPS) | Outbound |
Additionally ensure the following port is open on the NameNode of any target filesystems you will use with Data Migrator:
| Port | Description | Direction |
|---|---|---|
| 8020 | Namenode port (fs.defaultFS) | Inbound |
Enable one of these ports on all data nodes of any target filesystems you will use with Data Migrator:
| Port | Description | Direction |
|---|---|---|
| 9866 | DFS datanode address | Inbound |
| 1004 | Secure DFS datanode address | Inbound |
The port numbers 9866 and 1004 are default values and you should check the values in dfs.datanode.address and dfs.datanode.https.address properties for your cluster.
Open the following port between the host running Data Migrator and all hosts running data transfer agents:
| Port | Description | Direction |
|---|---|---|
| 1433 | Data transfer agent | Inbound |
For more information (such as port changes), see General configuration for Data Migrator.
Refer to the following diagram when configuring ports for Data Migrator and Hive Migrator:
Enable one of these ports depending on whether your connection uses HTTP or HTTPS:
| Port | Description | Direction |
|---|---|---|
| 6780 | Hive Migrator REST API (HTTP) | Inbound |
| 6781 | Hive Migrator REST API (HTTPS) | Inbound |
The following port must be open on the target server where the Hive Migrator remote server is installed:
| Port | Description | Direction |
|---|---|---|
| 5052 | Hive Migrator Remote Agent | Inbound |
You may view the remote agent port in the hdfs-site.xml file on your cluster.
Hive Migrator remote agents port requirements
The port used to communicate with a remote agent is defined in the remote agent's configuration. This port will be open for inbound connections on the remote agent side and Hive Migrator will connect to it as a client.
UI port requirements
Ensure the following ports are open on your platform to allow bi-directional communication for the UI.
| Port | Description | Direction |
|---|---|---|
| 8081 | UI | Inbound |
| 443 | Data transfer (HTTPS) | Outbound |
For more information (such as port changes), see General configuration for the UI.
Architecture diagram
This diagram gives you a simplified view of how Data Migrator and Hive Migrator fit together.
Command Line Interface (CLI) allows for terminal-based control of Data Migrator and Hive Migrator. Learn more about CLI.
Remote agents are optional - access a metastore directly or through a remote agent. Learn more about remote agents.
Data transfer agents are optional - for scaling data transfer from the source data. For small throughputs, transfer data directly using Data Migrator. Learn more about data transfer agents.
Registration and consumption reporting goes through the UI, which sends product usage data to the gateway service. Learn more about the Data Migrator Gateway
Troubleshooting
If you are having problems with networking, you can find solutions in the troubleshooting guide.
Next steps
Install Data Migrator after setting up your network.