LiveData uses a set of Paxos-based algorithms to continue to replicate even after brief networks outages, data changes will automatically catch up once connectivity between clusters is restored.

Below the coordination stream, actual data transfer is done as an asynchronous background process and doesn’t consume MapReduce resources.

WANdisco Fusion supports Selective replication, where you control which data is replicated to particular clusters, based on your security or data management policies. Data can be replicated globally if data is available to every cluster or just one cluster.

Replication networks are composed of a number of nodes, each node takes on one of a number of roles:

WANdisco Fusion is able to maintain HDFS replication even after the loss of WANdisco Fusion nodes from a cluster. However, there are some configuration rules that are worth considering:

This is related and why you will see coordinated operation’s written "out-of-order" in the filesystem. Internally within Fusion, "non-interfering" agreements are processed in parallel so that throughput is not hindered by operations that take a long time, such as a large file copy.

Consider the following global sequence, where /repl1 is the replicated directory:

1. Copy 10TB file to /repl1/dir1/file1
2. Copy 10TB file to /repl1/dir2/file1
3. Chown /repl/dir1

DConE’s Output Proposal Sequence (OPS) delivers agreed values in strict sequence, one-at-a-time, to an application. Applying these values to the application state in the sequence delivered by the OPS ensures the state is consistent with other replicas at that point in the sequence. However, an optimization can be made: if two or more values do not interfere with one another they may be applied in parallel without adverse effects. This parallelization has several benefits, for example:

The Fusion user interface provides an LDAP/AD connection, allowing Fusion users to be managed through a suitable Authorization Authority, such as an LDAP, Active Directory or Cloudera Manager-based system. Read more about connecting to LDAP/Active Directory.

Users can have their access to Fusion fine-tuned using assigned roles. Each Fusion user can be assigned one or more roles through the organization’s authorization authority. When the user logs into Fusion, their account’s associated roles are checked and their role with the highest priority is applied to their access.

Roles can be mapped to Fusion’s complete set of functions and features, so that user access can be as complete or as limited as your organization’s guidelines dictate. A set of roles are provided by default, you read about these in the Roles and Permissions, you can, instead, create your own user roles and limit permissions to specific Fusion functions.

The Fusion UI server uses a _User Authorization abstraction which incorporates the following sub-systems:

This component is responsible for mapping the authenticated user to one or more roles based on their presence in the "Authorization Authority".

Calls to Fusion’s REST APIs are guarded by a filter which checks the client calls against the roles specified and decides whether the call is authorized. The filter uses a 2 stage check:

Client access to Fusion is chiefly driven by URI selection. The Hadoop URI consists of a scheme, authority, and path, with the scheme and authority, together, determining the FileSystem implementation. The default is hdfs and is referred to by fs.hdfs.impl, which points to the Java class that handles references to files under the hdfs:// prefix. This prefix is entirely arbitrary, and you could use any prefix that you want, providing that it points to an appropriate ".impl." that will handle the filesystem commands that you need.

The following requirements come into play if you have deployed WANdisco Fusion using with its native fusion:/// URI.

In order to store a Hive table in WANdisco Fusion you specify a WANdisco Fusion URI when creating a table. E.g. consider creating a table called log that will be stored in a replicated directory.

CREATE TABLE log(requestline string) stored as textfile location 'fusion:///repl1/hive/log';. Note: Replicating table storage without sharing the Hive metadata will create a logical discrepancy in the Hive catalog. For example, consider a case where a table is defined on one cluster and replicated on the HCFS to another cluster. A Hive user on the other cluster would need to define the table locally in order to make use of it.

It’s possible to configure Hive to use WANdisco Fusion URIs as output paths for storing data, to do this you must specify a Fusion URI when writing data back to the underlying Hadoop-compatible file system (HCFS). For example, consider writing data out from a table called log to a file stored in a replicated directory:

In this section we’ll describe how to configure Hive to use fusion URIs as input paths for loading data.

It is not common to load data into a Hive table from a file using the fusion URI. When loading data into Hive from files the core-site.xml setting fs.default.name must also be set to fusion, which may not be desirable. It is much more common to load data from a local file using the LOCAL keyword:

If you do wish to use a fusion URI as a load path, you must change the fs.defaultFS setting to use WANdisco Fusion, as noted in a previous section. Then you may run:

Advanced configuration - please contact WANdisco before attempting
In this section we’ll describe how to share the Hive metastore between two clusters. Since WANdisco Fusion can replicate the file system that contains the Hive data storage, sharing the metadata presents a single logical view of Hive to users on both clusters.

When sharing the Hive metastore, note that Hive users on all clusters will know about all tables. If a table is not actually replicated, Hive users on other clusters will experience errors if they try to access that table.

There are two options available.

In this configuration, the Hive metastore is configured normally on one cluster. On other clusters, the metastore process points to a read-only copy of the metastore database. MySQL can be used in master-slave replication mode to provide the metastore.

In this configuration, the Hive metastore is writable on all clusters.

Standard

For Cloudera CDH: See Hive Metastore High Availability.

For Hortonworks/Ambari: High Availability for Hive Metastore.

Manual Replication

In order to manually replicate metastore data ensure that the DDLs are placed on two clusters, and perform a partitions rescan.

If you plan to use WANdisco Fusion’s own fusion:/// URI, then you will need to use the provided parcel (see the screenshot, below for link in the Client Download section of the Settings screen):

Follow the same steps described for installing the WANdisco Fusion client, downloading the parcel and SHA file, i.e.:

In order for Impala to load the Fusion Client jars, the user needs to make a small configuration change in their Impala service, through Cloudera Manager. In Cloudera Manager, the user needs to add an environment variable in the section Impala Service Environment Advanced Configuration Snippet (Safety Valve).

The following command gives an example of how to do this.

Presto is an open source distributed SQL query engine for running interactive analytic queries. It can query and interact with multiple data sources, and can be extended with plugins.

Presto requires the use of Java 8 and has internal dependencies on Java library versions that may conflict with those of the Hadoop distribution with which it communicates when using the “hive-hadoop2” plugin.

WANdisco Fusion leverages a replacement client library when overriding the hdfs:// scheme for access to the cluster file system in order to coordinate that access among multiple clusters. This replacement library is provided in a collection of jar files in the /opt/wandisco/fusion/client/lib directory for a standard installation. These jar files need to be available to any process that accesses the file system using the com.wandisco.fs.client.FusionHdfs implementation of the Apache Hadoop FileSystem API.

Because Presto requires these classes to be available to the hive-hadoop2 plugin, they must reside in the plugin/hive-hadoop2 directory of the Presto installation.

It is also important to confirm that the Presto configuration includes the necessary properties to function correctly with the hive-hadoop2 plugin.

The specific values below will need to be adjusted for the actual environment, including references to the WANdisco replicated metastore, the HDP cluster configuration that includes Fusion configuration, and Kerberos-specific information to allow Presto to interoperate with a secured cluster.

Keytabs and principals will need to be configured correctly, and as the hive-hadoop2 Presto plugin uses YARN for operation, the /user/yarn directory must exist and be writable by the yarn user in all clusters in which Fusion operates.

Presto embeds Hadoop configuration defaults into the hive-hadoop2 plugin, including a core-default.xml file that specifies the following property entry:

Although Presto allows the hive-hadoop2 plugin to use additional configuration properties by adding entries like the following in a .properties file in the etc/catalog directory:

This entry allows extra configuration properties to be loaded from a standard Hadoop configuration file, but those entries cannot override settings that are embedded in the core-default.xml that ships with the Presto hive-hadoop2 plugin.

In a kerberized implementation the Fusion client library relies on the ability to read the hadoop.security.authentication configuration property to determine if it should perform a secure handshake with the Fusion server. Without that property defined, the client and server will fail to perform their security handshake, and Presto queries will not succeed.

The solution to this issue is to update the core-default.xml file contained in the hive-hadoop2 plugin:

Edit the core-default.xml file to update the hadoop.security.authentication property so that its value is “kerberos”

Distribute the hadoop-apache2-0.10.jar to all Presto nodes, and restart the Presto coordinator.

Something to be aware of in Hyper Scale-Out Platform (HSP) installations - when you install the client stack, the fusion-client RPM creates symlinks in /usr/hdp/current/oozie-server/libext for the client jars. However, these get left behind if the client stack/RPM are removed.

If a new version of fusion-client is installed, Oozie server will refuse to start because of the broken symlinks.

Oracle:BDA is built on top of Cloudera’s Hadoop and requires some extra steps to support multi-homed network environment.

In order to make Hive with Tez work, you need to append the Fusion jar files in tez.cluster.additional.classpath.prefix under the Advanced tez-site section:

e.g. WANdisco Fusion tree

Running Hortonworks Data Platform, the tez.lib.uris parameter defaults to /hdp/apps/${hdp.version}/tez/tez.tar.gz. So, to add Fusion libs, there are three options.

Option 1: Delete the tez.lib.uris path, e.g. "/hdp/apps/${hdp.version}/tez/tez.tar.gz". Instead, use a list including the path where the tez.tar.gz file will unpack, and the path where Fusion libs are located.

Option 2: Or unpack tez.tar.gz, repack with WANdisco Fusion libs and re-upload to HDFS.

Option 3: Alternatively, you may set the tez.lib.uris property with the path to the WANdisco Fusion client jar files, e.g.

You can read about the results of testing Hive2 with LLAP - Low Latency Analytical Processing, using Apache Slider to run Tez Application Masters on YARN. Inevitably, running a Tez query through this interface results in a FusionHDFS class not found.

The following steps show an example remedy, through the bundling of the client jars into the tez.lib.uris tar.gz.

Getting set up with the default URI is simple, Solr just needs to be able to find the fusion client jar files that contain the FusionHdfs class.

This is a minimal working solution with Solr on top of fusion.

If you want to set up Flume through Cloudera Manager follow these steps:

There is a known issue where Spark is not picking up hive-site.xml, So that Hadoop configuration is not localised when submitting job in yarn-cluster mode (Fixed in version Spark 1.4).

You need to manually add it in by either:

Spark applications are run on a cluster as independent sets of processes, coordinated by the SparkContext object in the driver program. To run on a cluster, the SparkContext can connect to several types of cluster managers (either Spark’s own standalone cluster manager, Mesos or YARN), which allocate resources across applications. Once connected, Spark acquires executors on nodes in the cluster, which are processes that run computations and store data for your application. Next, it sends your application code (defined by JAR or Python files passed to SparkContext) to the executors. Finally, SparkContext sends tasks to the executors to run.

WANdisco Fusion uses a replacement client library when overriding the hdfs:// scheme for access to the cluster file system in order to coordinate that access among multiple clusters. This replacement library is provided in a collection of jar files in the /opt/wandisco/fusion/client/lib directory for a standard installation. These jar files need to be available to any process that accesses the file system using the com.wandisco.fs.client.FusionHdfs implementation of the Apache Hadoop File System API.

Because Spark does not provide a configurable mechanism for making the Fusion classes available to the Spark history server, the Spark Executor or Spark Driver programs, WANdisco Fusion client library classes need to be made available in the existing Spark assembly jar that holds the classes used by these Spark components. This requires updating that assembly jar to incorporate the Fusion client library classes.

This is one of a number of methods that may be employed to provide Fusion-Spark integration. We hope to cover some alternate methods at a later date.

The following example covers how you may upgrade the Spark Assembly as part of a Fusion upgrade. This example uses CDH 5.11, although it can be applied generically:

The resulting spark-assembly is now purged and requires one of two actions:

If Spark 2 is installed after WANdisco Fusion you will need to manually symlink the WANdisco Fusion client libraries.

For HDP, create the 3 symlinks as follows:

Cloudera will automatically handle the creation of symlinks for managed clusters. However if you are using unmanaged clusters you will need to create the symlinks using the following command:

For HBase to run with WANdisco Fusion, the following directories need to be created and permissioned, as shown below:

The steps below provide a method of handling a recovery using a cold back-up. Note that multiple HMaster/region servers restarts might be needed for certain steps, since hbck command generally requires master to be up, which may require fixing filesystem-level inconsistencies first.

To configure Fusion for access to encrypted zones, two aspects need to be considered:

The following items need to be considered within KMS configuration to ensure Fusion has access:

The kms-site configuration (such as Advanced kms-site in Ambari) contains its own auth_to_local type parameter called “hadoop.kms.authentication.kerberos.name.rules”

Ensure that any auth_to_local mapping used for the Fusion principal is also contained here. This can be most easily achieved via simple copy/paste from core-site.xml.

The kms-site configuration (such as Custom kms-site in Ambari) contains proxyuser parameters such as:

Entries should be created for the local Fusion user (after auth_to_local translation) to allow Fusion to proxy/impersonate other users requests. This could be as simple as.

In the dbks-site configuration, the parameter hadoop.kms.blacklist.DECRYPT_EEK exists. Ensure this does not contain the username that Fusion uses (after auth_to_local translation).

In the KMS ACLs, such as using Ranger KMS, ensure that the Fusion user (after auth_to_local translation) has "Get Metadata" and "Decrypt EEK" permissions to keys.

This could be granted access to all keys. This will avoid a need to review rules when new keys are added. However, Fusion will only need these permissions to keys that apply to zones that fall within a replicated path. Consideration is needed here based on the user that Fusion has been configured as - either "HDFS" will need access to EEKs, OR the fusion user will need access, OR the supergroup could be given access to EEKs (it is enabled by default on Ambari but disabled on CDH), and then make the Fusion user a member of the supergroup.

If you do not perform the correct configuration, both local operations (as performed by a client) and/or the replicated actions may fail when the Fusion client is invoked. This should only apply to replicated paths.

So to troubleshoot:

Amazon Linux 2
CentOS 6 x86_64
CentOS 7 x86_64
Oracle Linux 6 x86_64
Oracle Linux 7 x86_64
Red Hat Enterprise Linux (RHEL) 6 x86_64
RHEL 7 x86_64
SUSE Linux Enterprise Server (SLES) 11 x86_64
SLES 12 x86_64
Ubuntu 14.04LTS
Ubuntu 16.04LTS

We only support AMD64/Intel64 64-Bit (x86_64) architecture.

We develop and test using the following browsers:

Other browsers and older versions may be used but bugs may be encountered.

Java JRE 1.8.
Testing and development are done using a minimum of Java JRE 1.8. There is also support for Open JDK 8, which is used in cloud deployments. For other types of deployment we recommend running with Oracle’s Java as it has undergone more testing.

Maximum User Processes and Open Files limits are low by default on some systems. It is possible to check their value with the ulimit or limit command:

-u The maximum number of processes available to a single user.
-n The maximum number of open file descriptors.

For optimal performance, we recommend both hard and soft limits values to be set to 64000 or more:

RHEL6 and later: A file /etc/security/limits.d/90-nproc.conf explicitly overrides the settings in security.conf, i.e.:

Ambari and Cloudera manager will set various ulimit entries, you must ensure hard and soft limits are set to 64000 or higher. Check with the ulimit or limit command. If the limit is exceeded the JVM will throw an error: java.lang.OutOfMemoryError: unable to create new native thread.

iptables
Use the following procedure to temporarily disable iptables, during installation:

RHEL 6

RHEL 7

Comment out requiretty in /etc/sudoers
The installer’s use of sudo won’t work with some linux distributions (CentOS where /etc/sudoer sets enables requiretty, where sudo can only be invoked from a logged in terminal session, not through cron or a bash script. When enabled the installer will fail with an error:

Workaround
Upgrade to Java 8u60 or greater, or ensure WANdisco Fusion is able to make use of OpenSSL libraries instead of JDK. Requirements for this can be found at http://netty.io/wiki/requirements-for-4.x.html

Transport Layer Security (TLS) and its predecessor, Secure Socket Layer (SSL) are widely adopted protocols that are used transfer of data between the client and the server through authentication and encryption and integrity.

Recent research has indicated that some of the cipher systems that are commonly used in these protocols do not offer the level of security that was previously thought.

In order to stop WANdisco Fusion from using the disavowed ciphers (DES, 3DES, and RC4), use the following procedure on each node where the Fusion service runs:

When your license limits are exceeded, WANdisco Fusion will operate in a limited manner, but allows you to apply a new license to bring the system back to full operation. Once a license is no longer valid:

Each different type of license has different limits.

Unless there’s a problem that stops you from reaching the WANdisco Fusion UI, the correct way to upgrade a node license is through the License panel, under the Settings tab.

The time required to complete a deployment of WANdisco Fusion will in part be based on its size, larger deployments with more nodes and more complex replication rules will take correspondingly more time to set up. Use the guide below to help you plan for deployments.

Of course, this is a guideline to help you plan your deployment. You should think ahead and determine if there are additional steps or requirements introduced by your organization’s specific needs.

If you are running Kerberos on your cluster you should consider the following requirements:

For information about running Fusion with Kerberos, read this guide’s chapter on Kerberos.

For example, if you connect a Zone that runs CDH, which has superuser 'hdfs' with a zone running MapR, which has superuser 'mapr', you would need to create the user 'hdfs' on the MapR zone and 'mapr' on the CDH zone.

See the Knowledge base for instructions on setting up manual Kerberos settings. You only need these in special cases as the steps have been handled by the installer. See Manual Updates for WANdisco Fusion UI Configuration.

Instructions on setting up auth-to-local permissions, mapping a Kerberos principal onto a local system user. See the Knowledge base article - Setting up Auth-to-local.

Use the following steps to complete the initial installation on the command line interface (CLI) using the Fusion installer file. The administrator will need to enter some configuration details when prompted, these will be outlined below. Once the initial settings are entered through the terminal session, the installation is then completed via a web browser.

Follow this section to complete the installation by configuring WANdisco Fusion via the user interface on a web browser.

To start this process, open a web browser and point it at the URL provided in the last step of the Initial installation via the terminal section.

Once WANdisco Fusion has been installed on all Clusters/Zones, you can log into the Fusion UI for each node.

The Fusion UI will highlight tasks that need to be completed prior to setting up replication between Zones. Amber warnings may also be displayed that recommend enabling additional features (such as Kerberos authentication for the Fusion Core Server). Example below:

The following steps are used to start replicating HDFS data. The detail of each step will depend on your cluster setup and your specific replication requirements, although the basic steps remain the same.

On MapR clusters, you need to copy WANdisco Fusion configuration onto all other nodes in the cluster:

When deploying a LocalFileSystem installation, there is an additional property that will need to be set after the installation is complete.

LocalFileSystem installations do not set this property at install and the outcome is that Consistency Checks return results that have different usernames / groups, but Fusion cannot resolve them. This property allows Fusion to ignore user/group and perform other checks such as object presence, type and size.

LocalFileSystem files do have usernames, but as Fusion runs as a single user, all replicated objects will be owned by this user and not individual users like in HDFS.

The following directories will be created on the Fusion nodes if they did not exist previously.

Service scripts for Fusion will also be created in the following locations:

If using the uninstall Fusion script (documented here), the service scripts will be removed. The purge flag can be used when running the uninstall script to automatically remove the directories listed (they will not be removed by default).

Directories created on the Ambari Server:

Directories created on nodes with an Ambari Agent installed (i.e. all nodes being managed by Ambari):

All nodes that have a Fusion Client installed will have directories and contents created within (if they didn’t exist previously):

Client distribution will update the following directory with symlinks to the Fusion Client libraries:

Deletion of the Fusion service and removal of the Fusion Client will also remove these directories and files, except for the /var/lib/ambari-server/resources/stacks/HDP/<hdp-version>/services/FUSION directory. This will have to be removed manually.

Files added on the Cloudera Management Server for Client distribution:

Directories added on the Cloudera Management Server during Client distribution:

All nodes that have a Fusion Client installed will have a directory and contents created within (if it didn’t exist previously):

Deactivation, removal and deletion of the Fusion parcel, and removal of the Fusion Client will remove these directories and files.

If Spark 1 service is installed in the cluster, then it is highly likely that the Spark Assembly Jar will have been updated to include the Fusion client libraries. This will need to be reverted as part of the uninstallation of Fusion.

It is worth performing this step before proceeding onto the removal of the WANdisco Fusion service, as cluster services will be restarted as part of those steps.

Please see the Spark 1 for Ambari or Spark 1 for Cloudera notes referenced earlier in the uninstall section for background information.

Example to revert Spark assembly

Create a file (e.g. cluster_hosts) containing a list of all nodes in the cluster (one per line), and run a for loop command such as the example below:

This example assumes that the original Spark Assembly Jar is present in the same location with the .BACKUP suffix.

A restart of the Spark 1 service will be required after this, although this can be done as part of the cluster restarts that are performed once the WANdisco Fusion service is removed.

See the HttpFS for Ambari or HttpFS for Cloudera notes referenced earlier in the uninstall section for background information.

If a HttpFS server is installed on the cluster, then it is likely that it was made compatible for the Fusion client. In order to revert this, a script for Tomcat should be edited to remove references to the Fusion client libraries.

A restart of the HttpFS service will be required after this, although this can be done as part of the cluster restarts that are performed once the WANdisco Fusion service is removed.

Use these steps if wanting to remove the WANdisco Fusion service from the Ambari cluster.

Use these steps if wanting to remove the WANdisco Fusion service from the Cloudera cluster. This will also uninstall the Fusion client on all nodes in the cluster.

Running the script with -h outputs a list of options for the script.

Run the script with the -c option to back up your config and -l to back up WANdisco Fusion logs. The files will be backed up to the following location by default:

Use the -d option to test an uninstallation. This option lets you test the effects of an installation, without any actual file changes being made. Use this option to be sure that your uninstallation will do what you expect.

Running the script without using any additional options performs the following default actions:

If the default options are used, then the Fusion directories will need to manually deleted:

Example

In this example, the /opt/wandisco parent directory has been left intact. Please ensure that any potential files within are safely backed up before removing this.

Running the script with -p will also include the removal of all directories that were created as part of the WANdisco Fusion installation.

It is advisable to use the purge (-p) option in the event that you need to complete a fresh installation, although ensure that the current configuration is backed up (see the "Backup config/log files" section for details).

Please note that the backup option is for recording previous configuration and capturing logs for analysis. It isn’t practical for this option to be used to restore an installation.

When running the installer, first set the following environmental variable:

This will change the installation directory from the default to the one that you provide, e.g.

The above example would install fusion-ui-server into /CustomInstallLocation/fusion-ui-server. Also, the WANdisco Fusion server and IHC server will be installed under /CustomInstallLocation/fusion/server and /CustomInstallLocation/fusion/ihc/server/ respectively.

If you run with the FUSION_PREFIX, an additional line will appear on the summary screen of the installer:

You can also perform an installation to a custom directory using the following alternative:

Inject the environmental variable:

Run the installer as per the usual method, i.e.:

The installer will use the provided path for the installation, as described in the main procedure.

When installing to a custom location, you will need to ensure that your clients are configured with the matching location. You should be able to correctly install clients using the normal procedure, outlined above. See Client Installation, immediately below.

RPM package location
If you need to find the packages after leaving the installer page with the link, you can find them in your installation directory, here:

If you are installing the RPMs, download and install the package on each of your nodes.

Installing the client RPM is done in the usual way:

Install checks

Debian not supported
Although Ubuntu uses Debian’s packaging system, currently Debian itself is not supported. Note: Hortonworks HDP does not support Debian.

If you are running with an Ubuntu Linux distribution, you need to go through the following procedure for installing the clients using Debian’s DEB package:

DEB package location
If you need to find the packages after leaving the installer page with the link, you can find them in your installation directory, here:

By default local parcels are stored on the Cloudera Manager Server:/opt/cloudera/parcel-repo. To change this location, follow the instructions in Configuring Server Parcel Settings.

The location can be changed by setting the parcel_dir property in /etc/cloudera-scm-agent/config.ini file of the Cloudera Manager Agent and restart the Cloudera Manager Agent or by following the instructions in Configuring the Host Parcel Directory.

If you are replacing an existing package that was installed using a parcel, once the new package is activated you should remove the old package through Cloudera Manager. Use the Remove From Host button.

If using an ASF setup, follow the client installation instructions below to install the client on each applicable node. Once complete, distribute the core-site.xml to all nodes in the Hadoop cluster.

Be sure to point your replicated directory to your NFS mount, either directly or using a a symlink.

This is defined as the total amount of data held in AWS in directories that are replicated, excluding those files that are excluded from replication through regex-based filtering. WANdisco Fusion therefore measures the amount of data under replication each hour.

Follow these instructions to set up the metered option of WANdisco Fusion through the AWS Marketplace.

The following checks are made during installation to confirm that the zone has a working S3 bucket.

If you are using access key and secret key then additional fields are required. Please see the Use access key and secret key section for more information on this. These settings can also be updated on the Settings tab in the UI after installation.

Core-site.xml information:

IAM roles are the default method we use for S3 authentication, they are specific to the AWS platform.

If you don’t have the correct IAM roles to access your S3 bucket you can use the Access Key and Secret Key credentials. These are configured by running "aws configure" on the command line. This creates a .aws directory with a "credentials" file with the relevant keys, under a [default] section.

However, there is also the concept of "profiles", and you can store multiple different credentials for different profiles using the CLI command:

When using SSL between Fusion nodes, you can create dedicated truststores. However, when connecting WANdisco Fusion to external resources, such as AWS/Cloud nodes, the SSL connection will fail because these external nodes use CA certificates.

These instructions apply during the set up of WANdisco Fusion on a new AWS EMR cluster. This is the recommended approach, even if you already have an EMR cluster set up.

Install the fusion client (the one for EMR) on each node and after scaling, modify the core-site.xml file with the following: