Skip to main content

Troubleshooting

Solutions for common issues when using Ice Flow.

Catalog Issues​

Catalog connection fails : Verify the URI, credentials, and network connectivity between Symphony and the catalog endpoint. Open the catalog's Content tab to test the connection — if the catalog explorer loads namespaces and tables, the connection is working.

No tables appear in the catalog explorer : Confirm the catalog URI is correct and that the service account has read permissions on the catalog and its underlying storage. For JDBC catalogs, verify the JDBC driver JAR is provided via -Djdbc.extra.driver.jars.

Catalog cannot be deleted : A catalog that has active monitors or replications cannot be removed. Delete or stop those monitors and replications first, then retry the deletion.

Scope Issues​

No tables found by a scope : Namespace names are case-sensitive and must match exactly as they appear in the catalog. Use the catalog Content browser to verify the namespace name before creating scopes.

Replication Issues​

Continuous replication mode is unavailable : Continuous replication requires a Hive catalog as the source, because it relies on the Hive Metastore event stream to detect changes. For non-Hive sources, use one-time mode.

Replication stays in "replicating" state : For one-time replications, this means the replication is still running — check the Operations tab for active or errored operations. For continuous replications, "replicating" is the expected steady state — the replication runs until stopped.

Missing data files on the target : Verify that:

  1. The target warehouse storage location is accessible and writable
  2. Cloud credentials are configured in the target catalog properties
  3. A location mapping exists if source and target use different storage paths
  4. The mapping is associated with the replication (check the Mappings tab)

Older operations or events are missing : Ice Flow retains detailed monitoring events for 2 days and operation records for 7 days. Older data is automatically compacted into daily summaries. This is expected — see Data Retention.

Kerberos Issues​

Kerberos authentication fails : Verify that:

  1. The keytab file exists and is readable by the Ice Flow process user
  2. The principal name matches exactly (including realm)
  3. The principal has an entry in the keytab — check with klist -k <keytab>
  4. For headless keytabs, use the short form (e.g. hdfs@REALM not hdfs/_HOST@REALM)

"Unable to obtain password from user" : The principal does not match any entry in the keytab. Run klist -k /path/to/keytab to see the available principals and update the catalog configuration to match.

"GSSException: No valid credentials provided" : The keytab may have expired or the principal may not exist in the KDC. Re-export the keytab using kadmin and update the keytab path or re-upload the file in the catalog settings.

Hostname (service) principal fails to connect to the metastore (NoMatchingRule) : A service principal such as hive/host.example.com@OTHER.REALM is supported for catalogs in any realm, including realms with no cross-realm trust to the Ice Flow host. If you see KerberosName$NoMatchingRule or a metastore connection failure that only affects hostname-based keytabs (while headless keytabs in the same realm work), upgrade to a build that maps two-component principals from every realm to a local user. No catalog change is required.

"Cannot locate default realm" : The JVM cannot find the Kerberos realm configuration. Enable Override krb5.conf content in the catalog settings and provide the realm configuration. This is required when the Ice Flow host does not have a system-wide /etc/krb5.conf that includes the target realm.

Deployment Issues​

Extension cannot connect to Symphony : The SYMPHONY_TOKEN must be a valid, current API key. In container deployments, the token must be provided via the environment variable or Kubernetes secret.

Container crashes with NATS connection error : Ensure the container can reach the Symphony host on port 4222 (NATS). In Kubernetes, if Symphony runs in-cluster, set symphony.serviceName in the Helm values to the Service name. If it runs externally, set symphony.serviceName to empty and ensure external DNS resolves correctly.

Helm deployment init container fails : The resolve-symphony init container resolves the in-cluster Symphony service IP. If Symphony is not running in the same namespace, or the service name is wrong, the init container will fail. Set symphony.serviceName to empty to skip the init container and rely on external DNS.

License Enforcement​

Activities are paused with "License enforcement active" : Symphony license enforcement has been activated. All monitoring and replication activities are automatically paused. Activities will resume when enforcement is lifted. Contact your administrator to resolve the license issue.

"Ice Flow is disabled" : The Ice Flow extension has been specifically disabled by the license system. This is independent of global enforcement. Check your license coverage with your administrator.