Ice Flow Extension
Ice Flow is an extension for Symphony that provides monitoring, replication, and catalog management capabilities for Apache Iceberg data catalogs. It wraps the Iceberg library as a managed service, enabling you to connect multiple Iceberg catalogs and keep them synchronized.
note
Key Capabilities
- Monitor Iceberg catalogs for changes and metadata events
- Replicate tables between different Iceberg catalogs
- Browse catalog content including schemas and sample data
- Track replication operations and file transfers in real-time
Get Started
New to Ice Flow? Install the extension, walk through the first-catalog tutorial, and find the operational procedures for keeping it running.
- Installation — install Ice Flow via RPM, Docker, or Helm
- Getting Started — set up your first catalog, scope, and replication from scratch
- Upgrades — upgrade Ice Flow in place with minimal downtime
- Uninstallation — stop, remove, and optionally wipe stored state
Understand
Learn the concepts behind Ice Flow before diving in.
- Core Concepts — catalogs, warehouses, scopes, location mappings, and how they fit together
How-To Guides
Task-oriented guides for specific goals.
- Manage Catalogs — add, configure, and remove Iceberg catalog connections
- Browse Catalog Content — explore namespaces, tables, schemas, and sample data
- Set Up Replication — replicate tables between catalogs with one-time or continuous sync
- Set Up Monitoring — observe Hive Metastore events in real time
- Define Scopes — create reusable table selectors for monitors and replications
- Manage Warehouses — define storage locations and location mappings
- Configure Kerberos — enable per-catalog Kerberos authentication for secured clusters
- Authentication Settings —
inspect, validate, and override the merged JVM-wide
krb5.conf - Check Replication Consistency — verify that source and target catalogs match
Reference
Detailed specifications and configuration options.
- Catalog Type Reference — required properties for each catalog type
- Configuration Reference — environment variables, properties, and deployment options
- Data Retention — retention policies, TTLs, and automatic compaction
- Troubleshooting — solutions for common errors and unexpected behaviour