Connected data flows
01A connector name is the beginning of the design.
Connect the systems you already operate through a shared orchestration and processing model. Flower brings scheduling, transformation, validation, recovery and operational evidence into the route, so teams can manage the exchange as one coherent data flow.
Connect data sources
Determine how listing, pagination, snapshots, change cursors, and concurrent source mutation interact. The route needs a durable position and a rule for objects or rows created, changed, renamed, or removed during a scan.
Deliver to your destinations
Verify atomicity, partial-success reporting, overwrite or upsert behavior, visibility delay, and idempotency at the destination. Define what confirmation closes the unit and what must be reconciled before a timed-out write is repeated.
Design identity and secret rotation
Test least privilege, credential and certificate rotation, proxy or private-network requirements, and expiry during long work. Decide how disconnected runtimes obtain and refresh secrets without embedding them in flow definitions or blocking safe recovery.
Define the data contract
Specify schema evolution, type precision, null behavior, time zones, character encoding, ordering, and format conversion. Decide which changes are compatible, which require quarantine, and how validation evidence follows the transformed data.
Place processing and network paths
Record where bytes cross regions, public networks, gateways, proxies, or egress boundaries and where filtering, aggregation, compression, and encryption execute. Include residency, latency, outage isolation, and offline operation in the route design.
Align lifecycle semantics
Map source delete, rename, archive, retention, legal hold, immutability, target replacement, and cleanup behavior. A transfer is incomplete as an operating design until both endpoint lifecycles and the evidence required before destructive action agree.
Carry recovery evidence
Connect source identity, checkpoint, transformation, validation, destination result, attempts, reconciliation, quarantine, and final action in one trace. Alerts should expose that chain instead of asking operators to correlate unrelated connector logs.
Test throughput and total cost
Measure realistic object sizes, row batches, concurrency, API quotas, transactions, compression, backlog catch-up, and small-file effects. Price compute, storage, requests, egress, gateways, monitoring, support, and operator work together.
Connect the estate you have
02Storage & transfer
Use cloud object storage, HDFS, SMB shares, file services, secure transfer protocols, local files, and specialized APIs as sources or destinations. Every route can inherit scheduling, retries, reconciliation, validation, lineage, lifecycle policy, and proactive alerts instead of rebuilding those controls per connector.
Extend existing routes to MinIO, Backblaze B2, Huawei Cloud OBS, Tencent Cloud COS, Volcano Engine TOS and UPYUN, or move files through Koofr and pCloud. Native adapters bring private-cloud and regional storage choices into the same declarative flow, with shared orchestration, processing and operational visibility.
Bring team files into governed data flows with Box, Egnyte, Files.com, ShareFile and Seafile. MongoDB GridFS adds streamed file storage with native metadata and retained revisions, connecting document deliveries to the same orchestration and processing model.
Reuse existing folders and team file services as sources and destinations. Flower brings scheduling, record processing, delivery checks and lifecycle policies into the exchange, helping teams automate recurring work while retaining their established collaboration tools.
Amazon S3
Object storage
Read more →
Azure Blob
Object storage
Read more →Azure Files
File storage
Azure Data Lake
Data lake storage
Google Cloud Storage
Object storage
Read more →
Oracle Cloud
Object storage
Alibaba OSS
Object storage
OpenStack Swift
Object storage
MinIO
Object storage
Backblaze B2
Object storage
Huawei Cloud OBS
Object storage
Tencent Cloud COS
Object storage
Volcano Engine TOS
Object storage
UPYUN
Object storage
Apache Hadoop HDFS
Distributed file system
Google Drive
Cloud drive
Dropbox
Cloud drive
Koofr
Cloud drive
pCloud
Cloud drive
Microsoft OneDrive
Cloud drive
SharePoint libraries
Document library
Box
Native files and folders
Egnyte
Native files and folders
Files.com
Native files and folders
ShareFile
Native files and folders
Seafile
Document library
MongoDB GridFS
File storage in MongoDB
Databricks Volumes
Lakehouse storage
App Store Connect
Reporting API
SFTP / SSH
Secure transfer
Read more →FTP / FTPS
File transfer
WebDAV
Web storage
HTTP / HTTPS
Web transfer
SMB 2 / 3
Network file share
Local files
Device storage
IMAP
Mail source
03Database & query engines
Connect Snowflake, transactional databases, analytical engines, warehouses, lakehouses, distributed databases, and SQL protocols through a common orchestration model. Flower supports governed reads, writes, and explicit transactions, carrying integrity checks and operational evidence across the data flow.
Bring operational SQL data into the same governed flow. Flower provides native Firebird and Huawei GaussDB adapters, connects MariaDB through its MySQL adapter, and connects CockroachDB through PostgreSQL compatibility. These connections support reads, writes and explicit transactions; the Firebird client does not require a separately installed native client library.
Combine SQL extraction, loading and validation in a declarative route. Apply common scheduling, credential management and monitoring across operational and analytical systems, keeping data movement aligned with your existing database architecture.
For deployments standardized on ODBC, Flower provides backend-aware access to its application-state database as well as ODBC data connectivity. SQL profiles cover PostgreSQL, GaussDB, CockroachDB, MySQL, Microsoft SQL Server and SQLite, helping organizations reuse their database estate and established driver infrastructure.
PostgreSQL
SQL database
Read more →
MySQL
SQL database
Read more →
Microsoft SQL Server
SQL database
Read more →Oracle Database
SQL database
Read more →
SQLite
Embedded database
ClickHouse
Analytical database
Databricks SQL
Lakehouse storage
Snowflake
Cloud data platform
SAP HANA
SQL database
Cloud Spanner
Distributed database
Trino
Query engine
Presto
Query engine
Exasol
Analytical database
Vertica
Analytical database
IBM Netezza
Data warehouse
Arrow Flight SQL
Query protocol
ODBC / Avatica
Generic SQL access
Firebird
SQL database
Huawei GaussDB
SQL database
MariaDB
SQL via MySQL compatibility
CockroachDB
Distributed SQL via PostgreSQL
04Shared caches
Reuse intermediate state across Flower components with managed key-value services, persistent collections or distributed maps. The cache integrations below provide backend-aware key organization, expiry and timeouts within the same governed flow.
Share reusable state across components with backend-aware expiry and key organization. etcd leases, MongoDB collections and DynamoDB items support cache lifecycles within the same data flow, while Flower brings consistent configuration and operational visibility to the selected services.
Redis, Valkey and Dragonfly share Flower’s Redis adapter for RESP2/RESP3 endpoints, including key expiry and prefix-scoped operations. Reuse a familiar key-value model while bringing cache state into the same declarative configuration as the surrounding data flow.
etcd
Distributed key-value cache
MongoDB
Persistent collection-backed cache
Amazon DynamoDB
Managed table-backed cache
Couchbase
Persistent collection-backed cache
Aerospike
Distributed key-value cache
Hazelcast
Distributed map cache
Oracle Coherence
Named cache over gRPC
Infinispan
Distributed cache over REST
TiKV
Distributed RawKV cache
Redis
Distributed key-value cache
Valkey
Cache via Redis compatibility
Dragonfly
Cache via Redis compatibility
05AI and local classification
Enrich and classify records inside the data flow. Flower connects OpenAI and Anthropic Claude for structured transformations, offers optional local GGUF inference with llama.cpp, and uses ONNX models for local text and zero-shot classification. Select the processing location to match data confidentiality, available hardware and cost priorities.
Control which fields reach external models and define the expected output schema in the flow configuration. Flower coordinates requests, preserves record order and applies your chosen handling policies, bringing AI enrichment into the same governed route as validation, transformation and delivery.
Semantic filters use OpenAI, Claude or local GGUF models to select records using criteria expressed in natural language. Keep or exclude matching records while preserving their fields, values and order. Field selection, processing policies and execution metrics make these decisions part of an observable, configurable data flow.
Classify text locally with ONNX models and add labels and scores directly to your records, or use categories to guide filtering. Choose a model and tokenizer suited to your languages and data. Flower runs this classification without an ONNX Runtime shared library, keeping processing close to the data.
Choose optional local GGUF inference with llama.cpp to run language-model processing within your own infrastructure. Flower loads the model on demand and performs inference inside the runtime, bringing transformations and semantic filters close to the data with direct control over model choice and execution.
Recognize likely personal names locally using an embedded dictionary. Label or filter records within the data flow, with configurable scoring thresholds and handling policies. Processing stays inside Flower, without external model calls or model downloads.
OpenAI
Structured model responses
Anthropic Claude
Structured model responses
llama.cpp / GGUF
Optional local language-model inference
ONNX
Local text and zero-shot classification
Personal-name detection
Local dictionary-based classification
06Secrets and certificates
Bring secrets and certificates from cloud vaults, cluster stores, encrypted files, or local directories into a unified configuration. Flower prepares a consistent, read-only configuration snapshot before starting dependent components, making secure configuration part of the flow lifecycle.
Azure Key Vault
Cloud secrets and certificates
Google Cloud Secret Manager
Cloud secrets and certificates
AWS Secrets Manager + KMS
Secret manager and key service
AWS Systems Manager Parameter Store
Managed parameter store
Oracle Cloud Infrastructure Vault
Cloud vault and secrets
HashiCorp Vault / OpenBao
Enterprise secrets vault
Alibaba Cloud KMS Secrets Manager
Cloud secrets and certificates
Huawei Cloud CSMS
Cloud secrets and certificates
IBM Cloud Secrets Manager
Cloud secrets and certificates
Kubernetes Secrets
Cluster secret store
SOPS encrypted files
Encrypted configuration files
Local files
Constrained local secret source
07Optional processing runtimes
Standard Flower flows are declarative and require no programming. Optional JavaScript, Lua, WebAssembly, Starlark, and AWK runtimes add custom processing inside the flow, without an external compute service. Use Starlark for deterministic rules and AWK for text, CSV, and TSV records in motion.
JavaScript
JavaScript processing
Lua
Isolated Lua runtime
WebAssembly
Isolated WebAssembly modules
Starlark
Deterministic rule processing
AWK
Text, CSV and TSV processing
08Formats in motion
Decode, transform, aggregate, validate, encrypt, compress, transcode, and re-encode operational and analytical formats while records move between endpoints. Process data in motion to deliver the required shape without an unnecessary chain of external services.
Bring spreadsheet deliveries into the same flow as files and databases. Alongside Excel, Flower reads OpenDocument Spreadsheet (ODS) files and creates ODS 1.3 workbooks. Select a worksheet and cell range, map columns and validate headers, then convert records to CSV, JSON, Parquet or another supported destination. Processing runs directly in Flower, without an office application.
For ODS formulas, choose saved results, original expressions or OpenFormula recalculation to suit the flow. Bring spreadsheet values and formula results into validation, transformation and analytical delivery with the same declarative processing model.
Protect spreadsheet exchanges with password-based ODS encryption. Flower supports AES-CBC and AES-GCM for encrypted input and output, placing data protection in the same declarative flow as processing and delivery.
CSV
Format in motion
JSON
Format in motion
Avro
Format in motion
Parquet
Format in motion
Excel
Format in motion
ODS
OpenDocument spreadsheets
XML
Format in motion
CBOR
Format in motion
MessagePack
Format in motion
Fixed-width
Format in motion