Route qualification
01A connector name is the beginning of the design.
Two products can both list the same endpoint while exposing different guarantees, failure boundaries, security dependencies, and cost behavior. Qualify the complete source-to-destination route before treating catalog coverage as production readiness.
Establish the read boundary
Determine how listing, pagination, snapshots, change cursors, and concurrent source mutation interact. The route needs a durable position and a rule for objects or rows created, changed, renamed, or removed during a scan.
Establish the write boundary
Verify atomicity, partial-success reporting, overwrite or upsert behavior, visibility delay, and idempotency at the destination. Define what confirmation closes the unit and what must be reconciled before a timed-out write is repeated.
Design identity and secret rotation
Test least privilege, credential and certificate rotation, proxy or private-network requirements, and expiry during long work. Decide how disconnected runtimes obtain and refresh secrets without embedding them in flow definitions or blocking safe recovery.
Define the data contract
Specify schema evolution, type precision, null behavior, time zones, character encoding, ordering, and format conversion. Decide which changes are compatible, which require quarantine, and how validation evidence follows the transformed data.
Place processing and network paths
Record where bytes cross regions, public networks, gateways, proxies, or egress boundaries and where filtering, aggregation, compression, and encryption execute. Include residency, latency, outage isolation, and offline operation in the route design.
Align lifecycle semantics
Map source delete, rename, archive, retention, legal hold, immutability, target replacement, and cleanup behavior. A transfer is incomplete as an operating design until both endpoint lifecycles and the evidence required before destructive action agree.
Carry recovery evidence
Connect source identity, checkpoint, transformation, validation, destination result, attempts, reconciliation, quarantine, and final action in one trace. Alerts should expose that chain instead of asking operators to correlate unrelated connector logs.
Test throughput and total cost
Measure realistic object sizes, row batches, concurrency, API quotas, transactions, compression, backlog catch-up, and small-file effects. Price compute, storage, requests, egress, gateways, monitoring, support, and operator work together.
Connect the estate you have
02Storage & transfer
Use cloud object storage, HDFS, IPFS, SMB shares, file services, secure transfer protocols, local files, and specialized APIs as sources or destinations. Every route can inherit scheduling, retries, reconciliation, validation, lineage, lifecycle policy, and proactive alerts instead of rebuilding those controls per connector.
Amazon S3
Object storage
Read more →
Azure Blob
Object storage
Read more →Azure Files
File storage
Azure Data Lake
Data lake storage
Google Cloud Storage
Object storage
Read more →
Oracle Cloud
Object storage
Alibaba OSS
Object storage
OpenStack Swift
Object storage
Apache Hadoop HDFS
Distributed file system
IPFS
Content-addressed storage
Google Drive
Cloud drive
Dropbox
Cloud drive
Microsoft OneDrive
Cloud drive
SharePoint libraries
Document library
Databricks Volumes
Lakehouse storage
App Store Connect
Reporting API
SFTP / SSH
Secure transfer
Read more →FTP / FTPS
File transfer
TFTP
File transfer
WebDAV
Web storage
HTTP / HTTPS
Web transfer
SMB 2 / 3
Network file share
Local files
Device storage
IMAP
Mail source
03Database & query engines
Connect Snowflake, transactional databases, analytical engines, warehouses, lakehouses, distributed databases, and generic SQL protocols without coupling the flow to one vendor. Snowflake routes support governed reads, writes, and explicit transactions; qualify DML volume and loading strategy for each workload. Flower carries integrity checks and operational evidence across the hand-off, so teams can govern the movement as one path.
PostgreSQL
SQL database
Read more →
MySQL
SQL database
Read more →
Microsoft SQL Server
SQL database
Read more →Oracle Database
SQL database
Read more →
SQLite
Embedded database
ClickHouse
Analytical database
Databricks SQL
Lakehouse storage
Snowflake
Cloud data platform
SAP HANA
SQL database
Cloud Spanner
Distributed database
Trino
Query engine
Presto
Query engine
Exasol
Analytical database
Vertica
Analytical database
IBM Netezza
Data warehouse
Arrow Flight SQL
Query protocol
ODBC / Avatica
Generic SQL access
04Secrets and certificates
Resolve secrets and certificates from cloud vaults, cluster stores, encrypted files, or a constrained local directory before dependent components start. Flower publishes one read-only configuration snapshot only after every required source loads, so missing values fail startup instead of surfacing in the middle of a flow.
Azure Key Vault
Cloud secrets and certificates
Google Cloud Secret Manager
Cloud secrets and certificates
AWS Secrets Manager + KMS
Secret manager and key service
AWS Systems Manager Parameter Store
Managed parameter store
Oracle Cloud Infrastructure Vault
Cloud vault and secrets
HashiCorp Vault / OpenBao
Enterprise secrets vault
Alibaba Cloud KMS Secrets Manager
Cloud secrets and certificates
Huawei Cloud CSMS
Cloud secrets and certificates
IBM Cloud Secrets Manager
Cloud secrets and certificates
Kubernetes Secrets
Cluster secret store
SOPS encrypted files
Encrypted configuration files
Local files
Constrained local secret source
05Optional processing runtimes
Standard Flower flows remain declarative and require no programming. When a route needs custom behavior or processing, optional JavaScript, Lua, WebAssembly, Starlark, and AWK runtimes extend Flower capabilities without adding an external compute service. Starlark and AWK enforce configured execution and data budgets: Starlark supports deterministic rule logic, while AWK handles text, CSV, and TSV records in motion.
JavaScript
Bounded JavaScript runtime
Lua
Isolated Lua runtime
WebAssembly
Isolated WebAssembly modules
Starlark
Deterministic bounded runtime
AWK
Bounded text, CSV and TSV runtime
06Formats in motion
Decode, transform, aggregate, validate, encrypt, compress, transcode, and re-encode operational and analytical formats while records move between endpoints. Process data in motion to deliver the required shape without an unnecessary chain of external services.
CSV
Format in motion
JSON
Format in motion
Avro
Format in motion
Parquet
Format in motion
Excel
Format in motion
XML
Format in motion
CBOR
Format in motion
MessagePack
Format in motion
Fixed-width
Format in motion