Confluent Cloud Flink

• Process Data with Flink
• Overview
• Get Started
  • Overview
  • Quick Start with Cloud Console
  • Quick Start with SQL Shell in Confluent CLI
  • Quick Start with Java Table API
  • Quick Start with Python Table API
• Concepts
  • Overview
  • Autopilot
  • Batch and Stream Processing
  • Billing
  • Comparison with Apache Flink
  • Compute Pools
  • Delivery Guarantees and Latency
  • Determinism
  • Private Networking
  • Schema and Statement Evolution
  • Snapshot Queries
  • Statements
  • Statement CFU Metrics
  • Tables and Topics
  • Time and Watermarks
  • User-defined Functions
• How-To Guides
  • Overview
  • Aggregate a Stream in a Tumbling Window
  • Combine Streams and Track Most Recent Records
  • Compare Current and Previous Values in a Stream
  • Convert the Serialization Format of a Topic
  • Create a UDF
  • Deduplicate Rows in a Table
  • Generate Custom Sample Data
  • Handle Multiple Event Types
  • Log Debug Messages in UDFs
  • Mask Fields in a Table
  • Process Schemaless Events
  • Profile a Query
  • Resolve Statement Issues
  • Scan and Summarize Tables
  • Run a Snapshot Query
  • Transform a Topic
  • View Time Series Data
• Operate and Deploy
  • Overview
  • Carry-over Offsets
  • Deploy a Statement with CI/CD
  • Enable Private Networking
  • Generate a Flink API Key
  • Grant Role-Based Access
  • Manage Compute Pools
  • Manage Connections
  • Monitor and Manage Statements
  • Move SQL Statements to Production
  • Profile Queries
  • REST API
• Flink Reference
  • Overview
  • SQL Syntax
  • DDL Statements
    • Statements Overview
    • ALTER AGENT
    • ALTER CONNECTION
    • ALTER MODEL
    • ALTER TABLE
    • ALTER TOOL
    • ALTER VIEW
    • CREATE AGENT
    • CREATE CONNECTION
    • CREATE FUNCTION
    • CREATE MODEL
    • CREATE TABLE
    • CREATE TOOL
    • CREATE VIEW
    • DESCRIBE
    • DROP AGENT
    • DROP CONNECTION
    • DROP MODEL
    • DROP TABLE
    • DROP TOOL
    • DROP VIEW
    • HINTS
    • EXPLAIN
    • RESET
    • SET
    • SHOW
    • USE CATALOG
    • USE database_name
  • DML Statements
    • Queries Overview
    • Deduplication
    • Group Aggregation
    • INSERT INTO FROM SELECT
    • INSERT VALUES
    • Joins
    • LIMIT
    • Pattern Recognition
    • ORDER BY
    • OVER Aggregation
    • SELECT
    • Set Logic
    • EXECUTE STATEMENT SET
    • Top-N
    • Window Aggregation
    • Window Deduplication
    • Window Join
    • Window Top-N
    • Window Table-Valued Function
    • WITH
  • Functions
    • AI Model Inference
    • Aggregate
    • Collections
    • Comparison
    • Conditional
    • Datetime
    • Flink SQL Functions
    • Hashing
    • JSON
    • ML Preprocessing
    • Numeric
    • String
    • Table API
  • Data Types
  • Data Type Mappings
  • Time Zone
  • Keywords
  • Information Schema
  • Example Streams
  • Supported Cloud Regions
  • SQL Examples
  • Table API
  • CLI Reference
• Get Help
• FAQ

Flink SQL Autopilot for Confluent Cloud

Autopilot scales up and scales down the compute resources that SQL statements use in Confluent Cloud for Apache Flink®. Autopilot assigns resources efficiently to SQL statements submitted in Confluent Cloud and provides elastic autoscaling for the entire time the job is running. One of the biggest benefits of using Confluent Cloud for Apache Flink is the built-in Autopilot capability.

State size

The state size is the amount of state that a statement has accumulated. Confluent Cloud for Apache Flink enforces a soft limit of 500 GB and hard limit of 1000 GB on the state size for each statement.

Flink SQL Autopilot in Confluent Cloud for Apache Flink | Confluent Documentation

Topic Actions

Flink Updates: Topic Actions, Terraform support, GCP and Azure Availability

• Topic Actions, which are pre-packaged, turn-key stream processing workloads that handle common, domain-agnostic requirements. Users only need to provide a minimal amount of basic configuration before submitting an Action, which runs internally on Flink. Actions are designed to be extremely simple to use, allowing users to leverage the power of Flink with just a few clicks. We are starting with support for two common workloads—topic deduplication and field masking—and plan to add more in the future.
  • Transform topic
  • Deduplicate topic
  • Mask fields
  • Create embeddings

Others

• SQL Join Queries in Confluent Cloud for Apache Flink | Confluent Documentation
  • Multi-way join optimization - When joining three or more tables on a common join key, Flink can use a multi-way join operator instead of chaining multiple binary joins. This optimization reduces state by eliminating intermediate join results.
• Terraform Support for Flink
  • Deploy a Flink SQL Statement in Confluent Cloud for Apache Flink | Confluent Documentation