Use Python API in Apache Flink

History, Status Quo, and Future Development of Apache Flink Python API

Reasons Why Apache Flink Supports Python

  • Python is one of the most popular development languages
  • Python is supported by many open-source projects
  • Python is favored by machine learning (ML).

Python API — RoadMap

Apache Flink Python API Architecture and Development Environment

Python Table API Architecture

  • No need to create a new set of operators, instead easily maintain consistency with functions of the Java Table API.
  • Optimize the Python API by using the Java Table API optimization model. This ensures that jobs written by using the Python API provide optimal performance.

Python Table API — Job Development

Python Table API — Development Environment

//下载源代码
git clone https://github.com/apache/flink.git
// 拉取1.9分支
cd flink; git fetch origin release-1.9
git checkout -b release-1.9 origin/release-1.9
//构建二进制发布包
mvn clean install -DskipTests -Dfast
cd flink-dist/target/flink-1.9.0-bin/flink-1.9.0
tar -zcvf flink-1.9.0.tar.gz flink-1.9.0
cd flink-Python;Python setup.py sdist
cd dist/
pip install dist/*.tar.gz
pip list|grep flink
git clone https://github.com/sunjincheng121/enjoyment.code.git
cd enjoyment.code; Python word_count.py

Python Table API — Job Submission

./bin/flink run -py  ~/training/0806/enjoyment.code/myPyFlink/enjoyment/word_count_cli.py
bin/pyflink-shell.sh local

Introduction and Application of Key Operators of Flink Python APIs

Python Table API Operators

Python Table API Operators — Watermark Definition

How to define Watermark in Python API?

Python Table API — Java UDF

Reference Links of the Python Table API

Summary

Original Source:

--

--

--

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Websocket Deep Dive

Migrating from Monoliths to Micro-services

7 coding platforms for new learners

MongoDB Authentication in a React and Nodejs project

Essential Tools for Project Management

Introducing Solutions @ Klaviyo

How Coding Reshaped The Way I Think Now

Monitoring for the End-User Experience with Catchpoint and Google Lighthouse (Part 2)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com

More from Medium

Form Fits Function: A Case Study on Integrating MongoDB and Redis with Apache Spark

A Quick insight on Apache Airflow

Introduction Apache Kafka

Spark structured streaming