首页 笔记 图片 查字 
所属分类:Flink
浏览:79
内容:

DataStream 八种物理分区器:
包路径:org.apache.flink.streaming.runtime.partitioner
GlobalPartitioner
ShufflePartitioner
RebalancePartitioner
KeyGroupStreamPartitioner
BroadcastPartitioner
RescalePartitioner
ForwardPartitioner
CustomPartitionerWrapper

DataStream 重分区算子:
keyBy:先按照key分组,再按照key的hash来选择后面的分区,分区器:KeyGroupStreamPartitioner
shuffle:将数据随机发往下游的每一个suatask,分区器:ShufflePartitioner
rebalance:以轮询方式均分到下游每一个subtask,分区器:RebalancePartitioner
rescale:上游2个分区,下游4个分区,则上游每个分区的全量数据轮询发往下游的两个分区,分区器:RescalePartitioner
broadcast:将数据广播给下游每一个subtask,分区器:BroadcastPartitioner
global:所有数据发往下游第一个subtask,分区器:GlobalPartitioner
forward:只在当前分区做计算,分区器:ForwardPartitioner
partitionCustom:用户自定义分区方式,分区器:CustomPartitionerWrapper