org.apache.flink.api.java.operators

Class PartitionOperator<T>

java.lang.Object

org.apache.flink.api.java.DataSet<OUT>

org.apache.flink.api.java.operators.Operator<OUT,O>

org.apache.flink.api.java.operators.SingleInputOperator<T,T,PartitionOperator<T>>

org.apache.flink.api.java.operators.PartitionOperator<T>

Type Parameters:

T - The type of the data being partitioned.

@Public
public class PartitionOperator<T>
extends SingleInputOperator<T,T,PartitionOperator<T>>

This operator represents a partitioning.

Field Summary

Fields inherited from class org.apache.flink.api.java.operators.Operator

minResources, name, parallelism, preferredResources

Fields inherited from class org.apache.flink.api.java.DataSet

context

Constructor Summary

Constructors

Constructor and Description
PartitionOperator(DataSet<T> input, org.apache.flink.api.common.operators.Keys<T> pKeys, org.apache.flink.api.common.functions.Partitioner<?> customPartitioner, String partitionLocationName)
PartitionOperator(DataSet<T> input, org.apache.flink.api.common.operators.Keys<T> pKeys, org.apache.flink.api.common.functions.Partitioner<P> customPartitioner, org.apache.flink.api.common.typeinfo.TypeInformation<P> partitionerTypeInfo, String partitionLocationName)
PartitionOperator(DataSet<T> input, org.apache.flink.api.common.operators.base.PartitionOperatorBase.PartitionMethod pMethod, org.apache.flink.api.common.operators.Keys<T> pKeys, org.apache.flink.api.common.distributions.DataDistribution distribution, String partitionLocationName)
PartitionOperator(DataSet<T> input, org.apache.flink.api.common.operators.base.PartitionOperatorBase.PartitionMethod pMethod, org.apache.flink.api.common.operators.Keys<T> pKeys, String partitionLocationName)
PartitionOperator(DataSet<T> input, org.apache.flink.api.common.operators.base.PartitionOperatorBase.PartitionMethod pMethod, String partitionLocationName)

Method Summary

Modifier and Type	Method and Description
org.apache.flink.api.common.functions.Partitioner<?>	getCustomPartitioner() Gets the custom partitioner from this partitioning.
protected org.apache.flink.api.common.operators.SingleInputOperator<?,T,?>	translateToDataFlow(org.apache.flink.api.common.operators.Operator<T> input) Translates this operation to a data flow operator of the common data flow API.
PartitionOperator<T>	withOrders(org.apache.flink.api.common.operators.Order... orders) Sets the order of keys for range partitioning.

Methods inherited from class org.apache.flink.api.java.operators.SingleInputOperator

getInput, getInputType

Methods inherited from class org.apache.flink.api.java.operators.Operator

getMinResources, getName, getParallelism, getPreferredResources, getResultType, name, setParallelism

Methods inherited from class org.apache.flink.api.java.DataSet

aggregate, checkSameExecutionContext, clean, coGroup, collect, combineGroup, count, cross, crossWithHuge, crossWithTiny, distinct, distinct, distinct, distinct, fillInType, filter, first, flatMap, fullOuterJoin, fullOuterJoin, getExecutionEnvironment, getType, groupBy, groupBy, groupBy, iterate, iterateDelta, join, join, joinWithHuge, joinWithTiny, leftOuterJoin, leftOuterJoin, map, mapPartition, max, maxBy, min, minBy, output, partitionByHash, partitionByHash, partitionByHash, partitionByRange, partitionByRange, partitionByRange, partitionCustom, partitionCustom, partitionCustom, print, print, printOnTaskManager, printToErr, printToErr, project, rebalance, reduce, reduceGroup, rightOuterJoin, rightOuterJoin, runOperation, sortPartition, sortPartition, sortPartition, sum, union, write, write, writeAsCsv, writeAsCsv, writeAsCsv, writeAsCsv, writeAsFormattedText, writeAsFormattedText, writeAsText, writeAsText

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PartitionOperator

public PartitionOperator(DataSet<T> input,
                         org.apache.flink.api.common.operators.base.PartitionOperatorBase.PartitionMethod pMethod,
                         org.apache.flink.api.common.operators.Keys<T> pKeys,
                         String partitionLocationName)

PartitionOperator

public PartitionOperator(DataSet<T> input,
                         org.apache.flink.api.common.operators.base.PartitionOperatorBase.PartitionMethod pMethod,
                         org.apache.flink.api.common.operators.Keys<T> pKeys,
                         org.apache.flink.api.common.distributions.DataDistribution distribution,
                         String partitionLocationName)

PartitionOperator

public PartitionOperator(DataSet<T> input,
                         org.apache.flink.api.common.operators.base.PartitionOperatorBase.PartitionMethod pMethod,
                         String partitionLocationName)

PartitionOperator

public PartitionOperator(DataSet<T> input,
                         org.apache.flink.api.common.operators.Keys<T> pKeys,
                         org.apache.flink.api.common.functions.Partitioner<?> customPartitioner,
                         String partitionLocationName)

PartitionOperator

public PartitionOperator(DataSet<T> input,
                         org.apache.flink.api.common.operators.Keys<T> pKeys,
                         org.apache.flink.api.common.functions.Partitioner<P> customPartitioner,
                         org.apache.flink.api.common.typeinfo.TypeInformation<P> partitionerTypeInfo,
                         String partitionLocationName)

Method Detail

withOrders

@PublicEvolving
public PartitionOperator<T> withOrders(org.apache.flink.api.common.operators.Order... orders)

Sets the order of keys for range partitioning. NOTE: Only valid for PartitionOperatorBase.PartitionMethod.RANGE.

Parameters:

orders - array of orders for each specified partition key

Returns:

The partitioneOperator with properly set orders for given keys

getCustomPartitioner

@Internal
public org.apache.flink.api.common.functions.Partitioner<?> getCustomPartitioner()

Gets the custom partitioner from this partitioning.

Returns:

The custom partitioner.

translateToDataFlow

protected org.apache.flink.api.common.operators.SingleInputOperator<?,T,?> translateToDataFlow(org.apache.flink.api.common.operators.Operator<T> input)

Description copied from class: SingleInputOperator

Translates this operation to a data flow operator of the common data flow API.

Specified by:

translateToDataFlow in class SingleInputOperator<T,T,PartitionOperator<T>>

Parameters:

input - The data flow operator that produces this operation's input data.

Returns:

The translated data flow operator.