org.apache.flink.api.java.operators

Class DistinctOperator<T>

java.lang.Object

org.apache.flink.api.java.DataSet<OUT>

org.apache.flink.api.java.operators.Operator<OUT,O>

org.apache.flink.api.java.operators.SingleInputOperator<T,T,DistinctOperator<T>>

org.apache.flink.api.java.operators.DistinctOperator<T>

Type Parameters:

T - The type of the data set made distinct by the operator.

@Public
public class DistinctOperator<T>
extends SingleInputOperator<T,T,DistinctOperator<T>>

This operator represents the application of a "distinct" function on a data set, and the result data set produced by the function.

Field Summary

Fields inherited from class org.apache.flink.api.java.operators.Operator

minResources, name, parallelism, preferredResources

Fields inherited from class org.apache.flink.api.java.DataSet

context

Constructor Summary

Constructors

Constructor and Description
DistinctOperator(DataSet<T> input, org.apache.flink.api.common.operators.Keys<T> keys, String distinctLocationName)

Method Summary

Modifier and Type	Method and Description
DistinctOperator<T>	setCombineHint(org.apache.flink.api.common.operators.base.ReduceOperatorBase.CombineHint strategy) Sets the strategy to use for the combine phase of the reduce.
protected org.apache.flink.api.common.operators.SingleInputOperator<?,T,?>	translateToDataFlow(org.apache.flink.api.common.operators.Operator<T> input) Translates this operation to a data flow operator of the common data flow API.

Methods inherited from class org.apache.flink.api.java.operators.SingleInputOperator

getInput, getInputType

Methods inherited from class org.apache.flink.api.java.operators.Operator

getMinResources, getName, getParallelism, getPreferredResources, getResultType, name, setParallelism

Methods inherited from class org.apache.flink.api.java.DataSet

aggregate, checkSameExecutionContext, clean, coGroup, collect, combineGroup, count, cross, crossWithHuge, crossWithTiny, distinct, distinct, distinct, distinct, fillInType, filter, first, flatMap, fullOuterJoin, fullOuterJoin, getExecutionEnvironment, getType, groupBy, groupBy, groupBy, iterate, iterateDelta, join, join, joinWithHuge, joinWithTiny, leftOuterJoin, leftOuterJoin, map, mapPartition, max, maxBy, min, minBy, output, partitionByHash, partitionByHash, partitionByHash, partitionByRange, partitionByRange, partitionByRange, partitionCustom, partitionCustom, partitionCustom, print, print, printOnTaskManager, printToErr, printToErr, project, rebalance, reduce, reduceGroup, rightOuterJoin, rightOuterJoin, runOperation, sortPartition, sortPartition, sortPartition, sum, union, write, write, writeAsCsv, writeAsCsv, writeAsCsv, writeAsCsv, writeAsFormattedText, writeAsFormattedText, writeAsText, writeAsText

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DistinctOperator

public DistinctOperator(DataSet<T> input,
                        org.apache.flink.api.common.operators.Keys<T> keys,
                        String distinctLocationName)

Method Detail

translateToDataFlow

protected org.apache.flink.api.common.operators.SingleInputOperator<?,T,?> translateToDataFlow(org.apache.flink.api.common.operators.Operator<T> input)

Description copied from class: SingleInputOperator

Translates this operation to a data flow operator of the common data flow API.

Specified by:

translateToDataFlow in class SingleInputOperator<T,T,DistinctOperator<T>>

Parameters:

input - The data flow operator that produces this operation's input data.

Returns:

The translated data flow operator.

setCombineHint

@PublicEvolving
public DistinctOperator<T> setCombineHint(org.apache.flink.api.common.operators.base.ReduceOperatorBase.CombineHint strategy)

Sets the strategy to use for the combine phase of the reduce.

If this method is not called, then the default hint will be used. (ReduceOperatorBase.CombineHint.OPTIMIZER_CHOOSES)

Parameters:

strategy - The hint to use.

Returns:

The DistinctOperator object, for function call chaining.