org.apache.flink.api.java.operators

Class CoGroupOperator<I1,I2,OUT>

java.lang.Object

org.apache.flink.api.java.DataSet<OUT>

org.apache.flink.api.java.operators.Operator<OUT,O>

org.apache.flink.api.java.operators.TwoInputOperator<IN1,IN2,OUT,O>

org.apache.flink.api.java.operators.TwoInputUdfOperator<I1,I2,OUT,CoGroupOperator<I1,I2,OUT>>

org.apache.flink.api.java.operators.CoGroupOperator<I1,I2,OUT>

Type Parameters:

I1 - The type of the first input DataSet of the CoGroup transformation.

I2 - The type of the second input DataSet of the CoGroup transformation.

OUT - The type of the result of the CoGroup transformation.

All Implemented Interfaces:

UdfOperator<CoGroupOperator<I1,I2,OUT>>

@Public
public class CoGroupOperator<I1,I2,OUT>
extends TwoInputUdfOperator<I1,I2,OUT,CoGroupOperator<I1,I2,OUT>>

A DataSet that is the result of a CoGroup transformation.

See Also:

DataSet

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CoGroupOperator.CoGroupOperatorSets<I1,I2> Intermediate step of a CoGroup transformation.

Field Summary

Fields inherited from class org.apache.flink.api.java.operators.Operator

minResources, name, parallelism, preferredResources

Fields inherited from class org.apache.flink.api.java.DataSet

context

Constructor Summary

Constructors

Constructor and Description
CoGroupOperator(DataSet<I1> input1, DataSet<I2> input2, org.apache.flink.api.common.operators.Keys<I1> keys1, org.apache.flink.api.common.operators.Keys<I2> keys2, org.apache.flink.api.common.functions.CoGroupFunction<I1,I2,OUT> function, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> returnType, List<org.apache.commons.lang3.tuple.Pair<Integer,org.apache.flink.api.common.operators.Order>> groupSortKeyOrderFirst, List<org.apache.commons.lang3.tuple.Pair<Integer,org.apache.flink.api.common.operators.Order>> groupSortKeyOrderSecond, org.apache.flink.api.common.functions.Partitioner<?> customPartitioner, String defaultName)
CoGroupOperator(DataSet<I1> input1, DataSet<I2> input2, org.apache.flink.api.common.operators.Keys<I1> keys1, org.apache.flink.api.common.operators.Keys<I2> keys2, org.apache.flink.api.common.functions.CoGroupFunction<I1,I2,OUT> function, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> returnType, org.apache.flink.api.common.functions.Partitioner<?> customPartitioner, String defaultName)

Method Summary

Modifier and Type	Method and Description
protected org.apache.flink.api.common.functions.CoGroupFunction<I1,I2,OUT>	getFunction()
protected org.apache.flink.api.common.operators.Keys<I1>	getKeys1()
protected org.apache.flink.api.common.operators.Keys<I2>	getKeys2()
org.apache.flink.api.common.functions.Partitioner<?>	getPartitioner() Gets the custom partitioner used by this join, or null, if none is set.
org.apache.flink.api.common.operators.DualInputSemanticProperties	getSemanticProperties() Gets the semantic properties that have been set for the user-defined functions (UDF).
protected org.apache.flink.api.common.operators.base.CoGroupOperatorBase<?,?,OUT,?>	translateToDataFlow(org.apache.flink.api.common.operators.Operator<I1> input1, org.apache.flink.api.common.operators.Operator<I2> input2) Translates this java API operator into a common API operator with two inputs.
CoGroupOperator<I1,I2,OUT>	withPartitioner(org.apache.flink.api.common.functions.Partitioner<?> partitioner) Sets a custom partitioner for the CoGroup operation.

Methods inherited from class org.apache.flink.api.java.operators.TwoInputUdfOperator

extractSemanticAnnotationsFromUdf, getAnalyzedUdfSemanticsFlag, getBroadcastSets, getParameters, returns, returns, returns, setAnalyzedUdfSemanticsFlag, setSemanticProperties, udfWithForwardedFieldsFirstAnnotation, udfWithForwardedFieldsSecondAnnotation, withBroadcastSet, withForwardedFieldsFirst, withForwardedFieldsSecond, withParameters

Methods inherited from class org.apache.flink.api.java.operators.TwoInputOperator

getInput1, getInput1Type, getInput2, getInput2Type

Methods inherited from class org.apache.flink.api.java.operators.Operator

getMinResources, getName, getParallelism, getPreferredResources, getResultType, name, setParallelism

Methods inherited from class org.apache.flink.api.java.DataSet

aggregate, checkSameExecutionContext, clean, coGroup, collect, combineGroup, count, cross, crossWithHuge, crossWithTiny, distinct, distinct, distinct, distinct, fillInType, filter, first, flatMap, fullOuterJoin, fullOuterJoin, getExecutionEnvironment, getType, groupBy, groupBy, groupBy, iterate, iterateDelta, join, join, joinWithHuge, joinWithTiny, leftOuterJoin, leftOuterJoin, map, mapPartition, max, maxBy, min, minBy, output, partitionByHash, partitionByHash, partitionByHash, partitionByRange, partitionByRange, partitionByRange, partitionCustom, partitionCustom, partitionCustom, print, print, printOnTaskManager, printToErr, printToErr, project, rebalance, reduce, reduceGroup, rightOuterJoin, rightOuterJoin, runOperation, sortPartition, sortPartition, sortPartition, sum, union, write, write, writeAsCsv, writeAsCsv, writeAsCsv, writeAsCsv, writeAsFormattedText, writeAsFormattedText, writeAsText, writeAsText

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CoGroupOperator

public CoGroupOperator(DataSet<I1> input1,
                       DataSet<I2> input2,
                       org.apache.flink.api.common.operators.Keys<I1> keys1,
                       org.apache.flink.api.common.operators.Keys<I2> keys2,
                       org.apache.flink.api.common.functions.CoGroupFunction<I1,I2,OUT> function,
                       org.apache.flink.api.common.typeinfo.TypeInformation<OUT> returnType,
                       org.apache.flink.api.common.functions.Partitioner<?> customPartitioner,
                       String defaultName)

CoGroupOperator

public CoGroupOperator(DataSet<I1> input1,
                       DataSet<I2> input2,
                       org.apache.flink.api.common.operators.Keys<I1> keys1,
                       org.apache.flink.api.common.operators.Keys<I2> keys2,
                       org.apache.flink.api.common.functions.CoGroupFunction<I1,I2,OUT> function,
                       org.apache.flink.api.common.typeinfo.TypeInformation<OUT> returnType,
                       List<org.apache.commons.lang3.tuple.Pair<Integer,org.apache.flink.api.common.operators.Order>> groupSortKeyOrderFirst,
                       List<org.apache.commons.lang3.tuple.Pair<Integer,org.apache.flink.api.common.operators.Order>> groupSortKeyOrderSecond,
                       org.apache.flink.api.common.functions.Partitioner<?> customPartitioner,
                       String defaultName)

Method Detail

getFunction

protected org.apache.flink.api.common.functions.CoGroupFunction<I1,I2,OUT> getFunction()

Specified by:

getFunction in class TwoInputUdfOperator<I1,I2,OUT,CoGroupOperator<I1,I2,OUT>>

getSemanticProperties

public org.apache.flink.api.common.operators.DualInputSemanticProperties getSemanticProperties()

Description copied from interface: UdfOperator

Gets the semantic properties that have been set for the user-defined functions (UDF).

Specified by:

getSemanticProperties in interface UdfOperator<CoGroupOperator<I1,I2,OUT>>

Overrides:

getSemanticProperties in class TwoInputUdfOperator<I1,I2,OUT,CoGroupOperator<I1,I2,OUT>>

Returns:

The semantic properties of the UDF.

getKeys1

@Internal
protected org.apache.flink.api.common.operators.Keys<I1> getKeys1()

getKeys2

@Internal
protected org.apache.flink.api.common.operators.Keys<I2> getKeys2()

withPartitioner

public CoGroupOperator<I1,I2,OUT> withPartitioner(org.apache.flink.api.common.functions.Partitioner<?> partitioner)

Sets a custom partitioner for the CoGroup operation. The partitioner will be called on the join keys to determine the partition a key should be assigned to. The partitioner is evaluated on both inputs in the same way.

NOTE: A custom partitioner can only be used with single-field CoGroup keys, not with composite CoGroup keys.

Parameters:

partitioner - The custom partitioner to be used.

Returns:

This CoGroup operator, to allow for function chaining.

getPartitioner

public org.apache.flink.api.common.functions.Partitioner<?> getPartitioner()

Gets the custom partitioner used by this join, or null, if none is set.

Returns:

The custom partitioner used by this join;

translateToDataFlow

@Internal
protected org.apache.flink.api.common.operators.base.CoGroupOperatorBase<?,?,OUT,?> translateToDataFlow(org.apache.flink.api.common.operators.Operator<I1> input1,
                                                                                                                  org.apache.flink.api.common.operators.Operator<I2> input2)

Description copied from class: TwoInputOperator

Translates this java API operator into a common API operator with two inputs.

Specified by:

translateToDataFlow in class TwoInputOperator<I1,I2,OUT,CoGroupOperator<I1,I2,OUT>>

Parameters:

input1 - The first input of the operation, as a common API operator.

input2 - The second input of the operation, as a common API operator.

Returns:

The created common API operator.