org.apache.flink.api.java.operators

Class UnsortedGrouping<T>

java.lang.Object

org.apache.flink.api.java.operators.Grouping<T>

org.apache.flink.api.java.operators.UnsortedGrouping<T>

Type Parameters:

T - type of elements

@Public
public class UnsortedGrouping<T>
extends Grouping<T>

A Grouping that is unsorted.

Field Summary

Fields inherited from class org.apache.flink.api.java.operators.Grouping

customPartitioner, inputDataSet, keys

Constructor Summary

Constructors

Constructor and Description
UnsortedGrouping(DataSet<T> set, org.apache.flink.api.common.operators.Keys<T> keys)

Method Summary

Modifier and Type	Method and Description
AggregateOperator<T>	aggregate(Aggregations agg, int field) Applies an Aggregate transformation on a grouped Tuple DataSet.
<R> GroupCombineOperator<T,R>	combineGroup(org.apache.flink.api.common.functions.GroupCombineFunction<T,R> combiner) Applies a GroupCombineFunction on a grouped DataSet.
GroupReduceOperator<T,T>	first(int n) Returns a new set containing the first n elements in this grouped DataSet.
AggregateOperator<T>	max(int field) Syntactic sugar for aggregate (MAX, field).
ReduceOperator<T>	maxBy(int... fields) Applies a special case of a reduce transformation (maxBy) on a grouped DataSet.
AggregateOperator<T>	min(int field) Syntactic sugar for aggregate (MIN, field).
ReduceOperator<T>	minBy(int... fields) Applies a special case of a reduce transformation (minBy) on a grouped DataSet.
ReduceOperator<T>	reduce(org.apache.flink.api.common.functions.ReduceFunction<T> reducer) Applies a Reduce transformation on a grouped DataSet.
<R> GroupReduceOperator<T,R>	reduceGroup(org.apache.flink.api.common.functions.GroupReduceFunction<T,R> reducer) Applies a GroupReduce transformation on a grouped DataSet.
SortedGrouping<T>	sortGroup(int field, org.apache.flink.api.common.operators.Order order) Sorts Tuple elements within a group on the specified field in the specified Order.
<K> SortedGrouping<T>	sortGroup(org.apache.flink.api.java.functions.KeySelector<T,K> keySelector, org.apache.flink.api.common.operators.Order order) Sorts elements within a group on a key extracted by the specified KeySelector in the specified Order.
SortedGrouping<T>	sortGroup(String field, org.apache.flink.api.common.operators.Order order) Sorts Pojos within a group on the specified field in the specified Order.
AggregateOperator<T>	sum(int field) Syntactic sugar for aggregate (SUM, field).
UnsortedGrouping<T>	withPartitioner(org.apache.flink.api.common.functions.Partitioner<?> partitioner) Uses a custom partitioner for the grouping.

Methods inherited from class org.apache.flink.api.java.operators.Grouping

getCustomPartitioner, getInputDataSet, getKeys

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UnsortedGrouping

public UnsortedGrouping(DataSet<T> set,
                        org.apache.flink.api.common.operators.Keys<T> keys)

Method Detail

withPartitioner

public UnsortedGrouping<T> withPartitioner(org.apache.flink.api.common.functions.Partitioner<?> partitioner)

Uses a custom partitioner for the grouping.

Parameters:

partitioner - The custom partitioner.

Returns:

The grouping object itself, to allow for method chaining.

aggregate

public AggregateOperator<T> aggregate(Aggregations agg,
                                      int field)

Applies an Aggregate transformation on a grouped Tuple DataSet.

Note: Only Tuple DataSets can be aggregated. The transformation applies a built-in Aggregation on a specified field of a Tuple group. Additional aggregation functions can be added to the resulting AggregateOperator by calling AggregateOperator.and(Aggregations, int).

Parameters:

agg - The built-in aggregation function that is computed.

field - The index of the Tuple field on which the aggregation function is applied.

Returns:

An AggregateOperator that represents the aggregated DataSet.

See Also:

Tuple, Aggregations, AggregateOperator, DataSet

sum

public AggregateOperator<T> sum(int field)

Syntactic sugar for aggregate (SUM, field).

Parameters:

field - The index of the Tuple field on which the aggregation function is applied.

Returns:

An AggregateOperator that represents the summed DataSet.

See Also:

AggregateOperator

max

public AggregateOperator<T> max(int field)

Syntactic sugar for aggregate (MAX, field).

Parameters:

field - The index of the Tuple field on which the aggregation function is applied.

Returns:

An AggregateOperator that represents the max'ed DataSet.

See Also:

AggregateOperator

min

public AggregateOperator<T> min(int field)

Syntactic sugar for aggregate (MIN, field).

Parameters:

field - The index of the Tuple field on which the aggregation function is applied.

Returns:

An AggregateOperator that represents the min'ed DataSet.

See Also:

AggregateOperator

reduce

public ReduceOperator<T> reduce(org.apache.flink.api.common.functions.ReduceFunction<T> reducer)

Applies a Reduce transformation on a grouped DataSet.

For each group, the transformation consecutively calls a RichReduceFunction until only a single element for each group remains. A ReduceFunction combines two elements into one new element of the same type.

Parameters:

reducer - The ReduceFunction that is applied on each group of the DataSet.

Returns:

A ReduceOperator that represents the reduced DataSet.

See Also:

RichReduceFunction, ReduceOperator, DataSet

reduceGroup

public <R> GroupReduceOperator<T,R> reduceGroup(org.apache.flink.api.common.functions.GroupReduceFunction<T,R> reducer)

Applies a GroupReduce transformation on a grouped DataSet.

The transformation calls a RichGroupReduceFunction for each group of the DataSet. A GroupReduceFunction can iterate over all elements of a group and emit any number of output elements including none.

Parameters:

reducer - The GroupReduceFunction that is applied on each group of the DataSet.

Returns:

A GroupReduceOperator that represents the reduced DataSet.

See Also:

RichGroupReduceFunction, GroupReduceOperator, DataSet

combineGroup

public <R> GroupCombineOperator<T,R> combineGroup(org.apache.flink.api.common.functions.GroupCombineFunction<T,R> combiner)

Applies a GroupCombineFunction on a grouped DataSet. A GroupCombineFunction is similar to a GroupReduceFunction but does not perform a full data exchange. Instead, the CombineFunction calls the combine method once per partition for combining a group of results. This operator is suitable for combining values into an intermediate format before doing a proper groupReduce where the data is shuffled across the node for further reduction. The GroupReduce operator can also be supplied with a combiner by implementing the RichGroupReduce function. The combine method of the RichGroupReduce function demands input and output type to be the same. The CombineFunction, on the other side, can have an arbitrary output type.

Parameters:

combiner - The GroupCombineFunction that is applied on the DataSet.

Returns:

A GroupCombineOperator which represents the combined DataSet.

first

public GroupReduceOperator<T,T> first(int n)

Returns a new set containing the first n elements in this grouped DataSet.

Parameters:

n - The desired number of elements for each group.

Returns:

A GroupReduceOperator that represents the DataSet containing the elements.

minBy

public ReduceOperator<T> minBy(int... fields)

Applies a special case of a reduce transformation (minBy) on a grouped DataSet.

The transformation consecutively calls a ReduceFunction until only a single element remains which is the result of the transformation. A ReduceFunction combines two elements into one new element of the same type.

Parameters:

fields - Keys taken into account for finding the minimum.

Returns:

A ReduceOperator representing the minimum.

maxBy

public ReduceOperator<T> maxBy(int... fields)

Applies a special case of a reduce transformation (maxBy) on a grouped DataSet.

The transformation consecutively calls a ReduceFunction until only a single element remains which is the result of the transformation. A ReduceFunction combines two elements into one new element of the same type.

Parameters:

fields - Keys taken into account for finding the minimum.

Returns:

A ReduceOperator representing the minimum.

sortGroup

public SortedGrouping<T> sortGroup(int field,
                                   org.apache.flink.api.common.operators.Order order)

Sorts Tuple elements within a group on the specified field in the specified Order.

Note: Only groups of Tuple elements and Pojos can be sorted.

Groups can be sorted by multiple fields by chaining sortGroup(int, Order) calls.

Parameters:

field - The Tuple field on which the group is sorted.

order - The Order in which the specified Tuple field is sorted.

Returns:

A SortedGrouping with specified order of group element.

See Also:

Tuple, Order

sortGroup

public SortedGrouping<T> sortGroup(String field,
                                   org.apache.flink.api.common.operators.Order order)

Sorts Pojos within a group on the specified field in the specified Order.

Note: Only groups of Tuple elements and Pojos can be sorted.

Groups can be sorted by multiple fields by chaining sortGroup(String, Order) calls.

Parameters:

field - The Tuple or Pojo field on which the group is sorted.

order - The Order in which the specified field is sorted.

Returns:

A SortedGrouping with specified order of group element.

See Also:

Order

sortGroup

public <K> SortedGrouping<T> sortGroup(org.apache.flink.api.java.functions.KeySelector<T,K> keySelector,
                                       org.apache.flink.api.common.operators.Order order)

Sorts elements within a group on a key extracted by the specified KeySelector in the specified Order.

Chaining sortGroup(KeySelector, Order) calls is not supported.

Parameters:

keySelector - The KeySelector with which the group is sorted.

order - The Order in which the extracted key is sorted.

Returns:

A SortedGrouping with specified order of group element.

See Also:

Order