org.apache.calcite.sql2rel

Class RelDecorrelator

java.lang.Object

org.apache.calcite.sql2rel.RelDecorrelator

All Implemented Interfaces:

ReflectiveVisitor

public class RelDecorrelator
extends Object
implements ReflectiveVisitor

RelDecorrelator replaces all correlated expressions (corExp) in a relational expression (RelNode) tree with non-correlated expressions that are produced from joining the RelNode that produces the corExp with the RelNode that references it.

TODO:

• replace CorelMap constructor parameter with a RelNode
• make currentRel immutable (would require a fresh RelDecorrelator for each node being decorrelated)
• make fields of CorelMap immutable
• make sub-class rules static, and have them create their own de-correlator

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private class	RelDecorrelator.AdjustProjectForCountAggregateRule Planner rule that adjusts projects when counts are added.
(package private) static class	RelDecorrelator.CorDef A correlation and a field.
private static class	RelDecorrelator.CorelMap A map of the locations of LogicalCorrelate in a tree of RelNodes.
private static class	RelDecorrelator.CorelMapBuilder Builds a RelDecorrelator.CorelMap.
(package private) static class	RelDecorrelator.CorRef A unique reference to a correlation field.
private static class	RelDecorrelator.DecorrelateRexShuttle Shuttle that decorrelates.
(package private) static class	RelDecorrelator.Frame Frame describing the relational expression after decorrelation and where to find the output fields and correlation variables among its output fields.
private class	RelDecorrelator.RemoveCorrelationForScalarAggregateRule Planner rule that removes correlations for scalar aggregates.
private class	RelDecorrelator.RemoveCorrelationForScalarProjectRule Planner rule that removes correlations for scalar projects.
private class	RelDecorrelator.RemoveCorrelationRexShuttle Shuttle that removes correlations.
private class	RelDecorrelator.RemoveSingleAggregateRule Rule to remove single_value rel.

Field Summary

Fields

Modifier and Type	Field and Description
private RelDecorrelator.CorelMap	cm
private Context	context
private RelNode	currentRel
private ReflectUtil.MethodDispatcher<RelDecorrelator.Frame>	dispatcher
private HashSet<LogicalCorrelate>	generatedCorRels
private Map<RelNode,RelDecorrelator.Frame>	map Built during decorrelation, of rel to all the newly created correlated variables in its output, and to map old input positions to new input positions.
private RelBuilder	relBuilder
private RexBuilder	rexBuilder
private static org.slf4j.Logger	SQL2REL_LOGGER

Constructor Summary

Constructors

Modifier	Constructor and Description
private	RelDecorrelator(RelOptCluster cluster, RelDecorrelator.CorelMap cm, Context context)

Method Summary

Modifier and Type	Method and Description
private RelNode	aggregateCorrelatorOutput(Correlate correlate, LogicalProject project, Set<Integer> isCount) Pulls a Project above a Correlate from its RHS input.
(package private) static boolean	allLessThan(Collection<Integer> integers, int limit, Litmus ret)
private boolean	checkCorVars(LogicalCorrelate correlate, LogicalProject project, LogicalFilter filter, List<RexFieldAccess> correlatedJoinKeys) Checks whether the correlations in projRel and filter are related to the correlated variables provided by corRel.
private Function2<RelNode,RelNode,Void>	createCopyHook()
private HepPlanner	createPlanner(HepProgram program)
private RelNode	createProjectWithAdditionalExprs(RelNode input, List<Pair<RexNode,String>> additionalExprs) Projects all input output fields plus the additional expressions.
private RelNode	createValueGenerator(Iterable<RelDecorrelator.CorRef> correlations, int valueGenFieldOffset, SortedMap<RelDecorrelator.CorDef,Integer> corDefOutputs) Create RelNode tree that produces a list of correlated variables.
private RelNode	decorrelate(RelNode root)
protected RexNode	decorrelateExpr(RelNode currentRel, Map<RelNode,RelDecorrelator.Frame> map, RelDecorrelator.CorelMap cm, RexNode exp)
private RelDecorrelator.Frame	decorrelateInputWithValueGenerator(RelNode rel, RelDecorrelator.Frame frame)
static RelNode	decorrelateQuery(RelNode rootRel) Decorrelates a query.
RelDecorrelator.Frame	decorrelateRel(LogicalAggregate rel) Rewrites a LogicalAggregate.
RelDecorrelator.Frame	decorrelateRel(LogicalCorrelate rel) Rewrite Correlate into a left outer join.
RelDecorrelator.Frame	decorrelateRel(LogicalFilter rel) Rewrite LogicalFilter.
RelDecorrelator.Frame	decorrelateRel(LogicalJoin rel) Rewrite LogicalJoin.
RelDecorrelator.Frame	decorrelateRel(LogicalProject rel) Rewrite LogicalProject.
RelDecorrelator.Frame	decorrelateRel(RelNode rel) Fallback if none of the other decorrelateRel methods match.
RelDecorrelator.Frame	decorrelateRel(Sort rel) Rewrite Sort.
RelDecorrelator.Frame	decorrelateRel(Values rel) Rewrites a Values.
private void	findCorrelationEquivalent(RelDecorrelator.CorRef correlation, RexNode e) Finds a RexInputRef that is equivalent to a RelDecorrelator.CorRef, and if found, throws a Util.FoundOne.
private RelNode	getCorRel(RelDecorrelator.CorRef corVar)
private RelDecorrelator.Frame	getFrame(RelNode r, boolean safe)
RelDecorrelator.Frame	getInvoke(RelNode r, RelNode parent)
private static RexInputRef	getNewForOldInputRef(RelNode currentRel, Map<RelNode,RelDecorrelator.Frame> map, RexInputRef oldInputRef)
private boolean	has(Collection<RelDecorrelator.CorDef> corDefs, RelDecorrelator.CorRef corr) Returns whether a CorrelationId is satisfied by at least one of a collection of RelDecorrelator.CorDefs.
private boolean	hasAll(Collection<RelDecorrelator.CorRef> corRefs, Collection<RelDecorrelator.CorDef> corDefs) Returns whether all of a collection of RelDecorrelator.CorRefs are satisfied by at least one of a collection of RelDecorrelator.CorDefs.
(package private) static Map<Integer,Integer>	identityMap(int count)
private boolean	isWidening(RelDataType type, RelDataType type1) Returns whether one type is just a widening of another.
private RelDecorrelator.Frame	maybeAddValueGenerator(RelNode rel, RelDecorrelator.Frame frame) Adds a value generator to satisfy the correlating variables used by a relational expression, if those variables are not already provided by its input.
private static RexLiteral	projectedLiteral(RelNode rel, int i) Returns a literal output field, or null if it is not literal.
private RelNode	projectJoinOutputWithNullability(LogicalJoin join, LogicalProject project, int nullIndicatorPos) Pulls project above the join from its RHS input.
private boolean	references(RexNode e, RelDecorrelator.CorRef correlation)
(package private) RelDecorrelator.Frame	register(RelNode rel, RelNode newRel, Map<Integer,Integer> oldToNewOutputs, SortedMap<RelDecorrelator.CorDef,Integer> corDefOutputs) Registers a relational expression and the relational expression it became after decorrelation.
protected RexNode	removeCorrelationExpr(RexNode exp, boolean projectPulledAboveLeftCorrelator)
protected RexNode	removeCorrelationExpr(RexNode exp, boolean projectPulledAboveLeftCorrelator, RexInputRef nullIndicator)
protected RexNode	removeCorrelationExpr(RexNode exp, boolean projectPulledAboveLeftCorrelator, Set<Integer> isCount)
RelNode	removeCorrelationViaRule(RelNode root)
private void	removeCorVarFromTree(LogicalCorrelate correlate) Remove correlated variables from the tree at root corRel
private void	setCurrent(RelNode root, LogicalCorrelate corRel)
private static RelNode	stripHep(RelNode rel)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SQL2REL_LOGGER

private static final org.slf4j.Logger SQL2REL_LOGGER

relBuilder

private final RelBuilder relBuilder

cm

private RelDecorrelator.CorelMap cm

dispatcher

private final ReflectUtil.MethodDispatcher<RelDecorrelator.Frame> dispatcher

rexBuilder

private final RexBuilder rexBuilder

currentRel

private RelNode currentRel

context

private final Context context

map

private final Map<RelNode,RelDecorrelator.Frame> map

Built during decorrelation, of rel to all the newly created correlated variables in its output, and to map old input positions to new input positions. This is from the view point of the parent rel of a new rel.

generatedCorRels

private final HashSet<LogicalCorrelate> generatedCorRels

Constructor Detail

RelDecorrelator

private RelDecorrelator(RelOptCluster cluster,
                        RelDecorrelator.CorelMap cm,
                        Context context)

Method Detail

decorrelateQuery

public static RelNode decorrelateQuery(RelNode rootRel)

Decorrelates a query.

This is the main entry point to RelDecorrelator.

Parameters:

rootRel - Root node of the query

Returns:

Equivalent query with all LogicalCorrelate instances removed

setCurrent

private void setCurrent(RelNode root,
                        LogicalCorrelate corRel)

decorrelate

private RelNode decorrelate(RelNode root)

createCopyHook

private Function2<RelNode,RelNode,Void> createCopyHook()

createPlanner

private HepPlanner createPlanner(HepProgram program)

removeCorrelationViaRule

public RelNode removeCorrelationViaRule(RelNode root)

decorrelateExpr

protected RexNode decorrelateExpr(RelNode currentRel,
                                  Map<RelNode,RelDecorrelator.Frame> map,
                                  RelDecorrelator.CorelMap cm,
                                  RexNode exp)

removeCorrelationExpr

protected RexNode removeCorrelationExpr(RexNode exp,
                                        boolean projectPulledAboveLeftCorrelator)

removeCorrelationExpr

protected RexNode removeCorrelationExpr(RexNode exp,
                                        boolean projectPulledAboveLeftCorrelator,
                                        RexInputRef nullIndicator)

removeCorrelationExpr

protected RexNode removeCorrelationExpr(RexNode exp,
                                        boolean projectPulledAboveLeftCorrelator,
                                        Set<Integer> isCount)

decorrelateRel

public RelDecorrelator.Frame decorrelateRel(RelNode rel)

Fallback if none of the other decorrelateRel methods match.

decorrelateRel

public RelDecorrelator.Frame decorrelateRel(Sort rel)

Rewrite Sort.

Parameters:

rel - Sort to be rewritten

decorrelateRel

public RelDecorrelator.Frame decorrelateRel(Values rel)

Rewrites a Values.

Parameters:

rel - Values to be rewritten

decorrelateRel

public RelDecorrelator.Frame decorrelateRel(LogicalAggregate rel)

Rewrites a LogicalAggregate.

Parameters:

rel - Aggregate to rewrite

getInvoke

public RelDecorrelator.Frame getInvoke(RelNode r,
                                       RelNode parent)

projectedLiteral

private static RexLiteral projectedLiteral(RelNode rel,
                                           int i)

Returns a literal output field, or null if it is not literal.

decorrelateRel

public RelDecorrelator.Frame decorrelateRel(LogicalProject rel)

Rewrite LogicalProject.

Parameters:

rel - the project rel to rewrite

createValueGenerator

private RelNode createValueGenerator(Iterable<RelDecorrelator.CorRef> correlations,
                                     int valueGenFieldOffset,
                                     SortedMap<RelDecorrelator.CorDef,Integer> corDefOutputs)

Create RelNode tree that produces a list of correlated variables.

Parameters:

correlations - correlated variables to generate

valueGenFieldOffset - offset in the output that generated columns will start

corDefOutputs - output positions for the correlated variables generated

Returns:

RelNode the root of the resultant RelNode tree

getFrame

private RelDecorrelator.Frame getFrame(RelNode r,
                                       boolean safe)

getCorRel

private RelNode getCorRel(RelDecorrelator.CorRef corVar)

maybeAddValueGenerator

private RelDecorrelator.Frame maybeAddValueGenerator(RelNode rel,
                                                     RelDecorrelator.Frame frame)

Adds a value generator to satisfy the correlating variables used by a relational expression, if those variables are not already provided by its input.

hasAll

private boolean hasAll(Collection<RelDecorrelator.CorRef> corRefs,
                       Collection<RelDecorrelator.CorDef> corDefs)

Returns whether all of a collection of RelDecorrelator.CorRefs are satisfied by at least one of a collection of RelDecorrelator.CorDefs.

has

private boolean has(Collection<RelDecorrelator.CorDef> corDefs,
                    RelDecorrelator.CorRef corr)

Returns whether a CorrelationId is satisfied by at least one of a collection of RelDecorrelator.CorDefs.

decorrelateInputWithValueGenerator

private RelDecorrelator.Frame decorrelateInputWithValueGenerator(RelNode rel,
                                                                 RelDecorrelator.Frame frame)

findCorrelationEquivalent

private void findCorrelationEquivalent(RelDecorrelator.CorRef correlation,
                                       RexNode e)
                                throws Util.FoundOne

Finds a RexInputRef that is equivalent to a RelDecorrelator.CorRef, and if found, throws a Util.FoundOne.

Throws:

Util.FoundOne

references

private boolean references(RexNode e,
                           RelDecorrelator.CorRef correlation)

isWidening

private boolean isWidening(RelDataType type,
                           RelDataType type1)

Returns whether one type is just a widening of another.

For example:

• VARCHAR(10) is a widening of VARCHAR(5).
• VARCHAR(10) is a widening of VARCHAR(10) NOT NULL.

decorrelateRel

public RelDecorrelator.Frame decorrelateRel(LogicalFilter rel)

Rewrite LogicalFilter.

Parameters:

rel - the filter rel to rewrite

decorrelateRel

public RelDecorrelator.Frame decorrelateRel(LogicalCorrelate rel)

Rewrite Correlate into a left outer join.

Parameters:

rel - Correlator

decorrelateRel

public RelDecorrelator.Frame decorrelateRel(LogicalJoin rel)

Rewrite LogicalJoin.

Parameters:

rel - Join

getNewForOldInputRef

private static RexInputRef getNewForOldInputRef(RelNode currentRel,
                                                Map<RelNode,RelDecorrelator.Frame> map,
                                                RexInputRef oldInputRef)

projectJoinOutputWithNullability

private RelNode projectJoinOutputWithNullability(LogicalJoin join,
                                                 LogicalProject project,
                                                 int nullIndicatorPos)

Pulls project above the join from its RHS input. Enforces nullability for join output.

Parameters:

join - Join

project - Original project as the right-hand input of the join

nullIndicatorPos - Position of null indicator

Returns:

the subtree with the new Project at the root

aggregateCorrelatorOutput

private RelNode aggregateCorrelatorOutput(Correlate correlate,
                                          LogicalProject project,
                                          Set<Integer> isCount)

Pulls a Project above a Correlate from its RHS input. Enforces nullability for join output.

Parameters:

correlate - Correlate

project - the original project as the RHS input of the join

isCount - Positions which are calls to the COUNT aggregation function

Returns:

the subtree with the new Project at the root

checkCorVars

private boolean checkCorVars(LogicalCorrelate correlate,
                             LogicalProject project,
                             LogicalFilter filter,
                             List<RexFieldAccess> correlatedJoinKeys)

Checks whether the correlations in projRel and filter are related to the correlated variables provided by corRel.

Parameters:

correlate - Correlate

project - The original Project as the RHS input of the join

filter - Filter

correlatedJoinKeys - Correlated join keys

Returns:

true if filter and proj only references corVar provided by corRel

removeCorVarFromTree

private void removeCorVarFromTree(LogicalCorrelate correlate)

Remove correlated variables from the tree at root corRel

Parameters:

correlate - Correlate

createProjectWithAdditionalExprs

private RelNode createProjectWithAdditionalExprs(RelNode input,
                                                 List<Pair<RexNode,String>> additionalExprs)

Projects all input output fields plus the additional expressions.

Parameters:

input - Input relational expression

additionalExprs - Additional expressions and names

Returns:

the new Project

identityMap

static Map<Integer,Integer> identityMap(int count)

register

RelDecorrelator.Frame register(RelNode rel,
                               RelNode newRel,
                               Map<Integer,Integer> oldToNewOutputs,
                               SortedMap<RelDecorrelator.CorDef,Integer> corDefOutputs)

Registers a relational expression and the relational expression it became after decorrelation.

allLessThan

static boolean allLessThan(Collection<Integer> integers,
                           int limit,
                           Litmus ret)

stripHep

private static RelNode stripHep(RelNode rel)