org.apache.calcite.plan

Class SubstitutionVisitor

java.lang.Object

org.apache.calcite.plan.SubstitutionVisitor

Direct Known Subclasses:

MaterializedViewSubstitutionVisitor

public class SubstitutionVisitor
extends Object

Substitutes part of a tree of relational expressions with another tree.

The call new SubstitutionVisitor(target, query).go(replacement)) will return query with every occurrence of target replaced by replacement.

The following example shows how SubstitutionVisitor can be used for materialized view recognition.

• query = SELECT a, c FROM t WHERE x = 5 AND b = 4
• target = SELECT a, b, c FROM t WHERE x = 5
• replacement = SELECT * FROM mv
• result = SELECT a, c FROM mv WHERE b = 4

Note that result uses the materialized view table mv and a simplified condition b = 4.

Uses a bottom-up matching algorithm. Nodes do not need to be identical. At each level, returns the residue.

The inputs must only include the core relational operators: LogicalTableScan, LogicalFilter, LogicalProject, LogicalJoin, LogicalUnion, LogicalAggregate.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	SubstitutionVisitor.AbstractUnifyRule Abstract base class for implementing SubstitutionVisitor.UnifyRule.
private static class	SubstitutionVisitor.AggregateOnProjectToAggregateUnifyRule Implementation of SubstitutionVisitor.UnifyRule that matches a MutableAggregate on a MutableProject query to an MutableAggregate target.
private static class	SubstitutionVisitor.AggregateToAggregateUnifyRule Implementation of SubstitutionVisitor.UnifyRule that matches a LogicalAggregate to a LogicalAggregate, provided that they have the same child.
private static class	SubstitutionVisitor.AnyOperand Operand to a SubstitutionVisitor.UnifyRule that matches a relational expression of a given type.
static class	SubstitutionVisitor.FilterOnProjectRule Rule that converts a LogicalFilter on top of a LogicalProject into a trivial filter (on a boolean column).
private static class	SubstitutionVisitor.FilterToFilterUnifyRule Implementation of SubstitutionVisitor.UnifyRule that matches a MutableFilter.
private static class	SubstitutionVisitor.FilterToProjectUnifyRule Implementation of SubstitutionVisitor.UnifyRule that matches a MutableFilter to a MutableProject.
private static class	SubstitutionVisitor.InternalOperand Operand to a SubstitutionVisitor.UnifyRule that matches a relational expression of a given type.
protected static class	SubstitutionVisitor.MatchFailed Exception thrown to exit a matcher.
protected static class	SubstitutionVisitor.Operand Operand to a SubstitutionVisitor.UnifyRule.
private static class	SubstitutionVisitor.ProjectToFilterUnifyRule Implementation of SubstitutionVisitor.UnifyRule that matches a MutableProject to a MutableFilter.
private static class	SubstitutionVisitor.ProjectToProjectUnifyRule Implementation of SubstitutionVisitor.UnifyRule that matches LogicalProject.
private static class	SubstitutionVisitor.QueryOperand Operand that assigns a particular relational expression to a variable.
(package private) static class	SubstitutionVisitor.Replacement Represents a replacement action: before → after.
private static class	SubstitutionVisitor.ScanToProjectUnifyRule Implementation of SubstitutionVisitor.UnifyRule that matches LogicalTableScan.
private static class	SubstitutionVisitor.SlotCounter Visitor that counts how many SubstitutionVisitor.QueryOperand and SubstitutionVisitor.TargetOperand in an operand tree.
private static class	SubstitutionVisitor.TargetOperand Operand that checks that a relational expression matches the corresponding relational expression that was passed to a SubstitutionVisitor.QueryOperand.
private static class	SubstitutionVisitor.TrivialRule Implementation of SubstitutionVisitor.UnifyRule that matches if the query is already equal to the target.
protected static class	SubstitutionVisitor.UnifyResult Result of an application of a SubstitutionVisitor.UnifyRule indicating that the rule successfully matched query against target and generated a result that is equivalent to query and contains target.
protected static class	SubstitutionVisitor.UnifyRule Rule that attempts to match a query relational expression against a target relational expression.
protected class	SubstitutionVisitor.UnifyRuleCall Arguments to an application of a SubstitutionVisitor.UnifyRule.

Field Summary

Fields

Modifier and Type	Field and Description
private RelOptCluster	cluster
private static boolean	DEBUG
protected static com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule>	DEFAULT_RULES
(package private) com.google.common.collect.Multimap<MutableRel,MutableRel>	equivalents
private static org.slf4j.Logger	LOGGER
private Holder	query
(package private) List<MutableRel>	queryLeaves Nodes in query that have no children.
protected RelBuilder	relBuilder Factory for a builder for relational expressions.
(package private) Map<MutableRel,MutableRel>	replacementMap
private Map<Pair<Class,Class>,List<SubstitutionVisitor.UnifyRule>>	ruleMap
private com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule>	rules
private RexSimplify	simplify
protected MutableRel[]	slots Workspace while rule is being matched.
private MutableRel	target
(package private) List<MutableRel>	targetLeaves Nodes in target that have no children.

Constructor Summary

Constructors

Constructor and Description
SubstitutionVisitor(RelNode target_, RelNode query_) Creates a SubstitutionVisitor with the default rule set.
SubstitutionVisitor(RelNode target_, RelNode query_, com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule> rules) Creates a SubstitutionVisitor with the default logical builder.
SubstitutionVisitor(RelNode target_, RelNode query_, com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule> rules, RelBuilderFactory relBuilderFactory)

Method Summary

Modifier and Type	Method and Description
private List<SubstitutionVisitor.UnifyRule>	applicableRules(MutableRel query, MutableRel target)
private static List<AggregateCall>	apply(Mapping mapping, List<AggregateCall> aggCallList)
private SubstitutionVisitor.UnifyResult	apply(SubstitutionVisitor.UnifyRule rule, MutableRel query, MutableRel target)
private static RexNode	canonizeNode(RexBuilder rexBuilder, RexNode condition) Reorders some of the operands in this expression so structural comparison, i.e., based on string representation, can be more precise.
static boolean	equalType(String desc0, MutableRel rel0, String desc1, MutableRel rel1, Litmus litmus) Returns whether two relational expressions have the same row-type.
protected static RexShuttle	getRexShuttle(MutableProject target) Builds a shuttle that stores a list of expressions, and can map incoming expressions to references to them.
static SqlAggFunction	getRollup(SqlAggFunction aggregation)
private List<List<SubstitutionVisitor.Replacement>>	go(MutableRel replacement) Substitutes the query with replacement whenever possible but meanwhile keeps track of all the substitutions and their original rel before replacement, so that in later processing stage, the replacement can be recovered individually to produce a list of all possible rels with substitution in different places.
List<RelNode>	go(RelNode replacement_) Returns a list of all possible rels that result from substituting the matched RelNode with the replacement RelNode within the query.
RelNode	go0(RelNode replacement_)
private static boolean	isEquivalent(RexBuilder rexBuilder, RexNode condition, RexNode target)
protected boolean	isWeaker(MutableRel rel0, MutableRel rel) Returns if one rel is weaker than another.
private SubstitutionVisitor.UnifyResult	matchRecurse(MutableRel target)
static boolean	mayBeSatisfiable(RexNode e) Returns whether a boolean expression ever returns true.
private static boolean	mightMatch(SubstitutionVisitor.UnifyRule rule, Class queryClass, Class targetClass)
static MutableAggregate	permute(MutableAggregate aggregate, MutableRel input, Mapping mapping)
private static void	redoReplacement(List<SubstitutionVisitor.Replacement> replacement)
(package private) void	register(MutableRel result, MutableRel query)
static SubstitutionVisitor.Replacement	replace(MutableRel query, MutableRel find, MutableRel replace) Within a relational expression query, replaces occurrences of find with replace.
private static SubstitutionVisitor.Replacement	replaceRecurse(MutableRel query, MutableRel find, MutableRel replace) Helper for replace(org.apache.calcite.rel.mutable.MutableRel, org.apache.calcite.rel.mutable.MutableRel, org.apache.calcite.rel.mutable.MutableRel).
private static void	reverseSubstitute(RelBuilder relBuilder, Holder query, List<List<SubstitutionVisitor.Replacement>> matches, List<RelNode> sub, int replaceCount, int maxCount)
static RexNode	splitFilter(RexSimplify simplify, RexNode condition, RexNode target) Maps a condition onto a target.
private static RexNode	splitOr(RexBuilder rexBuilder, RexNode condition, RexNode target)
private static void	undoReplacement(List<SubstitutionVisitor.Replacement> replacement)
static MutableRel	unifyAggregates(MutableAggregate query, MutableAggregate target)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEBUG

private static final boolean DEBUG

LOGGER

private static final org.slf4j.Logger LOGGER

DEFAULT_RULES

protected static final com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule> DEFAULT_RULES

relBuilder

protected final RelBuilder relBuilder

Factory for a builder for relational expressions.

rules

private final com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule> rules

ruleMap

private final Map<Pair<Class,Class>,List<SubstitutionVisitor.UnifyRule>> ruleMap

cluster

private final RelOptCluster cluster

simplify

private final RexSimplify simplify

query

private final Holder query

target

private final MutableRel target

targetLeaves

final List<MutableRel> targetLeaves

Nodes in target that have no children.

queryLeaves

final List<MutableRel> queryLeaves

Nodes in query that have no children.

replacementMap

final Map<MutableRel,MutableRel> replacementMap

equivalents

final com.google.common.collect.Multimap<MutableRel,MutableRel> equivalents

slots

protected final MutableRel[] slots

Workspace while rule is being matched. Careful, re-entrant! Assumes no rule needs more than 2 slots.

Constructor Detail

SubstitutionVisitor

public SubstitutionVisitor(RelNode target_,
                           RelNode query_)

Creates a SubstitutionVisitor with the default rule set.

SubstitutionVisitor

public SubstitutionVisitor(RelNode target_,
                           RelNode query_,
                           com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule> rules)

Creates a SubstitutionVisitor with the default logical builder.

SubstitutionVisitor

public SubstitutionVisitor(RelNode target_,
                           RelNode query_,
                           com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule> rules,
                           RelBuilderFactory relBuilderFactory)

Method Detail

register

void register(MutableRel result,
              MutableRel query)

splitFilter

public static RexNode splitFilter(RexSimplify simplify,
                                  RexNode condition,
                                  RexNode target)

Maps a condition onto a target.

If condition is stronger than target, returns the residue. If it is equal to target, returns the expression that evaluates to the constant true. If it is weaker than target, returns null.

The terms satisfy the relation

 condition = target AND residue

and residue must be as weak as possible.

Example #1: condition stronger than target

• condition: x = 1 AND y = 2
• target: x = 1
• residue: y = 2

Note that residue x > 0 AND y = 2 would also satisfy the relation condition = target AND residue but is stronger than necessary, so we prefer y = 2.

Example #2: target weaker than condition (valid, but not currently implemented)

• condition: x = 1
• target: x = 1 OR z = 3
• residue: x = 1

Example #3: condition and target are equivalent

• condition: x = 1 AND y = 2
• target: y = 2 AND x = 1
• residue: TRUE

Example #4: condition weaker than target

• condition: x = 1
• target: x = 1 AND y = 2
• residue: null (i.e. no match)

There are many other possible examples. It amounts to solving whether condition AND NOT target can ever evaluate to true, and therefore is a form of the NP-complete Satisfiability problem.

canonizeNode

private static RexNode canonizeNode(RexBuilder rexBuilder,
                                    RexNode condition)

Reorders some of the operands in this expression so structural comparison, i.e., based on string representation, can be more precise.

splitOr

private static RexNode splitOr(RexBuilder rexBuilder,
                               RexNode condition,
                               RexNode target)

isEquivalent

private static boolean isEquivalent(RexBuilder rexBuilder,
                                    RexNode condition,
                                    RexNode target)

mayBeSatisfiable

public static boolean mayBeSatisfiable(RexNode e)

Returns whether a boolean expression ever returns true.

This method may give false positives. For instance, it will say that x = 5 AND x > 10 is satisfiable, because at present it cannot prove that it is not.

go0

public RelNode go0(RelNode replacement_)

go

public List<RelNode> go(RelNode replacement_)

Returns a list of all possible rels that result from substituting the matched RelNode with the replacement RelNode within the query.

For example, the substitution result of A join B, while A and B are both a qualified match for replacement R, is R join B, R join R, A join R.

go

private List<List<SubstitutionVisitor.Replacement>> go(MutableRel replacement)

Substitutes the query with replacement whenever possible but meanwhile keeps track of all the substitutions and their original rel before replacement, so that in later processing stage, the replacement can be recovered individually to produce a list of all possible rels with substitution in different places.

replace

public static SubstitutionVisitor.Replacement replace(MutableRel query,
                                                      MutableRel find,
                                                      MutableRel replace)

Within a relational expression query, replaces occurrences of find with replace.

Assumes relational expressions (and their descendants) are not null. Does not handle cycles.

replaceRecurse

private static SubstitutionVisitor.Replacement replaceRecurse(MutableRel query,
                                                              MutableRel find,
                                                              MutableRel replace)

Helper for replace(org.apache.calcite.rel.mutable.MutableRel, org.apache.calcite.rel.mutable.MutableRel, org.apache.calcite.rel.mutable.MutableRel).

undoReplacement

private static void undoReplacement(List<SubstitutionVisitor.Replacement> replacement)

redoReplacement

private static void redoReplacement(List<SubstitutionVisitor.Replacement> replacement)

reverseSubstitute

private static void reverseSubstitute(RelBuilder relBuilder,
                                      Holder query,
                                      List<List<SubstitutionVisitor.Replacement>> matches,
                                      List<RelNode> sub,
                                      int replaceCount,
                                      int maxCount)

matchRecurse

private SubstitutionVisitor.UnifyResult matchRecurse(MutableRel target)

apply

private SubstitutionVisitor.UnifyResult apply(SubstitutionVisitor.UnifyRule rule,
                                              MutableRel query,
                                              MutableRel target)

applicableRules

private List<SubstitutionVisitor.UnifyRule> applicableRules(MutableRel query,
                                                            MutableRel target)

mightMatch

private static boolean mightMatch(SubstitutionVisitor.UnifyRule rule,
                                  Class queryClass,
                                  Class targetClass)

permute

public static MutableAggregate permute(MutableAggregate aggregate,
                                       MutableRel input,
                                       Mapping mapping)

apply

private static List<AggregateCall> apply(Mapping mapping,
                                         List<AggregateCall> aggCallList)

unifyAggregates

public static MutableRel unifyAggregates(MutableAggregate query,
                                         MutableAggregate target)

getRollup

public static SqlAggFunction getRollup(SqlAggFunction aggregation)

getRexShuttle

protected static RexShuttle getRexShuttle(MutableProject target)

Builds a shuttle that stores a list of expressions, and can map incoming expressions to references to them.

isWeaker

protected boolean isWeaker(MutableRel rel0,
                           MutableRel rel)

Returns if one rel is weaker than another.

equalType

public static boolean equalType(String desc0,
                                MutableRel rel0,
                                String desc1,
                                MutableRel rel1,
                                Litmus litmus)

Returns whether two relational expressions have the same row-type.