public class VolcanoPlanner extends AbstractRelOptPlanner
| Modifier and Type | Class and Description |
|---|---|
private static class |
VolcanoPlanner.DeferringRuleCall
A rule call which defers its actions.
|
(package private) static class |
VolcanoPlanner.DirectProvenance
A RelNode that came directly from another RelNode via a copy.
|
private static class |
VolcanoPlanner.Provenance
Where a RelNode came from.
|
(package private) static class |
VolcanoPlanner.RuleProvenance
A RelNode that came via the firing of a rule.
|
private static class |
VolcanoPlanner.UnknownProvenance
We do not know where this RelNode came from.
|
RelOptPlanner.CannotPlanException, RelOptPlanner.Executor| Modifier and Type | Field and Description |
|---|---|
(package private) List<RelSet> |
allSets
List of all sets.
|
protected boolean |
ambitious
If true, the planner keeps applying rules as long as they continue to
reduce the cost.
|
private com.google.common.collect.Multimap<Class<? extends RelNode>,RelOptRuleOperand> |
classOperands
Operands that apply to a given class of
RelNode. |
protected static double |
COST_IMPROVEMENT |
protected boolean |
impatient
If true, and if
ambitious is true, the planner waits a finite
number of iterations for the cost to improve. |
private Map<List<String>,RelOptLattice> |
latticeByName
Map of lattices by the qualified name of their star table.
|
(package private) RelOptListener |
listener
Listener for this planner, or null if none set.
|
private boolean |
locked
Whether the planner can accept new rules.
|
private Map<Pair<String,RelDataType>,RelNode> |
mapDigestToRel
Canonical map from
digest to the unique
relational expression with that digest. |
private IdentityHashMap<RelNode,RelSubset> |
mapRel2Subset
|
private List<RelOptMaterialization> |
materializations |
private int |
nextSetId |
private RelNode |
originalRoot |
private String |
originalRootString
Dump of the root relational expression, as it was before any rules were
applied.
|
(package private) Map<RelNode,VolcanoPlanner.Provenance> |
provenanceMap |
private int |
registerCount
Incremented every time a relational expression is registered or two sets
are merged.
|
private Set<RelOptSchema> |
registeredSchemas
List of all schemas which have been registered.
|
(package private) Map<RelNode,Double> |
relImportances
The importance of relational expressions.
|
protected RelSubset |
root |
private Deque<VolcanoRuleCall> |
ruleCallStack |
private com.google.common.collect.SetMultimap<String,Class> |
ruleNames
Maps rule classes to their name, to ensure that the names are unique and
conform to rules.
|
(package private) RuleQueue |
ruleQueue
Holds rule calls waiting to be fired.
|
protected Set<RelOptRule> |
ruleSet
Set of all registered rules.
|
private List<RelTraitDef> |
traitDefs
Holds the currently registered RelTraitDefs.
|
private RelOptCost |
zeroCost
Zero cost, according to
AbstractRelOptPlanner.costFactory. |
context, costFactoryLOGGER| Constructor and Description |
|---|
VolcanoPlanner()
Creates a uninitialized
VolcanoPlanner. |
VolcanoPlanner(Context externalContext)
Creates a uninitialized
VolcanoPlanner. |
VolcanoPlanner(RelOptCostFactory costFactory,
Context externalContext)
Creates a
VolcanoPlanner with a given cost factory. |
| Modifier and Type | Method and Description |
|---|---|
void |
addLattice(RelOptLattice lattice)
Defines a lattice.
|
void |
addListener(RelOptListener newListener)
Adds a listener to this planner.
|
void |
addMaterialization(RelOptMaterialization materialization)
Defines a pair of relational expressions that are equivalent.
|
private RelSubset |
addRelToSet(RelNode rel,
RelSet set) |
boolean |
addRelTraitDef(RelTraitDef relTraitDef)
Registers a rel trait definition.
|
boolean |
addRule(RelOptRule rule)
Registers a rule.
|
private RelSubset |
canonize(RelSubset subset)
If a subset has one or more equivalent subsets (owing to a set having
merged with another), returns the subset which is the leader of the
equivalence class.
|
RelNode |
changeTraits(RelNode rel,
RelTraitSet toTraits)
Changes a relational expression to an equivalent one with a different set
of traits.
|
(package private) RelNode |
changeTraitsUsingConverters(RelNode rel,
RelTraitSet toTraits) |
private RelNode |
changeTraitsUsingConverters(RelNode rel,
RelTraitSet toTraits,
boolean allowAbstractConverters) |
(package private) void |
checkForSatisfiedConverters(RelSet set,
RelNode rel) |
RelOptPlanner |
chooseDelegate()
Negotiates an appropriate planner to deal with distributed queries.
|
void |
clear()
Removes all internal state, including all registered rules,
materialized views, and lattices.
|
private void |
clearImportanceBoost()
Clear all importance boosts.
|
void |
clearRelTraitDefs()
Clear all the registered RelTraitDef.
|
private RelNode |
completeConversion(RelNode rel,
boolean allowInfiniteCostConverters,
RelTraitSet toTraits,
Expressions.FluentList<RelTraitDef> usedTraits)
Converts traits using well-founded induction.
|
void |
dump(PrintWriter pw)
Dumps the internal state of this VolcanoPlanner to a writer.
|
RelTraitSet |
emptyTraitSet()
Creates an empty trait set.
|
RelSubset |
ensureRegistered(RelNode rel,
RelNode equivRel)
Registers a relational expression if it is not already registered.
|
void |
ensureRegistered(RelNode rel,
RelNode equivRel,
VolcanoRuleCall ruleCall) |
(package private) void |
ensureRootConverters()
Ensures that the subset that is the root relational expression contains
converters to all other subsets in its equivalence set.
|
private static RelSet |
equivRoot(RelSet s) |
RelNode |
findBestExp()
Finds the most efficient expression to implement the query given via
RelOptPlanner.setRoot(org.apache.calcite.rel.RelNode). |
(package private) void |
fireRules(RelNode rel,
boolean deferred)
Fires all rules matched by a relational expression.
|
private boolean |
fixUpInputs(RelNode rel) |
private static RelSet |
forward1(RelSet s,
RelSet p)
Moves forward one link, checking for a cycle.
|
private static RelSet |
forward2(RelSet s,
RelSet p)
Moves forward two links, checking for a cycle at each.
|
RelOptCost |
getCost(RelNode rel,
RelMetadataQuery mq)
Computes the cost of a RelNode.
|
RelOptLattice |
getLattice(RelOptTable table)
Retrieves a lattice, given its star table.
|
com.google.common.collect.ImmutableList<RelOptMaterialization> |
getMaterializations() |
protected VolcanoPlannerPhaseRuleMappingInitializer |
getPhaseRuleMappingInitializer() |
long |
getRelMetadataTimestamp(RelNode rel)
Gets a timestamp for a given rel's metadata.
|
List<RelTraitDef> |
getRelTraitDefs()
Returns the list of active trait types.
|
RelNode |
getRoot()
Returns the root node of this query.
|
List<RelOptRule> |
getRules()
Returns the list of all registered rules.
|
RelSet |
getSet(RelNode rel)
Finds an expression's equivalence set.
|
RelSubset |
getSubset(RelNode rel)
Returns the subset that a relational expression belongs to.
|
RelSubset |
getSubset(RelNode rel,
RelTraitSet traits) |
RelSubset |
getSubset(RelNode rel,
RelTraitSet traits,
boolean createIfMissing) |
private void |
injectImportanceBoost()
Finds RelSubsets in the plan that contain only rels of
Convention.NONE and boosts their importance by 25%. |
boolean |
isRegistered(RelNode rel)
Determines whether a relational expression has been registered.
|
private static Pair<String,RelDataType> |
key(RelNode rel)
Computes the key for
mapDigestToRel. |
private RelSet |
merge(RelSet set,
RelSet set2) |
static String |
normalizePlan(String plan)
Normalizes references to subsets within the string representation of a
plan.
|
protected void |
onNewClass(RelNode node)
Called when a new class of
RelNode is seen. |
private String |
provenance(RelNode root)
Returns a multi-line string describing the provenance of a tree of
relational expressions.
|
private void |
provenanceRecurse(PrintWriter pw,
RelNode node,
int i,
Set<RelNode> visited)
Helper for
provenance(org.apache.calcite.rel.RelNode). |
RelSubset |
register(RelNode rel,
RelNode equivRel)
Registers a relational expression in the expression bank.
|
void |
registerAbstractRelationalRules() |
private RelSubset |
registerImpl(RelNode rel,
RelSet set)
Registers a new expression
exp and queues up rule matches. |
private void |
registerMaterializations() |
void |
registerMetadataProviders(List<RelMetadataProvider> list)
Gives this planner a chance to register one or more
RelMetadataProviders in the chain which will be used to answer
metadata queries. |
private void |
registerMetadataRels()
Informs
JaninoRelMetadataProvider about the different kinds of
RelNode that we will be dealing with. |
void |
registerSchema(RelOptSchema schema)
Tells this planner that a schema exists.
|
private RelSubset |
registerSubset(RelSet set,
RelSubset subset) |
boolean |
removeRule(RelOptRule rule)
Removes a rule.
|
(package private) void |
rename(RelNode rel)
Re-computes the digest of a
RelNode. |
(package private) void |
reregister(RelSet set,
RelNode rel)
|
void |
setImportance(RelNode rel,
double importance)
Sets the importance of a relational expression.
|
private void |
setInitialImportance() |
void |
setLocked(boolean locked)
Sets whether this planner is locked.
|
void |
setRoot(RelNode rel)
Sets the root node of this query.
|
protected void |
validate()
Checks internal consistency.
|
checkCancel, fireRule, getContext, getCost, getCostFactory, getExecutor, getListener, getRuleByDescription, isRuleExcluded, mapRuleDescription, notifyChosen, notifyDiscard, notifyEquivalence, notifyTransformation, onCopy, registerClass, setCancelFlag, setExecutor, setRuleDescExclusionFilter, subClasses, unmapRuleDescriptionprotected static final double COST_IMPROVEMENT
protected RelSubset root
protected boolean ambitious
protected boolean impatient
ambitious is true, the planner waits a finite
number of iterations for the cost to improve.
The number of iterations K is equal to the number of iterations
required to get the first finite plan. After the first finite plan, it
continues to fire rules to try to improve it. The planner sets a target
cost of the current best cost multiplied by COST_IMPROVEMENT. If
it does not meet that cost target within K steps, it quits, and uses the
current best plan. If it meets the cost, it sets a new, lower target, and
has another K iterations to meet it. And so forth.
If false, the planner continues to fire rules until the rule queue is empty.
private final com.google.common.collect.Multimap<Class<? extends RelNode>,RelOptRuleOperand> classOperands
RelNode.
Any operand can be an 'entry point' to a rule call, when a RelNode is registered which matches the operand. This map allows us to narrow down operands based on the class of the RelNode.
private final Map<Pair<String,RelDataType>,RelNode> mapDigestToRel
digest to the unique
relational expression with that digest.
Row type is part of the key for the rare occasion that similar
expressions have different types, e.g. variants of
Project(child=rel#1, a=null) where a is a null INTEGER or a
null VARCHAR(10).
private final IdentityHashMap<RelNode,RelSubset> mapRel2Subset
RelNode) to its equivalence set
(RelSubset).
We use an IdentityHashMap to simplify the process of merging
RelSet objects. Most RelNode objects are identified by
their digest, which involves the set that their child relational
expressions belong to. If those children belong to the same set, we have
to be careful, otherwise it gets incestuous.
final Map<RelNode,Double> relImportances
The map contains only RelNodes whose importance has been overridden
using RelOptPlanner.setImportance(RelNode, double). Other
RelNodes are presumed to have 'normal' importance.
If a RelNode has 0 importance, all RelOptRuleCalls using it
are ignored, and future RelOptRuleCalls are not queued up.
private final Set<RelOptSchema> registeredSchemas
final RuleQueue ruleQueue
private final List<RelTraitDef> traitDefs
protected final Set<RelOptRule> ruleSet
private int nextSetId
private int registerCount
RelOptListener listener
private String originalRootString
private RelNode originalRoot
private boolean locked
private final List<RelOptMaterialization> materializations
private final Map<List<String>,RelOptLattice> latticeByName
final Map<RelNode,VolcanoPlanner.Provenance> provenanceMap
private final Deque<VolcanoRuleCall> ruleCallStack
private final RelOptCost zeroCost
AbstractRelOptPlanner.costFactory. Not necessarily a
VolcanoCost.public VolcanoPlanner()
VolcanoPlanner. To fully initialize
it, the caller must register the desired set of relations, rules, and
calling conventions.public VolcanoPlanner(Context externalContext)
VolcanoPlanner. To fully initialize
it, the caller must register the desired set of relations, rules, and
calling conventions.public VolcanoPlanner(RelOptCostFactory costFactory, Context externalContext)
VolcanoPlanner with a given cost factory.protected VolcanoPlannerPhaseRuleMappingInitializer getPhaseRuleMappingInitializer()
public boolean isRegistered(RelNode rel)
RelOptPlannerrel - expression to testpublic void setRoot(RelNode rel)
RelOptPlannerrel - Relational expressionpublic RelNode getRoot()
RelOptPlannerpublic com.google.common.collect.ImmutableList<RelOptMaterialization> getMaterializations()
public void addMaterialization(RelOptMaterialization materialization)
RelOptPlannerTypically tableRel is a
LogicalTableScan representing a
table that is a materialized view and queryRel is the SQL
expression that populates that view. The intention is that
tableRel is cheaper to evaluate and therefore if the query being
optimized uses (or can be rewritten to use) queryRel as a
sub-expression then it can be optimized by using tableRel
instead.
addMaterialization in interface RelOptPlanneraddMaterialization in class AbstractRelOptPlannerpublic void addLattice(RelOptLattice lattice)
RelOptPlannerThe lattice may have materializations; it is not necessary to call
RelOptPlanner.addMaterialization(org.apache.calcite.plan.RelOptMaterialization) for these; they are registered implicitly.
addLattice in interface RelOptPlanneraddLattice in class AbstractRelOptPlannerpublic RelOptLattice getLattice(RelOptTable table)
RelOptPlannergetLattice in interface RelOptPlannergetLattice in class AbstractRelOptPlannerprivate void registerMaterializations()
public RelSet getSet(RelNode rel)
rel - Relational expressionpublic boolean addRelTraitDef(RelTraitDef relTraitDef)
RelOptPlannerRelTraitDef has already
been registered, does nothing.addRelTraitDef in interface RelOptPlanneraddRelTraitDef in class AbstractRelOptPlannerCollection.add(E)public void clearRelTraitDefs()
RelOptPlannerclearRelTraitDefs in interface RelOptPlannerclearRelTraitDefs in class AbstractRelOptPlannerpublic List<RelTraitDef> getRelTraitDefs()
RelOptPlannergetRelTraitDefs in interface RelOptPlannergetRelTraitDefs in class AbstractRelOptPlannerpublic RelTraitSet emptyTraitSet()
RelOptPlannerThe empty trait set acts as the prototype (a kind of factory) for all subsequently created trait sets.
emptyTraitSet in interface RelOptPlanneremptyTraitSet in class AbstractRelOptPlannerpublic void clear()
RelOptPlannerclear in interface RelOptPlannerclear in class AbstractRelOptPlannerpublic List<RelOptRule> getRules()
RelOptPlannerpublic boolean addRule(RelOptRule rule)
RelOptPlannerIf the rule has already been registered, does nothing.
This method determines if the given rule is a
ConverterRule and pass the
ConverterRule to all
registered RelTraitDef
instances.
Collection.add(E)public boolean removeRule(RelOptRule rule)
RelOptPlannerCollection.remove(Object)protected void onNewClass(RelNode node)
AbstractRelOptPlannerRelNode is seen.onNewClass in class AbstractRelOptPlannerpublic RelNode changeTraits(RelNode rel, RelTraitSet toTraits)
RelOptPlannerrel - Relational expression (may or may not have been registered; must
not have the desired traits)toTraits - Trait set to convert the relational expression topublic RelOptPlanner chooseDelegate()
RelOptPlannerchooseDelegate in interface RelOptPlannerchooseDelegate in class AbstractRelOptPlannerpublic RelNode findBestExp()
RelOptPlanner.setRoot(org.apache.calcite.rel.RelNode).
The algorithm executes repeatedly in a series of phases. In each phase
the exact rules that may be fired varies. The mapping of phases to rule
sets is maintained in the ruleQueue.
In each phase, the planner sets the initial importance of the existing
RelSubSets (setInitialImportance()). The planner then iterates
over the rule matches presented by the rule queue until:
Furthermore, after every 10 iterations without an implementable plan,
RelSubSets that contain only logical RelNodes are given an importance
boost via injectImportanceBoost(). Once an implementable plan is
found, the artificially raised importance values are cleared (see
clearImportanceBoost()).
private void registerMetadataRels()
JaninoRelMetadataProvider about the different kinds of
RelNode that we will be dealing with. It will reduce the number
of times that we need to re-generate the provider.void ensureRootConverters()
Thus the planner tries to find cheap implementations of those other subsets, which can then be converted to the root. This is the only place in the plan where explicit converters are required; elsewhere, a consumer will be asking for the result in a particular convention, but the root has no consumers.
private String provenance(RelNode root)
Thus, every relational expression and rule invocation that affected the final outcome is described in the provenance. This can be useful when finding the root cause of "mistakes" in a query plan.
root - Root relational expression in a treeprivate void provenanceRecurse(PrintWriter pw, RelNode node, int i, Set<RelNode> visited)
provenance(org.apache.calcite.rel.RelNode).private void setInitialImportance()
private void injectImportanceBoost()
Convention.NONE and boosts their importance by 25%.private void clearImportanceBoost()
public RelSubset register(RelNode rel, RelNode equivRel)
RelOptPlannerAfter it has been registered, you may not modify it.
The expression must not already have been registered. If you are not
sure whether it has been registered, call
RelOptPlanner.ensureRegistered(RelNode, RelNode).
rel - Relational expression to register (must not already be
registered)equivRel - Relational expression it is equivalent to (may be null)public RelSubset ensureRegistered(RelNode rel, RelNode equivRel)
RelOptPlannerIf equivRel is specified, rel is placed in the same
equivalence set. It is OK if equivRel has different traits;
rel will end up in a different subset of the same set.
It is OK if rel is a subset.
rel - Relational expression to registerequivRel - Relational expression it is equivalent to (may be null)protected void validate()
public void registerAbstractRelationalRules()
public void registerSchema(RelOptSchema schema)
RelOptPlannerregisterSchema in interface RelOptPlannerregisterSchema in class AbstractRelOptPlannerpublic RelOptCost getCost(RelNode rel, RelMetadataQuery mq)
RelOptPlannerRelMetadataQuery.getCumulativeCost(org.apache.calcite.rel.RelNode).getCost in interface RelOptPlannergetCost in class AbstractRelOptPlannerrel - Relational expression of interestmq - Metadata querypublic RelSubset getSubset(RelNode rel)
rel - Relational expressionpublic RelSubset getSubset(RelNode rel, RelTraitSet traits)
public RelSubset getSubset(RelNode rel, RelTraitSet traits, boolean createIfMissing)
private RelNode changeTraitsUsingConverters(RelNode rel, RelTraitSet toTraits, boolean allowAbstractConverters)
private RelNode completeConversion(RelNode rel, boolean allowInfiniteCostConverters, RelTraitSet toTraits, Expressions.FluentList<RelTraitDef> usedTraits)
rel - Relational expressionallowInfiniteCostConverters - Whether to allow infinite converterstoTraits - Target trait setusedTraits - Traits that have been locked inRelNode changeTraitsUsingConverters(RelNode rel, RelTraitSet toTraits)
public void setImportance(RelNode rel, double importance)
RelOptPlannerAn important use of this method is when a RelOptRule has
created a relational expression which is indisputably better than the
original relational expression. The rule set the original relational
expression's importance to zero, to reduce the search space. Pending rule
calls are cancelled, and future rules will not fire.
setImportance in interface RelOptPlannersetImportance in class AbstractRelOptPlannerrel - Relational expressionimportance - Importancepublic void dump(PrintWriter pw)
pw - Print writernormalizePlan(String)private static Pair<String,RelDataType> key(RelNode rel)
mapDigestToRel.void rename(RelNode rel)
RelNode.
Since a relational expression's digest contains the identifiers of its children, this method needs to be called when the child has been renamed, for example if the child's set merges with another.
rel - Relational expressionvoid reregister(RelSet set, RelNode rel)
set - Setrel - Relational expressionprivate RelSubset canonize(RelSubset subset)
subset - Subsetvoid fireRules(RelNode rel, boolean deferred)
rel - Relational expression which has just been created (or maybe
from the queue)deferred - If true, each time a rule matches, just add an entry to
the queue.private boolean fixUpInputs(RelNode rel)
private static RelSet forward2(RelSet s, RelSet p)
private static RelSet forward1(RelSet s, RelSet p)
private RelSubset registerImpl(RelNode rel, RelSet set)
exp and queues up rule matches.
If set is not null, makes the expression part of that
equivalence set. If an identical expression is already registered, we
don't need to register this one and nor should we queue up rule matches.public void addListener(RelOptListener newListener)
RelOptPlanneraddListener in interface RelOptPlanneraddListener in class AbstractRelOptPlannernewListener - new listener to be notified of eventspublic void registerMetadataProviders(List<RelMetadataProvider> list)
RelOptPlannerRelMetadataProviders in the chain which will be used to answer
metadata queries.
Planners which use their own relational expressions internally to represent concepts such as equivalence classes will generally need to supply corresponding metadata providers.
registerMetadataProviders in interface RelOptPlannerregisterMetadataProviders in class AbstractRelOptPlannerlist - receives planner's custom providers, if anypublic long getRelMetadataTimestamp(RelNode rel)
RelOptPlannerCachingRelMetadataProvider to decide whether cached metadata has
gone stale.getRelMetadataTimestamp in interface RelOptPlannergetRelMetadataTimestamp in class AbstractRelOptPlannerrel - rel of interestpublic static String normalizePlan(String plan)
This is useful when writing tests: it helps to ensure that tests don't break when an extra rule is introduced that generates a new subset and causes subsequent subset numbers to be off by one.
For example,
FennelAggRel.FENNEL_EXEC(child=Subset#17.FENNEL_EXEC,groupCount=1, EXPR$1=COUNT())
FennelSortRel.FENNEL_EXEC(child=Subset#2.FENNEL_EXEC, key=[0], discardDuplicates=false)
FennelCalcRel.FENNEL_EXEC( child=Subset#4.FENNEL_EXEC, expr#0..8={inputs}, expr#9=3456, DEPTNO=$t7, $f0=$t9)
MockTableImplRel.FENNEL_EXEC( table=[CATALOG, SALES, EMP])
becomes
FennelAggRel.FENNEL_EXEC(child=Subset#{0}.FENNEL_EXEC, groupCount=1, EXPR$1=COUNT())
FennelSortRel.FENNEL_EXEC(child=Subset#{1}.FENNEL_EXEC, key=[0], discardDuplicates=false)
FennelCalcRel.FENNEL_EXEC( child=Subset#{2}.FENNEL_EXEC,expr#0..8={inputs},expr#9=3456,DEPTNO=$t7, $f0=$t9)
MockTableImplRel.FENNEL_EXEC( table=[CATALOG, SALES, EMP])
plan - Planpublic void setLocked(boolean locked)
addRule(org.apache.calcite.plan.RelOptRule) will do
nothing and return false.locked - Whether planner is lockedpublic void ensureRegistered(RelNode rel, RelNode equivRel, VolcanoRuleCall ruleCall)
Copyright © 2012–2018 The Apache Software Foundation. All rights reserved.