org.apache.calcite.profile

Class ProfilerImpl

java.lang.Object

org.apache.calcite.profile.ProfilerImpl

All Implemented Interfaces:

Profiler

public class ProfilerImpl
extends Object
implements Profiler

Implementation of Profiler that only investigates "interesting" combinations of columns.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ProfilerImpl.Builder Builds a ProfilerImpl.
(package private) static class	ProfilerImpl.Collector Collects values of a column or columns.
(package private) static class	ProfilerImpl.CompositeCollector Collector that collects two or more column values in a tree set.
(package private) static class	ProfilerImpl.HllCollector Collector that collects two or more column values into a HyperLogLog sketch.
(package private) static class	ProfilerImpl.HllCompositeCollector Collector that collects two or more column values into a HyperLogLog sketch.
(package private) static class	ProfilerImpl.HllSingletonCollector Collector that collects one column value into a HyperLogLog sketch.
(package private) class	ProfilerImpl.Run A run of the profiler.
(package private) static class	ProfilerImpl.SingletonCollector Collector that collects values of a single column.
(package private) static class	ProfilerImpl.Space Work space for a particular combination of columns.
(package private) static class	ProfilerImpl.SurpriseQueue A priority queue of the last N surprise values.

Nested classes/interfaces inherited from interface org.apache.calcite.profile.Profiler

Profiler.Column, Profiler.Distribution, Profiler.FunctionalDependency, Profiler.Profile, Profiler.RowCount, Profiler.Statistic, Profiler.Unique

Field Summary

Fields

Modifier and Type	Field and Description
private int	combinationsPerPass The number of combinations to consider per pass.
private int	interestingCount The minimum number of combinations considered "interesting".
private com.google.common.base.Predicate<Pair<ProfilerImpl.Space,Profiler.Column>>	predicate Whether a successor is considered interesting enough to analyze.

Constructor Summary

Constructors

Constructor and Description
ProfilerImpl(int combinationsPerPass, int interestingCount, com.google.common.base.Predicate<Pair<ProfilerImpl.Space,Profiler.Column>> predicate) Creates a ProfilerImpl.

Method Summary

Modifier and Type	Method and Description
static ProfilerImpl.Builder	builder()
Profiler.Profile	profile(Iterable<List<Comparable>> rows, List<Profiler.Column> columns, Collection<ImmutableBitSet> initialGroups) Creates a profile of a data set.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

combinationsPerPass

private final int combinationsPerPass

The number of combinations to consider per pass. The number is determined by memory, but a value of 1,000 is typical. You need 2KB memory per sketch, and one sketch for each combination.

interestingCount

private final int interestingCount

The minimum number of combinations considered "interesting". After that, a combination is only considered "interesting" if its surprise is greater than the median surprise.

predicate

private final com.google.common.base.Predicate<Pair<ProfilerImpl.Space,Profiler.Column>> predicate

Whether a successor is considered interesting enough to analyze.

Constructor Detail

ProfilerImpl

ProfilerImpl(int combinationsPerPass,
             int interestingCount,
             com.google.common.base.Predicate<Pair<ProfilerImpl.Space,Profiler.Column>> predicate)

Creates a ProfilerImpl.

Parameters:

combinationsPerPass - Maximum number of columns (or combinations of columns) to compute each pass

interestingCount - Minimum number of combinations considered interesting

predicate - Whether a successor is considered interesting enough to analyze

Method Detail

builder

public static ProfilerImpl.Builder builder()

profile

public Profiler.Profile profile(Iterable<List<Comparable>> rows,
                                List<Profiler.Column> columns,
                                Collection<ImmutableBitSet> initialGroups)

Description copied from interface: Profiler

Creates a profile of a data set.

Specified by:

profile in interface Profiler

Parameters:

rows - List of rows. Can be iterated over more than once (maybe not cheaply)

columns - Column definitions

initialGroups - List of combinations of columns that should be profiled early, because they may be interesting

Returns:

A profile describing relationships within the data set