org.elasticsearch.search.aggregations.metrics.cardinality

Class HyperLogLogPlusPlus

java.lang.Object

org.elasticsearch.search.aggregations.metrics.cardinality.HyperLogLogPlusPlus

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, Releasable

public final class HyperLogLogPlusPlus
extends java.lang.Object
implements Releasable

Hyperloglog++ counter, implemented based on pseudo code from http://static.googleusercontent.com/media/research.google.com/fr//pubs/archive/40671.pdf and its appendix https://docs.google.com/document/d/1gyjfMHy43U9OWBXxfaeG-3MjGzejW1dlpyMwEYAAWEI/view?fullscreen This implementation is different from the original implementation in that it uses a hash table instead of a sorted list for linear counting. Although this requires more space and makes hyperloglog (which is less accurate) used sooner, this is also considerably faster. Trying to understand what this class does without having read the paper is considered adventurous.

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_PRECISION
static int	MAX_PRECISION
static int	MIN_PRECISION

Constructor Summary

Constructors

Constructor and Description
HyperLogLogPlusPlus(int precision, BigArrays bigArrays, long initialBucketCount)

Method Summary

Modifier and Type	Method and Description
long	cardinality(long bucket)
void	close()
void	collect(long bucket, long hash)
boolean	equals(long bucket, HyperLogLogPlusPlus other)
int	hashCode(long bucket)
long	maxBucket()
static long	memoryUsage(int precision) Return the expected per-bucket memory usage for the given precision.
void	merge(long thisBucket, HyperLogLogPlusPlus other, long otherBucket)
int	precision()
static int	precisionFromThreshold(long count) Compute the required precision so that count distinct entries would be counted with linear counting.
static HyperLogLogPlusPlus	readFrom(StreamInput in, BigArrays bigArrays)
void	writeTo(long bucket, StreamOutput out)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MIN_PRECISION

public static final int MIN_PRECISION

See Also:

Constant Field Values

DEFAULT_PRECISION

public static final int DEFAULT_PRECISION

See Also:

Constant Field Values

MAX_PRECISION

public static final int MAX_PRECISION

See Also:

Constant Field Values

Constructor Detail

HyperLogLogPlusPlus

public HyperLogLogPlusPlus(int precision,
                           BigArrays bigArrays,
                           long initialBucketCount)

Method Detail

precisionFromThreshold

public static int precisionFromThreshold(long count)

Compute the required precision so that count distinct entries would be counted with linear counting.

memoryUsage

public static long memoryUsage(int precision)

Return the expected per-bucket memory usage for the given precision.

precision

public int precision()

maxBucket

public long maxBucket()

merge

public void merge(long thisBucket,
                  HyperLogLogPlusPlus other,
                  long otherBucket)

collect

public void collect(long bucket,
                    long hash)

cardinality

public long cardinality(long bucket)

close

public void close()

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface Releasable

hashCode

public int hashCode(long bucket)

equals

public boolean equals(long bucket,
                      HyperLogLogPlusPlus other)

writeTo

public void writeTo(long bucket,
                    StreamOutput out)
             throws java.io.IOException

Throws:

java.io.IOException

readFrom

public static HyperLogLogPlusPlus readFrom(StreamInput in,
                                           BigArrays bigArrays)
                                    throws java.io.IOException

Throws:

java.io.IOException