org.mapdb

Class HTreeMap<K,V>

java.lang.Object

java.util.AbstractMap<K,V>

org.mapdb.HTreeMap<K,V>

All Implemented Interfaces:

Closeable, Serializable, AutoCloseable, ConcurrentMap<K,V>, Map<K,V>, Bind.MapWithModificationListener<K,V>

public class HTreeMap<K,V>
extends AbstractMap<K,V>
implements ConcurrentMap<K,V>, Bind.MapWithModificationListener<K,V>, Closeable, Serializable

Thread safe concurrent HashMap

This map uses full 32bit hash from beginning, There is no initial load factor and rehash. Technically it is not hash table, but hash tree with nodes expanding when they become full.

This map is suitable for number of records 1e9 and over. Larger number of records will increase hash collisions and performance will degrade linearly with number of records (separate chaining).

Concurrent scalability is achieved by splitting HashMap into 16 segments, each with separate lock. Very similar to ConcurrentHashMap

Author:

Jan Kotek

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	HTreeMap.ExpireLinkNode
class	HTreeMap.KeySet
protected static class	HTreeMap.LinkedNode<K,V> node which holds key-value pair

Nested classes/interfaces inherited from class java.util.AbstractMap

AbstractMap.SimpleEntry<K,V>, AbstractMap.SimpleImmutableEntry<K,V>

Nested classes/interfaces inherited from interface java.util.Map

Map.Entry<K,V>

Field Summary

Fields

Modifier and Type	Field and Description
protected static int	BUCKET_OVERFLOW
protected boolean	closeEngine
protected boolean	closeExecutor Indicates if this collection collection was not made by DB by user.
protected Lock	consistencyLock
protected long[]	counterRecids
protected static Serializer<Object>	DIR_SERIALIZER
protected static int	DIV8
protected Engine[]	engines
protected ScheduledExecutorService	executor
protected long	expire
protected long	expireAccess
protected boolean	expireAccessFlag
protected boolean	expireFlag
protected long[]	expireHeads
protected long	expireLastTick
protected long	expireMaxSize
protected boolean	expireMaxSizeFlag
protected boolean	expireSingleThreadFlag
protected long	expireStoreSize
protected long[]	expireStoreSizes
protected long[]	expireStoreSizesCompact
protected long[]	expireTails
protected long	expireTick
protected long	expireTimeStart
protected int	hashSalt Salt added to hash before rehashing, so it is harder to trigger hash collision attack.
protected boolean	hasValues is this a Map or Set? if false, entries do not have values, only keys are allowed
protected Serializer<K>	keySerializer
protected Serializer<HTreeMap.LinkedNode<K,V>>	LN_SERIALIZER
protected static Logger	LOG
protected static int	MOD8
protected Bind.MapListener<K,V>[]	modListeners
protected Object	modListenersLock
protected ReentrantReadWriteLock[]	segmentLocks
protected long[]	segmentRecids list of segments, this is immutable
protected Fun.Function1<V,K>	valueCreator
protected Serializer<V>	valueSerializer

Constructor Summary

Constructors

Constructor and Description

HTreeMap(Engine[] engines, boolean closeEngine, long[] counterRecids, int hashSalt, long[] segmentRecids, Serializer<K> keySerializer, Serializer<V> valueSerializer, long expireTimeStart, long expire, long expireAccess, long expireMaxSize, long expireStoreSize, long expireTick, long[] expireHeads, long[] expireTails, Fun.Function1<V,K> valueCreator, ScheduledExecutorService executor, long executorPeriod, boolean closeExecutor, Lock consistencyLock) Opens HTreeMap

Method Summary

Methods

Modifier and Type	Method and Description
void	clear()
void	close()
boolean	containsKey(Object o)
boolean	containsValue(Object value)
protected void	counter(int segment, Engine engine, int plus)
protected static long	dirGet(Object dir, int pos)
protected long	dirGetSlot(Object dir, int slot)
protected static boolean	dirIsEmpty(Object dir)
protected static int	dirLen(Object dir)
protected static int	dirOffsetFromSlot(int[] dir, int slot) converts hash slot into actual offset in dir array, using bitmap
protected static int	dirOffsetFromSlot(long[] dir, int slot) converts hash slot into actual offset in dir array, using bitmap
protected static int	dirOffsetFromSlot(Object dir, int slot)
protected static Object	dirPut(Object dir, int slot, long newRecid)
protected static Object	dirRemove(Object dir, int slot)
protected static int	dirStart(Object dir)
Set<Map.Entry<K,V>>	entrySet()
protected void	expireCheckSegment(int segment)
protected void	expireLinkAdd(int segment, long expireNodeRecid, long keyRecid, int hash)
protected void	expireLinkBump(int segment, long nodeRecid, boolean access)
protected HTreeMap.ExpireLinkNode	expireLinkRemove(int segment, long nodeRecid)
protected HTreeMap.ExpireLinkNode	expireLinkRemoveLast(int segment)
protected void	expirePurge()
protected long	expirePurgeSegment(int seg, long removePerSegment)
V	get(Object o)
Engine	getEngine()
protected HTreeMap.LinkedNode<K,V>	getInner(Object o, int h, int segment)
long	getMaxExpireTime() Returns maximal (newest) expiration timestamp
long	getMinExpireTime() Returns minimal (oldest) expiration timestamp
V	getPeek(Object key) Return given value, without updating cache statistics if expireAccess() is true It also does not use valueCreator if value is not found (always returns null if not found)
protected int	hash(Object key)
boolean	isEmpty()
Set<K>	keySet()
long	mappingCount()
void	modificationListenerAdd(Bind.MapListener<K,V> listener) Add new modification listener notified when Map has been updated
void	modificationListenerRemove(Bind.MapListener<K,V> listener) Remove registered notification listener
protected void	notify(K key, V oldValue, V newValue)
protected static long[]	preallocateSegments(Engine[] engines)
V	put(K key, V value)
V	putIfAbsent(K key, V value)
V	remove(Object key)
boolean	remove(Object key, Object value)
protected V	removeInternal(Object key, int segment, int h, boolean removeExpire)
V	replace(K key, V value)
boolean	replace(K key, V oldValue, V newValue)
int	size()
long	sizeLong()
Map<K,V>	snapshot() Make readonly snapshot view of current Map.
Collection<V>	values()

Methods inherited from class java.util.AbstractMap

clone, equals, hashCode, putAll, toString

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.Map

equals, hashCode, putAll

Field Detail

LOG

protected static final Logger LOG

BUCKET_OVERFLOW

protected static final int BUCKET_OVERFLOW

See Also:

Constant Field Values

DIV8

protected static final int DIV8

See Also:

Constant Field Values

MOD8

protected static final int MOD8

See Also:

Constant Field Values

hasValues

protected final boolean hasValues

is this a Map or Set? if false, entries do not have values, only keys are allowed

hashSalt

protected final int hashSalt

Salt added to hash before rehashing, so it is harder to trigger hash collision attack.

counterRecids

protected final long[] counterRecids

keySerializer

protected final Serializer<K> keySerializer

valueSerializer

protected final Serializer<V> valueSerializer

engines

protected final Engine[] engines

closeEngine

protected final boolean closeEngine

expireFlag

protected final boolean expireFlag

expireSingleThreadFlag

protected final boolean expireSingleThreadFlag

expireTimeStart

protected final long expireTimeStart

expire

protected final long expire

expireAccessFlag

protected final boolean expireAccessFlag

expireAccess

protected final long expireAccess

expireMaxSize

protected final long expireMaxSize

expireStoreSize

protected final long expireStoreSize

expireTick

protected final long expireTick

expireMaxSizeFlag

protected final boolean expireMaxSizeFlag

expireHeads

protected final long[] expireHeads

expireTails

protected final long[] expireTails

expireStoreSizes

protected final long[] expireStoreSizes

expireStoreSizesCompact

protected final long[] expireStoreSizesCompact

valueCreator

protected final Fun.Function1<V,K> valueCreator

closeExecutor

protected final boolean closeExecutor

Indicates if this collection collection was not made by DB by user. If user can not access DB object, we must shutdown Executor and close Engine ourself in close() method.

executor

protected final ScheduledExecutorService executor

consistencyLock

protected final Lock consistencyLock

expireLastTick

protected volatile long expireLastTick

LN_SERIALIZER

protected final Serializer<HTreeMap.LinkedNode<K,V>> LN_SERIALIZER

DIR_SERIALIZER

protected static final Serializer<Object> DIR_SERIALIZER

segmentRecids

protected final long[] segmentRecids

list of segments, this is immutable

segmentLocks

protected final ReentrantReadWriteLock[] segmentLocks

modListenersLock

protected final Object modListenersLock

modListeners

protected Bind.MapListener<K,V>[] modListeners

Constructor Detail

HTreeMap

public HTreeMap(Engine[] engines,
        boolean closeEngine,
        long[] counterRecids,
        int hashSalt,
        long[] segmentRecids,
        Serializer<K> keySerializer,
        Serializer<V> valueSerializer,
        long expireTimeStart,
        long expire,
        long expireAccess,
        long expireMaxSize,
        long expireStoreSize,
        long expireTick,
        long[] expireHeads,
        long[] expireTails,
        Fun.Function1<V,K> valueCreator,
        ScheduledExecutorService executor,
        long executorPeriod,
        boolean closeExecutor,
        Lock consistencyLock)

Opens HTreeMap

Method Detail

preallocateSegments

protected static long[] preallocateSegments(Engine[] engines)

containsKey

public boolean containsKey(Object o)

Specified by:

containsKey in interface Map<K,V>

Overrides:

containsKey in class AbstractMap<K,V>

size

public int size()

Specified by:

size in interface Map<K,V>

Overrides:

size in class AbstractMap<K,V>

sizeLong

public long sizeLong()

Specified by:

sizeLong in interface Bind.MapWithModificationListener<K,V>

Returns:

size of map, but in 64bit long which does not overflow at 2e9 items.

mappingCount

public long mappingCount()

isEmpty

public boolean isEmpty()

Specified by:

isEmpty in interface Map<K,V>

Overrides:

isEmpty in class AbstractMap<K,V>

get

public V get(Object o)

Specified by:

get in interface Map<K,V>

Overrides:

get in class AbstractMap<K,V>

getPeek

public V getPeek(Object key)

Return given value, without updating cache statistics if expireAccess() is true It also does not use valueCreator if value is not found (always returns null if not found)

Parameters:

key - key to lookup

Returns:

value associated with key or null

getInner

protected HTreeMap.LinkedNode<K,V> getInner(Object o,
                                int h,
                                int segment)

dirIsEmpty

protected static boolean dirIsEmpty(Object dir)

dirLen

protected static int dirLen(Object dir)

dirStart

protected static int dirStart(Object dir)

dirGet

protected static long dirGet(Object dir,
          int pos)

dirGetSlot

protected long dirGetSlot(Object dir,
              int slot)

dirOffsetFromSlot

protected static int dirOffsetFromSlot(Object dir,
                    int slot)

dirOffsetFromSlot

protected static final int dirOffsetFromSlot(int[] dir,
                    int slot)

converts hash slot into actual offset in dir array, using bitmap

dirOffsetFromSlot

protected static final int dirOffsetFromSlot(long[] dir,
                    int slot)

converts hash slot into actual offset in dir array, using bitmap

dirPut

protected static final Object dirPut(Object dir,
            int slot,
            long newRecid)

dirRemove

protected static final Object dirRemove(Object dir,
               int slot)

put

public V put(K key,
    V value)

Specified by:

put in interface Map<K,V>

Overrides:

put in class AbstractMap<K,V>

counter

protected void counter(int segment,
           Engine engine,
           int plus)

remove

public V remove(Object key)

Specified by:

remove in interface Map<K,V>

Overrides:

remove in class AbstractMap<K,V>

removeInternal

protected V removeInternal(Object key,
               int segment,
               int h,
               boolean removeExpire)

clear

public void clear()

Specified by:

clear in interface Map<K,V>

Overrides:

clear in class AbstractMap<K,V>

containsValue

public boolean containsValue(Object value)

Specified by:

containsValue in interface Map<K,V>

Overrides:

containsValue in class AbstractMap<K,V>

keySet

public Set<K> keySet()

Specified by:

keySet in interface Map<K,V>

Overrides:

keySet in class AbstractMap<K,V>

values

public Collection<V> values()

Specified by:

values in interface Map<K,V>

Overrides:

values in class AbstractMap<K,V>

entrySet

public Set<Map.Entry<K,V>> entrySet()

Specified by:

entrySet in interface Map<K,V>

Specified by:

entrySet in class AbstractMap<K,V>

hash

protected int hash(Object key)

putIfAbsent

public V putIfAbsent(K key,
            V value)

Specified by:

putIfAbsent in interface ConcurrentMap<K,V>

remove

public boolean remove(Object key,
             Object value)

Specified by:

remove in interface ConcurrentMap<K,V>

replace

public boolean replace(K key,
              V oldValue,
              V newValue)

Specified by:

replace in interface ConcurrentMap<K,V>

replace

public V replace(K key,
        V value)

Specified by:

replace in interface ConcurrentMap<K,V>

expireLinkAdd

protected void expireLinkAdd(int segment,
                 long expireNodeRecid,
                 long keyRecid,
                 int hash)

expireLinkBump

protected void expireLinkBump(int segment,
                  long nodeRecid,
                  boolean access)

expireLinkRemoveLast

protected HTreeMap.ExpireLinkNode expireLinkRemoveLast(int segment)

expireLinkRemove

protected HTreeMap.ExpireLinkNode expireLinkRemove(int segment,
                                       long nodeRecid)

getMaxExpireTime

public long getMaxExpireTime()

Returns maximal (newest) expiration timestamp

getMinExpireTime

public long getMinExpireTime()

Returns minimal (oldest) expiration timestamp

expirePurge

protected void expirePurge()

expirePurgeSegment

protected long expirePurgeSegment(int seg,
                      long removePerSegment)

expireCheckSegment

protected void expireCheckSegment(int segment)

snapshot

public Map<K,V> snapshot()

Make readonly snapshot view of current Map. Snapshot is immutable and not affected by modifications made by other threads. Useful if you need consistent view on Map.

Maintaining snapshot have some overhead, underlying Engine is closed after Map view is GCed. Please make sure to release reference to this Map view, so snapshot view can be garbage collected.

Returns:

snapshot

modificationListenerAdd

public void modificationListenerAdd(Bind.MapListener<K,V> listener)

Description copied from interface: Bind.MapWithModificationListener

Add new modification listener notified when Map has been updated

Specified by:

modificationListenerAdd in interface Bind.MapWithModificationListener<K,V>

Parameters:

listener - callback interface notified when map changes

modificationListenerRemove

public void modificationListenerRemove(Bind.MapListener<K,V> listener)

Description copied from interface: Bind.MapWithModificationListener

Remove registered notification listener

Specified by:

modificationListenerRemove in interface Bind.MapWithModificationListener<K,V>

Parameters:

listener - callback interface notified when map changes

notify

protected void notify(K key,
          V oldValue,
          V newValue)

getEngine

public Engine getEngine()

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable