org.mapdb
Class DBMaker<DBMakerT extends DBMaker<DBMakerT>>

java.lang.Object
  org.mapdb.DBMaker<DBMakerT>

public class DBMaker<DBMakerT extends DBMaker<DBMakerT>>
extends Object

A builder class for creating and opening a database.

Author:

Jan Kotek

Nested Class Summary

protected static interface

DBMaker.Keys

Field Summary

protected Properties	props
protected String	TRUE

Constructor Summary

protected	DBMaker() use static factory methods, or make subclass
protected	DBMaker(File file)

Method Summary

protected DBMakerT	_newAppendFileDB(File file)
DBMakerT	_newFileDB(File file)
DBMakerT	_newHeapDB()
DBMakerT	_newMemoryDB()
DBMakerT	_newMemoryDirectDB()
DBMakerT	asyncWriteEnable() Enables mode where all modifications are queued and written into disk on Background Writer Thread.
DBMakerT	asyncWriteFlushDelay(int delay) Set flush interval for write cache, by default is 0 When BTreeMap is constructed from ordered set, tree node size is increasing linearly with each item added.
DBMakerT	asyncWriteQueueSize(int queueSize) Set size of async Write Queue.
DBMakerT	cacheDisable() Instance cache is enabled by default.
DBMakerT	cacheHardRefEnable() Enables unbounded hard reference cache.
DBMakerT	cacheLRUEnable() Enables Least Recently Used cache.
DBMakerT	cacheSize(int cacheSize) Set cache size.
DBMakerT	cacheSoftRefEnable() Enables unbounded cache which uses SoftReference.
DBMakerT	cacheWeakRefEnable() Enables unbounded cache which uses WeakReference.
DBMakerT	checksumEnable() Adds CRC32 checksum at end of each record to check data integrity.
DBMakerT	closeOnJvmShutdown() Adds JVM shutdown hook and closes DB just before JVM;
DBMakerT	commitFileSyncDisable() Disables file sync on commit.
DBMakerT	compressionEnable() Enables record compression.
DBMakerT	deleteFilesAfterClose() Try to delete files after DB is closed.
DBMakerT	encryptionEnable(byte[] password) Encrypt storage using XTEA algorithm.
DBMakerT	encryptionEnable(String password) Encrypt storage using XTEA algorithm.
protected void	extendArgumentCheck()
protected Engine	extendAsyncWriteEngine(Engine engine)
protected Engine	extendCacheHardRef(Engine engine)
protected Engine	extendCacheHashTable(Engine engine)
protected Engine	extendCacheLRU(Engine engine)
protected Engine	extendCacheSoftRef(Engine engine)
protected Engine	extendCacheWeakRef(Engine engine)
protected Engine	extendHeapStore()
protected Engine	extendSnapshotEngine(Engine engine)
protected Engine	extendStoreAppend()
protected Engine	extendStoreDirect(Volume.Factory folFac)
protected Volume.Factory	extendStoreVolumeFactory()
protected Engine	extendStoreWAL(Volume.Factory folFac)
protected Engine	extendWrapCache(Engine engine)
protected Engine	extendWrapSnapshotEngine(Engine engine)
protected Engine	extendWrapStore(Engine engine)
DBMakerT	freeSpaceReclaimQ(int q) Set free space reclaim Q.
protected static byte[]	fromHexa(String s)
protected DBMakerT	getThis()
protected static boolean	JVMSupportsLargeMappedFiles() Check if large files can be mapped into memory.
DB	make() constructs DB using current settings
Engine	makeEngine() constructs Engine using current settings
TxMaker	makeTxMaker()
DBMakerT	mmapFileEnable() Enables Memory Mapped Files, much faster storage option.
DBMakerT	mmapFileEnableIfSupported() Enable Memory Mapped Files only if current JVM supports it (is 64bit).
DBMakerT	mmapFileEnablePartial() Keeps small-frequently-used part of storage files memory mapped, but main area is accessed using Random Access File.
protected static DBMaker	newAppendFileDB(File file) Creates or open append-only database stored in file.
static <K,V> HTreeMap<K,V>	newCache(double size) Creates new cache with maximal size in GBs.
static <K,V> HTreeMap<K,V>	newCacheDirect(double size) Creates new off-heap cache with maximal size in GBs.
static DBMaker	newFileDB(File file) Creates or open database stored in file.
static DBMaker	newHeapDB() Creates new in-memory database which stores all data on heap without serialization.
static DBMaker	newMemoryDB() Creates new in-memory database.
static DBMaker	newMemoryDirectDB() Creates new in-memory database.
static DBMaker	newTempFileDB() Creates new database in temporary folder.
static <K,V> HTreeMap<K,V>	newTempHashMap() Create new HTreeMap backed by temporary file storage.
static <K> Set<K>	newTempHashSet() Create new HashSet backed by temporary file storage.
static <K,V> BTreeMap<K,V>	newTempTreeMap() Create new BTreeMap backed by temporary file storage.
static <K> NavigableSet<K>	newTempTreeSet() Create new TreeSet backed by temporary file storage.
protected boolean	propsGetBool(String key)
protected int	propsGetInt(String key, int defValue)
protected long	propsGetLong(String key, long defValue)
protected int	propsGetRafMode()
protected byte[]	propsGetXteaEncKey()
DBMakerT	readOnly() Open store in read-only mode.
DBMakerT	sizeLimit(double maxSize) Sets store size limit.
DBMakerT	snapshotEnable() MapDB supports snapshots.
DBMakerT	strictDBGet() DB Get methods such as DB.getTreeMap(String) or DB.getAtomicLong(String) auto create new record with default values, if record with given name does not exist.
protected static String	toHexa(byte[] bb)
DBMakerT	transactionDisable() Transaction journal is enabled by default You must call DB.commit() to save your changes.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TRUE

protected final String TRUE

See Also:

Constant Field Values

props

protected Properties props

Constructor Detail

DBMaker

protected DBMaker()

use static factory methods, or make subclass

DBMaker

protected DBMaker(File file)

Method Detail

newHeapDB

public static DBMaker newHeapDB()

Creates new in-memory database which stores all data on heap without serialization. This mode should be very fast, but data will affect Garbage Collector the same way as traditional Java Collections.

_newHeapDB

public DBMakerT _newHeapDB()

newMemoryDB

public static DBMaker newMemoryDB()

Creates new in-memory database. Changes are lost after JVM exits.

This will use HEAP memory so Garbage Collector is affected.

_newMemoryDB

public DBMakerT _newMemoryDB()

newMemoryDirectDB

public static DBMaker newMemoryDirectDB()

Creates new in-memory database. Changes are lost after JVM exits.

This will use DirectByteBuffer outside of HEAP, so Garbage Collector is not affected

_newMemoryDirectDB

public DBMakerT _newMemoryDirectDB()

newAppendFileDB

protected static DBMaker newAppendFileDB(File file)

Creates or open append-only database stored in file. This database uses format other than usual file db

Parameters:

file -

Returns:

maker

_newAppendFileDB

protected DBMakerT _newAppendFileDB(File file)

newTempTreeMap

public static <K,V> BTreeMap<K,V> newTempTreeMap()

Create new BTreeMap backed by temporary file storage. This is quick way to create 'throw away' collection.

Storage is created in temp folder and deleted on JVM shutdown

newTempHashMap

public static <K,V> HTreeMap<K,V> newTempHashMap()

Create new HTreeMap backed by temporary file storage. This is quick way to create 'throw away' collection.

Storage is created in temp folder and deleted on JVM shutdown

newTempTreeSet

public static <K> NavigableSet<K> newTempTreeSet()

Create new TreeSet backed by temporary file storage. This is quick way to create 'throw away' collection.

Storage is created in temp folder and deleted on JVM shutdown

newTempHashSet

public static <K> Set<K> newTempHashSet()

Create new HashSet backed by temporary file storage. This is quick way to create 'throw away' collection.

Storage is created in temp folder and deleted on JVM shutdown

newTempFileDB

public static DBMaker newTempFileDB()

Creates new database in temporary folder.

newCacheDirect

public static <K,V> HTreeMap<K,V> newCacheDirect(double size)

Creates new off-heap cache with maximal size in GBs. Entries are removed from cache in most-recently-used fashion if store becomes too big. This method uses off-heap direct ByteBuffers. See ByteBuffer.allocateDirect(int)

Parameters:

size - maximal size of off-heap store in gigabytes.

Returns:

map

newCache

public static <K,V> HTreeMap<K,V> newCache(double size)

Creates new cache with maximal size in GBs. Entries are removed from cache in most-recently-used fashion if store becomes too big. This cache uses on-heap `byte[]`, but does not affect GC since objects are serialized into binary form. This method uses ByteBuffers backed by on-heap byte[]. See ByteBuffer.allocate(int)

Parameters:

size - maximal size of off-heap store in gigabytes.

Returns:

map

newFileDB

public static DBMaker newFileDB(File file)

Creates or open database stored in file.

_newFileDB

public DBMakerT _newFileDB(File file)

getThis

protected DBMakerT getThis()

transactionDisable

public DBMakerT transactionDisable()

Transaction journal is enabled by default You must call DB.commit() to save your changes. It is possible to disable transaction journal for better write performance In this case all integrity checks are sacrificed for faster speed.

If transaction journal is disabled, all changes are written DIRECTLY into store. You must call DB.close() method before exit, otherwise your store WILL BE CORRUPTED

Returns:

this builder

cacheDisable

public DBMakerT cacheDisable()

Instance cache is enabled by default. This greatly decreases serialization overhead and improves performance. Call this method to disable instance cache, so an object will always be deserialized.

This may workaround some problems

Returns:

this builder

cacheHardRefEnable

public DBMakerT cacheHardRefEnable()

Enables unbounded hard reference cache. This cache is good if you have lot of available memory.

All fetched records are added to HashMap and stored with hard reference. To prevent OutOfMemoryExceptions MapDB monitors free memory, if it is bellow 25% cache is cleared.

Returns:

this builder

cacheWeakRefEnable

public DBMakerT cacheWeakRefEnable()

Enables unbounded cache which uses WeakReference. Items are removed from cache by Garbage Collector

Returns:

this builder

cacheSoftRefEnable

public DBMakerT cacheSoftRefEnable()

Enables unbounded cache which uses SoftReference. Items are removed from cache by Garbage Collector

Returns:

this builder

cacheLRUEnable

public DBMakerT cacheLRUEnable()

Enables Least Recently Used cache. It is fixed size cache and it removes less used items to make space.

Returns:

this builder

mmapFileEnable

public DBMakerT mmapFileEnable()

Enables Memory Mapped Files, much faster storage option. However on 32bit JVM this mode could corrupt your DB thanks to 4GB memory addressing limit. You may experience `java.lang.OutOfMemoryError: Map failed` exception on 32bit JVM, if you enable this mode.

mmapFileEnablePartial

public DBMakerT mmapFileEnablePartial()

Keeps small-frequently-used part of storage files memory mapped, but main area is accessed using Random Access File. This mode is good performance compromise between Memory Mapped Files and old slow Random Access Files. Index file is typically 5% of storage. It contains small frequently read values, which is where memory mapped file excel. With this mode you will experience `java.lang.OutOfMemoryError: Map failed` exceptions on 32bit JVMs eventually. But storage size limit is pushed to somewhere around 40GB.

mmapFileEnableIfSupported

public DBMakerT mmapFileEnableIfSupported()

Enable Memory Mapped Files only if current JVM supports it (is 64bit).

cacheSize

public DBMakerT cacheSize(int cacheSize)

Set cache size. Interpretations depends on cache type. For fixed size caches (such as FixedHashTable cache) it is maximal number of items in cache.

For unbounded caches (such as HardRef cache) it is initial capacity of underlying table (HashMap).

Default cache size is 32768.

Parameters:

cacheSize - new cache size

Returns:

this builder

snapshotEnable

public DBMakerT snapshotEnable()

MapDB supports snapshots. `TxEngine` requires additional locking which has small overhead when not used. Snapshots are disabled by default. This option switches the snapshots on.

Returns:

this builder

asyncWriteEnable

public DBMakerT asyncWriteEnable()

Enables mode where all modifications are queued and written into disk on Background Writer Thread. So all modifications are performed in asynchronous mode and do not block.

Enabling this mode might increase performance for single threaded apps.

Returns:

this builder

asyncWriteFlushDelay

public DBMakerT asyncWriteFlushDelay(int delay)

Set flush interval for write cache, by default is 0

When BTreeMap is constructed from ordered set, tree node size is increasing linearly with each item added. Each time new key is added to tree node, its size changes and storage needs to find new place. So constructing BTreeMap from ordered set leads to large store fragmentation.

Setting flush interval is workaround as BTreeMap node is always updated in memory (write cache) and only final version of node is stored on disk.

Parameters:

delay - flush write cache every N miliseconds

Returns:

this builder

asyncWriteQueueSize

public DBMakerT asyncWriteQueueSize(int queueSize)

Set size of async Write Queue. Default size is 32 000

Using too large queue size can lead to out of memory exception.

Parameters:

queueSize - of queue

Returns:

this builder

deleteFilesAfterClose

public DBMakerT deleteFilesAfterClose()

Try to delete files after DB is closed. File deletion may silently fail, especially on Windows where buffer needs to be unmapped file delete.

Returns:

this builder

closeOnJvmShutdown

public DBMakerT closeOnJvmShutdown()

Adds JVM shutdown hook and closes DB just before JVM;

Returns:

this builder

compressionEnable

public DBMakerT compressionEnable()

Enables record compression.

Make sure you enable this every time you reopen store, otherwise record de-serialization fails unpredictably.

Returns:

this builder

encryptionEnable

public DBMakerT encryptionEnable(String password)

Encrypt storage using XTEA algorithm.

XTEA is sound encryption algorithm. However implementation in MapDB was not peer-reviewed. MapDB only encrypts records data, so attacker may see number of records and their sizes.

Make sure you enable this every time you reopen store, otherwise record de-serialization fails unpredictably.

Parameters:

password - for encryption

Returns:

this builder

encryptionEnable

public DBMakerT encryptionEnable(byte[] password)

Encrypt storage using XTEA algorithm.

XTEA is sound encryption algorithm. However implementation in MapDB was not peer-reviewed. MapDB only encrypts records data, so attacker may see number of records and their sizes.

Make sure you enable this every time you reopen store, otherwise record de-serialization fails unpredictably.

Parameters:

password - for encryption

Returns:

this builder

checksumEnable

public DBMakerT checksumEnable()

Adds CRC32 checksum at end of each record to check data integrity. It throws 'IOException("Checksum does not match, data broken")' on de-serialization if data are corrupted

Make sure you enable this every time you reopen store, otherwise record de-serialization fails.

Returns:

this builder

strictDBGet

public DBMakerT strictDBGet()

DB Get methods such as DB.getTreeMap(String) or DB.getAtomicLong(String) auto create new record with default values, if record with given name does not exist. This could be problem if you would like to enforce stricter database schema. So this parameter disables record auto creation. If this set, `DB.getXX()` will throw an exception if given name does not exist, instead of creating new record (or collection)

Returns:

this builder

readOnly

public DBMakerT readOnly()

Open store in read-only mode. Any modification attempt will throw UnsupportedOperationException("Read-only")

Returns:

this builder

freeSpaceReclaimQ

public DBMakerT freeSpaceReclaimQ(int q)

Set free space reclaim Q. It is value from 0 to 10, indicating how eagerly MapDB searchs for free space inside store to reuse, before expanding store file. 0 means that no free space will be reused and store file will just grow (effectively append only). 10 means that MapDB tries really hard to reuse free space, even if it may hurt performance. Default value is 5;

Returns:

this builder

commitFileSyncDisable

public DBMakerT commitFileSyncDisable()

Disables file sync on commit. This way transactions are preserved (rollback works), but commits are not 'durable' and data may be lost if store is not properly closed. File store will get properly synced when closed. Disabling this will make commits faster.

Returns:

this builder

sizeLimit

public DBMakerT sizeLimit(double maxSize)

Sets store size limit. Disk or memory space consumed be storage should not grow over this space. Limit is not strict and does not apply to some parts such as index table. Actual store size might be 10% or more bigger.

Parameters:

maxSize - maximal store size in GB

Returns:

this builder

make

public DB make()

constructs DB using current settings

makeTxMaker

public TxMaker makeTxMaker()

makeEngine

public Engine makeEngine()

constructs Engine using current settings

propsGetInt

protected int propsGetInt(String key,
                          int defValue)

propsGetLong

protected long propsGetLong(String key,
                            long defValue)

propsGetBool

protected boolean propsGetBool(String key)

propsGetXteaEncKey

protected byte[] propsGetXteaEncKey()

JVMSupportsLargeMappedFiles

protected static boolean JVMSupportsLargeMappedFiles()

Check if large files can be mapped into memory. For example 32bit JVM can only address 2GB and large files can not be mapped, so for 32bit JVM this function returns false.

propsGetRafMode

protected int propsGetRafMode()

extendSnapshotEngine

protected Engine extendSnapshotEngine(Engine engine)

extendCacheLRU

protected Engine extendCacheLRU(Engine engine)

extendCacheWeakRef

protected Engine extendCacheWeakRef(Engine engine)

extendCacheSoftRef

protected Engine extendCacheSoftRef(Engine engine)

extendCacheHardRef

protected Engine extendCacheHardRef(Engine engine)

extendCacheHashTable

protected Engine extendCacheHashTable(Engine engine)

extendAsyncWriteEngine

protected Engine extendAsyncWriteEngine(Engine engine)

extendArgumentCheck

protected void extendArgumentCheck()

extendWrapStore

protected Engine extendWrapStore(Engine engine)

extendWrapCache

protected Engine extendWrapCache(Engine engine)

extendWrapSnapshotEngine

protected Engine extendWrapSnapshotEngine(Engine engine)

extendHeapStore

protected Engine extendHeapStore()

extendStoreAppend

protected Engine extendStoreAppend()

extendStoreDirect

protected Engine extendStoreDirect(Volume.Factory folFac)

extendStoreWAL

protected Engine extendStoreWAL(Volume.Factory folFac)

extendStoreVolumeFactory

protected Volume.Factory extendStoreVolumeFactory()

toHexa

protected static String toHexa(byte[] bb)

fromHexa

protected static byte[] fromHexa(String s)