java.lang.Object

org.elasticsearch.common.component.AbstractLifecycleComponent

org.elasticsearch.repositories.blobstore.BlobStoreRepository

All Implemented Interfaces:: java.io.Closeable, java.lang.AutoCloseable, LifecycleComponent, Releasable, Repository

Direct Known Subclasses:: FsRepository

public abstract class BlobStoreRepository
extends AbstractLifecycleComponent
implements Repository

BlobStore - based implementation of Snapshot Repository

This repository works with any BlobStore implementation. The blobStore could be (and preferred) lazy initialized in createBlobStore().

For in depth documentation on how exactly implementations of this class interact with the snapshot functionality please refer to the documentation of the package org.elasticsearch.repositories.blobstore.

Nested Class Summary

Nested classes/interfaces inherited from interface org.elasticsearch.repositories.Repository
Repository.Factory

Field Summary

Fields
Modifier and Type	Field	Description
`static Setting<java.lang.Boolean>`	`ALLOW_CONCURRENT_MODIFICATION`	When set to `true`, `bestEffortConsistency` will be set to `true` and concurrent modifications of the repository contents will not result in the repository being marked as corrupted.
`static Setting<java.lang.Boolean>`	`COMPRESS_SETTING`	When set to true metadata files are stored in compressed format.
`static java.lang.String`	`INDEX_FILE_PREFIX`
`static java.lang.String`	`INDEX_LATEST_BLOB`
`protected RepositoryMetaData`	`metadata`
`static java.lang.String`	`METADATA_NAME_FORMAT`
`static java.lang.String`	`METADATA_PREFIX`
`protected NamedXContentRegistry`	`namedXContentRegistry`
`static java.lang.String`	`SNAPSHOT_CODEC`
`static java.lang.String`	`SNAPSHOT_NAME_FORMAT`
`static java.lang.String`	`SNAPSHOT_PREFIX`
`protected ChecksumBlobStoreFormat<SnapshotInfo>`	`snapshotFormat`
`protected ThreadPool`	`threadPool`

Fields inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent

lifecycle

Constructor Summary

Constructors
Modifier	Constructor	Description
`protected`	`BlobStoreRepository(RepositoryMetaData metadata, boolean compress, NamedXContentRegistry namedXContentRegistry, ClusterService clusterService)`	Constructs new BlobStoreRepository

Method Summary

Modifier and Type	Method	Description
`protected void`	`assertSnapshotOrGenericThread()`
`abstract BlobPath`	`basePath()`	Returns base path of the repository
`protected BlobContainer`	`blobContainer()`	maintains single lazy instance of `BlobContainer`
`BlobStore`	`blobStore()`	Maintains single lazy instance of `BlobStore`.
`protected ByteSizeValue`	`chunkSize()`	Returns data file chunk size.
`void`	`cleanup(long repositoryStateId, boolean writeShardGens, ActionListener<RepositoryCleanupResult> listener)`	Runs cleanup actions on the repository.
`protected abstract BlobStore`	`createBlobStore()`	Creates new BlobStore to read and write data.
`void`	`deleteSnapshot(SnapshotId snapshotId, long repositoryStateId, boolean writeShardGens, ActionListener<java.lang.Void> listener)`	Deletes snapshot
`protected void`	`doClose()`
`protected void`	`doStart()`
`protected void`	`doStop()`
`void`	`endVerification(java.lang.String seed)`	Called at the end of repository verification process.
`void`	`finalizeSnapshot(SnapshotId snapshotId, ShardGenerations shardGenerations, long startTime, java.lang.String failure, int totalShards, java.util.List<SnapshotShardFailure> shardFailures, long repositoryStateId, boolean includeGlobalState, MetaData clusterMetaData, java.util.Map<java.lang.String,java.lang.Object> userMetadata, boolean writeShardGens, ActionListener<SnapshotInfo> listener)`	Finalizes snapshotting process
`protected BlobStore`	`getBlobStore()`
`RepositoryMetaData`	`getMetadata()`	Returns metadata about this repository.
`void`	`getRepositoryData(ActionListener<RepositoryData> listener)`	Returns a `RepositoryData` to describe the data in the repository, including the snapshots and the indices across all snapshots found in the repository.
`long`	`getRestoreThrottleTimeInNanos()`	Returns restore throttle time in nanoseconds
`IndexShardSnapshotStatus`	`getShardSnapshotStatus(SnapshotId snapshotId, IndexId indexId, ShardId shardId)`	Retrieve shard snapshot status for the stored snapshot
`MetaData`	`getSnapshotGlobalMetaData(SnapshotId snapshotId)`	Returns global metadata associated with the snapshot.
`IndexMetaData`	`getSnapshotIndexMetaData(SnapshotId snapshotId, IndexId index)`	Returns the index metadata associated with the snapshot.
`SnapshotInfo`	`getSnapshotInfo(SnapshotId snapshotId)`	Reads snapshot description from repository.
`long`	`getSnapshotThrottleTimeInNanos()`	Returns snapshot throttle time in nanoseconds
`void`	`initializeSnapshot(SnapshotId snapshotId, java.util.List<IndexId> indices, MetaData clusterMetaData)`	Starts snapshotting process
`protected boolean`	`isCompress()`	Returns true if metadata and snapshot files should be compressed
`boolean`	`isReadOnly()`	Returns true if the repository supports only read operations
`void`	`restoreShard(Store store, SnapshotId snapshotId, IndexId indexId, ShardId snapshotShardId, RecoveryState recoveryState, ActionListener<java.lang.Void> listener)`	Restores snapshot of the shard.
`void`	`snapshotShard(Store store, MapperService mapperService, SnapshotId snapshotId, IndexId indexId, org.apache.lucene.index.IndexCommit snapshotIndexCommit, IndexShardSnapshotStatus snapshotStatus, boolean writeShardGens, ActionListener<java.lang.String> listener)`	Creates a snapshot of the shard based on the index commit point.
`java.lang.String`	`startVerification()`	Verifies repository on the master node and returns the verification token.
`ThreadPool`	`threadPool()`
`java.lang.String`	`toString()`
`void`	`updateState(ClusterState state)`	Update the repository with the incoming cluster state.
`void`	`verify(java.lang.String seed, DiscoveryNode localNode)`	Verifies repository settings on data node.
`protected void`	`writeIndexGen(RepositoryData repositoryData, long expectedGen, boolean writeShardGens, ActionListener<java.lang.Void> listener)`	Writing a new index generation is a three step process.

Methods inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent

addLifecycleListener, close, lifecycleState, removeLifecycleListener, start, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.elasticsearch.common.component.LifecycleComponent

addLifecycleListener, lifecycleState, removeLifecycleListener, start, stop

Methods inherited from interface org.elasticsearch.common.lease.Releasable

close

Field Details
- metadata
  
  protected volatile RepositoryMetaData metadata
- namedXContentRegistry
  
  protected final NamedXContentRegistry namedXContentRegistry
- threadPool
  
  protected final ThreadPool threadPool
- SNAPSHOT_PREFIX
  
  public static final java.lang.String SNAPSHOT_PREFIX
  
  See Also:
  
  Constant Field Values
- SNAPSHOT_CODEC
  
  public static final java.lang.String SNAPSHOT_CODEC
  
  See Also:
  
  Constant Field Values
- INDEX_FILE_PREFIX
  
  public static final java.lang.String INDEX_FILE_PREFIX
  
  See Also:
  
  Constant Field Values
- INDEX_LATEST_BLOB
  
  public static final java.lang.String INDEX_LATEST_BLOB
  
  See Also:
  
  Constant Field Values
- METADATA_PREFIX
  
  public static final java.lang.String METADATA_PREFIX
  
  See Also:
  
  Constant Field Values
- METADATA_NAME_FORMAT
  
  public static final java.lang.String METADATA_NAME_FORMAT
  
  See Also:
  
  Constant Field Values
- SNAPSHOT_NAME_FORMAT
  
  public static final java.lang.String SNAPSHOT_NAME_FORMAT
  
  See Also:
  
  Constant Field Values
- COMPRESS_SETTING
  
  public static final Setting<java.lang.Boolean> COMPRESS_SETTING
  
  When set to true metadata files are stored in compressed format. This setting doesn’t affect index files that are already compressed by default. Changing the setting does not invalidate existing files since reads do not observe the setting, instead they examine the file to see if it is compressed or not.
- ALLOW_CONCURRENT_MODIFICATION
  
  public static final Setting<java.lang.Boolean> ALLOW_CONCURRENT_MODIFICATION
  
  When set to true, bestEffortConsistency will be set to true and concurrent modifications of the repository contents will not result in the repository being marked as corrupted. Note: This setting is intended as a backwards compatibility solution for 7.x and will go away in 8.
- snapshotFormat
  
  protected ChecksumBlobStoreFormat<SnapshotInfo> snapshotFormat
Constructor Details
- BlobStoreRepository
  
  protected BlobStoreRepository(RepositoryMetaData metadata, boolean compress, NamedXContentRegistry namedXContentRegistry, ClusterService clusterService)
  
  Constructs new BlobStoreRepository
  
  Parameters:
  
  metadata - The metadata for this repository including name and settings
  
  clusterService - ClusterService
Method Details
- doStart
  
  protected void doStart()
  
  Specified by:
  
  doStart in class AbstractLifecycleComponent
- doStop
  
  protected void doStop()
  
  Specified by:
  
  doStop in class AbstractLifecycleComponent
- doClose
  
  protected void doClose()
  
  Specified by:
  
  doClose in class AbstractLifecycleComponent
- updateState
  
  public void updateState(ClusterState state)
  
  Description copied from interface: Repository
  
  Update the repository with the incoming cluster state. This method is invoked from RepositoriesService.applyClusterState(org.elasticsearch.cluster.ClusterChangedEvent) and thus the same semantics as with ClusterStateApplier.applyClusterState(org.elasticsearch.cluster.ClusterChangedEvent) apply for the ClusterState that is passed here.
  
  Specified by:
  
  updateState in interface Repository
  
  Parameters:
  
  state - new cluster state
- threadPool
  
  public ThreadPool threadPool()
- getBlobStore
  
  protected BlobStore getBlobStore()
- blobContainer
  
  protected BlobContainer blobContainer()
  
  maintains single lazy instance of BlobContainer
- blobStore
  
  public BlobStore blobStore()
  
  Maintains single lazy instance of BlobStore. Public for testing.
- createBlobStore
  
  protected abstract BlobStore createBlobStore() throws java.lang.Exception
  
  Creates new BlobStore to read and write data.
  
  Throws:
  
  java.lang.Exception
- basePath
  
  public abstract BlobPath basePath()
  
  Returns base path of the repository
- isCompress
  
  protected final boolean isCompress()
  
  Returns true if metadata and snapshot files should be compressed
  
  Returns:
  
  true if compression is needed
- chunkSize
  
  protected ByteSizeValue chunkSize()
  
  Returns data file chunk size.
  This method should return null if no chunking is needed.
  
  Returns:
  
  chunk size
- getMetadata
  
  public RepositoryMetaData getMetadata()
  
  Description copied from interface: Repository
  
  Returns metadata about this repository.
  
  Specified by:
  
  getMetadata in interface Repository
- initializeSnapshot
  
  public void initializeSnapshot(SnapshotId snapshotId, java.util.List<IndexId> indices, MetaData clusterMetaData)
  
  Description copied from interface: Repository
  
  Starts snapshotting process
  
  Specified by:
  
  initializeSnapshot in interface Repository
  
  Parameters:
  
  snapshotId - snapshot id
  
  indices - list of indices to be snapshotted
  
  clusterMetaData - cluster metadata
- deleteSnapshot
  
  public void deleteSnapshot(SnapshotId snapshotId, long repositoryStateId, boolean writeShardGens, ActionListener<java.lang.Void> listener)
  
  Description copied from interface: Repository
  
  Deletes snapshot
  
  Specified by:
  
  deleteSnapshot in interface Repository
  
  Parameters:
  
  snapshotId - snapshot id
  
  repositoryStateId - the unique id identifying the state of the repository when the snapshot deletion began
  
  writeShardGens - if shard generations should be written to the repository
  
  listener - completion listener
- cleanup
  
  public void cleanup(long repositoryStateId, boolean writeShardGens, ActionListener<RepositoryCleanupResult> listener)
  Runs cleanup actions on the repository. Increments the repository state id by one before executing any modifications on the repository. TODO: Add shard level cleanups
  
  Deleting stale indices cleanupStaleIndices(java.util.Map<java.lang.String, org.elasticsearch.common.blobstore.BlobContainer>, java.util.Set<java.lang.String>)
  
  Deleting unreferenced root level blobs cleanupStaleRootFiles(java.util.List<java.lang.String>)
  Parameters:
  
  repositoryStateId - Current repository state id
  
  writeShardGens - If shard generations should be written to the repository
  
  listener - Listener to complete when done
- finalizeSnapshot
  
  public void finalizeSnapshot(SnapshotId snapshotId, ShardGenerations shardGenerations, long startTime, java.lang.String failure, int totalShards, java.util.List<SnapshotShardFailure> shardFailures, long repositoryStateId, boolean includeGlobalState, MetaData clusterMetaData, java.util.Map<java.lang.String,java.lang.Object> userMetadata, boolean writeShardGens, ActionListener<SnapshotInfo> listener)
  
  Description copied from interface: Repository
  
  Finalizes snapshotting process
  This method is called on master after all shards are snapshotted.
  
  Specified by:
  
  finalizeSnapshot in interface Repository
  
  Parameters:
  
  snapshotId - snapshot id
  
  shardGenerations - updated shard generations
  
  startTime - start time of the snapshot
  
  failure - global failure reason or null
  
  totalShards - total number of shards
  
  shardFailures - list of shard failures
  
  repositoryStateId - the unique id identifying the state of the repository when the snapshot began
  
  includeGlobalState - include cluster global state
  
  clusterMetaData - cluster metadata
  
  userMetadata - user metadata
  
  writeShardGens - if shard generations should be written to the repository
  
  listener - listener to be called on completion of the snapshot
- getSnapshotInfo
  
  public SnapshotInfo getSnapshotInfo(SnapshotId snapshotId)
  
  Description copied from interface: Repository
  
  Reads snapshot description from repository.
  
  Specified by:
  
  getSnapshotInfo in interface Repository
  
  Parameters:
  
  snapshotId - snapshot id
  
  Returns:
  
  information about snapshot
- getSnapshotGlobalMetaData
  
  public MetaData getSnapshotGlobalMetaData(SnapshotId snapshotId)
  
  Description copied from interface: Repository
  
  Returns global metadata associated with the snapshot.
  
  Specified by:
  
  getSnapshotGlobalMetaData in interface Repository
  
  Parameters:
  
  snapshotId - the snapshot id to load the global metadata from
  
  Returns:
  
  the global metadata about the snapshot
- getSnapshotIndexMetaData
  
  public IndexMetaData getSnapshotIndexMetaData(SnapshotId snapshotId, IndexId index) throws java.io.IOException
  
  Description copied from interface: Repository
  
  Returns the index metadata associated with the snapshot.
  
  Specified by:
  
  getSnapshotIndexMetaData in interface Repository
  
  Parameters:
  
  snapshotId - the snapshot id to load the index metadata from
  
  index - the IndexId to load the metadata from
  
  Returns:
  
  the index metadata about the given index for the given snapshot
  
  Throws:
  
  java.io.IOException
- getSnapshotThrottleTimeInNanos
  
  public long getSnapshotThrottleTimeInNanos()
  
  Description copied from interface: Repository
  
  Returns snapshot throttle time in nanoseconds
  
  Specified by:
  
  getSnapshotThrottleTimeInNanos in interface Repository
- getRestoreThrottleTimeInNanos
  
  public long getRestoreThrottleTimeInNanos()
  
  Description copied from interface: Repository
  
  Returns restore throttle time in nanoseconds
  
  Specified by:
  
  getRestoreThrottleTimeInNanos in interface Repository
- assertSnapshotOrGenericThread
  
  protected void assertSnapshotOrGenericThread()
- startVerification
  
  public java.lang.String startVerification()
  
  Description copied from interface: Repository
  
  Verifies repository on the master node and returns the verification token.
  If the verification token is not null, it's passed to all data nodes for verification. If it's null - no additional verification is required
  
  Specified by:
  
  startVerification in interface Repository
  
  Returns:
  
  verification token that should be passed to all Index Shard Repositories for additional verification or null
- endVerification
  
  public void endVerification(java.lang.String seed)
  
  Description copied from interface: Repository
  
  Called at the end of repository verification process.
  This method should perform all necessary cleanup of the temporary files created in the repository
  
  Specified by:
  
  endVerification in interface Repository
  
  Parameters:
  
  seed - verification request generated by Repository.startVerification() command
- getRepositoryData
  
  public void getRepositoryData(ActionListener<RepositoryData> listener)
  
  Description copied from interface: Repository
  
  Returns a RepositoryData to describe the data in the repository, including the snapshots and the indices across all snapshots found in the repository. Throws a RepositoryException if there was an error in reading the data.
  
  Specified by:
  
  getRepositoryData in interface Repository
- isReadOnly
  
  public boolean isReadOnly()
  
  Description copied from interface: Repository
  
  Returns true if the repository supports only read operations
  
  Specified by:
  
  isReadOnly in interface Repository
  
  Returns:
  
  true if the repository is read/only
- writeIndexGen
  
  protected void writeIndexGen(RepositoryData repositoryData, long expectedGen, boolean writeShardGens, ActionListener<java.lang.Void> listener)
  
  Writing a new index generation is a three step process. First, the RepositoryMetaData entry for this repository is set into a pending state by incrementing its pending generation P while its safe generation N remains unchanged. Second, the updated RepositoryData is written to generation P + 1. Lastly, the RepositoryMetaData entry for this repository is updated to the new generation P + 1 and thus pending and safe generation are set to the same value marking the end of the update of the repository data.
  
  Parameters:
  
  repositoryData - RepositoryData to write
  
  expectedGen - expected repository generation at the start of the operation
  
  writeShardGens - whether to write ShardGenerations to the new RepositoryData blob
  
  listener - completion listener
- snapshotShard
  
  public void snapshotShard(Store store, MapperService mapperService, SnapshotId snapshotId, IndexId indexId, org.apache.lucene.index.IndexCommit snapshotIndexCommit, IndexShardSnapshotStatus snapshotStatus, boolean writeShardGens, ActionListener<java.lang.String> listener)
  
  Description copied from interface: Repository
  
  Creates a snapshot of the shard based on the index commit point.
  The index commit point can be obtained by using Engine.acquireLastIndexCommit(boolean) method. Repository implementations shouldn't release the snapshot index commit point. It is done by the method caller.
  As snapshot process progresses, implementation of this method should update IndexShardSnapshotStatus object and check IndexShardSnapshotStatus.isAborted() to see if the snapshot process should be aborted.
  
  Specified by:
  
  snapshotShard in interface Repository
  
  Parameters:
  
  store - store to be snapshotted
  
  mapperService - the shards mapper service
  
  snapshotId - snapshot id
  
  indexId - id for the index being snapshotted
  
  snapshotIndexCommit - commit point
  
  snapshotStatus - snapshot status
  
  listener - listener invoked on completion
- restoreShard
  
  public void restoreShard(Store store, SnapshotId snapshotId, IndexId indexId, ShardId snapshotShardId, RecoveryState recoveryState, ActionListener<java.lang.Void> listener)
  
  Description copied from interface: Repository
  
  Restores snapshot of the shard.
  The index can be renamed on restore, hence different shardId and snapshotShardId are supplied.
  
  Specified by:
  
  restoreShard in interface Repository
  
  Parameters:
  
  store - the store to restore the index into
  
  snapshotId - snapshot id
  
  indexId - id of the index in the repository from which the restore is occurring
  
  snapshotShardId - shard id (in the snapshot)
  
  recoveryState - recovery state
  
  listener - listener to invoke once done
- getShardSnapshotStatus
  
  public IndexShardSnapshotStatus getShardSnapshotStatus(SnapshotId snapshotId, IndexId indexId, ShardId shardId)
  
  Description copied from interface: Repository
  
  Retrieve shard snapshot status for the stored snapshot
  
  Specified by:
  
  getShardSnapshotStatus in interface Repository
  
  Parameters:
  
  snapshotId - snapshot id
  
  indexId - the snapshotted index id for the shard to get status for
  
  shardId - shard id
  
  Returns:
  
  snapshot status
- verify
  
  public void verify(java.lang.String seed, DiscoveryNode localNode)
  
  Description copied from interface: Repository
  
  Verifies repository settings on data node.
  
  Specified by:
  
  verify in interface Repository
  
  Parameters:
  
  seed - value returned by Repository.startVerification()
  
  localNode - the local node information, for inclusion in verification errors
- toString
  
  public java.lang.String toString()
  
  Overrides:
  
  toString in class java.lang.Object

Class BlobStoreRepository

Nested Class Summary

Nested classes/interfaces inherited from interface org.elasticsearch.repositories.Repository