Package org.elasticsearch.index.engine

Class InternalEngine

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable
    public class InternalEngine
extends Engine

    • Field Detail

      • MAX_UNSAFE_AUTO_ID_TIMESTAMP_COMMIT_ID

        public static final java.lang.String MAX_UNSAFE_AUTO_ID_TIMESTAMP_COMMIT_ID
        See Also:
        Constant Field Values

    • Constructor Detail

      • InternalEngine

        public InternalEngine​(EngineConfig engineConfig)

    • Method Detail

      • restoreLocalCheckpointFromTranslog

        public void restoreLocalCheckpointFromTranslog()
                                        throws java.io.IOException
        Description copied from class: Engine
        Marks operations in the translog as completed. This is used to restore the state of the local checkpoint tracker on primary promotion.
        Specified by:
        restoreLocalCheckpointFromTranslog in class Engine
        Throws:
        java.io.IOException - if an I/O exception occurred reading the translog

      • fillSeqNoGaps

        public int fillSeqNoGaps​(long primaryTerm)
                  throws java.io.IOException
        Description copied from class: Engine
        Fills up the local checkpoints history with no-ops until the local checkpoint and the max seen sequence ID are identical.
        Specified by:
        fillSeqNoGaps in class Engine
        Parameters:
        primaryTerm - the shards primary term this engine was created for
        Returns:
        the number of no-ops added
        Throws:
        java.io.IOException

      • recoverFromTranslog

        public InternalEngine recoverFromTranslog()
                                   throws java.io.IOException
        Description copied from class: Engine
        Performs recovery from the transaction log. This operation will close the engine if the recovery fails.
        Specified by:
        recoverFromTranslog in class Engine
        Throws:
        java.io.IOException

      • skipTranslogRecovery

        public void skipTranslogRecovery()
        Description copied from class: Engine
        Do not replay translog operations, but make the engine be ready.
        Specified by:
        skipTranslogRecovery in class Engine

      • isTranslogSyncNeeded

        public boolean isTranslogSyncNeeded()
        Description copied from class: Engine
        Checks if the underlying storage sync is required.
        Specified by:
        isTranslogSyncNeeded in class Engine

      • ensureTranslogSynced

        public boolean ensureTranslogSynced​(java.util.stream.Stream<Translog.Location> locations)
                             throws java.io.IOException
        Description copied from class: Engine
        Ensures that all locations in the given stream have been written to the underlying storage.
        Specified by:
        ensureTranslogSynced in class Engine
        Throws:
        java.io.IOException

      • syncTranslog

        public void syncTranslog()
                  throws java.io.IOException
        Specified by:
        syncTranslog in class Engine
        Throws:
        java.io.IOException

      • newTranslogSnapshotFromMinSeqNo

        public Translog.Snapshot newTranslogSnapshotFromMinSeqNo​(long minSeqNo)
                                                  throws java.io.IOException
        Description copied from class: Engine
        Creates a new translog snapshot from this engine for reading translog operations whose seq# at least the provided seq#. The caller has to close the returned snapshot after finishing the reading.
        Specified by:
        newTranslogSnapshotFromMinSeqNo in class Engine
        Throws:
        java.io.IOException

      • estimateTranslogOperationsFromMinSeq

        public int estimateTranslogOperationsFromMinSeq​(long minSeqNo)
        Description copied from class: Engine
        Returns the estimated number of translog operations in this engine whose seq# at least the provided seq#.
        Specified by:
        estimateTranslogOperationsFromMinSeq in class Engine

      • getHistoryUUID

        public java.lang.String getHistoryUUID()
        Description copied from class: Engine
        returns the history uuid for the engine
        Specified by:
        getHistoryUUID in class Engine

      • getWritingBytes

        public long getWritingBytes()
        Returns how many bytes we are currently moving from indexing buffer to segments on disk
        Specified by:
        getWritingBytes in class Engine

      • assertOriginPrimarySequenceNumber

        protected boolean assertOriginPrimarySequenceNumber​(long seqNo)

      • doGenerateSeqNoForOperation

        protected long doGenerateSeqNoForOperation​(Engine.Operation operation)
        Generate the sequence number for the specified operation.
        Parameters:
        operation - the operation
        Returns:
        the sequence number

      • index

        public Engine.IndexResult index​(Engine.Index index)
                         throws java.io.IOException
        Description copied from class: Engine
        Perform document index operation on the engine
        Specified by:
        index in class Engine
        Parameters:
        index - operation to perform
        Returns:
        Engine.IndexResult containing updated translog location, version and document specific failures Note: engine level failures (i.e. persistent engine failures) are thrown
        Throws:
        java.io.IOException

      • delete

        public Engine.DeleteResult delete​(Engine.Delete delete)
                           throws java.io.IOException
        Description copied from class: Engine
        Perform document delete operation on the engine
        Specified by:
        delete in class Engine
        Parameters:
        delete - operation to perform
        Returns:
        Engine.DeleteResult containing updated translog location, version and document specific failures Note: engine level failures (i.e. persistent engine failures) are thrown
        Throws:
        java.io.IOException

      • maybePruneDeletes

        public void maybePruneDeletes()
        Description copied from class: Engine
        Tries to prune buffered deletes from the version map.
        Specified by:
        maybePruneDeletes in class Engine

      • refresh

        public void refresh​(java.lang.String source)
             throws EngineException
        Description copied from class: Engine
        Synchronously refreshes the engine for new search operations to reflect the latest changes.
        Specified by:
        refresh in class Engine
        Throws:
        EngineException

      • syncFlush

        public Engine.SyncedFlushResult syncFlush​(java.lang.String syncId,
                                          Engine.CommitId expectedCommitId)
                                   throws EngineException
        Description copied from class: Engine
        Attempts to do a special commit where the given syncID is put into the commit data. The attempt succeeds if there are not pending writes in lucene and the current point is equal to the expected one.
        Specified by:
        syncFlush in class Engine
        Parameters:
        syncId - id of this sync
        expectedCommitId - the expected value of
        Returns:
        true if the sync commit was made, false o.w.
        Throws:
        EngineException

      • shouldPeriodicallyFlush

        public boolean shouldPeriodicallyFlush()
        Description copied from class: Engine
        Checks if this engine should be flushed periodically. This check is mainly based on the uncommitted translog size and the translog flush threshold setting.
        Specified by:
        shouldPeriodicallyFlush in class Engine

      • flush

        public Engine.CommitId flush()
                      throws EngineException
        Description copied from class: Engine
        Flushes the state of the engine including the transaction log, clearing memory and persisting documents in the lucene index to disk including a potentially heavy and durable fsync operation. This operation is not going to block if another flush operation is currently running and won't write a lucene commit if nothing needs to be committed.
        Specified by:
        flush in class Engine
        Returns:
        the commit Id for the resulting commit
        Throws:
        EngineException

      • flush

        public Engine.CommitId flush​(boolean force,
                             boolean waitIfOngoing)
                      throws EngineException
        Description copied from class: Engine
        Flushes the state of the engine including the transaction log, clearing memory.
        Specified by:
        flush in class Engine
        Parameters:
        force - if true a lucene commit is executed even if no changes need to be committed.
        waitIfOngoing - if true this call will block until all currently running flushes have finished. Otherwise this call will return without blocking.
        Returns:
        the commit Id for the resulting commit
        Throws:
        EngineException

      • shouldRollTranslogGeneration

        public boolean shouldRollTranslogGeneration()
        Description copied from class: Engine
        Tests whether or not the translog generation should be rolled to a new generation. This test is based on the size of the current generation compared to the configured generation threshold size.
        Specified by:
        shouldRollTranslogGeneration in class Engine
        Returns:
        true if the current generation should be rolled to a new generation

      • forceMerge

        public void forceMerge​(boolean flush,
                       int maxNumSegments,
                       boolean onlyExpungeDeletes,
                       boolean upgrade,
                       boolean upgradeOnlyAncientSegments)
                throws EngineException,
                       java.io.IOException
        Description copied from class: Engine
        Triggers a forced merge on this engine
        Specified by:
        forceMerge in class Engine
        Throws:
        EngineException
        java.io.IOException

      • acquireLastIndexCommit

        public Engine.IndexCommitRef acquireLastIndexCommit​(boolean flushFirst)
                                             throws EngineException
        Description copied from class: Engine
        Snapshots the most recent index and returns a handle to it. If needed will try and "commit" the lucene index to make sure we have a "fresh" copy of the files to snapshot.
        Specified by:
        acquireLastIndexCommit in class Engine
        Parameters:
        flushFirst - indicates whether the engine should flush before returning the snapshot
        Throws:
        EngineException

      • maybeFailEngine

        protected boolean maybeFailEngine​(java.lang.String source,
                                  java.lang.Exception e)
        Description copied from class: Engine
        Check whether the engine should be failed
        Overrides:
        maybeFailEngine in class Engine

      • getLastCommittedSegmentInfos

        protected org.apache.lucene.index.SegmentInfos getLastCommittedSegmentInfos()
        Specified by:
        getLastCommittedSegmentInfos in class Engine

      • getIndexBufferRAMBytesUsed

        public long getIndexBufferRAMBytesUsed()
        Description copied from class: Engine
        How much heap is used that would be freed by a refresh. Note that this may throw AlreadyClosedException.
        Specified by:
        getIndexBufferRAMBytesUsed in class Engine

      • segments

        public java.util.List<Segment> segments​(boolean verbose)
        Description copied from class: Engine
        The list of segments in the engine.
        Specified by:
        segments in class Engine

      • closeNoLock

        protected final void closeNoLock​(java.lang.String reason,
                                 java.util.concurrent.CountDownLatch closedLatch)
        Closes the engine without acquiring the write lock. This should only be called while the write lock is hold or in a disaster condition ie. if the engine is failed.
        Specified by:
        closeNoLock in class Engine

      • acquireSearcher

        public Engine.Searcher acquireSearcher​(java.lang.String source,
                                       Engine.SearcherScope scope)
        Description copied from class: Engine
        Returns a new searcher instance. The consumer of this API is responsible for releasing the returned searcher in a safe manner, preferably in a try/finally block.
        Specified by:
        acquireSearcher in class Engine
        Parameters:
        source - the source API or routing that triggers this searcher acquire
        scope - the scope of this searcher ie. if the searcher will be used for get or search purposes
        See Also:
        Engine.Searcher.close()

      • getIndexThrottleTimeInMillis

        public long getIndexThrottleTimeInMillis()
        Description copied from class: Engine
        Returns the number of milliseconds this engine was under index throttling.
        Specified by:
        getIndexThrottleTimeInMillis in class Engine

      • commitIndexWriter

        protected void commitIndexWriter​(org.apache.lucene.index.IndexWriter writer,
                                 Translog translog,
                                 @Nullable
                                 java.lang.String syncId)
                          throws java.io.IOException
        Commits the specified index writer.
        Parameters:
        writer - the index writer to commit
        translog - the translog
        syncId - the sync flush ID (null if not committing a synced flush)
        Throws:
        java.io.IOException - if an I/O exception occurs committing the specfied writer

      • getLastSyncedGlobalCheckpoint

        public long getLastSyncedGlobalCheckpoint()
        Description copied from class: Engine
        Returns the latest global checkpoint value that has been persisted in the underlying storage (i.e. translog's checkpoint)
        Specified by:
        getLastSyncedGlobalCheckpoint in class Engine

      • getLocalCheckpoint

        public long getLocalCheckpoint()
        Specified by:
        getLocalCheckpoint in class Engine
        Returns:
        the local checkpoint for this Engine

      • waitForOpsToComplete

        public void waitForOpsToComplete​(long seqNo)
                          throws java.lang.InterruptedException
        Description copied from class: Engine
        Waits for all operations up to the provided sequence number to complete.
        Specified by:
        waitForOpsToComplete in class Engine
        Parameters:
        seqNo - the sequence number that the checkpoint must advance to before this method returns
        Throws:
        java.lang.InterruptedException - if the thread was interrupted while blocking on the condition

      • resetLocalCheckpoint

        public void resetLocalCheckpoint​(long localCheckpoint)
        Description copied from class: Engine
        Reset the local checkpoint in the tracker to the given local checkpoint
        Specified by:
        resetLocalCheckpoint in class Engine
        Parameters:
        localCheckpoint - the new checkpoint to be set

      • getSeqNoStats

        public SeqNoStats getSeqNoStats​(long globalCheckpoint)
        Specified by:
        getSeqNoStats in class Engine
        Returns:
        a SeqNoStats object, using local state and the supplied global checkpoint

      • isRecovering

        public boolean isRecovering()
        Description copied from class: Engine
        Returns true iff this engine is currently recovering from translog.
        Overrides:
        isRecovering in class Engine