org.apache.slider.server.avro

Class RoleHistoryWriter

java.lang.Object

org.apache.slider.server.avro.RoleHistoryWriter

public class RoleHistoryWriter
extends Object

Write out the role history to an output stream.

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	log
static int	ROLE_HISTORY_VERSION Although Avro is designed to handle some changes, we still keep a version marker in the file to catch changes that are fundamentally incompatible at the semantic level -changes that require either a different parser or get rejected outright.

Constructor Summary

Constructors

Constructor and Description
RoleHistoryWriter()

Method Summary

Methods

Modifier and Type	Method and Description
LoadedRoleHistory	attemptToReadHistory(org.apache.hadoop.fs.FileSystem fileSystem, List<org.apache.hadoop.fs.Path> paths) Iterate through the paths until one can be loaded
org.apache.hadoop.fs.Path	createHistoryFilename(org.apache.hadoop.fs.Path historyPath, long time) Create the filename for a history file
List<org.apache.hadoop.fs.Path>	findAllHistoryEntries(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dir, boolean includeEmptyFiles) Find all history entries in a dir.
LoadedRoleHistory	loadFromHistoryDir(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dir) Try to load the history from a directory -a failure to load a specific file is downgraded to a log and the next older path attempted instead
int	purgeOlderHistoryEntries(org.apache.hadoop.fs.FileSystem fileSystem, org.apache.hadoop.fs.Path keep) Delete all old history entries older than the one we want to keep.
LoadedRoleHistory	read(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path) Read a role history from a path in a filesystem
LoadedRoleHistory	read(InputStream in) Read a history, returning one that is ready to have its onThaw() method called
LoadedRoleHistory	read(String resource) Read from a resource in the classpath -used for testing
static void	sortHistoryPaths(List<org.apache.hadoop.fs.Path> paths)
long	write(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean overwrite, RoleHistory history, long savetime) Write the history information to a file
long	write(OutputStream out, RoleHistory history, long savetime) Write out the history.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected static final org.slf4j.Logger log

ROLE_HISTORY_VERSION

public static final int ROLE_HISTORY_VERSION

Although Avro is designed to handle some changes, we still keep a version marker in the file to catch changes that are fundamentally incompatible at the semantic level -changes that require either a different parser or get rejected outright.

See Also:

Constant Field Values

Constructor Detail

RoleHistoryWriter

public RoleHistoryWriter()

Method Detail

write

public long write(OutputStream out,
         RoleHistory history,
         long savetime)
           throws IOException

Write out the history. This does not update the history's dirty/savetime fields

Parameters:

out - outstream

history - history

savetime - time in millis for the save time to go in as a record

Returns:

no of records written

Throws:

IOException - IO failures

write

public long write(org.apache.hadoop.fs.FileSystem fs,
         org.apache.hadoop.fs.Path path,
         boolean overwrite,
         RoleHistory history,
         long savetime)
           throws IOException

Write the history information to a file

Parameters:

fs - filesystem

path - path

overwrite - overwrite flag

history - history

savetime - time in millis for the save time to go in as a record

Returns:

no of records written

Throws:

IOException - IO failures

createHistoryFilename

public org.apache.hadoop.fs.Path createHistoryFilename(org.apache.hadoop.fs.Path historyPath,
                                              long time)

Create the filename for a history file

Parameters:

time - time value

Returns:

a filename such that later filenames sort later in the directory

read

public LoadedRoleHistory read(InputStream in)
                       throws IOException,
                              BadConfigException

Read a history, returning one that is ready to have its onThaw() method called

Parameters:

in - input source

Returns:

no. of entries read

Throws:

IOException - problems

BadConfigException

read

public LoadedRoleHistory read(org.apache.hadoop.fs.FileSystem fs,
                     org.apache.hadoop.fs.Path path)
                       throws IOException,
                              BadConfigException

Read a role history from a path in a filesystem

Parameters:

fs - filesystem

path - path to the file

Returns:

the records read

Throws:

IOException - any problem

BadConfigException

read

public LoadedRoleHistory read(String resource)
                       throws IOException,
                              BadConfigException

Read from a resource in the classpath -used for testing

Parameters:

resource - resource

Returns:

the records read

Throws:

IOException - any problem

BadConfigException

findAllHistoryEntries

public List<org.apache.hadoop.fs.Path> findAllHistoryEntries(org.apache.hadoop.fs.FileSystem fs,
                                                    org.apache.hadoop.fs.Path dir,
                                                    boolean includeEmptyFiles)
                                                      throws IOException

Find all history entries in a dir. The dir is created if it is not already defined. The scan uses the match pattern SliderKeys.HISTORY_FILENAME_MATCH_PATTERN while dropping empty files and directories which match the pattern. The list is then sorted with a comparator that sorts on filename, relying on the filename of newer created files being later than the old ones.

Parameters:

fs - filesystem

dir - dir to scan

includeEmptyFiles - should empty files be included in the result?

Returns:

a possibly empty list

Throws:

IOException - IO problems

FileNotFoundException - if the target dir is actually a path

sortHistoryPaths

public static void sortHistoryPaths(List<org.apache.hadoop.fs.Path> paths)

attemptToReadHistory

public LoadedRoleHistory attemptToReadHistory(org.apache.hadoop.fs.FileSystem fileSystem,
                                     List<org.apache.hadoop.fs.Path> paths)
                                       throws BadConfigException

Iterate through the paths until one can be loaded

Parameters:

paths - paths to load

Returns:

the loaded history including the path -or null if all failed to load

Throws:

BadConfigException

loadFromHistoryDir

public LoadedRoleHistory loadFromHistoryDir(org.apache.hadoop.fs.FileSystem fs,
                                   org.apache.hadoop.fs.Path dir)
                                     throws IOException,
                                            BadConfigException

Try to load the history from a directory -a failure to load a specific file is downgraded to a log and the next older path attempted instead

Parameters:

fs - filesystem

dir - dir to load from

Returns:

the history loaded, including the path

Throws:

IOException - if indexing the history directory fails.

BadConfigException

purgeOlderHistoryEntries

public int purgeOlderHistoryEntries(org.apache.hadoop.fs.FileSystem fileSystem,
                           org.apache.hadoop.fs.Path keep)
                             throws IOException

Delete all old history entries older than the one we want to keep. This uses the filename ordering to determine age, not timestamps

Parameters:

fileSystem - filesystem

keep - path to keep -used in thresholding the files

Returns:

the number of files deleted

Throws:

FileNotFoundException - if the path to keep is not present (safety check to stop the entire dir being purged)

IOException - IO problems