org.apache.slider.core.launch

Class AbstractLauncher

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.slider.core.launch.AbstractLauncher

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

Direct Known Subclasses:

AppMasterLauncher, ContainerLauncher

public abstract class AbstractLauncher
extends org.apache.hadoop.conf.Configured

Launcher of applications: base class

Field Summary

Fields

Modifier and Type	Field and Description
static String	CLASSPATH
protected List<String>	commands
protected org.apache.hadoop.yarn.api.records.ContainerLaunchContext	containerLaunchContext
protected CoreFileSystem	coreFileSystem Filesystem to use for the launch
protected org.apache.hadoop.security.Credentials	credentials
protected MapOperations	env
protected Map<String,String>	envVars Env vars; set up at final launch stage
protected Map<String,org.apache.hadoop.yarn.api.records.LocalResource>	localResources
protected org.apache.hadoop.yarn.api.records.LogAggregationContext	logAggregationContext
static String	MAPREDUCE_JOB_CREDENTIALS_BINARY

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractLauncher(org.apache.hadoop.conf.Configuration conf, CoreFileSystem fs)
protected	AbstractLauncher(CoreFileSystem fs)

Method Summary

Methods

Modifier and Type	Method and Description
void	addCommand(String cmd)
void	addCommandLine(CommandLineBuilder cmd) Add a command line.
void	addCommands(List<String> commandList) Add a list of commands.
void	addLocalResource(String subpath, org.apache.hadoop.yarn.api.records.LocalResource resource)
void	addLocalResources(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> resourceMap) Add a set of local resources
org.apache.hadoop.yarn.api.records.ContainerLaunchContext	completeContainerLaunch() Complete the launch context (copy in env vars, etc).
boolean	copyEnvVars(MapOperations options) Important: the configuration must already be fully resolved in order to pick up global options Copy env vars into the launch context.
String[]	dumpEnvToString()
void	extractAmRetryCount(org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext submissionContext, Map<String,String> map) Extract the value for option yarn.resourcemanager.am.retry-count-window-ms and set it on the ApplicationSubmissionContext.
String	extractLabelExpression(Map<String,String> map) Return the label expression and if not set null
void	extractLogAggregationContext(Map<String,String> map)
void	extractResourceRequirements(org.apache.hadoop.yarn.api.records.Resource resource, Map<String,String> map) Extract any resource requirements from this component's settings.
List<String>	getCommands() Get the launch commands.
String	getCommandsAsString() Get all commands as a string, separated by ";".
org.apache.hadoop.yarn.api.records.ContainerLaunchContext	getContainerLaunchContext() Get the container.
org.apache.hadoop.security.Credentials	getCredentials() Accessor to the credentials
MapOperations	getEnv() Get the env vars to work on
Map<String,org.apache.hadoop.yarn.api.records.LocalResource>	getLocalResources() Get the map of local resources.
Map<String,ByteBuffer>	getServiceData()
protected void	propagateUsernameInInsecureCluster() This is critical for an insecure cluster -it passes down the username to YARN, and so gives the code running in containers the rights it needs to work with data.
void	putEnv(Map<String,String> map)
void	setClasspath(ClasspathConstructor classpath) Utility method to set up the classpath
void	setEnv(String var, String value)
void	submitDirectory(org.apache.hadoop.fs.Path srcDir, String destRelativeDir) Submit an entire directory

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CLASSPATH

public static final String CLASSPATH

See Also:

Constant Field Values

MAPREDUCE_JOB_CREDENTIALS_BINARY

public static final String MAPREDUCE_JOB_CREDENTIALS_BINARY

See Also:

Constant Field Values

coreFileSystem

protected final CoreFileSystem coreFileSystem

Filesystem to use for the launch

envVars

protected final Map<String,String> envVars

Env vars; set up at final launch stage

env

protected final MapOperations env

containerLaunchContext

protected final org.apache.hadoop.yarn.api.records.ContainerLaunchContext containerLaunchContext

commands

protected final List<String> commands

localResources

protected final Map<String,org.apache.hadoop.yarn.api.records.LocalResource> localResources

credentials

protected final org.apache.hadoop.security.Credentials credentials

logAggregationContext

protected org.apache.hadoop.yarn.api.records.LogAggregationContext logAggregationContext

Constructor Detail

AbstractLauncher

protected AbstractLauncher(org.apache.hadoop.conf.Configuration conf,
                CoreFileSystem fs)

AbstractLauncher

protected AbstractLauncher(CoreFileSystem fs)

Method Detail

getContainerLaunchContext

public org.apache.hadoop.yarn.api.records.ContainerLaunchContext getContainerLaunchContext()

Get the container. Until "completed", this isn't valid to launch.

Returns:

the container to launch

getEnv

public MapOperations getEnv()

Get the env vars to work on

Returns:

env vars

getCommands

public List<String> getCommands()

Get the launch commands.

Returns:

the live list of commands

getLocalResources

public Map<String,org.apache.hadoop.yarn.api.records.LocalResource> getLocalResources()

Get the map of local resources.

Returns:

the live map of local resources.

addLocalResource

public void addLocalResource(String subpath,
                    org.apache.hadoop.yarn.api.records.LocalResource resource)

addLocalResources

public void addLocalResources(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> resourceMap)

Add a set of local resources

Parameters:

resourceMap - map of name:resource to add

getServiceData

public Map<String,ByteBuffer> getServiceData()

getCredentials

public org.apache.hadoop.security.Credentials getCredentials()

Accessor to the credentials

Returns:

the credentials associated with this launcher

addCommandLine

public void addCommandLine(CommandLineBuilder cmd)

Add a command line. It is converted to a single command before being added.

Parameters:

cmd -

addCommand

public void addCommand(String cmd)

addCommands

public void addCommands(List<String> commandList)

Add a list of commands. Each element in the list becomes a single command

Parameters:

commandList - list of commands

getCommandsAsString

public String getCommandsAsString()

Get all commands as a string, separated by ";". This is for diagnostics

Returns:

a string descriptionof the commands

completeContainerLaunch

public org.apache.hadoop.yarn.api.records.ContainerLaunchContext completeContainerLaunch()
                                                                                  throws IOException

Complete the launch context (copy in env vars, etc).

Returns:

the container to launch

Throws:

IOException

propagateUsernameInInsecureCluster

protected void propagateUsernameInInsecureCluster()
                                           throws IOException

This is critical for an insecure cluster -it passes down the username to YARN, and so gives the code running in containers the rights it needs to work with data.

Throws:

IOException - problems working with current user

extractResourceRequirements

public void extractResourceRequirements(org.apache.hadoop.yarn.api.records.Resource resource,
                               Map<String,String> map)

Extract any resource requirements from this component's settings. All fields that are set will override the existing values -if unset that resource field will be left unchanged. Important: the configuration must already be fully resolved in order to pick up global options.

Parameters:

resource - resource to configure

map - map of options

extractAmRetryCount

public void extractAmRetryCount(org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext submissionContext,
                       Map<String,String> map)

Extract the value for option yarn.resourcemanager.am.retry-count-window-ms and set it on the ApplicationSubmissionContext. Use the default value if option is not set.

Parameters:

submissionContext -

map -

extractLogAggregationContext

public void extractLogAggregationContext(Map<String,String> map)

setClasspath

public void setClasspath(ClasspathConstructor classpath)

Utility method to set up the classpath

Parameters:

classpath - classpath to use

setEnv

public void setEnv(String var,
          String value)

putEnv

public void putEnv(Map<String,String> map)

copyEnvVars

public boolean copyEnvVars(MapOperations options)

Important: the configuration must already be fully resolved in order to pick up global options Copy env vars into the launch context.

dumpEnvToString

public String[] dumpEnvToString()

submitDirectory

public void submitDirectory(org.apache.hadoop.fs.Path srcDir,
                   String destRelativeDir)
                     throws IOException

Submit an entire directory

Parameters:

srcDir - src path in filesystem

destRelativeDir - relative path under destination local dir

Throws:

IOException - IO problems

extractLabelExpression

public String extractLabelExpression(Map<String,String> map)

Return the label expression and if not set null

Parameters:

map - map to look up

Returns:

extracted label or null