org.apache.slider.providers.slideram

Class SliderAMProviderService

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.slider.server.services.workflow.WorkflowSequenceService

org.apache.slider.providers.AbstractProviderService

org.apache.slider.providers.slideram.SliderAMProviderService

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.service.Service, org.apache.hadoop.service.ServiceStateChangeListener, SliderKeys, SliderXmlConfKeys, ExitCodeProvider, AgentKeys, ProviderCore, ProviderService, RMOperationHandlerActions, ServiceParent

public class SliderAMProviderService
extends AbstractProviderService
implements ProviderCore, AgentKeys, SliderKeys

Exists just to move some functionality out of AppMaster into a peer class of the actual service provider doing the real work

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields inherited from class org.apache.slider.providers.AbstractProviderService

amState, amWebAPI, queueAccess, restOps, yarnRegistry

Fields inherited from interface org.apache.slider.providers.agent.AgentKeys

ADDON_DEFINITION_DIR, ADDON_FOR_ALL_COMPONENTS, ADDON_PREFIX, ADDONS, AGENT_CONF, AGENT_CONFIG_FILE, AGENT_INSTALL_DIR, AGENT_INSTANCE_DEBUG_DATA, AGENT_MAIN_SCRIPT, AGENT_MAIN_SCRIPT_ROOT, AGENT_OUT_FILE, AGENT_VERSION, AGENT_VERSION_FILE, APP_DEF, APP_DEFINITION_DIR, APP_HOME, APP_ROOT, ARG_DEBUG, ARG_HOST, ARG_LABEL, ARG_PORT, ARG_SECURED_PORT, ARG_ZOOKEEPER_QUORUM, ARG_ZOOKEEPER_REGISTRY_PATH, CERT_FILE_LOCALIZATION_PATH, CONF_RESOURCE, CONTROLLER_URL, CREATE_DEF_ZK_NODE, HEARTBEAT_MONITOR_INTERVAL, INFRA_RUN_SECURITY_DIR, JAVA_HOME, KEY_AGENT_TWO_WAY_SSL_ENABLED, KEY_CONTAINER_LAUNCH_DELAY, PACKAGE_LIST, PACKAGE_PATH, PROVIDER_AGENT, PYTHON_EXE, ROLE_NODE, RUNAS_USER, SCRIPT_PATH, SERVICE_NAME, SYSTEM_CONFIGS, TEST_RELAX_VERIFICATION, WAIT_HEARTBEAT

Fields inherited from interface org.apache.slider.common.SliderKeys

ADDONS_DIR, AGENT_TAR, AM_DIR_PREFIX, AM_FILTER_NAME, APP_CONTAINER_HEARTBEAT_INTERVAL_SEC, APP_CONTAINER_RELEASE_TIMEOUT, APP_DEF_DIR, APP_TYPE, APP_VERSION, APP_VERSION_UNKNOWN, CLUSTER_DIRECTORY, COMP_KEYSTORE_PASSWORD_ALIAS_DEFAULT, COMP_KEYSTORE_PASSWORD_ALIAS_KEY, COMP_KEYSTORE_PASSWORD_PROPERTY_KEY, COMP_STORES_REQUIRED_KEY, COMP_TRUSTSTORE_PASSWORD_ALIAS_DEFAULT, COMP_TRUSTSTORE_PASSWORD_ALIAS_KEY, COMP_TRUSTSTORE_PASSWORD_PROPERTY_KEY, COMPONENT_AM, CRT_FILE_NAME, CRT_PASS_FILE_NAME, CSR_FILE_NAME, DATA_DIR_NAME, DEFAULT_APP_PKG, DEFAULT_GC_OPTS, DEFAULT_JVM_HEAP, DEFAULT_YARN_MEMORY, GENERATED_CONF_DIR_NAME, GSON_JAR, HADOOP_PROXY_USER, HADOOP_USER_NAME, HDP_VERSION_PROP_NAME, HISTORY_DIR_NAME, HISTORY_FILENAME_CREATION_PATTERN, HISTORY_FILENAME_GLOB_PATTERN, HISTORY_FILENAME_MATCH_PATTERN, HISTORY_FILENAME_PREFIX, HISTORY_FILENAME_SUFFIX, INFRA_DIR_NAME, JCOMMANDER_JAR, JVM_ENABLE_ASSERTIONS, JVM_ENABLE_SYSTEM_ASSERTIONS, JVM_FORCE_IPV4, JVM_JAVA_HEADLESS, KEY_ALLOWED_PORT_RANGE, KEY_AM_ALLOWED_PORT_RANGE, KEY_FILE_NAME, KEYSTORE_FILE_NAME, KEYTAB_DIR, LOCAL_TARBALL_INSTALL_SUBDIR, LOG4J_PROP_FILENAME, LOG4J_SERVER_PROP_FILENAME, PACKAGE_DIRECTORY, PASS_LEN, PROPAGATE_RESOURCE_OPTION, PROPAGATED_CONF_DIR_NAME, PROPERTY_CONF_DIR, PROPERTY_LIB_DIR, PYTHONPATH, ROLE_AM_PRIORITY_INDEX, SECURITY_DIR, SLIDER_BASE_DIRECTORY, SLIDER_CLASSPATH_EXTRA, SLIDER_CLIENT_XML, SLIDER_DEPENDENCY_DIR, SLIDER_DEPENDENCY_DIR_PERMISSIONS, SLIDER_DEPENDENCY_HDP_PARENT_DIR, SLIDER_DEPENDENCY_LOCALIZED_DIR_LINK, SLIDER_DEPENDENCY_TAR_GZ_FILE_EXT, SLIDER_DEPENDENCY_TAR_GZ_FILE_NAME, SLIDER_JAR, SLIDER_JVM_OPTS, SLIDER_PASSPHRASE, SLIDER_SERVER_XML, SLIDER_XML, SNAPSHOT_CONF_DIR_NAME, STDERR_AM, STDOUT_AM, SUBMITTED_CONF_DIR, SYSPROP_LOG_DIR, SYSPROP_LOG4J_CONFIGURATION, TMP_DIR_PREFIX, TMP_LOGDIR_PREFIX, YARN_CONTAINER_PATH

Fields inherited from interface org.apache.slider.common.SliderXmlConfKeys

CLUSTER_DIRECTORY_PERMISSIONS, DATA_DIRECTORY_PERMISSIONS, DEFAULT_CLUSTER_DIRECTORY_PERMISSIONS, DEFAULT_DATA_DIRECTORY_PERMISSIONS, DEFAULT_YARN_QUEUE, DEFAULT_YARN_QUEUE_PRIORITY, DFS_DATANODE_KERBEROS_PRINCIPAL_KEY, DFS_NAMENODE_DELEGATION_KEY_UPDATE_INTERVAL_DEFAULT, DFS_NAMENODE_DELEGATION_KEY_UPDATE_INTERVAL_KEY, DFS_NAMENODE_DELEGATION_TOKEN_ALWAYS_USE_DEFAULT, DFS_NAMENODE_DELEGATION_TOKEN_ALWAYS_USE_KEY, DFS_NAMENODE_DELEGATION_TOKEN_MAX_LIFETIME_DEFAULT, DFS_NAMENODE_DELEGATION_TOKEN_MAX_LIFETIME_KEY, DFS_NAMENODE_DELEGATION_TOKEN_RENEW_INTERVAL_DEFAULT, DFS_NAMENODE_DELEGATION_TOKEN_RENEW_INTERVAL_KEY, DFS_NAMENODE_DU_RESERVED_KEY, DFS_NAMENODE_KERBEROS_PRINCIPAL_KEY, DFS_NAMENODE_KEYTAB_FILE_KEY, FS_DEFAULT_NAME_CLASSIC, HADOOP_HTTP_FILTER_INITIALIZERS, IPC_CLIENT_FALLBACK_TO_SIMPLE_AUTH, IPC_CLIENT_RETRY_POLICY_ENABLED_DEFAULT, IPC_CLIENT_RETRY_POLICY_SPEC_DEFAULT, KEY_AM_KEYTAB_LOCAL_PATH, KEY_AM_LAUNCH_ENV, KEY_AM_LOGIN_KEYTAB_NAME, KEY_AM_LOGIN_KEYTAB_REQUIRED, KEY_AM_RESTART_LIMIT, KEY_HDFS_KEYTAB_DIR, KEY_IPC_CLIENT_RETRY_POLICY_ENABLED, KEY_IPC_CLIENT_RETRY_POLICY_SPEC, KEY_KERBEROS_PRINCIPAL, KEY_KEYSTORE_LOCATION, KEY_KEYTAB_PRINCIPAL, KEY_PROTOCOL_ACL, KEY_PROVIDER, KEY_SECURITY_ENABLED, KEY_SLIDER_AM_DEPENDENCY_CHECKS_DISABLED, KEY_SLIDER_BASE_PATH, KEY_TEMPLATE_ORIGIN, KEY_YARN_QUEUE, KEY_YARN_QUEUE_PRIORITY, PREFIX_PROVIDER, PYTHON_EXECUTABLE_PATH, REGISTRY_PATH, REGISTRY_ZK_QUORUM, X_DEV_INSECURE_DEFAULT, X_DEV_INSECURE_REQUIRED, X_DEV_INSECURE_WS

Constructor Summary

Constructors

Constructor and Description
SliderAMProviderService()

Method Summary

Methods

Modifier and Type	Method and Description
void	applyInitialRegistryDefinitions(URL amWebURI, URL agentOpsURI, URL agentStatusURI, org.apache.hadoop.registry.client.types.ServiceRecord serviceRecord) Prior to going live -register the initial service registry data
void	buildContainerLaunchContext(ContainerLauncher containerLauncher, AggregateConf instanceDefinition, org.apache.hadoop.yarn.api.records.Container container, String role, SliderFileSystem sliderFileSystem, org.apache.hadoop.fs.Path generatedConfPath, MapOperations resourceComponent, MapOperations appComponent, org.apache.hadoop.fs.Path containerTmpDirPath) Set up the entire container launch context
String	getHumanName() Get a human friendly name for web UIs and messages
List<ProviderRole>	getRoles()
org.apache.hadoop.conf.Configuration	loadProviderConfigurationInformation(File confDir) Load a specific XML configuration file for the provider config
void	validateInstanceDefinition(AggregateConf instanceDefinition) Verify that an instance definition is considered valid by the provider

Methods inherited from class org.apache.slider.providers.AbstractProviderService

addContainerRequest, bind, bindToYarnRegistry, buildEndpointDetails, buildMonitorDetails, buildProcess, buildProviderStatus, cancelContainerRequests, cancelSingleRequest, createContainerReleaseSelector, exec, execute, getAgentRestOperations, getAmState, getConf, getExitCode, getFPSFromParentService, getQueueAccess, getYarnRegistry, initializeApplicationConfiguration, isSupportedRole, latestProcess, loadProviderConfigurationInformation, maybeStartCommandSequence, notifyContainerCompleted, queueCommand, rebuildContainerDetails, releaseAssignedContainer, setAgentRestOperations, setAmState, validateApplicationConfiguration

Methods inherited from class org.apache.slider.server.services.workflow.WorkflowSequenceService

addService, getActiveService, getPreviousService, getServices, onServiceCompleted, serviceStart, serviceStop, setStopIfNoChildServicesAtStartup, startNextService, stateChanged, toString

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceInit, setConfig, start, stop, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.slider.providers.ProviderCore

getConf, getName

Methods inherited from interface org.apache.hadoop.service.Service

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, registerServiceListener, start, stop, unregisterServiceListener, waitForServiceToStop

Constructor Detail

SliderAMProviderService

public SliderAMProviderService()

Method Detail

getHumanName

public String getHumanName()

Description copied from interface: ProviderService

Get a human friendly name for web UIs and messages

Specified by:

getHumanName in interface ProviderService

Overrides:

getHumanName in class AbstractProviderService

Returns:

a name string. Default is simply the service instance name.

loadProviderConfigurationInformation

public org.apache.hadoop.conf.Configuration loadProviderConfigurationInformation(File confDir)
                                                                          throws BadCommandArgumentsException,
                                                                                 IOException

Description copied from interface: ProviderService

Load a specific XML configuration file for the provider config

Specified by:

loadProviderConfigurationInformation in interface ProviderService

Parameters:

confDir - configuration directory

Returns:

a configuration to be included in status

Throws:

BadCommandArgumentsException

IOException

buildContainerLaunchContext

public void buildContainerLaunchContext(ContainerLauncher containerLauncher,
                               AggregateConf instanceDefinition,
                               org.apache.hadoop.yarn.api.records.Container container,
                               String role,
                               SliderFileSystem sliderFileSystem,
                               org.apache.hadoop.fs.Path generatedConfPath,
                               MapOperations resourceComponent,
                               MapOperations appComponent,
                               org.apache.hadoop.fs.Path containerTmpDirPath)
                                 throws IOException,
                                        SliderException

Description copied from interface: ProviderService

Set up the entire container launch context

Specified by:

buildContainerLaunchContext in interface ProviderService

Throws:

IOException

SliderException

getRoles

public List<ProviderRole> getRoles()

Specified by:

getRoles in interface ProviderCore

validateInstanceDefinition

public void validateInstanceDefinition(AggregateConf instanceDefinition)
                                throws SliderException

Description copied from interface: ProviderCore

Verify that an instance definition is considered valid by the provider

Specified by:

validateInstanceDefinition in interface ProviderCore

Parameters:

instanceDefinition - instance definition

Throws:

SliderException - if the configuration is not valid

applyInitialRegistryDefinitions

public void applyInitialRegistryDefinitions(URL amWebURI,
                                   URL agentOpsURI,
                                   URL agentStatusURI,
                                   org.apache.hadoop.registry.client.types.ServiceRecord serviceRecord)
                                     throws IOException

Description copied from interface: ProviderService

Prior to going live -register the initial service registry data

Specified by:

applyInitialRegistryDefinitions in interface ProviderService

Overrides:

applyInitialRegistryDefinitions in class AbstractProviderService

Parameters:

amWebURI - URL to the AM. This may be proxied, so use relative paths

agentOpsURI - URI for agent operations. This will not be proxied

agentStatusURI - URI For agent status. Again: no proxy

serviceRecord - service record to build up

Throws:

IOException