| Modifier and Type | Field and Description |
|---|---|
protected SliderFileSystem |
SliderClient.sliderFileSystem |
| Modifier and Type | Method and Description |
|---|---|
protected void |
SliderClient.validateInstanceDefinition(AbstractClientProvider provider,
AggregateConf instanceDefinition,
SliderFileSystem fs)
Validate an instance definition against a provider.
|
| Modifier and Type | Method and Description |
|---|---|
static ClasspathConstructor |
SliderUtils.buildClasspath(String sliderConfDir,
String libdir,
org.apache.hadoop.conf.Configuration config,
SliderFileSystem sliderFileSystem,
boolean usingMiniMRCluster)
Build up the classpath for execution
-behaves very differently on a mini test cluster vs a production
production one.
|
static void |
SliderUtils.putAllJars(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> providerResources,
SliderFileSystem sliderFileSystem,
org.apache.hadoop.fs.Path tempPath,
String libDir,
String srcPath)
Submit a JAR containing and map it
|
static void |
SliderUtils.putAmTarGzipAndUpdate(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> providerResources,
SliderFileSystem sliderFileSystem)
Submit the AM tar.gz containing all dependencies and map it
|
static org.apache.hadoop.yarn.api.records.LocalResource |
SliderUtils.putJar(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> providerResources,
SliderFileSystem sliderFileSystem,
Class clazz,
org.apache.hadoop.fs.Path tempPath,
String libdir,
String jarName)
Submit a JAR containing a specific class and map it
|
static void |
SliderUtils.validateHDFSFile(SliderFileSystem sliderFileSystem,
String pathStr)
validate if a file on HDFS can be open
|
| Constructor and Description |
|---|
AppDefinitionPersister(SliderFileSystem sliderFileSystem) |
| Modifier and Type | Method and Description |
|---|---|
static boolean |
ProviderUtils.addAgentTar(Object provider,
String tarName,
SliderFileSystem sliderFileSystem,
org.apache.hadoop.fs.Path agentDir)
Add/overwrite the agent tarball (overwritten every time application is restarted)
|
static void |
ProviderUtils.addAllDependencyJars(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> providerResources,
SliderFileSystem sliderFileSystem,
org.apache.hadoop.fs.Path tempPath,
String libDir,
String libLocalSrcDir)
Loads all dependency jars from the default path
|
static void |
ProviderUtils.addDependencyJars(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> providerResources,
SliderFileSystem sliderFileSystem,
org.apache.hadoop.fs.Path tempPath,
String libdir,
String[] resources,
Class[] classes)
Add a set of dependencies to the provider resources being built up,
by copying them from the local classpath to the remote one, then
registering them
|
static boolean |
ProviderUtils.addProviderJar(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> providerResources,
Object provider,
String jarName,
SliderFileSystem sliderFileSystem,
org.apache.hadoop.fs.Path tempPath,
String libdir,
boolean miniClusterTestRun)
Add oneself to the classpath.
|
void |
ProviderService.buildContainerLaunchContext(ContainerLauncher containerLauncher,
AggregateConf instanceDefinition,
org.apache.hadoop.yarn.api.records.Container container,
String role,
SliderFileSystem sliderFileSystem,
org.apache.hadoop.fs.Path generatedConfPath,
MapOperations resourceComponent,
MapOperations appComponent,
org.apache.hadoop.fs.Path containerTmpDirPath)
Set up the entire container launch context
|
Set<String> |
AbstractClientProvider.getApplicationTags(SliderFileSystem fileSystem,
String appDef)
Return a set of application specific string tags.
|
void |
ProviderService.initializeApplicationConfiguration(AggregateConf instanceDefinition,
SliderFileSystem fileSystem)
The application configuration should be initialized here
|
void |
AbstractProviderService.initializeApplicationConfiguration(AggregateConf instanceDefinition,
SliderFileSystem fileSystem)
No-op implementation of this method.
|
void |
AbstractClientProvider.preflightValidateClusterConfiguration(SliderFileSystem sliderFileSystem,
String clustername,
org.apache.hadoop.conf.Configuration configuration,
AggregateConf instanceDefinition,
org.apache.hadoop.fs.Path clusterDirPath,
org.apache.hadoop.fs.Path generatedConfDirPath,
boolean secure)
This is called pre-launch to validate that the cluster specification
is valid.
|
void |
AbstractClientProvider.prepareAMAndConfigForLaunch(SliderFileSystem fileSystem,
org.apache.hadoop.conf.Configuration serviceConf,
AbstractLauncher launcher,
AggregateConf instanceDescription,
org.apache.hadoop.fs.Path snapshotConfDirPath,
org.apache.hadoop.fs.Path generatedConfDirPath,
org.apache.hadoop.conf.Configuration clientConfExtras,
String libdir,
org.apache.hadoop.fs.Path tempPath,
boolean miniClusterTestRun)
Prepare the AM settings for launch
|
void |
AbstractClientProvider.processClientOperation(SliderFileSystem fileSystem,
String operation,
File clientInstallPath,
File clientPackage,
org.codehaus.jettison.json.JSONObject config,
String name)
Process client operations for applications such as install, configure
|
void |
AbstractClientProvider.validateInstanceDefinition(AggregateConf instanceDefinition,
SliderFileSystem fs)
Verify that an instance definition is considered valid by the provider
|
| Modifier and Type | Method and Description |
|---|---|
void |
AgentProviderService.buildContainerLaunchContext(ContainerLauncher launcher,
AggregateConf instanceDefinition,
org.apache.hadoop.yarn.api.records.Container container,
String role,
SliderFileSystem fileSystem,
org.apache.hadoop.fs.Path generatedConfPath,
MapOperations resourceComponent,
MapOperations appComponent,
org.apache.hadoop.fs.Path containerTmpDirPath) |
protected Metainfo |
AgentProviderService.getApplicationMetainfo(SliderFileSystem fileSystem,
String appDef) |
static Metainfo |
AgentUtils.getApplicationMetainfo(SliderFileSystem fileSystem,
String metainfoPath,
boolean metainfoForAddon) |
protected Metainfo |
AgentProviderService.getApplicationMetainfo(SliderFileSystem fileSystem,
String appDef,
boolean addonPackage) |
Set<String> |
AgentClientProvider.getApplicationTags(SliderFileSystem fileSystem,
String appDef) |
void |
AgentProviderService.initializeApplicationConfiguration(AggregateConf instanceDefinition,
SliderFileSystem fileSystem) |
protected Map<String,DefaultConfig> |
AgentProviderService.initializeDefaultConfigs(SliderFileSystem fileSystem,
String appDef,
Metainfo metainfo)
Read all default configs
|
void |
AgentClientProvider.preflightValidateClusterConfiguration(SliderFileSystem sliderFileSystem,
String clustername,
org.apache.hadoop.conf.Configuration configuration,
AggregateConf instanceDefinition,
org.apache.hadoop.fs.Path clusterDirPath,
org.apache.hadoop.fs.Path generatedConfDirPath,
boolean secure) |
void |
AgentClientProvider.prepareAMAndConfigForLaunch(SliderFileSystem fileSystem,
org.apache.hadoop.conf.Configuration serviceConf,
AbstractLauncher launcher,
AggregateConf instanceDefinition,
org.apache.hadoop.fs.Path snapshotConfDirPath,
org.apache.hadoop.fs.Path generatedConfDirPath,
org.apache.hadoop.conf.Configuration clientConfExtras,
String libdir,
org.apache.hadoop.fs.Path tempPath,
boolean miniClusterTestRun) |
void |
AgentClientProvider.processClientOperation(SliderFileSystem fileSystem,
String operation,
File clientInstallPath,
File appPackage,
org.codehaus.jettison.json.JSONObject config,
String name) |
void |
AgentClientProvider.validateInstanceDefinition(AggregateConf instanceDefinition,
SliderFileSystem fs) |
| Modifier and Type | Method and Description |
|---|---|
protected void |
SliderAMClientProvider.addKeytabResourceIfNecessary(SliderFileSystem fileSystem,
AggregateConf instanceDescription,
Map<String,org.apache.hadoop.yarn.api.records.LocalResource> providerResources)
If the cluster is secure, and an HDFS installed keytab is available for AM
authentication, add this keytab as a local resource for the AM launch.
|
void |
SliderAMProviderService.buildContainerLaunchContext(ContainerLauncher containerLauncher,
AggregateConf instanceDefinition,
org.apache.hadoop.yarn.api.records.Container container,
String role,
SliderFileSystem sliderFileSystem,
org.apache.hadoop.fs.Path generatedConfPath,
MapOperations resourceComponent,
MapOperations appComponent,
org.apache.hadoop.fs.Path containerTmpDirPath) |
void |
SliderAMClientProvider.preflightValidateClusterConfiguration(SliderFileSystem sliderFileSystem,
String clustername,
org.apache.hadoop.conf.Configuration configuration,
AggregateConf instanceDefinition,
org.apache.hadoop.fs.Path clusterDirPath,
org.apache.hadoop.fs.Path generatedConfDirPath,
boolean secure) |
void |
SliderAMClientProvider.prepareAMAndConfigForLaunch(SliderFileSystem fileSystem,
org.apache.hadoop.conf.Configuration serviceConf,
AbstractLauncher launcher,
AggregateConf instanceDescription,
org.apache.hadoop.fs.Path snapshotConfDirPath,
org.apache.hadoop.fs.Path generatedConfDirPath,
org.apache.hadoop.conf.Configuration clientConfExtras,
String libdir,
org.apache.hadoop.fs.Path tempPath,
boolean miniClusterTestRun)
The Slider AM sets up all the dependency JARs above slider.jar itself
Prepare the AM settings for launch
|
void |
SliderAMClientProvider.validateInstanceDefinition(AggregateConf instanceDefinition,
SliderFileSystem fs)
Verify that an instance definition is considered valid by the provider
|
| Modifier and Type | Method and Description |
|---|---|
SliderFileSystem |
SliderAppMaster.getClusterFS()
Get the filesystem of this cluster
|
| Constructor and Description |
|---|
RoleLaunchService(QueueAccess queueAccess,
ProviderService provider,
SliderFileSystem fs,
org.apache.hadoop.fs.Path generatedConfDirPath,
Map<String,String> envVars,
org.apache.hadoop.fs.Path launcherTmpDirPath)
Construct an instance of the launcher
|
Copyright © 2014–2015 The Apache Software Foundation. All rights reserved.