org.apache.slider.client

Class SliderYarnClientImpl

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.yarn.client.api.YarnClient

org.apache.hadoop.yarn.client.api.impl.YarnClientImpl

org.apache.slider.client.SliderYarnClientImpl

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.service.Service

public class SliderYarnClientImpl
extends org.apache.hadoop.yarn.client.api.impl.YarnClientImpl

A class that extends visibility to some of the YarnClientImpl members and data structures, and factors out pure-YARN operations from the slider entry point service

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields

Modifier and Type	Field and Description
static String	KILL_ALL Keyword to use in the emergencyForceKill(String) operation to force kill all application instances belonging to a specific user
protected static org.slf4j.Logger	log

Fields inherited from class org.apache.hadoop.yarn.client.api.impl.YarnClientImpl

historyClient, rmClient, submitPollIntervalMillis, timelineClient, timelineServiceBestEffort, timelineServiceEnabled

Constructor Summary

Constructors

Constructor and Description
SliderYarnClientImpl()

Method Summary

Methods

Modifier and Type	Method and Description
void	emergencyForceKill(String applicationId) Force kill a yarn application by ID.
List<org.apache.hadoop.yarn.api.records.ApplicationReport>	findAllInstances(String user, String appname) find all instances of a specific app -if there is more than one in the YARN cluster, this returns them all
List<org.apache.hadoop.yarn.api.records.ApplicationReport>	findAllLiveInstances(String user, String appname) find all live instances of a specific app -if there is >1 in the cluster, this returns them all.
org.apache.hadoop.yarn.api.records.ApplicationReport	findAppInInstanceList(List<org.apache.hadoop.yarn.api.records.ApplicationReport> instances, String appname, org.apache.hadoop.yarn.api.records.YarnApplicationState desiredState) Find an app in the instance list in the desired state
org.apache.hadoop.yarn.api.records.ApplicationReport	findClusterInInstanceList(List<org.apache.hadoop.yarn.api.records.ApplicationReport> instances, String appname) Find a cluster in the instance list; biased towards live instances
org.apache.hadoop.yarn.api.ApplicationClientProtocol	getRmClient() Get the RM Client RPC interface
boolean	isApplicationLive(org.apache.hadoop.yarn.api.records.ApplicationReport app) Helper method to determine if a cluster application is running -or is earlier in the lifecycle
org.apache.hadoop.yarn.api.protocolrecords.KillApplicationResponse	killRunningApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String reason) Kill a running application
List<org.apache.hadoop.yarn.api.records.ApplicationReport>	listDeployedInstances(String user) List Slider deployedinstances belonging to a specific user.
List<org.apache.hadoop.yarn.api.records.ApplicationReport>	listInstances(String user) Deprecated. use listDeployedInstances(String)
NodeInformationList	listNodes(String label, boolean live) List the nodes in the cluster, possibly filtering by node state or label.
org.apache.hadoop.yarn.api.records.ApplicationReport	monitorAppToState(org.apache.hadoop.yarn.api.records.ApplicationId appId, org.apache.hadoop.yarn.api.records.YarnApplicationState desiredState, Duration duration) Monitor the submitted application for reaching the requested state.
protected void	serviceInit(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class org.apache.hadoop.yarn.client.api.impl.YarnClientImpl

createApplication, deleteReservation, getAllQueues, getAMRMToken, getApplicationAttemptReport, getApplicationAttempts, getApplicationReport, getApplications, getApplications, getApplications, getApplications, getChildQueueInfos, getClusterNodeLabels, getContainerReport, getContainers, getLabelsToNodes, getLabelsToNodes, getNodeReports, getNodeToLabels, getQueueAclsInfo, getQueueInfo, getRMDelegationToken, getRootQueueInfos, getYarnClusterMetrics, isSecurityEnabled, killApplication, moveApplicationAcrossQueues, serviceStart, serviceStop, setRMClient, submitApplication, submitReservation, updateReservation

Methods inherited from class org.apache.hadoop.yarn.client.api.YarnClient

createYarnClient

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

log

protected static final org.slf4j.Logger log

KILL_ALL

public static final String KILL_ALL

Keyword to use in the emergencyForceKill(String) operation to force kill all application instances belonging to a specific user

See Also:

Constant Field Values

Constructor Detail

SliderYarnClientImpl

public SliderYarnClientImpl()

Method Detail

serviceInit

protected void serviceInit(org.apache.hadoop.conf.Configuration conf)
                    throws Exception

Overrides:

serviceInit in class org.apache.hadoop.yarn.client.api.impl.YarnClientImpl

Throws:

Exception

getRmClient

public org.apache.hadoop.yarn.api.ApplicationClientProtocol getRmClient()

Get the RM Client RPC interface

Returns:

an RPC interface valid after initialization and authentication

listInstances

public List<org.apache.hadoop.yarn.api.records.ApplicationReport> listInstances(String user)
                                                                         throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                IOException

Deprecated. use listDeployedInstances(String)

List Slider runninginstances belonging to a specific user.

Parameters:

user - user: "" means all users

Returns:

a possibly empty list of Slider AMs

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

listDeployedInstances

public List<org.apache.hadoop.yarn.api.records.ApplicationReport> listDeployedInstances(String user)
                                                                                 throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                        IOException

List Slider deployedinstances belonging to a specific user.

Deployed means: known about in the YARN cluster; it will include any that are in the failed/finished state, as well as those queued for starting.

Parameters:

user - user: "" means all users

Returns:

a possibly empty list of Slider AMs

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

findAllInstances

public List<org.apache.hadoop.yarn.api.records.ApplicationReport> findAllInstances(String user,
                                                                          String appname)
                                                                            throws IOException,
                                                                                   org.apache.hadoop.yarn.exceptions.YarnException

find all instances of a specific app -if there is more than one in the YARN cluster, this returns them all

Parameters:

user - user; use "" for all users

appname - application name

Returns:

the list of all matching application instances

Throws:

IOException

org.apache.hadoop.yarn.exceptions.YarnException

isApplicationLive

public boolean isApplicationLive(org.apache.hadoop.yarn.api.records.ApplicationReport app)

Helper method to determine if a cluster application is running -or is earlier in the lifecycle

Parameters:

app - application report

Returns:

true if the application is considered live

killRunningApplication

public org.apache.hadoop.yarn.api.protocolrecords.KillApplicationResponse killRunningApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
                                                                                        String reason)
                                                                                          throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                 IOException

Kill a running application

Parameters:

applicationId - app Id

reason - reason: reason for log

Returns:

the response

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - YARN problems

IOException - IO problems

emergencyForceKill

public void emergencyForceKill(String applicationId)
                        throws org.apache.hadoop.yarn.exceptions.YarnException,
                               IOException

Force kill a yarn application by ID. No niceities here

Parameters:

applicationId - app Id. "all" means "kill all instances of the current user

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

monitorAppToState

public org.apache.hadoop.yarn.api.records.ApplicationReport monitorAppToState(org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                                     org.apache.hadoop.yarn.api.records.YarnApplicationState desiredState,
                                                                     Duration duration)
                                                                       throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                              IOException

Monitor the submitted application for reaching the requested state. Will also report if the app reaches a later state (failed, killed, etc) Kill application if duration!= null & time expires.

Parameters:

appId - Application Id of application to be monitored

duration - how long to wait -must be more than 0

desiredState - desired state.

Returns:

the application report -null on a timeout

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

findAllLiveInstances

public List<org.apache.hadoop.yarn.api.records.ApplicationReport> findAllLiveInstances(String user,
                                                                              String appname)
                                                                                throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                       IOException

find all live instances of a specific app -if there is >1 in the cluster, this returns them all. State should be running or less

Parameters:

user - user

appname - application name

Returns:

the list of all matching application instances

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

findClusterInInstanceList

public org.apache.hadoop.yarn.api.records.ApplicationReport findClusterInInstanceList(List<org.apache.hadoop.yarn.api.records.ApplicationReport> instances,
                                                                             String appname)

Find a cluster in the instance list; biased towards live instances

Parameters:

instances - list of instances

appname - application name

Returns:

the first found instance, else a failed/finished instance, or null if there are none of those

findAppInInstanceList

public org.apache.hadoop.yarn.api.records.ApplicationReport findAppInInstanceList(List<org.apache.hadoop.yarn.api.records.ApplicationReport> instances,
                                                                         String appname,
                                                                         org.apache.hadoop.yarn.api.records.YarnApplicationState desiredState)

Find an app in the instance list in the desired state

Parameters:

instances - instance list

appname - application name

desiredState - yarn state desired

Returns:

the match or null for none

listNodes

public NodeInformationList listNodes(String label,
                            boolean live)
                              throws IOException,
                                     org.apache.hadoop.yarn.exceptions.YarnException

List the nodes in the cluster, possibly filtering by node state or label.

Parameters:

label - label to filter by -or "" for any

live - flag to request running nodes only

Returns:

a possibly empty list of nodes in the cluster

Throws:

IOException - IO problems

org.apache.hadoop.yarn.exceptions.YarnException - YARN problems