org.apache.slider.server.appmaster.state

Class OutstandingRequestTracker

java.lang.Object

org.apache.slider.server.appmaster.state.OutstandingRequestTracker

public class OutstandingRequestTracker
extends Object

Tracks outstanding requests made with a specific placement option.

1. Used to decide when to return a node to 'can request containers here' list
2. Used to identify requests where placement has timed out, and so issue relaxed requests

If an allocation comes in that is not in the map: either the allocation was unplaced, or the placed allocation could not be met on the specified host, and the RM/scheduler fell back to another location.

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	log

Constructor Summary

Constructors

Constructor and Description
OutstandingRequestTracker()

Method Summary

Methods

Modifier and Type	Method and Description
List<AbstractRMOperation>	cancelOutstandingAARequests() Cancel all outstanding AA requests from the lists of requests.
List<AbstractRMOperation>	escalateOutstandingRequests(long now) Escalate operation as triggered by external timer.
List<OutstandingRequest>	extractOpenRequestsForRole(int roleId, int count) Extract a specific number of open requests for a role
List<OutstandingRequest>	extractPlacedRequestsForRole(int roleId, int count) Extract a specific number of placed requests for a role
List<OutstandingRequest>	listOpenRequests() Get a list of outstanding requests.
List<OutstandingRequest>	listPlacedRequests() Get a list of outstanding requests.
OutstandingRequest	lookupPlacedRequest(int role, String hostname) Look up any oustanding request to a (role, hostname).
OutstandingRequest	newAARequest(int role, List<NodeInstance> nodes, String label) Create a new Anti-affine request for the specific role
OutstandingRequest	newRequest(NodeInstance instance, int role) Create a new request for the specific role.
ContainerAllocationResults	onContainerAllocated(int role, String hostname, org.apache.hadoop.yarn.api.records.Container container) Notification that a container has been allocated drop it from the placedRequests structure. generate the cancellation request for AA placement, any actions needed
void	partitionRequests(RoleHistory rh, List<org.apache.hadoop.yarn.api.records.Container> inAllocated, List<org.apache.hadoop.yarn.api.records.Container> outPlaceRequested, List<org.apache.hadoop.yarn.api.records.Container> outUnplaced) Take a list of requests and split them into specific host requests and generic assignments.
OutstandingRequest	removePlacedRequest(OutstandingRequest request) Remove a request
List<NodeInstance>	resetOutstandingRequests(int role) Reset list all outstanding requests for a role: return the hostnames of any canceled requests

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected static final org.slf4j.Logger log

Constructor Detail

OutstandingRequestTracker

public OutstandingRequestTracker()

Method Detail

newRequest

public OutstandingRequest newRequest(NodeInstance instance,
                            int role)

Create a new request for the specific role.

If a location is set, the request is added to placedRequests. If not, it is added to openRequests

This does not update the node instance's role's request count

Parameters:

instance - node instance to manager

role - role index

Returns:

a new request

newAARequest

public OutstandingRequest newAARequest(int role,
                              List<NodeInstance> nodes,
                              String label)

Create a new Anti-affine request for the specific role

It is added to openRequests

This does not update the node instance's role's request count

Parameters:

role - role index

nodes - list of suitable nodes

label - label to use

Returns:

a new request

lookupPlacedRequest

public OutstandingRequest lookupPlacedRequest(int role,
                                     String hostname)

Look up any oustanding request to a (role, hostname).

Parameters:

role - role index

hostname - hostname

Returns:

the request or null if there was no outstanding one in the placedRequests

removePlacedRequest

public OutstandingRequest removePlacedRequest(OutstandingRequest request)

Remove a request

Parameters:

request - matching request to find

Returns:

the request or null for no match in the placedRequests

onContainerAllocated

public ContainerAllocationResults onContainerAllocated(int role,
                                              String hostname,
                                              org.apache.hadoop.yarn.api.records.Container container)

Notification that a container has been allocated

1. drop it from the placedRequests structure.
2. generate the cancellation request
3. for AA placement, any actions needed

Parameters:

role - role index

hostname - hostname

Returns:

the allocation outcome

partitionRequests

public void partitionRequests(RoleHistory rh,
                     List<org.apache.hadoop.yarn.api.records.Container> inAllocated,
                     List<org.apache.hadoop.yarn.api.records.Container> outPlaceRequested,
                     List<org.apache.hadoop.yarn.api.records.Container> outUnplaced)

Take a list of requests and split them into specific host requests and generic assignments. This is to give requested hosts priority in container assignments if more come back than expected

Parameters:

rh - RoleHistory instance

inAllocated - the list of allocated containers

outPlaceRequested - initially empty list of requested locations

outUnplaced - initially empty list of unrequested hosts

resetOutstandingRequests

public List<NodeInstance> resetOutstandingRequests(int role)

Reset list all outstanding requests for a role: return the hostnames of any canceled requests

Parameters:

role - role to cancel

Returns:

possibly empty list of hostnames

listPlacedRequests

public List<OutstandingRequest> listPlacedRequests()

Get a list of outstanding requests. The list is cloned, but the contents are shared

Returns:

a list of the current outstanding requests

listOpenRequests

public List<OutstandingRequest> listOpenRequests()

Get a list of outstanding requests. The list is cloned, but the contents are shared

Returns:

a list of the current outstanding requests

escalateOutstandingRequests

public List<AbstractRMOperation> escalateOutstandingRequests(long now)

Escalate operation as triggered by external timer.

Returns:

a (usually empty) list of cancel/request operations.

cancelOutstandingAARequests

public List<AbstractRMOperation> cancelOutstandingAARequests()

Cancel all outstanding AA requests from the lists of requests. This does not remove them from the role status; they must be reset by the caller.

extractOpenRequestsForRole

public List<OutstandingRequest> extractOpenRequestsForRole(int roleId,
                                                  int count)

Extract a specific number of open requests for a role

Parameters:

roleId - role Id

count - count to extract

Returns:

a list of requests which are no longer in the open request list

extractPlacedRequestsForRole

public List<OutstandingRequest> extractPlacedRequestsForRole(int roleId,
                                                    int count)

Extract a specific number of placed requests for a role

Parameters:

roleId - role Id

count - count to extract

Returns:

a list of requests which are no longer in the placed request data structure