org.apache.hadoop.yarn.util.resource
Class ResourceCalculator

java.lang.Object
  org.apache.hadoop.yarn.util.resource.ResourceCalculator

Direct Known Subclasses:

DefaultResourceCalculator, DominantResourceCalculator

@InterfaceAudience.Private
@InterfaceStability.Unstable
public abstract class ResourceCalculator
extends Object

A set of Resource comparison and manipulation interfaces.

Constructor Summary

ResourceCalculator()

Method Summary

abstract int	compare(org.apache.hadoop.yarn.api.records.Resource clusterResource, org.apache.hadoop.yarn.api.records.Resource lhs, org.apache.hadoop.yarn.api.records.Resource rhs)
abstract int	computeAvailableContainers(org.apache.hadoop.yarn.api.records.Resource available, org.apache.hadoop.yarn.api.records.Resource required) Compute the number of containers which can be allocated given available and required resources.
abstract float	divide(org.apache.hadoop.yarn.api.records.Resource clusterResource, org.apache.hadoop.yarn.api.records.Resource numerator, org.apache.hadoop.yarn.api.records.Resource denominator) Divide resource numerator by resource denominator using specified policy (domination, average, fairness etc.); hence overall clusterResource is provided for context.
static int	divideAndCeil(int a, int b)
abstract org.apache.hadoop.yarn.api.records.Resource	divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator, int denominator) Divide-and-ceil numerator by denominator.
abstract boolean	isInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r) Determine if a resource is not suitable for use as a divisor (will result in divide by 0, etc)
abstract org.apache.hadoop.yarn.api.records.Resource	multiplyAndNormalizeDown(org.apache.hadoop.yarn.api.records.Resource r, double by, org.apache.hadoop.yarn.api.records.Resource stepFactor) Multiply resource r by factor by and normalize down using step-factor stepFactor.
abstract org.apache.hadoop.yarn.api.records.Resource	multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r, double by, org.apache.hadoop.yarn.api.records.Resource stepFactor) Multiply resource r by factor by and normalize up using step-factor stepFactor.
org.apache.hadoop.yarn.api.records.Resource	normalize(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource minimumResource, org.apache.hadoop.yarn.api.records.Resource maximumResource) Normalize resource r given the base minimumResource and verify against max allowed maximumResource
abstract org.apache.hadoop.yarn.api.records.Resource	normalize(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource minimumResource, org.apache.hadoop.yarn.api.records.Resource maximumResource, org.apache.hadoop.yarn.api.records.Resource stepFactor) Normalize resource r given the base minimumResource and verify against max allowed maximumResource using a step factor for hte normalization.
abstract float	ratio(org.apache.hadoop.yarn.api.records.Resource a, org.apache.hadoop.yarn.api.records.Resource b) Ratio of resource a to resource b.
static int	roundDown(int a, int b)
abstract org.apache.hadoop.yarn.api.records.Resource	roundDown(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor) Round-down resource r given factor stepFactor.
static int	roundUp(int a, int b)
abstract org.apache.hadoop.yarn.api.records.Resource	roundUp(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor) Round-up resource r given factor stepFactor.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ResourceCalculator

public ResourceCalculator()

Method Detail

compare

public abstract int compare(org.apache.hadoop.yarn.api.records.Resource clusterResource,
                            org.apache.hadoop.yarn.api.records.Resource lhs,
                            org.apache.hadoop.yarn.api.records.Resource rhs)

divideAndCeil

public static int divideAndCeil(int a,
                                int b)

roundUp

public static int roundUp(int a,
                          int b)

roundDown

public static int roundDown(int a,
                            int b)

computeAvailableContainers

public abstract int computeAvailableContainers(org.apache.hadoop.yarn.api.records.Resource available,
                                               org.apache.hadoop.yarn.api.records.Resource required)

Compute the number of containers which can be allocated given available and required resources.

Parameters:

available - available resources

required - required resources

Returns:

number of containers which can be allocated

multiplyAndNormalizeUp

public abstract org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r,
                                                                                   double by,
                                                                                   org.apache.hadoop.yarn.api.records.Resource stepFactor)

Multiply resource r by factor by and normalize up using step-factor stepFactor.

Parameters:

r - resource to be multiplied

by - multiplier

stepFactor - factor by which to normalize up

Returns:

resulting normalized resource

multiplyAndNormalizeDown

public abstract org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeDown(org.apache.hadoop.yarn.api.records.Resource r,
                                                                                     double by,
                                                                                     org.apache.hadoop.yarn.api.records.Resource stepFactor)

Multiply resource r by factor by and normalize down using step-factor stepFactor.

Parameters:

r - resource to be multiplied

by - multiplier

stepFactor - factor by which to normalize down

Returns:

resulting normalized resource

normalize

public org.apache.hadoop.yarn.api.records.Resource normalize(org.apache.hadoop.yarn.api.records.Resource r,
                                                             org.apache.hadoop.yarn.api.records.Resource minimumResource,
                                                             org.apache.hadoop.yarn.api.records.Resource maximumResource)

Normalize resource r given the base minimumResource and verify against max allowed maximumResource

Parameters:

r - resource

minimumResource - step-factor

maximumResource - the upper bound of the resource to be allocated

Returns:

normalized resource

normalize

public abstract org.apache.hadoop.yarn.api.records.Resource normalize(org.apache.hadoop.yarn.api.records.Resource r,
                                                                      org.apache.hadoop.yarn.api.records.Resource minimumResource,
                                                                      org.apache.hadoop.yarn.api.records.Resource maximumResource,
                                                                      org.apache.hadoop.yarn.api.records.Resource stepFactor)

Normalize resource r given the base minimumResource and verify against max allowed maximumResource using a step factor for hte normalization.

Parameters:

r - resource

minimumResource - minimum value

maximumResource - the upper bound of the resource to be allocated

stepFactor - the increment for resources to be allocated

Returns:

normalized resource

roundUp

public abstract org.apache.hadoop.yarn.api.records.Resource roundUp(org.apache.hadoop.yarn.api.records.Resource r,
                                                                    org.apache.hadoop.yarn.api.records.Resource stepFactor)

Round-up resource r given factor stepFactor.

Parameters:

r - resource

stepFactor - step-factor

Returns:

rounded resource

roundDown

public abstract org.apache.hadoop.yarn.api.records.Resource roundDown(org.apache.hadoop.yarn.api.records.Resource r,
                                                                      org.apache.hadoop.yarn.api.records.Resource stepFactor)

Round-down resource r given factor stepFactor.

Parameters:

r - resource

stepFactor - step-factor

Returns:

rounded resource

divide

public abstract float divide(org.apache.hadoop.yarn.api.records.Resource clusterResource,
                             org.apache.hadoop.yarn.api.records.Resource numerator,
                             org.apache.hadoop.yarn.api.records.Resource denominator)

Divide resource numerator by resource denominator using specified policy (domination, average, fairness etc.); hence overall clusterResource is provided for context.

Parameters:

clusterResource - cluster resources

numerator - numerator

denominator - denominator

Returns:

numerator/denominator using specific policy

isInvalidDivisor

public abstract boolean isInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)

Determine if a resource is not suitable for use as a divisor (will result in divide by 0, etc)

Parameters:

r - resource

Returns:

true if divisor is invalid (should not be used), false else

ratio

public abstract float ratio(org.apache.hadoop.yarn.api.records.Resource a,
                            org.apache.hadoop.yarn.api.records.Resource b)

Ratio of resource a to resource b.

Parameters:

a - resource

b - resource

Returns:

ratio of resource a to resource b

divideAndCeil

public abstract org.apache.hadoop.yarn.api.records.Resource divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
                                                                          int denominator)

Divide-and-ceil numerator by denominator.

Parameters:

numerator - numerator resource

denominator - denominator

Returns:

resultant resource