org.apache.hadoop.hbase.procedure2

Class Procedure<TEnvironment>

java.lang.Object

org.apache.hadoop.hbase.procedure2.Procedure<TEnvironment>

All Implemented Interfaces:

Comparable<Procedure>

Direct Known Subclasses:

OnePhaseProcedure, SequentialProcedure, StateMachineProcedure, TwoPhaseProcedure

@InterfaceAudience.Private
@InterfaceStability.Evolving
public abstract class Procedure<TEnvironment>
extends Object
implements Comparable<Procedure>

Base Procedure class responsible to handle the Procedure Metadata e.g. state, startTime, lastUpdate, stack-indexes, ... execute() is called each time the procedure is executed. it may be called multiple times in case of failure and restart, so the code must be idempotent. the return is a set of sub-procedures or null in case the procedure doesn't have sub-procedures. Once the sub-procedures are successfully completed the execute() method is called again, you should think at it as a stack: -> step 1 ---> step 2 -> step 1 rollback() is called when the procedure or one of the sub-procedures is failed. the rollback step is supposed to cleanup the resources created during the execute() step. in case of failure and restart rollback() may be called multiple times, so the code must be idempotent.

Constructor Summary

Constructors

Constructor and Description
Procedure()

Method Summary

Methods

Modifier and Type	Method and Description
protected abstract boolean	abort(TEnvironment env) The abort() call is asynchronous and each procedure must decide how to deal with that, if they want to be abortable.
protected boolean	acquireLock(TEnvironment env) The user should override this method, and try to take a lock if necessary.
protected void	addStackIndex(int index) Called by the RootProcedureState on procedure execution.
protected void	beforeReplay(TEnvironment env) Called when the procedure is loaded for replay.
protected boolean	childrenCountDown() Called by the ProcedureExecutor to notify that one of the sub-procedures has completed.
int	compareTo(Procedure other)
protected void	completionCleanup(TEnvironment env) Called when the procedure is marked as completed (success or rollback).
static ProcedureProtos.Procedure	convert(Procedure proc) Helper to convert the procedure to protobuf.
static Procedure	convert(ProcedureProtos.Procedure proto) Helper to convert the protobuf procedure.
static ProcedureInfo	createProcedureInfo(Procedure proc, NonceKey nonceKey) Helper to create the ProcedureInfo from Procedure.
protected abstract void	deserializeStateData(InputStream stream) Called on store load to allow the user to decode the previously serialized state.
protected Procedure[]	doExecute(TEnvironment env) Internal method called by the ProcedureExecutor that starts the user-level code execute().
protected void	doRollback(TEnvironment env) Internal method called by the ProcedureExecutor that starts the user-level code rollback().
long	elapsedTime()
protected abstract Procedure[]	execute(TEnvironment env) The main code of the procedure.
RemoteProcedureException	getException()
long	getLastUpdate()
NonceKey	getNonceKey()
String	getOwner()
long	getParentProcId()
long	getProcId()
byte[]	getResult()
protected static Long	getRootProcedureId(Map<Long,Procedure> procedures, Procedure proc)
protected int[]	getStackIndexes()
long	getStartTime()
protected ProcedureProtos.ProcedureState	getState()
int	getTimeout()
long	getTimeRemaining()
boolean	hasException()
boolean	hasOwner()
boolean	hasParent()
boolean	hasTimeout()
protected void	incChildrenLatch() Called by the ProcedureExecutor on procedure-load to restore the latch state
boolean	isFailed()
boolean	isFinished()
boolean	isSuccess()
boolean	isWaiting()
protected static Procedure	newInstance(String className)
protected void	releaseLock(TEnvironment env) The user should override this method, and release lock if necessary.
protected boolean	removeStackIndex()
protected abstract void	rollback(TEnvironment env) The code to undo what done by the execute() code.
protected abstract void	serializeStateData(OutputStream stream) The user-level code of the procedure may have some state to persist (e.g.
protected void	setAbortFailure(String source, String msg)
protected void	setChildrenLatch(int numChildren) Called by the ProcedureExecutor on procedure-load to restore the latch state
protected void	setFailure(RemoteProcedureException exception)
protected void	setFailure(String source, Throwable cause)
protected void	setNonceKey(NonceKey nonceKey) Called by the ProcedureExecutor to set the value to the newly created procedure.
void	setOwner(String owner)
protected void	setParentProcId(long parentProcId) Called by the ProcedureExecutor to assign the parent to the newly created procedure.
protected void	setProcId(long procId) Called by the ProcedureExecutor to assign the ID to the newly created procedure.
protected void	setResult(byte[] result) The procedure may leave a "result" on completion.
protected void	setStackIndexes(List<Integer> stackIndexes) Called on store load to initialize the Procedure internals after the creation/deserialization.
protected void	setStartTime(long startTime) Called on store load to initialize the Procedure internals after the creation/deserialization.
protected void	setState(ProcedureProtos.ProcedureState state)
protected void	setTimeout(int timeout)
protected boolean	setTimeoutFailure()
String	toString()
protected String	toStringClass()
protected void	toStringClassDetails(StringBuilder builder) Extend the toString() information with the procedure details e.g.
String	toStringDetails() Extend the toString() information with more procedure details
protected StringBuilder	toStringSimpleSB() Build the StringBuilder for the simple form of procedure string.
protected void	updateTimestamp()
protected static void	validateClass(Procedure proc)
protected boolean	wasExecuted()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Procedure

public Procedure()

Method Detail

execute

protected abstract Procedure[] execute(TEnvironment env)
                                throws ProcedureYieldException,
                                       InterruptedException

The main code of the procedure. It must be idempotent since execute() may be called multiple time in case of machine failure in the middle of the execution.

Parameters:

env - the environment passed to the ProcedureExecutor

Returns:

a set of sub-procedures or null if there is nothing else to execute.

Throws:

ProcedureYieldException - the procedure will be added back to the queue and retried later

InterruptedException - the procedure will be added back to the queue and retried later

rollback

protected abstract void rollback(TEnvironment env)
                          throws IOException

The code to undo what done by the execute() code. It is called when the procedure or one of the sub-procedure failed or an abort was requested. It should cleanup all the resources created by the execute() call. The implementation must be idempotent since rollback() may be called multiple time in case of machine failure in the middle of the execution.

Parameters:

env - the environment passed to the ProcedureExecutor

Throws:

IOException - temporary failure, the rollback will retry later

abort

protected abstract boolean abort(TEnvironment env)

The abort() call is asynchronous and each procedure must decide how to deal with that, if they want to be abortable. The simplest implementation is to have an AtomicBoolean set in the abort() method and then the execute() will check if the abort flag is set or not. abort() may be called multiple times from the client, so the implementation must be idempotent. NOTE: abort() is not like Thread.interrupt() it is just a notification that allows the procedure implementor where to abort to avoid leak and have a better control on what was executed and what not.

Parameters:

env - the environment passed to the ProcedureExecutor

serializeStateData

protected abstract void serializeStateData(OutputStream stream)
                                    throws IOException

The user-level code of the procedure may have some state to persist (e.g. input arguments) to be able to resume on failure.

Parameters:

stream - the stream that will contain the user serialized data

Throws:

IOException - failure to stream data

deserializeStateData

protected abstract void deserializeStateData(InputStream stream)
                                      throws IOException

Called on store load to allow the user to decode the previously serialized state.

Parameters:

stream - the stream that contains the user serialized data

Throws:

IOException

acquireLock

protected boolean acquireLock(TEnvironment env)

The user should override this method, and try to take a lock if necessary. A lock can be anything, and it is up to the implementor. Example: in our Master we can execute request in parallel for different tables create t1 and create t2 can be executed at the same time. anything else on t1/t2 is queued waiting that specific table create to happen.

Parameters:

env - the environment passed to the ProcedureExecutor

Returns:

true if the lock was acquired and false otherwise

releaseLock

protected void releaseLock(TEnvironment env)

The user should override this method, and release lock if necessary.

Parameters:

env - the environment passed to the ProcedureExecutor

beforeReplay

protected void beforeReplay(TEnvironment env)

Called when the procedure is loaded for replay. The procedure implementor may use this method to perform some quick operation before replay. e.g. failing the procedure if the state on replay may be unknown.

Parameters:

env - the environment passed to the ProcedureExecutor

completionCleanup

protected void completionCleanup(TEnvironment env)

Called when the procedure is marked as completed (success or rollback). The procedure implementor may use this method to cleanup in-memory states. This operation will not be retried on failure.

Parameters:

env - the environment passed to the ProcedureExecutor

toString

public String toString()

Overrides:

toString in class Object

toStringSimpleSB

protected StringBuilder toStringSimpleSB()

Build the StringBuilder for the simple form of procedure string.

Returns:

the StringBuilder

toStringDetails

public String toStringDetails()

Extend the toString() information with more procedure details

toStringClass

protected String toStringClass()

toStringClassDetails

protected void toStringClassDetails(StringBuilder builder)

Extend the toString() information with the procedure details e.g. className and parameters

Parameters:

builder - the string builder to use to append the proc specific information

getResult

public byte[] getResult()

Returns:

the serialized result if any, otherwise null

setResult

protected void setResult(byte[] result)

The procedure may leave a "result" on completion.

Parameters:

result - the serialized result that will be passed to the client

getProcId

public long getProcId()

hasParent

public boolean hasParent()

hasException

public boolean hasException()

hasTimeout

public boolean hasTimeout()

getParentProcId

public long getParentProcId()

getNonceKey

public NonceKey getNonceKey()

isFailed

public boolean isFailed()

Returns:

true if the procedure has failed. true may mean failed but not yet rolledback or failed and rolledback.

isSuccess

public boolean isSuccess()

Returns:

true if the procedure is finished successfully.

isFinished

public boolean isFinished()

Returns:

true if the procedure is finished. The Procedure may be completed successfuly or failed and rolledback.

isWaiting

public boolean isWaiting()

Returns:

true if the procedure is waiting for a child to finish or for an external event.

getException

public RemoteProcedureException getException()

getStartTime

public long getStartTime()

getLastUpdate

public long getLastUpdate()

elapsedTime

public long elapsedTime()

setTimeout

protected void setTimeout(int timeout)

Parameters:

timeout - timeout in msec

getTimeout

public int getTimeout()

Returns:

the timeout in msec

getTimeRemaining

public long getTimeRemaining()

Returns:

the remaining time before the timeout

setOwner

@InterfaceAudience.Private
public void setOwner(String owner)

Parameters:

owner - the owner passed in

getOwner

public String getOwner()

hasOwner

public boolean hasOwner()

setState

@InterfaceAudience.Private
protected void setState(ProcedureProtos.ProcedureState state)

Parameters:

state - current procedure state

getState

@InterfaceAudience.Private
protected ProcedureProtos.ProcedureState getState()

setFailure

protected void setFailure(String source,
              Throwable cause)

Parameters:

source - exception string

cause - the cause of failure

setFailure

protected void setFailure(RemoteProcedureException exception)

Parameters:

exception - exception thrown

setAbortFailure

protected void setAbortFailure(String source,
                   String msg)

Parameters:

source - exception string

msg - message to pass on

setTimeoutFailure

@InterfaceAudience.Private
protected boolean setTimeoutFailure()

setProcId

@InterfaceAudience.Private
protected void setProcId(long procId)

Called by the ProcedureExecutor to assign the ID to the newly created procedure.

setParentProcId

@InterfaceAudience.Private
protected void setParentProcId(long parentProcId)

Called by the ProcedureExecutor to assign the parent to the newly created procedure.

Parameters:

parentProcId - parent procedure Id

setNonceKey

@InterfaceAudience.Private
protected void setNonceKey(NonceKey nonceKey)

Called by the ProcedureExecutor to set the value to the newly created procedure.

Parameters:

nonceKey - the key to detect duplicate call

doExecute

@InterfaceAudience.Private
protected Procedure[] doExecute(TEnvironment env)
                         throws ProcedureYieldException,
                                InterruptedException

Internal method called by the ProcedureExecutor that starts the user-level code execute().

Parameters:

env - the environment passed to the ProcedureExecutor

Returns:

a set of sub-procedures or null if there is nothing else to execute.

Throws:

ProcedureYieldException - the procedure will be added back to the queue and retried later

InterruptedException

doRollback

@InterfaceAudience.Private
protected void doRollback(TEnvironment env)
                   throws IOException

Internal method called by the ProcedureExecutor that starts the user-level code rollback().

Parameters:

env - the environment passed to the ProcedureExecutor

Throws:

IOException

setStartTime

@InterfaceAudience.Private
protected void setStartTime(long startTime)

Called on store load to initialize the Procedure internals after the creation/deserialization.

Parameters:

startTime - procedure start time

updateTimestamp

protected void updateTimestamp()

setChildrenLatch

@InterfaceAudience.Private
protected void setChildrenLatch(int numChildren)

Called by the ProcedureExecutor on procedure-load to restore the latch state

Parameters:

numChildren - children count

incChildrenLatch

@InterfaceAudience.Private
protected void incChildrenLatch()

Called by the ProcedureExecutor on procedure-load to restore the latch state

childrenCountDown

@InterfaceAudience.Private
protected boolean childrenCountDown()

Called by the ProcedureExecutor to notify that one of the sub-procedures has completed.

addStackIndex

@InterfaceAudience.Private
protected void addStackIndex(int index)

Called by the RootProcedureState on procedure execution. Each procedure store its stack-index positions.

Parameters:

index - the place where procedure is in

removeStackIndex

@InterfaceAudience.Private
protected boolean removeStackIndex()

setStackIndexes

@InterfaceAudience.Private
protected void setStackIndexes(List<Integer> stackIndexes)

Called on store load to initialize the Procedure internals after the creation/deserialization.

Parameters:

stackIndexes - the list of positions of procedures

wasExecuted

@InterfaceAudience.Private
protected boolean wasExecuted()

getStackIndexes

@InterfaceAudience.Private
protected int[] getStackIndexes()

compareTo

public int compareTo(Procedure other)

Specified by:

compareTo in interface Comparable<Procedure>

Parameters:

other - the procedure to compare to

getRootProcedureId

@InterfaceAudience.Private
protected static Long getRootProcedureId(Map<Long,Procedure> procedures,
                                                Procedure proc)

newInstance

protected static Procedure newInstance(String className)
                                throws IOException

Throws:

IOException

validateClass

protected static void validateClass(Procedure proc)
                             throws IOException

Throws:

IOException

createProcedureInfo

@InterfaceAudience.Private
public static ProcedureInfo createProcedureInfo(Procedure proc,
                                                          NonceKey nonceKey)

Helper to create the ProcedureInfo from Procedure.

Parameters:

proc - procedure

nonceKey - the key to detect duplicate call

convert

@InterfaceAudience.Private
public static ProcedureProtos.Procedure convert(Procedure proc)
                                         throws IOException

Helper to convert the procedure to protobuf. Used by ProcedureStore implementations.

Parameters:

proc - procedure

Throws:

IOException

convert

@InterfaceAudience.Private
public static Procedure convert(ProcedureProtos.Procedure proto)
                         throws IOException

Helper to convert the protobuf procedure. Used by ProcedureStore implementations. TODO: OPTIMIZATION: some of the field never change during the execution (e.g. className, procId, parentId, ...). We can split in 'data' and 'state', and the store may take advantage of it by storing the data only on insert().

Parameters:

proto - procedure protobuf

Throws:

IOException