org.apache.hadoop.fs.aliyun.oss

类 AliyunOSSUtils

java.lang.Object

org.apache.hadoop.fs.aliyun.oss.AliyunOSSUtils

public final class AliyunOSSUtils
extends Object

Utility methods for Aliyun OSS code.

方法概要

限定符和类型	方法和说明
static long	calculatePartSize(long contentLength, long minPartSize) Calculate a proper size of multipart piece.
static File	createTmpFileForWrite(String path, long size, org.apache.hadoop.conf.Configuration conf) Demand create the directory allocator, then create a temporary file.
static org.apache.hadoop.conf.Configuration	excludeIncompatibleCredentialProviders(org.apache.hadoop.conf.Configuration config, Class<? extends org.apache.hadoop.fs.FileSystem> fileSystemClass) Note: This function is copied from ProviderUtils There are certain integrations of the credential provider API in which a recursive dependency between the provider and the hadoop filesystem abstraction causes a problem.
static com.aliyun.oss.common.auth.CredentialsProvider	getCredentialsProvider(URI uri, org.apache.hadoop.conf.Configuration conf) Create credential provider specified by configuration, or create default credential provider if not specified.
static long	getMultipartSizeProperty(org.apache.hadoop.conf.Configuration conf, String property, long defVal) Get a size property from the configuration: this property must be at least equal to Constants.MULTIPART_MIN_SIZE.
static String	getValueWithKey(org.apache.hadoop.conf.Configuration conf, String key) Used to get password from configuration.
static int	intPositiveOption(org.apache.hadoop.conf.Configuration conf, String key, int defVal)
static String	maybeAddTrailingSlash(String key) Turns a path (relative or otherwise) into an OSS key, adding a trailing "/" if the path is not the root and does not already have a "/" at the end.
static boolean	objectRepresentsDirectory(String name, long size) Check if OSS object represents a directory.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

方法详细资料

intPositiveOption

public static int intPositiveOption(org.apache.hadoop.conf.Configuration conf,
                                    String key,
                                    int defVal)

getValueWithKey

public static String getValueWithKey(org.apache.hadoop.conf.Configuration conf,
                                     String key)
                              throws IOException

Used to get password from configuration.

参数:

conf - configuration that contains password information

key - the key of the password

返回:

the value for the key

抛出:

IOException - if failed to get password from configuration

calculatePartSize

public static long calculatePartSize(long contentLength,
                                     long minPartSize)

Calculate a proper size of multipart piece. If minPartSize is too small, the number of multipart pieces may exceed the limit of Constants.MULTIPART_UPLOAD_PART_NUM_LIMIT.

参数:

contentLength - the size of file.

minPartSize - the minimum size of multipart piece.

返回:

a revisional size of multipart piece.

excludeIncompatibleCredentialProviders

public static org.apache.hadoop.conf.Configuration excludeIncompatibleCredentialProviders(org.apache.hadoop.conf.Configuration config,
                                                                                          Class<? extends org.apache.hadoop.fs.FileSystem> fileSystemClass)
                                                                                   throws IOException

Note: This function is copied from ProviderUtils There are certain integrations of the credential provider API in which a recursive dependency between the provider and the hadoop filesystem abstraction causes a problem. These integration points need to leverage this utility method to remove problematic provider types from the existing provider path within the configuration.

参数:

config - the existing configuration with provider path

fileSystemClass - the class which providers must be compatible

返回:

Configuration clone with new provider path

抛出:

IOException

getCredentialsProvider

public static com.aliyun.oss.common.auth.CredentialsProvider getCredentialsProvider(URI uri,
                                                                                    org.apache.hadoop.conf.Configuration conf)
                                                                             throws IOException

Create credential provider specified by configuration, or create default credential provider if not specified.

参数:

uri - uri passed by caller

conf - configuration

返回:

a credential provider

抛出:

IOException - on any problem. Class construction issues may be nested inside the IOE.

maybeAddTrailingSlash

public static String maybeAddTrailingSlash(String key)

Turns a path (relative or otherwise) into an OSS key, adding a trailing "/" if the path is not the root and does not already have a "/" at the end.

参数:

key - OSS key or ""

返回:

the with a trailing "/", or, if it is the root key, "".

objectRepresentsDirectory

public static boolean objectRepresentsDirectory(String name,
                                                long size)

Check if OSS object represents a directory.

参数:

name - object key

size - object content length

返回:

true if object represents a directory

createTmpFileForWrite

public static File createTmpFileForWrite(String path,
                                         long size,
                                         org.apache.hadoop.conf.Configuration conf)
                                  throws IOException

Demand create the directory allocator, then create a temporary file.

参数:

path - prefix for the temporary file

size - the size of the file that is going to be written

conf - the Configuration object

返回:

a unique temporary file

抛出:

IOException - IO problems

getMultipartSizeProperty

public static long getMultipartSizeProperty(org.apache.hadoop.conf.Configuration conf,
                                            String property,
                                            long defVal)

Get a size property from the configuration: this property must be at least equal to Constants.MULTIPART_MIN_SIZE. If it is too small, it is rounded up to that minimum, and a warning printed.

参数:

conf - configuration

property - property name

defVal - default value

返回:

the value, guaranteed to be above the minimum size