public interface EncodingRegistry
| Modifier and Type | Method and Description |
|---|---|
Encoding |
getEncoding(EncodingType encodingType)
Returns the encoding with the given type.
|
java.util.Optional<Encoding> |
getEncoding(java.lang.String encodingName)
Returns the encoding with the given name, if it exists.
|
Encoding |
getEncodingForModel(ModelType modelType)
Returns the encoding that is used for the given model type.
|
java.util.Optional<Encoding> |
getEncodingForModel(java.lang.String modelName)
Returns the encoding that is used for the given model type, if it exists.
|
EncodingRegistry |
registerCustomEncoding(Encoding encoding)
Registers a new custom encoding with the given name.
|
EncodingRegistry |
registerGptBytePairEncoding(GptBytePairEncodingParams parameters)
Registers a new byte pair encoding with the given name.
|
java.util.Optional<Encoding> getEncoding(java.lang.String encodingName)
getEncoding(EncodingType) or getEncodingForModel(ModelType) for
built-in encodings.encodingName - the name of the encodingEncoding getEncoding(EncodingType encodingType)
encodingType - the type of the encodingjava.util.Optional<Encoding> getEncodingForModel(java.lang.String modelName)
getEncodingForModel(ModelType) for built-in encodings.
Note that you can use this method to retrieve the correct encodings for snapshots of models, for example "gpt-4-0314" or "gpt-3.5-turbo-0301".
modelName - the name of the model to get the encoding forEncoding getEncodingForModel(ModelType modelType)
modelType - the model typeEncodingRegistry registerGptBytePairEncoding(GptBytePairEncodingParams parameters)
parameters - the parameters for the encodingjava.lang.IllegalArgumentException - if the encoding name is already registeredGptBytePairEncodingParamsEncodingRegistry registerCustomEncoding(Encoding encoding)
encoding - the encodingjava.lang.IllegalArgumentException - if the encoding name is already registered