LlmInference.LlmInferenceOptions

public static abstract class LlmInference.LlmInferenceOptions

Options for setting up an LlmInference.

Nested Classes

class

LlmInference.LlmInferenceOptions.Builder

Builder for LlmInference.LlmInferenceOptions.

Public Constructors

LlmInferenceOptions()

Public Methods

static LlmInference.LlmInferenceOptions.Builder	builder() Instantiates a new LlmInferenceOptions builder.
abstract int	maxTokens() The total length of the kv-cache.
abstract String	modelPath() The path that points to the tflite model file.
abstract int	randomSeed() Random seed for sampling tokens.
abstract float	temperature() Randomness when decoding the next token.
abstract int	topK() Top K number of tokens to be sampled from for each decoding step.

Inherited Methods

From class com.google.mediapipe.tasks.core.TaskOptions

Any	convertToAnyProto() Converts a MediaPipe Tasks task-specific options to an proto3 ERROR(/Any) message.
CalculatorOptions	convertToCalculatorOptionsProto() Converts a MediaPipe Tasks task-specific options to a ERROR(/CalculatorOptions) protobuf message.

From class java.lang.Object

boolean	equals(Object arg0)
final Class<?>	getClass()
int	hashCode()
final void	notify()
final void	notifyAll()
String	toString()
final void	wait(long arg0, int arg1)
final void	wait(long arg0)
final void	wait()

Public Constructors

public LlmInferenceOptions ()

Public Methods

public static LlmInference.LlmInferenceOptions.Builder builder ()

Instantiates a new LlmInferenceOptions builder.

public abstract int maxTokens ()

The total length of the kv-cache. In other words, this is the total number of input + output tokens the model needs to handle.

public abstract String modelPath ()

The path that points to the tflite model file.

public abstract int randomSeed ()

Random seed for sampling tokens.

public abstract float temperature ()

Randomness when decoding the next token. A value of 0.0f means greedy decoding.

public abstract int topK ()

Top K number of tokens to be sampled from for each decoding step. A value of 1 means greedy decoding.