GpuAccelerationConfig.GpuInferenceUsage

Page Summary

• GpuAccelerationConfig.GpuInferenceUsage is an enum defining GPU inference preferences for initialization vs. inference time.

• GPU_INFERENCE_PREFERENCE_FAST_SINGLE_ANSWER is used when the delegate will be used only once, prioritizing faster initialization.

• GPU_INFERENCE_PREFERENCE_SUSTAINED_SPEED is used when the same delegate will be used repeatedly, prioritizing maximizing throughput.

public static final enum GpuAccelerationConfig.GpuInferenceUsage extends Enum<GpuAccelerationConfig.GpuInferenceUsage>

GPU inference preference for initialization time vs. inference time.

Inherited Method Summary

From class java.lang.Enum

final Object	clone()
final int	compareTo(GpuAccelerationConfig.GpuInferenceUsage arg0)
int	compareTo(Object arg0)
final Optional<EnumDesc<GpuAccelerationConfig.GpuInferenceUsage>>	describeConstable()
final boolean	equals(Object arg0)
final void	finalize()
final Class<GpuAccelerationConfig.GpuInferenceUsage>	getDeclaringClass()
final int	hashCode()
final String	name()
final int	ordinal()
String	toString()
static <T extends Enum<T>> T	valueOf(Class<T> arg0, String arg1)

From class java.lang.Object

Object	clone()
boolean	equals(Object arg0)
void	finalize()
final Class<?>	getClass()
int	hashCode()
final void	notify()
final void	notifyAll()
String	toString()
final void	wait(long arg0, int arg1)
final void	wait(long arg0)
final void	wait()

From interface java.lang.constant.Constable

abstract Optional<? extends ConstantDesc>

describeConstable()

From interface java.lang.Comparable

abstract int

compareTo(GpuAccelerationConfig.GpuInferenceUsage arg0)

Enum Values

public static final GpuAccelerationConfig.GpuInferenceUsage GPU_INFERENCE_PREFERENCE_FAST_SINGLE_ANSWER

Delegate will be used only once, therefore, bootstrap/init time should be taken into account.

public static final GpuAccelerationConfig.GpuInferenceUsage GPU_INFERENCE_PREFERENCE_SUSTAINED_SPEED

Prefer maximizing the throughput. Same delegate will be used repeatedly on multiple inputs.