Option

class Option(option: [<class 'tcim_lite.runtime.WeightManager'>, <class 'tcim_lite.runtime.DevManager'>, <class 'int'>] = 0)

Static configuration options for loading models.

Constructor for the Option class.

Parameters:

option (int or WeightManager) --

Initialization options:

  • The logical ID of the Houmo device used for loading and inferring a model. The device 0 is used by default.

  • The WeightManager object to manage the shared weight memory.

Methods

__init__

Constructor for the Option class.

enable_host_lazy_loading

Controls lazy buffer allocation strategy during model loading phase.

enable_io_lazy_mode

Controls when input and output buffers used during inference are allocated on the host.

enable_profile

Enable profiling mode.

set_dummy_tensors

Sets a list of tensor names of a model.

set_model_offset

Sets the byte offset and size of the model data within the file or buffer.