enable_host_lazy_loading

Option.enable_host_lazy_loading(enable=False)

Controls lazy buffer allocation strategy during model loading phase.

When enabled, buffer allocation and initialization on the host are deferred until they are actually needed. This strategy reduces the peak host memory footprint during model loading via load_from_file and load_model. This is beneficial for memory-constrained environments. The trade-off is increased model loading latency due to on-demand buffer initialization.

When disabled (default), all buffers are pre-allocated and initialized during model loading, resulting in faster model load completion but higher memory usage.

Parameters:

enable (bool, optional) --

Specifies the buffer allocation strategy for model loading.

• True: Save host memory by deferring buffer allocation and initialization.

• False (default): Speed up model loading (more memory).

Returns:

Returns the Option instance for method chaining.

Return type:

Option