Module

class Module(module)

Represents a module instance used for model inference in runtime. This is a wrapper of the TCIM module.

Example

Example1: Single model inference.

import tcim_lite as tcim

# Load the TCIM model
module = tcim.runtime.Module.load("model_name.hmm")

# Data preprocess
input_data = cv2.imread("image.png")
...

# Get the total number of model inputs
input_num = module.get_num_inputs()
# For each input
for id in range(0, input_num):
    # Get the input name
    input_name = module.get_input_name(id)
    # Get the information about input data
    input_info = module.get_input_info(input_name)
    # Set input data to the input with the given input name
    module.set_input(input_name, input_data)

# Infer model
module.run()
module.sync()

# Get the total number of outputs
output_num = module.get_num_outputs()
# For each output:
for id in range(0, output_num):
    # Get the output name
    output_name = module.get_output_name(id)
    # Get the information about output data
    output_info = module.get_output_info(output_name).astype(np.float32)
    # Get the output data
    output_data = module.get_output(output_name).astype(np.float32).numpy()

Example2: Multi-model inference.

# Load the first model
part1 = tcim.runtime.Module.load("petr_part1.hmm")
# Set the input of the first model
for name in inputs:
    input_data = inputs[name]
    input_data = np.concatenate([input_data for i in range(batch)], axis=0)
    part1.set_input(name, input_data)

# Load the second model
part2 = tcim.runtime.Module.load("petr_part2.hmm")
# Set the input name of the second model as the output name of the first model
part2_input_name = "pts_bbox_head_transformer_reshape_0_reshape"

# Infer the first model
part1.run()
part1.sync()
# Get the output data of first model
part1_output = part1.get_output(part2_input_name)
# Pass the output data of first model to the second model
part2.set_input(part2_input_name, part1_output)
# Infer the second model
part2.run()
part2.sync()

# Get the output of second model
outputs = []
for i in range(part2.get_num_outputs()):
    output_name = part2.get_output_name(i)
    output = part2.get_output(output_name)
    outputs.append(output)

Constructor for the Module class.

Methods

__init__	Constructor for the Module class.
dump_inout	Saves all input/output tensors of the current module to the specified directory, along with a tester-compatible model.json.
get_custom_msg	Retrieves the the custom message when the model is compiled by option '--custom_msg'.
get_dev_input	Gets the input tensor to the pre-allocated memory with the given tensor name.
get_dev_output	Gets output data from pre-allocated memory on host or Houmo device with the given tensor name.
get_input	Deprecated: Use get_dev_input instead.
get_input_info	Gets the tensor information, such as tensor shape and data type with the given input tensor name.
get_input_name	Gets the name of the id-th input tensor on device.
get_model_version	Retrieves the date when the model is compiled.
get_num_inputs	Gets the total number of input tensors in the network model.
get_num_outputs	Gets the total number of output tensors in the network model.
get_output	Gets output data from pre-allocated memory on host or Houmo device with the given tensor name.
get_output_info	Gets the information about the output tensor of model inference, such as tensor shape and data type with the given output tensor name.
get_output_name	Gets the name of the id-th output tensor.
init_status	Returns the status of the initialization.
load	Loads a built TCIM model with the specified model file and configuration options.
load_dump_input	Loads dumped input tensors from a DumpInOut directory and sets them into the current module.
load_model_info	Loads and returns the model information as a JSON string from a binary model file (.hmm or .hmms).
run	Infers a model.
set_dev_input	Sets the input data to the pre-allocated memory with the given tensor name and input data.
set_dev_output	Sets the output data to the pre-allocated memory on Houmo device with the given tensor name.
set_input	Sets the input data to the pre-allocated memory on host or device with the given tensor name and input data.
set_output	Deprecated: Use set_dev_output instead.
set_stream	Sets a stream to be used for model inference.
sync	Waits for the completion of all preceding tasks in the stream that is related to the module.