struct
Model
Represents a model that's loaded and ready for execution.
Do not instantiate this object directly. Instead, create it with
InferenceSession.load()
.
For example:
var session = engine.InferenceSession()
var model = session.load("bert-base-uncased")
Then you can run inference by passing your inputs to execute()
as a NumPy array, a
TensorMap
, or one
of the other tensor types.
Implemented traits
AnyType
,
CollectionElement
,
Copyable
,
Movable
Methods
__moveinit__
__moveinit__(inout self: Self, owned existing: Self)
Move initializer for model.
Args:
- existing (
Self
): Model to move.
__del__
__del__(owned self: Self)
Destructor for Model.
execute
execute(self: Self, inputs: TensorMap) -> TensorMap
Execute model with given inputs.
Args:
- inputs (
TensorMap
): A tensor map with input names as keys and inputs as values.
Returns:
A TensorMap with output names as keys.
execute(self: Self, inputs: PythonObject) -> TensorMap
Execute model with given inputs.
Args:
- inputs (
PythonObject
): Inputs as a Python object, which must be a dictionary with string keys (matching input names) and NumPy array values.
Returns:
A TensorMap with output names as keys.
execute(self: Self, *inputs: NamedTensor) -> TensorMap
Execute model with given inputs.
Args:
- *inputs (
NamedTensor
): A variadic list of NamedTensor values.
Returns:
A TensorMap with output names as keys.
execute(self: Self, *inputs: Tuple[StringLiteral, EngineNumpyView]) -> TensorMap
Execute model with given inputs.
Args:
- *inputs (
Tuple[StringLiteral, EngineNumpyView]
): A variadic list of tuples with first element of tuple is input name and second element is non owning view of a Numpy array.
Returns:
A TensorMap with output names as keys.
execute[type: DType](self: Self, name: String, input: Tensor[type]) -> TensorMap
Execute model with given input.
Parameters:
- type (
DType
): DType of input tensor.
Args:
- name (
String
): Name of the input tensor. - input (
Tensor[type]
): Input tensor to the model.
Returns:
A TensorMap with output names as keys.
execute(self: Self, name: String, inout input: PythonObject) -> TensorMap
Execute model with given input.
Args:
- name (
String
): Name of the input tensor. - input (
PythonObject
): Input to the model as numpy array.
Returns:
A TensorMap with output names as keys.
execute[type1: DType, type2: DType](self: Self, name1: String, input1: Tensor[type1], name2: String, input2: Tensor[type2]) -> TensorMap
Execute model with given inputs.
Parameters:
- type1 (
DType
): DType of first input tensor. - type2 (
DType
): DType of second input tensor.
Args:
- name1 (
String
): Name of the first input tensor. - input1 (
Tensor[type1]
): First Input tensor to the model. - name2 (
String
): Name of the second input tensor. - input2 (
Tensor[type2]
): Second Input tensor to the model.
Returns:
A TensorMap with output names as keys.
execute(self: Self, name1: String, inout input1: PythonObject, name2: String, inout input2: PythonObject) -> TensorMap
Execute model with given inputs.
Args:
- name1 (
String
): Name of the first input tensor. - input1 (
PythonObject
): First Input to the model as numpy array. - name2 (
String
): Name of the second input tensor. - input2 (
PythonObject
): Second Input to the model as numpy array.
Returns:
A TensorMap with output names as keys.
execute[type1: DType, type2: DType, type3: DType](self: Self, name1: String, input1: Tensor[type1], name2: String, input2: Tensor[type2], name3: String, input3: Tensor[type3]) -> TensorMap
Execute model with given inputs.
Parameters:
- type1 (
DType
): DType of first input tensor. - type2 (
DType
): DType of second input tensor. - type3 (
DType
): DType of third input tensor.
Args:
- name1 (
String
): Name of the first input tensor. - input1 (
Tensor[type1]
): First Input tensor to the model. - name2 (
String
): Name of the second input tensor. - input2 (
Tensor[type2]
): Second Input tensor to the model. - name3 (
String
): Name of the third input tensor. - input3 (
Tensor[type3]
): Third Input tensor to the model.
Returns:
A TensorMap with output names as keys.
execute(self: Self, name1: String, inout input1: PythonObject, name2: String, inout input2: PythonObject, name3: String, inout input3: PythonObject) -> TensorMap
Execute model with given inputs.
Args:
- name1 (
String
): Name of the first input tensor. - input1 (
PythonObject
): First Input to the model as numpy array. - name2 (
String
): Name of the second input tensor. - input2 (
PythonObject
): Second Input to the model as numpy array. - name3 (
String
): Name of the third input tensor. - input3 (
PythonObject
): Third Input to the model as numpy array.
Returns:
A TensorMap with output names as keys.
num_model_inputs
num_model_inputs(self: Self) -> Int
Gets the number of inputs of the model.
Returns:
Number of inputs of model.
get_model_input_names
get_model_input_names(self: Self) -> List[String]
Gets the names of model inputs.
Returns:
Input names of the model.
num_model_outputs
num_model_outputs(self: Self) -> Int
Gets the number of outputs of the model.
Returns:
Number of model outputs.
get_model_output_names
get_model_output_names(self: Self) -> List[String]
Gets the names of model outputs.
Returns:
Output names of the model.
get_model_input_metadata
get_model_input_metadata(self: Self) -> List[EngineTensorSpec]
Get metadata about the model's input tensors, as a list of EngineTensorSpec
objects.
Returns:
Metadata list of the model's input tensors.
get_model_output_metadata
get_model_output_metadata(self: Self) -> List[EngineTensorSpec]
Get metadata about the model's output tensors, as a list of EngineTensorSpec
objects.
Returns:
Metadata list of the model's output tensors.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!
If you'd like to share more information, please report an issue on GitHub
😔 What went wrong?