Compiled Partition#

Overview#

A compiled partition represents the generated kernels specialized for a partition on a target hardware (engine) with input and output information specified by the logical tensors. More…

// typedefs

typedef struct dnnl_graph_compiled_partition* dnnl_graph_compiled_partition_t;
typedef const struct dnnl_graph_compiled_partition* const_dnnl_graph_compiled_partition_t;

// structs

struct dnnl_graph_inplace_pair_t;

// classes

class dnnl::graph::compiled_partition;

// global functions

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_create(
    dnnl_graph_compiled_partition_t* compiled_partition,
    dnnl_graph_partition_t partition
    );

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_execute(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    dnnl_stream_t stream,
    size_t num_inputs,
    const_dnnl_graph_tensor_t* inputs,
    size_t num_outputs,
    const_dnnl_graph_tensor_t* outputs
    );

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_execute_v2(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    dnnl_stream_t stream,
    size_t num_inputs,
    const_dnnl_graph_tensor_t* inputs,
    size_t num_outputs,
    const_dnnl_graph_tensor_t* outputs,
    const_dnnl_graph_tensor_t scratchpad
    );

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_destroy(dnnl_graph_compiled_partition_t compiled_partition);

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_query_logical_tensor(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    size_t tid,
    dnnl_graph_logical_tensor_t* lt
    );

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_get_inplace_ports(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    size_t* num_inplace_pairs,
    const dnnl_graph_inplace_pair_t** inplace_pairs
    );

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_get_scratchpad_logical_tensor(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    dnnl_graph_logical_tensor_t* lt
    );

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_query_impl_info(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    const char** impl_info
    );

Detailed Documentation#

A compiled partition represents the generated kernels specialized for a partition on a target hardware (engine) with input and output information specified by the logical tensors.

Typedefs#

typedef struct dnnl_graph_compiled_partition* dnnl_graph_compiled_partition_t

A compiled partition handle.

typedef const struct dnnl_graph_compiled_partition* const_dnnl_graph_compiled_partition_t

A constant compiled partition handle.

Global Functions#

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_create(
    dnnl_graph_compiled_partition_t* compiled_partition,
    dnnl_graph_partition_t partition
    )

Creates a new compiled partition handle.

Parameters:

compiled_partition	The handle of output compiled partition.
partition	The handle of input partition.

Returns:

dnnl_success on success or a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_execute(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    dnnl_stream_t stream,
    size_t num_inputs,
    const_dnnl_graph_tensor_t* inputs,
    size_t num_outputs,
    const_dnnl_graph_tensor_t* outputs
    )

Executes a compiled partition.

Parameters:

compiled_partition	The handle of target compiled partition.
stream	The stream used for execution.
num_inputs	The number of input tensors.
inputs	A list of input tensors.
num_outputs	The number of output tensors.
outputs	A non-empty list of output tensors.

Returns:

dnnl_success on success or a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_execute_v2(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    dnnl_stream_t stream,
    size_t num_inputs,
    const_dnnl_graph_tensor_t* inputs,
    size_t num_outputs,
    const_dnnl_graph_tensor_t* outputs,
    const_dnnl_graph_tensor_t scratchpad
    )

Executes a compiled partition.

Note

The user can provide a scratchpad buffer for execution. If not provided, the library will allocate an internal scratchpad buffer for the execution. For user-provided scratchpad buffer, it should have a size no less than the value provided by dnnl_graph_compiled_partition_get_scratchpad_logical_tensor API. The user is responsible for the memory management of the user-provided scratchpad buffer, including allocation, deallocation, and thread-safety.

Parameters:

compiled_partition	The handle of target compiled partition.
stream	The stream used for execution.
num_inputs	The number of input tensors.
inputs	A list of input tensors.
num_outputs	The number of output tensors.
outputs	A non-empty list of output tensors.
scratchpad	User-provided scratchpad tensor.

Returns:

dnnl_success on success or a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_destroy(dnnl_graph_compiled_partition_t compiled_partition)

Destroys a compiled partition.

Parameters:

compiled_partition

The compiled partition to be destroyed.

Returns:

dnnl_success on success or a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_query_logical_tensor(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    size_t tid,
    dnnl_graph_logical_tensor_t* lt
    )

Queries an input or output logical tensor according to tensor ID. If the tensor ID doesn’t belong to any input or output of the compiled partition, an error status dnnl_invalid_arguments will be returned by the API.

Parameters:

compiled_partition	The handle of target compiled_partition.
tid	The unique id of required tensor.
lt	The output logical tensor.

Returns:

dnnl_success on success or a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_get_inplace_ports(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    size_t* num_inplace_pairs,
    const dnnl_graph_inplace_pair_t** inplace_pairs
    )

Returns the hint of in-place pairs from a compiled partition. It indicates that an input and an output of the partition can share the same memory buffer for computation. In-place computation helps to reduce the memory footprint and improves cache locality. But since the library may not have a global view of user’s application, it’s possible that the tensor with input_id is used at other places in user’s computation graph. In this case, the user should take the in-place pair as a hint and pass a different memory buffer for output tensor to avoid overwriting the input memory buffer which will probably cause unexpected incorrect results.

Parameters:

compiled_partition	The handle of target compiled_partition.
num_inplace_pairs	The number of in-place pairs.
inplace_pairs	The handle of in-place pairs.

Returns:

dnnl_success on success or a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_get_scratchpad_logical_tensor(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    dnnl_graph_logical_tensor_t* lt
    )

Queries the scratchpad logical tensor from a compiled partition. The scratchpad logical tensor describes the required scratchpad buffer for execution. The data type is u8 and the layout is strided with a single dimension equal to the scratchpad size in bytes. If no scratchpad is needed, the logical tensor will have a zero size dimension.

Parameters:

compiled_partition	The handle of target compiled_partition.
lt	The output logical tensor describing the scratchpad.

Returns:

dnnl_success on success or a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_query_impl_info(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    const char** impl_info
    )

Queries the implementation name of a compiled partition. The name identifies the kernel dispatched by the backend for the compiled partition and matches the kernel field reported in verbose output.

Note

The returned string is owned by the compiled partition and remains valid until the compiled partition is destroyed.

Parameters:

compiled_partition	The handle of target compiled_partition.
impl_info	The output implementation name.

Returns:

dnnl_success on success or a status describing the error otherwise.

Compiled Partition

Contents

Compiled Partition#

Overview#

Detailed Documentation#

Typedefs#

Global Functions#