Level Zero interoperability API#

Overview#

API extensions to interact with the underlying Level Zero run-time. More…

// namespaces

namespace dnnl::ze_interop;

// global functions

dnnl_status_t DNNL_API dnnl_ze_interop_engine_get_cache_blob_id(
    ze_driver_handle_t driver,
    ze_device_handle_t device,
    size_t* size,
    uint8_t* cache_blob_id
    );

dnnl_status_t DNNL_API dnnl_ze_interop_engine_get_cache_blob(
    dnnl_engine_t engine,
    size_t* size,
    uint8_t* cache_blob
    );

dnnl_status_t DNNL_API dnnl_ze_interop_engine_create_from_cache_blob(
    dnnl_engine_t* engine,
    ze_driver_handle_t driver,
    ze_device_handle_t device,
    ze_context_handle_t context,
    size_t size,
    const uint8_t* cache_blob
    );

dnnl_status_t DNNL_API dnnl_ze_interop_engine_create(
    dnnl_engine_t* engine,
    ze_driver_handle_t driver,
    ze_device_handle_t device,
    ze_context_handle_t context
    );

dnnl_status_t DNNL_API dnnl_ze_interop_engine_get_context(
    dnnl_engine_t engine,
    ze_context_handle_t* context
    );

dnnl_status_t DNNL_API dnnl_ze_interop_engine_get_device(
    dnnl_engine_t engine,
    ze_device_handle_t* device
    );

dnnl_status_t DNNL_API dnnl_ze_interop_engine_get_driver(
    dnnl_engine_t engine,
    ze_driver_handle_t* driver
    );

dnnl_status_t DNNL_API dnnl_ze_interop_stream_create(
    dnnl_stream_t* stream,
    dnnl_engine_t engine,
    ze_command_list_handle_t list,
    int profiling
    );

dnnl_status_t DNNL_API dnnl_ze_interop_stream_get_list(
    dnnl_stream_t stream,
    ze_command_list_handle_t* list
    );

dnnl_status_t DNNL_API dnnl_ze_interop_memory_create(
    dnnl_memory_t* memory,
    const_dnnl_memory_desc_t memory_desc,
    dnnl_engine_t engine,
    int nhandles,
    void** handles
    );

dnnl_status_t DNNL_API dnnl_ze_interop_primitive_execute(
    const_dnnl_primitive_t primitive,
    dnnl_stream_t stream,
    int nargs,
    const dnnl_exec_arg_t* args,
    int ndeps,
    const ze_event_handle_t* deps,
    ze_event_handle_t* return_event
    );

Detailed Documentation#

API extensions to interact with the underlying Level Zero run-time.

Global Functions#

dnnl_status_t DNNL_API dnnl_ze_interop_engine_get_cache_blob_id(
    ze_driver_handle_t driver,
    ze_device_handle_t device,
    size_t* size,
    uint8_t* cache_blob_id
    )

Retrieves a cache blob ID for the Level Zero device.

Warning

This API is intended to be used with dnnl_ze_interop_engine_get_cache_blob() and dnnl_ze_interop_engine_create_from_cache_blob(). The returned cache blob ID can only be used as an ID of the cache blob returned by dnnl_ze_interop_engine_get_cache_blob().

Note

The cache blob ID can be empty (size will be 0 and cache_blob_id will be nullptr) if oneDNN doesn’t have anything to put in the cache blob. (dnnl_ze_interop_engine_get_cache_blob will return an empty cache blob).

Parameters:

driver	A Level Zero driver.
device	A Level Zero device.
size	Size of the cache blob ID in bytes.
cache_blob_id	Cache blob id of size `size`. If the `cache_blob_id` is nullptr then the size of the cache blob ID is returned in `size`.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ze_interop_engine_get_cache_blob(
    dnnl_engine_t engine,
    size_t* size,
    uint8_t* cache_blob
    )

Retrieves a cache blob associated with the given engine.

Note

The cache blob can be empty (size will be 0 and cache_blob will be nullptr) if oneDNN doesn’t have anything to put in the cache blob. It’s the user’s responsibility to check whether it’s empty prior to passing it to dnnl_ze_interop_engine_create_from_cache_blob().

Parameters:

engine	Engine to query for the cache blob.
size	Size of the cache blob in bytes.
cache_blob	Cache blob of size `size`. If the `cache_blob` is nullptr then the size of the cache blob is returned in `size`.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ze_interop_engine_create_from_cache_blob(
    dnnl_engine_t* engine,
    ze_driver_handle_t driver,
    ze_device_handle_t device,
    ze_context_handle_t context,
    size_t size,
    const uint8_t* cache_blob
    )

Creates an engine from the given cache blob.

Parameters:

engine	Output engine.
driver	The Level Zero driver that this engine will encapsulate.
device	The Level Zero device that this engine will encapsulate.
context	The Level Zero context (containing the device) that this engine will use for all operations.
size	Size of the cache blob in bytes.
cache_blob	Cache blob of size `size`.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ze_interop_engine_create(
    dnnl_engine_t* engine,
    ze_driver_handle_t driver,
    ze_device_handle_t device,
    ze_context_handle_t context
    )

Creates an engine associated with a Level Zero device and a Level Zero context.

Parameters:

engine	Output engine.
driver	Pointer to the Level Zero driver to use for the engine.
device	Pointer to the Level Zero device to use for the engine.
context	Pointer to the Level Zero context to use for the engine.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ze_interop_engine_get_context(
    dnnl_engine_t engine,
    ze_context_handle_t* context
    )

Returns the Level Zero context associated with an engine.

Parameters:

engine	Engine to query.
context	Pointer to the underlying Level Zero context of the engine.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ze_interop_engine_get_device(
    dnnl_engine_t engine,
    ze_device_handle_t* device
    )

Returns the Level Zero device associated with an engine.

Parameters:

engine	Engine to query.
device	Pointer to the underlying Level Zero device of the engine.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ze_interop_engine_get_driver(
    dnnl_engine_t engine,
    ze_driver_handle_t* driver
    )

Returns the Level Zero driver associated with an engine.

Parameters:

engine	Engine to query.
driver	Pointer to the underlying Level Zero driver of the engine.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ze_interop_stream_create(
    dnnl_stream_t* stream,
    dnnl_engine_t engine,
    ze_command_list_handle_t list,
    int profiling
    )

Creates an execution stream for a given engine associated with a Level Zero command list.

Parameters:

stream	Output execution stream.
engine	Engine to create the execution stream on.
list	Level Zero command list to use.
profiling	Flag enabling GPU kernels profiling.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ze_interop_stream_get_list(
    dnnl_stream_t stream,
    ze_command_list_handle_t* list
    )

Returns the Level Zero command list associated with an execution stream.

Parameters:

stream	Execution stream to query.
list	Output Level Zero command list.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ze_interop_memory_create(
    dnnl_memory_t* memory,
    const_dnnl_memory_desc_t memory_desc,
    dnnl_engine_t engine,
    int nhandles,
    void** handles
    )

Creates a memory object.

Unless handle is equal to DNNL_MEMORY_NONE or DNNL_MEMORY_ALLOCATE, the constructed memory object will have the underlying buffer set. In this case, the buffer will be initialized as if dnnl_memory_set_data_handle() had been called.

Parameters:

memory	Output memory object.
memory_desc	Memory descriptor.
engine	Engine to use.
nhandles	Number of handles.
handles	Handles of the memory buffers to use as underlying storages. A USM pointer to the user-allocated buffer. In this case the library doesn’t own the buffer. The DNNL_MEMORY_ALLOCATE special value. Instructs the library to allocate the buffer for the memory object. In this case the library owns the buffer. The DNNL_MEMORY_NONE specific value. Instructs the library to create memory object without an underlying buffer.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ze_interop_primitive_execute(
    const_dnnl_primitive_t primitive,
    dnnl_stream_t stream,
    int nargs,
    const dnnl_exec_arg_t* args,
    int ndeps,
    const ze_event_handle_t* deps,
    ze_event_handle_t* return_event
    )

Executes computations specified by the primitive in a specified stream and returns a Level Zero event.

Parameters:

primitive	Primitive to execute.
stream	Stream to use.
nargs	Number of arguments.
args	Array of arguments. Each argument is an <index, dnnl_memory_t> pair. The index is one of the `DNNL_ARG_*` values such as `DNNL_ARG_SRC`. Unless runtime shapes are used (see DNNL_RUNTIME_DIM_VAL), the memory object must have the same memory descriptor as that returned by dnnl_primitive_desc_query_md (dnnl_query_exec_arg_md, index).
ndeps	Number of dependencies.
deps	A pointer to a vector of size `ndeps` that contains dependencies.
return_event	Output event.

Returns:

dnnl_success on success and a status describing the error otherwise.

Level Zero interoperability API

Contents

Level Zero interoperability API#

Overview#

Detailed Documentation#

Global Functions#