Level Zero Interoperability

API Reference

Overview

oneDNN uses the Level Zero runtime for GPU engines to interact with the GPU. Users may need to use oneDNN with other code that uses Level Zero. For that purpose, the library provides API extensions to interoperate with underlying Level Zero objects. This interoperability API is defined in the dnnl_ze.hpp header.

The interoperability API is provided for two scenarios:

• Construction of oneDNN objects based on existing Level Zero objects.

• Accessing Level Zero objects for existing oneDNN objects.

The mapping between oneDNN and Level Zero objects is provided in the following table:

oneDNN object	Level Zero object(s)
Engine	ze_driver_handle_t , ze_device_handle_t , and ze_context_handle_t
Stream	ze_command_list_handle_t
Memory (USM-based)	Unified Shared Memory (USM) pointer

The table below summarizes how to construct oneDNN objects based on Level Zero objects and how to query underlying Level Zero objects for existing oneDNN objects.

oneDNN object	API to construct oneDNN object	API to access Level Zero object(s)
Engine	dnnl::ze_interop::make_engine(ze_driver_handle_t, ze_device_handle_t, ze_context_handle_t)	dnnl::ze_interop::get_driver(const engine &)
		dnnl::ze_interop::get_device(const engine &)
		dnnl::ze_interop::get_context(const engine &)
Stream	dnnl::ze_interop::make_stream(const engine &, ze_command_list_handle_t, bool)	dnnl::ze_interop::get_list(const stream &)
Memory (USM-based)	dnnl::ze_interop::make_memory(const memory::desc &, const engine &, void *)	dnnl::memory::get_data_handle()

Level Zero USM Interfaces for Memory Objects

The memory model in Level Zero is based on Level Zero USM, which provides the ability to allocate and use memory in a uniform way on host and Level Zero devices.

To construct a oneDNN memory object, use the following interface:

• dnnl::ze_interop::make_memory(const memory::desc &, const engine &, void *)

  Constructs a USM-based memory object. The handle could be one of special values DNNL_MEMORY_ALLOCATE or DNNL_MEMORY_NONE, or it could be a user-provided USM pointer.

Handling Dependencies

Note

Only Level Zero in-order immediate command lists are supported.

Unlike the OpenCL API, the Level Zero API doesn’t have a notion of retain/release mechanics when it comes to managing its objects. The object, once created, must be managed by some party. Thus, objects created by oneDNN are managed by oneDNN, and objects created by the user and passed to oneDNN are not managed by oneDNN, the library only stores references to objects. It means it’s the user responsibility to manage the lifetime of objects created on their side while the library operates with them.

oneDNN provides two mechanisms to handle dependencies:

1. dnnl::ze_interop::execute() interface

   This interface enables the user to pass dependencies between primitives using Level Zero events. In this case, the user is responsible for passing proper dependencies for every primitive execution.

2. In-order oneDNN stream

   oneDNN enables the user to create in-order streams in which submitted primitives are executed in the order they were submitted. Using in-order streams prevents possible read-before-write or concurrent read/write issues.

Note

The access interfaces do not retain the Level Zero object. It is the user’s responsibility to retain the returned Level Zero object if necessary.

Note

Current version of API manages Level Zero events lifetime, and it’s attached to the stream lifetime. All returned events after stream destruction become invalidated.

Note

USM memory doesn’t support retain/release Level Zero semantics. When constructing a oneDNN memory object using a user-provided USM pointer oneDNN doesn’t own the provided memory. It’s the user’s responsibility to manage its lifetime.