Memory

Memory#

A container that describes and stores data. Memory objects can contain data of various types and formats. There are two levels of abstraction:

  1. Memory descriptor engine-agnostic logical description of data (number of dimensions, dimension sizes, and data type), and, optionally, the information about the physical format of data in memory. If this information is not known yet, a memory descriptor can be created with dnnl::memory::format_tag::any. This allows compute-intensive primitives to choose the best format for computation. The user is responsible for reordering the data into the chosen format when formats do not match.

    A memory descriptor can be initialized either by specifying dimensions and a memory format tag or strides for each of them, or by manipulating the dnnl_memory_desc_t structure directly.

    Warning

    The latter approach requires understanding how the physical data representation is mapped to the structure and is discouraged. This topic is discussed in Understanding Memory Formats.

    The user can query the amount of memory required by a memory descriptor using the dnnl::memory::desc::get_size() function. The size of data in general cannot be computed as the product of dimensions multiplied by the size of the data type. So users are required to use this function for better code portability.

    Two memory descriptors can be compared using the equality and inequality operators. The comparison is especially useful when checking whether it is necessary to reorder data from the user’s data format to a primitive’s format.

  2. Memory object an engine-specific object that handles the memory buffer and its description (a memory descriptor). For the CPU engine or with USM, the memory buffer handle is simply a pointer to void. The memory buffer can be queried using dnnl::memory::get_data_handle() and set using dnnl::memory::set_data_handle(). The underlying SYCL buffer, when used, can be queried using dnnl::sycl_interop::get_buffer and set using dnnl::sycl_interop::set_buffer. A memory object can also be queried for the underlying memory descriptor and for its engine using dnnl::memory::get_desc() and dnnl::memory::get_engine().

Along with ordinary memory descriptors with all dimensions being positive, the library supports zero-volume memory descriptors with one or more dimensions set to zero. This is used to support the NumPy* convention. If a zero-volume memory is passed to a primitive, the primitive typically does not perform any computations with this memory. For example:

  • A concatenation primitive would ignore all memory object with zeroes in the concat dimension / axis.

  • A forward convolution with a source memory object with zero in the minibatch dimension would always produce a destination memory object with a zero in the minibatch dimension and perform no computations.

  • However, a forward convolution with a zero in one of the weights dimensions is ill-defined and is considered to be an error by the library because there is no clear definition of what the output values should be.

Memory buffer of a zero-volume memory is never accessed.

// structs

struct dnnl::memory;

// global functions

bool dnnl::operator == (dnnl_data_type_t a, memory::data_type b);
bool dnnl::operator != (dnnl_data_type_t a, memory::data_type b);
bool dnnl::operator == (memory::data_type a, dnnl_data_type_t b);
bool dnnl::operator != (memory::data_type a, dnnl_data_type_t b);
bool dnnl::operator == (dnnl_format_tag_t a, memory::format_tag b);
bool dnnl::operator != (dnnl_format_tag_t a, memory::format_tag b);
bool dnnl::operator == (memory::format_tag a, dnnl_format_tag_t b);
bool dnnl::operator != (memory::format_tag a, dnnl_format_tag_t b);