.. index:: pair: page; Reorder .. _doxid-dev_guide_reorder: Reorder ======= :ref:`API Reference ` General ~~~~~~~ The reorder primitive copies data between different memory formats but doesn't change the tensor from mathematical perspective (the variable names follow the standard :ref:`Naming Conventions `): .. math:: \dst(\overline{x}) = \src(\overline{x}) As described in :ref:`Basic Concepts ` in order to achieve the best performance some primitives (such as convolution) require special memory format which is typically referred to as an optimized memory format. The optimized memory format may match or may not match memory format that data is currently kept in. In this case a user can use reorder primitive to copy (reorder) the data between the memory formats. Using the attributes and post-ops users can also use reorder primitive to quantize the data (and if necessary change the memory format simultaneously). Execution Arguments ~~~~~~~~~~~~~~~~~~~ When executed, the inputs and outputs should be mapped to an execution argument index as specified by the following table. ========================================================== ===================================== ======= Argument Index Type ========================================================== ===================================== ======= :math:`\src` DNNL_ARG_FROM Input :math:`\dst` DNNL_ARG_TO Output :math:`src scale` DNNL_ARG_ATTR_SCALES | DNNL_ARG_FROM Input :math:`dst scale` DNNL_ARG_ATTR_SCALES | DNNL_ARG_TO Input :ref:`scratchpad ` DNNL_ARG_SCRATCHPAD Output ========================================================== ===================================== ======= Implementation Details ~~~~~~~~~~~~~~~~~~~~~~ General Notes ------------- #. The reorder primitive requires the source and destination tensors to have the same shape. Implicit broadcasting is not supported. #. While in most of the cases the reorder should be able to handle arbitrary source and destination memory formats and data types, it might happen than some combinations are not implemented. For instance: * Reorder implementations between weights in non-plain memory formats might be limited (but if encountered in real practice should be treated as a bug and reported to oneDNN team); * Weights in one Winograd format cannot be reordered to the weights of the other Winograd format; * Quantized weights for convolution with :ref:`dnnl_s8 ` source data type cannot be dequantized back to the :ref:`dnnl_f32 ` data type; * Only reference support is available for reorders to or from ``f8_e4m3``. * Optimized implementation of reorders to or from ``f8_e5m2`` is available on Intel(R) Data Center GPU Max Series Only. #. To alleviate the problem a user may rely on fact that the reorder from original plain memory format and user's data type to the optimized format with chosen data type should be always implemented. Data Types Support ------------------ The reorder primitive supports arbitrary data types for the source and destination according to the :ref:`Data Types ` page. When converting the data from one data type to a smaller one saturation is used. For instance: .. ref-code-block:: cpp reorder(src={1024, data_type=f32}, dst={, data_type=s8}) // dst == {127} reorder(src={-124, data_type=f32}, dst={, data_type=u8}) // dst == {0} Data Representation ------------------- The reorder primitive works with arbitrary data tensors. There is no special meaning associated with any logical dimensions. Post-Ops and Attributes ----------------------- The reorder primitive support the following attributes and post-ops: ============================================================================================ ============================================================= Attributes / Post-ops Meaning ============================================================================================ ============================================================= :ref:`Scales ` Scales the corresponding tensor by the given scale factor(s) :ref:`Zero points ` Sets zero point(s) for the corresponding tensors :ref:`Sum post-op ` Instead of copy the data accumulate it to the previous data ============================================================================================ ============================================================= For instance, the following pseudo-code .. ref-code-block:: cpp reorder( src = {dims={N, C, H, W}, data_type=dt_src, memory_format=fmt_src}, dst = {dims={N, C, H, W}, data_type=dt_dst, memory_format=fmt_dst}, attr ={ scales={ src={mask=0} }, zero_points= { src={mask=0}, dst={mask=0} }, post-ops = { sum={scale=beta} }, }) would lead to the following operation: .. math:: \dst(\overline{x}) = scale_{src} \cdot (\src(\overline{x}) - shift_{src}) + \beta \cdot \dst(\overline{x}) + shift_{dst} .. note:: * The intermediate operations are being done using single precision floating point data type. * :math:`scale_{src}`, :math:`shift_{src}`, :math:`scale_{dst}`, and :math:`shift_{dst}` must be passed during execution runtime as a separate memory arguments. Using :math:`scale_{src}` argument will lead to multiplication of tensor values by a scale value. Using :math:`scale_{dst}` argument will lead to division of tensor values by a scale value. Sparsity -------- Currently, there is only one reorder for packing a dense tensor, i.e. converting a dense tensor that is in ``ab`` format to a sparse tensor that is encoded with the ``PACKED`` encoding. In general, it is expected that all reorder-related functionality (e.g. scales, zero-points, etc) that is supported for the dense destination tensor should also work for the sparse one. Common Limitations * The interoperability API to get/set data handles is not supported. Use the runtime agnostic API to do that. * Sparse memory and memory descriptor can only be used with the Matrix Multiplication and Reorder primitives. Refer to :ref:`Sparsity Advanced Topic ` page for more information on sparse encding. Implementation Limitations ~~~~~~~~~~~~~~~~~~~~~~~~~~ #. Refer to :ref:`Data Types ` for limitations related to data types support. #. CPU * Reorders between ``bf16``, ``f16`` and ``s32`` data types are not supported. #. GPU * Only tensors of 6 or fewer dimensions are supported. * Runtime dimensions are not supported. Performance Tips ~~~~~~~~~~~~~~~~ N/A Examples ~~~~~~~~ See :ref:`Examples and Tutorials ` page for a complete list. Reorder examples are listed in the :ref:`Memory Transformations ` section.