.. index:: pair: page; MatMul .. _doxid-dev_guide_op_matmul: MatMul ====== General ~~~~~~~ MatMul operation computes the product of two tensors with optional bias addition The variable names follow the standard :ref:`Naming Conventions `, typically taking 2D input tensors as an example, the formula is below: .. math:: \dst(m, n) = \sum_{k=0}^{K - 1} \left( \src(m, k) \cdot \weights(k, n) \right) + \bias(m, n) In the shape of a tensor, two right-most axes are interpreted as row and column dimensions of a matrix while all left-most axes (if present) are interpreted as batch dimensions. The operation supports broadcasting semantics for those batch dimensions. For example :math:`\src` can be broadcasted to :math:`\weights` if the corresponding dimension in :math:`\src` is ``1`` (and vice versa). Additionally, if ranks of :math:`\src` and :math:`\weights` are different, the tensor with a smaller rank will be unsqueezed from the left side of dimensions (inserting ``1``) to make sure two ranks matched. Operation attributes ~~~~~~~~~~~~~~~~~~~~ ============================================================================================================================== ======================================================================= =========== ============================================================================ ===================== Attribute Name Description Value Type Supported Values Required or Optional ============================================================================================================================== ======================================================================= =========== ============================================================================ ===================== :ref:`transpose_a ` Controls whether to transpose the last two dimensions of ``src`` . bool True, False (default) Optional :ref:`transpose_b ` Controls whether to transpose the last two dimensions of ``weights`` . bool True, False (default) Optional :ref:`accumulation_mode ` Specifies the accumulation mode to be used for MatMul calculation string ``strict`` (default), ``relaxed`` , ``any`` , ``f32`` , ``s32`` , ``f16`` . Optional ============================================================================================================================== ======================================================================= =========== ============================================================================ ===================== The above transpose attribute will not in effect when rank of an input tensor is less than 2. For example, in library implementation 1D tensor is unsqueezed firstly before compilation. The rule is applied independently. * For :math:`\src` tensor, the rule is defined like: ``[d] -> [1, d]``. * For :math:`\weights` tensor, the rule is defined like: ``[d] -> [d, 1]``. For the attribute ``accumulation_mode``, refer to `accumulation mode `__ for the definition for each supported value. Execution arguments ~~~~~~~~~~~~~~~~~~~ The inputs and outputs must be provided according to below index order when constructing an operation. Inputs ------ ====== ============== ===================== Index Argument Name Required or Optional ====== ============== ===================== 0 ``src`` Required 1 ``weights`` Required 2 ``bias`` Optional ====== ============== ===================== Outputs ------- ====== ============== ===================== Index Argument Name Required or Optional ====== ============== ===================== 0 ``dst`` Required ====== ============== ===================== Supported data types ~~~~~~~~~~~~~~~~~~~~ MatMul operation supports the following data type combinations. ===== ======== ===== ========== Src Weights Bias Dst ===== ======== ===== ========== f32 f32 f32 f32 bf16 bf16 bf16 f32, bf16 f16 f16 f16 f32, f16 ===== ======== ===== ==========