.. index:: pair: group; Layer Normalization .. _doxid-group__dnnl__api__layer__normalization: Layer Normalization =================== .. toctree:: :hidden: Overview ~~~~~~~~ .. ref-code-block:: cpp :class: doxyrest-overview-code-block // global functions :ref:`dnnl_status_t` DNNL_API :ref:`dnnl_layer_normalization_forward_primitive_desc_create`( :ref:`dnnl_primitive_desc_t`* primitive_desc, :ref:`dnnl_engine_t` engine, :ref:`dnnl_prop_kind_t` prop_kind, :ref:`const_dnnl_memory_desc_t` src_desc, :ref:`const_dnnl_memory_desc_t` dst_desc, :ref:`const_dnnl_memory_desc_t` stat_desc, float epsilon, unsigned flags, :ref:`const_dnnl_primitive_attr_t` attr ); :ref:`dnnl_status_t` DNNL_API :ref:`dnnl_layer_normalization_backward_primitive_desc_create`( :ref:`dnnl_primitive_desc_t`* primitive_desc, :ref:`dnnl_engine_t` engine, :ref:`dnnl_prop_kind_t` prop_kind, :ref:`const_dnnl_memory_desc_t` diff_src_desc, :ref:`const_dnnl_memory_desc_t` diff_dst_desc, :ref:`const_dnnl_memory_desc_t` src_desc, :ref:`const_dnnl_memory_desc_t` stat_desc, float epsilon, unsigned flags, :ref:`const_dnnl_primitive_desc_t` hint_fwd_pd, :ref:`const_dnnl_primitive_attr_t` attr ); :ref:`dnnl_status_t` DNNL_API :ref:`dnnl_layer_normalization_forward_primitive_desc_create_v2`( :ref:`dnnl_primitive_desc_t`* primitive_desc, :ref:`dnnl_engine_t` engine, :ref:`dnnl_prop_kind_t` prop_kind, :ref:`const_dnnl_memory_desc_t` src_desc, :ref:`const_dnnl_memory_desc_t` dst_desc, :ref:`const_dnnl_memory_desc_t` stat_desc, :ref:`dnnl_data_type_t` scale_shift_data_type, float epsilon, unsigned flags, :ref:`const_dnnl_primitive_attr_t` attr ); :ref:`dnnl_status_t` DNNL_API :ref:`dnnl_layer_normalization_backward_primitive_desc_create_v2`( :ref:`dnnl_primitive_desc_t`* primitive_desc, :ref:`dnnl_engine_t` engine, :ref:`dnnl_prop_kind_t` prop_kind, :ref:`const_dnnl_memory_desc_t` diff_src_desc, :ref:`const_dnnl_memory_desc_t` diff_dst_desc, :ref:`const_dnnl_memory_desc_t` src_desc, :ref:`const_dnnl_memory_desc_t` stat_desc, :ref:`dnnl_data_type_t` diff_scale_shift_data_type, :ref:`dnnl_data_type_t` scale_shift_data_type, float epsilon, unsigned flags, :ref:`const_dnnl_primitive_desc_t` hint_fwd_pd, :ref:`const_dnnl_primitive_attr_t` attr ); .. _details-group__dnnl__api__layer__normalization: Detailed Documentation ~~~~~~~~~~~~~~~~~~~~~~ Global Functions ---------------- .. index:: pair: function; dnnl_layer_normalization_forward_primitive_desc_create .. _doxid-group__dnnl__api__layer__normalization_1ga3f99050f79a43b697bf35cc0f39e21fe: .. ref-code-block:: cpp :class: doxyrest-title-code-block :ref:`dnnl_status_t` DNNL_API dnnl_layer_normalization_forward_primitive_desc_create( :ref:`dnnl_primitive_desc_t`* primitive_desc, :ref:`dnnl_engine_t` engine, :ref:`dnnl_prop_kind_t` prop_kind, :ref:`const_dnnl_memory_desc_t` src_desc, :ref:`const_dnnl_memory_desc_t` dst_desc, :ref:`const_dnnl_memory_desc_t` stat_desc, float epsilon, unsigned flags, :ref:`const_dnnl_primitive_attr_t` attr ) Creates a primitive descriptor for a layer normalization forward propagation primitive. .. note:: In-place operation is supported: the dst can refer to the same memory as the src. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - primitive_desc - Output primitive_descriptor. * - engine - Engine to use. * - prop_kind - Propagation kind. Possible values are :ref:`dnnl_forward_training ` and :ref:`dnnl_forward_inference `. * - src_desc - Source memory descriptor. * - dst_desc - Destination memory descriptor. * - stat_desc - Memory descriptor for mean and variance. If this parameter is NULL, a zero memory descriptor, or a memory descriptor with format_kind set to :ref:`dnnl_format_kind_undef `, then the memory descriptor for stats is derived from ``src_desc`` by removing the last dimension. * - epsilon - Layer normalization epsilon parameter. * - flags - Layer normalization flags (:ref:`dnnl_normalization_flags_t `). * - attr - Primitive attributes (can be NULL). .. rubric:: Returns: :ref:`dnnl_success ` on success and a status describing the error otherwise. .. index:: pair: function; dnnl_layer_normalization_backward_primitive_desc_create .. _doxid-group__dnnl__api__layer__normalization_1gaea99a932365f2e330420fdefabcfcd05: .. ref-code-block:: cpp :class: doxyrest-title-code-block :ref:`dnnl_status_t` DNNL_API dnnl_layer_normalization_backward_primitive_desc_create( :ref:`dnnl_primitive_desc_t`* primitive_desc, :ref:`dnnl_engine_t` engine, :ref:`dnnl_prop_kind_t` prop_kind, :ref:`const_dnnl_memory_desc_t` diff_src_desc, :ref:`const_dnnl_memory_desc_t` diff_dst_desc, :ref:`const_dnnl_memory_desc_t` src_desc, :ref:`const_dnnl_memory_desc_t` stat_desc, float epsilon, unsigned flags, :ref:`const_dnnl_primitive_desc_t` hint_fwd_pd, :ref:`const_dnnl_primitive_attr_t` attr ) Creates a primitive descriptor for a layer normalization backward propagation primitive. .. note:: In-place operation is supported: the diff_dst can refer to the same memory as the diff_src. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - primitive_desc - Output primitive_descriptor. * - engine - Engine to use. * - prop_kind - Propagation kind. Possible values are :ref:`dnnl_backward_data ` and :ref:`dnnl_backward ` (diffs for all parameters are computed in this case). * - diff_src_desc - Diff source memory descriptor. * - diff_dst_desc - Diff destination memory descriptor. * - src_desc - Source memory descriptor. * - stat_desc - Memory descriptor for mean and variance. If this parameter is NULL, a zero memory descriptor, or a memory descriptor with format_kind set to :ref:`dnnl_format_kind_undef `, then the memory descriptor for stats is derived from ``src_desc`` by removing the last dimension. * - epsilon - Layer normalization epsilon parameter. * - flags - Layer normalization flags (:ref:`dnnl_normalization_flags_t `). * - hint_fwd_pd - Primitive descriptor for a respective forward propagation primitive. * - attr - Primitive attributes (can be NULL). .. rubric:: Returns: :ref:`dnnl_success ` on success and a status describing the error otherwise. .. index:: pair: function; dnnl_layer_normalization_forward_primitive_desc_create_v2 .. _doxid-group__dnnl__api__layer__normalization_1ga9efbb60f29ecac498a78a828521cb0fd: .. ref-code-block:: cpp :class: doxyrest-title-code-block :ref:`dnnl_status_t` DNNL_API dnnl_layer_normalization_forward_primitive_desc_create_v2( :ref:`dnnl_primitive_desc_t`* primitive_desc, :ref:`dnnl_engine_t` engine, :ref:`dnnl_prop_kind_t` prop_kind, :ref:`const_dnnl_memory_desc_t` src_desc, :ref:`const_dnnl_memory_desc_t` dst_desc, :ref:`const_dnnl_memory_desc_t` stat_desc, :ref:`dnnl_data_type_t` scale_shift_data_type, float epsilon, unsigned flags, :ref:`const_dnnl_primitive_attr_t` attr ) Creates a primitive descriptor for a layer normalization forward propagation primitive with a user-provided data type for the scale and shift memory objects. .. note:: In-place operation is supported: the dst can refer to the same memory as the src. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - primitive_desc - Output primitive_descriptor. * - engine - Engine to use. * - prop_kind - Propagation kind. Possible values are :ref:`dnnl_forward_training ` and :ref:`dnnl_forward_inference `. * - src_desc - Source memory descriptor. * - dst_desc - Destination memory descriptor. * - stat_desc - Memory descriptor for mean and variance. If this parameter is NULL, a zero memory descriptor, or a memory descriptor with format_kind set to :ref:`dnnl_format_kind_undef `, then the memory descriptor for stats is derived from ``src_desc`` by removing the last dimension. * - scale_shift_data_type - Data type of scale and shift memory. If neither scale nor shift flag are specified the parameter is ignored. * - epsilon - Layer normalization epsilon parameter. * - flags - Layer normalization flags (:ref:`dnnl_normalization_flags_t `). * - attr - Primitive attributes (can be NULL). .. rubric:: Returns: :ref:`dnnl_success ` on success and a status describing the error otherwise. .. index:: pair: function; dnnl_layer_normalization_backward_primitive_desc_create_v2 .. _doxid-group__dnnl__api__layer__normalization_1ga4f398ee5311dcdb475ac18f379958b6a: .. ref-code-block:: cpp :class: doxyrest-title-code-block :ref:`dnnl_status_t` DNNL_API dnnl_layer_normalization_backward_primitive_desc_create_v2( :ref:`dnnl_primitive_desc_t`* primitive_desc, :ref:`dnnl_engine_t` engine, :ref:`dnnl_prop_kind_t` prop_kind, :ref:`const_dnnl_memory_desc_t` diff_src_desc, :ref:`const_dnnl_memory_desc_t` diff_dst_desc, :ref:`const_dnnl_memory_desc_t` src_desc, :ref:`const_dnnl_memory_desc_t` stat_desc, :ref:`dnnl_data_type_t` diff_scale_shift_data_type, :ref:`dnnl_data_type_t` scale_shift_data_type, float epsilon, unsigned flags, :ref:`const_dnnl_primitive_desc_t` hint_fwd_pd, :ref:`const_dnnl_primitive_attr_t` attr ) Creates a primitive descriptor for a layer normalization backward propagation primitive with a user-provided data type for the scale and shift memory objects. .. note:: In-place operation is supported: the diff_dst can refer to the same memory as the diff_src. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - primitive_desc - Output primitive_descriptor. * - engine - Engine to use. * - prop_kind - Propagation kind. Possible values are :ref:`dnnl_backward_data ` and :ref:`dnnl_backward ` (diffs for all parameters are computed in this case). * - diff_src_desc - Diff source memory descriptor. * - diff_dst_desc - Diff destination memory descriptor. * - src_desc - Source memory descriptor. * - stat_desc - Memory descriptor for mean and variance. If this parameter is NULL, a zero memory descriptor, or a memory descriptor with format_kind set to :ref:`dnnl_format_kind_undef `, then the memory descriptor for stats is derived from ``src_desc`` by removing the last dimension. * - diff_scale_shift_data_type - Data type of diff scale and shift memory. If neither scale nor shift flag are specified the parameter is ignored. * - scale_shift_data_type - Data type of scale and shift memory. If neither scale nor shift flag are specified the parameter is ignored. * - epsilon - Layer normalization epsilon parameter. * - flags - Layer normalization flags (:ref:`dnnl_normalization_flags_t `). * - hint_fwd_pd - Primitive descriptor for a respective forward propagation primitive. * - attr - Primitive attributes (can be NULL). .. rubric:: Returns: :ref:`dnnl_success ` on success and a status describing the error otherwise.