.rst .pdf

Objective Function

In oneDAL, the objective function represents an interface of objective functions $K(\theta )=F(\theta )+M(\theta )$, where $F(\theta )$ is a smooth and $M(\theta )$ is a non-smooth functions, that accepts input argument $\theta \in R^p$ and returns:

• The value of objective function, $y=K(\theta )$

• The value of $M(\theta )$, $y_{ns}=M(\theta )$

• The gradient of $F(\theta )$:

  $$g(\theta )=\mathrm{\nabla }F(\theta )=\{\frac{\partial F}{\partial {\theta }_1},\dots ,\frac{\partial F}{\partial {\theta }_p}\}$$

• The Hessian of $F(\theta )$:

  $$\begin{array}{r}H=={\mathrm{\nabla }}^{2}F(\theta )={\mathrm{\nabla }}^2{F}_i=\left[\begin{array}{ccc}\frac{\partial F_i}{\partial {\theta }_1\partial {\theta }_1}& \cdots & \frac{\partial F_i}{\partial {\theta }_1\partial {\theta }_p}\\ ⋮& \ddots & ⋮\\ \frac{\partial F_i}{\partial p\partial {\theta }_1}& \cdots & \frac{\partial F_i}{\partial {\theta }_p\partial {\theta }_p}\end{array}\right]\end{array}$$

• The objective function specific projection of proximal operator (see [MSE, Log-Loss, Cross-Entropy] for details):

  $$\begin{array}{r}\begin{array}{c}{\text{prox}}_{\eta }^M(x)={\text{argmin}}_{u\in R^p}(M(u)+\frac{1}{2\eta }|u-x{|}_2^2)\\ x\in R^p\end{array}\end{array}$$

• The objective function specific Lipschitz constant, $\text{constantOfLipschitz}\le |\mathrm{\nabla }|F(\theta )$.

Objective functions

• Computation
• Sum of Functions
• Mean Squared Error Algorithm
• Objective Function with Precomputed Characteristics Algorithm
• Logistic Loss
• Cross-entropy Loss

Note

On GPU, only Logistic Loss and Cross-entropy Loss are supported, Mean Squared Error Algorithm is not supported.

previous

Optimization Solvers

next

Computation