NAME

MPSCNNLoss

SYNOPSIS

#import <MPSCNNLoss.h>

Inherits MPSCNNKernel.

Detailed Description

This depends on Metal.framework. The MPSCNNLoss filter is only used for training. This filter performs both the forward and backward pass computations. Specifically, it computes the loss between the input (predictions) and target data (labels) and the loss gradient. The loss value can be a 1 x 1 x 1 image containing a scalar loss value or an image (of the same size as the input source image) with per feature channel losses. The loss value is used to determine whether to continue the training operation or to terminate it, once satisfactory results are achieved. The loss gradient is the first gradient computed for the backward pass and serves as input to the next gradient filter (in the backward direction).

The MPSCNNLoss filter is created with a MPSCNNLossDescriptor describing the type of a loss filter and the type of a reduction to use for computing the overall loss.

The MPSCNNLoss filter takes the output of the inference pass (predictions) as input. It also requires the target data (labels) and optionally, weights for the labels. If per-label weights are not supplied, there is an option to use a single weight value by setting the 'weight' properly on the MPSCNNLossDescriptor object. The labels and optional weights need to be supplied by the user using the MPSCNNLossLabels object. The labels and weights are described via the MPSCNNLossDataDescriptor objects, which are in turn used to initialize the MPSCNNLossLabels object.

If the specified reduction operation is MPSCNNReductionTypeNone, the destinationImage should be at least as large as the specified clipRect. The detinationImage will then contain per-element losses. Otherse, a reduction operation will be performed, according to the specified reduction type, and the filter will return a scalar value containing the overall loss. For more information on the available reduction types, see MPSCNNTypes.h. Also see MPSCNNLossDescriptor for the description of optional parameters.

Here is a code example:

// Setup MPSCNNLossDataDescriptor* labelsDescriptor = [MPSCNNLossDataDescriptor cnnLossDataDescriptorWithData: labelsData layout: MPSDataLayoutHeightxWidthxFeatureChannels size: labelsDataSize]; MPSCNNLossLabels* labels = [[MPSCNNLossLabels alloc] initWithDevice: device labelsDescriptor: labelsDescriptor]; MPSCNNLossDescriptor lossDescriptor = [MPSCNNLossDescriptor cnnLossDescriptorWithType: (MPSCNNLossType)MPSCNNLossTypeMeanAbsoluteError reductionType: (MPSCNNReductionType)MPSCNNReductionTypeSum]; MPSCNNLoss lossFilter = [[MPSCNNLoss alloc] initWithDevice: device lossDescriptor: lossDescriptor];

// Encode loss filter. // The sourceImage is the output of a previous layer, for example, the SoftMax layer. The lossGradientsImage // is the sourceGradient input image to the first gradient layer (in the backward direction), for example, // the SoftMax gradient filter. [lossFilter encodeToCommandBuffer: commandBuffer sourceImage: sourceImage labels: labels destinationImage: lossGradientsImage];

// In order to gaurantee that the loss image data is correctly synchronized for CPU side access, // it is the application's responsibility to call the [labels synchronizeOnCommandBuffer:] // method before accessing the loss image data. [labels synchronizeOnCommandBuffer:commandBuffer]; MPSImage* lossImage = [labels lossImage];

        For predictions (y) and labels (t), the available loss filter types are the following:


        Mean Absolute Error loss filter. This filter measures the absolute error of the element-wise


        difference between the predictions and labels.


        This loss function is computed according to the following formulas:


            Compute losses:          losses = |y - t|


            Compute weighted losses: weighted_losses = weight(s) * losses


            Compute overall loss:    loss = reduce(weighted_losses, reductionType)


        Mean Squared Error loss filter. This filter measures the squared error of the element-wise


        difference between the predictions and labels.


        This loss function is computed according to the following formulas:


            Compute losses:          losses = (y - t)^2


            Compute weighted losses: weighted_losses = weight(s) * losses


            Compute overall loss:    loss = reduce(weighted_losses, reductionType)


        SoftMax Cross Entropy loss filter. This loss filter is applied element-wise.


        This loss filter combines the LogSoftMax and Negative Log Likelihood operations in a


        single filter. It is useful for training a classification problem with multiple classes.


        This loss function is computed according to the following formulas:


            Compute losses:          losses = -t * LogSoftMax(y)


            Compute weighted losses: weighted_losses = weight(s) * losses


            Compute overall loss:    loss = reduce(weighted_losses, reductionType)


                                     If reductionType is MPSCNNReductionTypeMean, the accumulated


                                     loss value is divided by width * height instead of


                                     width * height * featureChannels.


        Sigmoid Cross Entropy loss filter. This loss filter is applied element-wise.


        This loss function is computed according to the following formulas:


            Compute losses:          losses = max(y, 0) - y * t + log(1 + exp(-|y|))


            Compute weighted losses: weighted_losses = weight(s) * losses


            Compute overall loss:    loss = reduce(weighted_losses, reductionType)


        Categorical Cross Entropy loss filter. This loss filter is applied element-wise.


        This loss function is computed according to the following formulas:


            Compute losses:          losses = -t * log(y)


            Compute weighted losses: weighted_losses = weight(s) * losses


            Compute overall loss:    loss = reduce(weighted_losses, reductionType)


        Hinge loss filter. This loss filter is applied element-wise.


        The labels are expected to be 0.0 or 1.0.


        This loss function is computed according to the following formulas:


            Compute losses:          losses = max(1 - (t * y), 0.0f)


            Compute weighted losses: weighted_losses = weight(s) * losses


            Compute overall loss:    loss = reduce(weighted_losses, reductionType)


        Huber loss filter. This loss filter is applied element-wise.


        This loss function is computed according to the following formulas:


            Compute losses:          if (|y - t| <= delta, losses = 0.5 * y^2


                                     if (|y - t| >  delta, losses = 0.5 * delta^2 + delta * (|y - t| - delta)


            Compute weighted losses: weighted_losses = weight(s) * losses


            Compute overall loss:    loss = reduce(weighted_losses, reductionType)


        Cosine Distance loss filter. This loss filter is applied element-wise.


        The only valid reduction type for this loss filter is MPSCNNReductionTypeSum.


        This loss function is computed according to the following formulas:


            Compute losses:          loss = 1 - reduce_sum(y * t)


            Compute overall loss:    weighted_loss = weight * loss


        Log loss filter. This loss filter is applied element-wise.


        This loss function is computed according to the following formulas:


            Compute losses:          losses = -(t * log(y + epsilon)) - ((1 - t) * log(1 - y + epsilon))


            Compute weighted losses: weighted_losses = weight(s) * losses


            Compute overall loss:    loss = reduce(weighted_losses, reductionType)


        Kullback-Leibler Divergence loss filter. This loss filter is applied element-wise.


        The input (predictions) is expected to contain log-probabilities.


            This loss function is computed according to the following formulas:


            Compute losses:          losses = t * (log(t) - y)


            Compute weighted losses: weighted_losses = weight(s) * losses


            Compute overall loss:    loss = reduce(weighted_losses, reductionType)


        For predictions (y) and labels (t), the loss gradient for each available loss filter type


        is computed as follows:


        Mean Absolute Error loss.


        The loss gradient is computed according to the following formulas:


            Compute gradient:          d/dy = (y - t) / |y - t|


            Compute weighted gradient: weighted_gradient = weight(s) * gradient


        Mean Squared Error loss.


        The loss gradient is computed according to the following formulas:


            Compute gradient:          d/dy = 2 * (y - t)


            Compute weighted gradient: weighted_gradient = weight(s) * gradient


        SoftMax Cross Entropy loss.


        The loss gradient is computed according to the following formulas:


            First, apply the same label smoothing as in the MPSCNNLoss filter.


            Compute gradient:          d/dy = y - t


            Compute weighted gradient: weighted_gradient = weight(s) * gradient


        Sigmoid Cross Entropy loss.


        The loss gradient is computed according to the following formulas:


        First, apply the same label smoothing as in the MPSCNNLoss filter.


            Compute gradient:          d/dy = (1 / (1 + exp(-y)) - t


            Compute weighted gradient: weighted_gradient = weight(s) * gradient


        Categorical Cross Entropy loss.


        The loss gradient is computed according to the following formulas:


            Compute gradient:          d/dy = -t / y


            Compute weighted gradient: weighted_gradient = weight(s) * gradient


        Hinge loss.


        The loss gradient is computed according to the following formulas:


            Compute gradient:          d/dy = ((1 + ((1 - (2 * t)) * y)) > 0) ? 1 - (2 * t) : 0


            Compute weighted gradient: weighted_gradient = weight(s) * gradient


        Huber loss.


        The loss gradient is computed according to the following formulas:


            Compute gradient:          d/dy = |y - t| > delta ? delta : y - t


            Compute weighted gradient: weighted_gradient = weight(s) * gradient


        Cosine Distance loss.


        The loss gradient is computed according to the following formulas:


            Compute gradient:          d/dy = -t


            Compute weighted gradient: weighted_gradient = weight(s) * gradient


        Log loss.


        The loss gradient is computed according to the following formulas:


            Compute gradient:          d/dy = (-2 * epsilon * t - t + y + epsilon) / (y * (1 - y) + epsilon * (epsilon + 1))


            Compute weighted gradient: weighted_gradient = weight(s) * gradient


        Kullback-Leibler Divergence loss.


        The loss gradient is computed according to the following formulas:


            Compute gradient:          d/dy = -t / y


            Compute weighted gradient: weighted_gradient = weight(s) * gradient


        The number of output feature channels remains the same as the number of input feature


        channels..fi

Method Documentation

Property Documentation

Author

Generated automatically by Doxygen for MetalPerformanceShaders.framework from the source code.