NAME

MPSGRUDescriptor

SYNOPSIS

#import <MPSRNNLayer.h>

Inherits MPSRNNDescriptor.

Detailed Description

This depends on Metal.framework The MPSGRUDescriptor specifies a GRU (Gated Recurrent Unit) block/layer descriptor. The RNN layer initialized with a MPSGRUDescriptor transforms the input data (image or matrix), and previous output with a set of filters, each producing one feature map in the output data according to the Gated unit formulae detailed below. The user may provide the GRU unit a single input or a sequence of inputs. The layer also supports p-norm gating (Detailed in: https://arxiv.org/abs/1608.03639 ).

Description of operation:

Let x_j be the input data (at time index t of sequence, j index containing quadruplet: batch index, x,y and feature index (x=y=0 for matrices)). Let h0_j be the recurrent input (previous output) data from previous time step (at time index t-1 of sequence). Let h_i be the proposed new output. Let h1_i be the output data produced at this time step.

Let Wz_ij, Uz_ij, be the input gate weights for input and recurrent input data respectively Let bi_i be the bias for the input gate

Let Wr_ij, Ur_ij be the recurrent gate weights for input and recurrent input data respectively Let br_i be the bias for the recurrent gate

Let Wh_ij, Uh_ij, Vh_ij, be the output gate weights for input, recurrent gate and input gate respectively Let bh_i be the bias for the output gate

Let gz(x), gr(x), gh(x) be the neuron activation function for the input, recurrent and output gates Let p > 0 be a scalar variable (typicall p >= 1.0) that defines the p-norm gating norm value.

Then the output of the Gated Recurrent Unit layer is computed as follows:

    z_i = gz(  Wz_ij * x_j  +  Uz_ij * h0_j  +  bz_i  )


    r_i = gr(  Wr_ij * x_j  +  Ur_ij * h0_j  +  br_i  )


    c_i =      Uh_ij * (r_j h0_j)  +  Vh_ij * (z_j h0_j)


    h_i = gh(  Wh_ij * x_j  + c_i + bh_i  )
h1_i = ( 1 - z_i ^ p)^(1/p) h_i + z_i h0_i

The '*' stands for convolution (see MPSRNNImageInferenceLayer) or matrix-vector/matrix multiplication (see MPSRNNMatrixInferenceLayer). Summation is over index j (except for the batch index), but there is no summation over repeated index i - the output index. Note that for validity all intermediate images have to be of same size and all U and V matrices have to be square (ie. outputFeatureChannels == inputFeatureChannels in those). Also the bias terms are scalars wrt. spatial dimensions. The conventional GRU block is achieved by setting Vh = 0 (nil) and the so-called Minimal Gated Unit is achieved with Uh = 0. (The Minimal Gated Unit is detailed in: https://arxiv.org/abs/1603.09420 and there they call z_i the value of the forget gate).

Method Documentation

Property Documentation

Author

Generated automatically by Doxygen for MetalPerformanceShaders.framework from the source code.