MPSKernel(3) MetalPerformanceShaders.framework MPSKernel(3)

MPSKernel

#import <MPSKernel.h>

Inherits NSObject, <NSCopying>, and <NSSecureCoding>.

Inherited by MPSAccelerationStructure, MPSBinaryImageKernel, MPSCNNBinaryKernel, MPSCNNKernel, MPSImageCopyToMatrix, MPSImageFindKeypoints, MPSImageGuidedFilter, MPSImageHistogram, MPSImageNormalizedHistogram, MPSMatrixBinaryKernel, MPSMatrixCopy, MPSMatrixCopyToImage, MPSMatrixMultiplication, MPSMatrixSum, MPSMatrixUnaryKernel, MPSNNGraph, MPSNNOptimizer, MPSRayIntersector, MPSRNNMatrixInferenceLayer, MPSRNNMatrixTrainingLayer, and MPSUnaryImageKernel.


(nonnull instancetype) - initWithDevice:
(nonnull instancetype) - copyWithZone:device:
(nullable instancetype) - initWithCoder:
(nullable instancetype) - initWithCoder:device:
() - MPSCopyAllocator


const MTLRegion MPSRectNoClip


MPSKernelOptions options
id< MTLDevice > device
NSString * label

MPSKernel.h MPSCore.framework

Copyright:

Copyright (c) 2015-2017 Apple Inc. All rights reserved.

MPSKernel objects encode tuned image processing operations into a MTLCommandBuffer.

This depends on Metal.framework The MPSKernel class is the base class for all MPS objects. It defines a standard interface for MPS kernels. You should not use the MPSKernel class directly. Instead, a number of MPSKernel subclasses are available in MetalPerformanceShaders.framework that define specific high-performance image processing operations.

The basic sequence for applying a MPSKernel to an image is as follows:

1.
Create a MPSKernel corresponding to the operation you wish to perform:

MPSImageSobel *sobel = [[MPSImageSobel alloc] initWithDevice: mtlDevice];
2.
Encode the filter into a command buffer:

sobel.offset = ...;
sobel.clipRect = ...;
sobel.options = ...;
[sobel encodeToCommandBuffer: commandBuffer

sourceTexture: inputImage
destinationTexture: resultImage ];


Encoding the kernel merely encodes the operation into a MTLCommandBuffer. It does not modify any pixels, yet. All MPSKernel state has been copied to the command buffer. MPSKernels may be reused. If the texture was previously operated on by another command encoder (e.g. MTLRenderCommandEncoder), you should call -endEncoding on the other encoder before encoding the filter.

Some MPS filters work in place (inputImage = resultImage) even in situations where Metal might not normally allow in place operation on textures. If in-place operation is desired, you may attempt to call [MPSKernel encodeKernelInPlace...]. If the operation can not be completed in place, then NO will be returned and you will have to create a new result texture and try again. To make an in-place image filter reliable, pass a fallback MPSCopyAllocator to the method to create a new texture to write to in the event that a filter can not operate in place.

(Repeat steps 2 for more filters, as desired.)

It should be self evident that step 2 may not be thread safe. That is, you can not have
multiple threads manipulating the same properties on the same MPSKernel object at the
same time and achieve coherent output. In common usage, the MPSKernel properties don't
often need to be changed from their default values, but if you need to apply the same
filter to multiple images on multiple threads with cropping / tiling, make additional
MPSKernel objects per thread. They are cheap. You can use multiple MPSKernel objects on
multiple threads, as long as only one thread is operating on any particular MPSKernel
object at a time.
3.
After encoding any additional work to the command buffer using other encoders, submit the MTLCommandBuffer to your MTLCommandQueue, using:

[mtlCommandBuffer commit];

A MPSKernel can be saved to disk / network using NSCoders such as NSKeyedArchiver. When decoding, the system default MTLDevice will be chosen unless the NSCoder adopts the <MPSDeviceProvider> protocol. To accomplish this, subclass or extend your unarchiver to add this method.

- (nonnull instancetype) copyWithZone: (nullable NSZone *) zone(nullable id< MTLDevice >) device

Make a copy of this MPSKernel for a new device -copyWithZone: will call this API to make a copy of the MPSKernel on the same device. This interface may also be called directly to make a copy of the MPSKernel on a new device. Typically, the same MPSKernels should not be used to encode kernels on multiple command buffers from multiple threads. Many MPSKernels have mutable properties that might be changed by the other thread while this one is trying to encode. If you need to use a MPSKernel from multiple threads make a copy of it for each additional thread using -copyWithZone: or -copyWithZone:device:

Parameters:

zone The NSZone in which to allocate the object
device The device for the new MPSKernel. If nil, then use self.device.

Returns:

a pointer to a copy of this MPSKernel. This will fail, returning nil if the device is not supported. Devices must be MTLFeatureSet_iOS_GPUFamily2_v1 or later.

Reimplemented in MPSRNNMatrixTrainingLayer, MPSRNNMatrixInferenceLayer, MPSRayIntersector, MPSRNNImageInferenceLayer, MPSAccelerationStructure, MPSMatrixFullyConnectedGradient, MPSMatrixNeuronGradient, MPSMatrixBatchNormalizationGradient, MPSMatrixSoftMaxGradient, MPSMatrixFindTopK, MPSMatrixFullyConnected, MPSMatrixBatchNormalization, MPSMatrixNeuron, and MPSMatrixSoftMax.

- (nullable instancetype) initWithCoder: (NSCoder *__nonnull) aDecoder

Called by NSCoder to decode MPSKernels This isn't the right interface to decode a MPSKernel, but it is the one that NSCoder uses. To enable your NSCoder (e.g. NSKeyedUnarchiver) to set which device to use extend the object to adopt the MPSDeviceProvider protocol. Otherwise, the Metal system default device will be used.

- (nullable instancetype) initWithCoder: (NSCoder *__nonnull) aDecoder(nonnull id< MTLDevice >) device

NSSecureCoding compatability While the standard NSSecureCoding/NSCoding method -initWithCoder: should work, since the file can't know which device your data is allocated on, we have to guess and may guess incorrectly. To avoid that problem, use initWithCoder:device instead.

Parameters:

aDecoder The NSCoder subclass with your serialized MPSKernel
device The MTLDevice on which to make the MPSKernel

Returns:

A new MPSKernel object, or nil if failure.

Reimplemented in MPSCNNBinaryConvolution, MPSCNNBinaryFullyConnected, MPSCNNConvolutionTranspose, MPSRNNMatrixTrainingLayer, MPSCNNConvolutionGradient, MPSCNNFullyConnected, MPSCNNFullyConnectedGradient, MPSCNNBinaryKernel, MPSCNNGradientKernel, MPSRNNMatrixInferenceLayer, MPSCNNConvolution, MPSCNNYOLOLoss, MPSRayIntersector, MPSRNNImageInferenceLayer, MPSCNNPoolingAverageGradient, MPSCNNPoolingMaxGradient, MPSCNNPoolingL2NormGradient, MPSCNNDilatedPoolingMaxGradient, MPSCNNSoftMaxGradient, MPSCNNLogSoftMaxGradient, MPSCNNCrossChannelNormalizationGradient, MPSCNNLoss, MPSCNNPoolingGradient, MPSImagePyramid, MPSCNNCrossChannelNormalization, MPSBinaryImageKernel, MPSImageHistogramSpecification, MPSCNNDilatedPoolingMax, MPSCNNLocalContrastNormalizationGradient, MPSAccelerationStructure, MPSCNNBatchNormalization, MPSCNNBatchNormalizationStatistics, MPSCNNBatchNormalizationGradient, MPSCNNBatchNormalizationStatisticsGradient, MPSImageHistogramEqualization, MPSImageSobel, MPSCNNNeuronGradient, MPSCNNPoolingAverage, MPSCNNPoolingL2Norm, MPSMatrixFullyConnectedGradient, MPSCNNKernel, MPSImageThresholdToZeroInverse, MPSMatrixNeuronGradient, MPSMatrixBatchNormalizationGradient, MPSCNNLocalContrastNormalization, MPSMatrixSoftMaxGradient, MPSCNNInstanceNormalization, MPSCNNNeuron, MPSImageThresholdToZero, MPSImageNormalizedHistogram, MPSCNNDropoutGradient, MPSUnaryImageKernel, MPSMatrixCopyToImage, MPSImageEuclideanDistanceTransform, MPSImageBox, MPSImageGaussianBlur, MPSMatrixCopy, MPSImageStatisticsMean, MPSImageThresholdBinary, MPSCNNSpatialNormalizationGradient, MPSImageThresholdTruncate, MPSNNCropAndResizeBilinear, MPSMatrixSum, MPSMatrixFindTopK, MPSImageDilate, MPSCNNDropout, MPSMatrixFullyConnected, MPSImageScale, MPSImageLanczosScale, MPSImageBilinearScale, MPSImageStatisticsMeanAndVariance, MPSMatrixBatchNormalization, MPSMatrixNeuron, MPSImageConvolution, MPSMatrixSoftMax, MPSImageThresholdBinaryInverse, MPSImageHistogram, MPSImageGuidedFilter, MPSImageCopyToMatrix, MPSCNNSpatialNormalization, MPSImageFindKeypoints, MPSNNResizeBilinear, MPSImageStatisticsMinAndMax, MPSCNNPooling, MPSCNNPoolingMax, MPSImageMedian, MPSImageAreaMax, and MPSNNGraph.

- (nonnull instancetype) initWithDevice: (nonnull id< MTLDevice >) device

Standard init with default properties per filter type

Parameters:

device The device that the filter will be used on. May not be NULL.

Returns:

a pointer to the newly initialized object. This will fail, returning nil if the device is not supported. Devices must be MTLFeatureSet_iOS_GPUFamily2_v1 or later.

Reimplemented in MPSCNNBinaryConvolution, MPSCNNBinaryFullyConnected, MPSCNNConvolutionTranspose, MPSRNNMatrixTrainingLayer, MPSCNNConvolutionGradient, MPSCNNFullyConnected, MPSCNNFullyConnectedGradient, MPSCNNGradientKernel, MPSRNNMatrixInferenceLayer, MPSCNNConvolution, MPSNNOptimizerAdam, MPSCNNYOLOLoss, MPSRayIntersector, MPSRNNImageInferenceLayer, MPSCNNSoftMaxGradient, MPSCNNLogSoftMaxGradient, MPSNNOptimizerRMSProp, MPSCNNLoss, MPSCNNPoolingGradient, MPSCNNCrossChannelNormalization, MPSBinaryImageKernel, MPSNNReduceFeatureChannelsAndWeightsSum, MPSNNReshape, MPSCNNArithmeticGradient, MPSNNReduceBinary, MPSNNReduceFeatureChannelsAndWeightsMean, MPSImagePyramid, MPSAccelerationStructure, MPSCNNBatchNormalization, MPSCNNBatchNormalizationStatistics, MPSNNOptimizerStochasticGradientDescent, MPSNNReduceFeatureChannelsSum, MPSImageSobel, MPSCNNNeuronGradient, MPSCNNNeuronLinear, MPSCNNNeuronReLU, MPSCNNNeuronPReLU, MPSCNNNeuronSigmoid, MPSCNNNeuronHardSigmoid, MPSCNNNeuronTanH, MPSCNNNeuronAbsolute, MPSCNNNeuronSoftPlus, MPSCNNNeuronSoftSign, MPSCNNNeuronELU, MPSCNNNeuronReLUN, MPSCNNNeuronPower, MPSCNNNeuronExponential, MPSCNNNeuronLogarithm, MPSMatrixFullyConnectedGradient, MPSCNNBinaryKernel, MPSNNOptimizer, MPSImageThresholdToZeroInverse, MPSMatrixNeuronGradient, MPSMatrixSum, MPSMatrixBatchNormalizationGradient, MPSCNNLocalContrastNormalization, MPSMatrixSoftMaxGradient, MPSCNNInstanceNormalization, MPSCNNKernel, MPSCNNNeuron, MPSCNNUpsamplingGradient, MPSImageThresholdToZero, MPSCNNDropoutGradient, MPSUnaryImageKernel, MPSImageEuclideanDistanceTransform, MPSImageBox, MPSImageGaussianBlur, MPSImageStatisticsMean, MPSImageThresholdBinary, MPSImageThresholdTruncate, MPSCNNArithmetic, MPSCNNAdd, MPSCNNSubtract, MPSCNNMultiply, MPSCNNDivide, MPSNNCropAndResizeBilinear, MPSNNSlice, MPSMatrixFindTopK, MPSImageDilate, MPSCNNDropout, MPSMatrixFullyConnected, MPSImageLanczosScale, MPSImageBilinearScale, MPSImageArithmetic, MPSImageAdd, MPSImageSubtract, MPSImageMultiply, MPSImageDivide, MPSImageStatisticsMeanAndVariance, MPSMatrixMultiplication, MPSMatrixVectorMultiplication, MPSMatrixBatchNormalization, MPSMatrixNeuron, MPSMatrixSoftMax, MPSImageThresholdBinaryInverse, MPSImageGuidedFilter, MPSCNNUpsampling, MPSCNNSpatialNormalization, MPSImageFindKeypoints, MPSNNReduceUnary, MPSNNReduceRowMin, MPSNNReduceColumnMin, MPSNNReduceFeatureChannelsMin, MPSNNReduceFeatureChannelsArgumentMin, MPSNNReduceRowMax, MPSNNReduceColumnMax, MPSNNReduceFeatureChannelsMax, MPSNNReduceFeatureChannelsArgumentMax, MPSNNReduceRowMean, MPSNNReduceColumnMean, MPSNNReduceFeatureChannelsMean, MPSNNReduceRowSum, MPSNNReduceColumnSum, MPSNNResizeBilinear, MPSImageReduceUnary, MPSImageReduceRowMin, MPSImageReduceColumnMin, MPSImageReduceRowMax, MPSImageReduceColumnMax, MPSImageReduceRowMean, MPSImageReduceColumnMean, MPSImageReduceRowSum, MPSImageReduceColumnSum, MPSImageScale, MPSImageStatisticsMinAndMax, MPSCNNPooling, MPSImageMedian, MPSImageAreaMax, and MPSMatrixCopy.

- MPSCopyAllocator

MPSImageKernel.h MetalPerformanceShaders.framework

Copyright:

Copyright (c) 2015 Apple Inc. All rights reserved. MetalPerformanceShaders filter base classes

A block to make a copy of sourceTexture for MPSKernels that can only execute out of place. Some MPSKernel objects may not be able to operate in place. When that occurs, and in-place operation is requested, MPS will call back to this block to get a new texture to return instead. To avoid spending long periods of time allocating pages to back the MTLTexture, the block should attempt to reuse textures. The texture returned from the MPSCopyAllocator will be returned instead of the sourceTexture from the MPSKernel method on return.

// A MPSCopyAllocator to handle cases where in-place operation fails.
MPSCopyAllocator myAllocator = ^id <MTLTexture>( MPSKernel * __nonnull filter,

__nonnull id <MTLCommandBuffer> cmdBuf,
__nonnull id <MTLTexture> sourceTexture) {
MTLPixelFormat format = sourceTexture.pixelFormat; // FIXME: is this format writable?
MTLTextureDescriptor *d = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat: format
width: sourceTexture.width
height: sourceTexture.height
mipmapped: NO];
d.usage = MTLTextureUsageShaderRead | MTLTextureUsageShaderWrite;
//FIXME: Allocating a new texture each time is slow. They take up to 1 ms each.
// There are not too many milliseconds in a video frame! You can recycle
// old textures (or MTLBuffers and make textures from them) and reuse
// the memory here.
id <MTLTexture> result = [cmdBuf.device newTextureWithDescriptor: d];
// FIXME: If there is any metadata associated with sourceTexture such as colorspace
// information, MTLResource.label, MTLResource.cpuCacheMode mode,
// MTLResource.MTLPurgeableState, etc., it may need to be similarly associated
// with the new texture to avoid losing your metadata.
// FIXME: If filter.clipRect doesn't cover the entire image, you may need to copy
// pixels from sourceTexture to the new texture or regions of the new texture
// will be uninitialized. You can make a MTLCommandEncoder to encode work on
// the MTLCommandBuffer here to do that work, if necessary. It will be
// scheduled to run immediately before the MPSKernel work. Do not call
// [MTLCommandBuffer enqueue/commit/waitUntilCompleted/waitUntilScheduled]
// in the MPSCopyAllocator block. Make sure to call -endEncoding on the
// MTLCommandEncoder so that the MTLCommandBuffer has no active encoder
// before returning.
// CAUTION: The next command placed on the MTLCommandBuffer after the MPSCopyAllocator
// returns is almost assuredly going to be encoded with a MTLComputeCommandEncoder.
// Creating any other type of encoder in the MPSCopyAllocator will probably cost
// an additional 0.5 ms of both CPU _AND_ GPU time (or more!) due to a double
// mode switch penalty.
// CAUTION: If other objects (in addition to the caller of -encodeToCommandBuffer:inPlaceTexture:...)
// own a reference to sourceTexture, they may need to be notified that
// sourceTexture has been replaced so that they can release that resource
// and adopt the new texture.
// The reference to sourceTexture owned by the caller of
// -encodeToCommandBuffer:inPlaceTexture... will be released by
// -encodeToCommandBuffer:inPlaceTexture:... after the kernel is encoded if
// and only if the MPSCopyAllocator is called, and the operation is successfully
// encoded out of place.
return result;
// d is autoreleased };


If nil is returned by the allocator, NO will be returned by the calling function.

When the MPSCopyAllocator is called, no MTLCommandEncoder is active on the commandBuffer. You may create a MTLCommandEncoder in the block to initialize the texture. Make sure to call -endEncoding on it before returning, if you do.

Parameters:

filter A valid pointer to the MPSKernel that is calling the MPSCopyAllocator. From it you can get the clipRect of the intended operation.
commandBuffer A valid MTLCommandBuffer. It can be used to obtain the device against which to allocate the new texture. You may also enqueue operations on the commandBuffer to initialize the texture on a encoder allocated in the block. You may not submit, enqueue or wait for scheduling/completion of the command buffer.
sourceTexture The texture that is providing the source image for the filter. You may wish to use its size and MTLPixelFormat for the new texture, but it is not requred.

Returns:

A new valid MTLTexture to use as the destination for the MPSKernel. If the calling function succeeds, its texture parameter will be overwritten with a pointer to this texture. If the calling function fails (highly unlikely, except for user error) then the texture will be released before the calling function returns.

- (const MTLRegion) MPSRectNoClip

MPSRectNoClip This is a special constant to indicate no clipping is to be done. The entire image will be used. This is the default clipping rectangle or the input extent for MPSKernels.

- device [read], [nonatomic], [retain]

The device on which the kernel will be used

- label [read], [write], [atomic], [copy]

A string to help identify this object.

- options [read], [write], [nonatomic], [assign]

The set of options used to run the kernel. MPSKernelOptions

Generated automatically by Doxygen for MetalPerformanceShaders.framework from the source code.

Mon Jul 9 2018 Version MetalPerformanceShaders-119.3