| MPSKernel(3) | MetalPerformanceShaders.framework | MPSKernel(3) |
MPSKernel
#import <MPSKernel.h>
Inherits NSObject, <NSCopying>, and <NSSecureCoding>.
Inherited by MPSAccelerationStructure, MPSBinaryImageKernel, MPSCNNBinaryKernel, MPSCNNKernel, MPSImageCopyToMatrix, MPSImageFindKeypoints, MPSImageGuidedFilter, MPSImageHistogram, MPSImageNormalizedHistogram, MPSMatrixBinaryKernel, MPSMatrixCopy, MPSMatrixCopyToImage, MPSMatrixMultiplication, MPSMatrixSum, MPSMatrixUnaryKernel, MPSNNGraph, MPSNNOptimizer, MPSRayIntersector, MPSRNNMatrixInferenceLayer, MPSRNNMatrixTrainingLayer, and MPSUnaryImageKernel.
(nonnull instancetype) - initWithDevice:
(nonnull instancetype) - copyWithZone:device:
(nullable instancetype) - initWithCoder:
(nullable instancetype) - initWithCoder:device:
() - MPSCopyAllocator
const MTLRegion MPSRectNoClip
MPSKernelOptions options
id< MTLDevice > device
NSString * label
MPSKernel.h MPSCore.framework
Copyright:
MPSKernel objects encode tuned image processing operations into a MTLCommandBuffer.
This depends on Metal.framework The MPSKernel class is the base class for all MPS objects. It defines a standard interface for MPS kernels. You should not use the MPSKernel class directly. Instead, a number of MPSKernel subclasses are available in MetalPerformanceShaders.framework that define specific high-performance image processing operations.
The basic sequence for applying a MPSKernel to an image is as follows:
MPSImageSobel *sobel = [[MPSImageSobel alloc] initWithDevice: mtlDevice];
sobel.offset = ...; sobel.clipRect = ...; sobel.options = ...; [sobel encodeToCommandBuffer: commandBuffer
sourceTexture: inputImage
destinationTexture: resultImage ];
Encoding the kernel merely encodes the operation into a MTLCommandBuffer. It
does not modify any pixels, yet. All MPSKernel state has been copied
to the command buffer. MPSKernels may be reused. If the texture was
previously operated on by another command encoder (e.g.
MTLRenderCommandEncoder), you should call -endEncoding on the other encoder
before encoding the filter.
Some MPS filters work in place (inputImage = resultImage) even in situations where Metal might not normally allow in place operation on textures. If in-place operation is desired, you may attempt to call [MPSKernel encodeKernelInPlace...]. If the operation can not be completed in place, then NO will be returned and you will have to create a new result texture and try again. To make an in-place image filter reliable, pass a fallback MPSCopyAllocator to the method to create a new texture to write to in the event that a filter can not operate in place.
(Repeat steps 2 for more filters, as desired.)
It should be self evident that step 2 may not be thread safe. That is, you can not have multiple threads manipulating the same properties on the same MPSKernel object at the same time and achieve coherent output. In common usage, the MPSKernel properties don't often need to be changed from their default values, but if you need to apply the same filter to multiple images on multiple threads with cropping / tiling, make additional MPSKernel objects per thread. They are cheap. You can use multiple MPSKernel objects on multiple threads, as long as only one thread is operating on any particular MPSKernel object at a time.
[mtlCommandBuffer commit];
A MPSKernel can be saved to disk / network using NSCoders such as NSKeyedArchiver. When decoding, the system default MTLDevice will be chosen unless the NSCoder adopts the <MPSDeviceProvider> protocol. To accomplish this, subclass or extend your unarchiver to add this method.
Make a copy of this MPSKernel for a new device -copyWithZone: will call this API to make a copy of the MPSKernel on the same device. This interface may also be called directly to make a copy of the MPSKernel on a new device. Typically, the same MPSKernels should not be used to encode kernels on multiple command buffers from multiple threads. Many MPSKernels have mutable properties that might be changed by the other thread while this one is trying to encode. If you need to use a MPSKernel from multiple threads make a copy of it for each additional thread using -copyWithZone: or -copyWithZone:device:
Parameters:
Returns:
Reimplemented in MPSRNNMatrixTrainingLayer, MPSRNNMatrixInferenceLayer, MPSRayIntersector, MPSRNNImageInferenceLayer, MPSAccelerationStructure, MPSMatrixFullyConnectedGradient, MPSMatrixNeuronGradient, MPSMatrixBatchNormalizationGradient, MPSMatrixSoftMaxGradient, MPSMatrixFindTopK, MPSMatrixFullyConnected, MPSMatrixBatchNormalization, MPSMatrixNeuron, and MPSMatrixSoftMax.
Called by NSCoder to decode MPSKernels This isn't the right interface to decode a MPSKernel, but it is the one that NSCoder uses. To enable your NSCoder (e.g. NSKeyedUnarchiver) to set which device to use extend the object to adopt the MPSDeviceProvider protocol. Otherwise, the Metal system default device will be used.
NSSecureCoding compatability While the standard NSSecureCoding/NSCoding method -initWithCoder: should work, since the file can't know which device your data is allocated on, we have to guess and may guess incorrectly. To avoid that problem, use initWithCoder:device instead.
Parameters:
Returns:
Reimplemented in MPSCNNBinaryConvolution, MPSCNNBinaryFullyConnected, MPSCNNConvolutionTranspose, MPSRNNMatrixTrainingLayer, MPSCNNConvolutionGradient, MPSCNNFullyConnected, MPSCNNFullyConnectedGradient, MPSCNNBinaryKernel, MPSCNNGradientKernel, MPSRNNMatrixInferenceLayer, MPSCNNConvolution, MPSCNNYOLOLoss, MPSRayIntersector, MPSRNNImageInferenceLayer, MPSCNNPoolingAverageGradient, MPSCNNPoolingMaxGradient, MPSCNNPoolingL2NormGradient, MPSCNNDilatedPoolingMaxGradient, MPSCNNSoftMaxGradient, MPSCNNLogSoftMaxGradient, MPSCNNCrossChannelNormalizationGradient, MPSCNNLoss, MPSCNNPoolingGradient, MPSImagePyramid, MPSCNNCrossChannelNormalization, MPSBinaryImageKernel, MPSImageHistogramSpecification, MPSCNNDilatedPoolingMax, MPSCNNLocalContrastNormalizationGradient, MPSAccelerationStructure, MPSCNNBatchNormalization, MPSCNNBatchNormalizationStatistics, MPSCNNBatchNormalizationGradient, MPSCNNBatchNormalizationStatisticsGradient, MPSImageHistogramEqualization, MPSImageSobel, MPSCNNNeuronGradient, MPSCNNPoolingAverage, MPSCNNPoolingL2Norm, MPSMatrixFullyConnectedGradient, MPSCNNKernel, MPSImageThresholdToZeroInverse, MPSMatrixNeuronGradient, MPSMatrixBatchNormalizationGradient, MPSCNNLocalContrastNormalization, MPSMatrixSoftMaxGradient, MPSCNNInstanceNormalization, MPSCNNNeuron, MPSImageThresholdToZero, MPSImageNormalizedHistogram, MPSCNNDropoutGradient, MPSUnaryImageKernel, MPSMatrixCopyToImage, MPSImageEuclideanDistanceTransform, MPSImageBox, MPSImageGaussianBlur, MPSMatrixCopy, MPSImageStatisticsMean, MPSImageThresholdBinary, MPSCNNSpatialNormalizationGradient, MPSImageThresholdTruncate, MPSNNCropAndResizeBilinear, MPSMatrixSum, MPSMatrixFindTopK, MPSImageDilate, MPSCNNDropout, MPSMatrixFullyConnected, MPSImageScale, MPSImageLanczosScale, MPSImageBilinearScale, MPSImageStatisticsMeanAndVariance, MPSMatrixBatchNormalization, MPSMatrixNeuron, MPSImageConvolution, MPSMatrixSoftMax, MPSImageThresholdBinaryInverse, MPSImageHistogram, MPSImageGuidedFilter, MPSImageCopyToMatrix, MPSCNNSpatialNormalization, MPSImageFindKeypoints, MPSNNResizeBilinear, MPSImageStatisticsMinAndMax, MPSCNNPooling, MPSCNNPoolingMax, MPSImageMedian, MPSImageAreaMax, and MPSNNGraph.
Standard init with default properties per filter type
Parameters:
Returns:
Reimplemented in MPSCNNBinaryConvolution, MPSCNNBinaryFullyConnected, MPSCNNConvolutionTranspose, MPSRNNMatrixTrainingLayer, MPSCNNConvolutionGradient, MPSCNNFullyConnected, MPSCNNFullyConnectedGradient, MPSCNNGradientKernel, MPSRNNMatrixInferenceLayer, MPSCNNConvolution, MPSNNOptimizerAdam, MPSCNNYOLOLoss, MPSRayIntersector, MPSRNNImageInferenceLayer, MPSCNNSoftMaxGradient, MPSCNNLogSoftMaxGradient, MPSNNOptimizerRMSProp, MPSCNNLoss, MPSCNNPoolingGradient, MPSCNNCrossChannelNormalization, MPSBinaryImageKernel, MPSNNReduceFeatureChannelsAndWeightsSum, MPSNNReshape, MPSCNNArithmeticGradient, MPSNNReduceBinary, MPSNNReduceFeatureChannelsAndWeightsMean, MPSImagePyramid, MPSAccelerationStructure, MPSCNNBatchNormalization, MPSCNNBatchNormalizationStatistics, MPSNNOptimizerStochasticGradientDescent, MPSNNReduceFeatureChannelsSum, MPSImageSobel, MPSCNNNeuronGradient, MPSCNNNeuronLinear, MPSCNNNeuronReLU, MPSCNNNeuronPReLU, MPSCNNNeuronSigmoid, MPSCNNNeuronHardSigmoid, MPSCNNNeuronTanH, MPSCNNNeuronAbsolute, MPSCNNNeuronSoftPlus, MPSCNNNeuronSoftSign, MPSCNNNeuronELU, MPSCNNNeuronReLUN, MPSCNNNeuronPower, MPSCNNNeuronExponential, MPSCNNNeuronLogarithm, MPSMatrixFullyConnectedGradient, MPSCNNBinaryKernel, MPSNNOptimizer, MPSImageThresholdToZeroInverse, MPSMatrixNeuronGradient, MPSMatrixSum, MPSMatrixBatchNormalizationGradient, MPSCNNLocalContrastNormalization, MPSMatrixSoftMaxGradient, MPSCNNInstanceNormalization, MPSCNNKernel, MPSCNNNeuron, MPSCNNUpsamplingGradient, MPSImageThresholdToZero, MPSCNNDropoutGradient, MPSUnaryImageKernel, MPSImageEuclideanDistanceTransform, MPSImageBox, MPSImageGaussianBlur, MPSImageStatisticsMean, MPSImageThresholdBinary, MPSImageThresholdTruncate, MPSCNNArithmetic, MPSCNNAdd, MPSCNNSubtract, MPSCNNMultiply, MPSCNNDivide, MPSNNCropAndResizeBilinear, MPSNNSlice, MPSMatrixFindTopK, MPSImageDilate, MPSCNNDropout, MPSMatrixFullyConnected, MPSImageLanczosScale, MPSImageBilinearScale, MPSImageArithmetic, MPSImageAdd, MPSImageSubtract, MPSImageMultiply, MPSImageDivide, MPSImageStatisticsMeanAndVariance, MPSMatrixMultiplication, MPSMatrixVectorMultiplication, MPSMatrixBatchNormalization, MPSMatrixNeuron, MPSMatrixSoftMax, MPSImageThresholdBinaryInverse, MPSImageGuidedFilter, MPSCNNUpsampling, MPSCNNSpatialNormalization, MPSImageFindKeypoints, MPSNNReduceUnary, MPSNNReduceRowMin, MPSNNReduceColumnMin, MPSNNReduceFeatureChannelsMin, MPSNNReduceFeatureChannelsArgumentMin, MPSNNReduceRowMax, MPSNNReduceColumnMax, MPSNNReduceFeatureChannelsMax, MPSNNReduceFeatureChannelsArgumentMax, MPSNNReduceRowMean, MPSNNReduceColumnMean, MPSNNReduceFeatureChannelsMean, MPSNNReduceRowSum, MPSNNReduceColumnSum, MPSNNResizeBilinear, MPSImageReduceUnary, MPSImageReduceRowMin, MPSImageReduceColumnMin, MPSImageReduceRowMax, MPSImageReduceColumnMax, MPSImageReduceRowMean, MPSImageReduceColumnMean, MPSImageReduceRowSum, MPSImageReduceColumnSum, MPSImageScale, MPSImageStatisticsMinAndMax, MPSCNNPooling, MPSImageMedian, MPSImageAreaMax, and MPSMatrixCopy.
MPSImageKernel.h MetalPerformanceShaders.framework
Copyright:
A block to make a copy of sourceTexture for MPSKernels that can only execute out of place. Some MPSKernel objects may not be able to operate in place. When that occurs, and in-place operation is requested, MPS will call back to this block to get a new texture to return instead. To avoid spending long periods of time allocating pages to back the MTLTexture, the block should attempt to reuse textures. The texture returned from the MPSCopyAllocator will be returned instead of the sourceTexture from the MPSKernel method on return.
// A MPSCopyAllocator to handle cases where in-place operation fails. MPSCopyAllocator myAllocator = ^id <MTLTexture>( MPSKernel * __nonnull filter,
__nonnull id <MTLCommandBuffer> cmdBuf,
__nonnull id <MTLTexture> sourceTexture) {
MTLPixelFormat format = sourceTexture.pixelFormat; // FIXME: is this format writable?
MTLTextureDescriptor *d = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat: format
width: sourceTexture.width
height: sourceTexture.height
mipmapped: NO];
d.usage = MTLTextureUsageShaderRead | MTLTextureUsageShaderWrite;
//FIXME: Allocating a new texture each time is slow. They take up to 1 ms each.
// There are not too many milliseconds in a video frame! You can recycle
// old textures (or MTLBuffers and make textures from them) and reuse
// the memory here.
id <MTLTexture> result = [cmdBuf.device newTextureWithDescriptor: d];
// FIXME: If there is any metadata associated with sourceTexture such as colorspace
// information, MTLResource.label, MTLResource.cpuCacheMode mode,
// MTLResource.MTLPurgeableState, etc., it may need to be similarly associated
// with the new texture to avoid losing your metadata.
// FIXME: If filter.clipRect doesn't cover the entire image, you may need to copy
// pixels from sourceTexture to the new texture or regions of the new texture
// will be uninitialized. You can make a MTLCommandEncoder to encode work on
// the MTLCommandBuffer here to do that work, if necessary. It will be
// scheduled to run immediately before the MPSKernel work. Do not call
// [MTLCommandBuffer enqueue/commit/waitUntilCompleted/waitUntilScheduled]
// in the MPSCopyAllocator block. Make sure to call -endEncoding on the
// MTLCommandEncoder so that the MTLCommandBuffer has no active encoder
// before returning.
// CAUTION: The next command placed on the MTLCommandBuffer after the MPSCopyAllocator
// returns is almost assuredly going to be encoded with a MTLComputeCommandEncoder.
// Creating any other type of encoder in the MPSCopyAllocator will probably cost
// an additional 0.5 ms of both CPU _AND_ GPU time (or more!) due to a double
// mode switch penalty.
// CAUTION: If other objects (in addition to the caller of -encodeToCommandBuffer:inPlaceTexture:...)
// own a reference to sourceTexture, they may need to be notified that
// sourceTexture has been replaced so that they can release that resource
// and adopt the new texture.
// The reference to sourceTexture owned by the caller of
// -encodeToCommandBuffer:inPlaceTexture... will be released by
// -encodeToCommandBuffer:inPlaceTexture:... after the kernel is encoded if
// and only if the MPSCopyAllocator is called, and the operation is successfully
// encoded out of place.
return result;
// d is autoreleased };
If nil is returned by the allocator, NO will be returned by the calling
function.
When the MPSCopyAllocator is called, no MTLCommandEncoder is active on the commandBuffer. You may create a MTLCommandEncoder in the block to initialize the texture. Make sure to call -endEncoding on it before returning, if you do.
Parameters:
Returns:
MPSRectNoClip This is a special constant to indicate no clipping is to be done. The entire image will be used. This is the default clipping rectangle or the input extent for MPSKernels.
The device on which the kernel will be used
A string to help identify this object.
The set of options used to run the kernel. MPSKernelOptions
Generated automatically by Doxygen for MetalPerformanceShaders.framework from the source code.
| Mon Jul 9 2018 | Version MetalPerformanceShaders-119.3 |