ReplayKit2 屏幕录制
如果你需要录制苹果手机屏幕，ReplayKit肯定需要了解。本文主要介绍Replaykit2 在iOS12后的一些技巧及使用方法。为啥不介绍iOS12前的录制呢，因为操作起来太麻烦了，麻烦到开发的我们使用起来都很不顺手，别说用户使用，而且现在苹果都iOS14了...
ReplayKit2 使用的技巧
由于系统提供的是个RPSystemBroadcastPickerView类型的View，需要用户点击这个View才能弹出录制界面。那如何才能优雅的把它给隐藏呢？答案是我们在它上面覆盖一层View，然后把点击事件传递给它达到点击效果。事件类型根据不同系统版本会稍有不同,直接贴代码：
@property (nonatomic, strong) RPSystemBroadcastPickerView *sysTemBroadCastPickerView; //录制view
@property (nonatomic, strong) UIButton *startPushStreamBtn; //开始录制按钮
- (void)showReplayKitView
{
if (@available(iOS 12.0, *)) {
for (UIView *view in _sysTemBroadCastPickerView.subviews) {
if ([view isKindOfClass:[UIButton class]]) {
float iOSVersion = [[UIDevice currentDevice].systemVersion floatValue];
UIButton *button = (UIButton *)view;
if (button != self.startPushStreamBtn) {
if (iOSVersion >= 13) {
[(UIButton *)view sendActionsForControlEvents:UIControlEventTouchDown];
[(UIButton *)view sendActionsForControlEvents:UIControlEventTouchUpInside];
} else {
[(UIButton *)view sendActionsForControlEvents:UIControlEventTouchDown];
}
}
}
}
}
}
如何指定录制应用，是否使用麦克风。注意BundleID为录制Target的BundleID，和主工程的BundleID区分下。如果不填写，弹窗的录制界面将会带上手机上所有支持录制的应用，是不是很不友好..
- (RPSystemBroadcastPickerView *)sysTemBroadCastPickerView
API_AVAILABLE(ios(12.0))
{
if (!_sysTemBroadCastPickerView) {
_sysTemBroadCastPickerView = [[RPSystemBroadcastPickerView alloc] init];
_sysTemBroadCastPickerView.showsMicrophoneButton = NO;//是否显示麦克风
_sysTemBroadCastPickerView.preferredExtension = [MnaConfig replayKitBundleID];//指定录制应用BundleID
}
return _sysTemBroadCastPickerView;
}
点击开始直播后有个倒计时，倒计时结束后如何优雅的退出录制界面？首先我们需要捕获系统录制弹窗，看弹出效果应该是 presentViewController 然后采用Method Swizzling尝试下，发现可以拿到。然后我们可以在录制进程启动后发送进程通知，主进程收到后进行dismiss：代码如下：
@implementation UIViewController (MnaPresentSwizzleAdd)
+ (void)load
{
static dispatch_once_t onceToken;
dispatch_once(&onceToken, ^{
[self swizzleSelector:@selector(presentViewController:animated:completion:) withAnotherSelector:@selector(mna_presentViewController:animated:completion:)];
});
}
+ (void)swizzleSelector:(SEL)originalSelector withAnotherSelector:(SEL)swizzledSelector
{
Class aClass = [self class];
Method originalMethod = class_getInstanceMethod(aClass, originalSelector);
Method swizzledMethod = class_getInstanceMethod(aClass, swizzledSelector);
BOOL didAddMethod =
class_addMethod(aClass,
originalSelector,
method_getImplementation(swizzledMethod),
method_getTypeEncoding(swizzledMethod));
if (didAddMethod) {
class_replaceMethod(aClass,
swizzledSelector,
method_getImplementation(originalMethod),
method_getTypeEncoding(originalMethod));
} else {
method_exchangeImplementations(originalMethod, swizzledMethod);
}
}
#pragma mark - Method Swizzling
- (void)mna_presentViewController:(UIViewController *)viewControllerToPresent animated:(BOOL)flag completion:(void (^)(void))completion
{
if ([NSStringFromClass(viewControllerToPresent.class) isEqualToString:@"RPBroadcastPickerStandaloneViewController"]) {
MnaReplayKitHiddenManager.sharedInstance.replayKitBraodViewControler = viewControllerToPresent; //该管理类监听录制进程启动完成的通知然后进行Dismiss
[self mna_presentViewController:viewControllerToPresent animated:flag completion:completion];
} else {
[self mna_presentViewController:viewControllerToPresent animated:flag completion:completion];
}
}
@end
更新：iOS14后弹出方法更改，暂时还不知道如何显示，该方法只适用iOS14以下
进程通知，进程间通讯非常麻烦。可以采用CFNotificationCenterPostNotification进行消息传递，不过有个问题是不能传递数据。不过我们可以使用App Groups可以数据共享的方式来进行传递。推荐一个已经封装好的开源库 MMWormhole
踩过的坑
Extension进程最麻烦的就是调试。如果要查看Log，录制调试可以选择 录制进程->运行->选择主进程。这样启动后可以在终端看到Log，不过以前同事遇到过某些Xcode版本不能运行，并看不了Log。测试可行的Xcode版本:Version 12.1 (12A7403)
如果你需要打印一些日志保存,进行问题定位，需要保存到Group共享区。获取方式：
[[NSFileManager defaultManager] containerURLForSecurityApplicationGroupIdentifier:goupName];
内存限制50M这个非常重要，因为如果超过这个大小直接被系统杀掉。如果你做推流：必须控制缓冲队列大小,做好内存管理。
音频输出为大端，如果需要，可以转换下大小端。转换为小端代码如下：
- (NSData *)convertAudioSamepleBufferToPcmData:(CMSampleBufferRef)sampleBuffer
{
CMBlockBufferRef blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer);
if (blockBuffer == nil) {
return nil;
}
AudioBufferList bufferList;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBuffer,
NULL,
&bufferList,
sizeof(bufferList),
NULL,
NULL,
kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment,
&blockBuffer);
int8_t *audioBuffer = (int8_t *)bufferList.mBuffers[0].mData;
UInt32 audioBufferSizeInBytes = bufferList.mBuffers[0].mDataByteSize;
CMFormatDescriptionRef formatDescription = CMSampleBufferGetFormatDescription(sampleBuffer);
const AudioStreamBasicDescription *asbd = CMAudioFormatDescriptionGetStreamBasicDescription(formatDescription);
// Perform an endianess conversion, if needed. A TVIAudioDevice should deliver little endian samples.
if (asbd->mFormatFlags & kAudioFormatFlagIsBigEndian) { //大端
for (int i = 0; i < (audioBufferSizeInBytes - 1); i += 2) {
int8_t temp = audioBuffer[i];
audioBuffer[i] = audioBuffer[i + 1];
audioBuffer[i + 1] = temp;
}
} else { //小端
}
NSData *data = [NSData dataWithBytes:audioBuffer length:audioBufferSizeInBytes];
CFRelease(blockBuffer);
return data;
}
不管你手机竖屏还是横屏，非常无语的是视频输出，都是竖屏。还好iOS11后有个方法可以判定当前输出的视频帧方向:
CGImagePropertyOrientation oritation = ((__bridge NSNumber *)CMGetAttachment(buffer, (__bridge CFStringRef)RPVideoSampleOrientationKey, NULL)).unsignedIntValue;
typedef CF_CLOSED_ENUM(uint32_t, CGImagePropertyOrientation) {
kCGImagePropertyOrientationUp = 1, // 0th row at top, 0th column on left - default orientation
kCGImagePropertyOrientationUpMirrored, // 0th row at top, 0th column on right - horizontal flip
kCGImagePropertyOrientationDown, // 0th row at bottom, 0th column on right - 180 deg rotation
kCGImagePropertyOrientationDownMirrored, // 0th row at bottom, 0th column on left - vertical flip
kCGImagePropertyOrientationLeftMirrored, // 0th row on left, 0th column at top
kCGImagePropertyOrientationRight, // 0th row on right, 0th column at top - 90 deg CW
kCGImagePropertyOrientationRightMirrored, // 0th row on right, 0th column on bottom
kCGImagePropertyOrientationLeft // 0th row on left, 0th column at bottom - 90 deg CCW
};
由于视频都是竖屏输出，所以需要旋转方向然后再转换为CVPixelBufferRef进行硬编码。找过很多资料，这块介绍很少。有介绍使用开源库libyuv 不过是转换为i420后调用的腾讯云接口无相关涉及。现在使用的是方法：
#pragma mark - Rotation default stream
- (void)dealWithSampleBuffer:(CMSampleBufferRef)buffer timeStamp:(uint64_t)timeStamp
{
if (@available(iOS 11.0, *)) {
CGImagePropertyOrientation oritation = ((__bridge NSNumber *)CMGetAttachment(buffer, (__bridge CFStringRef)RPVideoSampleOrientationKey, NULL)).unsignedIntValue;
CIImage *outputImage = nil;
CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(buffer);
CGFloat outputWidth = self.session.videoConfiguration.videoSize.width;
CGFloat outputHeight = self.session.videoConfiguration.videoSize.height;
BOOL isLandScape = self.session.videoConfiguration.landscape;
size_t inputWidth = CVPixelBufferGetWidth(pixelBuffer);
size_t inputHeight = CVPixelBufferGetHeight(pixelBuffer);
CGAffineTransform lastRotateTransform = CGAffineTransformMakeScale(0.5, 0.5);
CIImage *sourceImage = nil;
CGImagePropertyOrientation lastRotateOritation = oritation;
sourceImage = [CIImage imageWithCVPixelBuffer:pixelBuffer];
// 如果是横屏且输入源为横屏（iPad Pro）或者 竖屏且输入源为竖屏
if ((inputWidth > inputHeight && isLandScape) || (inputWidth <= inputHeight && !isLandScape)) {
if (oritation == kCGImagePropertyOrientationUp) {
lastRotateOritation = kCGImagePropertyOrientationUp;
} else if (oritation == kCGImagePropertyOrientationDown) {
lastRotateOritation = kCGImagePropertyOrientationDown;
}
lastRotateTransform = CGAffineTransformMakeScale(outputWidth / inputWidth, outputHeight / inputHeight);
} else {
if (oritation == kCGImagePropertyOrientationLeft) {
lastRotateOritation = kCGImagePropertyOrientationRight;
} else if (oritation == kCGImagePropertyOrientationRight) {
lastRotateOritation = kCGImagePropertyOrientationLeft;
} else {
lastRotateOritation = kCGImagePropertyOrientationLeft;
}
lastRotateTransform = CGAffineTransformMakeScale(outputWidth / inputHeight, outputHeight / inputWidth);
}
sourceImage = [sourceImage imageByApplyingCGOrientation:lastRotateOritation];
outputImage = [sourceImage imageByApplyingTransform:lastRotateTransform];
if (outputImage) {
NSDictionary *pixelBufferOptions = @{(NSString *)kCVPixelBufferWidthKey : @(outputWidth),
(NSString *)kCVPixelBufferHeightKey : @(outputHeight),
(NSString *)kCVPixelBufferOpenGLESCompatibilityKey : @YES,
(NSString *)kCVPixelBufferIOSurfacePropertiesKey : @{} };
CVPixelBufferLockBaseAddress(pixelBuffer, 0);
CVPixelBufferRef newPixcelBuffer = nil;
CVPixelBufferCreate(kCFAllocatorDefault, outputWidth, outputHeight, kCVPixelFormatType_420YpCbCr8BiPlanarFullRange, (__bridge CFDictionaryRef)pixelBufferOptions, &newPixcelBuffer);
[_ciContext render:outputImage toCVPixelBuffer:newPixcelBuffer];
CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
CMVideoFormatDescriptionRef videoInfo = nil;
CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault, newPixcelBuffer, &videoInfo);
CMTime duration = CMSampleBufferGetDuration(buffer);
CMTime presentationTimeStamp = CMSampleBufferGetPresentationTimeStamp(buffer);
CMTime decodeTimeStamp = CMSampleBufferGetDecodeTimeStamp(buffer);
CMSampleTimingInfo sampleTimingInfo;
sampleTimingInfo.duration = duration;
sampleTimingInfo.presentationTimeStamp = presentationTimeStamp;
sampleTimingInfo.decodeTimeStamp = decodeTimeStamp;
//
CMSampleBufferRef newSampleBuffer = nil;
CMSampleBufferCreateForImageBuffer(kCFAllocatorMalloc, newPixcelBuffer, true, nil, nil, videoInfo, &sampleTimingInfo, &newSampleBuffer);
// 对新buffer做处理
[self.session pushVideoBuffer:newSampleBuffer timeStamp:timeStamp];
// release
if (newPixcelBuffer) {
CVPixelBufferRelease(newPixcelBuffer);
}
if (newSampleBuffer) {
CFRelease(newSampleBuffer);
}
}
} else {
// Fallback on earlier versions
[self.session pushVideoBuffer:buffer timeStamp:timeStamp];
}
}
推流。可以参考 LFLiveKit ，需要注意的是前面说的缓冲区大小设置，还有就是里面LibRTMP调整输出块大小，减少CPU消耗。