OpenHarmony开发者论坛
标题:
OpenHarmony 3.2 Release版本Codec HDI适配过程
[打印本页]
作者:
Laval社区小助手
时间:
2023-11-1 09:41
标题:
OpenHarmony 3.2 Release版本Codec HDI适配过程
[md]详情：[OpenHarmony 3.2 Release版本Codec HDI适配过程](
https://laval.csdn.net/64c8b1587 ... tml?login=from_csdn
)
# 简介
OpenHarmony Codec HDI（Hardware Device Interface）驱动框架基于OpenMax实现了视屏硬件编解码驱动，提供Codec基础能力接口供上层媒体服务调用，包括获取组件编解码能力、创建组件、参数设置、数据的轮转和控制、以及销毁组件等功能，实现对视频数据的编解码处理。
## 视频编解码驱动架构
**Codec HDI 2.0接口依赖OpenMax IL的标准接口。OMX Wrapper将OMX接口的实现封装成libOMX\_Core.z.so供HDI层调用。如果codec驱动为实现OpenMax标准接口，则需根据驱动适配实现OMX Interface接口。**
Codec HDI 2.0接口列表：
| 头文件 | 接口名称 | 功能描述 |
| ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------ |
| codec\_component \_manager.h | int32\_t (\*GetComponentNum)(); | 获取Codec编解码组件数量 |
| | int32\_t (\*GetComponentCapabilityList)(CodecCompCapability \*capList, int32\_t count); | 获取编解码能力集表 |
| | int32\_t (\*CreateComponent)(struct CodecComponentType \*\*component, uint32\_t \*componentId, char \*compName, int64\_t appData, struct CodecCallbackType \*callbacks); | 创建Codec组件实例 |
| | int32\_t (\*DestroyComponent)(uint32\_t componentId); | 销毁组件实例 |
| codec\_component \_if.h | int32\_t (\*GetComponentVersion)(struct CodecComponentType \*self, struct CompVerInfo \*verInfo); | 获取Codec组件版本号 |
| | int32\_t (\*SendCommand)(struct CodecComponentType \*self, enum OMX\_COMMANDTYPE cmd, uint32\_t param, int8\_t \*cmdData, uint32\_t cmdDataLen); | 发送命令给组件 |
| | int32\_t (\*GetParameter)(struct CodecComponentType \*self, uint32\_t paramIndex, int8\_t \*paramStruct, uint32\_t paramStructLen); | 获取组件参数设置 |
| | int32\_t (\*SetParameter)(struct CodecComponentType \*self, uint32\_t index, int8\_t \*paramStruct, uint32\_t paramStructLen); | 设置组件需要的参数 |
| | int32\_t (\*GetConfig)(struct CodecComponentType \*self, uint32\_t index, int8\_t \*cfgStruct, uint32\_t cfgStructLen); | 获取组件的配置结构 |
| | int32\_t (\*SetConfig)(struct CodecComponentType \*self, uint32\_t index, int8\_t \*cfgStruct, uint32\_t cfgStructLen); | 设置组件的配置 |
| | int32\_t (\*GetExtensionIndex)(struct CodecComponentType \*self, const char \*paramName, uint32\_t \*indexType); | 根据字符串获取组件的扩展索引 |
| | int32\_t (\*GetState)(struct CodecComponentType \*self, enum OMX\_STATETYPE \*state); | 获取组件的状态 |
| | int32\_t (\*ComponentTunnelRequest)(struct CodecComponentType \*self, uint32\_t port, int32\_t tunneledComp, uint32\_t tunneledPort, struct OMX\_TUNNELSETUPTYPE \*tunnelSetup); | 设置组件Tunneled方式通信 |
| | int32\_t (\*UseBuffer)(struct CodecComponentType \*self, uint32\_t portIndex, struct OmxCodecBuffer \*buffer); | 指定组件端口的buffer |
| | int32\_t (\*AllocateBuffer)(struct CodecComponentType \*self, uint32\_t portIndex, struct OmxCodecBuffer \*buffer); | 向组件申请端口buffer |
| | int32\_t (\*FreeBuffer)(struct CodecComponentType \*self, uint32\_t portIndex, const struct OmxCodecBuffer \*buffer); | 释放buffer |
| | int32\_t (\*EmptyThisBuffer)(struct CodecComponentType \*self, const struct OmxCodecBuffer \*buffer); | 编解码输入待处理buffer |
| | int32\_t (\*FillThisBuffer)(struct CodecComponentType \*self, const struct OmxCodecBuffer \*buffer); | 编解码输出填充buffer |
| | int32\_t (\*SetCallbacks)(struct CodecComponentType \*self, struct CodecCallbackType \*callback, int64\_t appData); | 设置Codec组件的回调函数 |
| | int32\_t (\*ComponentDeInit)(struct CodecComponentType \*self); | 组件去初始化 |
| | int32\_t (\*UseEglImage)(struct CodecComponentType \*self, struct OmxCodecBuffer \*buffer, uint32\_t portIndex, int8\_t \*eglImage, uint32\_t eglImageLen); | 使用已在ELG中申请的空间 |
| | int32\_t (\*ComponentRoleEnum)(struct CodecComponentType \*self, uint8\_t \*role, uint32\_t roleLen, uint32\_t index); | 获取组件角色 |
| codec\_callback\_if.h | int32\_t (\*EventHandler)(struct CodecCallbackType \*self, enum OMX\_EVENTTYPE event, struct EventInfo \*info); | 事件上报 |
| | int32\_t (\*EmptyBufferDone)(struct CodecCallbackType \*self, int64\_t appData, const struct OmxCodecBuffer \*buffer); | 上报输入buffer编码或者解码处理完毕 |
| | int32\_t (\*FillBufferDone)(struct CodecCallbackType \*self, int64\_t appData, const struct OmxCodecBuffer \*buffer); | 上报输出buffer填充完毕 |
## Codec HDI相关目录接口
```
├── //drivers/peripheral/codec
│ ├── hal
│ │ ├── BUILD.gn
│ │ ├── idl_service
│ │ ├── include
│ │ ├── passthrough # v2.0到v1.0的转换，v1.0接口已弃用，无需实现
│ │ ├── src
│ │ ├── v1.0 # codec hdi v1.0接口的实现，已弃用，MediaService已不对接相关接口。
│ │ └── v2.0 # codec hdi v2.0接口的实现，依赖OpenMax接口，需封装实现libOMX_Core.z.so
│ ├── hdi_service # codec_host相关实现
│ │ ├── BUILD.gn
│ │ ├── codec_proxy
│ │ ├── codec_service_stub
│ │ └── common
│ ├── interfaces
│ │ └── include
│ └── test
```
## OMX\_Core相关接口
codec hdi V2.0的实现依赖libOMX\_Core.z.so，需根据OpenMax标准接口封装实现。
参考drivers/peripheral/codec/hal/v2.0/hdi\_impl/include/codec\_omx\_core.h的定义调用过程。
```
typedef OMX_ERRORTYPE (*InitFunc)();
typedef OMX_ERRORTYPE (*DeinitFunc)();
typedef OMX_ERRORTYPE (*ComponentNameEnumFunc)(OMX_STRING, OMX_U32, OMX_U32);
typedef OMX_ERRORTYPE (*GetHandleFunc)(OMX_HANDLETYPE *, OMX_STRING, OMX_PTR, OMX_CALLBACKTYPE *);
typedef OMX_ERRORTYPE (*FreeHandleFunc)(OMX_HANDLETYPE);
typedef OMX_ERRORTYPE (*GetRolesOfComponentFunc)(OMX_STRING, OMX_U32 *, OMX_U8 **);
```
## HCS配置
### 配置codec\_host服务
./hdf\_config/uhdf/device\_info.hcs
```
codec :: host {
hostName = "codec_host";
priority = 50;
gid = ["codec_host", "uhdf_driver", "vendor_mpp_driver"];
codec_omx_device :: device {
device0 :: deviceNode {
policy = 2;
priority = 100;
moduleName = "libcodec_hdi_omx_server.z.so";
serviceName = "codec_hdi_omx_service";
deviceMatchAttr = "media_codec_capabilities";
}
}
}
```
### 配置codec\_capabilities
根据codec::host中配置的deviceMatchAttr，配置hdf.hcs
./hdf\_config/uhdf/hdf.hcs
```
#include "media_codec_capabilitie.hcs"
```
参考media\_codec\_capabilitie.hcs
```
root {
module = "master";
codec_config {
match_attr = "media_codec_capabilities";
use_openmax = true;
// capsMask: 0x01, Adaptive playback; 0x02, Secure playback; 0x04, Tunnel playback.
// allocateMask: 0x01, Input buffer allocated within the Codec module;
// allocateMask: 0x02, Input buffer allocated by an external user;
// allocateMask: 0x04, Output buffer allocated within the Codec module;
// allocateMask: 0x08, Output buffer allocated by an external user.
VideoHwEncoders {
/* node name explanation -- HDF_video_hw_enc_avc_rk：
**
** HDF____________video__________________hw____________________enc____________avc_______rk
** | | | | | |
** HDF or OMX video or audio hardware or software encoder or decoder mime vendor
*/
HDF_video_hw_enc_avc_rk {
role = 1;
type = 1;
name = "OMX.rk.video_encoder.avc";
supportProfiles = [1, 32768, 2, 32768, 8, 32768];
maxInst = 4;
isSoftwareCodec = false;
processModeMask = [];
capsMask = [0x01];
minBitRate = 1;
maxBitRate = 40000000;
minWidth = 176;
minHeight = 144;
maxWidth = 1920;
maxHeight = 1088;
widthAlignment = 16;
heightAlignment = 8;
minBlockCount = 99;
maxBlockCount = 8160;
minBlocksPerSecond = 99;
maxBlocksPerSecond = 489600;
blockSizeWidth = 16;
blockSizeHeight = 16;
supportPixelFmts = [28, 24, 20, 12];
measuredFrameRate = [320, 240, 165, 165, 720, 480, 149, 149, 1280, 720, 73, 73, 1920, 1080, 18, 18];
bitRateMode = [1, 2];
minFrameRate = 1;
maxFrameRate = 60;
}
......
......
......
}
}
```
## RK3568的参考适配过程
### OMX Wrapper的封装
根据gn文件：//drivers/peripheral/codec/BUILD.gn
```
OMX_IL_PATH = rebase_path(
"//device/soc/${device_company}/${product_name}/hardware/omx_il")
cmd = "if [ -f ${OMX_IL_PATH}/BUILD.gn ]; then echo true; else echo false; fi"
HAVE_OMX_IL_PATH =
exec_script("//build/lite/run_shell_cmd.py", [ cmd ], "value")
if (HAVE_OMX_IL_PATH) {
deps += [ "${OMX_IL_PATH}:lib_omx" ]
}
```
需创建lib\_omx工程，并封装实现libOMX\_Core.z.so，供codec hdi接口调用。
如果codec驱动已实现OpenMax标准接口，则可直接封装libOMX\_Core库，否则需要根据私有驱动实现OpenMax接口。
参考RK3568的适配过程，因codec驱动使用rockchip的mpp平台实现，需根据私有驱动实现OpenMax的接口。
参考gn文件：//device/soc/rockchip/rk3568/hardware/omx\_il/BUILD.gn
```
group("lib_omx") {
if (product_name == "rk3568") {
deps = [
"//device/soc/rockchip/rk3568/hardware/omx_il/component/video/dec:libomxvpu_dec",
"//device/soc/rockchip/rk3568/hardware/omx_il/component/video/enc:libomxvpu_enc",
"//device/soc/rockchip/rk3568/hardware/omx_il/core:libOMX_Core",
"//device/soc/rockchip/rk3568/hardware/omx_il/libOMXPlugin:libOMX_Pluginhw",
]
}
}
```
### hcs配置
* 配置codec\_host服务
//vendor/hihope/rk3568/hdf\_config/uhdf/device\_info.hcs
```
codec :: host {
hostName = "codec_host";
priority = 50;
gid = ["codec_host", "uhdf_driver", "vendor_mpp_driver"];
codec_omx_device :: device {
device0 :: deviceNode {
policy = 2;
priority = 100;
moduleName = "libcodec_hdi_omx_server.z.so";
serviceName = "codec_hdi_omx_service";
deviceMatchAttr = "codec_component_capabilities";
}
}
}
```
* 配置codec hcs
需根据硬件信息配置编解码组件的配置参数
//vendor/hihope/rk3568/hdf\_config/uhdf/hdf.hcs
```
#include "media_codec/codec_component_capabilities.hcs"
```
//vendor/hihope/rk3568/hdf\_config/uhdf/media\_codec/codec\_component\_capabilities.hcs
```
root {
module = "master";
codec_config {
match_attr = "codec_component_capabilities";
use_openmax = true;
// capsMask: 0x01, Adaptive playback; 0x02, Secure playback; 0x04, Tunnel playback.
// allocateMask: 0x01, Input buffer allocated within the Codec module;
// allocateMask: 0x02, Input buffer allocated by an external user;
// allocateMask: 0x04, Output buffer allocated within the Codec module;
// allocateMask: 0x08, Output buffer allocated by an external user.
VideoHwEncoders {
/* node name explanation -- HDF_video_hw_enc_avc_rk：
**
** HDF____________video__________________hw____________________enc____________avc_______rk
** | | | | | |
** HDF or OMX video or audio hardware or software encoder or decoder mime vendor
*/
HDF_video_hw_enc_avc_rk {
role = 1;
type = 1;
name = "OMX.rk.video_encoder.avc";
supportProfiles = [1, 32768, 2, 32768, 8, 32768];
maxInst = 4;
isSoftwareCodec = false;
processModeMask = [];
capsMask = [0x01];
minBitRate = 1;
maxBitRate = 40000000;
minWidth = 176;
minHeight = 144;
maxWidth = 1920;
maxHeight = 1088;
widthAlignment = 16;
heightAlignment = 8;
minBlockCount = 0xFFFFFFFF;
maxBlockCount = 0xFFFFFFFF;
minBlocksPerSecond = 0xFFFFFFFF;
maxBlocksPerSecond = 0xFFFFFFFF;
blockSizeWidth = 0xFFFFFFFF;
blockSizeHeight = 0xFFFFFFFF;
supportPixelFmts = [28, 24, 20, 12];
measuredFrameRate = [320, 240, 165, 165, 720, 480, 149, 149, 1280, 720, 73, 73, 1920, 1080, 18, 18];
bitRateMode = [1, 2];
minFrameRate = 0;
maxFrameRate = 0;
}
}
VideoHwDecoders {
HDF_video_hw_dec_avc_rk {
role = 1;
type = 0;
name = "OMX.rk.video_decoder.avc";
supportProfiles = [1, 32768, 2, 32768, 8, 32768];
maxInst = 6;
isSoftwareCodec = false;
processModeMask = [];
capsMask = [0x01];
minBitRate = 1;
maxBitRate = 10000000;
minWidth = 176;
minHeight = 144;
maxWidth = 4096;
maxHeight = 2160;
widthAlignment = 8;
heightAlignment = 8;
minBlockCount = 0xFFFFFFFF;
maxBlockCount = 0xFFFFFFFF;
minBlocksPerSecond = 1;
maxBlocksPerSecond = 244800;
blockSizeWidth = 16;
blockSizeHeight = 16;
supportPixelFmts = [24];
measuredFrameRate = [320, 240, 617, 617, 720, 480, 559, 559, 1280, 720, 276, 276, 1920, 1080, 164, 164, 3840, 2160, 30, 30];
bitRateMode = [];
minFrameRate = 0;
maxFrameRate = 0;
}
HDF_video_hw_dec_mpeg2_rk {
role = 0xFFFFFFFF;
type = 0;
name = "OMX.rk.video_decoder.m2v";
supportProfiles = [0, 3, 1, 3];
maxInst = 6;
isSoftwareCodec = false;
processModeMask = [];
capsMask = [0x01];
minBitRate = 1;
maxBitRate = 10000000;
minWidth = 176;
minHeight = 144;
maxWidth = 1920;
maxHeight = 1088;
widthAlignment = 8;
heightAlignment = 8;
minBlockCount = 0xFFFFFFFF;
maxBlockCount = 0xFFFFFFFF;
minBlocksPerSecond = 1;
maxBlocksPerSecond = 244800;
blockSizeWidth = 16;
blockSizeHeight = 8;
supportPixelFmts = [24];
measuredFrameRate = [];
bitRateMode = [];
minFrameRate = 0;
maxFrameRate = 0;
}
HDF_video_hw_dec_v8p_rk {
role = 0xFFFFFFFF;
type = 0;
name = "OMX.rk.video_decoder.vp8";
supportProfiles = [];
maxInst = 6;
isSoftwareCodec = false;
processModeMask = [];
capsMask = [0x01];
minBitRate = 1;
maxBitRate = 10000000;
minWidth = 176;
minHeight = 144;
maxWidth = 1920;
maxHeight = 1088;
widthAlignment = 8;
heightAlignment = 8;
minBlockCount = 0xFFFFFFFF;
maxBlockCount = 0xFFFFFFFF;
minBlocksPerSecond = 1;
maxBlocksPerSecond = 244800;
blockSizeWidth = 16;
blockSizeHeight = 16;
supportPixelFmts = [24];
measuredFrameRate = [320, 180, 500, 500, 640, 360, 387, 387, 1280, 720, 112, 112, 1920, 1080, 77, 77];
bitRateMode = [];
minFrameRate = 0;
maxFrameRate = 0;
}
HDF_video_hw_dec_h263_rk {
role = 0xFFFFFFFF;
type = 0;
name = "OMX.rk.video_decoder.h263";
supportProfiles = [1, 1, 1, 2, 1, 4, 1, 16, 8, 1, 8, 2, 8, 4, 8, 16];
maxInst = 6;
isSoftwareCodec = false;
processModeMask = [];
capsMask = [0x01];
minBitRate = 1;
maxBitRate = 10000000;
minWidth = 176;
minHeight = 144;
maxWidth = 1920;
maxHeight = 1088;
widthAlignment = 8;
heightAlignment = 8;
minBlockCount = 0xFFFFFFFF;
maxBlockCount = 0xFFFFFFFF;
minBlocksPerSecond = 1;
maxBlocksPerSecond = 244800;
blockSizeWidth = 16;
blockSizeHeight = 16;
supportPixelFmts = [24];
measuredFrameRate = [176, 144, 600, 600, 352, 288, 600, 600];
bitRateMode = [];
minFrameRate = 0;
maxFrameRate = 0;
}
HDF_video_hw_dec_m4v_rk {
role = 3;
type = 0;
name = "OMX.rk.video_decoder.m4v";
supportProfiles = [1, 1, 1, 2, 1, 4, 1, 8, 1, 16];
maxInst = 6;
isSoftwareCodec = false;
processModeMask = [];
capsMask = [0x01];
minBitRate = 1;
maxBitRate = 10000000;
minWidth = 176;
minHeight = 144;
maxWidth = 1920;
maxHeight = 1088;
widthAlignment = 8;
heightAlignment = 8;
minBlockCount = 0xFFFFFFFF;
maxBlockCount = 0xFFFFFFFF;
minBlocksPerSecond = 1;
maxBlocksPerSecond = 244800;
blockSizeWidth = 16;
blockSizeHeight = 16;
supportPixelFmts = [24];
measuredFrameRate = [176, 144, 600, 600];
bitRateMode = [];
minFrameRate = 0;
maxFrameRate = 0;
}
HDF_video_hw_dec_flv_rk {
role = 0xFFFFFFFF;
type = 0;
name = "OMX.rk.video_decoder.flv1";
supportProfiles = [];
maxInst = 6;
isSoftwareCodec = false;
processModeMask = [];
capsMask = [0x01];
minBitRate = 1;
maxBitRate = 10000000;
minWidth = 176;
minHeight = 144;
maxWidth = 1920;
maxHeight = 1088;
widthAlignment = 8;
heightAlignment = 8;
minBlockCount = 0xFFFFFFFF;
maxBlockCount = 0xFFFFFFFF;
minBlocksPerSecond = 1;
maxBlocksPerSecond = 244800;
blockSizeWidth = 16;
blockSizeHeight = 16;
supportPixelFmts = [24];
measuredFrameRate = [];
bitRateMode = [];
minFrameRate = 0;
maxFrameRate = 0;
}
HDF_video_hw_dec_mjpeg_rk {
role = 0;
type = 0;
name = "OMX.rk.video_decoder.mjpeg";
supportProfiles = [];
maxInst = 6;
isSoftwareCodec = false;
processModeMask = [];
capsMask = [0x01];
minBitRate = 1;
maxBitRate = 10000000;
minWidth = 176;
minHeight = 144;
maxWidth = 1920;
maxHeight = 1088;
widthAlignment = 8;
heightAlignment = 8;
minBlockCount = 0xFFFFFFFF;
maxBlockCount = 0xFFFFFFFF;
minBlocksPerSecond = 1;
maxBlocksPerSecond = 244800;
blockSizeWidth = 16;
blockSizeHeight = 16;
supportPixelFmts = [24];
measuredFrameRate = [];
bitRateMode = [];
minFrameRate = 0;
maxFrameRate = 0;
}
HDF_video_hw_dec_hevc_rk {
role = 2;
type = 0;
name = "OMX.rk.video_decoder.hevc";
supportProfiles = [1, 1, 1, 4, 1, 16, 1, 64, 1, 256, 1, 1024, 1, 4096, 1, 16384, 1, 65536, 2, 65536];
maxInst = 6;
isSoftwareCodec = false;
processModeMask = [];
capsMask = [0x01];
minBitRate = 1;
maxBitRate = 160000000;
minWidth = 176;
minHeight = 144;
maxWidth = 1920;
maxHeight = 1088;
widthAlignment = 2;
heightAlignment = 2;
minBlockCount = 0xFFFFFFFF;
maxBlockCount = 0xFFFFFFFF;
minBlocksPerSecond = 1;
maxBlocksPerSecond = 244800;
blockSizeWidth = 16;
blockSizeHeight = 16;
supportPixelFmts = [24];
measuredFrameRate = [352, 288, 700, 700, 720, 480, 700, 700, 640, 360, 980, 980, 1280, 720, 600, 600, 1920, 1080, 130, 130, 3840, 2160, 130, 130];
bitRateMode = [];
minFrameRate = 0;
maxFrameRate = 0;
}
}
VideoSwEncoders {
}
VideoSwDecoders {
}
AudioHwEncoders {
}
AudioHwDecoders {
}
AudioSwEncoders {
}
AudioSwDecoders {
}
}
}
```
## 适配验证
* 当系统启动后，拉起codec\_host进程，日志中有加载hcs配置的component相关组件的log，则初步判定适配过程正常。
* 当前系统通过gstreamer插件实现视频编解码功能。当未实现硬件编解码时，默认使用FFmpeg软件编解码。
codec hdi插件加载过程如下：
///foundation/multimedia/player\_framework/services/engine/gstreamer/plugins/codec/hdi\_plugins/hdi\_init.cpp
```
void HdiInit::AddHdiCap(CodecCompCapability &hdiCap)
{
MEDIA_LOGI("Add codec name %{public}s", hdiCap.compName);
CapabilityData codecCap;
codecCap.codecName = hdiCap.compName;
codecCap.codecType = GetCodecType(hdiCap.type);
codecCap.mimeType = GetCodecMime(hdiCap.role);
codecCap.isVendor = !hdiCap.isSoftwareCodec;
codecCap.alignment = {hdiCap.port.video.whAlignment.widthAlignment, hdiCap.port.video.whAlignment.heightAlignment};
codecCap.bitrateMode = GetBitrateMode(hdiCap.port.video);
codecCap.width = {hdiCap.port.video.minSize.width, hdiCap.port.video.maxSize.width};
codecCap.height = {hdiCap.port.video.minSize.height, hdiCap.port.video.maxSize.height};
codecCap.bitrate = {hdiCap.bitRate.min, hdiCap.bitRate.max};
codecCap.frameRate = {hdiCap.port.video.frameRate.min, hdiCap.port.video.frameRate.max};
codecCap.format = GetCodecFormats(hdiCap.port.video);
codecCap.blockPerFrame = {hdiCap.port.video.blockCount.min, hdiCap.port.video.blockCount.max};
codecCap.blockPerSecond = {hdiCap.port.video.blocksPerSecond.min, hdiCap.port.video.blocksPerSecond.max};
codecCap.blockSize = {hdiCap.port.video.blockSize.width, hdiCap.port.video.blockSize.height};
codecCap.measuredFrameRate = GetMeasuredFrameRate(hdiCap.port.video);
codecCap.profileLevelsMap = GetCodecProfileLevels(hdiCap);
capabilitys_.push_back(codecCap);
}
```
[/md]
欢迎光临 OpenHarmony开发者论坛 (https://forums.openharmony.cn/)
Powered by Discuz! X3.5