submitting_target居然可以这样

今天有个为 FFmpeg 贡献高质量 patch 的朋友问到了一个问题，如何编写一个 FATE 测试用例，当

 

今天有个为 FFmpeg 贡献高质量 patch 的朋友问到了一个问题，如何编写一个 FATE 测试用例，当时简单地回复了操作步骤，那么接下来我将会在这里详细介绍一下如何编写 FFmpeg 的自动化测试用例。

一. 测试用例结果参考数据说明首先看一下测试用例参考数据内容大概的展现形态bogon:ffmpeg_up liuqi$ cat tests/ref/fate/hls-segment-size#tb 0:

1/44100#media_type 0: audio#codec_id 0: pcm_s16le#sample_rate 0: 44100#channel_layout 0: 4#channel_layout_name

0: mono0, 0, 0, 1152, 2304, 0x907cb7fa0, 1152, 1152, 1152,

2304, 0xb8dc75250, 2304, 2304, 1152, 2304, 0x3e7d69050, 3456, 3456,

1152, 2304, 0xef47877b0, 4608, 4608, 1152, 2304, 0xfe916b7e0, 5760,

5760, 1152, 2304, 0xe3d08cde0, 6912, 6912, 1152, 2304, 0xff7f86cf0,

8064, 8064, 1152, 2304, 0x843e6f950, 9216, 9216, 1152, 2304, 0x81577c26

0, 10368, 10368, 1152, 2304, 0x04a085d50, 11520, 11520, 1152, 2304

, 0x1c5a76f50, 12672, 12672, 1152, 2304, 0x4ee786230, 13824, 13824, 1152

, 2304, 0x8ec861dc0, 14976, 14976, 1152, 2304, 0x0ca179d80, 16128, 16128

, 1152, 2304, 0xc6da750f0, 17280, 17280, 1152, 2304, 0xf6bf79b50, 18432

, 18432, 1152, 2304, 0x97b88a430, 19584, 19584, 1152, 2304, 0xf13c7b9c

0, 20736, 20736, 1152, 2304, 0xdfba83af0, 21888, 21888, 1152, 2304

, 0xc9467d4b0, 23040, 23040, 1152, 2304, 0xbbb58e2b0, 24192, 24192, 1152

, 2304, 0x3a1078ea0, 25344, 25344, 1152, 2304, 0xe9587a5c此后省略好几百行0, 880128,

880128, 1152, 2304, 0x3cb185300, 881280, 881280, 1152, 2304, 0x5a0c5e7bbogon:ffmpeg_up liuqi$

从内容上看，这是一个音频数据的测试文件，timebase是1/44100，采样率是44100，媒体类型是音频，音频格式是pcm_s16le，音频通道布局是mono然后下面就是每一帧音频数据的输出信息，一共 6 列，分别代表，。

当前 packet 的流所在的索引就是 packet->stream_index当前 packet 的 dts当前 packet 的 pts当前 packet 的 duration当前 packet 的数据包大小

当前数据包的 crc 值二. 实际操作那么这一堆数据是怎么生成的呢？下面从FATE中找到一个样例看一下tests/data/hls_segment_size.m3u8: TAG = GENtests/data/hls_segment_size.m3u8: ffmpeg

$(PROGSSUF)$(EXESUF) | tests/data$(M)$(TARGET_EXEC)$(TARGET_PATH)/$< \ -f lavfi -i "aevalsrc=cos(2*PI*t)*sin(2*PI*(440+4*t)*t):d=20"

-f hls -hls_segment_size 300000 -map 0 \ -hls_list_size 0 -codec:a mp2fixed -hls_segment_filename $(TARGET_PATH)

/tests/data/hls_segment_size_%d.ts \$(TARGET_PATH)/tests/data/hls_segment_size.m3u8 2>/dev/nullFATE_AFILTER-

$(call ALLYES, HLS_DEMUXER MPEGTS_MUXER MPEGTS_DEMUXER AEVALSRC_FILTER LAVFI_INDEV MP2FIXED_ENCODER) += fate-hls-segment-size

fate-hls-segment-size: tests/data/hls_segment_size.m3u8fate-hls-segment-size: CMD = framecrc -flags +bitexact -i

$(TARGET_PATH)/tests/data/hls_segment_size.m3u8 -vf setpts=N*23从例子中可以看到，首先是生成一个m3u8列表，也就是这一段tests/data/hls_segment_size.m3u8: TAG = GEN

tests/data/hls_segment_size.m3u8: ffmpeg$(PROGSSUF)$(EXESUF) | tests/data$(M)$(TARGET_EXEC)$(TARGET_PATH)

/$< \ -f lavfi -i "aevalsrc=cos(2*PI*t)*sin(2*PI*(440+4*t)*t):d=20" -f hls -hls_segment_size 300000 -map 0 \

-hls_list_size 0 -codec:a mp2fixed -hls_segment_filename $(TARGET_PATH)/tests/data/hls_segment_size_%d.ts \

$(TARGET_PATH)/tests/data/hls_segment_size.m3u8这一段最终执行时也就是这样的/Users/liuqi/project/ffmpeg_up/dash/ffmpeg \

-f lavfi -i "aevalsrc=cos(2*PI*t)*sin(2*PI*(440+4*t)*t):d=20" -f hls -hls_segment_size 300000 -map

0 \ -hls_list_size 0 -codec:a mp2fixed -hls_segment_filename /Users/liuqi/project/ffmpeg_up/dash/tests/

data/hls_segment_size_%d.ts \ /Users/liuqi/project/ffmpeg_up/dash/tests/data/hls_segment_size.m3u8 2

>/dev/null然后是下面那一段生成framecrc数据的操作FATE_AFILTER-$(call ALLYES, HLS_DEMUXER MPEGTS_MUXER MPEGTS_DEMUXER AEVALSRC_FILTER LAVFI_INDEV MP2FIXED_ENCODER) += fate-hls-

segment-sizefate-hls-segment-size: tests/data/hls_segment_size.m3u8fate-hls-segment-size: CMD = framecrc -flags +bitexact -i $(TARGET_PATH)/tests/

data/hls_segment_size.m3u8 -vf setpts=N*23内容也比较好理解，首先是执行一句tests/data/hls_segment_size.m3u8，在make fate-hls-segment-size的时候首先会执行tests/data/hls_segment_size.m3u8，然后会执行这一句

CMD = framecrc -flags +bitexact -i $(TARGET_PATH)/tests/data/hls_segment_size.m3u8 -vf setpts=N*23这里的framecrc是一个shell封装函数，这个函数是封装在tests/fate-run.sh文件中

124 ffmpeg(){125 dec_opts="-hwaccel $hwaccel -threads $threads -thread_type $thread_type"126 ffmpeg_args=

"-nostdin -nostats -cpuflags $cpuflags"127 for arg in$@; do128 [ x${arg} = x-i ] && ffmpeg_args=

"${ffmpeg_args}${dec_opts}"129 ffmpeg_args="${ffmpeg_args}${arg}"130 done131 run ffmpeg

${PROGSUF}${EXECSUF}${ffmpeg_args}132 }133134 framecrc(){135 ffmpeg "$@" -bitexact -f framecrc -136 }

137138 ffmetadata(){139 ffmpeg "$@" -bitexact -f ffmetadata -140 }141142 framemd5(){143 ffmpeg

"$@" -bitexact -f framemd5 -144 }145146 crc(){147 ffmpeg "$@" -f crc -148 }149150 md5pipe(){151 ffmpeg

"$@" md5:152 }153154 md5(){155 encfile="${outdir}/${test}.out"156 cleanfiles="$cleanfiles$encfile

"157 ffmpeg "$@"$encfile158 do_md5sum $encfile | awk {print $1}159 }最后拼出来的fate执行的效果大概是这样的/Users/liuqi/project/ffmpeg_up/dash/ffmpeg -nostdin -nostats -cpuflags all -flags +bitexact -hwaccel none -threads

1 -thread_type frame+slice -i /Users/liuqi/project/ffmpeg_up/dash/tests/data/hls_segment_size.m3u8 -vf setpts=N*

23 -bitexact -f framecrc -然后将最后一个字符-替换成自己的文件名即可，那么这里可以替换成tests/ref/fate/hls-segment-size，也就基本上大功告成了然后剩下的FATE相关信息，可以参考FFmpeg的官方文档：

FFmpeg Automated Testing EnvironmentTable of Contents1 Introduction2 Using FATE from your FFmpeg source directory

3 Submitting the results to the FFmpeg result aggregation server4 Uploading new samples to the fate suite

5 FATE makefile targets and variables5.1 Makefile targets5.2 Makefile variables5.3 Examples1 Introduction

FATE is an extended regression suite on the client-side and a means for results aggregation and presentation on the server-side.

The first part of this document explains how you can use FATE from your FFmpeg sourcedirectorytotest your ffmpeg binary. The

second part describes how you can run FATE to submit the results to FFmpeg’s FATE server.Inany way you can have a look

at the publicly viewable FATE results by visiting this website:http://fate.ffmpeg.org/This is especially recommended

forall people contributing source code to FFmpeg, as it can be seen ifsometestonsome platform broke with

their recent contribution. This usually happens on the platforms the developers could nottest on.The

second part of this document describes how you can run FATE to submit your results to FFmpeg’s FATE server.

If you want to submit your results be sure tocheck that your combination of CPU, OS and compiler isnot

already listed on the above mentioned website.In the third part you can find a comprehensive listing

of FATE makefile targets and variables.2Using FATE from your FFmpeg sourcedirectoryIf you want to run FATE

on your machine you need to have the samples in place. You can get the samples via the build target fate-rsync.

Use this command from the top-levelsourcedirectory:make fate-rsync SAMPLES=fate-suite/make fate SAMPLES=fate-suite/

The above commands set the samples location bypassing a makefile variable via command line. It is also possible

toset the samples location atsource configuration timeby invoking configure with--samples=. Afterwards you can invoke the makefile targets without setting the SAMPLES makefile variable. This is illustrated by the following commands:

./configure --samples=fate-suite/make fate-rsyncmake fateYet another way to tell FATE about the location

of the sampledirectoryisby making sure the environment variable FATE_SAMPLES contains the pathto your samples directory. This can be achieved

by e.g. putting that variablein your shell profile orby setting it in your interactive session.FATE_SAMPLES=fate-suite/ make fate

Donot put a ’~’ characterin the samples pathto indicate a home directory. Because of shell nuances, this will cause FATE

to fail.Touse a custom wrapper to run the test, pass --target-exec to configure or set the TARGET_EXEC Make variable.

3 Submitting the results to the FFmpeg result aggregation serverTo submit your results to the server you should run fate

through the shell script tests/fate.sh from the FFmpeg sources. This script needs to be invoked with a configuration

fileas its first argument.tests/fate.sh /path/to/fate_configA configuration filetemplatewith comments describing the individual configuration

variables can be foundat doc/fate_config.sh.template.The mentioned configuration templateis also available here:

slot= # some unique identifierrepo=git://source.ffmpeg.org/ffmpeg.git

# the source repository#branch=release/2.6 # the branch to testsamples=

# path to samples directoryworkdir= # directory in which to do all the work

#fate_recv="ssh -T fate@fate.ffmpeg.org" # command to submit reportcomment=

# optional descriptionbuild_only= # set to "yes" for a compile-only instance that skips testsignore_tests=

# the following are optional and map to configure optionsarch=cpu=cross_prefix=as=cc=ld=target_os=sysroot=

target_exec=target_path=target_samples=extra_cflags=extra_ldflags=extra_libs=extra_conf= # extra configure options not covered above

#make= # name of GNU make if not makemakeopts= # extra options passed to make#makeopts_fate= # extra options passed to make when running tests,

# defaulting to makeopts above if this is not set#tar= # command to create a tar archive from its arguments on stdout,

# defaults to tar cCreate a configuration that suits your needs, based on the configuration template. The slot configuration

variable can be anystring that isnot yet used, but it is suggested that you name it adhering to the following

pattern ‘arch-os-compiler-compiler version’. The configuration file itself will be sourced in a shell script, therefore

all shell features may be used. This enables you to setup the environment as you need it for your build.

For your firsttest runs the fate_recv variable should be emptyor commented out. This will run everything

asnormalexcept that it will omit the submission of the results to the server. The following files should be

presentin $workdir as specified in the configuration file:configure.logcompile.logtest.logreportversion

When you have everything working properly you can create an SSH key pair and send the publickeyto the FATE

server administrator who can be contacted at the email address fate-admin@ffmpeg.org.Configure your SSH

clienttousepublickeyauthenticationwith that keywhen connecting to the FATE server. Also donot forget

tocheck the identityof the serverandtoaccept its host key. This can usually be achieved by running your SSH

client manually and killing it after you accepted the key. The FATE server’s fingerprint is:‘RSA’d3:f1:

83:97:a4:75:2b:a6:fb:d6:e8:aa:81:93:97:51‘ECDSA’76:9f:68:32:04:1e:d5:d4:ec:47:3f:dc:fc:18:17:86If you have problems connecting

to the FATE server, it may helpto try out the ssh command with one or more -v options. You should get

detailed output concerning your SSH configuration and the authentication process.The only thing left

isto automate the execution of the fate.sh script and the synchronisation of the samples directory.4 Uploading

new samples to the fate suiteIf you need a sample uploaded send a mail to samples-request.This isfor developers who have an

accounton the fate suite server. If you upload new samples, please make sure they areas small as possible,

spaceoneachclient, network bandwidth and so on benefit from smaller test cases. Also keepin mind older checkouts

use existing sample files, that means in practice generally donotreplace, remove or overwrite files as

it likely would break older checkouts or releases. Also all needed samples for a commit should be uploaded, ideally

24hours, before the push. If you need an accountfor frequently uploading samples or you wish tohelp others

by doing that send a mail to ffmpeg-devel.#First update your local samples copy:rsync -vauL --chmod=Dg+s,Duo+x,ug+rw,o+r,o-w,+X fate-suite.ffmpeg.org:/home/samples/fate-suite/ ~/fate-suite

#Then do a dry run checking what would be uploaded:rsync -vanL --no-g --chmod=Dg+s,Duo+x,ug+rw,o+r,o-w,+X ~/fate-suite/ fate-suite.ffmpeg.org:/home/samples/fate-suite

#Upload the files:rsync -vaL --no-g --chmod=Dg+s,Duo+x,ug+rw,o+r,o-w,+X ~/fate-suite/ fate-suite.ffmpeg.org:/home/samples/fate-suite

5 FATE makefile targets andvariables5.1 Makefile targetsfate-rsyncDownload/synchronize sample files to

the configured samples directory.fate-listWill listall fate/regression test targets.fateRun the FATE

test suite (requires the fate-suite dataset).5.2 Makefile variablesVVerbosity level, can be setto0, 1

or2.0: show just the test arguments1: show just the command used in the test2: show everythingSAMPLES

Specify or override the pathto the FATE samples at make time, it has a meaning onlywhile running the regression tests.

THREADSSpecify how many threads tousewhile running regression tests, it is quite useful to detect thread

-related regressions.THREAD_TYPESpecify which threading strategy test, either ‘slice’ or ‘frame’, bydefault

‘slice+frame’CPUFLAGSSpecify CPU flags.TARGET_EXECSpecify or override the wrapper used to run the tests. The TARGET_EXEC

option provides a way to run FATE wrappedin valgrind, qemu-useror wine oron remote targets through ssh.

GENSetto ‘1’ to generate the missingor mismatched references.HWACCELSpecify which hardware acceleration

tousewhile running regression tests, bydefault ‘none’ is used.KEEPSetto ‘1’ tokeep temp files generated

by fate test(s) whentestis successful. Defaultis ‘0’, which removes these files. Files arealways kept

when a test fails.5.3 Examplesmake V=1 SAMPLES=/var/fate/samples THREADS=2 CPUFLAGS=mmx fate三 结语其实要看过程的话，make 自己的tag就可以了，例如这样

bogon:dash liuqi$ make fate-hls-segment-size V=1/Users/liuqi/project/ffmpeg_up/dash/ffmpeg \ -f lavfi -i

"aevalsrc=cos(2*PI*t)*sin(2*PI*(440+4*t)*t):d=20" -f hls -hls_segment_size 300000 -map 0 \ -hls_list_size

0 -codec:a mp2fixed -hls_segment_filename /Users/liuqi/project/ffmpeg_up/dash/tests/data/hls_segment_size_%d.ts \

/Users/liuqi/project/ffmpeg_up/dash/tests/data/hls_segment_size.m3u8 2>/dev/nullTEST hls-segment-size

src/tests/fate-run.sh fate-hls-segment-size """""/Users/liuqi/project/ffmpeg_up/dash"framecrc -flags +bitexact -i /Users/liuqi/project/ffmpeg_up/dash/tests/data/hls_segment_size.m3u8 -vf setpts=N*23

1 /Users/liuqi/project/ffmpeg_up/dash/ffmpeg -nostdin -nostats -cpuflags all -flags +bitexact -hwaccel none -threads

1 -thread_type frame+slice -i /Users/liuqi/project/ffmpeg_up/dash/tests/data/hls_segment_size.m3u8 -vf setpts=N*

23 -bitexact -f framecrc -bogon:dash liuqi$读这篇内容需要一些基础：Makefile 基本原理和用法bash 脚本基本用法