maryjane
/
syncevolution
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Testing: add 'known failure' support in nightly (MBC#2870) 

When a failure is known and it should not be treated as
a 'failed' case. Instead, 'knownfailure' is used to indicate
it. Thus, a new testing status for each case is shown in
the nightly testing outputs.
This commit is contained in:
Zhu, Yongsheng 2010-06-11 09:50:21 +08:00 committed by Patrick Ohly
parent ebe8078624
commit deee96a368
2 changed files with 15 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
test/generate-html.xsl
View File

@ -234,6 +234,7 @@
<th width="60">Categories</th>
<th width="60">Total Cases</th>
<th width="60">Passed</th>
<th width="60">Known Failure</th>
<th width="60">Failed</th>
<th width="60">Skipped</th>
<th width="60">Passrate</th>
@ -248,6 +249,7 @@
<xsl:variable name="categories" select="$item/*"/>
<xsl:variable name="all" select="count($unit-cases) * count($categories)"/>
<xsl:variable name="passed" select="count($item/*/*[.='okay'])"/>
<xsl:variable name="knownfailure" select="count($item/*/*[.='knownfailure'])"/>
<xsl:variable name="failed" select="count($item/*/*[.='failed'])"/>
<xsl:variable name="status" select="$cmp-result/*[name(.)=name($item)]/@summary"/>
<xsl:variable name="real-regression" select="$cmp-result/*[name(.)=name($item)]/*/*[.=$regression]"/>
@ -279,6 +281,9 @@
<td>
<xsl:value-of select="$passed"/>
</td>
<td>
<xsl:value-of select="$knownfailure"/>
</td>
<td>
<xsl:value-of select="$failed"/>
</td>
@ -422,6 +427,7 @@
<th width="60">Sources</th>
<th width="60">Total Cases</th>
<th width="60">Passed</th>
<th width="60">Known Failure</th>
<th width="60">Failed</th>
<th width="60">Skipped</th>
<th width="60">Passrate</th>
@ -440,6 +446,7 @@
<xsl:variable name="item" select="."/>
<xsl:variable name="all" select="count($item/*)"/>
<xsl:variable name="passed" select="count($item/*/*[.='okay'])"/>
<xsl:variable name="knownfailure" select="count($item/*/*[.='knownfailure'])"/>
<xsl:variable name="failed" select="count($item/*/*[.='failed'])"/>
<xsl:variable name="status" select="$cmp-result/*[name(.)=name($item)]/@summary"/>
<xsl:variable name="valgrind" select="$item/@result"/>
@ -459,6 +466,9 @@
<td>
<xsl:value-of select="$passed"/>
</td>
<td>
<xsl:value-of select="$knownfailure"/>
</td>
<td>
<xsl:value-of select="$total - $passed"/>
</td>

7
test/resultchecker.py
View File

@ -254,8 +254,11 @@ def step2(resultdir, result, servers, indents, srcdir, shellprefix, backenddir):
casename = case.rpartition('_')[2].partition('.')[0]
result.write(indent+'<'+casename+'>')
match=format+'::'+casename
match=match+": okay \*\*\*"
if(os.system("grep -q '" + match + "' "+case)):
matchOk=match+": okay \*\*\*"
matchKnownFailure=match+": \*\*\* failure ignored \*\*\*"
if(os.system("grep -q '" + matchKnownFailure + "' "+case)==0):
result.write('knownfailure')
elif(os.system("grep -q '" + matchOk + "' "+case)):
result.write('failed')
else:
result.write('okay')