Commit graph

65 commits

Author SHA1 Message Date
Patrick Ohly d741059975 runtest.py: check Synthesis source
We check out the Synthesis source first, then the SyncEvolution source.
That let's us use SyncEvolution's autogen.sh/gen-autotools.sh scripts
for checking whether both sources are clean and properly tagged.

For Synthesis, we have to set the SYNTHESISSRC env variable.
2010-03-16 13:24:20 +01:00
Patrick Ohly 73195a730b testing: insert valgrind before syncevolution (MB #6457)
Previously, the valgrind command was inserted in front of "env",
which always succeeded because child processes were not checked
(would include EDS). Now we run exactly the right process under
valgrind.
2010-01-22 16:34:54 +01:00
Chen Congwu 837c0429c8 Testing: Fix no html report error (MB#8391)
Do not write 'parameter' property, as the value sometimes fails the XML parser.
Generate a default HTML report if the report failed to generate.
2010-01-22 16:34:54 +01:00
Patrick Ohly 210a2656f7 nightly testing: git fetch fixes
Instead of fetching a specific repo, use just "git fetch" to
pull updates into an existing repo. Specifying the repo
should have worked, but apparently is not done correctly (not
yet?) by the git in Debian Stable.

Using "git fetch" assumes that the URL of the repo does not
change between updates. That should be the case for us.
2009-12-01 21:45:34 +01:00
tester 2d0da511f0 test/runtest.py: git checkout debugging
Added both 'set -x' and 'git show-ref' to get more information
because it seems that sometimes the latest source is not checked
out.
2009-12-01 21:45:34 +01:00
Patrick Ohly 70329670bf runtests.py: run setup command twice, just to be sure
In a setup where it was run inside schroot we got error
messages from schroot because gconfd got stuck and prevented
a clean unmount of /tmp. Even "kill -9" wouldn't get rid of
such a hanging gconfd.

As a workaround, running the setup command twice seems to
succeed, including getting rid of the handing gconfd.
2009-12-01 21:45:34 +01:00
Zhu, Yongsheng ca5a4d76f4 Testing: clean log/test files before a new testing
Previously do cleanup operation after implementing
a syncevolution setup. But if this setup raises
exceptions, we might get incorrect test results.
so clean all these files before doing any other
preparations
2009-12-01 21:45:34 +01:00
Patrick Ohly 653c1986b9 runtest.py: fixed checking out branch from git
Checking out a branch which only exists remotely requires
a bit more work. Checking out tags might also have failed
due to an unnecessary "git pull". Now "git tag" and "git branch"
are used to check what the requested revision is and do the
necessary opertions.
2009-12-01 21:45:33 +01:00
Patrick Ohly 8e6bfc24e5 runtest.py: point prebuilt binaries to correct location of backends
The "prebuilt" test ran, but because it used the build directory
of the current platform it didn't find any backends, or the wrong
ones. Fixed by only using the backend setting from the previous
testing if the directory really exists and falling back to the
relative "backends" directory otherwise.

It might be possible to always use "backends". In order to not
change the running testing this wasn't done.
2009-12-01 21:45:33 +01:00
Zhu, Yongsheng 280e67bddc Testing: refine return code checkings(bug#6457)
when memory leaks occur, the return code should be '100'
but python on unix will re-encode it and put them in a high byte
in a 16-bit number.
2009-12-01 21:45:33 +01:00
Zhu, Yongsheng 46a3ed30ee Mobical: skip unit test testOneWayFromClient(MB#8121)
Mobical has a known server issue that it initiates a
slow sync when one client syncs with it in one-way-from-client
mode from the second time.

Because Mobical has no schedule on this, we skip this
test for Mobical interoperability test.
2009-12-01 21:42:41 +01:00
Congwu Chen 6d2be17e6f runtests.py: added configuration of Resend test
This change was originally committed a while back to the
runtest.py used in the nightly testing. Now we are merging
it back to the public runtest.py.

This patch adds the "Resend" test to various _SKIP and
_FAILURES lists and adds CLIENT_TEST_RESEND_TIMEOUT and
CLIENT_TEST_INTERRUPT_AT.
2009-10-16 14:25:32 +02:00
Patrick Ohly 00cb014971 runtests.py: reformatted source code to split up long lines
The long and unreadable environment strings for client-test
are now in the Python code as multiple independent strings
which get concatenated by Python:
function("foo"
         "bar")
2009-10-16 14:25:32 +02:00
Patrick Ohly ddaf76ec1b runtests.py: build RPMs in addition to DEBs
checkinstall *does* work on a Debian system, even under fakeroot.
The trick is to make /usr/src/rpm/RPMS writable for normal users.
The Makefile then copies the resulting .rpm back to the current
working directory.
2009-10-16 14:25:32 +02:00
Patrick Ohly 1cffe8fe6f runtests.py: fixed "prebuilt" tests
The invocation of the Client::Source tests for prebuilts binaries
was incorrect, the new source arraty was missing.
2009-10-16 14:25:32 +02:00
Patrick Ohly fd40f4f87a runtests.py: fixed building lpia .deb
The "lpia" (Low Power Intel Architecture) .debs are our normal
x86 binaries packaged with a different architecture name. The check
whether runtests.py was building for x86 broke when moving to an
64 bit bost. It must be run inside the build environment.
2009-10-16 14:25:32 +02:00
Zhu, Yongsheng a2cd73595b Testing: add nightly test for memotoo
Memotoo is ready to test so enable it
2009-10-16 14:25:31 +02:00
Chen Congwu a2b079298e Testing: add time info for mail title
add time info for mail title for the purpose
of clear report
2009-10-16 14:25:31 +02:00
Chen Congwu d1e9e24721 Testing: change know failures to skip for google testing
Change CLIENT_TEST_FAIURES to CLIENT_TEST_SKIP for google
testing
2009-10-16 14:25:31 +02:00
Chen Congwu 06fd773fd9 Testing: add valgrind check result and backend scan dir
1)Get return code from valgrind check result
2)Add environment variable SYNCEVOLUION_BACKEND_DIR when starting testing
2009-10-16 14:25:31 +02:00
Chen Congwu 950e9251c2 Testing: add valgrind check result and backend scan dir
Change failure to skip for google known failures
Change report title to have more acurate time information
2009-10-16 14:25:31 +02:00
Chen Congwu 015e1d3d9c Testing: a fix to avoid null servers list if compile error 2009-08-27 10:21:58 +02:00
Zhu, Yongsheng b338591bae Testing: added Mobical.net in nightly test
Mobical.net supports 4 kinds of PIMs, e.g.,
contacts, calendar, task and memo.
Enable them in nightly test
2009-08-24 15:18:56 +02:00
Zhu, Yongsheng c234306473 The main purpose is to refine nightly test report.
1) extract test results from many result files and combine them
into a xml format file to summary each unit test status
2) use 'compare.xsl' to compare two days' result and generate
comparison result
3) generate more visiable html output file by nightly result
and comparison result to make tracking regressions easily
2009-08-24 15:18:47 +02:00
Patrick Ohly 9c8eb81c45 testing: removed incorrect CLIENT_TEST_FAILURES=EvolutionCalendarTest::testOpenDefaultMemo
First, the test really shouldn't fail. Need to investigate why it does on
one test platform. Second, it was applied to all test runs, overwriting
the desired value of CLIENT_TEST_FAILURE for that run.
2009-07-24 17:00:50 +02:00
Patrick Ohly 89d1ae3b35 testing: don't compress .out files
These files used to contain the complete sync session
output. Now they contain just the user-visible output,
so don't compress anymore because that doesn't save much
space and makes the files harder to read.
2009-07-22 11:54:30 +02:00
Patrick Ohly b69430fb98 testing: added --sanity-checks mode
In this mode, Client::Sync is replaced with
just Client::Sync::<first source>::testItems.
This assumes that all servers support WBXML
with our test data.
2009-07-22 10:53:01 +02:00
Patrick Ohly 275e561d02 testing: copy log directory in addition to stdout
.log.gz is the stdout of a complete test. It covers
multiple sync sessions, which are in the corresponding
directories.
2009-07-22 10:53:01 +02:00
Patrick Ohly 07d526945c ScheduleWorld testing: ignore text::testManyItems failure (Bugzilla #4596)
The server doesn't do twinning during a slow sync correctly,
thus duplicating all text items.
2009-07-21 11:09:59 +02:00
Patrick Ohly a7edbb3cd8 testing: added Google contact sync testing
Only contacts in vCard 2.1 are tested. Several
failures are ignored because Google does not support
them:
* refresh from client
* one-way from client
* XML format

These tests are run although we know that they
fail, just in case that at some point they start
to work.
2009-07-21 11:06:49 +02:00
Patrick Ohly d630535cfd testing: added ZYB.com for contacts
Calendar sync would also work, but is no longer officially supported, thus not enabling it.
2009-07-20 15:30:11 +02:00
Patrick Ohly 2daf66bf38 testing: removed ScheduleWorld failure suppression for new clients
vcard30::testCopy used to fail because adding new clients forced
unexpected slow syncs on all other clients. This should be fixed
now, so the FAILURES entry could be removed.
2009-07-20 15:30:11 +02:00
Patrick Ohly 98df00d558 testing: fixed typo in recent commit
The local "evolution" test failed because of a typo
in the Python function call: comma instead of %.
2009-07-15 17:03:33 +02:00
Patrick Ohly a47d7ff1d2 testing: allow parallel makes
Added --make-command to runtests.py: can be used to ask for a parallel
make.
2009-07-15 17:03:33 +02:00
Patrick Ohly 3c2bf7ea49 testing: workaround for running client-test with --enable-shared --disable-static
For weird reasons that I don't have the patience to analyze running
the client-test wrapper script as generated by libtool fails
to set up the search path for libsyncevolution.so. It works
for syncevolution.

As a workaround let's simply set LD_LIBRARY_PATH ourselves, regardless
whether it is needed.
2009-07-15 17:03:32 +02:00
Patrick Ohly 76e190ebaf testing: avoid running Retry and Suspend tests for ScheduleWorld and Funambol
These tests are both slow and tend to fail with these servers
because of deficiencies in the servers. Disabled these tests
as discussed in Bugzilla #4282.
2009-07-14 10:02:16 +02:00
Patrick Ohly 2dbb186ab2 testing: run distcheck with building GUI
distcheck is useful to ensure that all files are distributed,
the result builds and cleans up properly. Doing this with
building of the GUI is important to avoid surprises during
packaging.
2009-07-14 10:02:16 +02:00
Patrick Ohly 8c3157d93f testing: ignore EvolutionCalendarTest::testOpenDefaultMemo failure
This is currently broken because opening the default system
addressbook depends on e_cal_new_system_memos(), which is
not available in all Evolution versions.

It could be added, but first we should sort out issue #3732.
2009-07-13 15:30:56 +02:00
Patrick Ohly 53573ba030 testing: enable ical20 and itodo20 for Funambol testing 2009-07-13 15:29:22 +02:00
Patrick Ohly 269983003a runtests.py: updated to work with current source code
Added support for git checkout. Replaced Funambol with
Synthesis. Added --mailhost option.
2009-06-26 08:30:09 +02:00
Patrick Ohly 9bbd6c92ac on x86 also build lpia .deb
git-svn-id: https://zeitsenke.de/svn/SyncEvolution/trunk@772 15ad00c4-1369-45f4-8270-35d70d36bdcd
2008-12-07 20:02:14 +00:00
Patrick Ohly d93c67c2de checking out client library is optional now (done automatically by newer SyncEvolution configure script)
git-svn-id: https://zeitsenke.de/svn/SyncEvolution/trunk@746 15ad00c4-1369-45f4-8270-35d70d36bdcd
2008-10-08 20:53:58 +00:00
Patrick Ohly 14a59ea744 speed up testing by reducing the default delay in Evolution calendar/task/text sync sources
git-svn-id: https://zeitsenke.de/svn/SyncEvolution/trunk@737 15ad00c4-1369-45f4-8270-35d70d36bdcd
2008-08-26 16:15:35 +00:00
Patrick Ohly bb23c15f26 only invoke client-test once; suppress known SW 'new client' problem; test 'text' source with Funambol
git-svn-id: https://zeitsenke.de/svn/SyncEvolution/trunk@727 15ad00c4-1369-45f4-8270-35d70d36bdcd
2008-08-17 08:19:45 +00:00
Patrick Ohly 26633a6a27 added the possibility to run a setup script as part of each test
git-svn-id: https://zeitsenke.de/svn/SyncEvolution/trunk@662 15ad00c4-1369-45f4-8270-35d70d36bdcd
2008-07-10 18:09:36 +00:00
Patrick Ohly f3c0b7c2ba added MemoToo
git-svn-id: https://zeitsenke.de/svn/SyncEvolution/trunk@647 15ad00c4-1369-45f4-8270-35d70d36bdcd
2008-06-29 17:59:27 +00:00
Patrick Ohly e08bed66cb added --prebuilt: testing with precompiled SyncEvolution
git-svn-id: https://zeitsenke.de/svn/SyncEvolution/trunk@636 15ad00c4-1369-45f4-8270-35d70d36bdcd
2008-06-26 20:29:56 +00:00
Patrick Ohly b35f44f984 also test SyncEvolution unit tests in addition to testing framework tests
git-svn-id: https://zeitsenke.de/svn/SyncEvolution/trunk@608 15ad00c4-1369-45f4-8270-35d70d36bdcd
2008-05-02 17:22:44 +00:00
Patrick Ohly b141163fd0 fixed rebuild of Evolution trunk: install stamp must be removed, otherwise it won't reinstall
git-svn-id: https://zeitsenke.de/svn/SyncEvolution/trunk@519 15ad00c4-1369-45f4-8270-35d70d36bdcd
2008-03-16 08:48:11 +00:00
Patrick Ohly c1c17699e5 check out SyncEvolution from Subversion
git-svn-id: https://zeitsenke.de/svn/SyncEvolution/trunk@502 15ad00c4-1369-45f4-8270-35d70d36bdcd
2008-02-13 22:28:53 +00:00