Occasionally in the nightly testing reading the existing akonadi.db
produced incomplete output which could not be used to create the
modified akonadi.db. This patch works around that by using pre-generated
akonadi.db.dump files as input if found.
"evolution" tests used to be a traditional name for all Client::Source
and SyncEvolution unit tests. This caused problems when adding
Akonadi (crash in Akonadi client lib when EDS was used before) and
was not nice in the test reporting (multiple different tests all mixed
into one summary line). It also prevented running these tests in parallel.
All of this gets fixed by splitting "evolution" into "eds", "kde",
"file" and "unittests". To avoid updating the nightly test run scripts,
--enable evolution=.... is still understood.
More recent distros (for example, Ubuntu Saucy) rely on
XDG_RUNTIME_DIR. Each time dbus-session.sh runs, it must
ensure that the runtime dir exists and is empty.
Not doing so may have caused random concurrency issues in
parallel nightly testing.
It also was a problem when trying to run activesyncd + SyncEvolution
on a headless Ubuntu Saucy server (see FDO #76273).
Better assume that the program may change directories. In that
case we need an absolute path for the output file.
Nightly testing failed once because valgrind was not able to
write its output file in the new directory. It remained unclear
why the directory was changed and to what, though. But that is
a different problem.
Nightly testing uses a "Client_Sync_Current" symlink to point
the server to a location where it shall write its per-test logs
without having to reconfigure the server.
When the server-side session runs longer than a specific test, it ends up
using the "Client_Sync_Current" link of the next test. "permission denied"
errors were also seen, although it is less certain whether that had the same
root cause.
To avoid such issues, the sync session now tries to resolve the log dir
path to an absolute path at the start of the session.
When spawning the dlt daemon under "unbuffer", a lingering tclsh process
exists before the test starts and the test ended up waiting for that to
stop.
We need to be more specific with identifying sync processes: only those
processes started as part of syncing matter.
Specifying the language of the email address did not make much sense
to start with, even if EDS does (did?) it that way originally. Akonadi
strips LANGUAGE=en. Instead of filtering that out for Akonadi, better
simplify the test data.
The KDE Notes resources store items under a different MIME type than the one
used in AKonadi (see "[Kde-pim] note format"). SyncEvolution use the same type
as Akonadi and thus did not find existing KDE Notes resources.
To support both while KDE and Akonadi transition to the same type,
SyncEvolution now looks for notes resources using both MIME types and accepts
both kinds of items when reading. When writing, SyncEvolution picks the MIME
type that is supported by the resource, which hopefully avoids confusing the
KDE app using the resource (untested).
As a positive side effect, the "database" value used for opening a resource is
now checked more thoroughly. Non-existent resources and the type mismatches
like pointing a "kde-contacts" backend to a calendar resource are now detected
early.
Akonadi resources do not enforce iCalendar 2.0 semantic like
"each VEVENT must have a UID" (see "[Kde-pim] iCalendar semantic").
When receiving an event from a peer which itself does not enforce
that semantic (Funambol, vCalendar 1.0 based phones), then we
need to generate a UID, otherwise KOrganizer will ignore the
imported event.
The tests depend on the source re-adding UID and/or RECURRENCE-ID during an
update. The Akonadi source and file source do not do that and thus we cannot
run the test against them.
Akonadi and Akonadi resources do not enforce iCalendar 2.0 semantic
like "UID+RECURRENCE-ID must be unique in a collection". Therefore we
must disable all tests expecting that behavior.
Regular syncing should work okay as long as the peer behaves. A misbehaving
peer will be able to send us invalid sets of items and syncing will not be
able to detect that because the sync engine itself is also agnostic to the
special iCalendar 2.0 semantic.
See "[Kde-pim] iCalendar semantic".
Under heavy load while testing, syncing occasionally failed with a D-Bus
message timeout, for example in the Close() call. Let's disable these timeouts
by explicitly passing something very large (infinite is not supported by the
Python bindings).
If HTTP processing takes to long, the client may resend, but in contrast to a
failed D-Bus method call that case is handled.
The increased timeout has no negative effect on detecting when
syncevo-dbus-server dies, because then the D-Bus daemon will notify
us immediately.
The previous solution left the "sleep 60" shell command running despite
killing the shell that spawned it. That lingering process caused schroot to
wait during session shutdown, which slowed down result checking considerably.
Now a single Perl process does the sleep and the killing. Killing that
process allows schroot to proceed immediately.
filekde covers running Akonadi in HTTP server mode, which was broken in
SyncEvolution 1.4 (see FDO 75672). kdekde and filefile get added for
the sake of completeness.
When cloning the home dir template, take care of adapting
some hard-coded paths in akonadiserverrc and the akonadi.db
used for sqlite. This only works when Akonadi was configured
to use sqlite as backend in the home dir template.
When used as storage in a server, Akonadi got called in a background thread
that gets created to handle slow initialization of sources and preventing
ensuing timeouts in HTTP clients (probably not needed for Akonadi itself,
but may still be useful when combining it with other sources).
Akonadi cannot be used like that, leading to false "Akonadi not running"
errors or (if one got past that check) failing item operations.
This commit avoids the situation by shifting the work back into the main
thread.
This is needed to move one-time operations from AkonadiSyncSource into the
main thread if (and only if) the source is used in a background
thread. Akonadi can't be used in a thread unless the thread was created with
QThread, which is not the case in SyncEvolution.
The basic functionality is the same as the older GRunWhile(). It just has to
be extended so that the action is guaranteed to be called only in the main
thread and there runs only once. This is achieved by turning the action
into a check which immediately returns false.
AkonadiSyncSource cannot include GLibSupport.h due to conflicting usage of
"signal" (a define in Qt, parameter name in glib), therefore these functions
are now made available via util.h.
Writing a test that ensures that libsynthesis gets the right
timezone was hard, because the original problem only occurs
when syncing a vCalendar 1.0 based device with SyncEvolution.
Emulating libsynthesis at least comes close.
libical 1.0 started to return VTIMEZONE definitions with multiple
absolute transition times instead of RRULEs. This causes problems when
exchanging data with peers (see
https://sourceforge.net/p/freeassociation/bugs/95/).
In SyncEvolution, this affected sending an event using New Zealand
time in vCalendar 1.0 format to a phone, because the internal,
out-dated definition of the time zone in libsynthesis was used as
fallback when loading RRULE-based timezone definitions from libical
failed (see "[SyncEvolution] Some events showing wrong time on
phone"). It might also affect exchanging data with CalDAV peers (not
tested).
The workaround is to include the original code from libical from
before the change in SyncEvolution and override
icaltimezone_get_component() with a version that uses the original
timezone loading code. This does not fix cases where other code causes
libical itself to load a timezone, but for libsynthesis it is good
enough because it does the loading early when no other code should
have used libical.
The downside is that now we need to maintain the RRULE heuristics and
ensure that they actually work. Copying libical/src/test/timezones.c
would be useful for that.
Long-term it would be good to enhance libical itself such that it can
return a VTIMEZONE with suitable RRULEs matching a specific event,
point in time or time range.
Check for icaltimezone_get_component() before actually loading libical.
That way our own code uses our own copy of that method from the copied
icaltz-util.c instead of always using the one from libical.
For that we must not overwriting already found functions.
gcc 4.4 complains about "num_types" being used potentially uninitialized.
This probably refers to bailing out before num_types gets set. However,
the use is conditional on "types", which is only set after "num_types",
so this is a false warning.
Prevent the warning anyway for the sake of clean -Wall -Werror compilation.
When EFREAD() does a "goto error", the code leaks the "r_trans"
memory. Found with cppcheck. Adding cleanup code fixes that.
To avoid a false cppcheck warning, the free() call must use the same
variable name as the calloc() assignment. It is a bit more readable
that way, too.
Can be compiled separately when linking against libical. For this
to work it must use header file locations as in libical client
programs.
icaltzutil_get_zone_directory() can come from either libical or
the icatz-util.c copy, depending on DISABLE_ICALTZUTIL_GET_ZONE_DIRECTORY.
We have to use singleton methods to ensure that the global std::string
instances are initialized before use. There was some non-determinism before
because SyncSource loading happens as part of the executable initialization,
too, and can cause a call to EDSAbiWrapperInit().
On some platforms, LDFLAGS is used during configure to set rpath
options. runtests.py must pass the -no-install to the linker
of client-test without overwriting those flags.
This fixes an issue where configure fails to find Akonadi when
test programs do not compile because QString is not found:
checking for Akonadi/Collection... no
configure: error: akonadi.pc not found. Install it to compile with the Akonadi backend enabled.
...
configure:21857: checking Akonadi/Collection presence
configure:21857: g++ -E -I/usr/include/ -I/usr/include//KDE -I/usr/include/qt4 conftest.cpp
In file included from /usr/include//KDE/Akonadi/../../akonadi/collection.h:25:0,
from /usr/include//KDE/Akonadi/Collection:1,
from conftest.cpp:44:
/usr/include/akonadi/entity.h:24:19: fatal error: QString: No such file or directory
#include <QString>
The libsynthesis_3.4.0.47+syncevolution-1-3-99-7-20140203 tag
accidentally matched the check for a non-exact-tag "git-describe"
output (-<number of changes>-g<hash>). Therefore the actual tags
weren't even checked.
Reversing the check such that we look at tags ourselves and proceed with
them if one matches avoids this problem.
The test randomly failed under load because the client's GetVersion
call did not make it to the server in time. It seems to work better
with a smaller delay.
We must use the temporary file that was created for us securily, not
a temp file named after that file. This caused a temp file vulnerability
and the real temporary files were not deleted by the script.
We need the right file suffix for the compiler. "mktemp --suffix"
would be useful, but it not yet available in Ubuntu Lucid. So instead
create a temp dir (which works in Ubuntu Lucid) and create files
inside that.
With DAViCal, the "EST/EDT" VTIMEZONE gets renamed by e_cal_check_timezones()
because DAViCal reorders properties and thus breaking the simple string
comparison even though the semantic of the timezone did not change.
If this happens multiple times (for example, in testTwinning), the test
failed because the " 1" suffix was only stripped once. We also need to
ignore multiple repeated suffices (" 1 1") because e_cal_check_timezones()
is not intelligent enough to bump an existing suffix.
Long term, it would be better to teach e_cal_check_timezones() how to
normalize timezones before the comparison and thus avoiding the duplicated
timezones in the first place.
The test relied on fixed timeouts for a) killing syncevo-dbus-server
and b) completion of the sync. The second timeout sometimes occurred
to soon, causing the testNoParent itself to fail and also the following
test (because log files from testNoParent were not removed).
Now be more careful about really killing in the middle of the sync (thanks
to delaying the sync and not the helper startup) and really wait for process
termination (with getChildren() and os.kill() checking).
Google CalDAV server does not handle \\\n properly. It adds
an extra backslash. Avoid this aspect of the test case because
a fix on the server side has been slow in coming.
eds_event.ics.googlecalendar.tem.patch is used with
Client::Sync::eds_event and Google CalDAV as server. google_event.ics
is used for Client::Source::google_caldav.
Patrick Ohly
Niels Ole Salscheider
Tino Mettler