Suppress the following differences:
- SEQUENCE removed
- X-MEEGO-ACTIVESYNCD-MeetingStatus/NativeBodyType added
- ORGANIZER added to all events, ignored for events without ATTENDEEs
Improved valgrindcheck.sh so that if VALGRIND_LEAK_CHECK_ONLY_FIRST
is set, memory leaks in forked processes are filtered out of the
valgrind log file.
Also improved the detection whether an error was reported. This is
the case if and only if a suppression was created. This allows
running valgrind with normal output, no need for --quiet anymore.
Memotoo replaces the UTC UNTIL time with floating time, just
as it changes the DTSTART/DTEND. Let's ignore that by using
test data with that UNTIL clause for Memotoo.
An invalid syncFormat (like text/vcard for "Evolution Calendar")
can prevent instantiating the source. Therefore include that
value in the error message and clearly distinguish between
sync and database format.
Sony Ericsson seems to use ISO-8859-1 for all their phones. This
causes two problems:
- mangled characters and/or
- crashes inside libecal/gdbus
It is uncertain whether all devices have this problem. A poll did not
yield any results
(http://syncevolution.org/blogs/pohly/2011/question-sony-ericsson-users-charset). So
let's change it for those who have reported problems.
To revert the change, copy
/usr/share/syncevolution/xml/remoterules/server/00_sony_ericsson.xml
into $HOME/.config/syncevolution-xml/remoterules/server (after
creating that directory) and remove the lines with ISO-8859-1.
This was originally reported for contacts, and now also for calendar
data. The calendar case was seen as a crash of the syncevo-dbus-server:
GLib-CRITICAL **: g_variant_new_string: assertion `g_utf8_validate (string, -1, NULL)' failed
Program received signal SIGSEGV, Segmentation fault.
g_variant_is_trusted (value=0x0)
at /build/buildd/glib2.0-2.28.6/./glib/gvariant-core.c:600
in /build/buildd/glib2.0-2.28.6/./glib/gvariant-core.c
Thread 1 (Thread 0xb7fce850 (LWP 3402)):
at /build/buildd/glib2.0-2.28.6/./glib/gvariant-core.c:600
at /build/buildd/glib2.0-2.28.6/./glib/gvariant.c:3081
at /build/buildd/glib2.0-2.28.6/./glib/gvariant.c:4093
app=0xbfffca4c) at /build/buildd/glib2.0-2.28.6/./glib/gvariant.c:4248
at /build/buildd/glib2.0-2.28.6/./glib/gvariant.c:4188
in_calobj=0x8435fe8 "BEGIN:VEVENT\r\nSUMMARY:THW Sim Pr\374fung\r\nDTSTART:20070420T230000Z\r\nDTEND:20070421T225900Z\r\nBEGIN:VALARM\r\nTRIGGER;VALUE=DURATION:PT45M\r\nACTION:DISPLAY\r\nEND:VALARM\r\nEND:VEVENT\r\n",
Evolution shows a meeting twice on the day of a modified recurrence,
if the meeting series was originally created and modified in Exchange,
then imported into Google Calendar.
The reason is that the RECURRENCE-ID in that case ends up being
in UTC, even if the parent event has a time zone. Evolution and/or
libical seem to have a bug here, IMHO they should recognize that the
RECURRENCE-ID time is the same.
SyncEvolution now works around this by transforming the UTC
RECURRENCE-ID time into the time zone of the DTSTART time of the
parent event. This is combined with removal of X-LIC-ERRORS into
a new fixIncomingCalendar() method which needs to be called
whenever a complete VCALENDAR is received from the CalDAV server
(adding data from a report, GET).
This transformation changes the "rid" part of the item IDs. It should
be okay in a two-way sync (remove one sub item, add another) and
a slow sync (libsynthesis recognizes the times as equal).
Added some more functions needed to work with DTSTART and RECURRENCE-ID.
Removed icalproperty_vanew_lastmodified() because its entry was incomplete
and unused.
ShutdownProcess() has a race condition between calling popen.poll()
and os.kill(): the process might quit in the meantime. The check
also was missing for the initial kill().
Better check first and continue when kill() fails because the
process was not found.
Nightly testing did not resend a PUT when it could (should?)
have because the deadline was exceeded. Added logging to
track down what the chosen deadline is.
Instead of aborting the test at the first item which
fails to import, collect the error messages (from the
exceptions), do the comparison of imported data
against the reference data and then either report
the import failures (if there were any) or the comparison
failure (if that broke).
The advantage is that a single run of the test shows
all problems that exist.
An empty argument (like the server list in tests where no server
was involved) must be quoted, too, otherwise it gets dropped.
Broke parsing of some test results.
When sudo is necessary to enter the target platform, environment
variables no longer can be sent via the environment because
sudo ignores those. They must be be put onto the command line
of sudo, which then puts them into then environment of the invoked
process (depends on SETENV: permission in /etc/sudoers!).
This commit reorders the command line so that sudo comes
first, followed by env variables.
Use checkPassword() before trying to --print-items. This will either
prompt for the password at command line, or ask it from the keyring
if --keyring is specified (or if using the dbus server).
Nightly testing got stuck in test-dbus.py when syncevo-dbus-server
somehow didn't shut down properly after a D-Bus test ran: test-dbus.py
must have been in subprocess.communicate() after sending a SIGTERM,
but syncevo-dbus-server did not shut down.
Now such a situation is detected by (semi-)busy polling for process
completion and sending a SIGKILL after 5 seconds. A timeout in
subprocess.communicate() would have been nicer, but doesn't seem to
be supported by Python.
Client::Source::*::testLinkedItems* tested local storage operations
for events sharing the same UID. For syncing, only testItems had
a little bit like it. This commit adds more tests specifically
designed to exercise corner cases of syncing such linked items:
- sync parent, then sync with child and other event added on different
sides (corner case in CalDAV change tracking, BMC #22329)
- sync child
- sync child, sync parent later
Trigger "item already deleted" in syncing, check that syncing
completes and leaves no items around.
Somehow the problem in BMC #22327 must have been different: when
running the test, the Synthesis engine recognizes that the item
was removed on both sides and logs:
So this test doesn't really cover BMC 22327. But it is worthwile
to run it anyway.
Google Calendar checks whether a CalDAV client is allowed to update
particular events. In combination with meeting invitations this has
led to know issues where desirable changes were rejected (see
http://code.google.com/p/google-caldav-issues/issues/detail?id=38).
Details are murky whether that bug is still open. I hit the "403 You
don't have access to change that event." problem both in updating
a meeting series and creating it.
This commit ensures that when updating fails, the errors is treated as
a temporary, per item error (417). The sync session then continues.
The overall result will be STATUS_PARTIAL_FAILURE = 22001 and the
next session will retry the same item. This is better than aborting
the session (situation without this patch) or ignoring the problem
(alternative solution).
The same error is not handled when creation fails. This might need
further investigations.
During an incremental sync, when an unmodified meeting series on the CalDAV
server had to be extended (= adding a detached recurrence), the operation failed
with "event not found".
The root cause was a bug in updateAllSubItems(): due to a copy-and-paste bug,
it cleared the cache instead of adding the modified items to it. Therefore
unmodified items, added to the cache earlier, where not found later on.
Normally they weren't needed. The exception is reading in preparation
for adding a detached recurrences.
Fixed by removing the m_cache.clear(). That is valid in this case even
if the operation is repeated, because adding already read items will
simply overwrite them.
Also added some debug logging which helped to track this down.
If the same event was deleted both locally and in the CalDAV server, syncing
failed with "event not found".
It is normal that the Synthesis engine requests the removal of
non-existent items. CalDAVSource needs to handle that in
removeSubItem() and getSubDescription().
Item operations like --print-items failed if the configuration didn't exist:
[ERROR] : virtual read-only configuration node, cannot write property webDAVCredentialsOkay = 1
This was caused by trying to write the webDAVCredentialsOkay property
into a temporary, read-only configuration. The fix is to check for
read-only configs before attempting to use the property.
Alternatively it would have been possible to catch exceptions, but it
is not obvious which errors can be ignored.
Some users of ConfigNode (like the WebDAV backend) need to know
whether setting a optional property will succeed before attempting
it. When they know that it fails because the node is read-only, they
can skip setting it.
The alternative would be to throw a well-defined exception and catch
it, but that is not in line with the SyncEvolution design. Exceptions
should be for real, unexpected errors.
Notifications were meant to be shown for all errors except temporary
ones (http://bugzilla.moblin.org/web/bug_report_10000.html). This has
never been implemented correctly since the feature was introduced:
instead of hiding known temporary errors, all errors except 500 (fatal
error) were suppressed.
This commit switches to white listing the known errors which are
temporary and suppresses those. Right now that happens to be only one,
network problems.
Use the new READ() script method in libsynthesis to inline local photo
data right before sending to a remote peer.
Tests were added in combination with SyncEvolution server for inlining
a special well-known file (testcases/local.png) and failing to inline
(file doesn't exist). In the latter case the URI is sent unchanged.
The VALUE parameter was ignored completely and thus got lost in the
Synthesis engine. Added it as a string.
The TYPE parameter was an incomplete enum. Better allow arbitrary
strings.
Both parameters must be copied together with the PHOTO data they
belong to. In combination with merge="fillempty" this can be
problematic: if one of the parameters is empty, it may be overwritten
although the PHOTO data is not copied.
This problem is solved by ensuring that the internal field list never
has empty PHOTO_TYPE/VALUE fields. This is done by setting "binary"
resp. "unknown" when importing contact data from a peer or the local
Evolution backend and removing them again before sending to a peer or
storing.
The same change needs to be done with other backends. It is not made
mandatory because some backends (like file) might want to store these
values explicitly.
Because it is unknown which peers support VALUE=uri, only the
CLIENT_TEST_SERVER=syncevolution testItem testcases contain a contact
with such a PHOTO.
The PHOTO property was removed completely for Memotoo because the
literal comparison used to fail due to transcoding the images. Now
synccompare keeps the PHOTO property and only removes the data itself.
This was done in order to test PHOTO;VALUE=uri. It turned out that
Memotoo does not support those.
The auto sync tests failed when run in combination with other tests.
The root cause turned out to be side effects of these other tests,
but hardening the SessionChanged signal checking is useful anyway.
The TestDBusServerPresence tests have to invoke syncevo-dbus-server with
DBUS_TEST_CONNMAN set in the environment, to get it to use the mock
implementation on the session D-Bus.
They did this by modifying os.environment *permanently*. That had the
effect that all later tests also ended up using the mock ConnMan. This
made testing confusing because tests failed only when run in combination
with the presence tests (fixed previously by improving the mock ConnMan
implementation).
This commit addresses the root cause by putting the env variable only
temporarily into the environment of those tests which really need it.
When one test requested signals for D-Bus server 'org.syncevolution' and
then didn't consume them before completing, these signals were delivered
to the next test because the "dbus" instance is shared. When that
next test happens to listen for the same signals, it could get confused.
Happened with testAutoSync* because TestMultipleConfigs did not consume
all SessionChanged signals.
Instead of forcing all tests to consume all signals (might be too
restrictive and definitely was error-prone), ensure that each test
only processes the signals emitted by its own private syncevo-dbus-server
instance. This is achieved by requesting the signals for that server's
bus name instead of the generic 'org.syncevolution' name.
Ensure that all output is unbuffered by adding the -u command line
option.
Create separate D-Bus and syncevo-dbus-server log files
for each test, with the name derived from the current class and
method. str(self) gives that without depending on internals.
Add environment and arguments to syncevo-dbus-server log. Make
detection of "server is running" more resilient.
When neither Network Manager nor ConnMan are running, network presence was "not
online". This prevented running automatic syncs. Primarily (exclusively?) affected
nightly testing in the minimal chroots.
The solution is to detect when both managers were not found on D-Bus and
then set the status to "online".
Mangling the VJOURNAL test data by prepending "U " to the DESCRIPTION
caused problems because the first line has to match the SUMMARY (in
plain text, only the body is sent and SUMMARY is taken from the first
line).
This commit fixes these problems by also inserting the "U " into the
SUMMARY of memos.
Google Calendar replaces explicit DESCRIPTIONs in a VALARM with
a generic "This is an event reminder" and adds it when there
is no DESCRIPTION.
Let's ignore that while testing by setting that Google default text in
the Google testItem test data.
The Google server seems to think that it can take over meeting
acceptance/decline handling when new events are stored on the server.
For example, PARTSTAT=ACCEPTED gets replaced with PARTSTAT=NEEDS-ACTION.
It has also been observed that participants and organizer get overwritten.
This needs further investigation, see BMC #19484. In the meantime I am
removing the problematic test case to get the testItems test to pass.
Same as commit 789550 for testImport.
Google Calendar modifies the RRULE of a recurring event. The
transformation is semantically correct, but breaks the strict
comparison in the Client::Sync::eds_event::testItems test.
Fixed by making the same change in the reference test data for Google
Calendar.
Google Calendar recognizes the "custom US time zone" despite using
different names for TZID and TZNAME. Probably uses the offsets. When
it recognizes the zone, it replaces it with its own internal zone,
which breaks the test.
That's okay, so let's stop using this particular test event to get the
test to pass.
Google Calendar converts UTC time stamps into the time zone of the
calendar as set when the event is stored in the calendar. This changes
the semantic of the event slight, in particular for recurring events,
but is acceptable and is done correctly for the event's point in time
by Google, so let's accept this change.
This commit adds the Europe/Berlin time zone to the test data used for
CLIENT_TEST_SERVER=googlecalendar Client::Sync::eds_event
so that the change above no longer occurs in the testing and thus doesn't
break the test.
Because a time zone is set explicitly and preserved by Google, the Google
Calendar time zone setting doesn't matter for testing.
The Client::Source::eas_event::testChanges test failed incorrectly
because an ActiveSyncSource was created before deleting the previous
one. That breaks change tracking because the previous instance must
flush its in-memory item list first.
Adding an explicit source.reset() fixes the problem.
Evolution and the MeeGo UX assume that first/middle/last name are set.
That is not the case when a contact is created in the Google Contacts
web interface. Such contacts are sent by Google without the N
property. That is surprising because Google supports the N property
(it is returned for contacts created via SyncML, and internally Google
distinguishes between the name components).
This commit adds a workaround for all peers (not just Google) which
ensures that first/middle/last name are set whenever possible. If FN
is set, it is split so that the first and last word are first/last
name, the rest is the middle name. As a special case a trailing comma
at the end of the first words is recognized as "Doe, John Middle"
format.
If the full name is empty, the first email address and (if that is also
empty) the first phone number are used as additional fallbacks for the
first name.
