Had UID conflicts between VTODO and VEVENT. Now they all use @dummy
as tail in their UID and mangleICalendar20 ensures that the item type
is embedded there. Could have done it without mangling, but it seemed
safer that way.
Removed some redundant IGNORE entries (aren't enabled in the first
place) and told client-test to use refresh sync - helps to avoid
slow syncs and the 417 'retry later' throttling.
client-test avoided "refresh-from-server" in favor of delete+slow sync
because the Funambol SyncML did a literal "refresh-from-server" sync.
libsynthesis is smarter and does the delete+slow sync or literal
refresh sync, depending on the configuration for the peer.
Changed because Funambol's OneMedia throttles clients with 417 which
ask for slow syncs.
Added INFO output about checking sources. This helps with WebDAV when
the server cannot be contacted (dead, misconfigured) because otherwise
there would be no indication at all why the --configure operation
seems to hang.
Here is some example output, including aborting:
$ syncevolution --configure --template webdav \
syncURL=http://192.168.1.100:9000/ \
username=foo password=bar retryDuration=2s \
target-config@webdav-temp
[INFO] creating configuration target-config@webdav-temp
[INFO] addressbook: looking for databases...
[INFO] addressbook: no database to synchronize
[INFO] calendar: looking for databases...
[INFO] calendar: no database to synchronize
[INFO] memo: looking for databases...
[INFO] memo: no database to synchronize
[INFO] todo: looking for databases...
[INFO] todo: no database to synchronize
It timed out fairly quickly here because of the retryDuration=2s. That
also gets placed in the resulting config, which is probably not desired.
$ syncevolution --configure \
--template webdav \
syncURL=http://192.168.1.100:9000/ \
username=foo password=bar \
target-config@webdav-temp
[INFO] creating configuration target-config@webdav-temp
[INFO] addressbook: looking for databases...
^C[INFO] Asking to suspend...
[INFO] Press CTRL-C again quickly (within 2s) to stop immediately (can cause problems in the future!)
^C[INFO] Aborting immediately ...
[ERROR] error code from SyncEvolution aborted on behalf of user (local, status 20017): aborting as requested by user
It would be good to make the CTRL-C handling code aware that it can
abort immediately instead of doing the intermediate "asking to suspend"
step, which only makes sense for sync sessions.
Also added task and memo sources to the WebDAV template.
Both requires changes to the testing. Instead of updating the C++
CmdlineTest, the affected tests were removed and only the
corresponding Python tests were updated.
A --configure operation checks whether backends are usable before
enabling them. This can take a while or fail for every single source
after a long timeout, in particular when the backend needs to contact
a remote server and that server was not configured correctly or is
dead (WebDAV!).
Now the command line checks for abort requests in the loop over
sources.
The message resend loop in the WebDAV backend did not check abort
requests. Now the error which caused the resend is reported
immediately in case of an abort. The actual send/receive still cannot
be interrupted, as it happens inside libneon with little chances to
influence the execution (?).
When waiting for resending a failed message, the sleeping couldn't be
interrupted when using the D-Bus server. It now uses the SuspendFlags
infrastructure and glib, which detects abort requests sent via D-Bus
in addition to those sent via signals (which already worked earlier).
When a memo in VJOURNAL format from the CalDAV backend gets converted
to plain text *and* the description starts with summary plus newline,
then only the description is used in the plain text version, to avoid
duplicating the summary.
Such entries are produced by SyncEvolution when importing plain text
into EDS. The conversion in the CalDAV backend uses a slightly
different logic and strips the first line from the incoming text
instead of just copying it.
The main reason for this change is that the new mechanism (based
libsynthesis text profile) makes it easier to implement the conversion
that way.
The TestFileNotify.testSession test failed when using GIO D-Bus: when
Session.Detach() caused the shutdown, the shutdown happened before
returning the method call reply to the caller.
Fixed by moving the g_main_loop_quit() into a timeout callback with
zero delay. This ensures (in practice - not sure whether it is
guaranteed) that it will get invoked by the main event loop after
processing the current events.
This covers Client::Source testing (including the new
testLinkedSources) and (with DAViCal) also syncing.
For VJOURNAL a server is needed which supports it. Should
set up Radicale.
The "text/calendar" type now also contains rules for parsing and
generating VJOURNAL in iCalendar 2.0 and vCalendar 1.0. When using
"text/calendar+plain", the internal format is iCalendar 2.0 with plain
text used as alternative if the sync peer does not support VJOURNAL
for memos.
These types can be used as "databaseFormat" of the file backend and by
other backends which want to replace plain text as exchange format for
memos.
Somehow syncevo-local-sync got stuck waiting for its parents reply to
a D-Bus method call when the parent had already quit. The expected
error response did not occur in this case.
Fixed by watching whether the parent is around while waiting for the
response. Also got rid of the event loop start/stop in favor of
g_main_context_iteration(). That is more explicit about the reason for
stopping the waiting.
CalDAV VJOURNAL and VTODO need similar UID handling as VCARD: when
creating or updating an item, ensure that it has a UID in the
payload.
It may also help to match that with the resource name and use
.ics/.vcf as suffix of the resource name. It was necessary for Yahoo's
CardDAV support and doesn't hurt elsewhere.
When updating, the merging with local data now ensures that any UID
different from the resource is properly preserved when the incoming
data has no UID.
All backends except for EDS replaced local data wholesale with an
incoming update, even if that update came from a peer which did not
store all properties. The EDS backend had already been configured
earlier to always merge remote and local data before writing it back.
Instead of updating each backend individually it makes more sense to
make the more intelligent (and more expensive) merging the default in
backends derived from SyncSourceSerialize/TrackingSyncSource. Backends
which do not want it can remove
"<updateallfields>true</updateallfields>" from
SynthesisInfo::m_datastoreOptions.
The new backend property values "CalDAVTodo" and "CalDAVJournal"
select tasks resp. memos stored in a CalDAV collection. "CalDAV"
continues to select events.
Events, tasks and journals can be mixed in the same resource (=
URL). However, this is less efficient than storing them separately.
A good CalDAV server allows filtering items by type, and SyncEvolution
uses that. However, it was found that Radicale 0.7 ignores this
filtering, which could have led to data loss (SyncEvolution asks for
all VTODOs in preparation for a "delete all items" operation in a
"CalDAVTodo" source, gets also VJOURNALs, then deletes them).
Therefore SyncEvolution plays it safe and downloads the VTODO and
VJOURNAL data to double-check that it is working on the right items.
This causes additional traffic for well-behaving servers; currently
it cannot be turned off.
What is missing for VJOURNAL is the conversion to plain text (see BMC
not possible yet.
Backends are now allowed to create RegisterSyncSourceTest instances
with empty name. They will be ignored except that their init() method
will be called after main() and before the list of source tests is
used.
This can be used to create additional RegisterSyncSourceTest instances
at a time when the test of libsyncevolution is initialized. This is
needed by the WebDAV backend, which has to instantiate sources and use
the config system. That crashed when done before the ConfigProperty
instances were initialized.
The WebDAV backend must support different kinds of items in the same
collection. The new testLinkedSources covers this by adding, updating
and deleting an item of one kind and checking that other sources
referencing the same database do not see these changes.
This test must be activated for a specific source by adding links to
other sources using the same database. A separate commit will do that
for WebDAV.
Radicale reports 200 'Okay' instead of 204 when asked to delete an
existing item. Was triggering a status error. Found when running
Client::Source test against Radicale 0.7-7-g186c59c; 0.7 still
returned 204, see http://redmine.kozea.fr/issues/802.
Radicale reports 412 'Preconditiona Failed' instead of 404 when asked
to delete a non-existent item. Translate that into the 404 expected by
the Synthesis engine. Found when running Client::Source test against
Radicale 0.7-7-g186c59c.
Radicale reports 410 'Gone' instead of 404 when asked to read a
non-existent item. Translate that into the 404 expected by the
Synthesis engine. Found when running Client::Source test against
Radicale 0.7-7-g186c59c.
Google does not implement refresh-from-server, therefore using it has
to be made configurable. It is enabled by default for Funambol and
disabled for everything else.
Existing user configs must be updated to use refresh-from-server with
Funambol:
syncevolution --configure enableRefreshSync=1 funambol
libsynthesis has traditionally implemented "refresh-from-server" as
"delete local data" plus "slow" sync. This is more compatible, because
some servers did not support "refresh-from-server".
But it has the downside that the server cannot know that the client
won't send any data, and Funambol now only allows one slow sync before
blocking the next one for a certain period of time. Probably this is
done to prevent excessive resource usage by badly behaving clients.
SyncEvolution now uses a new configure option in libsynthesis which
enables the use of "refresh-from-server". This is set unconditionally;
all modern SyncML servers are expected to support it.
libsynthesis console is logged with a "SYSYNC: " prefix for each line
if (and only if) SYNCEVOLUTION_DEBUG is set to a value of 4 or higher.
Showing it by default is not useful (because it can be quite verbose,
in particular when a server triggers decoding error messages in SyncML
TK, like Funambol) and using the normal loglevel doesn't help because
that loglevel is not available outside of sync sessions.
The previous commit broke parallel "make deb" (because mv *.deb
matched different package builds and then mv only succeeded for one)
and "make rpm" (because the mv was looking in the wrong place).
The packages unintentionally contained README.rst, README.rst.patched
and README files in a second /usr/share/doc directory.
They were picked up by checkinstall because they happened to be in the
current directory. Fixed by preparing running checkinstall inside
a "checkinstall" directory which contains just the content needed
for packaging.
Using a config name which refers to a context now triggers
a specific error. The operation that cannot be continued
is named explicitly:
$ ./syncevolution @default
[INFO] Configuration "@default" does not refer to a sync peer.
[ERROR] Cannot proceed with sync without a configuration.
The error message about missing configuration also became
better:
$ ./syncevolution foo@default
[INFO] Configuration "foo@default" does not exist.
[ERROR] Cannot proceed with sync without a configuration.
Running a sync in "dry run" mode is not supported. The error
message about that was reported like an internal error.
Also added tests covering such output.
Due to the way how properties were handled internally, it wasn't
possible to explicitly set a property to its default value. Instead
the property was unset. For example, explicitly setting database= was
not possible.
This is necessary for client-test and ActiveSync, because client-test
needs to know that the testing is expected to run with the default
databases (something which normally is avoided by overwriting empty
database properties).
Now the "is set" state is tracked explicitly in the config storage and
command line property APIs. Unsetting a property via the command line
could be implemented with an explicit command line option, but is not
supported at the moment.
Tests were extended to cover the new functionality and adapted to the
change behavior for "type" migration: syncFormat was empty
already (because the empty string matched the default), but
forceSyncFormat was unnecessarily set explicitly. Now it is not.
Commit 006bcf26 introduced a more versatile replacement for
FileConfigNode, but didn't switch over old code to the replacement
because of a code freeze. Switching now.
Also use the hash variant in VolatileConfigNode, because it is more
efficient by dropping the (in that case irrelevant) order of
properties.
"database", "databaseUser", "databasePassword" must not be set by
client-test when explicitly set to empty. That is necessary for
ActiveSync, where database= means "use default database" and
explicitly selecting that database via --print-database is not
currently supported.
Google doesn't seem to support the Fetch operation, which is used
during testing to retrieve unchanged items. During syncing it will
only be needed when merging incoming data with an existing item, which
should not be necessary... except that testing shows that it is
necessary. The case where it is needed is:
- incremental sync (cache empty)
- calendar event series needs to be modified
- items from that series need to be fetched in preparation
for updating it
To allow testing, several workarounds are necessary:
- request *all* data before doing a data dump in testImport (and friends),
to ensure that the cache in the backend is fully populated
- use the cache in ActiveSyncCalendarSource instead of accessing
the base class directly, because that would trigger a Fetch
Accessing the base class is still useful for Exchange+calendar,
because testing then avoids the cache (and thus can expose bugs
in it).
While at it, adapted the README. "database" needs to be set
explicitly, "client-test" only does it when creating configs.
Fixed ESourceList leak. Was never freed?! Now assigned to a new GObject
C++ wrapper as owner.
client API: Use EvolutionSyncSource::throwError() consistently, because it
relates the error to the source (nicer).
client API: fixed "only-if-exists" issue for contacts.
client API: fixed leak of uid after creating calendar object.
The Qt headers define "signal", which breaks GNOME header
files if those get included later. Rearranged header
files to avoid ugly "undefine" statements.
Found when using EDS client API header files, because those
include different GNOME headers.
e_cal_client_tzlookup() doesn't detect that lookup
fails because of non-existant time zone, because it gets
a non-descriptive error code from EDS ("Invalid object").
Work around that by ignoring all errors in our own lookup
function.
See "[Evolution-hackers] e_cal_client_check_timezones() + e_cal_client_tzlookup() + Could not retrieve calendar time zone: Invalid object"
Tino Keitel
Tino Keitel
Tino Keitel
Patrick Ohly