and allow this UID to be used for selecting a particular calendar.
Since all listed calendars are in the default storage anyway,
the UID is far more useful to have. (On the N9, knowing the
physical storage does not help the user at all anyway, as access
to it is restricted and needs to go through the API anyway.)
(cherry picked from commit a5c2939c1d)
Some unnamed version of KDE crashes in KApplication when invoked
without a D-Bus session. The reporter ran into this when compiling
from source, because the SyncEvolution binary is invoked as part of
the build process, which ran outside of a D-Bus session.
Avoid the crash by checking for a D-Bus session bus with
QDBusConnection::sessionBus().isConnected() before instantiating
KApplication. The QDBusConnection API does not say explicitly when it
connects to the daemon, but testing shows that in practice this
detects missing env variables and an unreachable daemon right away as
expected, while passing when the daemon can be contacted.
Instantiating KApplication was added for KWallet support. Without
D-Bus, KWallet does not work either, therefore throw an explicit error
when the lack of D-Bus was detected.
When compiled against EDS 3.5.x or later, SyncEvolution now uses
the backend code originally written for the EClient API introduced
in EDS 3.2. That code was changed so that it works with the new
include file rules and ESourceRegistry in EDS 3.5.x. Support
for using the EClient API with EDS 3.4 was removed because maintaining
three different flavors of the EDS backend code would be too much
work and not gain much (just the possibility to test the EDSClient
code with 3.4).
At the moment, this is a compile time choice made automatically
by configure. syncevolution.org binaries are compiled against
an older EDS and thus do not work with EDS 3.5.x or later.
EDS 3.5.x handles authentication itself, using a standard system
prompt if necessary. SyncEvolution can no longer provide the password,
and thus the "databaseUser/Password" options have no effect when using
EDS 3.5.x.
The patch leaves code for older EDS almost completely unchanged and
therefore is considered safe for the stable release series leading to
1.3. Using EClient is an all-or-nothing choice now, because the common
EvolutionSyncSource needs to be compiled differently. Thanks to the
reorganized API, a lot more common code for ECal and EBook sources
could be moved into EvolutionSyncSource.
Instantiating an ActiveSyncSource for testing must work with
empty database names now. Testing no longer forces the
database to be set.
While at it rewrote the code to avoid the explicit pointer.
A backend cannot know whether it'll be used together
with a client-test which supports integration testing.
Therefore the backend should always register its tests.
Updating failed when using Google because the Synthesis engine
tried to read the existing item in order to merge it with
the update. This failed because Google does not implement the
Fetch command.
Pretending to update the item intelligently avoids that. It
also helps to improve performance of updates with Exchange.
The downside is that syncing with local storages which do
not support all ActiveSync fields will cause data loss.
Need to check whether Exchange-only attributes get lost
also when the local storage supports everything, for
example because activesynd unintentionally removes data.
The error message for "Invalid synchronization key" changed in
activesyncd, now it contains the D-Bus error type as prefix. Fixed by
doing a substring search.
Also, not freeing the GError before trying again is a bug. Apparently
that was ignored earlier, now it triggers an assert.
As mentioned by Tino Keitel on the mailing list, some libs and
executables were only implicitly linked against libraries that they
called directly. This happened to work by chance because these libraries
ended up in the running executable anyway, due to indirect loading.
To catch such problems, the "make installcheck" was extended:
dpkg-shlibdeps is run, if available, and the error output is scanned
for the messages which indicate that a symbol is used without linking
to the right library (example output below).
Had to fix quite a few _LIBADD lines to pass the test.
Some exceptions are allowed:
- libsmltk depends on the caller providing SySync logging support.
- libneon is intentionally not linked explicitly for syncevolution.org
binaries, to make resulting binaries work with GNUTLS and OpenSSL.
dpkg-shlibdeps: warning: debian/syncevolution-libs/usr/lib/syncevolution/backends/syncdav.so contains an unresolvable reference to symbol icalparameter_new_from_value_string: it's probably a plugin.
dpkg-shlibdeps: warning: 51 other similar warnings have been skipped (use -v to see them all).
...
dpkg-shlibdeps: warning: symbol dlsym used by debian/libsyncevolution0/usr/lib/libsyncevolution.so.0.0.0 found in none of the libraries.
dpkg-shlibdeps: warning: symbol dlerror used by debian/libsyncevolution0/usr/lib/libsyncevolution.so.0.0.0 found in none of the libraries.
dpkg-shlibdeps: warning: symbol dlopen used by debian/libsyncevolution0/usr/lib/libsyncevolution.so.0.0.0 found in none of the libraries.
When libical and libecal were not installed, trying to use the CalDAV
backend for VEVENTs segfaulted because it depends on libical and did
not check properly for it. Only affected syncevolution.org binaries.
The root cause is that libical functions were only looked up in
combination with libecal, when compiled with
--enable-evolution-compatibility. Now they are first checked via
libecal (for old libecal which embedded libical) and separately in
libical itself as fallback.
Google Calendar sometimes returns redirect requests to human-readable
web sites (an "unavailable" page, a login form). This is of course
bogus when the client is an automated CalDAV client.
The "unavailable.html" case was already handled. Made it a bit more
flexible to also catch possible variations of it (additional
parameters, https instead of http).
Added the https://accounts.google.com/ServiceLogin case. Not sure
whether retrying will help in that case, but there's not much else
that SyncEvolution can do.
When asked to insert a VJOURNAL which already existed (= same UID),
CalDAV servers respond with a 412 "Precondition failed" error. This
needs to be detected and translated into an "item needs to be merged"
result so that the engine can load the existing item, merge the data,
and then write back.
A test for this, testInsertTwice, will be committed separately. The
code was written so that it handles the same error when using CardDAV.
However, this was not tested because CardDAV test data does not have a
UID (wouldn't trigger the problem) and Radicale did not report 412 when
adding the UID.
Instead of relying on catching an exception, better use the new
"expected status codes" feature and check for 412 as part of the
normal switch statement.
Some WebDAV requests might fail with a non-okay status code that the
caller expects. This was reported via an exception. But
SyncEvolution's design uses exceptions only for non-normal
incidencences. Therefore better allow the caller to indicate which
status codes are expected and return normally from run() and
checkError() when those are encountered, without retrying and without
throwing an exception.
In some cases data with a very long UID wasn't handled correctly,
causing the out-going data to be malformed and probably causing a
rejection by the server.
The root-cause was two-fold:
- extractUID() didn't expect folding. Normally it deals only with
data encoded by libsynthesis, which does not use folding; the unexpected
exception was data which gets imported directly (--import).
- setResourceName() used std::string::replace() incorrectly: second
parameter is length, not end offset, of the data to be replaced.
The command line --status operation did not complete when applied to a
CalDAV/CardDAV source. Instead it aborted because the operation took a
code path where the backend was not fully initialized.
Sometimes the backend has to resend requests because of temporary
issues. If the problem turned out to be permanent, there was a long
period of time, retryDuration=5 minutes to be precice, in which no
visible progress happened.
Now SyncEvolution's WebDAV backend will print a message like this
before going to sleep until it is time to retry:
[INFO @client-test-googlecalendar] operation temporarily (?) failed, going to retry in 5.0s before giving up in 18.4s: PROPFIND: Neon error code 1: 401 Unauthorized
The uncertainty comes from several factors. In this example, the 401
might indicate a permanent problem (wrong credentials), or it could be
Google reporting a temporary authorization problem which is (probably)
meant to slow down the client while it asks the user to re-enter the
password. SyncEvolution only asks for passwords once, so it tries
again with the same password if it was successful with it in the
past. Otherwise it gives up immediately.
Another dubious example are name server lookup errors. They can be
ermanent (wrong host name) or temporary (name server
down). SyncEvolution errs on the side of retrying, to avoid
interrupting an operation which still has a chance to continue.
Currently this output is only visible in combination with --daemon=no,
because when using syncevo-dbus-server the output is not passed on
properly.
The message resend loop in the WebDAV backend did not check abort
requests. Now the error which caused the resend is reported
immediately in case of an abort. The actual send/receive still cannot
be interrupted, as it happens inside libneon with little chances to
influence the execution (?).
When waiting for resending a failed message, the sleeping couldn't be
interrupted when using the D-Bus server. It now uses the SuspendFlags
infrastructure and glib, which detects abort requests sent via D-Bus
in addition to those sent via signals (which already worked earlier).
CalDAV VJOURNAL and VTODO need similar UID handling as VCARD: when
creating or updating an item, ensure that it has a UID in the
payload.
It may also help to match that with the resource name and use
.ics/.vcf as suffix of the resource name. It was necessary for Yahoo's
CardDAV support and doesn't hurt elsewhere.
When updating, the merging with local data now ensures that any UID
different from the resource is properly preserved when the incoming
data has no UID.
All backends except for EDS replaced local data wholesale with an
incoming update, even if that update came from a peer which did not
store all properties. The EDS backend had already been configured
earlier to always merge remote and local data before writing it back.
Instead of updating each backend individually it makes more sense to
make the more intelligent (and more expensive) merging the default in
backends derived from SyncSourceSerialize/TrackingSyncSource. Backends
which do not want it can remove
"<updateallfields>true</updateallfields>" from
SynthesisInfo::m_datastoreOptions.
The new backend property values "CalDAVTodo" and "CalDAVJournal"
select tasks resp. memos stored in a CalDAV collection. "CalDAV"
continues to select events.
Events, tasks and journals can be mixed in the same resource (=
URL). However, this is less efficient than storing them separately.
A good CalDAV server allows filtering items by type, and SyncEvolution
uses that. However, it was found that Radicale 0.7 ignores this
filtering, which could have led to data loss (SyncEvolution asks for
all VTODOs in preparation for a "delete all items" operation in a
"CalDAVTodo" source, gets also VJOURNALs, then deletes them).
Therefore SyncEvolution plays it safe and downloads the VTODO and
VJOURNAL data to double-check that it is working on the right items.
This causes additional traffic for well-behaving servers; currently
it cannot be turned off.
What is missing for VJOURNAL is the conversion to plain text (see BMC
not possible yet.
Radicale reports 200 'Okay' instead of 204 when asked to delete an
existing item. Was triggering a status error. Found when running
Client::Source test against Radicale 0.7-7-g186c59c; 0.7 still
returned 204, see http://redmine.kozea.fr/issues/802.
Radicale reports 412 'Preconditiona Failed' instead of 404 when asked
to delete a non-existent item. Translate that into the 404 expected by
the Synthesis engine. Found when running Client::Source test against
Radicale 0.7-7-g186c59c.
Radicale reports 410 'Gone' instead of 404 when asked to read a
non-existent item. Translate that into the 404 expected by the
Synthesis engine. Found when running Client::Source test against
Radicale 0.7-7-g186c59c.
Due to the way how properties were handled internally, it wasn't
possible to explicitly set a property to its default value. Instead
the property was unset. For example, explicitly setting database= was
not possible.
This is necessary for client-test and ActiveSync, because client-test
needs to know that the testing is expected to run with the default
databases (something which normally is avoided by overwriting empty
database properties).
Now the "is set" state is tracked explicitly in the config storage and
command line property APIs. Unsetting a property via the command line
could be implemented with an explicit command line option, but is not
supported at the moment.
Tests were extended to cover the new functionality and adapted to the
change behavior for "type" migration: syncFormat was empty
already (because the empty string matched the default), but
forceSyncFormat was unnecessarily set explicitly. Now it is not.
Google doesn't seem to support the Fetch operation, which is used
during testing to retrieve unchanged items. During syncing it will
only be needed when merging incoming data with an existing item, which
should not be necessary... except that testing shows that it is
necessary. The case where it is needed is:
- incremental sync (cache empty)
- calendar event series needs to be modified
- items from that series need to be fetched in preparation
for updating it
To allow testing, several workarounds are necessary:
- request *all* data before doing a data dump in testImport (and friends),
to ensure that the cache in the backend is fully populated
- use the cache in ActiveSyncCalendarSource instead of accessing
the base class directly, because that would trigger a Fetch
Accessing the base class is still useful for Exchange+calendar,
because testing then avoids the cache (and thus can expose bugs
in it).
While at it, adapted the README. "database" needs to be set
explicitly, "client-test" only does it when creating configs.
Fixed ESourceList leak. Was never freed?! Now assigned to a new GObject
C++ wrapper as owner.
client API: Use EvolutionSyncSource::throwError() consistently, because it
relates the error to the source (nicer).
client API: fixed "only-if-exists" issue for contacts.
client API: fixed leak of uid after creating calendar object.
The Qt headers define "signal", which breaks GNOME header
files if those get included later. Rearranged header
files to avoid ugly "undefine" statements.
Found when using EDS client API header files, because those
include different GNOME headers.
e_cal_client_tzlookup() doesn't detect that lookup
fails because of non-existant time zone, because it gets
a non-descriptive error code from EDS ("Invalid object").
Work around that by ignoring all errors in our own lookup
function.
See "[Evolution-hackers] e_cal_client_check_timezones() + e_cal_client_tzlookup() + Could not retrieve calendar time zone: Invalid object"
KWallet support was broken: syncevo-dbus-server checked
KDE_FULL_SESSION to determine whether it should use KWallet instead of
GNOME Keyring. That did not work, because the env variable was not set
for D-Bus daemons.
Automatically detecting KDE users is not possible at the
moment. Instead KDE users have to manually set the new "keyring"
global config property to "KDE" (case insensitive) if the
SyncEvolution installation supports both, because GNOME Keyring is the
default to avoid surprises for traditional users. If only KWallet
support is enabled, then this is not necessary.
"GNOME" and "true/false/1/0/yes/no" can also be set. This has the
advantage that keyring usage can be enabled permanently for the
command line in --daemon=no mode; normally keyrings are not used in
that mode because accessing them can bring up UI dialogs.
It also becomes possible to disable keyring usage in syncevo-dbus-server,
something which couldn't be done before.
The --keyring command line option is still supported, as an alias for
"[--sync-property] keyring=<value>". The default value for --keyring
is true, to match the traditional behavior. In contrast to other sync
properties, setting "keyring" does not require an explicit --run
parameter. Again this is done to mirror traditional usage.
Reading a password also (unintentionally) checked all supported
storages while searching for the password. Now it uses exactly
one storage and falls back to asking for the password directly.
The commit itself also cleans up the code a bit (reformatted, fixed
comments). Choosing the right slot in the password signals is done via
a new InitStateTri parameter which contains the "keyring" setting.
Error checking (unsupported keyring string, --keyring=yes and no
keyring enabled) is done in additional slots which run after all the
regular ones.
Parameter parsing for --sync and --keyring were unified. However,
there is the difference that --keyring has an implicit default value
("yes") and never has an additional parameter, in contrast to --sync,
which always is followed by one.
The new CmdlineTest::testKeyring covers different ways of using
--keyring. It relies on actually invoking keyring backends, something
not done by the default SyncContext UI. Therefore
CmdlineSyncClient+KeyringSyncCmdline were moved into libsyncevolution,
to be used by CmdlineTest.
A normal KDE user should already have the akonadi server running.
Others probably don't want it to be started, for example as part
of --print-databases.
Therefore auto-starting Akonadi is now disabled. Attempts to use
the Akonadi sources results in an error message with a hint
about akonadictl when Akonadi is not already running.
The same problem is less relevant for EDS (but still exists to
some extend): here listing databases "merely" starts gconfd.
Merged branch 'eds-api'. Several changes that were made in the
meantime on the master branch caused conflicts and/or required
updating the new code to avoid regressions. The code was also
updated to address some issues:
- g_propagate_error() steals ownership from GErrorCXX,
thus causing a double free. Replaced with consistent use
of GErrorCXX.
- Putting opening brackets into the middle of ifdefs
breaks automatic code formatting. Now the code has "if("
and ") {" outside of the ifdef/else/endif block.
The old code was updated to also use GErrorCXX. This revealed a memory
leak in the error path for opening databases, where the GError was not
cleared before returning.
SyncEvolution EDS backend currently uses the ECal API which is
deprecated as of EDS v3.2 and replaced by the new ECalClient API.
This patch brings support for this new ECalClient API (when EDS
3.2 is detected at compile time).
The new code based on the ECalClient API will enable a few
optimizations, especially the partial fetching of components to
avoid useless DBus trafic. This results in better component change
tracking by fetching only the UID/RID/REV attributes for each
component.
Note: The new code uses the new GLibSupport header instead of the
deprecated SmartPointer (eptr).
SyncEvolution EDS backend currently uses the EBook API which is
deprecated as of EDS v3.2 and replaced by the new EBookClient API.
This patch brings support for this new EBookClient API (when EDS
3.2 is detected at compile time).
The new code based on the EBookClient API enables a few
optimizations, especially the partial fetching of contacts to
avoid useless DBus trafic. This results in better contact change
tracking by fetching only the UID/REV attributes for each contact.
Note: The new code uses the new GLibSupport header instead of the
deprecated SmartPointer (eptr).
