Matches the code which creates the configs. Has to recreate the
properties based on the information found in the SyncEvolution
configuration.
Runs outside of a Session because it works with an atomic snapshot of
the SyncEvolution configs (no other code can modify it while this code
reads it).
The peer properties are are stored as normal SyncEvolution
configurations. Every peer gets its own context, which contains both
the local source definition for the peer and sync plus target configs.
The special name space prefix "pim-manager-" used to identify them and
avoid potential conflicts with normal SyncEvolution configurations.
Local EDS databases are created with a fixed UID and name that also
have that prefix. One could go one step further than it is currently
done and set these databases to "disabled" in the ESourceRegistry,
which would hide them in Evolution.
Creating the databases via the synchronous SyncEvolution SyncSource
API is cheating a bit - strictly speaking, syncevo-dbus-server should
not call methods which can block, to keep it responsive. The long term
goal for syncevo-dbus-server is to move all SyncSource instantiation
and usage into separate processes, to avoid tainting the D-Bus service
with the mess that loading arbitrary third-party libraries can cause.
Config changes and syncing must be serialized, to avoid race
conditions. The Server class tracks Sessions and only activates one at
a time. This patch hooks into that concept and executes the different
operations once the requested session is the active one.
The PIM Manager doesn't have to keep track of the pending D-Bus calls.
That is done implicitly as part of binding an operation to the Session
instance via the "active" signal. What does have to track are pending
sessions, because it is the owner of those. The shared_ptr in
m_pending are the only strong reference to these sessions.
Removing them (as done in stopSync()) automatically destroys the
session and the pending D-Bus call, which (via the default behavior of
DBusResult in GDBus GIO) will send a generic "method failed" error to
the caller.
If the caller needs to distinguish between "operation cancelled" and
"fatal error", then this indirect tracking of the DBusResult must be
changed so that removing a session can call the failed() method with a
suitable error (to be defined in the API).
The Server and Session classes get extended so that it becomes
possibly to create sessions which do not belong to a D-Bus client and
to query the result of a sync directly.
It is possible to sync source A in context @X with source B in the
same context @X. The PIM manager is doing that with one context per
peer and databases that are guaranteed to not conflict with anything
else.
Therefore allow such configs. A better check is left for later, if it
really should be needed at all.
The key trick is the mapping between FolksIndividual and D-Bus. The
patch verifies that this can be done via D-Bus traits, without
fleshing out more than the very basics (sending full name).
Adds a subset of the D-Bus API to syncevo-dbus-server. Views are
resources that are attached to the client and thus get destroyed
automatically when the client disconnects from D-Bus.
In addition, failures for calls to the client's ViewAgent delete the
view. In this case, the client stays connected, because it might only
have destroyed one agent with others still active.
The choice of std::vector in getContacts() comes from the goal of
using this function more or less directly in a D-Bus implementation.
It works efficiently because the size can be computed in advance.
A more general API, like for example one which stores via some append
iterator, would also have been possible but probably isn't worth
the effort.
The classes on top of libfolks are designed such that they can
be tested without involving libfolks. Right now the goal is only
to get the API right; most of the actual functionality is missing.
The terminology (added/modified/removed) is aligned with libebook
terminology and the PIM Manager API description.
Initial step towards using SyncEvolution, PBAP and libfolks in the
context of IVI (in-vehicle-infotainment): D-Bus API definition for the
org._01.pim.contact API, --enable-dbus-service-pim, find libs, compile
into syncevo-dbus-server and client-test.
The only functional code at this time is the unit testing of libfolks,
GValueCXX and libgee.
Creating a database is only possible with a chosen name. The UID is
chosen automatically by the storage. A new property would be needed
to also specify the UID.
There are no command line tests for the new functionality because
support for it in backends is limited.
Depends on the EDS 3.6 ESourceRegistry API. Needs the very latest EDS
with the following commit. There's no configure check yet because EDS
3.6 is not released yet.
commit 6df76009318eac9dbe3dd49165394d389102764e
Author: Matthew Barnes <mbarnes@redhat.com>
Date: Tue Sep 11 22:56:08 2012 -0400
Bug 683785 - Add e_source_new_with_uid()
Variation of e_source_new() which allows a predetermined UID to be
specified for a scratch source. This changes the "uid" property from
read-only to read/write + construct-only, and eliminates the need for
EServerSideSource to override the property.
The API supports creating a database with a specific name and UID. To
avoid ambiguities, deleting is done via the UID.
Looking up the UID for a "database" property value is meant to be
supported by opening the SyncSource and then getting the current
database that was opened.
This allows implementing a delete operation that uses the same
database lookup method as syncing.
All of the new methods have stubs which reject the operation
respectively tell the caller that no information is available. This
was done to avoid having to adapt all derived SyncSources. Support for
the new functionality is optional.
Looks up a value in a map and returns a found value or a default. This
mimicks Python's dict.get().
Because it is not possible in C++ to return None as default, the
returned value is wrapped in a InitState to tell the caller whether
the value was found.
Depends on the unified InitState template.
The new InitState template works for both classes and generic types.
The difference is in the kind of base class that it inherits from: the
one for classes "is" an instance of the wrapped type, the other "has"
such an instance. boost::is_class is used to pick the right utility
class, something that had to be done manually previously.
While touching the classes, the explicit initialization with "const
char *" was replace with a more general template constructor.
The main advantage is that other template code, like the D-Bus traits,
no longer needs to distinguish between InitState and
InitStateClass. The InitState implementation did not become
smaller. At least it avoids code duplication (m_wasSet handled in one
place).
Using a DBusClient* to make an actual call does not change the object.
The call is independent of the instance used to make it (see also the
previous commit about handling method results for asynchronous calls
after destroying the call instance). Therefore it makes sense to mark
the methods as const.
Now DBusResult itself tracks whether a reply was sent. If not, it
reports a generic "processing the method call failed" error. If that
error is not good enough for the method implementation, then it must
call failed() itself, as before.
Remove global variables and allow addditional names to be owned via
DBusConnectionPtr::ownName().
g_bus_own_name_on_connection() is given shared access to a ref-counted
OwnNameAsyncData (instead of full ownership) because that way it is
guaranteed that the instance is still around while waiting for name
acquisition in undelay().
Primarily this is meant to be used for iterating over collections
in libfolks, which come straight from libgee.
The header file assumes that the corresponding gee header files are
available. A user of it must link against libgee.
The wrapper enhances type safety (can only assign values of
the right type, enforced at compile time) and automates resource
tracking (g_value_unset() called be destructor). Generic,
boxed values need special treatment because their GType
is not a constant.
The header file assumes that the corresponding glib header files
are available. A user of it must link against libgobject.
The synchronous version used to deadlock frequently, the asynchronous
one doesn't, so use that instead. The problem was also worked around
later in EDS (commit below), but let's stick to the asynchronous
version anyway to be closer to Evolution.
commit 6c4c1c6a1f40a6873be954a0b9d770e31726bd50
Author: Matthew Barnes <mbarnes@redhat.com>
Date: Fri Sep 7 07:40:09 2012 -0400
ESourceRegistry: Work around GType deadlock.
Work around http://bugzilla.gnome.org/show_bug.cgi?id=683519
until GObject's type initialization deadlock issue is fixed.
Apparently only the synchronous instantiation is affected.
Instead of manually written callback functions, use the C++ helper
code to get C++ code invoked from inside glib. Reduces the code
size and ensures that no exceptions escape into glib C code.
gcc failed because the type of the function is not a pointer, and it
does not allow that as template parameter (in contrast to
clang). Explicitly turn the function name into a pointer.
Some asynchronous operations are guaranteed to not fail, in which case
the finish method will have two parameters (object + handle), or just
one (handle).
This patch adds support for that by checking the type of the first
parameter to distinguish between these additional cases. It switches
to function_traits to determine function attributes. Using
boost::function also works, but was a bit hacky.
e_source_registry_new_finish() does not take a GObject pointer
as first parameter. Added another flavor of GAsyncReadyCXX
for _finish() calls with only two instead of three parameters.
A common pattern when the result is needed immediately is to write a
callback function which just stores the result and then iterate the
main loop until the asynchronous method completes. The new
SYNCEVO_GLIB_CALL_SYNC() macro provides such storage methods in the
GAsyncReadyDoneCXX helper classes and waits until the method is done.
The connectSignal() utility code can be used instead of manually
written static callback functions. The direct C callbacks are provided
by the header file for up to nine parameters and invoke a
boost::function (or any class which can be turned into a
boost::function, like the result of boost::bind).
The provided C callbacks ensure that exceptions do not escape into the
calling glib. Any exception making it that far aborts the
program. This is often missing from manually written callbacks.
The trick is to allocate a boost::function which is owned by glib and
gets destroyed with another callback when no longer needed.
With Google Contact + CardDAV the auto-discovery failed after
finding the default address, without reporting that result.
The reason was that the search continued at the root of the server
where PROPFIND triggers an error when using the Google server. Because
of a missing check for "have result", that error was treated as
something which needs to be reported to the user.
Fixed by unifying the various checks in a singe class.
The randomness of readdir() can cause heisenbugs. It also makes
operations like --import of files in a directory unpredictable, unless
the caller remembers to sort himself. Better make sorting
mandatory. There shouldn't be any need for performance critical
directory reading.
Previously, if http_proxy was set, a proxy was used even if
explicitly disabled. This prevented disabling the use of a proxy
which only made sense in some cases, like accessing something
that runs locally. Explicitly telling SyncEvolution to ignore
http_proxy is necessary because it doesn't support no_proxy.
This change became necessary in the new nightly test server,
which depends on http_* env vars for communication with external
servers.
Because of incomplete support for time conversion, the due date
could get mixed up when phone and PC were set to something other
than UTC.
Reported and fixed by Peter Jan.
In the past, using curl as HTTP transport in the syncevo-dbus-server
was prevented, leading to "unsupported transport type is specified in
the configuration". The reason was that using curl would block the
server and make it unresponsive on D-Bus.
This reason has gone away, because now the HTTP traffic happens in a
separate process. Thus now it is allowed to use curl in the
syncevo-dbus-server.
Fallback to old obexd API was broken because creating the
DBusRemoteObject does not verify whether the service really exists and
thus always succeeds. Fixed by checking for existence as part of the
actual CreateSession method call.
The new code needs glib. Include header file and declare dependency in
configure.
The backend must throw errors when something fatal happens. Logging
the error is not enough, because that can't be checked by the
caller. Throwing errors is best done via the utility methods in
SyncSource, because those include the source name in the exception.
Memory handling was broken: nothing owned the memory in the
StringPiece instances, munmap() was missing. Fixed by making
PbabSyncSource the owner of both.
Unified the parsing of the result. The new code based on pcrecpp is
used for both old and new obexd API.
File name and GError allocated by g_file_open_tmp() were leaked. The
file descriptor and the file were leaked in case of aborting via an
exception. Now these resources are owned by a class which will always
clean up when getting destructed.
A failed transfer was not checked for when using the new API. Probably
failed when trying to use the file (because obexd deletes it), but
better show the error message that we got for the failed transfer.
Remove the obsolete vcardParse().
The backend is not useful for most users, therefore it has to be enable
during compilation with --enable-pbap. The code for "PBAP disabled"
had to be adapted to a backend API change.
Instead of using explicit g_variant_unref() calls, let a smart pointer
class own the instance. This allows to remove code and (more
important) fixes leaks when something throws an exception or the
D-Bus traits for boost::variant return prematurely.
The signal name was not stored, making the actual filtering more
relaxed than it should have been. Worse, the path was redundantly
checked for a complete match even when SIGNAL_FILTER_PATH_PREFIX was
set, thus preventing signal deliver in that case.
Working with obexd 0.47's Transfer object depends on path prefix
filtering of signals to be efficient. This patch adds that support in
the GDBus C++ bindings, together with some documentation on how to
use it.
It also allows relaxing the signal matching by interpreting empty
strings as "match any".
Finally it adds support for passing of sender, path, interface and
member (= signal name) information to the user of the D-Bus
binding. This is necessary because a handler with wildcards for
matching cannot know the actual values unless they are passed to the
callback explicitly. This was not possible at all before for signals
(because extracting that information relied on a message pointer,
which wasn't set while decoding signal messages) and limited to just
the sender in method implementations.
Besides in functionality, GDBus for GIO and for libdbus now also
differ slightly in their API. In order to pass meta data about a
signal into the demarshalling get() methods, they have to be stored in
ExtractArgs first and passed to get() via that struct, which is an
interface change.
Derived code which extends marshalling/demarshalling needs to be
adapted accordingly. Allowing some ifdefs elsewhere seems simpler than
adapting GDBus for libdbus. Those ifdefs can be removed once libdbus
support is no longer needed.
The patch relaxes access control to DBusObject and DBusRemoteObject
because that simplifies the code which works better with std::string.
Traditionally the sender of a signal was not checked. This is probably
related to unsolved questions like "should watching org.foo.bar follow
owner changes", but not doing any checking is just not right
either. Not fixed in this patch.
and allow this UID to be used for selecting a particular calendar.
Since all listed calendars are in the default storage anyway,
the UID is far more useful to have. (On the N9, knowing the
physical storage does not help the user at all anyway, as access
to it is restricted and needs to go through the API anyway.)
(cherry picked from commit a5c2939c1d)
The PBAP backend must link against pcrecpp explicitly. The header
files are part of the global search path, but the library is not.
Having to specify the library explicitly is right (avoids
adding it to modules which don't need it); perhaps the header
file flags also should have to be added explicitly.
It worked when linked into an executable which also uses pcrecpp, but
can fail to build depending on how strict the toolchain is. Found
by dist checks in the nightly testing.
When using libdbus instead of GIO D-Bus (as done by syncevolution.org
binaries and SyncEvolution on Maemo), local sync may have aborted
after 25 seconds when syncing many items with a D-Bus timeout error:
[ERROR] sending message to child failed: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
Reported by Toke Høiland-Jørgensen for Harmattan. Somehow not encountered
elsewhere.
Root cause is the use of the default D-Bus timeout of 25 seconds
instead of disabling the timeout altogether. Fixed by using
DBUS_TIMEOUT_INFINITE and INT_MAX as fallback.
(cherry picked from commit a15e8be1c2)
When using GIO D-Bus, sometimes messages created shortly before
shutting down were not sent. Must flush a connection explicitly.
A NOP when using libdbus.
First, clarified that a source has to set all revision strings or
none. This was implied, but not spelled out explicitly.
Second, use that to detect when a sync must be done in slow mode.
Third, avoid calculating changes when that isn't needed. This last
point was left as TODO while in release preparations and can
be committed now.
This changes is needed for the PBAP backend which does not currently
set revision strings and thus cannot be used in incremental syncing.
This patch introduces support for true one-way syncing ("caching"):
the local datastore is meant to be an exact copy of the data on the
remote side. The assumption is that no modifications are ever made
locally outside of syncing. This is different from one-way sync modes,
which allows local changes and only temporarily disables sending them
to the remote side.
Another goal of the new mode is to avoid data writes as much as
possible.
This new mode only works on the server side of a sync, where the
engine has enough control over the data flow.
Most of the changes are in libsynthesis. SyncEvolution only needs to
enable the new mode, which is done via an extension of the "sync"
property:
- "local-cache-incremental" will do an incremental sync (if possible)
or a slow sync (otherwise). This is usually the right mode to use,
and thus has "local-cache" as alias.
- "local-cache-slow" will always do a slow sync. Useful for
debugging or after (accidentally) making changes on the server side.
An incremental sync will ignore such changes because they are not
meant to happen and thus leave client and sync out-of-sync!
Both modes are recorded in the sync report of the local side. The
target side is the client and records the normal "two-way" or "slow"
sync modes.
With the current SyncEvolution contact field list, first, middle and
last name are used to find matches during any kind of slow sync. The
organization field is ignored for matching during the initial slow
sync and used in all following ones. That's okay, the difference won't
matter in practice because the initial slow sync in PBAP caching will
be done with no local data. The test achieve the same result in both
cases by keeping the organization set in the reduced data set.
It's also okay to include the property in the comparison, because it
might help to distinguish between "John Doe" in different companies.
It might be worthwhile to add more fields as match criteria, for
example the birthday. Currently they are excluded, probably because
they are not trusted to be supported by SyncML peers. In caching mode
the situation is different, because all our data came from the peer.
The downside is that in cases where matching has to be done all the
time because change detection is not supported (PBAP), including the
birthday as criteria will cause unnecessary contact removed/added
events (and thus disk IO) when a contact was originally created
without birthday locally and then a birthday gets added on the phone.
Testing is done as part of the D-Bus testing framework, because usually
this functionality will be used as part of the D-Bus server and writing
tests in Python is easier.
A new test class "TestLocalCache" contains the new tests. They include
tests for removing extra items during a slow sync (testItemRemoval),
adding new client items under various conditions (testItemAdd*) and
updating/removing an item during incremental syncing
(testItemUpdate/Delete*). Doing these changes during a slow sync could
also be tested (not currently covered).
The tests for removing properties (testPropertyRemoval*) cover
removing almost all contact properties during an initial slow sync, a
second slow sync (which is treated differently in libsynthesis, see
merge=always and merge=slowsync), and an incremental sync.
For example in the new TestLocalCache.testItemDelete100, the
percentage value in the ProgressChanged signal become larger
than 100 and then revert to 100 at the end of the sync.
Seems the underlying calculation is faulty or simply inaccurate.
This patch does not fix that. Instead it just clips the final
result to the valid range. It also cleans up ownership of the
actual int32_t progress value.
Instad of having three of the helper classes as members,
only store pointers to them in Server. The advantage is
that including Server.h becomes simpler for .cpp files
which do not need to use these classes and that they
can be constructed/destructed more explicitly. This is
particularly important because they get access to the
Server instance which is still in the process of
initializing itself.
Change tracking in the file backend used to be based on the
modification time in seconds. When running many syncs quickly (as in
testing), that can lead to changes not being detected when they happen
within a second.
Now the file backend also includes the sub-second part of the
modification time stamp, if available. The revision string
intentionally uses no leading zeros, because that would make it
unnecessarily larger.
This change is relevant when upgrading SyncEvolution: most of the
items will be considered "updated" once during the first sync after
the upgrade (or a downgrade) because the revision strings get
calculated differently.
With this patch, the databaseFormat can be used as follows:
[(2.1|3.0)][:][^]propname,propname,...
3.0 = download in vCard 3.0 instead of the default 2.1
3.0:^PHOTO = download in vCard 3.0 format, excluding PHOTO
PHOTO = download in vCard 2.1 format, only the PHOTO
Valid property names are pulled from obexd with ListFilterFields().
Binding a temporary std::string (the result of getDatabase() in this
case) to a const reference is broken, because the temporary instance
will get deleted before the reference.
Local IDs across sessions are only useful when we also have useful
revision strings. For debugging it is easier to just enumerate the
contacts. Would be nice to use the same number as in the PBAP session,
but that information is not currently available via obexd (see "PBAP +
two-step download" on Bluez mailing list).
As it stands now, the PBAP backend can only be used in slow syncs
where the engine does the matching. Perhaps that's the right way to do
it, instead of adding redundant code to the backends.
When using libdbus instead of GIO D-Bus (as done by syncevolution.org
binaries and SyncEvolution on Maemo), local sync may have aborted
after 25 seconds when syncing many items with a D-Bus timeout error:
[ERROR] sending message to child failed: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
Reported by Toke Høiland-Jørgensen for Harmattan. Somehow not encountered
elsewhere.
Root cause is the use of the default D-Bus timeout of 25 seconds
instead of disabling the timeout altogether. Fixed by using
DBUS_TIMEOUT_INFINITE and INT_MAX as fallback.
When using libdbus instead of GIO D-Bus (as done by syncevolution.org
binaries and SyncEvolution on Maemo), local sync may have aborted
after 25 seconds when syncing many items with a D-Bus timeout error:
[ERROR] sending message to child failed: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
Reported by Toke Høiland-Jørgensen for Harmattan. Somehow not encountered
elsewhere.
Root cause is the use of the default D-Bus timeout of 25 seconds
instead of disabling the timeout altogether. Fixed by using
DBUS_TIMEOUT_INFINITE and INT_MAX as fallback.
Some unnamed version of KDE crashes in KApplication when invoked
without a D-Bus session. The reporter ran into this when compiling
from source, because the SyncEvolution binary is invoked as part of
the build process, which ran outside of a D-Bus session.
Avoid the crash by checking for a D-Bus session bus with
QDBusConnection::sessionBus().isConnected() before instantiating
KApplication. The QDBusConnection API does not say explicitly when it
connects to the daemon, but testing shows that in practice this
detects missing env variables and an unreachable daemon right away as
expected, while passing when the daemon can be contacted.
Instantiating KApplication was added for KWallet support. Without
D-Bus, KWallet does not work either, therefore throw an explicit error
when the lack of D-Bus was detected.
A combination of Funambol Android and Funambol server recently led to
the Funambol server sending PHOTO data with TYPE=image/jpeg. This is
invalid and caused EDS to reject the photo (Vladimir Elisseev,
"[SyncEvolution] issues with syncing photos").
Work around the problem by only keeping the part of the type after the
last slash, if there is any. For image/jpeg and similar types that
leads to the desired value and does not affect valid values, because
those do not contain a slash
(http://www.iana.org/assignments/media-types/image/index.html).
When syncevo-dbus-server was started on demand by the D-Bus daemon,
then it registered itself with the daemon before it was ready to
serve requests. Only happened in combination with GIO D-Bus and
thus was not a problem before 1.2.99.x.
One user-visible effect was that the GTK UI did select the default
service when it was started for the first time, because it could not
retrieve that information from syncevo-dbus-server.
The fix consists of delaying the name acquisition. That gives the
caller a chance to register D-Bus objects first, before completing the
connection setup. The semantic change is that
dbus_bus_connection_undelay() must be called on new connections which
have a name (a NOP with libdbus).
This patch tries to minimize code changes. The downside of not
changing the GDBusCXX API more radically is that the bus name must be
attached to DBusConnectionPtr, where it will be copied into each
reference to the connection. Hopefully std::string is smart enough to
share the (small) data in this case. Should be solved cleaner once
libdbus support can be deprecated.
A test for auto-activation and double start of syncevo-dbus-server
is also added.
and allow this UID to be used for selecting a particular calendar.
Since all listed calendars are in the default storage anyway,
the UID is far more useful to have. (On the N9, knowing the
physical storage does not help the user at all anyway, as access
to it is restricted and needs to go through the API anyway.)
One method was still virtual, from the time when the class had a base
class. Remove the "virtual" keyword because a) it causes a compiler
warning in recent g++ about a class with non-virtual destructor with a
virtual method and b) it makes the generated code unnecessarily
complex.
The old explanation made it sound like nothing would get deleted by
default ("If set, ..."). That's not correct, by default only 10
sessions are kept.
Also explain the behavior of deleting intermediate sessions first.
Auto syncing was not getting triggered when using an autoSyncDelay > 0;
by default it is 5 minutes. Thanks to Vladimir Elisseev for reporting
this problem.
The root cause was a "greater than" instead of a "less than"
comparison. This was not found by the tests because they set
autoSyncDelay to zero to speed up testing.
Changing one test (TestSessionAPIsDummy.testAutoSyncNetworkFailure) so
that it uses non-zero autoSyncDelay triggered the problem. Also added
a variation of that test that simulates the "no Connman and no
NetworkManager" setup used by Vladimir.
Using "template" as parameter name causes problems for Qt, because the
name gets used in the generated C++ stubs. The earlier solution for
the problem (spelling it as "tmplate") was accidentally removed in
e4bd5bd while preparing 1.2.99.1.
Let's use "getTemplate" instead of "templateName", because the
parameter itself does not actually contain the name; it's just
a boolean.
This reverts commit 2d3153bc48.
Reverting to the Qt 4 compatible annotations and fixing
the accidental removal of the "template" parameter in
this method.
When compiled against EDS 3.5.x or later, SyncEvolution now uses
the backend code originally written for the EClient API introduced
in EDS 3.2. That code was changed so that it works with the new
include file rules and ESourceRegistry in EDS 3.5.x. Support
for using the EClient API with EDS 3.4 was removed because maintaining
three different flavors of the EDS backend code would be too much
work and not gain much (just the possibility to test the EDSClient
code with 3.4).
At the moment, this is a compile time choice made automatically
by configure. syncevolution.org binaries are compiled against
an older EDS and thus do not work with EDS 3.5.x or later.
EDS 3.5.x handles authentication itself, using a standard system
prompt if necessary. SyncEvolution can no longer provide the password,
and thus the "databaseUser/Password" options have no effect when using
EDS 3.5.x.
The patch leaves code for older EDS almost completely unchanged and
therefore is considered safe for the stable release series leading to
1.3. Using EClient is an all-or-nothing choice now, because the common
EvolutionSyncSource needs to be compiled differently. Thanks to the
reorganized API, a lot more common code for ECal and EBook sources
could be moved into EvolutionSyncSource.
Instantiating an ActiveSyncSource for testing must work with
empty database names now. Testing no longer forces the
database to be set.
While at it rewrote the code to avoid the explicit pointer.
synccompare on the target side of a local sync was invoked with its
output being redirected via an unreliable socket to the local sync
parent. When the output was large, some of it might have been lost.
Fixed by reading the output locally in the syncevo-local-sync process
with reliable output redirection and sending it to the parent via
D-Bus. Execute() does that when both stdout and stderr are redirected
via LogRedirect, which makes sense also for other stdout output.
When processing stdout from syncevo-local-child in
syncevo-dbus-helper, the LogRedirect class was invoked recursively and
tried to print the same stdout data repeatedly until the
syncevo-dbus-helper crashed due to the infinite recurssion.
The output shouldn't have been sent via stdout in the first place
(will be fixed separately), but nor should LogRedirect have crashed.
Fixed by swapping the data into a temporary buffer and thus not using
it again.
A backend cannot know whether it'll be used together
with a client-test which supports integration testing.
Therefore the backend should always register its tests.
Updating failed when using Google because the Synthesis engine
tried to read the existing item in order to merge it with
the update. This failed because Google does not implement the
Fetch command.
Pretending to update the item intelligently avoids that. It
also helps to improve performance of updates with Exchange.
The downside is that syncing with local storages which do
not support all ActiveSync fields will cause data loss.
Need to check whether Exchange-only attributes get lost
also when the local storage supports everything, for
example because activesynd unintentionally removes data.
The error message for "Invalid synchronization key" changed in
activesyncd, now it contains the D-Bus error type as prefix. Fixed by
doing a substring search.
Also, not freeing the GError before trying again is a bug. Apparently
that was ignored earlier, now it triggers an assert.
As mentioned by Tino Keitel on the mailing list, some libs and
executables were only implicitly linked against libraries that they
called directly. This happened to work by chance because these libraries
ended up in the running executable anyway, due to indirect loading.
To catch such problems, the "make installcheck" was extended:
dpkg-shlibdeps is run, if available, and the error output is scanned
for the messages which indicate that a symbol is used without linking
to the right library (example output below).
Had to fix quite a few _LIBADD lines to pass the test.
Some exceptions are allowed:
- libsmltk depends on the caller providing SySync logging support.
- libneon is intentionally not linked explicitly for syncevolution.org
binaries, to make resulting binaries work with GNUTLS and OpenSSL.
dpkg-shlibdeps: warning: debian/syncevolution-libs/usr/lib/syncevolution/backends/syncdav.so contains an unresolvable reference to symbol icalparameter_new_from_value_string: it's probably a plugin.
dpkg-shlibdeps: warning: 51 other similar warnings have been skipped (use -v to see them all).
...
dpkg-shlibdeps: warning: symbol dlsym used by debian/libsyncevolution0/usr/lib/libsyncevolution.so.0.0.0 found in none of the libraries.
dpkg-shlibdeps: warning: symbol dlerror used by debian/libsyncevolution0/usr/lib/libsyncevolution.so.0.0.0 found in none of the libraries.
dpkg-shlibdeps: warning: symbol dlopen used by debian/libsyncevolution0/usr/lib/libsyncevolution.so.0.0.0 found in none of the libraries.
Testing with libdbus 1.6.0 on Debian Testing failed because the
lib changed some behavior: instead of looking up the owner of
a certain bus name immediately, it now does that when invoking
a method. Therefore the check for "have connection" in NetworkManager
and ConnMan client code was too simplistic and missed the fact that
both were not usable, causing the server to assume that HTTP was down
while in reality it should have assumed it to be up.
When libical and libecal were not installed, trying to use the CalDAV
backend for VEVENTs segfaulted because it depends on libical and did
not check properly for it. Only affected syncevolution.org binaries.
The root cause is that libical functions were only looked up in
combination with libecal, when compiled with
--enable-evolution-compatibility. Now they are first checked via
libecal (for old libecal which embedded libical) and separately in
libical itself as fallback.
Instead of using delays to kill processes at the right time, watch old
and new debug output via D-Bus and then kill the processes. To avoid
race conditions, these processes get delayed at the right point.
Added tests for local sync and command line, covering killing of
all involved processes.
The session output needs to go through the server's
message handling, to get the same kind of treatment.
Sending it to the server's parent logger was missing
previously and thus it did not get sent to the
servers stdout. Found while looking at test-dbus.py
logs, which were incomplete.
When the D-Bus helper quit unexpectedly, the session
was closed immediately. That is necessary as a workaround
for D-Bus libdbus, which does not properly shut down
pending method calls when the connection goes down.
However, it prevents processing further events that might
still be pending. So as a workaround for the workaround
let's run the session shutdown with an internal failure
in an idle callback, which will happen after processing all
pending events.
The API supported repeating timeouts, but none of the users of
it needed that. To fit the API, they were forced to return a boolean
value, which prevented binding to callbacks without such a
return value.
That'll be useful, so let's simplify the code...
The helper process only detected that the parent failed when
it tried to log something while the parent had already shut down
the D-Bus connection. Even that did not work reliably and differed
between D-Bus libdbus and GIO.
Now logging ignores failures to send the message (done the same way
as in the syncevo-dbus-helper -> syncevo-dbus-server communication).
Parent failures are detected via the ForkExecChild::m_onQuit signal.
They set the "abort" state permanently until the helper process
terminates.
Found while testing some new failure test cases in combination with
D-Bus libdbus.
The EDS backend in the syncevolution.org packages are compiled
for the API in EDS <= 3.4. With some tricks (dynamically
loading libs) it works across a range of libecal and libebook
releases.
Now the syncevolution-evolution meta package declares that
it depends on one of these libecal/ebook libraries (was missing
earlier), to ensure that the libs get installed on a system
which did not have them already.
The dynamic loading will no longer attempt to work with
EDS 3.6. For EDS 3.6 it will be necessary to update the EClient
variant of the EDS backend and compile the syncevolution.org
binaries differently.
Avoid the abstract base class and implement the GDBusXX::Watch
class directly. This became necessary because moving the GIO
implementation into the .cpp resulted in strange crashes around
the virtual setCallback() method - but only when optimization
was enabled. Compiler bug?!
Simplifying the class hierarchy is the right step either way. It was
done with separate classes initially to avoid dependencies on
gdbus-cxx-bridge.h in code which just needs to interact with the base
functionality, but it turned out that no code needs that.
The previous implementation did not work at all. It relied
on "NameLost" signals with the watched D-Bus name as sender,
but that is not how the signal works. It is sent by the D-Bus
daemon to the old owner of that name if it looses the name.
Instead we need to watch the "NameOwnerChanged" signal and
detect when the peer we are watching is the one who disconnected
from the bus.
Also moved the actual code into the .cpp file because there is
no need to inline all of it.
This problem was found after adding more failure tests to
test-dbus.py. The real-world effect was that syncevo-dbus-server did
not clean up properly when clients disconnected early and that the
command line kept running after syncevo-dbus-server crashed. Did not
affect syncevolution.org binaries, which do not use D-Bus GIO.
The static template method doesn't have to be in the header
file (doesn't depend on template parameters). Moving it into
the .cpp file to minimize compile and executable size.
Runtime work can be minimized by holding the instance
temporarily in a simpler auto pointer before moving it
into the final shared pointer.
Another reason was that compiling with optimization enabled
produced binaries which crashed in that code. However, moving
it did not help.
An asynchronous D-Bus call will invoke the callback even if
the call instance itself was already deleted. Therefore binding
the this pointer of the call instance owner is not safe. This
caused use-after-free errors in local sync (found during testing).
Add a weak pointer to itself to each LocalTransportAgent
instance and use that weak pointers so that the callback
does not crash when it happens to late.
An asynchronous D-Bus call will invoke the callback even if
the call instance itself was already deleted. Therefore binding
the this pointer of the call instance owner is not safe. This
caused use-after-free errors in local sync (found during testing).
Use weak pointers as a precaution where it can be done easily.
Some remaining start() calls still need to be converted.
On several occasions, boost::bind() is used with a this
pointer as second parameter. This is only safe as long as
the functor is only called when the instance is still valid.
After including BoostHelper.h it is also possible to use
a weak pointer. The result is a functor which does not
prevent deleting the instance and doesn't do anything
when the instance is gone when called.
Nightly builds with --enable-doc tended to fail occassionally due to
download errors of the current docbook XSL from SourceForge. Using
a local copy of those files avoids that problem, speeds up compilation
and gives us some control over potentially incompatible changes
in the upstream docbook XSL.
Google Calendar sometimes returns redirect requests to human-readable
web sites (an "unavailable" page, a login form). This is of course
bogus when the client is an automated CalDAV client.
The "unavailable.html" case was already handled. Made it a bit more
flexible to also catch possible variations of it (additional
parameters, https instead of http).
Added the https://accounts.google.com/ServiceLogin case. Not sure
whether retrying will help in that case, but there's not much else
that SyncEvolution can do.
When asked to insert a VJOURNAL which already existed (= same UID),
CalDAV servers respond with a 412 "Precondition failed" error. This
needs to be detected and translated into an "item needs to be merged"
result so that the engine can load the existing item, merge the data,
and then write back.
A test for this, testInsertTwice, will be committed separately. The
code was written so that it handles the same error when using CardDAV.
However, this was not tested because CardDAV test data does not have a
UID (wouldn't trigger the problem) and Radicale did not report 412 when
adding the UID.
Instead of relying on catching an exception, better use the new
"expected status codes" feature and check for 412 as part of the
normal switch statement.
Some WebDAV requests might fail with a non-okay status code that the
caller expects. This was reported via an exception. But
SyncEvolution's design uses exceptions only for non-normal
incidencences. Therefore better allow the caller to indicate which
status codes are expected and return normally from run() and
checkError() when those are encountered, without retrying and without
throwing an exception.
Funambol's OneMedia sends UID, but not RECURRENCE-ID. That becomes a
problem when multiple events of the same event series are added to a
backend which follows the iCalendar 2.0 standard (CalDAV, EDS, KDE),
because these events all look like the master event, and there can be
only one of those.
SyncEvolution now strips the UID from all events coming from any
Funambol server (regardless of the version). If a future Funambol
server release adds support for both UID and RECURRENCE-ID, then
SyncEvolution will have to be updated to take advantage of the
improved server. Because the RECURRENCE-ID is also getting
stripped (despite not being set at the moment), SyncEvolution should
continue to work as it does now even if the server changes.
It would have been nice to limit this workaround to affected Funambol
server versions, but an inquiry on the Funambol mailing list didn't
get a reply, therefore SyncEvolution is playing it safe and assumes
that all future Funambol releases will have the same problem.
In some cases data with a very long UID wasn't handled correctly,
causing the out-going data to be malformed and probably causing a
rejection by the server.
The root-cause was two-fold:
- extractUID() didn't expect folding. Normally it deals only with
data encoded by libsynthesis, which does not use folding; the unexpected
exception was data which gets imported directly (--import).
- setResourceName() used std::string::replace() incorrectly: second
parameter is length, not end offset, of the data to be replaced.
When writing calendar items into a backend storage as iCalendar 2.0 or
vCalendar 1.0, they should have DTSTAMP and LAST-MODIFIED values. DTSTAMP
is expected by some CalDAV servers (like Apple). LAST-MODIFIED is usually
added by the storage, but not always.
In the text/plain -> syncevolution -> text/calendar -> Radicale -> EDS
-> syncevolution chain the LAST-MODIFIED was not added by Radicale, which caused
problems for change tracking in an EDS-based SyncEvolution.
The command line --status operation did not complete when applied to a
CalDAV/CardDAV source. Instead it aborted because the operation took a
code path where the backend was not fully initialized.
