libical 1.0 started to return VTIMEZONE definitions with multiple
absolute transition times instead of RRULEs. This causes problems when
exchanging data with peers (see
https://sourceforge.net/p/freeassociation/bugs/95/).
In SyncEvolution, this affected sending an event using New Zealand
time in vCalendar 1.0 format to a phone, because the internal,
out-dated definition of the time zone in libsynthesis was used as
fallback when loading RRULE-based timezone definitions from libical
failed (see "[SyncEvolution] Some events showing wrong time on
phone"). It might also affect exchanging data with CalDAV peers (not
tested).
The workaround is to include the original code from libical from
before the change in SyncEvolution and override
icaltimezone_get_component() with a version that uses the original
timezone loading code. This does not fix cases where other code causes
libical itself to load a timezone, but for libsynthesis it is good
enough because it does the loading early when no other code should
have used libical.
The downside is that now we need to maintain the RRULE heuristics and
ensure that they actually work. Copying libical/src/test/timezones.c
would be useful for that.
Long-term it would be good to enhance libical itself such that it can
return a VTIMEZONE with suitable RRULEs matching a specific event,
point in time or time range.
On some platforms, LDFLAGS is used during configure to set rpath
options. runtests.py must pass the -no-install to the linker
of client-test without overwriting those flags.
Only the "client-test" executable and its test data were installed so far, in
/usr/bin resp. the doc dir. Now also test-dbus.py and testpim.py plus their
data files get installed, using a new /usr/lib/syncevolution/test directory
for all test files and commands.
"runtests.py --prebuilt" can take one or more .deb files including these
tests, will install them as the "compile" operation and then will use the
installed SyncEvolution, without overriding any paths. This relies on having
suitable rights for "dpkg" and /usr/lib/syncevolution/test; the way how that
is done in the nightly testing is via chroots where the entire file system
is writable by the normal test user.
Normal users without such write access to /usr/lib/syncevolution/test need
to copy that directory into their home directory first.
When compiling source files of client-test, use -g as default CXXFLAGS
instead of the "-g -O2" that autotools normally picks. That speeds up
compilation significantly (on some platforms, gcc can't deal with the
many templates in ClientTest.cpp well) and leads to more useful
executables (suitable for interactive debugging) even when the rest of
the code gets optimized.
Explicitly specifying CXXFLAGS still overrides this per-target
default.
This feature depends on GNU make. A configure check is in place
to disable it when not using GNU make.
Compilation takes a lot longer when using -O2 and the result
is less debugger friendly, so filter out the flag. This is
relevant when building a release where the flag is set for the
rest of the compilation.
Initial step towards using SyncEvolution, PBAP and libfolks in the
context of IVI (in-vehicle-infotainment): D-Bus API definition for the
org._01.pim.contact API, --enable-dbus-service-pim, find libs, compile
into syncevo-dbus-server and client-test.
The only functional code at this time is the unit testing of libfolks,
GValueCXX and libgee.
As mentioned by Tino Keitel on the mailing list, some libs and
executables were only implicitly linked against libraries that they
called directly. This happened to work by chance because these libraries
ended up in the running executable anyway, due to indirect loading.
To catch such problems, the "make installcheck" was extended:
dpkg-shlibdeps is run, if available, and the error output is scanned
for the messages which indicate that a symbol is used without linking
to the right library (example output below).
Had to fix quite a few _LIBADD lines to pass the test.
Some exceptions are allowed:
- libsmltk depends on the caller providing SySync logging support.
- libneon is intentionally not linked explicitly for syncevolution.org
binaries, to make resulting binaries work with GNUTLS and OpenSSL.
dpkg-shlibdeps: warning: debian/syncevolution-libs/usr/lib/syncevolution/backends/syncdav.so contains an unresolvable reference to symbol icalparameter_new_from_value_string: it's probably a plugin.
dpkg-shlibdeps: warning: 51 other similar warnings have been skipped (use -v to see them all).
...
dpkg-shlibdeps: warning: symbol dlsym used by debian/libsyncevolution0/usr/lib/libsyncevolution.so.0.0.0 found in none of the libraries.
dpkg-shlibdeps: warning: symbol dlerror used by debian/libsyncevolution0/usr/lib/libsyncevolution.so.0.0.0 found in none of the libraries.
dpkg-shlibdeps: warning: symbol dlopen used by debian/libsyncevolution0/usr/lib/libsyncevolution.so.0.0.0 found in none of the libraries.
KWallet support was broken: syncevo-dbus-server checked
KDE_FULL_SESSION to determine whether it should use KWallet instead of
GNOME Keyring. That did not work, because the env variable was not set
for D-Bus daemons.
Automatically detecting KDE users is not possible at the
moment. Instead KDE users have to manually set the new "keyring"
global config property to "KDE" (case insensitive) if the
SyncEvolution installation supports both, because GNOME Keyring is the
default to avoid surprises for traditional users. If only KWallet
support is enabled, then this is not necessary.
"GNOME" and "true/false/1/0/yes/no" can also be set. This has the
advantage that keyring usage can be enabled permanently for the
command line in --daemon=no mode; normally keyrings are not used in
that mode because accessing them can bring up UI dialogs.
It also becomes possible to disable keyring usage in syncevo-dbus-server,
something which couldn't be done before.
The --keyring command line option is still supported, as an alias for
"[--sync-property] keyring=<value>". The default value for --keyring
is true, to match the traditional behavior. In contrast to other sync
properties, setting "keyring" does not require an explicit --run
parameter. Again this is done to mirror traditional usage.
Reading a password also (unintentionally) checked all supported
storages while searching for the password. Now it uses exactly
one storage and falls back to asking for the password directly.
The commit itself also cleans up the code a bit (reformatted, fixed
comments). Choosing the right slot in the password signals is done via
a new InitStateTri parameter which contains the "keyring" setting.
Error checking (unsupported keyring string, --keyring=yes and no
keyring enabled) is done in additional slots which run after all the
regular ones.
Parameter parsing for --sync and --keyring were unified. However,
there is the difference that --keyring has an implicit default value
("yes") and never has an additional parameter, in contrast to --sync,
which always is followed by one.
The new CmdlineTest::testKeyring covers different ways of using
--keyring. It relies on actually invoking keyring backends, something
not done by the default SyncContext UI. Therefore
CmdlineSyncClient+KeyringSyncCmdline were moved into libsyncevolution,
to be used by CmdlineTest.
This commit moves the blocking syncing, database restore and command
line execution into a separate, short-lived process executing the
syncevo-dbus-helper. The advantage is that the main
syncevo-dbus-server remains responsive under all circumstances (fully
asynchronous now) and suffers less from memory leaks and/or crashes
during a sync.
The core idea behind the new architecture is that Session remains the
D-Bus facing side of a session. It continues to run inside
syncevo-dbus-server and uses the syncevo-dbus-helper transparently via
a custom D-Bus interface between the two processes. State changes of
the helper are mirrored in the server.
Later the helper might also be used multiple times in a Session. For
example, anything related to loading backends should be moved into the
helper (currently the "is config usable" check still runs in the
syncevo-dbus-server and needs to load/initialize backends). The
startup code of the helper already handles that (see boolean result of
operation callback), but it is not used yet in practice.
At the moment, only the helper provides a D-Bus API. It sends out
signals when it needs information from the server. The server watches
those and replies when ready. The helper monitors the connection to
the parent and detects that it won't get an answer if that connection
goes down.
The problem of "helper died unexpectedly" is also handled, by not
returning a D-Bus method reply until the requested operation is
completed (different from the way how the public D-Bus API is
defined!).
The Connection class continues to use such a Session, as before. It's
now fully asynchronous and exchanges messages with the helper via the
Session class.
Inside syncevo-dbus-server, boost::signals2 and the dbus-callbacks
infrastructure for asynchronous methods execution are used heavily
now. The glib event loop is entered exactly once and only left to shut
down.
Inside syncevo-dbus-helper, the event loop is entered only as
needed. Password requests sent from syncevo-local-sync to
syncevo-dbus-helper are handled asynchronously inside the event loop
driven by the local transport.
syncevo-dbus-helper and syncevo-local-sync are conceptually very
similar. Should investigate whether a single executable can serve both
functions.
The AutoSyncManager was completely rewritten. The data structure is a
lot simpler now (basically just a cache of transient information about
a sync config and the relevant config properties that define auto
syncing). The main work happens inside the schedule() call, which
verifies whether a session can run and, if not possible for some
reasons, ensures that it gets invoked again when that blocker is
gone (timeout over, server idle, etc.). The new code also uses
signals/slots instead of explicit coupling between the different
classes.
All code still lives inside the src/dbus/server directory. This
simplifies checking differences in partly modified files like
dbus-sync.cpp. A future commit will move the helper files.
The syslog logger code is referenced by the server, but never used.
This functionality needs further thought:
- Make usage depend on command line option? Beware that test-dbus.py
looks for the "ready to run" output and thus startup breaks when
all output goes to syslog instead of stdout.
- Redirect glib messages into syslog (done by LogRedirect, disabled when
using LoggerSyslog)?
The syncevo-dbus-server now sends the final "Session.StatusChanged
done" signal immediately. The old implementation accidentally delayed
sending that for 100 seconds. The revised test-dbus.py checks for
more "session done" quit events to cover this fix.
Only user-visible messages should have the INFO level in any of the
helpers. Messages about starting and stopping processes are related to
implementation details and thus should only have DEBUG level.
The user also doesn't care about where the operation eventually
runs. All messages related to it should be in INFO/DEBUG/ERROR
messages without a process name. Therefore now syncevo-dbus-server
logs with a process name (also makes some explicit argv[0] logging
redundant; requires changes in test-dbus.py) and syncevo-dbus-helper
doesn't.
syncevo-local-sync is different from syncevo-dbus-helper: it produces
user-relevant output (the other half of the local sync). It's output
is carefully chosen so that the process name is something the user
understands (target context) and output can be clearly related to one
side or the other (for example, context names are included in the sync
table).
Output handling is based on the same idea as output handling in the
syncevo-dbus-server:
- Session registers itself as the top-most logger and sends
SyncEvolution logging via D-Bus to the parent, which re-sends
it with the right D-Bus object path as output of the session.
- Output redirection catches all other output and feeds it back
to the Session log handler, from where it goes via D-Bus
to the parent.
The advantage of this approach is that level information is made
available directly to the parent and that message boundaries are
preserved properly.
stderr and stdout are redirected into the parent and logged there as
error. Normally the child should not print anything. While it runs,
LogRedirect inside it will capture output and log it
internally. Anything reaching the parent thus must be from early
process startup or shutdown.
Almost all communication from syncevo-dbus-helper to
syncevo-dbus-server is purely information for the syncevo-dbus-server;
syncevo-dbus-helper doesn't care whether the signal can be
delivered. The only exception is the information request, which must
succeed.
Instead of catching exceptions everywhere, the optional signals are
declared as such in the EmitSignal template parameterization and
no longer throw exceptions when something goes wrong. They also don't
log anything, because that could lead to quite a lof of output.
The old ui.glade was still being referenced by the list of translatable
files. The corresponding files from the GTK3 version were missing.
automake doesn't seem to handle conditional _DATA files correctly. In contrast
to source files, they end up not being distributed when disabled. To solve
this, src/gtk*-ui/gtk-ui.am must be included unconditionally in src.am and
contain unconditional EXTRA_DIST entries for the two ui.xml files.
We want to support both GTK+-2.0 and GTK+-3.0 for the time being
(even if the former is just maintenance). This is not possible in
the same codebase without large amounts of ifdefs so we fork the UI.
The platform specific code which is of no value unless you run a
specific desktop now gets compiled as part of shared libraries, just
like the storage backends. The advantage is that the rest of
SyncEvolution keeps running even if one of these shared libraries
cannot be loaded due to missing depdendencies. syncevolution.org
packages will not declared these dependencies, to allow installing
each package without forcing the installation of unwanted libraries.
Distros can package the platform code separately.
Another advantage is reduced code duplication (password load/store
was duplicated in command line and D-Bus server).
Technically this uses almost the same mechnism as loadable sync
sources. The code resides in src/backends/[kde|gnome], where the
autotool magic finds the *Register.cpp files automatically and
includes them into executables. These files contain global singletons
which, when initialized, connect platform specific code to new signals
in the core (init, password load/save).
The actual code is in the backend libraries. Because
SE_ARG_ENABLE_BACKEND() is not used (in favor of the traditional
enable macros), linking against these libs must be set up by adding
them to the (now slightly misnamed) SYNCSOURCES variable in the
configure fragments.
libsyncevolution depends on GDBus, which therefore must be dealt
with first.
Mostly the dependencies take care of the right compilation order.
Installing the results is different, items of the same kind are
installed in the order in which they are added to their variables,
which in turn is defined by the order in which .am files are
included.
That broke when making libgdbussyncevo a normal lib like
libsyncevolution (see next commit).
Due to bitrot the Akonadi backend and KWallet support code no longer
worked. Moved the common code for KApplication initialization into
libsyncevolution's SyncContext::initMain() and fixed autotools rules.
The old code always tried to contact an X server (default constructor
of KApplication). That doesn't seem to be necessary and is avoided now.
Even better might be to skip KApplication entirely and instead use
QCoreApplication and KComponentData, as suggested by
http://api.kde.org/4.x-api/kdelibs-apidocs/kdeui/html/classKApplication.html
KAboutData was incorrectly passed the address of a string pointer, not
the pointer itself.
Testing the Akonadi backend in client-test failed because client-test
always overwrites the "backend" value with
"Test_kde_[contact/event/..]._[1/2]". Now this special case is
detected. The backend then uses the first resp. second resource that
it finds.
Instead of forking and continuing to sync in the forked process
without an explicit exec(), exec() the 'syncevo-local-sync' helper in
the forked process. The syncevo-local-sync helper binary gets
installed into libexec. SYNCEVOLUTION_LIBEXEC_DIR must be set if that
helper is not installed yet, not in the PATH, or an old version would
be found without that env variable ("make" without "make install"
during development!).
Main advantage is the cleaner environment for running the child side
of local sync. Required for getting ActiveSync to work again (GDBus
GIO as used by recent activesyncd client libraries did not work in the
forked process without the exec()).
Full D-Bus communication gets established between parent and child.
The downside is the hard dependency of local sync on the D-Bus
libraries (not the daemon!).
D-Bus communication allowed implementing interactive password requests
from the child side through the parent to the UI using the parent,
something that wasn't implemented before.
The child asks its parent for the password, which in turn
passes the request to its SyncContext. This happens to work
when that SyncContext is a normal instance (reads from stdin,
the "syncevolution --daemon" case) and the syncevo-dbus-server
(sends out an Info Request signal and waits for a response).
The info request and response are handled in the blocking
askPassword() by polling the running mail loop, so the parent should
remain responsive. Overall it is still a pretty difficult setup; it
would be better if askPassword() was asynchronous.
Describing the required password also is sub-optimal: the sync-ui just
asks for a password in its current config (even though that might not
be the config which currently gets synced) and crashes if no config is
currently selected. The command line uses the description derived from
the property and config name, which is a bit technical, but at least
correct.
Syncing uses the child's error string as "first error" in the parent,
too, by logging it anew on the parent side. That puts it into the
parent's sync report ahead of any additional error that it might
encounter while the child shuts down. Also use the child's status when
available instead of a misleading TransportError.
In addition, suppress as many of these errors as possible when we know
already that the child reported an error in its sync report. Not all
"transport errors" are currently avoided that way, but this is at
least a first step.
src/src.am still used its local variables instead of the top-level
ones. In addition, it got the paths slightly wrong, causing build
errors in the nighty testing.
Conflicts:
Makefile.am
configure-post.in
configure.ac
src/Makefile-gen.am
src/dbus/qt/Makefile.am
src/syncevo-dbus-server.cpp
test/test-dbus.py
Conflicts caused by renaming of files on master. Manually applied the
relevant changes to the renamed files.
When compiling libsynthesis as part of the normal make invocation,
libsyncevolution.la had a dependency on libsynthesis.la, but no rule
to generate it. That worked for "make all" (because it entered the
subdir first), but not for "make <something>" where <something>
dependended on libsynthesis.
This commit adds such a rule. It is enough for initial builds, but not
complete yet because the outer make doesn't know the dependencies of
libsynthesis.la and thus won't trigger rebuilds. Add the complete file
list?
syncEvolution has used an in-tree copy of Bluez's gdbus dbus
implentation.
With this patch there is now a configure switch which enables the use
of the GLib's GDBus dbus implmentation if the --with-gio-gdbus
configure flag is use or, if no flag is used, an adequate glib version
(2.26) is found.
This commit does *not* actually implement the use of GIO's GDBus. The
implmentation is coming soon. Thus only --without-gio-gdbus will
currently result in a working build.
Added the necessary autotools rules to use pcre in the src binaries
and libsyncevolution. It's unconditionally required now, which makes
it possible to add code depending on it without having to worry about
the case without pcre.
The alternative to pcre with its C++ binding would have been
boost::Xpressive. pcre was chosen because it has UTF-8 support and is
more common.
boost::regex introduces an undesirable library dependency. The C POSIX
regex code has no C++ binding and does not support easy replacement.
README was updated accordingly.
The HTML version of the .log files links to ClientTest.cpp.html,
colorizes important parts and links to the sync session
directories.
Because it is much more useful than the plain text version, the
nightly.html now links to these .html files.
ClientTest.cpp.html is built with Python pygments if installed,
otherwise some builtin fallback code is used.
Exceptions reported by CppUnit only contained one file+line pair. If
that location was called multiple times inside a larger test, then it
was impossible to tell where it was called. The new assertion macros
and in particular CT_ASSERT_NO_THROW() solve this problem by catching
exceptions, adding the current file+line information and then
rethrowing an extended exception. When CppUnit finally logs the
problem, it will contain a complete call stack.
For this to work, every single line which might throw an exception
must be wrapped in a macro. Entering and leaving the line is logged
together with the wrapped expression as part of the test .log file.
doSync() is handled as a special case and gets the file+line info of
its caller via parameters.
New logging macros are introduced and used in LocalTests::testChanges:
instead of writing comments, call the logging macros and the string
will appear also in the .log file of the test.
Further areas for improvements:
- use CLIENT_TEST_LOG() everywhere
- reduce file names to just the base name is logged
- convert .log file into HTML with links into session logs and
ClientTest.cpp source file
The bundled libsynthesis source needs to go into "src/synthesis".
None of the XML config files and templates were installed when using
out-of-tree compilation in a clean build directory, because then the
$(wildcard) macro didn't actually match any files. Now it checks the
source directory and the extra directory prefix is removed before
using the resulting file list.
All but toplevel Makefile.am are replaced with their non-recursive
counterparts. The generation of configure.in was removed (and thus
configure-{pre,post}.in are also removed) in favor of configure.ac
and m4 macros adding backend specific configure parts.
Version number is generated like in old build system.
There are still many things to improve, but for now there are no
immediate regressions. AUTOTOOLS-TODO contains a list of possible
improvements and fixes. AUTOTOOLS-TESTING contains what was tested
with current build system (configure flags, make options).
