Klocwork complained about not checking the result. Depending on the
situation we now give up or use some kind of fallback (for example,
when logging).
SyncEvolution itself is not multithreaded, but utility libraries
might be, so better avoid the localtime() function and its global
state. May also be relevant when function A gets a tm * pointer,
calls function B which calls localtime() again, then A tries to
use its original value.
Without the explicit TYPE=INTERNET, email addresses sent to a Nokia
e51 were not shown by the phone and even got lost eventually (when
syncing back?).
This commit ensures that the type is set for all emails sent to any
Nokia phone, because there may be other phones which need it and
phones which don't, shouldn't mind. This was spot-checked with a N97
mini, which works fine with and without the INTERNET type.
This behavior can be disabled again for specific Nokia phones by
adding a remote rule which sets the addInternetEmail session variable
to FALSE again.
Non-Nokia phones can enable the feature in a similar way, by setting
the variable to TRUE.
Some peers have problems with meta data (CtCap, old Nokia phones) and
the sync mode extensions required for advertising the restart
capability (Oracle Beehive).
Because the problem occurs when SyncEvolution contacts the peers
before it gets the device information from the peer, dynamic rules
based on the peer identifiers cannot be used. Instead the local config
must already disable these extra features in advance.
The "SyncMLVersion" property gets extended for this. Instead of just
"SyncMLVersion = 1.0" (as before) it now becomes possible to say
"SyncMLVersion = 1.0, noctcap, norestart".
"noctcap" disables sending CtCap. "norestart" disables the sync mode
extensions and thus doing multiple sync cycles in the same session
(used between SyncEvolution instances in some cases to get client and
server into sync in one session).
Both keywords are case-insensitive. There's no error checking for
typos, so beware!
The "SyncMLVersion" property was chosen because it was already in use
for configuring SyncML compatibility aspects and adding a new property
would have been harder.
In the previous attempt (commit a0375e) setting the <syncmodeextensions>
option was only done on the client side, thus breaking multi-cycle
syncing with SyncEvolution as server. On the server side the option
shouldn't be needed (the server never sends the extensions unless
the client did first), but for symmetry and testing reasons it makes
sense to also offer the options there.
This reverts commit 3a71a2bf53 and
commit a0375e0160.
Need to back out the workaround for broken peers because it breaks
SyncEvolution<->SyncEvolution syncing.
Avoid writing config file changes to disk by enabling a new
"ephemeral" mode for syncing via the PIM Manager. This mode is enabled
via the sync mode parameter of the D-Bus API and from there passed
through to the SyncConfig and local sync helper. It cannot be enabled
in the config files (yet?).
In this mode, config file changes are not flushed resp. discarded
directly in the config nodes. This prevents writing to .ini files in
~/.config.
The "synthesis" binfile client files are still written, but they get
redirected into the session directory, which can (and should) be set
to a temp file system and get deleted again quickly.
Data dumps are turned off now in the configs created by the PIM
Manager.
Some peers have problems with meta data (CtCap, old Nokia phones) and
the sync mode extensions required for advertising the restart
capability (Oracle Beehive).
Because the problem occurs when SyncEvolution contacts the peers
before it gets the device information from the peer, dynamic rules
based on the peer identifiers cannot be used. Instead the local config
must already disable these extra features in advance.
The "SyncMLVersion" property gets extended for this. Instead of just
"SyncMLVersion = 1.0" (as before) it now becomes possible to say
"SyncMLVersion = 1.0, noctcap, norestart".
"noctcap" disables sending CtCap. "norestart" disables the sync mode
extensions and thus doing multiple sync cycles in the same session
(used between SyncEvolution instances in some cases to get client and
server into sync in one session).
Both keywords are case-insensitive. There's no error checking for
typos, so beware!
The "SyncMLVersion" property was chosen because it was already in use
for configuring SyncML compatibility aspects and adding a new property
would have been harder.
Fetched the code and its history from the 1.3.1 archives at:
http://people.debian.org/~ovek/maemo/http://people.debian.org/~ovek/harmattan/
Merged almost everything, except for Maemo/Harmattan specific build files:
autogen-maemo.sh builddeb buildsrc debian
The following changes were also removed, because they are either local
workarounds or merge artifacts which probably also don't belong into
the Maemo/Harmattan branch:
diff --git a/configure.ac b/configure.ac
index cb66617..2c4403c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -44,7 +44,7 @@ if test "$enable_release_mode" = "yes"; then
AC_DEFINE(SYNCEVOLUTION_STABLE_RELEASE, 1, [binary is meant for end-users])
fi
-AM_INIT_AUTOMAKE([1.11.1 tar-ustar silent-rules subdir-objects -Wno-portability])
+AM_INIT_AUTOMAKE([subdir-objects -Wno-portability])
AM_PROG_CC_C_O
diff --git a/src/backends/webdav/CalDAVSource.cpp b/src/backends/webdav/CalDAVSource.cpp
index decd170..7d338ac 100644
--- a/src/backends/webdav/CalDAVSource.cpp
+++ b/src/backends/webdav/CalDAVSource.cpp
@@ -1282,6 +1282,7 @@ void CalDAVSource::Event::fixIncomingCalendar(icalcomponent *calendar)
// time.
bool ridInUTC = false;
const icaltimezone *zone = NULL;
+ icalcomponent *parent = NULL;
for (icalcomponent *comp = icalcomponent_get_first_component(calendar, ICAL_VEVENT_COMPONENT);
comp;
@@ -1295,6 +1296,7 @@ void CalDAVSource::Event::fixIncomingCalendar(icalcomponent *calendar)
// is parent event? -> remember time zone unless it is UTC
static const struct icaltimetype null = { 0 };
if (!memcmp(&rid, &null, sizeof(null))) {
+ parent = comp;
struct icaltimetype dtstart = icalcomponent_get_dtstart(comp);
if (!icaltime_is_utc(dtstart)) {
zone = icaltime_get_timezone(dtstart);
diff --git a/src/backends/webdav/CalDAVSource.h b/src/backends/webdav/CalDAVSource.h
index 517ac2f..fa7c2ca 100644
--- a/src/backends/webdav/CalDAVSource.h
+++ b/src/backends/webdav/CalDAVSource.h
@@ -45,6 +45,10 @@ class CalDAVSource : public WebDAVSource,
virtual void removeMergedItem(const std::string &luid);
virtual void flushItem(const string &uid);
virtual std::string getSubDescription(const string &uid, const string &subid);
+ virtual void updateSynthesisInfo(SynthesisInfo &info,
+ XMLConfigFragments &fragments) {
+ info.m_backendRule = "HAVE-SYNCEVOLUTION-EXDATE-DETACHED";
+ }
// implementation of SyncSourceLogging callback
virtual std::string getDescription(const string &luid);
Making SySync_ConsolePrintf a real instance inside SyncEvolution leads
to link errors in other configurations. It really has to be extern. Added
a comment to the master branch to make that more obvious:
-extern "C" { // without curly braces, g++ 4.2 thinks the variable is extern
- int (*SySync_ConsolePrintf)(FILE *stream, const char *format, ...);
-}
+// This is just the declaration. The actual function pointer instance
+// is inside libsynthesis, which, for historic purposes, doesn't define
+// it in its header files (yet).
+extern "C" int (*SySync_ConsolePrintf)(FILE *stream, const char *format, ...);
In the past, using curl as HTTP transport in the syncevo-dbus-server
was prevented, leading to "unsupported transport type is specified in
the configuration". The reason was that using curl would block the
server and make it unresponsive on D-Bus.
This reason has gone away, because now the HTTP traffic happens in a
separate process. Thus now it is allowed to use curl in the
syncevo-dbus-server.
This patch introduces support for true one-way syncing ("caching"):
the local datastore is meant to be an exact copy of the data on the
remote side. The assumption is that no modifications are ever made
locally outside of syncing. This is different from one-way sync modes,
which allows local changes and only temporarily disables sending them
to the remote side.
Another goal of the new mode is to avoid data writes as much as
possible.
This new mode only works on the server side of a sync, where the
engine has enough control over the data flow.
Most of the changes are in libsynthesis. SyncEvolution only needs to
enable the new mode, which is done via an extension of the "sync"
property:
- "local-cache-incremental" will do an incremental sync (if possible)
or a slow sync (otherwise). This is usually the right mode to use,
and thus has "local-cache" as alias.
- "local-cache-slow" will always do a slow sync. Useful for
debugging or after (accidentally) making changes on the server side.
An incremental sync will ignore such changes because they are not
meant to happen and thus leave client and sync out-of-sync!
Both modes are recorded in the sync report of the local side. The
target side is the client and records the normal "two-way" or "slow"
sync modes.
With the current SyncEvolution contact field list, first, middle and
last name are used to find matches during any kind of slow sync. The
organization field is ignored for matching during the initial slow
sync and used in all following ones. That's okay, the difference won't
matter in practice because the initial slow sync in PBAP caching will
be done with no local data. The test achieve the same result in both
cases by keeping the organization set in the reduced data set.
It's also okay to include the property in the comparison, because it
might help to distinguish between "John Doe" in different companies.
It might be worthwhile to add more fields as match criteria, for
example the birthday. Currently they are excluded, probably because
they are not trusted to be supported by SyncML peers. In caching mode
the situation is different, because all our data came from the peer.
The downside is that in cases where matching has to be done all the
time because change detection is not supported (PBAP), including the
birthday as criteria will cause unnecessary contact removed/added
events (and thus disk IO) when a contact was originally created
without birthday locally and then a birthday gets added on the phone.
Testing is done as part of the D-Bus testing framework, because usually
this functionality will be used as part of the D-Bus server and writing
tests in Python is easier.
A new test class "TestLocalCache" contains the new tests. They include
tests for removing extra items during a slow sync (testItemRemoval),
adding new client items under various conditions (testItemAdd*) and
updating/removing an item during incremental syncing
(testItemUpdate/Delete*). Doing these changes during a slow sync could
also be tested (not currently covered).
The tests for removing properties (testPropertyRemoval*) cover
removing almost all contact properties during an initial slow sync, a
second slow sync (which is treated differently in libsynthesis, see
merge=always and merge=slowsync), and an incremental sync.
Instead of using delays to kill processes at the right time, watch old
and new debug output via D-Bus and then kill the processes. To avoid
race conditions, these processes get delayed at the right point.
Added tests for local sync and command line, covering killing of
all involved processes.
An asynchronous D-Bus call will invoke the callback even if
the call instance itself was already deleted. Therefore binding
the this pointer of the call instance owner is not safe. This
caused use-after-free errors in local sync (found during testing).
Add a weak pointer to itself to each LocalTransportAgent
instance and use that weak pointers so that the callback
does not crash when it happens to late.
Funambol's OneMedia sends UID, but not RECURRENCE-ID. That becomes a
problem when multiple events of the same event series are added to a
backend which follows the iCalendar 2.0 standard (CalDAV, EDS, KDE),
because these events all look like the master event, and there can be
only one of those.
SyncEvolution now strips the UID from all events coming from any
Funambol server (regardless of the version). If a future Funambol
server release adds support for both UID and RECURRENCE-ID, then
SyncEvolution will have to be updated to take advantage of the
improved server. Because the RECURRENCE-ID is also getting
stripped (despite not being set at the moment), SyncEvolution should
continue to work as it does now even if the server changes.
It would have been nice to limit this workaround to affected Funambol
server versions, but an inquiry on the Funambol mailing list didn't
get a reply, therefore SyncEvolution is playing it safe and assumes
that all future Funambol releases will have the same problem.
Added a "target side of local sync ready" INFO message to introduce
the output which has the target context in the [INFO] tag. The sync report
from the target side now has the target context embedded in brackets
after the "Changes applied during synchronization" header, to avoid
ambiguities.
Output from the target side of a local sync was passed through stderr
redirection as chunks of text to the frontends. This had several
drawbacks:
- forwarding only happened when the local sync parent was processing
the output redirection, which (due to limitations of the implementation)
only happens when it needs to print something itself
- debug messages were not forwarded
- message boundaries might have been lost
In particular INFO messages about delays on the target side are
relevant while the sync runs and need to be shown immediately.
Now the output is passed through D-Bus, which happens immediately,
preserves message boundaries and is done for all output. The frontend
can decide separately whether it shows debug messages (not currently
supported by the command line tool).
Implementing this required extending the D-Bus API. The
Server.LogOutput signal now has an additional "process name"
parameter. Normally it is empty. For messages originating from the
target side, it carries that extra target context string.
This D-Bus API change is backward compatible. Older clients can still
subscribe to and decode the LogOutput messages, they'll simply ignore
the extra parameter. Newer clients expecting that extra parameter
won't work with an older D-Bus daemon: they'll fail to decode the
D-Bus message.
This revealed that the last error messages in a session was
incorrectly attributed to the syncevo-dbus-server. Might also have
happened with several other error messages. Now everything happening
in the server while working on code related to a session is logged as
coming from that sessions. It's not perfect either (some of the output
could be from unrelated events encountered indirectly while running
that code), but it should be better than before.
The patch changes the handling or errors in the local sync parent
slightly: because it logs real ERROR messages now instead of plain
text, it'll record the child's errors in its own sync report. That's
okay, user's typically shouldn't have to care about where the error
occurred. The D-Bus tests need to be adapted for this, because it
removes the "failure in local sync child" from the sync report.
Another, more internal change is that the syncevo-local-sync helper
does its own output redirection instead of relying on the stderr
handling of the parent. That way libneon debug output ends up in the
log file of the child side (where it belongs) and not in the parent's
log.
When waiting for resending a failed message, the sleeping couldn't be
interrupted when using the D-Bus server. It now uses the SuspendFlags
infrastructure and glib, which detects abort requests sent via D-Bus
in addition to those sent via signals (which already worked earlier).
Google does not implement refresh-from-server, therefore using it has
to be made configurable. It is enabled by default for Funambol and
disabled for everything else.
Existing user configs must be updated to use refresh-from-server with
Funambol:
syncevolution --configure enableRefreshSync=1 funambol
libsynthesis has traditionally implemented "refresh-from-server" as
"delete local data" plus "slow" sync. This is more compatible, because
some servers did not support "refresh-from-server".
But it has the downside that the server cannot know that the client
won't send any data, and Funambol now only allows one slow sync before
blocking the next one for a certain period of time. Probably this is
done to prevent excessive resource usage by badly behaving clients.
SyncEvolution now uses a new configure option in libsynthesis which
enables the use of "refresh-from-server". This is set unconditionally;
all modern SyncML servers are expected to support it.
libsynthesis console is logged with a "SYSYNC: " prefix for each line
if (and only if) SYNCEVOLUTION_DEBUG is set to a value of 4 or higher.
Showing it by default is not useful (because it can be quite verbose,
in particular when a server triggers decoding error messages in SyncML
TK, like Funambol) and using the normal loglevel doesn't help because
that loglevel is not available outside of sync sessions.
Using a config name which refers to a context now triggers
a specific error. The operation that cannot be continued
is named explicitly:
$ ./syncevolution @default
[INFO] Configuration "@default" does not refer to a sync peer.
[ERROR] Cannot proceed with sync without a configuration.
The error message about missing configuration also became
better:
$ ./syncevolution foo@default
[INFO] Configuration "foo@default" does not exist.
[ERROR] Cannot proceed with sync without a configuration.
Due to the way how properties were handled internally, it wasn't
possible to explicitly set a property to its default value. Instead
the property was unset. For example, explicitly setting database= was
not possible.
This is necessary for client-test and ActiveSync, because client-test
needs to know that the testing is expected to run with the default
databases (something which normally is avoided by overwriting empty
database properties).
Now the "is set" state is tracked explicitly in the config storage and
command line property APIs. Unsetting a property via the command line
could be implemented with an explicit command line option, but is not
supported at the moment.
Tests were extended to cover the new functionality and adapted to the
change behavior for "type" migration: syncFormat was empty
already (because the empty string matched the default), but
forceSyncFormat was unnecessarily set explicitly. Now it is not.
Commit 006bcf26 introduced a more versatile replacement for
FileConfigNode, but didn't switch over old code to the replacement
because of a code freeze. Switching now.
Also use the hash variant in VolatileConfigNode, because it is more
efficient by dropping the (in that case irrelevant) order of
properties.
The destructor of LogDir used the SyncReport passed to its
startSession() method. Depending on the destruction order in
sync(), that report was already destructed. Only seen in
very few suspend/resume tests.
Fixed by only accessing the SyncReport when explicitly asked to.
Also cleaned up resource tracking of SafeConfigNode to use
smart pointer instead of plain pointer while touching the code.
The doSync() method does several steps which can take a long time
(for example, send SAN message) before finally entering the while
loop which checks for abort/suspend. In the case that a suspend or
abort request arrives early, doSync() should not complete the
sync setup. Now it checks more often and returns early.
This was found because the fork/exec sync changed the timing so that
the syncevo-dbus-helper ended up being killed by SIGTERM before it
even started syncing. That had the downside that the process didn't
clean up, resulting in many "potentially lost" memory chunks.
Better choose the timing so that doSync() really is ready to handle
the signal. Aborting at that time is also more realistic (normally
sync startup isn't that slow, only valgrind makes it slow).
This fixes an infinite loop when the engine is waiting for data and
the transport is dead: trying to abort just re-triggers the transport
problem, so to abort, the code noq proceeds to shutdown when it all
involved parties (engine, transport) were already notified of the
abort.
This uses the new libsynthesis support for adding and checking entries
in the SyncCap to detect per datastore whether UID/RECURRENCE-ID are
truly globally unique and thus can be used to finding pairs. The
presence of the property alone is no guarantee for that.
Previously this kind of pairing was enabled only for local sync, which
was a hack which didn't work for local backends which didn't support
UID (for example, Maemo 5 calendar). It also didn't work for mixtures
of datastores with and without that kind of support.
"1122583000" was randomly chosen as pseudo sync mode. It is a number
because strings confuse Funambol. Note that SYNCMODESUPPORTED() only
works inside the compare script.
gcc is unhappy about empty format strings when format checking is
enabled. -Wno-format-zero-length did not help because it is not
supported for C++ ("enabled by default" - then why does the compiler
complain?!). pragmas are ugly and hiding them inside the logging
macro definition didn't work anyway.
Therefore the easiest solution is to use "\n" instead of "" - also
makes it explicit that a line break is meant with the call.
The user-visible part of this change is that command line output now
uses the same [ERROR/INFO] prefixes like the rest of SyncEvolution,
instead of "Error:". Several messages were split into [ERROR] and
[INFO] parts on seperate lines. Multi-line messages with such a prefix
now have the prefix at the start of each line. Full sentences start
with captital letters.
All usage errors related to the synopsis of the command line now
include the synopsis, without the detailed documentation of all
options. Some of those errors dumped the full documentation, which was
way too much information and pushed the actual synopsis off the
screen. Some other errors did not include usage information at all.
All output still goes to stdout, stderr is not used at all. Should be
changed in a seperate patch, because currently error messages during
operations like "--export -" get mixed with the result of the
operation.
Technically the output handling was simplified. All output is printed
via the logging system, instead of using a mixture of logging and
streaming into std::cout. The advantage is that it will be easier to
redirect all regular output inside the syncevo-dbus-helper to the
parent. In particular, the following code could be removed:
- the somewhat hacky std::streambuf->logging bridge code (CmdlineStreamBuf)
- SyncContext set/getOutput()
- ostream constructor parameters for Cmdline and derived classes
The new code uses SE_LOG_SHOW() to produce output without prefix. Each
call ends at the next line, regardless whether the string ends in a
newline or not. The LoggerStdout was adapted to behave according to
that expectation, and it inserts the line prefix at the start of each
line - probably didn't matter before, because hardly any (no?!)
message had line breaks.
Because of this implicit newline in the logging code, some newlines
become redundant; SE_LOG_SHOW("") is used to insert an empty line
where needed. Calls to the logging system are minimized if possible by
assembling output in buffers first, to reduce overhead and to adhere
to the "one call per message" guideline.
Testing was adapted accordingly. It's a bit stricter now, too, because
it checks the entire error output instead of just the last line. The
previous use of Cmdline ostreams to capture output from the class was
replaced with loggers which hook into the logging system while the
test runs and store the output. Same with SyncContext testing.
Conflicts:
src/dbus/server/cmdline-wrapper.h
The child in a local sync asks its parent for a password,
which in turn has to use the normal password request mechanism
on that side. Added this test because the code refactoring
of ConfigUserInterface->UserInterface broke that without
causing test failures... this test caught that problem.
All user-facing functions (password handling, reading from
stdin) are now in a dedicated "UserInterface" class, instead of
spreading that between ConfigUserInterface and SyncContext.
SyncContext no longer has the misleading "is a" [Config]UserInterface]
relationship. Instead it "has a" UserInterface instance, which is set
at runtime. Long term the plan is remove the need to subclass
SyncContext. In the local sync that was already possible
(LocalTransportContext->LocalTransportUI).
The guarantee that there always is a usable instance that can be used
without checking for NULL is provided by the getUserInterfaceNonNull()
method, which falls back to a dummy instance if necessary.
Reading data and password from stdin is moved out of the core
libsyncevolution into the syncevolution binary. That way the D-Bus
server and client-test do not accidentally attempt to read from stdin
(has happened when setting up testing incorrectly).
The platform specific code which is of no value unless you run a
specific desktop now gets compiled as part of shared libraries, just
like the storage backends. The advantage is that the rest of
SyncEvolution keeps running even if one of these shared libraries
cannot be loaded due to missing depdendencies. syncevolution.org
packages will not declared these dependencies, to allow installing
each package without forcing the installation of unwanted libraries.
Distros can package the platform code separately.
Another advantage is reduced code duplication (password load/store
was duplicated in command line and D-Bus server).
Technically this uses almost the same mechnism as loadable sync
sources. The code resides in src/backends/[kde|gnome], where the
autotool magic finds the *Register.cpp files automatically and
includes them into executables. These files contain global singletons
which, when initialized, connect platform specific code to new signals
in the core (init, password load/save).
The actual code is in the backend libraries. Because
SE_ARG_ENABLE_BACKEND() is not used (in favor of the traditional
enable macros), linking against these libs must be set up by adding
them to the (now slightly misnamed) SYNCSOURCES variable in the
configure fragments.
A few operations had a callback registration mechanism, but not
all. For testing multi-cycle syncing and injecting concurrent changes
while a sync runs it is necessary to add code for several more
operations (begin data read, add/modify/delete item).
Instead of adding more custom code, this commit replaces the
home-grown callback lists with a boost::function wrapper which
contains pre- and post-signals based on boost::signals2.
Also simplifies the Synthesis DB Plugin implementation by moving the
common code (null check, exception handling) into the wrapper.
This removed the LOCERR_OK return code from some plugin
implementations (end data read, start data write). Implementations
returning LOCERR_OK must be provided by the SyncSource. Done by
SyncSourceSession (used by TrackingSyncSource) automatically.
The "before sync" database dump is always done for a single
database. Adapted the INFO message and function prototype accordingly.
Also detect the case when the "before sync" dump is requested at a
time when it already was done and then just skip it - can happen
when running multiple syncs in a single session.
The long term goal is that using a specific sync source once more in
another, new sync session can be requested per source.
The actual implementation however is per-session, using the new
libsynthesis "restartsync" session variable. This functionality
sits in SyncContext, because it is the one which has access to
the session variables.
Don't overwrite the initially recorded sync mode when the source is
restarted again later. Instead bump the "restart" counter.
To the user this is shown as "number of cycles" in a sync session
in the sync report. "Restart" is the process of entering a new cycle;
hopefully users will understand the concept.
The cycles are also visible in the command line output as multiple
INFO lines:
[INFO] eds_contact: starting first time sync from client (peer is server)
[INFO] creating complete data backup of source eds_contact before sync (enabled with dumpData and needed for printChanges)
Local data changes to be applied during synchronization:
*** eds_contact ***
no changes
[INFO] eds_contact: sent 1/1
[INFO] eds_contact: started
[INFO] eds_contact: first time sync done successfully
[INFO] eds_contact: starting normal sync from client (peer is server) <===
[INFO] eds_contact: started <===
[INFO] eds_contact: normal sync done successfully <===
[INFO] creating complete data backup after sync (enabled with dumpData and needed for printChanges)
Synchronization successful.
Changes applied during synchronization:
+---------------|-----------------------|-----------------------|-CON-+
| | LOCAL | REMOTE | FLI |
| Source | NEW | MOD | DEL | ERR | NEW | MOD | DEL | ERR | CTS |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| eds_contact | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
| refresh-from-local, 2 cycles, 0 KB sent by client, 0 KB received |
| item(s) in database backup: 1 before sync, 1 after it |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| start Tue Feb 7 17:07:49 2012, duration 0:03min |
| synchronization completed successfully |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
The updated "link to source code" feature in libsynthesis
requires that apps select which mode they want to use.
In SyncEvolution we want the "link to Doxygen source"
variant; adding the corresponding <sourcelink>.
Noticed that KApplication messes with signal handlers (although
not necessarily SIGINT/SIGTERM). Either way, restore the state
of those two after creating the KApplication instance.
Added debug logging to valgrindcheck.sh, suspend flags, fork/exec
and main() functions. Helps with tracing where signal handlers
are installed and which processes run.
Due to bitrot the Akonadi backend and KWallet support code no longer
worked. Moved the common code for KApplication initialization into
libsyncevolution's SyncContext::initMain() and fixed autotools rules.
The old code always tried to contact an X server (default constructor
of KApplication). That doesn't seem to be necessary and is avoided now.
Even better might be to skip KApplication entirely and instead use
QCoreApplication and KComponentData, as suggested by
http://api.kde.org/4.x-api/kdelibs-apidocs/kdeui/html/classKApplication.html
KAboutData was incorrectly passed the address of a string pointer, not
the pointer itself.
Testing the Akonadi backend in client-test failed because client-test
always overwrites the "backend" value with
"Test_kde_[contact/event/..]._[1/2]". Now this special case is
detected. The backend then uses the first resp. second resource that
it finds.
Shutting down syncevo-local-sync in a timely manner when
aborting is hard: the process might be stuck in a blocking
call which cannot be made to check the abort request (blocking
libneon, activesyncd client library, ...).
The best that can be done is to let the process be killed by the
SIGTERM. To have some trace of that, catch the signal and log the
signal; there's a slight risk that the logging system is in an
inconsistent state, but overall that risk is minor.
Because syncevo-local-sync catches SIGINT, ForkExec::stop() must send
SIGTERM in addition to SIGINT. To suppress redundant and misleading
ERROR messages when the bad child status is handled, the
ForkExecParent remembers that itself asked the child to stop and only
treats unexpected "killed by signal" results as error.
The local transport must call that stop() in its cancel(). It enters
the "canceled" state which prevents all further communication with the
child, in particular waiting for the child sync report; doing that
would produce another redundant error message about "child exited
without sending report".
Calling stop() in the local transport's shutdown() is no longer
possible, because it would kill the child right away. Before it simply
had no effect, because SIGINT was ignored. This points towards an
unsolved problem: how long should the parent wait for the child after
the sync is done? If the child gets stuck hard after sending its last
message, the parent currently waits forever until the user aborts.
In the sync event loop the caller of the transport must recognize
CANCELED as something which might be desired and thus should not be
logged as ERROR. That way the Synthesis engine is called one more time
with STEPCMD_ABORT also in those cases where the transport itself
detected the abort request first.
Having the signal handling code in SyncContext created an unnecessary
dependency of some classes (in particular the transports) on
SyncContext.h. Now the code is in its own SuspendFlags.cpp/h files.
Cleaning up when the caller is done with signal handling is now part
of the utility class (removed automatically when guard instance is
freed).
The signal handlers now push one byte for each caught signal into a
pipe. That byte tells the rest of the code which message it needs to
print, which cannot be done in the signal handlers (because the
logging code is not reentrant and thus not safe to call from a signal
handler).
Compared to the previous solution, this solves several problems:
- no more race condition between setting and printing the message
- the pipe can be watched in a glib event loop, thus removing
the need to poll at regular intervals; polling is still possible
(and necessary) in those transports which do not integrate with
the event loop (CurlTransport) while it can be removed from
others (SoupTransport, OBEXTransport)
A boost::signal is emitted when the global SuspendFlags change.
Automatic connection management is used to disconnect instances which
are managed by boost::shared_ptr. For example, the current transport's
cancel() method is called when the state changes to "aborted".
The early connection phase of the OBEX transport now also can be
aborted (required cleaning up that transport!).
Currently watching for aborts via the event loop only works for real
Unix signals, but not for "abort" flags set in derived SyncContext
instances. The plan is to change that by allowing a "set abort" on
SuspendFlags and thus making
SyncContext::checkForSuspend/checkForAbort() redundant.
The new class is used as follows:
- syncevolution command line without daemon uses it to control
suspend/abort directly
- syncevolution command line as client of syncevo-dbus-server
connects to the state change signal and relays it to the
syncevo-dbus-server session via D-Bus; now all operations
are protected like that, not just syncing
- syncevo-dbus-server installs its own handlers for SIGINT
and SIGTERM and tries to shut down when either of them
is received. SuspendFlags then doesn't activate its own
handler. Instead that handler is invoked by the
syncevo-dbus-server niam() handler, to suspend or abort
a running sync. Once syncs run in a separate process, the
syncevo-dbus-server should request that these processes
suspend or abort before shutting down itself.
- The syncevo-local-sync helper ignores SIGINT after a sync
has started. It would receive that signal when forked by
syncevolution in non-daemon mode and the user presses
CTRL-C. Now the signal is only handled in the parent
process, which suspends as part of its own side of
the SyncML session and aborts by sending a SIGTERM+SIGINT
to syncevo-local-sync. SIGTERM in syncevo-local-sync is
handled by SuspendFlags and is meant to abort whatever
is going on there at the moment (see below).
Aborting long-running operations like import/export or communication
via CardDAV or ActiveSync still needs further work. The backends need
to check the abort state and return early instead of continuing.
Only the parent process is receiving a sync report. Don't try that in
the child. Relevant for local sync with fork/exec, because there the
child properly deconstructs all classes including its SyncContext.
Dead code, doesn't work. Merely committing as reminder. As before,
anything written by the child's logging code to stdout is printed
directly to the parent's stdout (no redirection anywhere).
The problems with this patch were:
- when SYNCEVOLUTION_DEBUG was set, the code still redirected
the child's stdout to stderr
- the child prints INFO messages which are relevant for the user;
with redirection, those do not reach the user
- the child's ERROR messages end up being printed as
[ERROR] stderr output: [ERROR] ....
Conflicts:
Makefile.am
configure-post.in
configure.ac
src/Makefile-gen.am
src/dbus/qt/Makefile.am
src/syncevo-dbus-server.cpp
test/test-dbus.py
Conflicts caused by renaming of files on master. Manually applied the
relevant changes to the renamed files.
The logic for dumpData and printChanges wasn't implemented correctly.
- run sync with dumpData=0 printChanges=0 => writes database dumps
("before/after" in logdir) although it shouldn't
- run status check (--status) with dumpData=1 printChanges=0 => dumps and
prints changes although it should do neither
- same for --restore
The fix is to check the "print changes" setting in addition to (and
not instead of) m_doLogging. m_doLogging is merely an override used in
testing. If false, then the logdir handling and thus data dumps are
disabled.
For --status only check printChanges. This used to print changes and
thus dump data when either dumpData or printChanges were set. It makes
more sense to only produce the output when exactly that is asked for.
In SyncContext::initMain(), set a default handler which routes
all glog output into the SyncEvolution logging system. That's more
reliable than trying to capture redirected stderr and also works
for output which would go to stdout.
At the property level, the isDefault retval exposed whether the
property value was set explicitly in the config or taken from the
property default. That information got lost at the
SyncConfig/SyncSourceConfig level although there are cases where that
is relevant (like providing better error messages, BMC #23783).
Now that level uses the new InitState classes instead of plain
int/bool/std::string return values. Code which assigns these return
values to local variables doesn't need to be adapted. Directly using
the return value in an expression might need some work (typically
adding a get() if the compiler cannot infer the desired
type). Overriding the virtual methods always needs to be adapted.
When deleting an item on phone and locally, the next sync fails with
ERROR messages about "object not found". This has several reasons:
- libsynthesis super data store attempts to read items
which may or may not exist (triggers ERROR message)
- it checks for 404 but Evolution backends only return a generic
database error (causes sync to fail)
It turned out that ReadItem and DeleteItem are expected to return a
404 status when the requested item does not exist. This patch documents
that (only in the TrackingSyncSource, though), adds tests and fixes
EDS, WebDAV, file and sqlite backends accordingly.
This patch also suppresses the 404 error logging inside DeleteItem(),
while still returning that error code to the Synthesis engine. Not
logging that particular situation is consistent with the previous
SyncEvolution behavior of silently returning successfully when there
wasn't anything to delete.
In addition, more recent libsynthesis versions also no longer do
a ReadItem() call to test for existence. That would still trigger
a spurious (albeit now harmless) ERROR message.
(cherry picked from commit ba289c899f)
Conflicts:
src/backends/webdav/CalDAVSource.cpp
test/ClientTest.cpp
test/ClientTest.h
Frederik Elwert reported that running a local sync with a phone via
Bluetooth caused the syncevo-dbus-server to shut down during a sync.
A log showed that a SIGPIPE is triggered, apparently by a writev()
inside libdbus.
Explicitly telling the process to ignore the signal solved that
problem.
When deleting an item on phone and locally, the next sync fails with
ERROR messages about "object not found". This has several reasons:
- libsynthesis super data store attempts to read items
which may or may not exist (triggers ERROR message)
- it checks for 404 but Evolution backends only return a generic
database error (causes sync to fail)
It turned out that ReadItem and DeleteItem are expected to return a
404 status when the requested item does not exist. This patch documents
that (only in the TrackingSyncSource, though), adds tests and fixes
EDS, WebDAV, file and sqlite backends accordingly.
This patch also suppresses the 404 error logging inside DeleteItem(),
while still returning that error code to the Synthesis engine. Not
logging that particular situation is consistent with the previous
SyncEvolution behavior of silently returning successfully when there
wasn't anything to delete.
In addition, more recent libsynthesis versions also no longer do
a ReadItem() call to test for existence. That would still trigger
a spurious (albeit now harmless) ERROR message.
The -from-client/server sync modes are confusing because the direction
of the data exchange depends on which side acts as SyncML server or
client.
This patch introduces new modes which use -from-local/remote
instead. The statistics and messages also use these variants now. The
old modes are still understood, but are declared as "not recommended"
in the documentation.
When using a config which has sources with a backend type set which is
not currently available, an error was thrown even if those sources
weren't even part of the current operation (for example, syncing
another source which is currently supported).
Fixed by checking for "source active" before checking its backend.
The memset/memcpy of the embedded boost::function instances inside the
old ClientTestConfig was causing segfaults at the end of a client-test
run if compiled with optimization.
Therefore this commit turns ClientTestConfig into a proper class
containing members which initialize themselves (Bool wrapper class,
std::string), thus memset is no longer needed and used. Also added the
standard m_ prefix.
m_numItems is gone, was never set by any backend anyway and even
expected to be consistent in one test. Now CLIENT_TEST_NUM_ITEMS is
read by defNumItems() each time it is needed.
Removed "const char *" strings from method parameters. This revealed
that config.itemType (a const char *) was incorrectly passed to
insert() where the boolean "relax" parameter should have been given.
Replaced by "false" (= strict checking) even though the old code
must have run with an implicit "true" (= relaxed checking). Let's see
whether any tests fail now.
(cherry-picked from commit 6399bd8181)
Commit 3f1185, contained in 1.1.99.3, changed
SyncContext::throwError() so that it throws a StatusException with
STATUS_FATAL. Previously a runtime exception was thrown, which
Exception::handle() recorded as a local error.
This commit fixes that regression by throwing a STATUS_FATAL +
LOCAL_STATUS_CODE, which restores the traditional result of
throwError().
Found by test-dbus.py TestDBusSyncError.testSyncNoConfig.
This patch replaces the ugly configuration translation with a runtime
check in a comparescript.
This is a first step towards detecting properly at runtime whether a
peer supports UID/RECURRENCE-ID semantic in calendar data. Currently
this check is still based on the "local sync == use UID/RECURRENCE-ID"
shortcut.
This patch depends on a libsynthesis which supports the COMPAREMODE()
method.
The distinction between vcard21 and vcard30 became mute in the Evolution
backend a while ago. Both tests ended up using the vCard 3.0 Evolution
tests data and the default uri for each server. This patch removes
the vCard 2.1 special case.
It also renames the tests and test data to reflect that they always
were Evolution specific. The new naming convention, also applied
to file, QtContacts, KCalExtended, XMLRPC, Maemo and Akonadi backends, is
now <backend>_contact/event/task/memo, with eds/file/qt/kcal/maemo/kde
as backend names.
The reasoning is:
- results in unique string (in particular no overlap with
backend type names), easier to search for
- underscore already used before (in contrast to hyphen)
- no plural-s to keep the name shorter
The Akonadi backend should be using its own test data instead of
the Evolution ones.
The "source-config" peer in the target context used to be required for
local sync. Now it is optional. The reasoning is:
- For WebDAV, copying username/password from the main config may
already be enough to find the server.
- For local file sources, no sync properties ared needed
at all.
In both cases, "source-config" is used if available. That can be useful
to set loglevel separately, for example.
For SyncEvolution<->SyncEvolution synchronization there should be
no need to set the uri property of each source explicitly, because
typically it is the same as the source name.
With this patch, the source name is used as fallback. Because some
code needs to check whether the URI is set, getURI() returns the
setting directly (as before) whereas getURINonEmpty() always returns a
non-empty result with the name as fallback.
testOneWayFromClient was still failing in server mode because of a
similar issue as with slow sync as fallback for refresh-from-server:
the client can ask for a two-way sync and then withhold all of its own
changes to achieve the same effect.
Detect this based on the requested sync mode.
When deleting items on the server due to "refresh-from-client" sync,
we want to have these items counted in the statistics, just as on
a client in a "refresh-from-server".
The code doing this had to be updated to distinguish between client and
server mode.
This patch relies on libsynthesis reporting a stable sync mode
of each source in the PEV_ALERTED progress event. This was added
recently to libsynthesis in server mode.
This patch deals with a client that falls back to "delete data + slow
sync" as a way of doing a refresh-from-server (as Synthesis engine
itself does) at least in one case: server-alerted syncs know the originally
requested sync mode and assume that a slow sync is the intended mode,
which then gets translated back into refresh-from-server.
A session with such a sync initiated by a client is still recorded as
"slow sync".
The methods related to sync username/password were just called
get/setUsername/Password(), with additional check/savePassword()
methods. The savePassword() method clashed with a semantically
different method for arbitrary passwords (as pointed out by clang
2.9), so let's make it more clear that one set of methods specifically
is for the sync username/password.
clang 2.9 pointed out that getLogDir() didn't have the desired effect
anymore after a signature change in the base class. Not sure why the
test passed regardless of that, but it was indeed broken.
clang 2.8 compiles SyncEvolution + Synthesis faster than g++ 4.4.5
(3:40min instead of 4:10min on my laptop) and produces more useful
error reports. This patch fixes the code so that it compiles cleanly
with clang when using "-Wall -Werror -Wno-unknown-pragmas". Note that
clang 2.6 (Debian Squeeze) goes into an infinite recursion compiling
code using gdbus-cxx-bridge.h and dies eventually with a stack
overflow - can't be used.
Changes necessary for clang:
- eptr pointer referencing ambiguous, use *x.get() instead
- boost::intrusive_ptr* must be defined before code using it
- two-phase template checking requires explicitly specifying
members in base classes
- name clashes with plain C structs (DBusServer, DBusWatch) are
an error and need to be avoided (done with namespaces GDBusCXX and
SyncEvo)
- floats cannot be inline constants
- unused methods in local classes are warned about (left() in SyncML.cpp)
When the glib event loop is left because the D-Bus client has
requested an abort, the LocalTransportAgent should simply return a
"failed" status and let the caller handle the abort. The return code
of write/readMessage() must be able to convey that - extended from
boolean to an enum.
SyncContext did not do that correctly in server mode: the check for
abort must be done before giving up by throwing an exception.
The D-Bus test now checks that the right status is recorded (wasn't
the case earlier).
The final sync status is reported by the Synthesis engine as a
progress event. SyncEvolution ignored that overall status. In most
cases, it ended up using either one of its own errors (like transport
problem) or the status of a specific source as the final status of
sync.
When synchronization was aborted by the user before the first message
(as in an artificial test), then the source did not encounter a
problem and thus no error was set for the session, although it
failed. Fixed by remembering the Synthesis status code.
Instead of allowing users of the API to register a callback
which can choose between aborting and time out, only accept
the timeout duration and always treat that as a timeout.
The advanced functionality wasn't used and the simplification
makes implementing the API easier.
In some cases (aborted sync, source status was set to a remote error),
the child process has more detailed and accurate information about a failure
related to a source. Copy that to the parent's reports.
Patch depends on still having the LocalTransportAgent instance (moved
m_agent.reset()) and updating the source should be reflectd in final
report (moved updateSyncReport()). The later probably also was broken
for the "partial success case".
Calling the parent logger may invoke flushing in LogRedirect. This
needs to be done before writing the log message which causes that
flushing, otherwise the order of lines in the -log.html file is
slightly wrong.
SyncConfig inherited "const char *" from the Funambol C++ API and some
other methods used the same approach for efficient access to plain
strings. However, this has the disadvantage that dynamically generated
strings cannot be returned. SyncConfig had to use an awkward
workaround with a local string cache.
This patch converts most of that code to a normal std::string return
value and removes the string cache.
Out-of-tree backends must be adapted, otherwise they won't compile.
KCalExtended does not support the operation needed to obtain changes since
the last sync. That is because it relies on the anchor, which is inaccessible
in the client binfile and only available in a real beginSync().
The command line output contained no line for the KCalExtended backend
because it got no information. It is better to explicitly mark unknown
entries with -1.
gen-autotools.sh now detects pre-releases: have "99" in the version or
the source isn't cleanly tagged. "syncevolution --version" includes
"(pre-release)" if in pre-release mode, otherwise just the version (as
before).
This patch adds version numbers to the on-disk config and files. The
goals are:
1. refuse to use a config which was written by a SyncEvolution release
in a format that is too recent to be handled correctly by the
current release ("detect invalid downgrades")
2. refuse to modify a config in such a way that the previous release
using that config will not be able to use it anymore ("prevent
unintentional upgrade of config")
In the first case the user is told:
SyncEvolution <version> is too old to read configuration '<config>',
please upgrade SyncEvolution.
In the second case the user is told to migrate the configuration manually:
Proceeding would modify config '<config>' such that the
previous SyncEvolution release will not be able to use it.
Stopping now. Please explicitly acknowledge this step by
running the following command on the command line:
syncevolution --migrate '<config>'
These are printed as [ERROR] messages on the command line and shown as
error codes 22004 resp. 22005 in front-ends which don't know about these
scenarios.
The first problem should be rare, so presenting a nicer error messages
in UIs is not essential. But the second problem will occur. The plan
(not implemented yet) is to automatically migrate in stable releases
without asking.
When running client-test, the normal stdout text is copied into
a .log text file named after the currently running test. The output
of the child process should also go there.
This patch achieves that by being more selective about which loggers
in the stack of loggers it removes: LogDir must be removed because it
writes per-process Synthesis .html files, LoggerStdout must remain for
.log. The difference is abstracted away behind a new
Logger::isProcessSafe() method, which tells LocalTransportAgent whether
sharing the logger instance between processes is okay.
If both sides support iCalendar 2.0, then UID/RECURRENCE-ID should be
used for pairing instead of looking at data. The other user-visible
fields are still relevant and should be merged and updated. The new
Synthesis compare="scripted" achieves the later.
This patch enables this mode for local sync, which is assumed to use
iCalendar 2.0. This assumption and the way how the different compare
values are implemented (global search/replace via a poor man's regex)
is a hack: it would be much nicer to have this mode configurable for
each peer and source or even recognize it at runtime.
But runtime detection is unlikely to work without fundamental changes
in libsynthesis because the field list must be configured before the
peer is even known. One solution might be to evaluate the compare level
later at runtime, instead of at config parsing time.
Database backups used to be created unconditionally, even if not
needed for comparison. Now the new "dumpData" variable is
checked. If change printing is enabled, dumping data is implied.
Resetting username/password temporarily was a hack which
had undesired side effects for local sync (credentials
were not available for other side of sync).
Changed this so that disabling of credential checking is done
at the point where it is needed: when creating Synthesis XML
config. Username/password settings are left untouched.
The code setting up a sync session disabled printing of DEBUG
messages to the console, even if it was enabled earlier because
of SYNCEVOLUTION_DEBUG.
Now check for SYNCEVOLUTION_DEBUG before making that change.
Not sure whether it is the right solution (perhaps just keep
the old level?), but it is known to have the desired effect.
In local sync the terms "local" and "remote" (in SyncReport, "Data
modified locally") do not always apply and can be confusing. Replaced
with explicitly mentioning the context.
The source name also no longer is unique. Extended in the local sync
case (and only in that case) by adding a <context>/ prefix to the
source name.
Here is an example of the modified output:
$ syncevolution google
[INFO] @default/itodo20: inactive
[INFO] @default/addressbook: inactive
[INFO] @default/calendar+todo: inactive
[INFO] @default/memo: inactive
[INFO] @default/ical20: inactive
[INFO] @default/todo: inactive
[INFO] @default/file_calendar+todo: inactive
[INFO] @default/file_vcard21: inactive
[INFO] @default/vcard30: inactive
[INFO] @default/text: inactive
[INFO] @default/file_itodo20: inactive
[INFO] @default/vcard21: inactive
[INFO] @default/file_ical20: inactive
[INFO] @default/file_vcard30: inactive
[INFO] @google/addressbook: inactive
[INFO] @google/memo: inactive
[INFO] @google/todo: inactive
[INFO] @google/calendar: starting normal sync, two-way
Local data changes to be applied remotely during synchronization:
*** @google/calendar ***
after last sync | current data
removed since last sync <
> added since last sync
-------------------------------------------------------------------------------
BEGIN:VCALENDAR BEGIN:VCALENDAR
...
END:VCALENDAR END:VCALENDAR
-------------------------------------------------------------------------------
[INFO] @google/calendar: sent 1/2
[INFO] @google/calendar: sent 2/2
Local data changes to be applied remotely during synchronization:
*** @default/calendar ***
no changes
[INFO] @default/calendar: started
[INFO] @default/calendar: updating "created in Google, online"
[INFO] @default/calendar: updating "created in Google - mod2, online"
[INFO] @google/calendar: started
[INFO] @default/calendar: inactive
[INFO] @google/calendar: normal sync done successfully
Synchronization successful.
Changes applied during synchronization:
+---------------|-----------------------|-----------------------|-CON-+
| | @default | @google | FLI |
| Source | NEW | MOD | DEL | ERR | NEW | MOD | DEL | ERR | CTS |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| calendar | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| disabled, 0 KB sent by client, 2 KB received |
| item(s) in database backup: 3 before sync, 3 after it |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| start Mon Oct 25 10:03:24 2010, duration 0:13min |
| synchronization completed successfully |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
Data modified @default during synchronization:
*** @default/calendar ***
before sync | after sync
removed during sync <
> added during sync
-------------------------------------------------------------------------------
BEGIN:VCALENDAR BEGIN:VCALENDAR
VERSION:2.0 VERSION:2.0
...
END:VCALENDAR END:VCALENDAR
-------------------------------------------------------------------------------
pohly@pohly-mobl1:/tmp/syncevolution/src$
Synchronization successful.
Changes applied during synchronization:
+---------------|-----------------------|-----------------------|-CON-+
| | @google | @default | FLI |
| Source | NEW | MOD | DEL | ERR | NEW | MOD | DEL | ERR | CTS |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| calendar | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 | 0 |
| two-way, 2 KB sent by client, 0 KB received |
| item(s) in database backup: 2 before sync, 2 after it |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| start Mon Oct 25 10:03:24 2010, duration 0:13min |
| synchronization completed successfully |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
Data modified @google during synchronization:
*** @google/calendar ***
no changes
During local sync names like "addressbook" are no longer unique,
because they may exist in both the local and the remote context. This
patch introduces a "display name" composed from context and source name,
like this: "@<context>/<source>" (@default/addressbook).
The context is only used if needed, which currently is the case during
a local sync.
Changing the SyncSourceBase::getName() result was also considered, but
several places expect this to be the name of the source inside its
context, so an explicit SyncSourceBase::getDisplayName() turned out
to be safer.
Backends like XMLRPC need information about URL, proxy and SSL
settings, etc. This can be done via source specific properties, like
evolutionsource, but this is not how this is normally done. It would
be nicer if the exising per-peer properties could be used. The goal is
that a normal peer configuration can be created from a template with
the necessary information to enable sources using that information.
This patch makes this possible by adding a context parameter to
SyncSourceParams:
* @param context Additional non-source config settings.
* When running as part of a normal sync, these are the
* settings for the peer. When running in a local sync,
* these settings come from the "source-config" peer
* config inside the config context of the source.
* Testing uses "source-config@client-test". On the
* command line, this is the config chosen by the
* user, which may or may not have peer-specific settings!
Note that this still doesn't solve the problem for XMLRPC to SyncML
peer sync, because in that case ("normal sync") the context will be
the one describing the peer. SyncURL is already used and proxy
settings might not match.
The XMLRPC backends therefore was not changed and continues to use
evolutionsource.
Local sync is configured with a new syncURL = local://<context> where
<context> identifies the set of databases to synchronize with. The
URI of each source in the config identifies the source in that context
to synchronize with.
The databases in that context run a SyncML session as client. The
config itself is for a server. Reversing these roles is possible by
putting the config into the other context.
A sync is started by the server side, via the new LocalTransportAgent.
That agent forks, sets up the client side, then passes messages
back and forth via stream sockets. Stream sockets are useful because
unexpected peer shutdown can be detected.
Running the server side requires a few changes:
- do not send a SAN message, the client will start the
message exchange based on the config
- wait for that message before doing anything
The client side is more difficult:
- Per-peer config nodes do not exist in the target context.
They are stored in a hidden .<context> directory inside
the server config tree. This depends on the new "registering nodes
in the tree" feature. All nodes are hidden, because users
are not meant to edit any of them. Their name is intentionally
chosen like traditional nodes so that removing the config
also removes the new files.
- All relevant per-peer properties must be copied from the server
config (log level, printing changes, ...); they cannot be set
differently.
Because two separate SyncML sessions are used, we end up with
two normal session directories and log files.
The implementation is not complete yet:
- no glib support, so cannot be used in syncevo-dbus-server
- no support for CTRL-C and abort
- no interactive password entry for target sources
- unexpected slow syncs are detected on the client side, but
not reported properly on the server side
LogDir is not copyable when it contains an m_info pointer. std::sort()
inside LogDir::getLogDirs() for a context is called with LogDir as
binary predicate, which causes a crash because sort() copies the
predicate.
This was found when calling getLogDirs() in a LogDir used for local
sync, without a peer. It did not happen before because then the
LogDir instance did not contain an m_info pointer. It was merely
inefficient.
This patch moves the static log dir name methods into a new
LogDirNames class and uses that as binary predicate. LogDir itself
inherits these methods via a private base class, which is similar
to the previous private static methods (avoids touching all of
the calls) and also inherits from boost::copyable to catch future
errors related to copying LogDir.
The new method is needed for importing item data from stdin, a new
feature for the command line. The default implementation which is
used when executing the command line locally reads from std::cin
directly.
The syncevo-dbus-server would have to ask its client for the data;
this hasn't been implemented yet. To catch the problem and inform
the user, an exception is thrown there.
4 cases in LogDirTest classes are failed due to logging ouput
missing. The root cause is that the 'session' function in LogDirTest
sets the top logger as 'INFO' level and thus the testing output of
'DEBUG' info can't be written in the log file. Thus nightly testing
can't check the result.
The solution is to save the logging level before creating a new session
and restore it again when completion.
This problem is specific to SyncEvolution running as HTTP server:
when a second session is requested, SyncContext::analyzeSyncMLMessage()
creates a second sync session that is active temporarily instead
of the main one. This is necessary to detect when the same peer
connects again.
The static m_sourceListPtr was not reset properly by that function,
causing the main session to fail later on when reading it. This patch
solves the problem by making m_sourceListPtr a normal member of
SyncContext and relying on m_activeContext instead to find the current
source list.
The latest Synthesis engine also emits progress events in server mode.
Before this patch, the events were ignored because the code was
conservative about not relying on something not supported. Now
client and server mode report and record progress and statistics
the same way.
The signal handler in syncevolution interpret CTRL-C as suspend
request. When invoked by syncevo-phone-config, that is not what is
intended by the user and usually had the effect that a sync session
failed without flagging the abort request, thus keeping the script
running.
This patch adds SYNCEVOLUTION_NO_SYNC_SIGNALS, which prevents
installing the signal handlers, and uses that in the script.
As a result, CTRL-C leads to "aborted prematurely", for which
a test was added earlier.
Because syncevolution has to do no cleanup work, aborting it
like this is okay.
Only apply conversion of alarm time to UTC time when server is
Mobical.net. The way is to define a global flag and set the flag
when 'remoterule' of 'Mobical' is matched.
The global flag is declared as a context variable and the script
reads it via built-in function 'SESSIONVAR'.
Patrick Ohly
Ove Kåven
Zhu, Yongsheng
Chen Congwu