When a single sub item in a merged item is updated, we modify the
calendar and sent the complete item to the server. Google didn't
like that and rejected the update because of "409 - Can only store an
event with a newer DTSTAMP." Found in testLinkedItemsInsertBothUpdateParent.
The solution is to update SEQUENCE and DTSTAMP/LAST-MODIFIED to that of
the recently updated sub item, then Google accepts the update. Updating
just one or the other was not enough.
This change should work even if more than one item is updated, as long
as a) item changes are reported in the right order (oldest change
first) or b) all sub items get updated.
a) is not guaranteed, b) might be. To be tested...
This patch implements the SubSyncSource interface in CalDAVSource,
then instantiates that class as backend for a MapSyncSource which
bridges between the core engine (one VEVENT per item) and CalDAV (all
VEVENTs with same UID in the same item).
The tests must run with a modified ical20.ics.google.tem which
does not contain RECURRENCE-ID, because that is not yet supported.
Also, our complex 1234567890!@#$%^&*()<>@dummy must be avoided
because Google has an issue with it (doesn't find resulting resource):
CLIENT_TEST_SERVER=google CLIENT_TEST_SIMPLE_UID=1 ./client-test Client::Source::caldav_ical20::testImport
This patch fixes some (but not all) issues found in testImport:
- readItem() did not clear the item string before appending
the new item.
- Detection of "item existed already" cannot use the alternative name
returned by the server, because Google does that also for items
which are really new.
- The test registered itself as "ical20", which conflicts with the EDS
backend. Now use "caldav_ical20".
This replaces the WEBDAV_* env variables with access to the
normal per-peer properties, passed into the source via the new
SyncSourceParams context member.
When used on the command line, @foobar context configs are not
enough to use the WebDAV sources. A properly configured peer
context (foo@bar) is needed. There is no explicit check for this,
the resulting error is about not being able to access the empty
URL `' (Neon uses these quotation marks).
Mapping removeItem() to DELETE is straight-forward. Except that CalDAV
will require additional support for removing individual VEVENTs from
a resource that contains more than one VEVENT...
This works for updating and adding an item. Storing a new item under a
different resouce URI is also handled, which can happen with CalDAV
when there is already an item with the same UID.
It would be nice if the server returned a ETag in response to a PUT,
but Google Calendar doesn't do that. The header is checked neverthless
(untested!), with an explicit PROPGET as fallback.
This patch also clarifies LUID (relative if possible, absolute if not)
and revision string handling (without W/ and quotation marks). We
don't care about weak ETags at the moment, Google doesn't use
them. Even if a server did, what would we do with the information that
the item wasn't stored verbatim?
The current implementation is not yet good enough for CalDAV:
- only one VEVENT per UID supported: must merge and split
related VEVENTs into one CalDAV calendar resource
- adding or updating VEVENT without UID probably fails:
must insert new or old UID
Listing items in WebDAV consists of getting the ETag for each URI
directly underneath the collection. Checking for "empty source"
is done by reading all pairs and then checking for non-empty map.
It would be nice to abort the processing once an error is encountered
or once "not empty" has been observed, but Neon does not support that.
Probably for a good reason, because it would render the TCP connection
useless.
This wraps the "simple" Neon propfind method which handles plain
strings as values. For more complex values, the value is an XML
fragment which should better be parsed using the "complex" Neon
propfind method with XML handlers.
The callbacks are defined by boost::function, which allows attaching
additional parameters to each callback. Exceptions must not cross Neon
library calls.
The Neon::Session wraps most of the relevant calls. It is
parameterized by Neon::Settings. It is uncertain where all of these
are meant to come from, because there is no peer configuration in many
cases. Perhaps we can enforce that a WebDAV source may only be created
in a context which has one and exactly one peer config?
The current intermediate solution in WebDAVSourceRegister.cpp grabs
all settings from WEBDAV_* env variables.
Disabling SSL verification and Neon debug logging are implemented.
Opening the source runs a few checks on the URL. Disabling SSL
certificate checking turned out to be necessary, probably because of
the known issue of gnutls not trusting the weak Google certificate
chain.
This patch maps QContactDetails which have no mapping to vCard to
X-SYNCEVO-QTCONTACTS. The value of that extension property is a dump
of the complete QContactDetail. Reading it again later can restore
that detail.
The goal is to make this additional information available to the
Synthesis engine such that it can preserve the additional information.
Some peers might even be able to store them.
One of the features of the Synthesis engine is that it can preserve
properties locally which are not supported by the
peer. "testExtensions" covers that, by updating all items on the
server (via client B) and reimporting them as updates into client A.
The data comparison then is done without the usual "allow server to
drop data" simplifications in synccompare.
This test is not enabled yet. "config.update" needs to be set for it,
but doing so then trips over other changes introduced by servers, like
for example re-encoding photos. Needs some more thoughts and testing...
The QVersitContactHandlerFactory::ProfileBackup profile enables a
vCard flavor which contains all information stored in a QtContact. It
might make sense to use that format for raw access (--import/export).
Not currently enabled though because the format is pretty verbose.
clang 2.8 compiles SyncEvolution + Synthesis faster than g++ 4.4.5
(3:40min instead of 4:10min on my laptop) and produces more useful
error reports. This patch fixes the code so that it compiles cleanly
with clang when using "-Wall -Werror -Wno-unknown-pragmas". Note that
clang 2.6 (Debian Squeeze) goes into an infinite recursion compiling
code using gdbus-cxx-bridge.h and dies eventually with a stack
overflow - can't be used.
Changes necessary for clang:
- eptr pointer referencing ambiguous, use *x.get() instead
- boost::intrusive_ptr* must be defined before code using it
- two-phase template checking requires explicitly specifying
members in base classes
- name clashes with plain C structs (DBusServer, DBusWatch) are
an error and need to be avoided (done with namespaces GDBusCXX and
SyncEvo)
- floats cannot be inline constants
- unused methods in local classes are warned about (left() in SyncML.cpp)
A lot of the code was roled out explicitly. This patch replaces
that with an approach similar to iostreams: remember context, then
work on one parameter at a time.
Because the decision what to do with a certain method parameter
depends on the type of the parameter, that type must be used to pick
between Get/Set classes which skip resp. work on the parameter.
Allows comparison, substraction and addition. Reads monotonic and system
time. This is a better alternative to time(NULL) + time_t, because it
has sub-second resolution and (for timeouts) is resilient against manually
setting system time.
When the glib event loop is left because the D-Bus client has
requested an abort, the LocalTransportAgent should simply return a
"failed" status and let the caller handle the abort. The return code
of write/readMessage() must be able to convey that - extended from
boolean to an enum.
SyncContext did not do that correctly in server mode: the check for
abort must be done before giving up by throwing an exception.
The D-Bus test now checks that the right status is recorded (wasn't
the case earlier).
The final sync status is reported by the Synthesis engine as a
progress event. SyncEvolution ignored that overall status. In most
cases, it ended up using either one of its own errors (like transport
problem) or the status of a specific source as the final status of
sync.
When synchronization was aborted by the user before the first message
(as in an artificial test), then the source did not encounter a
problem and thus no error was set for the session, although it
failed. Fixed by remembering the Synthesis status code.
Local sync inside syncevo-dbus-server must keep the main event
loop running, otherwise the D-Bus server would stop responding
to D-Bus calls while a sync runs.
This is tested as part of new test-dbus.py tests. They set up local
sync between two directories and then test syncing, timeout detection
when using the glib support code, and aborting via the D-Bus API
while a sync runs.
This testing depends on delaying the sync inside the child
process. Setting the SYNCEVOLUTION_LOCAL_CHILD_DELAY env variable
achieves that (might also be useful for interactive debugging). It is
set for those tests which need it via additional function properties,
similar to the older @timeout() mechanism, which now uses the
more general @property function decorator.
LocalTransportAgent needs to wait temporarily for message transfers.
Without other activity, that is done with select(). But inside syncevo-dbus-server,
the main glib event loop must be served while waiting. GLibSelect()
does that by setting up a source which returns a) after the timeout
or b) when the file descriptor is ready for the next read or write.
As a special case, other returns from the event loop must also be reported.
Doing setSourceType() unconditionally unintentionally also removed a
configured syncFormat and forceSyncFormat setting for an existing
configuration. Only setBackend() (as with this patch) is necessary
and valid in all cases (never run a test with the wrong backend!).
If the context "type" was wrong and only the peer "type" was right,
the old config was usable for syncing. It seems that the "Memotoo"
template created such broken configs. After migration, the wrong
backend was selected based on the context's broken "type", thus
breaking syncing.
This patch fixes that problem by overriding the context "type" with
the corresponding values from one peer which has the source enabled.
The rationale is that this peer used to work and should be kept
working.
There's still a chance that different peers had different backends
configured, which is neither checked nor detected at the moment.
The new call is meant to be used after forking: in such a situation,
redirected stdout/stderr output can be read by both the parent and the
child, which leads to a race condition and makes it non-deterministic
where that output will be handled. redoRedirect() closes the child's
connection to the parent's redirection and sets up its own instead.
Timeouts for the SyncML message are detected on the server side.
No resending is possible, nor needed: this is a reliable local
transport after all, not HTTP.
The implementation uses select() and non-blocking sockets to time out
at the right time. While at it, it also sets CLOEXEC to ensure that
the pipes are not accidentally inherited by other child programs.
Instead of allowing users of the API to register a callback
which can choose between aborting and time out, only accept
the timeout duration and always treat that as a timeout.
The advanced functionality wasn't used and the simplification
makes implementing the API easier.
In some cases (aborted sync, source status was set to a remote error),
the child process has more detailed and accurate information about a failure
related to a source. Copy that to the parent's reports.
Patch depends on still having the LocalTransportAgent instance (moved
m_agent.reset()) and updating the source should be reflectd in final
report (moved updateSyncReport()). The later probably also was broken
for the "partial success case".
In contrast to getSyncSourceReport(), findSyncSourceReport() can be
called on a const SyncReport and returns a pointer to an existing
SyncSourceReport or NULL if not present. Useful when
getSyncSourceReport() would unintentionally create the result.
After the removal of the "type" property, listing of sources broke
because the command kind code kept setting that to select a backend.
Fixed by parsing the type and setting "backend" and "databaseFormat"
instead.
SyncEvolution 1.1.1 did not build without this patch on S360, because
the source used an integer typedef which turned out to be incompatible
with size_t, although it should have the same size. Apparently the
compiler distinguishes between "unsigned int" and "unsigned long",
even if both have the same size.
One of them ("We don't know what this device...") was already
translated elsewhere. It might be that it is already translated in the
UI. The other ("Use these settings") was not and thus really showed up
as untranslated.
Running gen-autotools.sh as part of "make distcheck" fails because the
source directory is read-only. It shouldn't be needed and wouldn't
replicate the same autotools files (due to the missing "git describe"
output), so disable running it unless the source directory contains a
.git repo.
"gen-autotools.sh" (invoked by "autogen.sh") is needed to create some
of the autotools .am and .in files. This is necessary to avoid
hard-coding certain lists of dirs (backends) and files (templates).
The generated makefiles updated themselves when these files were
modified, but not when files were added or removed.
This patch fixes that. It also avoids some code duplication. Instead
of adding explicit rules to the makefiles for the file dependencies,
now gen-autotools.sh is run on each "make all" or "make Makefile"
invocation. It modifies the generated files only if their content has
changed. The dependency rules for "Makefile.am" and "configure.in" are
constructed so that make re-checks their time stamps after running the
gen-autotools.sh script.
