Commit Graph

94 Commits

Author SHA1 Message Date
Patrick Ohly 04f11b422e source -> datastore rename, improved terminology
The word "source" implies reading, while in fact access is read/write.
"datastore" avoids that misconception. Writing it in one word emphasizes
that it is single entity.

While renaming, also remove references to explicit --*-property
parameters. The only necessary use today is "--sync-property ?"
and "--datastore-property ?".

--datastore-property was used instead of the short --store-property
because "store" might be mistaken for the verb. It doesn't matter
that it is longer because it doesn't get typed often.

--source-property must remain valid for backward compatility.

As many user-visible instances of "source" as possible got replaced in
text strings by the newer term "datastore". Debug messages were left
unchanged unless some regex happened to match it.

The source code will continue to use the old variable and class names
based on "source".

Various documentation enhancements:
  Better explain what local sync is and how it involves two sync
  configs. "originating config" gets introduces instead of just
  "sync config".

  Better explain the relationship between contexts, sync configs,
  and source configs ("a sync config can use the datastore configs in
  the same context").

  An entire section on config properties in the terminology
  section. "item" added (Todd Wilson correctly pointed out that it was
  missing).

  Less focus on conflict resolution, as suggested by Graham Cobb.

  Fix examples that became invalid when fixing the password
  storage/lookup mechanism for GNOME keyring in 1.4.

  The "command line conventions", "Synchronization beyond SyncML" and
  "CalDAV and CardDAV" sections were updated. It's possible that the
  other sections also contain slightly incorrect usage of the
  terminology or are simply out-dated.
2014-07-28 15:29:41 +02:00
Patrick Ohly 4b2b2bb64e engine: UID support in contact data
Before, the UID property in a vCard was ignored by the engine.
Backends were responsible for ensuring that the property is
set if required by the underlying storage.

This change moves this into the engine:
- UID is now field. It does not get used for matching
  because the engine cannot rely on it being stored
  by both sides.
- It gets parsed if present, but only generated if
  explicitly enabled (because that is the traditional
  behavior).
- It is never shown in the DevInf's CtCap
  because the Synthesis engine would always show it
  regardless whether a rule enabled the property.
  That's because rules normally only get triggered
  after exchanging DevInf and thus DevInf has to
  be rule-independent. We don't want it shown because
  then merging the incoming item during a local sync
  would use the incoming UID, even if it is empty.
- Before writing, ensure that UID is set.

When updating an existing item, the Synthesis engine reads
the existing item, preserves the existing UID unless the peer
claims to support UID, and then updates with the existing UID.

This works for local sync (where SyncEvolution never claims
to support UID when talking to the other side). It will break
with peers which have UID in their CtCap although they
rewrite the UID and backends whose underlying storage cannot
handle UID changes during an update (for example, CardDAV).

A backend must still activate the UID property by setting m_backendRule
to LOCALSTORAGE-WITH-UID instead of the default LOCALSTORAGE.
Custom backend rules must include the new sub-rule HAVE-VCARD-UID.

See "[os-libsynthesis] UID + CardDAV" for a discussion of this change.
Long term it would be better to configure profiles without depending
on the rule hack.
2014-05-02 16:43:53 +02:00
Patrick Ohly 02088c22e0 SyncSource: add source name to all exception handling
Not all exceptions thrown while executing a source operation contain
the source name. SyncSource::throwError() does that, but SE_THROW() as
used in helper code does not. It is better to add the source name as
logging prefix. The other advantage is that all lines will have the
prefix, not just the first one.

The SyncSource operations need access to the SyncSource class which
contains them to access the display name of the SyncSource instance.
A positive a side effect of telling the wrappers about the instance at
construction time is that the caller no longer has to pass the source
reference.

This change allows removing the SyncSource::throwError() special
cases, which completes the transition towards having correct source
code location information for all exceptions.
2014-05-02 16:43:52 +02:00
Patrick Ohly de8461f802 code restructing: Exception, throwError()
Raising exceptions via throwError() looses the original source code
location information. Fixing that by explicitly passing that
information as additional parameter, created with the preprocessor
macro SE_HERE.

Several files included the complex SyncContext.h only because needed
throwError(). A better place for the revised implementation is the
Exception class which used to be in util.h, now Exception.h.

Simplifying the include statements exposed indirect include
dependencies and removed "using namespace std" from the compilation of
some source files which happened to rely on it. We want to get rid of
that name space polution, so fix the code instead of adding it back.
2014-05-02 16:43:52 +02:00
Patrick Ohly 3c01a1ebf6 PIM: enhanced progress notifications (FDO #72114)
This adds GetPeerStatus() and "progress" events.

To detect DB changes as they happen, the SyncSource operations are
monitored. Upon entry, a counter gets increased and transmitted
through to the PIM manager in syncevo-dbus-server using extended
SourceProgress structs (only visible internally - public API must not
be extended!). This will count operations which fail and count those
twice which get resumed, so the counts will be too high
occasionally. That is in line with the API definition; because it is
not exact, the API only exposes a "modified" flag.

Progress is reported based on the "item received" Synthesis event and
the total item count. A modified libsynthesis is needed where the
SyncML binfile client on the target side of the local sync actually
sends the total item count (via NumberOfChanges). This cannot be done
yet right at the start of the sync, only the second SyncML message
will have it. That is acceptable, because completion is reached very
quickly anyway for syncs involving only one message.

At the moment, SyncContext::displaySourceProgress() holds back "item
received" events until a different event needs to be emitted. Progress
reporting might get more fine-grained when adding allowing held back
events to be emitted at a fixed rate, every 0.1s. This is not done yet
because it seems to work well enough already.

For testing and demonstration purposes, sync.py gets command line
arguments for setting progress frequency and showing progress either
via listening to signals or polling.
2014-04-01 16:45:09 +02:00
Patrick Ohly ef9a89f788 EDS: only load one backend plugin of each kind
SyncEvolution was meant to load the syncecal or syncebook shared object
which uses the most recent libraries (libical, libecal/libebook) on
the system and then stop loading further backends. Due to a string
handling bug the check for already backends always found nothing,
leading to multiple conflicting backends loaded on some systems (for
example, those with libical0 and libical1 installed).

If that happened, the backend became unusable.
2014-03-31 16:08:08 +02:00
Patrick Ohly b27e560085 cppcheck performance: possible inefficient checking for emptiness
cppcheck correctly identified some places where the weaker empty()
call can be used instead of size() to determine whether an STL
data structure contains something.

None of them were really performance critical, but let's fix them
anyway for the sake of clean cppcheck scan results.
2014-01-17 16:15:15 +01:00
Patrick Ohly b3b0f3c7e1 EDS: support dual-install of ebook/ecal backends for EDS < 3.6 and EDS >= 3.6
The backends must be compiled differently for EDS < 3.6 (using the old
API before EBookClient, ECalClient and ESource, ideally in
compatibility mode) and for EDS >= 3.6 (using the new API, with hard
linking against libebook-1.2.so.14, libecal-1.2.so.15,
libedataserver-1.2.so.17).

With these changes, a SyncEvolution binary compiled for the older EDS
API will be able to load and use backends compiled against the current
one. Both backends can be installed in the same
lib/syncevolution/backends dir under different names. The newer ones
must have an additional -<version> appendix before the .so suffix.

Then loading will attempt to use those backends first and if
successful, skip the older ones. This is necessary because loading
both into the same process will fail due to symbol clashes. If
unsuccessful (as on systems with only the older EDS), the dynamic
linker will fail and SyncEvolution proceeds with the backends for the
old API.

Packaging of binaries as .dev/.rpm/.tar.gz includes these additional
backends if given in the EXTRA_BACKENDS variable when invoking "make
deb rpm distbin".
2013-10-25 21:07:26 +01:00
Patrick Ohly c16f1b0756 sync: avoid maintaining suspend/resume meta data during ephemeral sync
Both maintaining the map items inside the Synthesis engine and storing
them in .ini hash config nodes inside SyncEvolution are fairly heavy
operations which are not needed at all during an ephemeral sync (= no
meta data stored, done by the PIM Manager when triggering a pure PBAP
sync).

Using the new Synthesis CA_ResumeSupported DB capability it is
possible to suppress these operations without having to fiddle with
the Synthesis DB API that SyncEvolution provides. To make it possible
at the DB layer to detect that the meta data is not needed, the
ConfigNode passed to it must be marked as volatile.

This change sped up a sync with 10000 unmodified, known items from 38s
to 23s.
2013-07-12 11:44:39 +02:00
Patrick Ohly 309bed01e1 SyncSource: avoid ERROR logging for 508 status code
Returning a 508 status from beginSync() via a StatusException is
valid, this should only be logged by the originator if it deems that
an error.
2013-07-10 13:08:02 +02:00
Patrick Ohly 7d12c0a586 read-ahead: tell SyncSource about the upcoming read accesses
Trying to predict in the SyncSource which items will be needed is
hard. It depends what the item is retrieved for (sync or
backup/printing) and on the sync mode (during a sync).

Instead of guessing, better have the user of the source tell the
source in advance what it will read. In most cases this is cheap
because it does not involve copying of items luids ("read all items",
"read new or modified items"). The source then can use its own internal
mechanisms to figure out what that means.

Only extracting specific items specified on the command line and the
backup operation must explicitly pass an ordered list of luids. For
the sake of simplicity, do that in a std::vector even though it
involves copying.
2013-07-10 13:07:44 +02:00
Patrick Ohly 6c551167c4 engine: change tracking optional for caching mode and item modification
Recording the revision of items during a caching sync is unnecessary
because the next sync will not use the information. For item
modifications, the information was not even recorded.

Now a "don't need changes" mode can be set in SyncSource, which is
done for caching and command line item operations. This is better than
second-guessing the mode in which a source is used.

SyncSourceRevision checks that flag and skips updating the
luid->revision mapping if set.

Individual backends can also check the flag and skip calculating the
revision to begin with.
2013-07-05 17:44:10 +02:00
Patrick Ohly ebef300ae5 SyncSource: support asynchronous add/update in utility classes
SyncSourceSerialize uses ITEM_AGAIN plus a callback in
InsertItemResult to indicate that and how an insert (aka add or
update) operation must be continued.

TrackingSyncSource merely passes that result through until it finally
completes.

Direct, raw access to items as done by the command line and restore
operations is not done asynchronously. The backend does not need to
know how it is used, the SyncSourceSerialize wrapper around the
backend will flush and wait.

Asynchronous deleting is not supported. It would require throwing an
exception or an API change (better - exceptions are for errors!)
because removeItem() has no return code which could be extended.
2013-07-05 17:44:09 +02:00
Patrick Ohly 4f8615ee8b Logging: eliminate _instance from SE_LOG* macros
With the _instance parameter always being NULL thanks to the previous
patch, it can be removed completely.
2013-05-06 16:28:13 +02:00
Patrick Ohly b5befe6cbf Logging: remove usage of Logger instance
Passing an explicit Logger instance to the SE_LOG* macros was hardly
ever used and only made the macros more complex.

The only usage of it was in some backends, which then added a prefix
string automatically. The same effect can be achieved by passing
getDisplayName(). Exception handling no longer needs a Logger instance,
the prefix alone is enough.

The other intended usage, avoiding global variables for logging by
passing a logger known to the caller, was not possible at all.

To make prefix handling more flexible, it is now passed as a "const
std::string *" instead of a "const char *".
2013-05-06 16:28:12 +02:00
Patrick Ohly f1bbe9802a SyncSource: add API for creating and deleting databases
The API supports creating a database with a specific name and UID. To
avoid ambiguities, deleting is done via the UID.

Looking up the UID for a "database" property value is meant to be
supported by opening the SyncSource and then getting the current
database that was opened.

This allows implementing a delete operation that uses the same
database lookup method as syncing.

All of the new methods have stubs which reject the operation
respectively tell the caller that no information is available. This
was done to avoid having to adapt all derived SyncSources. Support for
the new functionality is optional.
2012-10-25 16:43:46 +02:00
Patrick Ohly 43371ac180 sources: force slow sync when listAllItems() returns no revisions
First, clarified that a source has to set all revision strings or
none. This was implied, but not spelled out explicitly.
Second, use that to detect when a sync must be done in slow mode.
Third, avoid calculating changes when that isn't needed. This last
point was left as TODO while in release preparations and can
be committed now.

This changes is needed for the PBAP backend which does not currently
set revision strings and thus cannot be used in incremental syncing.
2012-08-31 15:00:25 +02:00
Patrick Ohly 266656d8bd engine: add DTSTAMP+LAST-MODIFIED before writing calendar items
When writing calendar items into a backend storage as iCalendar 2.0 or
vCalendar 1.0, they should have DTSTAMP and LAST-MODIFIED values. DTSTAMP
is expected by some CalDAV servers (like Apple). LAST-MODIFIED is usually
added by the storage, but not always.

In the text/plain -> syncevolution -> text/calendar -> Radicale -> EDS
-> syncevolution chain the LAST-MODIFIED was not added by Radicale, which caused
problems for change tracking in an EDS-based SyncEvolution.
2012-06-29 11:36:32 +02:00
Patrick Ohly 3716ae2c9a core, WebDAV: improved support for aborting while sleeping
When waiting for resending a failed message, the sleeping couldn't be
interrupted when using the D-Bus server. It now uses the SuspendFlags
infrastructure and glib, which detects abort requests sent via D-Bus
in addition to those sent via signals (which already worked earlier).
2012-06-20 12:26:43 +02:00
Patrick Ohly 92d7e7cc48 engine: support VJOURNAL and VJOURNAL+text/plain
The "text/calendar" type now also contains rules for parsing and
generating VJOURNAL in iCalendar 2.0 and vCalendar 1.0. When using
"text/calendar+plain", the internal format is iCalendar 2.0 with plain
text used as alternative if the sync peer does not support VJOURNAL
for memos.

These types can be used as "databaseFormat" of the file backend and by
other backends which want to replace plain text as exchange format for
memos.
2012-06-15 14:20:25 +02:00
Patrick Ohly 5014f9b105 backends: merge with incoming data by default
All backends except for EDS replaced local data wholesale with an
incoming update, even if that update came from a peer which did not
store all properties. The EDS backend had already been configured
earlier to always merge remote and local data before writing it back.

Instead of updating each backend individually it makes more sense to
make the more intelligent (and more expensive) merging the default in
backends derived from SyncSourceSerialize/TrackingSyncSource. Backends
which do not want it can remove
"<updateallfields>true</updateallfields>" from
SynthesisInfo::m_datastoreOptions.
2012-06-15 12:25:52 +02:00
Patrick Ohly eafc81c08d command line: allow setting empty properties
Due to the way how properties were handled internally, it wasn't
possible to explicitly set a property to its default value. Instead
the property was unset. For example, explicitly setting database= was
not possible.

This is necessary for client-test and ActiveSync, because client-test
needs to know that the testing is expected to run with the default
databases (something which normally is avoided by overwriting empty
database properties).

Now the "is set" state is tracked explicitly in the config storage and
command line property APIs. Unsetting a property via the command line
could be implemented with an explicit command line option, but is not
supported at the moment.

Tests were extended to cover the new functionality and adapted to the
change behavior for "type" migration: syncFormat was empty
already (because the empty string matched the default), but
forceSyncFormat was unnecessarily set explicitly. Now it is not.
2012-06-07 14:16:58 +02:00
Patrick Ohly 314f073638 local + remote sync: negotiate UID support via SyncCap (BMC #22783)
This uses the new libsynthesis support for adding and checking entries
in the SyncCap to detect per datastore whether UID/RECURRENCE-ID are
truly globally unique and thus can be used to finding pairs. The
presence of the property alone is no guarantee for that.

Previously this kind of pairing was enabled only for local sync, which
was a hack which didn't work for local backends which didn't support
UID (for example, Maemo 5 calendar). It also didn't work for mixtures
of datastores with and without that kind of support.

"1122583000" was randomly chosen as pseudo sync mode. It is a number
because strings confuse Funambol. Note that SYNCMODESUPPORTED() only
works inside the compare script.
2012-05-03 09:45:36 +02:00
Patrick Ohly 12334eb676 SyncSource: remove special RegisterSyncSource::InactiveSource pointer
The special semantic of the former RegisterSyncSource::InactiveSource
(invalid pointer of value 1) caused bugs, like using it in
--print-databases (=> segfault) or not being able to store the result
of a createSource() directly in a smart pointer (=> potential leak in
SyncSource::createSource()).

Obviously a bad idea to start with. Replaced with a
RegisterSyncSource::InactiveSource() method which creates a real,
inactive SyncSource instance which can and must be deleted by the
caller.

This is a SyncSource API change for backend developers.
Instead of RegisterSyncSource::InactiveSource, return
RegisterSyncSource::InactiveSource().

Comparisons against RegisterSyncSource::InactiveSource needs
to be replaced with a call to the new SyncSource::isInactive().

User visible fixes:
* --print-databases: no longer crashes when EDS or KDE backends
  are not usable. Instead it prints "not enabled during compilation or
  not usable in the current environment".
* --print-databases: continues with other backends even if
  one backend throws an exception, as the KDE backend does
  when it cannot find Akonadi. Error messages are printed.
2012-03-09 07:25:11 +00:00
Patrick Ohly b5410acbd9 SyncSource: rewrote callback handling for operations
A few operations had a callback registration mechanism, but not
all. For testing multi-cycle syncing and injecting concurrent changes
while a sync runs it is necessary to add code for several more
operations (begin data read, add/modify/delete item).

Instead of adding more custom code, this commit replaces the
home-grown callback lists with a boost::function wrapper which
contains pre- and post-signals based on boost::signals2.

Also simplifies the Synthesis DB Plugin implementation by moving the
common code (null check, exception handling) into the wrapper.

This removed the LOCERR_OK return code from some plugin
implementations (end data read, start data write). Implementations
returning LOCERR_OK must be provided by the SyncSource. Done by
SyncSourceSession (used by TrackingSyncSource) automatically.
2012-03-06 14:03:22 +01:00
Patrick Ohly 4c270e4e80 SyncSource: support multiple sync cycles in the same session
The state of SyncSourceChanges and SyncSourceRevisions must be reset
in beginSync() to get change tracking right when the same session
includes more than the traditional single sync cycle.

SyncSourceRevisions::initRevisions() must clear its set of revisions
before filling the data structure again. Otherwise items removed as
part of a sync are not detected as deleted in the next cycle.

Any backend derived using these classes (like any backend derived from
TrackingSyncSource) should work now and all other backends should be
fixed to support multiple cycles, so the <canrestart> flag is set
unconditionally.

Still need Client::Source tests for this expected behavior.
2012-03-06 14:03:21 +01:00
Patrick Ohly 00912e9e18 SyncSource + SyncContext: added requestAnotherSync()
The long term goal is that using a specific sync source once more in
another, new sync session can be requested per source.

The actual implementation however is per-session, using the new
libsynthesis "restartsync" session variable. This functionality
sits in SyncContext, because it is the one which has access to
the session variables.
2012-03-06 14:03:21 +01:00
Patrick Ohly 26ebe2f299 SyncSource: grant control over "read-only" mode
The Synthesis engine already supports read-only datastores. All writes
for such a store are silently discarded. Now a SyncSource can select
that mode by setting SynthesisInfo::m_readOnly to true in its
getSynthesisInfo() implementation.
2012-03-06 14:03:20 +01:00
Patrick Ohly e69a818eb4 SyncSource: detect ambiguous "backend" value
Some backends react to the "backend" aliases
("calendar/todo/memo/addressbook"). If more than one such backend
is enabled and installed, then these aliases become ambiguous.
Used to be the case when Akonadi and EDS were both enabled.

This commit introduces an error message for that case.

In the case of Akonadi and EDS it is (temporarily) avoided
by not choosing Akonadi when only an alias is used.
2012-02-01 15:29:01 +00:00
Patrick Ohly cd7cc5e7bc testing: added activesyncd compilation + coverage
activesyncd gets compiled completely (which should always succeed) and
then only the required pieces are installed (without the things which
are hard-coded for /usr, because installing those will fail).

Testing is done by running a similar set of client-test tests as for remove
CalDAV/CardDAV servers.

activesyncd is started anew each time client-test is run, done in
the new wrappercheck.sh script. Can be combined with valgrindcheck.sh:
wrappercheck.sh valgrindcheck.sh activesyncd -- valgrindcheck.sh client-test ...

The return code of wrappercheck.sh is the return code of the real command
or the daemon, if the real command succeeded. This way the special 100 return
code from valgrindcheck.sh is returned if the only problems were memory
issues.
2011-12-08 13:59:32 +00:00
Patrick Ohly 0f1bed33fe command line: config and source names are optional (BMC #23783)
The need to add "foo" and "bar" pseudo config and source names to the
command line even when all parameters for the operation where
explicitly specified on the command line was confusing.

Now it is possible to invoke item operations without the config and
source name. Names which refer to non-existent configs are still
accepted, as in previous releases. Typos are handled better by
producing a detailed error report which includes (as applicable):
- config doesn't exist
- source doesn't exist or not selected
- backend property not set

This error report is created only if SyncSource was unable to create a
SyncSource, which is detected by catching the status exception with
the right error code. Trying to anticipate the error at the Cmdline
level would duplicate code found in the SyncSource and/or impose
limitations which might not hold in all cases.

Because luids used to be positional arguments after <config> and
<source>, a new --luids keyword is necessary to indicate that the
following parameters are luids and not <config> and <source>.

Added a CmdlineTest::testItemOperations for this new command line
syntax. It uses the file backend to run some real operations (missing
earlier).

Error reporting in the command line uses the same mix of writing error
messages and returning early (Cmdline) and throwing exceptions (rest
of SyncEvolution). Perhaps the Cmdline class should also use
exceptions - but not now.
2011-11-28 10:18:19 +01:00
Patrick Ohly ba289c899f phone sync: delete<->delete conflict + phone calendar+todo sync (BMC #23744)
When deleting an item on phone and locally, the next sync fails with
ERROR messages about "object not found". This has several reasons:
- libsynthesis super data store attempts to read items
  which may or may not exist (triggers ERROR message)
- it checks for 404 but Evolution backends only return a generic
  database error (causes sync to fail)

It turned out that ReadItem and DeleteItem are expected to return a
404 status when the requested item does not exist. This patch documents
that (only in the TrackingSyncSource, though), adds tests and fixes
EDS, WebDAV, file and sqlite backends accordingly.

This patch also suppresses the 404 error logging inside DeleteItem(),
while still returning that error code to the Synthesis engine. Not
logging that particular situation is consistent with the previous
SyncEvolution behavior of silently returning successfully when there
wasn't anything to delete.

In addition, more recent libsynthesis versions also no longer do
a ReadItem() call to test for existence. That would still trigger
a spurious (albeit now harmless) ERROR message.
2011-11-14 21:17:02 +01:00
Murray Cumming 6d57734e0a SyncSource testing: add unit test to check that some backends exist 2011-10-21 09:37:10 +02:00
Patrick Ohly fbe71f0ef7 engine: allow StartDataRead/beginSync to start early and late, as needed (BMC #22881)
The unconditional change of calling StartDataRead (aka beginSync)
early enough so that ActiveSync can force a slow sync had negative
consequences, because now it was called before the peer was contacted
and credentials were accepted:
- broke the "sync started successfully" logic, resulting in
  notifications for syncs which were supposed to be retried silently
  (showed up in TestSessionAPIsDummy.testAutoSyncNetworkFailure)
- database dumps were done even if not needed because sync never
  starts

Now all backends are called as before unless they explicitly ask for
the early call. The ActiveSync backend does that. The downsides of
that approach do not matter much because syncing will start okay and
dumping of data is typically disable on that side of a local sync.
2011-10-20 15:18:48 +02:00
Patrick Ohly 1421769db8 SyncSource: allow StartDataRead/beginSync() to force a slow sync
This is needed for ActiveSync, which can detect inside beginSync()
that the sync key became invalid and then has to fall back to a slow
sync.

It is done by asking the engine to call StartDataRead earlier (via
<plugin_earlystartdataread>) and documenting the special 508 error
code. Depends on support for both in libsynthesis, thus the bumped
version number of it.

Normally SyncEvolution is meant to never throw an exception unless
something fatal happens, so it would have been better to add a return
code to beginSync(). But that is a fairly intrusive change, so instead
the simpler "use exception" solution is used. The change can still be
made later on.
2011-10-17 13:17:58 +02:00
Patrick Ohly 765ed0f833 TestingSyncSource+CalDAV: added "delete all" API call
During testing, removing all items is done via a special call in
TestingSyncSource. It used to be an utility method which fell back to
the normal SyncSource API.

This patch changes several things
- Reversed order in which items are deleted in that utility method,
  because removing children (= longer IDs) first tends to be supported
  better by servers (bug in CalDAV server, but still...).
- Allow backends to implement their own removeAllItems().
- Implement that in CalDAV + MapSyncSource as removing the merged
  items directly, instead of using a sequence of PUT+final DELETE.

Found while testing with Bedework CalDAV server. Makes testing more
robust and efficient.
2011-10-17 13:17:40 +02:00
Patrick Ohly 8a640d2ae2 SyncSource API: support and use Synthesis DB_DataMerged/Replace/Conflict result codes (BMC #22783)
Returning 207 = DB_DataMerged when an item replaced an existing one
wasn't correct (the data wasn't really merged) and also wasn't handled
as intended by the Synthesis engine. Now a backend can properly report what it did:
- data fully replaced
- data was merged
- data needs to be merged by engine

The last option is used by EDS and CalDAV when an add<->add conflict
is detected by the backends. In that case the Synthesis engine will
read the local item, merge it with the incoming item and write back
the result.

At the moment the tests assume that the more recent item wins
completely. But our field list config specifies a merge=lines mode for
some fields, which results in concatenating these fields in the merged
item. Thus the tests currently fail. Need to decide which behavior is
desired.
2011-09-14 15:07:30 +02:00
Patrick Ohly 2773dfe47d testing: added test case for add<->add conflict (part of BMC #22783)
The test can be used in combination with a SyncEvolution HTTP server
or local sync. Currently doesn't pass because a libsynthesis with
support for DB_DataMerged is needed.
2011-09-14 15:07:30 +02:00
Patrick Ohly d65ed40287 backends: apply PHOTO import/export scripts by default (Apple CardDAV)
A recent Apple Calendar server (correctly) rejects the invalid
PHOTO;TYPE=unknown: property in a vCard. This internal representation
must be cleared before serializing the field list. Done by using the
VCARD_OUTGOING_PHOTO_VALUE_SCRIPT by default for all backends which do
not set there own script. The Evolution backend already included the
script.

Likewise, check PHOTO data after reading from the backend by
default (VCARD_INCOMING_PHOTO_VALUE_SCRIPT).
2011-08-26 08:31:41 +00:00
Patrick Ohly c293f51bba CalDAV/CardDAV + local storage: avoid empty properties
The main motivation for this change is that a recent Apple Calendar
server rejects vCards with empty BDAY property. Another reason is that
keeping the data as small as possible is desirable by itself.

Sending an empty property serves as a hint for the peer that the
property is supported. This is not necessary when storing an item in a
backend. Therefore this commit disables empty properties for all
backends which do not themselves set the m_backendRule Synthesis info
value.

The Evolution backend is not affected by this because it has its own
backend rule. That will be updated separately.
2011-08-26 09:45:04 +02:00
Patrick Ohly 6099214016 syncevolution.org binaries: work with both libneon and libneon-gnutls
When --enable-neon-compatibility is used, libneon.so.27 and
libneon-gnutls.so.27 are opened dynamically instead of linking against
them. The Debian package specifies that it depends on one of these two
libs in this case. Don't use --enable-neon-compatibility when not
enabling WebDAV to avoid this dependency.

This change is necessary for Debian Testing, which no longer has
libneon.so.27 at all. Note that the very latest Debian Testing also
has another problem because libnotify1 was dropped in favor of
libnotify4.
2011-08-17 10:04:21 +00:00
Patrick Ohly 24efbe11db testing: use target-config@client-test-$CLIENT_TEST_SERVER for Client::Source
For testing sources instantiated via SyncSource.cpp (for example, EDS)
and client-test-app.cpp (ActiveSync) append the name of the server set
via the CLIENT_TEST_SERVER env variable to target-config@client-test,
separated by a hyphen.

This allows configuring testing differently for different servers.
Was already done for CalDAV sources, inside WebDAVSourceRegister.cpp.
2011-08-15 17:32:01 +02:00
Patrick Ohly 4424c38548 source config: improved error message
An invalid syncFormat (like text/vcard for "Evolution Calendar")
can prevent instantiating the source. Therefore include that
value in the error message and clearly distinguish between
sync and database format.
2011-08-10 15:07:18 +02:00
Patrick Ohly 8032247a5a local sync: renamed "source-config" to "target-config"
As discussed on the mailing list, "source-config" is ambiguous because
the "addressbook/calendar/..." configs are also called "source
configs".

Now the naming is "sync" config (for the config with syncURL=local://,
because it is used for syncing) and "target" config (because it is
used as target in a sync config's syncURL).

Rejected:
"local" config - because the databases are not necessarily local
"source" config - see above
"client" or "server" config - because both sides might use local data
                              and/or client/server could refer to the role
                              of the peer or the SyncML client/server model
                              used internally
2011-06-28 18:42:43 -07:00
Patrick Ohly 7e542c596d SyncSourceRevisions: added updateAll[Sub]Items
There may be backends (like CalDAV) where updating existing
information about items is more efficient than reading them
from scratch. This commit adds updateAll[Sub]Items in SyncSourceRevisions/TrackingSyncSource resp. the SubSyncSource API for that purpose.

If it is called when some information exits (without it, the attempt
would be useless) and when change tracking may resuse existing
information. So an explicit slow sync still reads all items and wipes
out all old information, just in case that it is somehow
broken. Should not matter in most cases, though, because wrong luids
and mismatching revisions will be detected and corrected. The only
situation where incorrect information would matter is correct
luid+revision with wrong meta information attached in MapSyncSource
(subids, UID, ...).
2011-06-25 17:23:37 +02:00
Patrick Ohly 415e33bada sync sources: added support for avoiding listAllItems()
If a sync source can quickly determine that nothing has changed,
then SyncSourceRevisions::detectChanges() can use a shortcut and
simply copy the list of known uids => CHANGES_NONE mode.

Such a change detection will be possible in the WebDAV backends (using
the Calendar Collection Entity Tag (CTag) as "database revision"
string) and perhaps in the future also in Evolution Data Server (after
adding a new API).

This commit also adds a CHANGES_SLOW mode. This is meant as hint that
detecting changes is not necessary. Right now, this mode is the same
as CHANGES_FULL because the code changes should be minimized at this
time (in preparation for 1.2). More work will probably be needed
to distinguish between unit testing and real real slow syncs.

Finally, a way to pass information about the cached item list is added
with the SyncSourceRevision::setAllItems() call. setAllItems() and
listAllItems() are mutually exclusive: either the backend delivers
the information, or it receives it. The CalDAV backend depends on this
because it needs to maintain a cache with information about all items.
2011-06-24 17:09:18 +02:00
Patrick Ohly 86ad81e932 Synthesis engine: allow arbitrary <datastore> config properties
SyncSources are now allowed to insert arbitrary config properties
into the <datastore></datastore> config. The main motivation is that
some backends need to enable <updateallfields>.
2011-06-21 11:42:33 +02:00
Patrick Ohly c1a2d81e91 virtual source: added better config checking and error reporting
An empty "database" property for a virtual source lead to a useless
error about instantiating a source with an empty name. Now the error
is reported for the virtual source and includes the "database" value.

A sanity check that the source really had exactly two sub-sources
was missing. Added.
2011-04-19 13:58:04 +02:00
Patrick Ohly 0574084362 SyncSource: better error message for incomplete config
The file backend refuses to create a source if backendFormat is unset.
This was reported as "<source>: backend 'file' not supported",
which was confusing because the backend was in fact supported.

Now the error message is "<source>: backend 'file' not supported or
not fully configured (format '<format>')".
2011-04-19 13:56:02 +02:00
Patrick Ohly fe95a19f4a ItemCache: allow backup dumps to restart
Added ItemCache::reset(), will be used by WebDAV backend if dumping
data fails in the middle and needs to be retried.
2011-04-15 14:24:30 +02:00