The main motivation for this change is that a recent Apple Calendar
server rejects vCards with empty BDAY property. Another reason is that
keeping the data as small as possible is desirable by itself.
Sending an empty property serves as a hint for the peer that the
property is supported. This is not necessary when storing an item in a
backend. Therefore this commit disables empty properties for all
backends which do not themselves set the m_backendRule Synthesis info
value.
The Evolution backend is not affected by this because it has its own
backend rule. That will be updated separately.
If --enable-notify-compatibility is used, libnotify is not linked
directly into syncevo-dbus-server. Instead libnotify.so.1 till .so.4
(current Debian Testing) are opened opened dynamically and the
necessary functions are looked up via dlsym(). Not finding the
libraries or the functions silently disables this notification
mechanism.
When (auto-)migrating a config, it was possible that a name for the
peer, say foo.old, was chosen for the renamed config although there
was already such a config, for example foo.old in ~/.sync4j. Besides
being confusing for users, this also led to a bug in the code where it
copied from the older config with the foo.old name.
Found when adding unit testing of migrating configs with auto syncing,
because then the content of the migrated config mattered.
The main problem fixed in this commit is the disabling of auto syncing
in the old config. Done similar to ConsumerRead, by reading/writing
the .ini (resp. .txt) file directly.
"syncURL = insert your URL here" with "autoSync = 0" did lead to auto
sync attempts although it wasn't enabled. A check for "auto syncing
enabled" was missing for the "unknown transport" case.
When --enable-neon-compatibility is used, libneon.so.27 and
libneon-gnutls.so.27 are opened dynamically instead of linking against
them. The Debian package specifies that it depends on one of these two
libs in this case. Don't use --enable-neon-compatibility when not
enabling WebDAV to avoid this dependency.
This change is necessary for Debian Testing, which no longer has
libneon.so.27 at all. Note that the very latest Debian Testing also
has another problem because libnotify1 was dropped in favor of
libnotify4.
The code which removed TZIDs after a e_cal_get_component_as_string()
fell back to icalcomponent_as_ical_str(). That is incorrect, because
we get a VEVENT instead of the required VCALENDAR that way. Must
continue to use e_cal_get_component_as_string().
Found when running with a broken libical (compiled from source) that
couldn't resolve its own internal TZIDs.
For testing sources instantiated via SyncSource.cpp (for example, EDS)
and client-test-app.cpp (ActiveSync) append the name of the server set
via the CLIENT_TEST_SERVER env variable to target-config@client-test,
separated by a hyphen.
This allows configuring testing differently for different servers.
Was already done for CalDAV sources, inside WebDAVSourceRegister.cpp.
The gdbus utility library crashed here if interface->properties was NULL:
for (property = interface->properties; property->name; property++)
^^^^^^^^^^^^^^
Fixed by checking property for non-NULL in the for loop.
More global initialization/destruction ordering problems appeared
on current Debian Testing (gcc 4.6.1, binutils 2.21.52.20110606-2):
- WebDAVCredentialsOkay registered and visibility modified before
constructed, which later resets visibility
- test setup uses config before config properties are registered
- logger stack deallocated before later destructor attempt to use it
Fixed with:
- WebDAVCredentialsOkay() singleton
- delay all work in test setup until really needed
- never deallocate logger stack, use singleton
The code which removed TZIDs after a e_cal_get_component_as_string()
had a memory handling problem: icalproperty_get_next_parameter()
after removing the parameter accesses freed memory.
Found when running with a broken libical (compiled from source) that
couldn't resolve its own internal TZIDs.
Instead of fixing the "remove all TZIDs" approach, let's use the
simpler "remove first (and probably only) TZID". Also added some
debugging code.
This finishes the partial commit for this problem which was accidentally
committed earlier as part of af43ac9.
An invalid syncFormat (like text/vcard for "Evolution Calendar")
can prevent instantiating the source. Therefore include that
value in the error message and clearly distinguish between
sync and database format.
Sony Ericsson seems to use ISO-8859-1 for all their phones. This
causes two problems:
- mangled characters and/or
- crashes inside libecal/gdbus
It is uncertain whether all devices have this problem. A poll did not
yield any results
(http://syncevolution.org/blogs/pohly/2011/question-sony-ericsson-users-charset). So
let's change it for those who have reported problems.
To revert the change, copy
/usr/share/syncevolution/xml/remoterules/server/00_sony_ericsson.xml
into $HOME/.config/syncevolution-xml/remoterules/server (after
creating that directory) and remove the lines with ISO-8859-1.
This was originally reported for contacts, and now also for calendar
data. The calendar case was seen as a crash of the syncevo-dbus-server:
GLib-CRITICAL **: g_variant_new_string: assertion `g_utf8_validate (string, -1, NULL)' failed
Program received signal SIGSEGV, Segmentation fault.
g_variant_is_trusted (value=0x0)
at /build/buildd/glib2.0-2.28.6/./glib/gvariant-core.c:600
in /build/buildd/glib2.0-2.28.6/./glib/gvariant-core.c
Thread 1 (Thread 0xb7fce850 (LWP 3402)):
at /build/buildd/glib2.0-2.28.6/./glib/gvariant-core.c:600
at /build/buildd/glib2.0-2.28.6/./glib/gvariant.c:3081
at /build/buildd/glib2.0-2.28.6/./glib/gvariant.c:4093
app=0xbfffca4c) at /build/buildd/glib2.0-2.28.6/./glib/gvariant.c:4248
at /build/buildd/glib2.0-2.28.6/./glib/gvariant.c:4188
in_calobj=0x8435fe8 "BEGIN:VEVENT\r\nSUMMARY:THW Sim Pr\374fung\r\nDTSTART:20070420T230000Z\r\nDTEND:20070421T225900Z\r\nBEGIN:VALARM\r\nTRIGGER;VALUE=DURATION:PT45M\r\nACTION:DISPLAY\r\nEND:VALARM\r\nEND:VEVENT\r\n",
Evolution shows a meeting twice on the day of a modified recurrence,
if the meeting series was originally created and modified in Exchange,
then imported into Google Calendar.
The reason is that the RECURRENCE-ID in that case ends up being
in UTC, even if the parent event has a time zone. Evolution and/or
libical seem to have a bug here, IMHO they should recognize that the
RECURRENCE-ID time is the same.
SyncEvolution now works around this by transforming the UTC
RECURRENCE-ID time into the time zone of the DTSTART time of the
parent event. This is combined with removal of X-LIC-ERRORS into
a new fixIncomingCalendar() method which needs to be called
whenever a complete VCALENDAR is received from the CalDAV server
(adding data from a report, GET).
This transformation changes the "rid" part of the item IDs. It should
be okay in a two-way sync (remove one sub item, add another) and
a slow sync (libsynthesis recognizes the times as equal).
Added some more functions needed to work with DTSTART and RECURRENCE-ID.
Removed icalproperty_vanew_lastmodified() because its entry was incomplete
and unused.
Nightly testing did not resend a PUT when it could (should?)
have because the deadline was exceeded. Added logging to
track down what the chosen deadline is.
Instead of aborting the test at the first item which
fails to import, collect the error messages (from the
exceptions), do the comparison of imported data
against the reference data and then either report
the import failures (if there were any) or the comparison
failure (if that broke).
The advantage is that a single run of the test shows
all problems that exist.
Use checkPassword() before trying to --print-items. This will either
prompt for the password at command line, or ask it from the keyring
if --keyring is specified (or if using the dbus server).
Google Calendar checks whether a CalDAV client is allowed to update
particular events. In combination with meeting invitations this has
led to know issues where desirable changes were rejected (see
http://code.google.com/p/google-caldav-issues/issues/detail?id=38).
Details are murky whether that bug is still open. I hit the "403 You
don't have access to change that event." problem both in updating
a meeting series and creating it.
This commit ensures that when updating fails, the errors is treated as
a temporary, per item error (417). The sync session then continues.
The overall result will be STATUS_PARTIAL_FAILURE = 22001 and the
next session will retry the same item. This is better than aborting
the session (situation without this patch) or ignoring the problem
(alternative solution).
The same error is not handled when creation fails. This might need
further investigations.
During an incremental sync, when an unmodified meeting series on the CalDAV
server had to be extended (= adding a detached recurrence), the operation failed
with "event not found".
The root cause was a bug in updateAllSubItems(): due to a copy-and-paste bug,
it cleared the cache instead of adding the modified items to it. Therefore
unmodified items, added to the cache earlier, where not found later on.
Normally they weren't needed. The exception is reading in preparation
for adding a detached recurrences.
Fixed by removing the m_cache.clear(). That is valid in this case even
if the operation is repeated, because adding already read items will
simply overwrite them.
Also added some debug logging which helped to track this down.
If the same event was deleted both locally and in the CalDAV server, syncing
failed with "event not found".
It is normal that the Synthesis engine requests the removal of
non-existent items. CalDAVSource needs to handle that in
removeSubItem() and getSubDescription().
Item operations like --print-items failed if the configuration didn't exist:
[ERROR] : virtual read-only configuration node, cannot write property webDAVCredentialsOkay = 1
This was caused by trying to write the webDAVCredentialsOkay property
into a temporary, read-only configuration. The fix is to check for
read-only configs before attempting to use the property.
Alternatively it would have been possible to catch exceptions, but it
is not obvious which errors can be ignored.
Some users of ConfigNode (like the WebDAV backend) need to know
whether setting a optional property will succeed before attempting
it. When they know that it fails because the node is read-only, they
can skip setting it.
The alternative would be to throw a well-defined exception and catch
it, but that is not in line with the SyncEvolution design. Exceptions
should be for real, unexpected errors.
Notifications were meant to be shown for all errors except temporary
ones (http://bugzilla.moblin.org/web/bug_report_10000.html). This has
never been implemented correctly since the feature was introduced:
instead of hiding known temporary errors, all errors except 500 (fatal
error) were suppressed.
This commit switches to white listing the known errors which are
temporary and suppresses those. Right now that happens to be only one,
network problems.
Use the new READ() script method in libsynthesis to inline local photo
data right before sending to a remote peer.
Tests were added in combination with SyncEvolution server for inlining
a special well-known file (testcases/local.png) and failing to inline
(file doesn't exist). In the latter case the URI is sent unchanged.
The VALUE parameter was ignored completely and thus got lost in the
Synthesis engine. Added it as a string.
The TYPE parameter was an incomplete enum. Better allow arbitrary
strings.
Both parameters must be copied together with the PHOTO data they
belong to. In combination with merge="fillempty" this can be
problematic: if one of the parameters is empty, it may be overwritten
although the PHOTO data is not copied.
This problem is solved by ensuring that the internal field list never
has empty PHOTO_TYPE/VALUE fields. This is done by setting "binary"
resp. "unknown" when importing contact data from a peer or the local
Evolution backend and removing them again before sending to a peer or
storing.
The same change needs to be done with other backends. It is not made
mandatory because some backends (like file) might want to store these
values explicitly.
Because it is unknown which peers support VALUE=uri, only the
CLIENT_TEST_SERVER=syncevolution testItem testcases contain a contact
with such a PHOTO.
When neither Network Manager nor ConnMan are running, network presence was "not
online". This prevented running automatic syncs. Primarily (exclusively?) affected
nightly testing in the minimal chroots.
The solution is to detect when both managers were not found on D-Bus and
then set the status to "online".
Evolution and the MeeGo UX assume that first/middle/last name are set.
That is not the case when a contact is created in the Google Contacts
web interface. Such contacts are sent by Google without the N
property. That is surprising because Google supports the N property
(it is returned for contacts created via SyncML, and internally Google
distinguishes between the name components).
This commit adds a workaround for all peers (not just Google) which
ensures that first/middle/last name are set whenever possible. If FN
is set, it is split so that the first and last word are first/last
name, the rest is the middle name. As a special case a trailing comma
at the end of the first words is recognized as "Doe, John Middle"
format.
If the full name is empty, the first email address and (if that is also
empty) the first phone number are used as additional fallbacks for the
first name.
syncevolution.org binaries are compatible with Evolution 2.32 thanks
to loading libecal/libebook dynamically. Release 1.1.99.5 warned about
the libs from Evolution 2.32 as "might not be compatible!" which is
unnecessary. They are compatible.
Bumping the version numbers to avoid the warning.
Most of the selected theme icons don't exist, so adding these values
only provides the possibility to add icons later on.
For Google Contacts, the existing gmail icon is used as a short-term
solution until the theme gets a proper google-contacts icon.
Due to too restrictive checking of the syncURL, configs without http
or obex-bt were never executed automatically. This commit adds a
fallback which enables "other" configs to run without checking for
peer presence.
In addition it marks all "local" syncs as needing HTTP
connectivity. This is a simplification that fits the current use
cases, but needs to be enhanced later on.
Auto-sync sessions did not properly activate their D-Bus support and
thus couldn't be accessed via the Session D-Bus API. Must have
affected showing progress of such sessions in the GTK sync-ui.
They also weren't kept around for one minute, like the sessions
started by a client. Therefore UIs which need to retrieve information
about a completed session failed for a second reason.
Fixed by adding the necessary "activate()" and use "addTimeout()"
trick for session expiry also in the AutoSyncManager. The later was
moved into DBusServer for that.
These issues were found with the new
TestSessionAPIsDummy.testAutoSyncFailure D-Bus test.
When a config was set to "auto-sync on", the AutoSyncManager added it
to a list, but did not prevent the server from shutting down. Not sure
how this was meant to work or when it broke.
Fixing it is easy: as long as AutoSyncManager has at least one config
lined up for auto-syncing, it holds a reference on the AutoTerm
instance and thus prevents shutting down.
This commit also adds test cases for various situations:
- prevent shutdown while auto-sync on
- re-enable shutdown while timer is running
- re-enable shutdown while timer is off
g++ 4.6 and ld 2.21.52.20110707 (Debian Unstable) led to a different
order of global instance construction:
1. WebDAV constructor calls SyncConfig::getRegistry()
2. getRegistry() adds the (uninitialized!) property instances
and modifies them
3. SyncConfig.cpp instances are initializes, which resets
some of the values modified by getRegistry()
The result was that, for example, the "defaultPeer" property was
treated like an unshared property and written into the wrong config
file.
The assumption that variables in a compilation unit are initialized
before methods in that unit can be called is not based on anything in
the C++ standard. Therefore this commit rewrites the code so that
properties are not added/updated inside the getRegistry()
methods. Instead this is done in separate classes which (and that is
guaranteed by the C++ standard) are constructed after the properties
defined earlier in the compilation unit.
g++ 4.6 complains about the unused assignment. Probably this boolean
result needs to be checked. But as GDBus will be replaced soon anyway,
don't bother now.
In contrast to the Yahoo template, this one doesn't mention
a specific service and enables both contact and calendar sync.
To be used with a service that supports auto-discovery.
The previous commit added a check for qmake, but then used QMAKE
without checking whether qmake was found at all. This caused configure
problems on systems where qmake wasn't available.
The error handling also wasn't correct. A "test" was missing in front
of the comparison.
Commit 3f1185, contained in 1.1.99.3, changed
SyncContext::throwError() so that it throws a StatusException with
STATUS_FATAL. Previously a runtime exception was thrown, which
Exception::handle() recorded as a local error.
This commit fixes that regression by throwing a STATUS_FATAL +
LOCAL_STATUS_CODE, which restores the traditional result of
throwError().
Found by test-dbus.py TestDBusSyncError.testSyncNoConfig.
Parsing the revision map extracted the wrong subset of the string. As
a result, the revision comparison was broken and reported more changes
than really existed. Showed up as a failure in
Client::Sync::eds_event::testOneWayFromClient and requests for items
in a multiget when it wasn't needed.
The "retry on 401" code wasn't active during the initial sync
because the fact that the credentials had been accepted before
was only recorded on disk, but not in memory.
Moving the response handling from the data element to the response
element caused problems with Google, because it sends a 404 status for
the collection with no data. Apple Calendar Server didn't do that when
testing the change manually, so the problem only showed up in the
nightly testing.
This patch restores the previous behavior of simply ignoring responses
with no data. Some better error handling might be useful.
As discussed on the mailing list, "source-config" is ambiguous because
the "addressbook/calendar/..." configs are also called "source
configs".
Now the naming is "sync" config (for the config with syncURL=local://,
because it is used for syncing) and "target" config (because it is
used as target in a sync config's syncURL).
Rejected:
"local" config - because the databases are not necessarily local
"source" config - see above
"client" or "server" config - because both sides might use local data
and/or client/server could refer to the role
of the peer or the SyncML client/server model
used internally
The Funambol template hadn't been updated and the command line
tests failed because the didn't expect the PeerName to be set.
The normalization of "= F" to "= 0" broke the "= Funambol" peer name.
Doesn't seem to server any useful purpose anymore, so removed.
The code which caught the 404 status had the unintended side effect of
also catching 401 errors and then not reporting them. Fixed by
handling the exception as in the default "Exception" case if it does
not fit the 404 special case.
Saw an unexpected 401 error in the middle of a sync. At that point
the credentials should have been recognized as valid, but somehow
weren't added debug output to track down the problem.
Updating an item must be done with the same UID that was originally
set by the server. The Maemo 5 backend replaces the UID received from
the server with its own sequential numbering of events in the SQLite
database.
This commit is an attempt to catch this situation and restore the
correct the UID before sending the update item content to the server.
The value in the key/value pairs now start with a slash. The intention
is that if the content ever has to be extended, it can be done by
adding a version number or something like that in front of the
slash. Right now, that version is implicitly empty. Without the slash
it wouldn't be possible to distinguish to distinguish the future
version number from the revision.
All members except for the integer port were auto-initialized.
This commit fixes valgrind warnings in WebDAV in isEmpty()
by initializing the port in a new constructor.
Trying to reuse the TrackingSyncSource change tracking was a dead-end
that just led to horribly complex scaffolding classes (like the key/value
node which had to keep revisions synchronized and mix in UID).
This is a complete rewrite where change detection is done in
MapSyncSource, using a similar approach as in TrackingSyncSource. Some
of the session life cycle is now cut-and-pasted from
TrackingSyncSource (primarily checking the overall database
revision). Eventually this common logic might get refactored into a
SyncSource utility class, but for now let's keep it separate.
This solution is much cleaner and uses simpler key/value storage
with one item-<mainid> entry for each merged item, mapping to the
revision, UID, and list of subids.
The multiget result processing had to problems: etag wasn't requested
and thus not stored and existing subids were not removed for already
existing items.
The comparison between full path and local ID and between etag and
revision was completely mixed up, with the result that all items were
always loaded anew even if unchanged.
Fixed by storing the reduced luid+revision mapping and using it in all
further comparisons against the SubRevisionsMap_t.
The API was intentionally changed to notify backend developers of the
new possibilities. The SQLite backend hadn't been adapted and failed
to compile...
The recent commit for storing UID in the tracking source together with
revision information was incomplete.
First, the transformation from %d-%s to %d/%s/%s hadn't been finished
and wouldn't have worked because one return parameter of
splitMainIDValue() was not declared as reference (found via compiler
warnings in the nightly build by g++; I was using clang, which didn't
complain).
Second, there were cases that led to MapConfigNode::setProperty()
being called without having m_uids set first. For example,
listAllItems() + detectChanges(). Now all methods which return luids also
set UID values in the MapConfigNode. In addition, setProperty() is
more careful about preserving existing UID information.
This incidence shows that coding on a plane is no good, and that the
reusal of TrackingSyncSource as base for MapSyncSource really was a
mistake because of the complexity. Should really be rewritten.
The recent commit for the multiget REPORT used the temporary
string from stringstream after it was freed, because Neon::Request
doesn't copy the request body. Must make a copy in the caller.
This commit implements updateAllSubItems(). A first query
retrieves the etags of all items. A comparison determines
removed items and those which are new or updated. Those items
are then fetched with a multiget REPORT and used to complete
the cache and item list.
404 errors, as they are possible when Google Calendar gets
confused, are intentionally not handled. The rationale is
that a slow sync has a suitable workaround (use data from the
query REPORT) and hopefully the problem will occur less often
for future calendar changes.
The handling of href and etag data had to be done by all
classes using initResponseHandler(). Now it is in the XML
handler itself.
In addition, users of the class process the gathered results
at the end of the results XML element. At that point, all
the response parts are guaranteed to be processed.
There may be backends (like CalDAV) where updating existing
information about items is more efficient than reading them
from scratch. This commit adds updateAll[Sub]Items in SyncSourceRevisions/TrackingSyncSource resp. the SubSyncSource API for that purpose.
If it is called when some information exits (without it, the attempt
would be useless) and when change tracking may resuse existing
information. So an explicit slow sync still reads all items and wipes
out all old information, just in case that it is somehow
broken. Should not matter in most cases, though, because wrong luids
and mismatching revisions will be detected and corrected. The only
situation where incorrect information would matter is correct
luid+revision with wrong meta information attached in MapSyncSource
(subids, UID, ...).
The testLinkedItem tests updated items without actually
modifying their content. That worked as long as LAST-MODIFIED
increased, but because of the recent speed-ups that was no
longer always the case. The test then failed because the ETag
of a CalDAV resource didn't changed on the Apple Calendar Server.
This commit fixes that by modifying the item content when it is
to be used as an update.
One test didn't take that into account and continued using the
unmodified item content. Fixed.
The UID for each CalDAV resource is required to handle additions to
that meeting series correctly. The previous commits for using cached
results broke the linked items tests because the UID wasn't available
unless the calendar was read completely.
This commit fixes that by caching the UID in the rev- entries of the
tracking node. It also cleans up the naming to avoid confusion around
luid (resource path), uid (part of the item data), and subid (again
part of the item data, but needed to address individual VEVENTs).
Note that the code has become fairly complex due to reusing the
TrackingSyncSource as base class of MapSyncSource. In retrospect it
seems better to not reuse that particular logic and instead do all of
the change tracking directly in MapSyncSource. But that is a more
intrusive change and thus not done at this time.
This commit changes the trade-off between "duration of sync" and
"memory consumption" in favor of "faster sync": when all items need to
be listed (currently the case in an initial sync or if the server's
database has changed, according to the CTag), then the complete
calendar is downloaded in one go with REPORT and cached in memory.
It used to be downloaded already before without caching it. The reason
was that there was a (futile) attempt at reducing the download size by
requesting only the minimum set of properties. It was futile because
all servers ignored that hint and sent complete items. Therefore, and
because the case isn't occuring as often as it used to, it makes sense
to avoid the expensive (latency!) GET requests in favor of using more
memory.
Each sync involving WebDAV did a complete data dump (dumpData) and
showed differences (printChanges) for the WebDAV side of the sync.
This could be used to restore the WebDAV server after a sync, but it
seems a bit excessive and not useful for most users because the same
is also done on the local side of the sync.
Therefore this patch sets these two options to off in the
configuration templates.
There was a GET before a DELETE, for two reasons:
- If the item still has VEVENTs left, a PUT instead of
a DELETE is necessary. Now the code checks for this
special case and only GETs the item when a PUT follows.
- Providing a description for debug output required
access to the item content. Now the item content is
only used if already loaded.
The CTag mechanism allows to quickly check whether data has changed.
With WebDAV and the recent infrastructure changes in
SyncSourceRevisions/TrackingSyncSource, that is straight-forward.
With CalDAV it is a bit more complicated because the m_cache needs to
be populated for some of the operations to succeed. This is
accomplished via the setAll[Sub]Items() calls.
If a sync source can quickly determine that nothing has changed,
then SyncSourceRevisions::detectChanges() can use a shortcut and
simply copy the list of known uids => CHANGES_NONE mode.
Such a change detection will be possible in the WebDAV backends (using
the Calendar Collection Entity Tag (CTag) as "database revision"
string) and perhaps in the future also in Evolution Data Server (after
adding a new API).
This commit also adds a CHANGES_SLOW mode. This is meant as hint that
detecting changes is not necessary. Right now, this mode is the same
as CHANGES_FULL because the code changes should be minimized at this
time (in preparation for 1.2). More work will probably be needed
to distinguish between unit testing and real real slow syncs.
Finally, a way to pass information about the cached item list is added
with the SyncSourceRevision::setAllItems() call. setAllItems() and
listAllItems() are mutually exclusive: either the backend delivers
the information, or it receives it. The CalDAV backend depends on this
because it needs to maintain a cache with information about all items.
The flag should only have been added to old configurations. Adding it
to templates (which happen to match the version check) is wrong and
must be avoided. Because of the wrong flag, the "SyncEvolution"
template was shown in the MeeGo UX UI although it shouldn't have been.
If we get a 404 error while contacting the server, it might mean
that the username was wrong, so the server gave us a not found
error. It's better to let the user know that, because we don't
have a clear heuristic to determin whether this might have been
a true 404 error.
The convertion of 404 errors to 401 should happen only if the URL
we're trying to open is one in which it was us who injected the
username into the URL. This was achieved by removing the username
injection from the context creation code, and moving it into the
loop that does the autodiscovery, adding it path by path as it
was necessary.
Notice: this required NeonCXX to be aware of the "%u" semantic,
something I'm not completely comfortable with.
See also: https://bugs.meego.com/show_bug.cgi?id=17862
The TYPE=HOME seems to be redundant. The Evolution UI doesn't distinguish
types, and therefore our internal field list also doesn't.
The motivation for removing the TYPE is that it breaks the
testExtentions test with Google, because the TYPE gets lost in our
sent data.
Local data not supported by a peer (for example, X- extensions and
vCard 3.0 properties) was lost when importing updates from such a peer.
The Synthesis engine can preserve such extensions, but doesn't
apply that more expensive merging unless a backend is configured
with <updateallfields>.
This patch adds it for the Evolution contact and calendar backends.
SyncSources are now allowed to insert arbitrary config properties
into the <datastore></datastore> config. The main motivation is that
some backends need to enable <updateallfields>.
EDS can store arbitrary vCard extensions. These used to get lost in
two-way syncing because the engine couldn't convert them into the
internal field list. This patch adds a catch-all field (XPROPS) and a
match for unknown X- extensions.
Enabling the preservation of local extensions still needs to be enabled
separately for each backend which wants to use the logic (<updateallfields>).
The CtCap information is necessary for preserving fields not supported
by the Google SyncML server. It depends on the "overridedevinf"
patches for libsynthesis, currently under review.
Backends don't have access to ClientTest::update. Must provide
a pointer to it in the test config => ClientTestConfig::genericUpdate.
Manipulating N/FN is problematic because some peers support FN, some
don't => better update or add a NOTE instead when dealing with vCards.
Must also work for NOTE;CHARSET="UTF-8": test case, so the matching
against properties in the item is very inprecise.
The rule="KDE" properties are only used internally. They are never
sent to peers via SyncML. It is debatable whether they should be
listed in the CtCap sent to peers: on the one hand, receiving them
probably works (untested). On the other hand, the peer will never get
them back because the content will be encoded using the other
properties instead.
Without further hints, the Synthesis engine includes all properties in
the outgoing CtCap. This patch changes that by explicitly setting the
show="no" parameter to the internal properties.
This patch replaces the ugly configuration translation with a runtime
check in a comparescript.
This is a first step towards detecting properly at runtime whether a
peer supports UID/RECURRENCE-ID semantic in calendar data. Currently
this check is still based on the "local sync == use UID/RECURRENCE-ID"
shortcut.
This patch depends on a libsynthesis which supports the COMPAREMODE()
method.
Testing with Google Calendar showed another abort due to connection
problems:
"Could not read status line: SSL error: decryption failed or bad record mac"
Retry in this case, as in the "Secure connection truncated" case before.
contactServer() wasn't called before SyncSourceRevisions asked
for all items, which then failed with a boost::shared_ptr exception
on m_session.
Fixed this by wrapping the operations in
WebDAVSource::backup/restoreData which call contactServer() before
invoking the original implementation.
Function composition with boost::bind() might be nicer, but didn't
work right away (compile errors due to invalid syntax?) and thus
isn't used.
GErrorCXX was originally added to KCal-EDS. Copying it back because
it is useful.
GListCXX wraps a GList or GSList in a STL compatible list with forward
iterators. Appending (= push_back) is supported with both, but will be
slow for GSList.
The "database" property now can hold the final URL of the collection
(aka database) on the WebDAV server which is used for the
source. Setting it skips the entire auto-discovery process, which
makes access quite a bit faster and more reliable.
Because most UIs won't know initially how to find these URLs (listing
them not supported by the core SyncEvolution) and/or won't have the
necessary user dialogs, the property is set automatically after a
successful sync. This avoids the accidental switching between databases
when the user adds or removes databases on the server (which can lead
to different results of the auto discovery).
There's still no guarantee that the database picked by default is the
"right" one. That can only be solved in the UI because servers
typically don't have a "default" or "personal" flag for their
collections.
The conversion from a parsed URI to a string URL dropped the query
part and introduced redundant characters when port, userinfo and
fragment were empty.
It also introduced an extra slash before the path, which broke Google
Calendar with cached URLs (400 "bad request" error when using a path
with double slashes at the beginning).
NetworkManager 0.9 changes the values of
org.freedesktop.NetworkManager.Status property. Fortunately the new
and old values are not in conflict.
Commit also starts setting presence to false only when we know this
should happen (and not the other way around): It's better to fail
this way than prevent user from syncing if things like this happen.
NetworkManager 0.9 is more strict about the call arguments: it seems
newer dbus-glib requires that the interface name is specified, otherwise
there is an AccessDenied error.
Google CalDAV does not deal well with a detached recurrence that has
no parent. The "Google child hack" avoids *adding* such a recurrence,
but it missed the case where the same problem occurs when removing the
parent before the children. Added similar code to the partial removal
code path.
There were still several cases where the "Google delete hack" (= catch
409 "Can't delete a recurring event except on its organizer's
calendar", update item, delete again) did not work.
Sometimes it failed when only an EXDATE was set => also remove
EXDATE, not just RRULE, to convince the server that the event is not
recurring.
Sometimes it failed because the DELETE came too soon after the PUT
(?!). Added a retry loop.
When updating a meeting series by adding a child, the series' SEQ
value was not increased if the new child already had a higher SEQ
value. Then the updating failed because the other events in the series
didn't have a higher SEQ than on the server.
Fix this by always bumping first, then comparing against the new
event.
Timeouts for the SyncML messages between master and child did not
make sense. In HTTP they are necessary to detect dead peers, but with
permanent pipes between both sides that isn't an issue.
Because the timeouts triggered incorrectly, this patch removes them by
never setting the m_timeoutSeconds member variable. For sending the
child's status report a fixed timeout of 5 minutes (the previous
default) continues to be used, just in case that something goes
horribly wrong (software bug) and sending the report somehow hangs
(which it shouldn't).
Header files must be listed explicitly in autotools. Otherwise
they are not getting included in source tar balls. "make distcheck"
exposes that problem.
Google temporarily redirects to a special URL when the calendar is
down. The check() function already recognized this and just told
the caller to try again. But Session::propfind*() methods threw
a redirect exception themselves before giving check() a chance to
catch the special case.
Solved by keeping ne_propfind_handler instance valid during the check()
call (necessary because deleting it would also delete the ne_status
needed by check()) and calling check() directly with all the needed
information.
The handler will be deleted by a destructor class in combination with
boost::shared_ptr when leaving propfindURI() or when trying again
with a fresh handler in the retry loop.
The description of the "password" property was "SyncML server",
which was used by the command line password prompt:
"Enter password for SyncML server: ..."
Given that the property now also is used for CalDAV credentials and
the prompt did not tell the user for which SyncML server they were
asked, it is better to use the config name as description. Now the
password prompt is, for example:
"Enter password for @google-calendar: ..."
Keyring access is not affected by this change because the password
description was not used to find or describe the stored password.
There still was a TODO in the code for handling "-" as password value.
No surprise, not having that implemented broke CalDAV sync in
syncevo-dbus-server because it would try to read the password from
stdin (the default in SyncContext).
Probably SyncContext shouldn't provide such an unsafe fallback, but
that's something for another patch.
This patch addresses the immediate problem by moving the
initialization of the SyncContext used by the child process into the
master process and adding the password checking directly afterwards
(LocalTransportAgent::start()). It runs in the main process
(syncevolution or syncevo-dbus-server) and uses the "request password"
method of the main sync context. Passwords are then stored
temporarily, so the same check doesn't have to ask for passwords again
in the child process.
Long term we'll need to rewrite the complete password handling...
This was replaced at some point with iterating over registered properties
(see SyncContext::sync()) without removing the obsolete methods. Removing
them now to avoid further confusion.
Explicit --enable-mlite didn't enable mlite (enable_mlite not set).
The default is now to not enable it in any case (even if available),
so traditional users won't have to add --disable-mlite to suppress
the error about it being not installed.
The code which depends on mlite must not be compiled if the feature
is disabled, because if mlite is not installed, it would cause compile
errors.
The notifications system has been made template based. There is
a Factory object that creates NotificationManagers with the correct
template (mlite, libnotify, or a dummy no-op) according to the
platform.
If (for whatever reason) the patch file is empty, we
shouldn't invoke the "patch" command. Some implementations
of it then complain instead of copying the input file.
This commit adds a copying of the original test case file
if the corresponding patch file exists and is empty.
This reverts commit c1aaf7128e.
This patch doesn't work because the composed patch for valid
patches is no longer piped into the "patch" command due to
the additional test command in the middle. Will commit a different
solution.
Ignore "peerType = WebDAV" configurations as well as the
configurations with syncURL starting with "local://@" temporarily
(before we actually support WebDAV in the UI).
This patch adds .xml configs which replace the google-contacts from
the patched Buteo plugins. The glue code configures it as the normal
"google" config.
This patch teaches the command line how to infer the right template
for a source-config@<something> config creation:
syncevolution --configure source-config@google-calendar
For this particular example, "google-calendar" must match the "Google
Calendar" fingerprint in the template. Spaces, hyphen and underscores
are now all considered equal in TemplateConfig::fingerprintMatch().
Also added CmdlineTest::testWebDAV unit test. The test can only run if
WebDAV support is enabled, because otherwise the
backend=CardDAV/CalDAV would be rejected.
This patch introduces the new "peerType" property which marks
templates and configs as something that can be used for the
'source-config@<target>' configs necessary for local sync.
Only "WebDAV" is used. If peerType is not set, the template or config
is traditional SyncML.
This patch also adds two templates, one for Google Calendar and one
for Yahoo CardDAV and CalDAV. Because Yahoo CardDAV is unreliable,
it is not enabled.
The code for builtin templates had side effects, like always adding
all four standard sources to a template, even if the template itself
didn't have all of them defined. It also hid the problem that listing
templates didn't work for templates on disk.
Another benefit is that template files can be packaged separately. By
choosing the packages which are to be installed, a distributor of
SyncEvolution (like MeeGo) can choose which services to offer by
default.
Therefore this patch removes the "builtin templates" feature, which
was only useful in unusual use cases anyway (for example, single-binary
distribution).
Because there are no more default values for source properties, all
templates must specify the "backend" explicitly. syncevo-phone-config
was adapted accordingly, and also updated to use the current names of
the properties in the process.
As part of moving the templates into separate files, some of them
were cleaned up:
- Mobical: now points to Everdroid, its new name
- Google, Ovi: SSL verification is always enabled in the templates;
the workaround for old libsoup should no longer be
necessary for most users
- Google: renamed to "Google_Contacts", with "Google" as alias,
because there will be two Google templates soon
- Scheduleworld: use "server no longer in operation" instead of
an invalid URL
The finger print match had a special case for "default". The exact
intention of that is unknown. Perhaps it was meant to give that
template a boost when it wouldn't match the string that is getting
searched for at all.
But it had the effect that an exact match when searching for the
"default" template was not found and thus that template couldn't be
used in the command line after moving it from builtin to external.
Removed the complete check.
In TemplateConfig::fingerprintMatch the peerIsClient property only
matters when the mode is either "match only clients" or "match only
servers".
This patch checks for this first before reading the template.
Minor optimization, not performance critical.
The check with "starts with ." probably was meant to filter out hidden
files. Because it was applied to the full path, including the
directory names, it didn't have that effect. Instead it skipped all
entries of the template dir started with a dot, as in "./templates".
That was used in one of the Cmdline unit tests and only worked while
none of the templates there were needed for the test. It started to
fail after removing the builtin templates.
Better check error message before return code. That way
a non-zero error is visible in the output instead of
only the failed run check.
Calling doit() would be shorter, but hide the actual location
of the failure.
Moved removal and creation of the test directory (= "CmdlineTest")
into the test setup method. That way all tests are guaranteed to
start in a clean state, without having to duplicate that all over
the place.
Motivated by the observation that at least one test didn't have the
necessary cleanup, which caused a failure when creating more
templates.
If compilation and testing runs outside of the original source,
the src/testcases must be made available to "client-test". This
used to be done with a full copy, but that means that changes
made later are not reflected in later tests. Better use a symlink.
Also remember to remove it as part of "make clean".
Same bug as in command line (previous commit). Scanning
for on-disk templates only considered Bluetooth devices.
For server templates, only the ones built into SyncEvolution
were returned.
For some reason the 'valgrind' target attempted to run valgrind on a
./test executable. Perhaps it's a leftover from the past, as it should
be ./client-test.
Nightly compilation was failing because empty template patch files
would result in an error like the following:
patch: **** Only garbage was found in the patch input.
Making sure that the patch file is not empty before trying to apply
the patch fixes the problem.
"syncevolution --templates ?" only showed builtin templates
because the scanning for template on disk was called without
any match definition => empty results, only builtin templates
used as fallback.
After moving the main initialization from open() into
beginSync()/contactServer() two other functions failed because they
assumed that open() had done the initialization:
- WebDAVSource::isEmpty()
- CalDAVSource::backupData()
These functions are (correctly) called before beginSync() because they
need to work before resp. without a sync session.
Fixed this by tracking whether the server has been contacted and
calling contactServer() multiple times.
