maryjane
/
syncevolution
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
syncevolution/NEWS

9191 lines
403 KiB
Plaintext
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

SyncEvolution 1.5.3 -> 2.0.0, 21.03.2021
========================================
This release modernizes the code base and removes usage of out-dated
libraries and APIs. All Python scripts require Python 3. The major
version bump reflects that this release is not just a minor bug fix.
However, no new features were added.
Binaries on syncevolution.org get built for distros >= Ubuntu Bionic
and Debian Buster. Testing is now based on Docker containers instead
of a custom schroot solution, so adding testing against other distros
will be easier in the future. Compilation on Fedora Rawhide was
already added.
Some features are no longer built and thus untested:
- ActiveSync
- KDE
The code is still there, but needs help from interested developers to
ensure that it really works. It may get removed in a future version if
there is no interest.
SyncEvolution 1.5.2 -> 1.5.3, 03.01.2018
========================================
Maintenance release. syncevolution.org binaries are now getting
compiled for distros >= Ubuntu Xenial 16.04 LTS. Usage of deprecated
libraries (GNOME keyring) and APIs (SoupAsyncSession) was
replaced. libical v3 is supported.
The code now compiles more cleanly with recent compilers and depends
on C++11 support.
Details:
* EDS: more generic open retry handling
Recent EDS started to exhibit race conditions when opening a database (for
example, https://bugzilla.gnome.org/show_bug.cgi?id=791306). Opening was
already tried again for a certain known error in some old EDS version. Now it
is tried five times with a delay of one second for all errors.
* SoupTransportAgent: require libsoup 2.42, no deprecated methods
This allows us to get rid of deprecated function calls. We no longer
need to set a default proxy either, the newer libsoup does that itself
by default.
* C++: replace auto_ptr with unique_ptr, require C++11
auto_ptr has been deprecated for a while now. unique_ptr can
be taken for granted now, so use that instead.
* testing: work around Google CalDAV RECURRENCE-ID
Stand-alone events with RECURRENCE-ID get mangled by the server:
it converts the RECURRENCE-ID time to UTC. Reported in:
https://stackoverflow.com/questions/47811670/detached-recurrence-without-parent-event
* GNOME: replace gnome-keyring with libsecret (FDO #104219)
The GNOME keyring library has been obsoleted for a long time now,
long enough that the replacement libsecret is available on all
supported distros. Therefore we can switch unconditionally.
* libical: support libical v3 (FDO #104220)
libical v3 removes some deprecated functions (like icaltime_from_timet)
and removes the "is_utc" member from icaltimetype. The replacement
code works with old and new libical and thus needs no ifdefs.
Original author: Milan Crha
* syncevolution.org: fixed packaging (FDO #98014, FDO #100549)
The activesyncd package missing dependencies on libgnome-keyring0 and
libglib2.0-bin and therefore failed to work when installed on a minimal
system without those.
* various build and test fixes/workarounds
SyncEvolution 1.5.1 -> 1.5.2, 08.11.2016
========================================
Maintenance release. syncevolution.org binaries are now getting
compiled for distros >= Ubuntu Trusty 14.04 LTS, which allowed
removing several hacks that were needed when building binaries that
also had to run on older distros. Compilation from source for old
distros should still work as before, but is not getting tested
anymore.
Compile problems with recent libraries (libical v2) and tools (GCC v6)
were resolved. Syncing via Bluetooth with certain phones now should
work reliably in incremental mode.
New backends for the Trinity Desktop Environment (TDE) were added to
the source code.
Details:
* ObexTransportAgent.cpp: properly shut down connection (FDO #91485)
Apparently there's a race condition in the OBEX transport that causes the
connection to phones via Bluetooth to be shut down prematurely. Some phones
react by doing a slow sync instead of an incremental sync the next time.
* support non-readable parent directories (FDO #91000)
The previous mkdir_p() walked down top to bottom and checked each path
entry as it went along. That approach failed unnecessarily when some
existing parent directory could not be read (non-readable /home, for
example).
* avoid using dbus-launch (Debian #836399)
dbus-launch is considered deprecated because of the X11 dependency.
See https://lists.debian.org/debian-devel/2016/08/msg00554.html "Mass bug
filing: use and misuse of dbus-launch (dbus-x11)"
The dbus-session.sh script still needs to start the D-Bus daemon when used in the nightly
testing, so the code now does it by invoking the dbus-daemon directly.
syncevo-http-server still has some usage of dbus-launch left, but that's
strictly for systems which don't have the more modern D-Bus.
* syncevo-dbus-server integrates better with systemd (FDO #92164)
A .service file allows the D-Bus daemon to start the service via systemd,
thus ensuring that the process environment is correct. Patch from Simon
McVittie. Auto-starting as part of the desktop login uses D-Bus activation
if the "dbus-send" tool is installed.
* syncevolution.org: compile on Ubuntu Trusty, libical v1/v2 compatibility
syncevolution.org binaries are now getting compiled on Ubuntu Trusty and thus
no longer support distros with older EDS. The code should still compile
against older EDS (for example, for Maemo), but that is not getting tested
anymore.
This allows removing the dynamic linker hacks related to older libraries,
which was only used in those binaries. Instead, backends using libical or EDS
get compiled on Ubuntu Trusty and then the soname of those libs get patched to
make the backend module usable in combination with a different set of
libs. That patching is part of a script maintained in the syncevolution.org
build infrastructure.
This approach was already used before to generate different EDS backends
for EDS versions with the newer EClient API, because that turned out to be
easier than the dynamic loading approach. It works because none of the methods
used by SyncEvolution changed their ABI, only some other parts of the
libraries did. Should there ever be a situation again that cannot be handled
like this, then backends also get compiled on different distros than
Ubuntu Trusty (for example, the ActiveSync backend for Debian Stretch
is built on Debian Stretch).
libical still requires one special hack: system time zone loading in
libical v1 (and only in that version, v2 has builtin support again) must
be overridden such that time zones are generated with rules instead
of transitions because that is more compatible with the peers that
SyncEvolution exchanges data with.
That hack now relies on overriding the two relevant functions inside the main
binaries (has to be there, otherwise libical still ends up calling its own
internal implementation). The overriding code is in
libsyncevo-icaltz-util.so.0 and depends on libical.so.1. If
libsyncevo-icaltz-util.so.0 can be loaded, the wrappers in the main binary use
it, otherwise they fall through to the code from the current libical.so, which
then should be libical.so.2 or more recent.
This hack is active by default when libical v1 is detected during configuration.
* optionally show debug output in --version output
SYNCEVOLUTION_DEBUG=1 syncevolution --daemon=no --version now
dumps also the debug information gathered by the binary
compatibility code. It was only available in sync logs before.
* various build fixes for libical v2, GCC v6/C++14
SyncEvolution 1.5 -> 1.5.1, 05.06.2015
======================================
Maintenance release. Binaries now also get compiled for Debian 8.0
"Jessie".
Details:
* avoid time zone issue with Funambol server
The Funambol iCalendar 2.0 parser fails to handle time zones
with quotation marks around the TZID value, which is something
that SyncEvolution started to add in 1.4.99.3. While it is valid
to quote like that, it is not necessary, so avoid quoting in
this case to restore interoperability.
* syncevo-http-server: stop using deprecated twisted.web.error (FDO #90419)
This has become a real problem for example on Fedora 22 where the
old name is no longer available.
* syncevo-http-server: use TLS instead of SSLv3
This fixes a potential security risk and connection problems with clients
that don't support SSLv3 anymore.
* syncing: avoid segfault for invalid text inside items (FDO #90118)
As reported by Canonical, syncing fails if data items contain
text which is not correct UTF-8 in one of the fields that
SyncEvolution logs in the command line output (like SUMMARY of
a calendar event).
That is because the byte string coming from the item is passed
unchecked to the D-Bus implementation for transmission via D-Bus. But
D-Bus strings must be correct UTF-8, so depending on the D-Bus library
in use, one gets a segfault (GIO D-Bus, due to an unchecked NULL
pointer access) or an "out of memory" error (libdbus, which checks for
NULL).
SyncEvolution now replaces invalid bytes with a question mark in its
output while preserving the rest of the text.
* file backend: log item manipulation
Extracting a meaningful description of each item from the Synthesis
engine when updating and adding items is easy to do for items of
certain known types (contacts and calendar items).
* command line: preserve log prefix of target side of local sync
In some cases, the prefix which was supposed to be embedded
in the log messages from the target side of a local sync got
lost on the way to the command line tool.
Primarily this affected the added/updated/deleted messages, as in:
[INFO remote@client] @client/addressbook: started
[INFO remote@client] updating "Joan Doe"
[INFO remote@client] @client/addressbook: received 1/1
* compile fix: use ${PKG_CONFIG} instead of pkg-config.
This fixes the build on Exherbo that only has prefixed versions of
pkg-config.
* WebDAV: handle 403 during Google OAuth authentication
When sending an access token with insufficient scope (for example,
because the Ubuntu Online Accounts service definition was incomplete,
as documented in FDO #86824), Google responds with a 403 "service
denied" error.
This is now dealt with by retrying, just as for a transient 401 error.
* CalDAV: more efficient "is empty" check (FDO #86335)
Since 1.4.99.4, syncing WebDAV collections always checks first
whether there are items in the collections. This was partly done for
slow sync prevention (which is not necessary for empty collections),
partly for the "is the datastore usable" check.
However, this did not take into account that for CalDAV collections,
the entire content gets downloaded for this check. That is because
filtering by item type (VEVENT vs. VJOURNAL) is not implemented
correctly by all servers. So now all CalDAV syncs, whether incremental
or slow, always transfered all items, which is not the
intention (incremental syncs should be fast and efficient).
This release adds a more efficient isEmpty() check: for simple CardDAV
collections, only luid and etag get transferred, as in
listAllItems(). This is the behavior from 1.5.
For CalDAV, a report with a filter for the content type is used and
the transfer gets aborted after the first item, without actually
double-checking the content of the item. This is different from
listAllItems(), which really transfers the content. This extra content
check would only be needed for some old servers (Radical 0.7) and is
not essential, because reporting "not empty" even when empty is safe.
* WebDAV: send Basic Auth via http in some cases (FDO #57248)
It turned out that finding databases on an Apple Calendar server accessed via
http depends on sending Basic Auth even when the server does not ask for it:
without authentication, there is no information about the current principal,
which is necessary for finding the user's databases.
To make this work again, sending the authentication header is now forced for
plain http if (and only if) the request which should have returned the
principal URL fails to include it. This implies sending the same request
twice, but as this scenario should be rare in practise (was only done for
testing), this is acceptable.
* Ubuntu Online Accounts: support plain text credentials
The backend for UOA was rewritten by Alberto Mardegan and now also
can use plain username/password credentials stored in UOA.
* various compiler error and warning fixes
SyncEvolution 1.4.1 -> 1.5, 31.10.2014
======================================
Based on community feedback and discussions, the terminology used in
SyncEvolution for configuration, local sync and database access was
revised. Some usability issues with setting up access to databases
were addressed.
Interoperability with WebDAV servers and in particular Google Contacts
was enhanced considerably. Access to iCloud contacts was reported as
working when using username=foobar@icloud.com and password, but is not
formally tested. Syncing with iCloud calendars ran into a server
limitation (reported as 17001498 "CalDAV REPORT drops calendar data")
and needs further work (FDO #72133).
Contact data gets converted to and from the format typically used by
CardDAV servers, so now anniversary, spouse, manager, assistant and
instant message information are exchanged properly. Custom labels get
stored in EDS as extensions and no longer get lost when updating some
other aspects of a contact. However, Evolution does not show custom
labels and removes them when editing a property which has a custom
label (BGO #730636).
Scanning for CardDAV/CalDAV resources was enhanced. It now finds
additional calendars with Google CalDAV. For Google, the obsolete
SyncML config template was removed and CalDAV/CardDAV were merged into
a single "Google" template.
Using Google Calendar/Contacts with OAuth2 authentication on a
headless server becomes a bit easier: it is possible to set up access
on one system with a GUI using either gSSO or GNOME Online Accounts,
then take the OAuth2 refresh token and use it in SyncEvolution on a
different system. See
http://cgit.freedesktop.org/SyncEvolution/syncevolution/tree/src/backends/oauth2/README
The PIM Manager API also supports Google Contact syncing. Some
problems with suspending a PBAP sync were fixed. Suspend/abort can
be tested with the sync.py example.
Performance is better for local syncs and PBAP caching. The most
common case, a two-way sync with no changes on either side, no longer
rewrites any meta data files. CPU consumption during local sync was
reduced to one third by exchanging messages via shared memory instead
of internal D-Bus. Redundant vCard decode/encode on the sending side
of PBAP and too agressive flushing of meta data during a normal sync
were removed.
The EDS memo backend is able to switch between syncing in plain
text and iCalendar 2.0 VJOURNAL automatically.
Details:
* source -> datastore rename, improved terminology
The word "source" implies reading, while in fact access is read/write.
"datastore" avoids that misconception. Writing it in one word emphasizes
that it is single entity.
While renaming, also remove references to explicit --*-property
parameters. The only necessary use today is "--sync-property ?"
and "--datastore-property ?".
--datastore-property was used instead of the short --store-property
because "store" might be mistaken for the verb. It doesn't matter
that it is longer because it doesn't get typed often.
--source-property must remain valid for backward compatility.
As many user-visible instances of "source" as possible got replaced in
text strings by the newer term "datastore". Debug messages were left
unchanged unless some regex happened to match it.
The source code will continue to use the old variable and class names
based on "source".
Various documentation enhancements:
Better explain what local sync is and how it involves two sync
configs. "originating config" gets introduces instead of just
"sync config".
Better explain the relationship between contexts, sync configs,
and source configs ("a sync config can use the datastore configs in
the same context").
An entire section on config properties in the terminology
section. "item" added (Todd Wilson correctly pointed out that it was
missing).
Less focus on conflict resolution, as suggested by Graham Cobb.
Fix examples that became invalid when fixing the password
storage/lookup mechanism for GNOME keyring in 1.4.
The "command line conventions", "Synchronization beyond SyncML" and
"CalDAV and CardDAV" sections were updated. It's possible that the
other sections also contain slightly incorrect usage of the
terminology or are simply out-dated.
* local sync: allow config name in syncURL=local://
Previously, only syncURL=local://@<context name> was allowed and used
the "target-config@context name" config as target side in the local
sync.
Now "local://config-name@context-name" or simply "local://config-name"
are also allowed. "target-config" is still the fallback if only a
context is give.
It also has one more special meaning: "--configure
target-config@google" will pick the "Google" template automatically
because it knows that the intention is to configure the target side
of a local sync. It does not know that when using some other name
for the config, in which case the template (if needed) must be
specified explicitly.
The process name in output from the target side now also includes the
configuration name if it is not the default "target-config".
* command line: revise usability checking of datastores
When configuring a new sync config, the command line checks whether a
datastore is usable before enabling it. If no datastores were listed
explicitly, only the usable ones get enabled. If unusable datastores
were explicitly listed, the entire configure operation fails.
This check was based on listing databases, which turned out to be too
unspecific for the WebDAV backend: when "database" was set to some URL
which is good enough to list databases, but not a database URL itself,
the sources where configured with that bad URL.
Now a new SyncSource::isUsable() operation is used, which by default
just falls back to calling the existing Operations::m_isEmpty. In
practice, all sources either check their config in open() or the
m_isEmpty operation, so the source is usable if no error is
enountered.
For WebDAV, the usability check is skipped because it would require
contacting a remote server, which is both confusing (why does a local
configure operation need the server?) and could fail even for valid
configs (server temporarily down). The check was incomplete anyway
because listing databases gave a fixed help text response when no
credentials were given. For usability checking that should have
resulted in "not usable" and didn't.
The output during the check was confusing: it always said "listing
databases" without giving a reason why that was done. The intention
was to give some feedback while a potentially expensive operation
ran. Now the isUsable() method itself prints "checking usability" if
(and only if!) such a check is really done.
Sometimes datastores were checked even when they were about to be
configure as "disabled" already. Now checking such datastores is
skipped.
* command line: fix --update from directory
The "--update <dir name>" operation was supposed to take the
item luids from the file names inside the directory. That part
had not been implemented, turning the operation accidentally
into an "--import".
Also missing was the escaping/unescaping of luids. Now the
same escaping is done as in command line output and command
line parsing to make the luids safe for use as file name.
* sync output: hide "<source>: started" INFO messages
These messages get printed at the start of processing each
SyncML message. This is not particularly useful and just
adds noise to the output.
* config: allow storing credentials for email address
When configuring a WebDAV server with username = email address and no
URL (which is possible if the server supports service discovery via
the domain in the email address), then storing the credentials in the
GNOME keyring used to fail with "cannot store password in GNOME
keyring, not enough attributes".
That is because GNOME keyring seemed to get confused when a network
login has no server name and some extra safeguards were added to
SyncEvolution to avoid this.
To store the credentials in the case above, the email address now gets
split into user and domain part and together get used to look up the
password.
* config: ignore unnecessary username property
A local sync or Bluetooth sync do not need the 'username' property.
When it is set despite that, issue a warning.
Previously, the value was checked even when not needed, which
caused such syncs to fail when set to something other than a plain
username.
* config templates: Funambol URLs
Funambol turned of the URL redirect from my.funambol.com to
onemedia.com. The Funambol template now uses the current URL. Users
with existing Funambol configs must updated the syncURL property
manually to https://onemediahub.com/sync
Kudos to Daniel Clement for reporting the change.
* EDS: memo syncing as iCalendar 2.0 (FDO #52714)
When syncing memos with a peer which also supports iCalendar 2.0 as
data format, the engine will now pick iCalendar 2.0 instead of
converting to/from plain text. The advantage is that some additional
properties like start date and categories can also be synchronized.
The code is a lot simpler, too, because the EDS specific iCalendar 2.0
<-> text conversion code can be removed.
* SoupTransport: drop CA file check
It used to be necessary to specify a CA file for libsoup to enable SSL
certificate checking. Nowadays libsoup uses the default CA store
unless told otherwise, so the check in SyncEvolution became
obsolete. However, now there is a certain risk that no SSL checking is
done although the user asked for it (when libsoup is not recent enough
or compiled correctly).
* CardDAV: use Apple/Google/CardDAV vCard flavor
In principle, CardDAV servers support arbitrary vCard 3.0
data. Extensions can be different and need to be preserved. However,
when multiple different clients or the server's Web UI interpret the
vCards, they need to agree on the semantic of these vCard extensions.
In practice, CardDAV was pushed by Apple and Apple clients are
probably the most common clients of CardDAV services. When the Google
Contacts Web UI creates or edits a contact, Google CardDAV will
send that data using the vCard flavor used by Apple.
Therefore it makes sense to exchange contacts with *all* CardDAV
servers using that format. This format could be made configurable in
SyncEvolution on a case-by-case basis; at the moment, it is
hard-coded.
During syncing, SyncEvolution takes care to translate between the
vCard flavor used internally (based on Evolution) and the CardDAV
vCard flavor. This mapping includes:
X-AIM/JABBER/... <-> IMPP + X-SERVICE-TYPE
Any IMPP property declared as X-SERVICE-TYPE=AIM will get
mapped to X-AIM. Same for others. Some IMPP service types
have no known X- property extension; they are stored in
EDS as IMPP. X- property extensions without a known X-SERVICE-TYPE
(for example, GaduGadu and Groupwise) are stored with
X-SERVICE-TYPE values chosen by SyncEvolution so that
Google CardDAV preserves them (GroupWise with mixed case
got translated by Google into Groupwise, so the latter is used).
Google always sends an X-ABLabel:Other for IMPP. This is ignored
because the service type overrides it.
The value itself also gets transformed during the mapping. IMPP uses
an URI as value, with a chat protocol (like "aim" or "xmpp") and
some protocol specific identifier. For each X- extension the
protocol is determined by the property name and the value is the
protocol specific identifier without URL encoding.
X-SPOUSE/MANAGER/ASSISTANT <-> X-ABRELATEDNAMES + X-ABLabel
The mapping is based on the X-ABLabel property attached to
the X-ABRELATEDNAMES property. This depends on the English
words "Spouse", "Manager", "Assistant" that Google CardDAV
and Apple devices seem to use regardless of the configured
language.
As with IMPP, only the subset of related names which have
a corresponding X- property extension get mapped. The rest
is stored in EDS using the X-ABRELATEDNAMES property.
X-ANNIVERSARY <-> X-ABDATE
Same here, with X-ABLabel:Anniversary as the special case
which gets mapped.
X-ABLabel parameter <-> property
CardDAV vCards have labels attached to arbitrary other properties
(TEL, ADR, X-ABDATE, X-ABRELATEDNAMES, ...) via vCard group tags:
item1.X-ABDATE:2010-01-01
item1.X-ABLabel:Anniversary
The advantage is that property values can contain arbitrary
characters, including line breaks and double quotation marks,
which is not possible in property parameters.
Neither EDS nor KDE (judging from the lack of responses on the
KDE-PIM mailing list) support custom labels. SyncEvolution could
have used grouping as it is done in CardDAV, but grouping is not
used much (not at all?) by the UIs working with the vCards in EDS
and KDE. It seemed easier to use a new X-ABLabel parameter.
Characters which cannot be stored in a parameter get converted
(double space to single space, line break to space, etc.) during
syncing. In practice, these characters don't appear in X-ABLabel
properties anyway because neither Apple nor Google UIs allow entering
them for custom labels.
The "Other" label is used by Google even in case where it adds no
information. For example, all XMPP properties have an associated
X-ABLabel=Other although the Web UI does not provide a means to edit
or show such a label. Editing the text before the value in the UI
changes the X-SERVICE-TYPE parameter value, not the X-ABLabel as for
other fields.
Therefore the "Other" label is ignored by removing it during syncing.
X-EVOLUTION-UI-SLOT (the parameter used in Evolution to determine the
order of properties in the UI) gets stored in CardDAV. The only exception
is Google CardDAV which got confused when an IMPP property had both
X-SERVICE-TYPE and X-EVOLUTION-UI-SLOT parameters set. For Google,
X-EVOLUTION-UI-SLOT is only sent on other properties and thus ordering
of chat information can get lost when syncing with Google.
* synccompare: support grouping and quoted parameter strings
Grouped properties are sorted first according to the actual property
name, then related properties are moved to the place where their group
tag appears first. The first grouped property gets a "- " prefix, all
following ones are just indended with " ". The actual group tag is not
part of the normalized output, because its value is irrelevant:
BDAY:19701230
- EMAIL:john@custom.com
X-ABLabel:custom-label2
...
FN:Mr. John 1 Doe Sr.
- IMPP;X-SERVICE-TYPE=AIM:aim:aim
X-ABLabel:Other
...
- X-ABDATE:19710101
X-ABLabel:Anniversary
Redundant tags (those set for only a single property, X-ABLabel:Other)
get removed as part of normalizing an item.
* WebDAV: use server's order when listing collections
When doing a recursive scan of the home set, preserve the order of
entries as reported by the server and check the first one first. The
server knows better which entries are more relevant for the user (and
thus should be the default) or may have some other relevant
order. Previously, SyncEvolution replaced that order with sorting by
URL, which led to a predictable, but rather meaningless order.
For example, Google lists the users own calendar first, followed by
the shared calendars sorted alphabetical by their name. Now
SyncEvolution picks the main calendar as default correctly when
scanning from https://www.google.com/calendar/dav/.
* WebDAV: improved database search (Google, Zimbra)
Zimbra has a principal URL that also serves as home set. When using it
as start URL, SyncEvolution only looked the URL once, without listing
its content, and thus did not find the databases.
When following the Zimbra principal URL indirectly, SyncEvolution did
check all of the collections there recursively. Unfortunately that
also includes many mail folders, causing the scan to abort after
checking 1000 collections (an internal safe guard).
The solution for both includes tracking what to do with a URL. For the
initial URL, only meta data about the URL itself gets
checked. Recursive scanning is only done for the home set. If that
home set contains many collections, scanning is still slow and may run
into the internal safe guard limit. This cannot be avoided because the
CalDAV spec explicitly states that the home set may contain normal
collections which contain other collections, so a client has to do the
recursive scan.
When looking at a specific calendar, Google CalDAV does not report
what the current principal or the home set is and therefore
SyncEvolution stopped after finding just the initial calendar. Now it
detects the lack of meta information and adds all parents also as
candidates that need to be looked at. The downside of this is that it
doesn't know anything about which parents are relevant, so it ends up
checking https://www.google.com/calendar/ and
https://www.google.com/.
In both cases Basic Auth gets rejected with a temporary redirect to
the Google login page, which is something that SyncEvolution must
ignore immediately during scanning without applying the resend
workaround for "temporary rejection of valid credentials" that can
happen for valid Google CalDAV URLs.
* WebDAV: enhanced database search (Google Calendar)
Additional databases where not found for several
reasons. SyncEvolution ignored all shared calendars
(http://calendarserver.org/ns/shared) and Google marks the additional
calendars that way. The other problem was that the check for leaf
collections (= collections which cannot contain other desired
collections) incorrectly excluded those collections instead of only
preventing listing of their content.
With this change,
https://www.google.com/calendar/dav/?SyncEvolution=Google can be used
as starting point for Google Calendar.
* WebDAV: fix database scan on iCloud
The calendar home set URL on iCloud (the one ending in /calendars/) is
declared as containing calendar data. That was enough for
SyncEvolution to accept it incorrectly as calendar. However, the home
set only contains calendar data indirectly.
* WebDAV: support redirects between hosts and DNS SRV lookup based on URL
When finding a new URL, we must be prepared to reinitialize the Neon
session with the new host settings.
iCloud does not have .well-known support on its www.icloud.com
server. To support lookup with a non-icloudd.com email address, we
must do DNS SRV lookup when access to .well-known URLs fails. We do
this without a www prefix on the host first, because that is what happens
to work for icloud.com.
With these changes it becomes possible to do database scans on Apple
iCloud, using syncURL=https://www.icloud.com or
syncURL=https://icloud.com. Giving the syncURL like this is only
necessary for a username that does not end in @icloud.com. When
the syncURL is not set, the domain for DNS SRV lookup is taken
from the username.
* WebDAV: more efficient item creation
PUT has the disadvantage that a client needs to choose a name and then
figure out what the real name on the server is. With Google CardDAV that
requires sending another request and only works because the server happens
to remember the original name (which is not guaranteed!).
POST works for new items without a name and happens to be implemented
by Google such that the response already includes all required
information (new name and revision string).
POST is checked for as described in RFC 5995 once before creating a new
item. Servers which don't support it continue to get a PUT.
* WebDAV: send "User-Agent: SyncEvolution"
Apple iCloud servers reject requests unless they contain a User-Agent
header. The exact value doesn't seem to matter. Making the string
configurable might be better, but can still be done later when it
is more certain whether and for what it is needed.
* WebDAV: refactor and fix DNS SRV lookup
The syncevo-webdav-lookup script was not packaged. It did not report
"not found" DNS results correctly and the caller did not check for
this either, so when looking up the information for a domain which
does not have DNS SRV entries, SyncEvolution ended up retrying for
while as if there had been a temporary lookup problem.
* Google: remove SyncML template, combine CalDAV/CardDAV
Google has turned off their SyncML server, so the corresponding
"Google Contacts" template became useless and needs to be removed. It
gets replaced by a "Google" template which combines the three
different URLs currently used by Google for CalDAV/CardDAV.
This new template can be used to configure a "target-config@google"
with default calendar and address book database already enabled. The
actual URL of these databases will be determined during the first
sync using them.
The template relies on the WebDAV backend's new capability to search
multiple different entries in the syncURL property for databases. To
avoid listing each calendar twice (once for the legacy URL, once with
the new one) when using basic username/password authentication, the
backend needs a special case for Google and detect that the legacy URL
does not need to be checked.
* Google Calendar: remove child hack, improve alarm hack (FDO #63881)
Google recently enhanced support for RECURRENCE-ID, so SyncEvolution
no longer needs to replace the property when uploading a single
detached event with RECURRENCE-ID. However, several things are still
broken in the server, with no workaround in SyncEvolution:
- Removing individual events gets ignored by the server;
a full "wipe out server data" might work (untested).
- When updating the parent event, all child events also get
updated even though they were included unchanged in the data
sent by SyncEvolution.
- The RECURRENCE-ID of a child event of an all-day recurring event
does not get stored properly.
- The update hack seems to fail for complex meetings: uploading them
once and then deleting them seems to make uploading them again
impossible.
All of these issues were reported to Google and are worked on there,
so perhaps the situation will improve. In the meantime, syncing with
Google CalDAV should better be limited to:
- Downloading a Google calendar in one-way mode.
- Two-way syncing of simple calendars without complex meeting
serieses.
While updating the Google workarounds, the alarm hack (sending a
new event without alarms twice to avoid the automatic server side
alarm) was simplified. Now the new event gets sent only once with a
pseudo-alarm.
* CardDAV: implement read-ahead
Instead of downloading contacts one-by-one with GET, SyncEvolution now
looks at contacts that are most likely going to be needed soon and
gets all of them at once with addressbook-multiget REPORT.
The number of contacts per REPORT is 50 by default, configurable by
setting the SYNCEVOLUTION_CARDDAV_BATCH_SIZE env variable.
This has two advantages:
- It avoids round-trips to the server and thus speeds up a large
download (100 small contacts with individual GETs took 28s on
a fast connection, 3s with two REPORTs).
- It reduces the overall number of requests. Google CardDAV is known
to start issuing intermittent 401 authentication errors when the
number of contacts retrieved via GET is too large. Perhaps this
can be avoided with addressbook-multiget.
* oauth2: new backend using libsoup/libcurl
New backend implements identity provider for obtaining OAuth2 access
token for systems without HMI support.
Access token is obtained by making direct HTTP request to OAuth2 server
and using refresh token obtained by user in some other way.
New provider automatically updates stored refresh token when OAuth2
server is issuing new one.
* signon: make Accounts optional
The new "signon" provider only depends on lib[g]signon-glib. It uses
gSSO if found, else UOA. Instead of pulling parameters and the
identity via libaccounts-glib, the user of SyncEvolution now has to
ensure that the identity exists and pass all relevant parameters
in the "signon:" username.
* gSSO: adapt to gSSO >= 2.0
* signon: fix build
Static build was broken for gSSO and UOA (wrong path name to .la file)
and gSSO was not enabled properly (wrong condition check).
* datatypes: raw text items with minimal conversion (FDO #52791)
When using "raw/text/calendar" or "raw/text/vcard" as SyncEvolution
"databaseFormat", all parsing and conversion is skipped. The backend's
data is identical to the item data in the engine. Finding duplicates
in a slow sync is very limited when using these types because the entire
item data must match exactly.
This is useful for the file backend when the goal is to store an exact copy
of what a peer has or for limited, read-only backends (PBAP). The downside
of using the raw types is that the peer is not given accurate information
about which vCard or iCalendar properties are supported, which may cause
some peers to not send all data.
* datatypes: text/calendar+plain revised heuristic
When sending a VEVENT, DESCRIPTION was set to the SUMMARY if empty. This may
have been necessary for some peers, but for notes (= VJOURNAL) we don't know
that (hasn't been used in the past) and don't want to alter the item
unnecessarily, so skip that part and allow empty DESCRIPTION.
When receiving a plain text note, the "text/calendar+plain" type
used to store the first line as summary and the rest as description.
This may be correct in some cases and wrong in others.
The EDS backend implemented a different heuristic: there the first
line is copied into the summary and stays in the description. This
makes a bit more sense (the description alone is always enough to
understand the note). Therefore and to avoid behavioral changes
for EDS users when switching the EDS backend to use text/calendar+plain,
the engine now uses the same approach.
* datatypes: avoid PHOTO corruption during merge (FDO #77065)
When handling an update/update conflict (both sides of the sync have an
updated contact) and photo data was moved into a local file by EDS, the engine
merged the file path and the photo data together and thus corrupted the photo.
The engine does not know about the special role of the photo property.
This needs to be handled by the merge script, and that script did not
cover this particular situation. Now the loosing side is cleared,
causing the engine to then copy the winning side over into the loosing
one.
Found by Renato Filho/Canonical when testing SyncEvolution for Ubuntu 14.04.
* vcard profile: avoid data loss during merging
When resolving a merge conflict, repeating properties were taken
wholesale from the winning side (for example, all email addresses). If
a new email address had been added on the loosing side, it got lost.
Arguably it is better to preserve as much data as possible during a
conflict. SyncEvolution now does that in a merge script by checking
which properties in the loosing side do not exist in the winning side
and copying those entries.
Typically only the main value (email address, phone number) is checked
and not the additional meta data (like the type). Otherwise minor
differences (for example, both sides have same email address, but with
different types) would lead to duplicates.
Only addresses are treated differently: for them all attributes
(street, country, city, etc.) are compared, because there is no single
main value.
* engine: UID support in contact data
Before, the UID property in a vCard was ignored by the engine.
Backends were responsible for ensuring that the property is
set if required by the underlying storage. This turned out to be
handled incompletely in the WebDAV backend.
This change moves this into the engine:
- UID is now field. It does not get used for matching
because the engine cannot rely on it being stored
by both sides.
- It gets parsed if present, but only generated if
explicitly enabled (because that is the traditional
behavior).
- It is never shown in the DevInf's CtCap
because the Synthesis engine would always show it
regardless whether a rule enabled the property.
That's because rules normally only get triggered
after exchanging DevInf and thus DevInf has to
be rule-independent. We don't want it shown because
then merging the incoming item during a local sync
would use the incoming UID, even if it is empty.
- Before writing, ensure that UID is set.
When updating an existing item, the Synthesis engine reads
the existing item, preserves the existing UID unless the peer
claims to support UID, and then updates with the existing UID.
This works for local sync (where SyncEvolution never claims
to support UID when talking to the other side). It will break
with peers which have UID in their CtCap although they
rewrite the UID and backends whose underlying storage cannot
handle UID changes during an update (for example, CardDAV).
* engine: flush map items less frequently
The Synthesis API does not say explicitly, but in practice all map
items get updated in a tight loop. Rewriting the m_mappingNode (case
insensitive string comparisons) and serialization to disk
(std::ostrstream) consume a significant amount of CPU cycles and cause
extra disk writes that can be avoided by making some assumptions about
the sequence of API calls and flushing only once.
* local sync: exchange SyncML messages via shared memory
Encoding/decoding of the uint8_t array in D-Bus took a surprisingly
large amount of CPU cycles relative to the rest of the SyncML message
processing. Now the actual data resides in memory-mapped temporary
files and the D-Bus messages only contain offset and size inside these
files. Both sides use memory mapping to read and write directly.
For caching 1000 contacts with photos on a fast laptop, total sync
time roughly drops from 6s to 3s.
To eliminate memory copies, memory handling in libsynthesis or rather,
libsmltk is tweaked such that it allocates the buffer used for SyncML
message data in the shared memory buffer directly. This relies on
knowledge of libsmltk internals, but those shouldn't change and if they
do, SyncEvolution will notice ("unexpected send buffer").
* local sync: avoid updating meta data when nothing changed
The sync meta data (sync anchors, client change log) get updated after
a sync even if nothing changed and the existing meta data could be
used again. This can be skipped for local sync, because then
SyncEvolution can ensure that both sides skip updating the meta
data. With a remote SyncML server that is not possible and thus
SyncEvolution has to update its data.
It is based on the observation that when the server side calls
SaveAdminData, the client has sent its last message and the sync is
complete. At that point, SyncEvolution can check whether anything has
changed and if not, skip saving the server's admin data and stop the
sync without sending the real reply to the client.
Instead the client gets an empty message with "quitsync" as content
type. Then it takes shortcuts to close down without finalizing the
sync engine, because that would trigger writing of meta data
changes. The server continues its shutdown normally.
This optimization is limited to syncs with a single source, because
the assumption about when aborting is possible is harder to verify
when multiple sources are involved.
* PBAP: support SYNCEVOLUTION_PBAP_CHUNK_TRANSFER_TIME <= 0
When set to 0 or less, the chunk size is not getting adapted at all
while still using transfers in chunks.
* PBAP: use raw text items
This avoids the redundant parse/generate step on the sending
side of the PBAP sync.
* PBAP syncing: updated photo not always stored
Because photo data was treated like a C string, changes after any
embedded null byte were ignored during a comparison.
* ephemeral sync: don't write binfile client files (FDO #55921)
When doing PBAP caching, we don't want any meta data written because
the next sync would not use it anyway. With the latest libsynthesis
we can configure "/dev/null" as datadir for the client's binfiles and
libsynthesis will avoid writing them.
The PIM manager uses this for PBAP syncing automatically. For testing
it can be enabled by setting the SYNCEVOLUTION_EPHEMERAL env variable.
* PBAP: avoid empty field filter
Empty field filter is supposed to mean "return all supported
fields". This used to work and stopped working with Android phones
after an update to 4.3 (seen on Galaxy S3); now the phone only
returns the mandatory TEL, FN, N fields.
The workaround is to replace the empty filter list with the list of
known and supported properties. This means we only pull data we really
need, but it also means we won't get to see any additional properties
that the phone might support.
* PBAP: transfer in chunks (FDO #77272)
If enabled via env variables, PullAll transfers will be limited to
a certain numbers contacts at different offsets until all data got
pulled. See PBAP README for details.
When transfering in chunks, the enumeration of contacts for the engine
no longer matches the PBAP enumeration. Debug output uses "offset #x"
for PBAP and "ID y" for the engine.
* PBAP: remove transfer via pipe
Using a pipe was never fully supported by obexd (blocks
obexd). Transfering in suitably sized chunks (FDO #77272) will be a
more obexd friendly solution with a similar effect (not having to
buffer the entire address book in memory).
* PBAP: Suspend/ResumeSync() (FDO #72112)
By default, the new API freezes a sync by stopping to consume data on the
local side of the sync.
In addition, the information that the sync is freezing is now also handed
down to the transport and all sources. In the case of PBAP caching, the local
transport notifies the child where the PBAP source then uses Bluez
5.15 Transfer1.Suspend/Resume to freeze/thaw the actual OBEX transfer.
If that fails (for example, not implemented because Bluez is too old
or the transfer is still queueing), then the transfer gets cancelled
and the entire sync fails. This is desirable for PBAP caching and
Bluetooth because a failed sync can easily be recovered from (just
start it again) and the overall goal is to free up Bluetooth bandwidth
quickly.
* PBAP: transfer data via pipe (part of FDO #72112)
The main advantage is that processed data can be discarded
immediately. When using a plain file, the entire address book must be
stored in it.
The drawback is that obexd does not react well to a full pipe. It
simply gets stuck in a blocking write(); in other words, all obexd
operations get frozen and obexd stops responding on D-Bus.
* PIM: include CardDAV in CreatePeer()
This adds "protocol: CardDAV" as a valid value, with corresponding
changes to the interpretation of some existing properties and some new
ones. The API itself is not changed.
Suspending a CardDAV sync is possible. This freezes the internal
SyncML message exchange, so data exchange with the CardDAV server may
continue for a while after SuspendPeer().
Photo data is always downloaded immediately. The "pbap-sync" flag
in SyncPeerWithFlags() has no effect.
Syncing can be configured to be one-way (local side is read-only
cache) or two-way (local side is read/write). Meta data must be
written either way, to speed up caching or allow two-way syncing. The
most common case (no changes on either side) will have to be optimized
such that existing meta data is not touched and thus no disk writes
occur.
* PIM: handle SuspendPeer() before and after transfer (FDO #82863)
A SuspendPeer() only succeeded while the underlying Bluetooth transfer
was active. Outside of that, Bluez errors caused SyncEvolution to
attempt a cancelation of the transfer and stopped the sync.
When the transfer was still queueing, obexd returns
org.bluez.obex.Error.NotInProgress. This is difficult to handle for
SyncEvolution: it cannot prevent the transfer from starting and has to
let it become active before it can suspend the transfer. Canceling
would lead to difficult to handle error cases (like partially parsed
data) and therefore is not done.
The Bluez team was asked to implement suspending of queued transfers
(see "org.bluez.obex.Transfer1 Suspend/Resume in queued state" on
linux-bluetooth@vger.kernel.org), so this case might not happen
anymore with future Bluez.
When the transfer completes before obexd processes the Suspend(),
org.freedesktop.DBus.Error.UnknownObject gets returned by
obexd. SyncEvolution can ignore errors which occur after the active
transfer completed. In addition, it should prevent starting the next
one. This may be relevant for transfer in chunks, although the sync
engine will also stop asking for data and thus typically no new
transfer gets triggered anyway.
* PIM: add suspend/resume/abort to sync.py
CTRL-C while waiting for the end of a sync causes an interactive
prompt to appear where one can choose been suspend/resume/abort and
continuing to wait. CTRL-C again in the prompt aborts the script.
* PIM: enhanced progress notifications (FDO #72114)
This adds GetPeerStatus() and "progress" events. Progress is reported based
on the "item received" Synthesis event and the total item count. A modified
libsynthesis is needed where the SyncML binfile client on the target side of
the local sync actually sends the total item count (via NumberOfChanges).
This cannot be done yet right at the start of the sync, only the second SyncML
message will have it. That is acceptable, because completion is reached very
quickly anyway for syncs involving only one message.
At the moment, SyncContext::displaySourceProgress() holds back "item
received" events until a different event needs to be emitted. Progress
reporting might get more fine-grained when adding allowing held back
events to be emitted at a fixed rate, every 0.1s. This is not done yet
because it seems to work well enough already.
For testing and demonstration purposes, sync.py gets command line
arguments for setting progress frequency and showing progress either
via listening to signals or polling.
* PIM: add SyncPeerWithFlags() and 'pbap-sync' flag (FDO #70950)
The is new API and flag grant control over the PBAP sync mode.
* PIM: fix phone number normalization
The parsed number always has a country code, whereas SyncEvolution expected it
to be zero for strings without an explicit country code. This caused a caller
ID lookup of numbers like "089788899" in DE to find only telephone numbers in
the current default country, instead of being more permissive and also finding
"+189788899". The corresponding unit test was broken and checked for the wrong
result. Found while investigating an unrelated test failure when updating
libphonenumber.
* engine: enable batching by default (FDO #52669)
This reverts commit c435e937cd406e904c437eec51a32a6ec6163102.
Commit 7b636720a in libsynthesis fixes an unitialized memory read in
the asynchronous item update code path.
Testing confirms that we can now used batched writes reliably with EDS
(the only backend currently supporting asynchronous writes +
batching), so this change enables it again also for local and
SyncEvolution<->SyncEvolution sync (with asynchronous execution of
contact add/update overlapped with SyncML message exchanges) and other
SyncML syncs (with changes combined into batches and executed at the
end of each message).
* Various compiler problems and warnings fixed; compiles with
--with-warnings=fatal on current Debian Testing and Ubuntu Trusty
(FDO #79316).
* D-Bus server: fix unreliable shutdown handling
Occassionally, syncevo-dbus-server locked up after receiving
a CTRL-C. This primarily affected nightly testing, in particular (?)
on Ubuntu Lucid.
* D-Bus: use streams for direct IPC with GIO
When using GIO, it is possible to avoid the DBusServer listening on a
publicly accessible address. Connection setup becomes more reliable,
too, because the D-Bus server side can detect that a child died
because the connection will be closed.
When using libdbus, the traditional server/listen and client/connect
model is still used.
* LogRedirect: safeguard against memory corruption
When aborting, our AbortHandler gets called to close down logging.
This may involve memory allocation, which is unsafe. In FDO #76375, a
deadlock on a libc mutex was seen.
To ensure that the process shuts down anyway, install an alarm and give
the process five seconds to shut down before the SIGALRM signal will kill
it.
Upgrading from releases <= 1.3.99.4:
If the value of "username/databaseUser/proxyUser" contains a colon,
the "user:" prefix must be added to the value, to continue treating it
like a plain user name and not some reference to an unknown identity
provider (like "id:", "goa:", "signon:", etc.).
The lookup of passwords in GNOME Keyring was updated slightly in
1.3.99.5. It may be necessary to set passwords anew if the old one is
no longer found.
Upgrading from release 1.2.x:
The sync format of existing configurations for Mobical (aka Everdroid)
must be updated manually, because the server has encoding problems when
using vCard 3.0 (now the default for Evolution contacts):
syncevolution --configure \
syncFormat=text/x-vcard \
mobical addressbook
The Funambol template explicitly enables usage of the
"refresh-from-server" sync mode to avoid getting throttled with 417
'retry later' errors. The same must be added to existing configs
manually:
syncevolution --configure \
enableRefreshSync=TRUE \
funambol
Upgrading from releases before 1.2:
Old configurations can still be read. But writing, as it happens
during a sync, must migrate the configuration first. Releases >= 1.2
automatically migrates configurations. The old configurations
will still be available (see "syncevolution --print-configs") but must
be renamed manually to use them again under their original names with
older SyncEvolution releases.
SyncEvolution 1.4.99.4 -> 1.5, 31.10.2014
=========================================
Mostly a bug fix release.
Details:
* vcard: fix caching of PBAP contacts (FDO #84710)
After changing PBAP to send raw items, caching them led to unnecessary
disk writes and bogus "contacts changed" reports. That's because
the merge script relied on the exact order of properties, which was
only the same when doing the redundant decode/encode on the PBAP side.
Instead of reverting back to sending re-encoded items, better enhance
the contact merge script such that it detects contacts as unchanged
when just the order of entries in the property arrays is different.
This relies on an enhanced libsynthesis with the new RELAXEDCOMPARE()
and modified MERGEFIELDS().
* sync: ignore unnecessary username property
A local sync or Bluetooth sync do not need the 'username' property.
When it is set despite that, issue a warning.
Previously, the value was checked even when not needed, which
caused such syncs to fail when set to something other than a plain
username.
* D-Bus server: fix unreliable shutdown handling
Occassionally, syncevo-dbus-server locked up after receiving
a CTRL-C. This primarily affected nightly testing, in particular (?)
on Ubuntu Lucid.
* scripting: prevent premature loop timeouts
The more complex "avoid data loss during merging" scripting ran for longer
than 5s limit under extreme conditions (full logging, busy system, running
under valgrind), which resulted in aborting the script and a 10500 "local
internal error" sync failure.
* signon: fix providersignon.so (TC-1667)
The shared providersignon.so ended up being compiled with "gsso" as
prefix for the username. There also was a problem with invalid
reference counting.
* PBAP: support SYNCEVOLUTION_PBAP_CHUNK_TRANSFER_TIME <= 0
When set to 0 or less, the chunk size is not getting adapted at all
while still using transfers in chunks.
SyncEvolution 1.4.99.3 -> 1.4.99.4, 10.09.2014
==============================================
One focus in this release was on minimizing CPU consumption and disk
writes. The most common case, a two-way sync with no changes on either
side, no longer rewrites any meta data files. CPU consumption during
local sync was reduced to one third by exchanging messages via shared
memory instead of internal D-Bus. Redundant vCard decode/encode on the
sending side of PBAP and too agressive flushing of meta data during a
normal sync were removed. Altogether, sending 1000 contacts with photo
data in a refresh-from-server local sync takes only one sixth of the
CPU cycles compared to 1.3.99.3 (measured with valgrind's callgrind on
x86_64).
Based on community feedback and discussions, the terminology used in
SyncEvolution for configuration, local sync and database access was
revised. Some usability issues with setting up access to databases
were addressed. For Google, the obsolete SyncML config template was
removed and CalDAV/CardDAV were merged into a single "Google"
template.
Using Google Calendar/Contacts with OAuth2 authentication on a
headless server becomes a bit easier: it is possible to set up access
on one system with a GUI using either gSSO or GNOME Online Accounts,
then take the OAuth2 refresh token and use it in SyncEvolution on a
different system. See
http://cgit.freedesktop.org/SyncEvolution/syncevolution/tree/src/backends/oauth2/README
Some issues accessing Apple iCloud were fixed such that CardDAV works
by just giving SyncEvolution username=foobar@icloud.com and password. No
throrough testing was done, so iCloud support is still experimental.
The PIM Manager API also supports Google Contact syncing. Some
problems with suspending a PBAP sync were fixed. Suspend/abort can
be tested with the sync.py example.
The EDS memo backend is able to switch between syncing in plain
text and iCalendar 2.0 VJOURNAL automatically.
Details:
* oauth2: new backend using libsoup/libcurl
New backend implements identity provider for obtaining OAuth2 access
token for systems without HMI support.
Access token is obtained by making direct HTTP request to OAuth2 server
and using refresh token obtained by user in some other way.
New provider automatically updates stored refresh token when OAuth2
server is issuing new one.
* PBAP: use raw text items
This avoids the redundant parse/generate step on the sending
side of the PBAP sync.
* datatypes: raw text items with minimal conversion (FDO #52791)
When using "raw/text/calendar" or "raw/text/vcard" as SyncEvolution
"databaseFormat", all parsing and conversion is skipped. The backend's
data is identical to the item data in the engine. Finding duplicates
in a slow sync is very limited when using these types because the entire
item data must match exactly.
This is useful for the file backend when the goal is to store an exact copy
of what a peer has or for limited, read-only backends (PBAP). The downside
of using the raw types is that the peer is not given accurate information
about which vCard or iCalendar properties are supported, which may cause
some peers to not send all data.
* engine: flush map items less frequently
The Synthesis API does not say explicitly, but in practice all map
items get updated in a tight loop. Rewriting the m_mappingNode (case
insensitive string comparisons) and serialization to disk
(std::ostrstream) consume a significant amount of CPU cycles and cause
extra disk writes that can be avoided by making some assumptions about
the sequence of API calls and flushing only once.
* SoupTransport: drop CA file check
It used to be necessary to specify a CA file for libsoup to enable SSL
certificate checking. Nowadays libsoup uses the default CA store
unless told otherwise, so the check in SyncEvolution became
obsolete. However, now there is a certain risk that no SSL checking is
done although the user asked for it (when libsoup is not recent enough
or compiled correctly).
* local sync: exchange SyncML messages via shared memory
Encoding/decoding of the uint8_t array in D-Bus took a surprisingly
large amount of CPU cycles relative to the rest of the SyncML message
processing. Now the actual data resides in memory-mapped temporary
files and the D-Bus messages only contain offset and size inside these
files. Both sides use memory mapping to read and write directly.
For caching 1000 contacts with photos on a fast laptop, total sync
time roughly drops from 6s to 3s.
To eliminate memory copies, memory handling in libsynthesis or rather,
libsmltk is tweaked such that it allocates the buffer used for SyncML
message data in the shared memory buffer directly. This relies on
knowledge of libsmltk internals, but those shouldn't change and if they
do, SyncEvolution will notice ("unexpected send buffer").
* local sync: avoid updating meta data when nothing changed
The sync meta data (sync anchors, client change log) get updated after
a sync even if nothing changed and the existing meta data could be
used again. This can be skipped for local sync, because then
SyncEvolution can ensure that both sides skip updating the meta
data. With a remote SyncML server that is not possible and thus
SyncEvolution has to update its data.
This optimization is only used for local syncs with one source. It is
based on the observation that when the server side calls
SaveAdminData, the client has sent its last message and the sync is
complete. At that point, SyncEvolution can check whether anything has
changed and if not, skip saving the server's admin data and stop the
sync without sending the real reply to the client.
Instead the client gets an empty message with "quitsync" as content
type. Then it takes shortcuts to close down without finalizing the
sync engine, because that would trigger writing of meta data
changes. The server continues its shutdown normally.
This optimization is limited to syncs with a single source, because
the assumption about when aborting is possible is harder to verify
when multiple sources are involved.
* PIM: include CardDAV in CreatePeer()
This adds "protocol: CardDAV" as a valid value, with corresponding
changes to the interpretation of some existing properties and some new
ones. The API itself is not changed.
Suspending a CardDAV sync is possible. This freezes the internal
SyncML message exchange, so data exchange with the CardDAV server may
continue for a while after SuspendPeer().
Photo data is always downloaded immediately. The "pbap-sync" flag
in SyncPeerWithFlags() has no effect.
Syncing can be configured to be one-way (local side is read-only
cache) or two-way (local side is read/write). Meta data must be
written either way, to speed up caching or allow two-way syncing. The
most common case (no changes on either side) will have to be optimized
such that existing meta data is not touched and thus no disk writes
occur.
* PIM: handle SuspendPeer() before and after transfer (FDO #82863)
A SuspendPeer() only succeeded while the underlying Bluetooth transfer
was active. Outside of that, Bluez errors caused SyncEvolution to
attempt a cancelation of the transfer and stopped the sync.
When the transfer was still queueing, obexd returns
org.bluez.obex.Error.NotInProgress. This is difficult to handle for
SyncEvolution: it cannot prevent the transfer from starting and has to
let it become active before it can suspend the transfer. Canceling
would lead to difficult to handle error cases (like partially parsed
data) and therefore is not done.
The Bluez team was asked to implement suspending of queued transfers
(see "org.bluez.obex.Transfer1 Suspend/Resume in queued state" on
linux-bluetooth@vger.kernel.org), so this case might not happen
anymore with future Bluez.
When the transfer completes before obexd processes the Suspend(),
org.freedesktop.DBus.Error.UnknownObject gets returned by
obexd. SyncEvolution can ignore errors which occur after the active
transfer completed. In addition, it should prevent starting the next
one. This may be relevant for transfer in chunks, although the sync
engine will also stop asking for data and thus typically no new
transfer gets triggered anyway.
* PIM: add suspend/resume/abort to sync.py
CTRL-C while waiting for the end of a sync causes an interactive
prompt to appear where one can choose been suspend/resume/abort and
continuing to wait. CTRL-C again in the prompt aborts the script.
* PIM: fix sync.py --sync-flags
The help text used single quotes for the JSON example instead of
the required double quotes. Running without --sync-flags was broken
because of trying to parse the empty string as JSON.
* command line: revise usability checking of datastores
When configuring a new sync config, the command line checks whether a
datastore is usable before enabling it. If no datastores were listed
explicitly, only the usable ones get enabled. If unusable datastores
were explicitly listed, the entire configure operation fails.
This check was based on listing databases, which turned out to be too
unspecific for the WebDAV backend: when "database" was set to some URL
which is good enough to list databases, but not a database URL itself,
the sources where configured with that bad URL.
Now a new SyncSource::isUsable() operation is used, which by default
just falls back to calling the existing Operations::m_isEmpty. In
practice, all sources either check their config in open() or the
m_isEmpty operation, so the source is usable if no error is
enountered.
For WebDAV, the usability check is skipped because it would require
contacting a remote server, which is both confusing (why does a local
configure operation need the server?) and could fail even for valid
configs (server temporarily down). The check was incomplete anyway
because listing databases gave a fixed help text response when no
credentials were given. For usability checking that should have
resulted in "not usable" and didn't.
The output during the check was confusing: it always said "listing
databases" without giving a reason why that was done. The intention
was to give some feedback while a potentially expensive operation
ran. Now the isUsable() method itself prints "checking usability" if
(and only if!) such a check is really done.
Sometimes datastores were checked even when they were about to be
configure as "disabled" already. Now checking such datastores is
skipped.
* EDS: memo syncing as iCalendar 2.0 (FDO #52714)
When syncing memos with a peer which also supports iCalendar 2.0 as
data format, the engine will now pick iCalendar 2.0 instead of
converting to/from plain text. The advantage is that some additional
properties like start date and categories can also be synchronized.
The code is a lot simpler, too, because the EDS specific iCalendar 2.0
<-> text conversion code can be removed.
* datatypes: text/calendar+plain revised heuristic
When sending a VEVENT, DESCRIPTION was set to the SUMMARY if empty. This may
have been necessary for some peers, but for notes (= VJOURNAL) we don't know
that (hasn't been used in the past) and don't want to alter the item
unnecessarily, so skip that part and allow empty DESCRIPTION.
When receiving a plain text note, the "text/calendar+plain" type
used to store the first line as summary and the rest as description.
This may be correct in some cases and wrong in others.
The EDS backend implemented a different heuristic: there the first
line is copied into the summary and stays in the description. This
makes a bit more sense (the description alone is always enough to
understand the note). Therefore and to avoid behavioral changes
for EDS users when switching the EDS backend to use text/calendar+plain,
the engine now uses the same approach.
* source -> datastore rename, improved terminology
The word "source" implies reading, while in fact access is read/write.
"datastore" avoids that misconception. Writing it in one word emphasizes
that it is single entity.
While renaming, also remove references to explicit --*-property
parameters. The only necessary use today is "--sync-property ?"
and "--datastore-property ?".
--datastore-property was used instead of the short --store-property
because "store" might be mistaken for the verb. It doesn't matter
that it is longer because it doesn't get typed often.
--source-property must remain valid for backward compatility.
As many user-visible instances of "source" as possible got replaced in
text strings by the newer term "datastore". Debug messages were left
unchanged unless some regex happened to match it.
The source code will continue to use the old variable and class names
based on "source".
Various documentation enhancements:
Better explain what local sync is and how it involves two sync
configs. "originating config" gets introduces instead of just
"sync config".
Better explain the relationship between contexts, sync configs,
and source configs ("a sync config can use the datastore configs in
the same context").
An entire section on config properties in the terminology
section. "item" added (Todd Wilson correctly pointed out that it was
missing).
Less focus on conflict resolution, as suggested by Graham Cobb.
Fix examples that became invalid when fixing the password
storage/lookup mechanism for GNOME keyring in 1.4.
The "command line conventions", "Synchronization beyond SyncML" and
"CalDAV and CardDAV" sections were updated. It's possible that the
other sections also contain slightly incorrect usage of the
terminology or are simply out-dated.
* local sync: allow config name in syncURL=local://
Previously, only syncURL=local://@<context name> was allowed and used
the "target-config@context name" config as target side in the local
sync.
Now "local://config-name@context-name" or simply "local://config-name"
are also allowed. "target-config" is still the fallback if only a
context is give.
It also has one more special meaning: "--configure
target-config@google" will pick the "Google" template automatically
because it knows that the intention is to configure the target side
of a local sync. It does not know that when using some other name
for the config, in which case the template (if needed) must be
specified explicitly.
The process name in output from the target side now also includes the
configuration name if it is not the default "target-config".
* Google: remove SyncML template, combine CalDAV/CardDAV
Google has turned off their SyncML server, so the corresponding
"Google Contacts" template became useless and needs to be removed. It
gets replaced by a "Google" template which combines the three
different URLs currently used by Google for CalDAV/CardDAV.
This new template can be used to configure a "target-config@google"
with default calendar and address book database already enabled. The
actual URL of these databases will be determined during the first
sync using them.
The template relies on the WebDAV backend's new capability to search
multiple different entries in the syncURL property for databases. To
avoid listing each calendar twice (once for the legacy URL, once with
the new one) when using basic username/password authentication, the
backend needs a special case for Google and detect that the legacy URL
does not need to be checked.
* config: allow storing credentials for email address
When configuring a WebDAV server with username = email address and no
URL (which is possible if the server supports service discovery via
the domain in the email address), then storing the credentials in the
GNOME keyring used to fail with "cannot store password in GNOME
keyring, not enough attributes".
That is because GNOME keyring seemed to get confused when a network
login has no server name and some extra safeguards were added to
SyncEvolution to avoid this.
To store the credentials in the case above, the email address now gets
split into user and domain part and together get used to look up the
password.
SyncEvolution 1.4.99.2 -> 1.4.99.3, 23.07.2014
==============================================
This release enhances CalDAV/CardDAV and PBAP syncing and fixes some
problems. The enhanced conflict handling introduced 1.4.99.2 was
unintentionally limited to syncs with EDS on the server side; it is
now also available for example in WebDAV<->SyncML bridge setups.
Details:
* CardDAV: implement read-ahead
Instead of downloading contacts one-by-one with GET, SyncEvolution now
looks at contacts that are most likely going to be needed soon and
gets all of them at once with addressbook-multiget REPORT.
The number of contacts per REPORT is 50 by default, configurable by
setting the SYNCEVOLUTION_CARDDAV_BATCH_SIZE env variable.
This has two advantages:
- It avoids round-trips to the server and thus speeds up a large
download (100 small contacts with individual GETs took 28s on
a fast connection, 3s with two REPORTs).
- It reduces the overall number of requests. Google CardDAV is known
to start issuing intermittent 401 authentication errors when the
number of contacts retrieved via GET is too large. Perhaps this
can be avoided with addressbook-multiget.
* Google Calendar: remove child hack, improve alarm hack (FDO #63881)
Google recently enhanced support for RECURRENCE-ID, so SyncEvolution
no longer needs to replace the property when uploading a single
detached event with RECURRENCE-ID. However, several things are still
broken in the server, with no workaround in SyncEvolution:
- Removing individual events gets ignored by the server;
a full "wipe out server data" might work (untested).
- When updating the parent event, all child events also get
updated even though they were included unchanged in the data
sent by SyncEvolution.
- The RECURRENCE-ID of a child event of an all-day recurring event
does not get stored properly.
- The update hack seems to fail for complex meetings: uploading them
once and then deleting them seems to make uploading them again
impossible.
All of these issues were reported to Google and are worked on there,
so perhaps the situation will improve. In the meantime, syncing with
Google CalDAV should better be limited to:
- Downloading a Google calendar in one-way mode.
- Two-way syncing of simple calendars without complex meeting
serieses.
While updating the Google workarounds, the alarm hack (sending a
new event without alarms twice to avoid the automatic server side
alarm) was simplified. Now the new event gets sent only once with a
pseudo-alarm.
* ephemeral sync: don't write binfile client files (FDO #55921)
When doing PBAP caching, we don't want any meta data written because
the next sync would not use it anyway. With the latest libsynthesis
we can configure "/dev/null" as datadir for the client's binfiles and
libsynthesis will avoid writing them.
The PIM manager uses this for PBAP syncing automatically. For testing
it can be enabled by setting the SYNCEVOLUTION_EPHEMERAL env variable.
* PBAP: avoid empty field filter
Empty field filter is supposed to mean "return all supported
fields". This used to work and stopped working with Android phones
after an update to 4.3 (seen on Galaxy S3); now the phone only
returns the mandatory TEL, FN, N fields.
The workaround is to replace the empty filter list with the list of
known and supported properties. This means we only pull data we really
need, but it also means we won't get to see any additional properties
that the phone might support.
* PBAP: transfer in chunks (FDO #77272)
If enabled via env variables, PullAll transfers will be limited to
a certain numbers contacts at different offsets until all data got
pulled. See PBAP README for details.
When transfering in chunks, the enumeration of contacts for the engine
no longer matches the PBAP enumeration. Debug output uses "offset #x"
for PBAP and "ID y" for the engine.
* PBAP: remove transfer via pipe
Using a pipe was never fully supported by obexd (blocks
obexd). Transfering in suitably sized chunks (FDO #77272) will be a
more obexd friendly solution with a similar effect (not having to
buffer the entire address book in memory).
* engine: enable batching by default (FDO #52669)
This reverts commit c435e937cd406e904c437eec51a32a6ec6163102.
Commit 7b636720a in libsynthesis fixes an unitialized memory read in
the asynchronous item update code path.
Testing confirms that we can now used batched writes reliably with EDS
(the only backend currently supporting asynchronous writes +
batching), so this change enables it again also for local and
SyncEvolution<->SyncEvolution sync (with asynchronous execution of
contact add/update overlapped with SyncML message exchanges) and other
SyncML syncs (with changes combined into batches and executed at the
end of each message).
* datatypes: fix contact caching
Adding grouping to the contact datatype in 1.4.99.2 broke PBAP caching: when
sending an empty URL, for example, during the sync, the parsed contact
had different field arrays than the locally stored contact, because the
latter was saved without the empty URL.
This caused the field-based comparison to detect a difference even when
the final, reencoded contact wasn't different at all.
To solve this, syncing now uses the same "don't send empty properties"
configuration as local storages. Testing shows that this resolves
the difference for EDS.
A more resilient solution would be to add a check based on the encoded
data, but that's more costly performance wise.
* datatypes: fix vCard handling
The new "preserve repeating properties during conflict resolution"
feature from 1.4.99.2 was only active when using EDS as storage. The relevant
merge script must be applied to all datatypes, not just the EDS
flavor.
The feature was also unintentionally active when running in
caching mode. This caused two problems:
- The cached item was updated even though only the
ordering of repeating properties had been modified during
merging.
- The merged item was sent back to the client side, which
was undesirable (caching is supposed to be one-way) or even
impossible (PBAP is read-only, causing sync failures eith error 20030).
We must check for caching mode and disable merging when it is active.
We also must not tell the engine that we updated the photo property
in the winning item, because then that item would get sent to the
read-only side of the sync.
Perhaps a better solution would be to actually tell the engine
that the remote side is read-only when we activate caching mode.
* datatypes: avoid PHOTO corruption during merge (FDO #77065)
When handling an update/update conflict (both sides of the sync have an
updated contact) and photo data was moved into a local file by EDS, the engine
merged the file path and the photo data together and thus corrupted the photo.
The engine does not know about the special role of the photo property.
This needs to be handled by the merge script, and that script did not
cover this particular situation. Now the loosing side is cleared,
causing the engine to then copy the winning side over into the loosing
one.
Found by Renato Filho/Canonical when testing SyncEvolution for Ubuntu 14.04.
* PBAP syncing: updated photo not always stored
Because photo data was treated like a C string, changes after any
embedded null byte were ignored during a comparison.
* PIM: fix phone number normalization
The parsed number always has a country code, whereas SyncEvolution expected it
to be zero for strings without an explicit country code. This caused a caller
ID lookup of numbers like "089788899" in DE to find only telephone numbers in
the current default country, instead of being more permissive and also finding
"+189788899". The corresponding unit test was broken and checked for the wrong
result. Found while investigating an unrelated test failure when updating
libphonenumber.
* Various compiler problems and warnings fixed; compiles with
--with-warnings=fatal on current Debian Testing and Ubuntu Trusty
(FDO #79316).
SyncEvolution 1.4.99.1 -> 1.4.99.2, 23.05.2014
==============================================
1.4.99.2 enhances interoperability with CardDAV servers and in
particular Google Contacts considerably. Contact data gets converted
to and from the format typically used by CardDAV servers, so now
anniversary, spouse, manager, assistant and instant message
information are exchanged properly.
Custom labels get stored in EDS as extensions and no longer get lost
when updating some other aspects of a contact. However, Evolution does
not show custom labels and removes them when editing a property which
has a custom label (BGO #730636).
Scanning for CardDAV/CalDAV resources was enhanced. It now finds
additional calendars with Google CalDAV and works with iCloud.
However, syncing with iCloud ran into a server bug (reported as
17001498 "CalDAV REPORT drops calendar data") and needs further work.
Details:
* vcard profile: avoid data loss during merging
When resolving a merge conflict, repeating properties were taken
wholesale from the winning side (for example, all email addresses). If
a new email address had been added on the loosing side, it got lost.
Arguably it is better to preserve as much data as possible during a
conflict. SyncEvolution now does that in a merge script by checking
which properties in the loosing side do not exist in the winning side
and copying those entries.
Typically only the main value (email address, phone number) is checked
and not the additional meta data (like the type). Otherwise minor
differences (for example, both sides have same email address, but with
different types) would lead to duplicates.
Only addresses are treated differently: for them all attributes
(street, country, city, etc.) are compared, because there is no single
main value.
* engine: UID support in contact data
Before, the UID property in a vCard was ignored by the engine.
Backends were responsible for ensuring that the property is
set if required by the underlying storage. This turned out to be
handled incompletely in the WebDAV backend.
This change moves this into the engine:
- UID is now field. It does not get used for matching
because the engine cannot rely on it being stored
by both sides.
- It gets parsed if present, but only generated if
explicitly enabled (because that is the traditional
behavior).
- It is never shown in the DevInf's CtCap
because the Synthesis engine would always show it
regardless whether a rule enabled the property.
That's because rules normally only get triggered
after exchanging DevInf and thus DevInf has to
be rule-independent. We don't want it shown because
then merging the incoming item during a local sync
would use the incoming UID, even if it is empty.
- Before writing, ensure that UID is set.
When updating an existing item, the Synthesis engine reads
the existing item, preserves the existing UID unless the peer
claims to support UID, and then updates with the existing UID.
This works for local sync (where SyncEvolution never claims
to support UID when talking to the other side). It will break
with peers which have UID in their CtCap although they
rewrite the UID and backends whose underlying storage cannot
handle UID changes during an update (for example, CardDAV).
* CardDAV: use Apple/Google/CardDAV vCard flavor
In principle, CardDAV servers support arbitrary vCard 3.0
data. Extensions can be different and need to be preserved. However,
when multiple different clients or the server's Web UI interpret the
vCards, they need to agree on the semantic of these vCard extensions.
In practice, CardDAV was pushed by Apple and Apple clients are
probably the most common clients of CardDAV services. When the Google
Contacts Web UI creates or edits a contact, Google CardDAV will
send that data using the vCard flavor used by Apple.
Therefore it makes sense to exchange contacts with *all* CardDAV
servers using that format. This format could be made configurable in
SyncEvolution on a case-by-case basis; at the moment, it is
hard-coded.
During syncing, SyncEvolution takes care to translate between the
vCard flavor used internally (based on Evolution) and the CardDAV
vCard flavor. This mapping includes:
X-AIM/JABBER/... <-> IMPP + X-SERVICE-TYPE
Any IMPP property declared as X-SERVICE-TYPE=AIM will get
mapped to X-AIM. Same for others. Some IMPP service types
have no known X- property extension; they are stored in
EDS as IMPP. X- property extensions without a known X-SERVICE-TYPE
(for example, GaduGadu and Groupwise) are stored with
X-SERVICE-TYPE values chosen by SyncEvolution so that
Google CardDAV preserves them (GroupWise with mixed case
got translated by Google into Groupwise, so the latter is used).
Google always sends an X-ABLabel:Other for IMPP. This is ignored
because the service type overrides it.
The value itself also gets transformed during the mapping. IMPP uses
an URI as value, with a chat protocol (like "aim" or "xmpp") and
some protocol specific identifier. For each X- extension the
protocol is determined by the property name and the value is the
protocol specific identifier without URL encoding.
X-SPOUSE/MANAGER/ASSISTANT <-> X-ABRELATEDNAMES + X-ABLabel
The mapping is based on the X-ABLabel property attached to
the X-ABRELATEDNAMES property. This depends on the English
words "Spouse", "Manager", "Assistant" that Google CardDAV
and Apple devices seem to use regardless of the configured
language.
As with IMPP, only the subset of related names which have
a corresponding X- property extension get mapped. The rest
is stored in EDS using the X-ABRELATEDNAMES property.
X-ANNIVERSARY <-> X-ABDATE
Same here, with X-ABLabel:Anniversary as the special case
which gets mapped.
X-ABLabel parameter <-> property
CardDAV vCards have labels attached to arbitrary other properties
(TEL, ADR, X-ABDATE, X-ABRELATEDNAMES, ...) via vCard group tags:
item1.X-ABDATE:2010-01-01
item1.X-ABLabel:Anniversary
The advantage is that property values can contain arbitrary
characters, including line breaks and double quotation marks,
which is not possible in property parameters.
Neither EDS nor KDE (judging from the lack of responses on the
KDE-PIM mailing list) support custom labels. SyncEvolution could
have used grouping as it is done in CardDAV, but grouping is not
used much (not at all?) by the UIs working with the vCards in EDS
and KDE. It seemed easier to use a new X-ABLabel parameter.
Characters which cannot be stored in a parameter get converted
(double space to single space, line break to space, etc.) during
syncing. In practice, these characters don't appear in X-ABLabel
properties anyway because neither Apple nor Google UIs allow entering
them for custom labels.
The "Other" label is used by Google even in case where it adds no
information. For example, all XMPP properties have an associated
X-ABLabel=Other although the Web UI does not provide a means to edit
or show such a label. Editing the text before the value in the UI
changes the X-SERVICE-TYPE parameter value, not the X-ABLabel as for
other fields.
Therefore the "Other" label is ignored by removing it during syncing.
X-EVOLUTION-UI-SLOT (the parameter used in Evolution to determine the
order of properties in the UI) gets stored in CardDAV. The only exception
is Google CardDAV which got confused when an IMPP property had both
X-SERVICE-TYPE and X-EVOLUTION-UI-SLOT parameters set. For Google,
X-EVOLUTION-UI-SLOT is only sent on other properties and thus ordering
of chat information can get lost when syncing with Google.
* synccompare: support grouping and quoted parameter strings
Grouped properties are sorted first according to the actual property
name, then related properties are moved to the place where their group
tag appears first. The first grouped property gets a "- " prefix, all
following ones are just indended with " ". The actual group tag is not
part of the normalized output, because its value is irrelevant:
BDAY:19701230
- EMAIL:john@custom.com
X-ABLabel:custom-label2
...
FN:Mr. John 1 Doe Sr.
- IMPP;X-SERVICE-TYPE=AIM:aim:aim
X-ABLabel:Other
...
- X-ABDATE:19710101
X-ABLabel:Anniversary
Redundant tags (those set for only a single property, X-ABLabel:Other)
get removed as part of normalizing an item.
* WebDAV: use server's order when listing collections
When doing a recursive scan of the home set, preserve the order of
entries as reported by the server and check the first one first. The
server knows better which entries are more relevant for the user (and
thus should be the default) or may have some other relevant
order. Previously, SyncEvolution replaced that order with sorting by
URL, which led to a predictable, but rather meaningless order.
For example, Google lists the users own calendar first, followed by
the shared calendars sorted alphabetical by their name. Now
SyncEvolution picks the main calendar as default correctly when
scanning from https://www.google.com/calendar/dav/.
* WebDAV: improved database search (Google, Zimbra)
Zimbra has a principal URL that also serves as home set. When using it
as start URL, SyncEvolution only looked the URL once, without listing
its content, and thus did not find the databases.
When following the Zimbra principal URL indirectly, SyncEvolution did
check all of the collections there recursively. Unfortunately that
also includes many mail folders, causing the scan to abort after
checking 1000 collections (an internal safe guard).
The solution for both includes tracking what to do with a URL. For the
initial URL, only meta data about the URL itself gets
checked. Recursive scanning is only done for the home set. If that
home set contains many collections, scanning is still slow and may run
into the internal safe guard limit. This cannot be avoided because the
CalDAV spec explicitly states that the home set may contain normal
collections which contain other collections, so a client has to do the
recursive scan.
When looking at a specific calendar, Google CalDAV does not report
what the current principal or the home set is and therefore
SyncEvolution stopped after finding just the initial calendar. Now it
detects the lack of meta information and adds all parents also as
candidates that need to be looked at. The downside of this is that it
doesn't know anything about which parents are relevant, so it ends up
checking https://www.google.com/calendar/ and
https://www.google.com/.
In both cases Basic Auth gets rejected with a temporary redirect to
the Google login page, which is something that SyncEvolution must
ignore immediately during scanning without applying the resend
workaround for "temporary rejection of valid credentials" that can
happen for valid Google CalDAV URLs.
* WebDAV: enhanced database search (Google Calendar)
Additional databases where not found for several
reasons. SyncEvolution ignored all shared calendars
(http://calendarserver.org/ns/shared) and Google marks the additional
calendars that way. The other problem was that the check for leaf
collections (= collections which cannot contain other desired
collections) incorrectly excluded those collections instead of only
preventing listing of their content.
With this change,
https://www.google.com/calendar/dav/?SyncEvolution=Google can be used
as starting point for Google Calendar.
* WebDAV: fix database scan on iCloud
The calendar home set URL on iCloud (the one ending in /calendars/) is
declared as containing calendar data. That was enough for
SyncEvolution to accept it incorrectly as calendar. However, the home
set only contains calendar data indirectly.
* WebDAV: support redirects between hosts and DNS SRV lookup based on URL
When finding a new URL, we must be prepared to reinitialize the Neon
session with the new host settings.
iCloud does not have .well-known support on its www.icloud.com
server. To support lookup with a non-icloudd.com email address, we
must do DNS SRV lookup when access to .well-known URLs fails. We do
this without a www prefix on the host first, because that is what happens
to work for icloud.com.
With these changes it becomes possible to do database scans on Apple
iCloud, using syncURL=https://www.icloud.com or
syncURL=https://icloud.com. Giving the syncURL like this is only
necessary for a username that does not end in @icloud.com. When
the syncURL is not set, the domain for DNS SRV lookup is taken
from the username.
* WebDAV: more efficient item creation
PUT has the disadvantage that a client needs to choose a name and then
figure out what the real name on the server is. With Google CardDAV that
requires sending another request and only works because the server happens
to remember the original name (which is not guaranteed!).
POST works for new items without a name and happens to be implemented
by Google such that the response already includes all required
information (new name and revision string).
POST is checked for as described in RFC 5995 once before creating a new
item. Servers which don't support it continue to get a PUT.
* WebDAV: send "User-Agent: SyncEvolution"
Apple iCloud servers reject requests unless they contain a User-Agent
header. The exact value doesn't seem to matter. Making the string
configurable might be better, but can still be done later when it
is more certain whether and for what it is needed.
* WebDAV: refactor and fix DNS SRV lookup
The syncevo-webdav-lookup script was not packaged. It did not report
"not found" DNS results correctly and the caller did not check for
this either, so when looking up the information for a domain which
does not have DNS SRV entries, SyncEvolution ended up retrying for
while as if there had been a temporary lookup problem.
* signon: make Accounts optional
The new "signon" provider only depends on lib[g]signon-glib. It uses
gSSO if found, else UOA. Instead of pulling parameters and the
identity via libaccounts-glib, the user of SyncEvolution now has to
ensure that the identity exists and pass all relevant parameters
in the "signon:" username.
* gSSO: adapt to gSSO >= 2.0
* config templates: Funambol URLs
Funambol turned of the URL redirect from my.funambol.com to
onemedia.com. The Funambol template now uses the current URL. Users
with existing Funambol configs must updated the syncURL property
manually to https://onemediahub.com/sync
Kudos to Daniel Clement for reporting the change.
* command line: fix --update from directory
The "--update <dir name>" operation was supposed to take the
item luids from the file names inside the directory. That part
had not been implemented, turning the operation accidentally
into an "--import".
Also missing was the escaping/unescaping of luids. Now the
same escaping is done as in command line output and command
line parsing to make the luids safe for use as file name.
* testing: added server-specific tests for CardDAV covering
remote item formats and edit conflicts.
SyncEvolution 1.4.1 -> 1.4.99.1, 01.04.2014
===========================================
1.4.99.1 includes several PIM Manager improvements plus some unrelated
fixes.
Details:
* LogRedirect: safeguard against memory corruption
When aborting, our AbortHandler gets called to close down logging.
This may involve memory allocation, which is unsafe. In FDO #76375, a
deadlock on a libc mutex was seen.
To ensure that the process shuts down anyway, install an alarm and give
the process five seconds to shut down before the SIGALRM signal will kill
it.
* PBAP: Suspend/ResumeSync() (FDO #72112)
By default, the new API freezes a sync by stopping to consume data on the
local side of the sync.
In addition, the information that the sync is freezing is now also handed
down to the transport and all sources. In the case of PBAP caching, the local
transport notifies the child where the PBAP source then uses Bluez
5.15 Transfer1.Suspend/Resume to freeze/thaw the actual OBEX transfer.
If that fails (for example, not implemented because Bluez is too old
or the transfer is still queueing), then the transfer gets cancelled
and the entire sync fails. This is desirable for PBAP caching and
Bluetooth because a failed sync can easily be recovered from (just
start it again) and the overall goal is to free up Bluetooth bandwidth
quickly.
* PBAP: transfer data via pipe (part of FDO #72112)
The main advantage is that processed data can be discarded
immediately. When using a plain file, the entire address book must be
stored in it.
The drawback is that obexd does not react well to a full pipe. It
simply gets stuck in a blocking write(); in other words, all obexd
operations get frozen and obexd stops responding on D-Bus.
* PIM: enhanced progress notifications (FDO #72114)
This adds GetPeerStatus() and "progress" events. Progress is reported based
on the "item received" Synthesis event and the total item count. A modified
libsynthesis is needed where the SyncML binfile client on the target side of
the local sync actually sends the total item count (via NumberOfChanges).
This cannot be done yet right at the start of the sync, only the second SyncML
message will have it. That is acceptable, because completion is reached very
quickly anyway for syncs involving only one message.
At the moment, SyncContext::displaySourceProgress() holds back "item
received" events until a different event needs to be emitted. Progress
reporting might get more fine-grained when adding allowing held back
events to be emitted at a fixed rate, every 0.1s. This is not done yet
because it seems to work well enough already.
For testing and demonstration purposes, sync.py gets command line
arguments for setting progress frequency and showing progress either
via listening to signals or polling.
* PIM: add SyncPeerWithFlags() and 'pbap-sync' flag (FDO #70950)
The is new API and flag grant control over the PBAP sync mode.
* D-Bus: use streams for direct IPC with GIO
When using GIO, it is possible to avoid the DBusServer listening on a
publicly accessible address. Connection setup becomes more reliable,
too, because the D-Bus server side can detect that a child died
because the connection will be closed.
When using libdbus, the traditional server/listen and client/connect
model is still used.
* sync output: hide "<source>: started" INFO messages
These messages get printed at the start of processing each
SyncML message. This is not particularly useful and just
adds noise to the output.
* signon: fix build
Static build was broken for gSSO and UOA (wrong path name to .la file)
and gSSO was not enabled properly (wrong condition check).
SyncEvolution 1.4 -> 1.4.1, 31.03.2014
======================================
The first bug fix release in the 1.4 series addresses some issues which
occurred on some systems.
Details:
* EDS: only load one backend plugin of each kind
SyncEvolution was meant to load the syncecal or syncebook shared object
which uses the most recent libraries (libical, libecal/libebook) on
the system and then stop loooking for alternatives. Due to a string
handling bug the check for already backends always found nothing,
leading to multiple conflicting backends loaded on some systems (for
example, those with libical0 and libical1 installed).
If that happened, the backend became unusable.
* ical: workaround for libical 1.0 builtin timezone change
libical 1.0 started to return VTIMEZONE definitions with multiple
absolute transition times instead of RRULEs. This causes problems when
exchanging data with peers (see
https://sourceforge.net/p/freeassociation/bugs/95/).
In SyncEvolution, this affected sending an event using New Zealand
time in vCalendar 1.0 format to a phone, because the internal,
out-dated definition of the time zone in libsynthesis was used as
fallback when loading RRULE-based timezone definitions from libical
failed (see "[SyncEvolution] Some events showing wrong time on
phone"). It might also affect exchanging data with CalDAV peers (not
tested).
The workaround is to include the original code from libical.
* dbus-session.sh: create XDG_RUNTIME_DIR
More recent distros (for example, Ubuntu Saucy) rely on
XDG_RUNTIME_DIR. Each time dbus-session.sh runs, it must
ensure that the runtime dir exists and is empty.
This was a problem when trying to run activesyncd + SyncEvolution
on a headless Ubuntu Saucy server (see FDO #76273).
* Akonadi: support KDE Notes, enhanced "database" check
The KDE Notes resources store items under a different MIME type than the one
used in AKonadi (see "[Kde-pim] note format"). SyncEvolution use the same type
as Akonadi and thus did not find existing KDE Notes resources.
To support both while KDE and Akonadi transition to the same type,
SyncEvolution now looks for notes resources using both MIME types and accepts
both kinds of items when reading. When writing, SyncEvolution picks the MIME
type that is supported by the resource, which hopefully avoids confusing the
KDE app using the resource (untested).
As a positive side effect, the "database" value used for opening a resource is
now checked more thoroughly. Non-existent resources and the type mismatches
like pointing a "kde-contacts" backend to a calendar resource are now detected
early.
* Akonadi: ensure that UID is set (FDO #74342)
Akonadi resources do not enforce iCalendar 2.0 semantic like
"each VEVENT must have a UID" (see "[Kde-pim] iCalendar semantic").
When receiving an event from a peer which itself does not enforce
that semantic (Funambol, vCalendar 1.0 based phones), then we
need to generate a UID, otherwise KOrganizer will ignore the
imported event.
* Akonadi: avoid threading problem in HTTP server mode (FDO #75672)
When used as storage in a server, Akonadi got called in a background thread
that gets created to handle slow initialization of sources and preventing
ensuing timeouts in HTTP clients (probably not needed for Akonadi itself,
but may still be useful when combining it with other sources).
Akonadi cannot be used like that, leading to false "Akonadi not running"
errors or (if one got past that check) failing item operations.
* autotools: Add QtCore include path to KDEPIM_CFLAGS (FDO #75670)
This fixes an issue where configure fails to find Akonadi when
test programs do not compile because QString is not found.
* Enhanced testing again: faster execution, less false negatives
under load. Re-enabled testing of Akonadi.
SyncEvolution 1.3.2 -> 1.4, 16.02.2014
======================================
The 1.4 relase of SyncEvolution is the first stable version with the
in-vehicle infotainment (IVI) PIM Manager included. GENIVI Diagnostic
Log and Trace (DLT) is also supported. For more information about this
aspect of SyncEvolution, see the PBAP and PIM entries in the 1.3.99.x
release notes and
https://syncevolution.org/blogs/pohly/2013/pim-its-all-about-contacts
The biggest change for normal Linux users is Google CalDAV/CardDAV
authentication with OAuth2. These are the open protocol that Google
currently supports and thus the recommended way of syncing with
Google, replacing ActiveSync and SyncML (both no longer available to
all Google customers).
Support for Google CardDAV is new. Like Evolution, SyncEvolution does
not yet support some of the advanced features of the server, in
particular custom labels for phone numbers, emails and
addresses. Likewise, some client properties are not supported by the
server: CALURI, CATEGORIES, FBURL, GEO and ROLE are not supported. Of
ORG, only the first two components are supported. Currently,
properties not supported by one side get lost in a full roundtrip
sync.
SyncEvolution depends on external components for OAuth2. It can be
compiled to use gSSO [1] or GNOME Online Accounts [2]. The latter is
enabled in binaries from syncevolution.org. GNOME Online Accounts >=
3.10 works out of the box for CalDAV and CardDAV. 3.8 is guaranteed to
work for CardDAV and may also work for CalDAV, if the Linux
distribution ships a patched version (like Debian Wheezy does). If it
does not, then GNOME Online Accounts 3.8 binary can be patched to also
support CalDAV, see [2]. Anything older than 3.8 does not
work. Support for Ubuntu Online Accounts is available when compiling
from source. For setup instructions see these READMEs.
[1] https://01.org/gsso and
http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backends/signon/README
[2] http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backends/goa/README
Binary packages of 1.4 on syncevolution.org have enhanced support for
recent distros. They now work with EDS >= 3.6 *and* < 3.6. Distros
with libical1 like Ubuntu Saucy are also supported.
The HTTP server became better at handling message resends when the
server is slow with processing a message. The server is able to keep a
sync session alive while loading the initial data set by sending
acknowledgement replies before the client times out.
Some issues in CalDAV, WebDAV and SyncML were fixed.
Graham R. Cobb contributed several patches for enhancing ActiveSync
support and making it work with Exchange 2010. Guido Günther provided
some patches addressing problems when compiling SyncEvolution for
Maemo.
Details:
* D-Bus server: support DLT (FDO #66769)
Diagnostic Log and Trace (DLT) manages a sequence of log messages,
with remote controllable level of detail. SyncEvolution optionally
(can be chosen at compile time and again at runtime) uses DLT
instead of its own syncevolution-log.html files. See README-DLT.rst
for more information.
To use the feature, configure SyncEvolution with
"--enable-dbus-server=--dlt --no-syslog"
* D-Bus server: fix abort when mixing auto-sync and manual operations (FDO #73562)
When enabling auto-sync for a config and then accessing or syncing the
config manually via the command line tool, the server would abort at
the time when the auto-sync was originally scheduled.
* D-Bus server: accept WBXML with charset in incoming connections
A user reported via email that the Nokia 515 sends
'application/vnd.syncml+wbxml; charset=UTF-8' as type of its messages
this tripped up the syncevo-http-server, leading to:
[ERROR] syncevo-dbus-server: /org/syncevolution/Server: message type
'application/vnd.syncml+wbxml; charset=UTF-8' not supported for starting
a sync
* D-Bus server: command line options for controlling output and startup
The system log is used by default now. New command line options can be
used to change this:
-d, --duration=seconds/'unlimited' Shut down automatically
when idle for this duration (default 300 seconds)
-v, --verbosity=level Choose amount of output, 0 = no output,
1 = errors, 2 = info, 3 = debug; default is 1.
--dbus-verbosity=level Choose amount of output via D-Bus signals, 0 = no output,
1 = errors, 2 = info, 3 = debug; default is 2.
-o, --stdout Enable printing to stdout (result of operations)
and stderr (errors/info/debug).
-s, --no-syslog Disable printing to syslog.
-p, --start-pim Activate the PIM Manager (= unified address book)
immediately.
* D-Bus: missing out parameters in D-Bus introspection XML (FDO #57292)
The problem was in the C++ D-Bus binding. If the method that gets bound
to D-Bus returns a value, that value was ignored in the signature:
int foo() => no out parameter
It works when the method was declared as having a retval:
void foo (int &result) => integer out parameter
This problem existed for both the libdbus and the GIO D-Bus
bindings. In SyncEvolution it affected methods like GetVersions().
* D-Bus server: avoid progress outside of 0-100% range
For example in the new TestLocalCache.testItemDelete100, the
percentage value in the ProgressChanged signal become larger
than 100 and then revert to 100 at the end of the sync.
Seems the underlying calculation is faulty or simply inaccurate.
This is not fixed. Instead the result is just clipped to the valid
range.
* sync: less verbose output, shorter runtime
For each incoming change, one INFO line with "received x[/out of y]"
was printed, immediately followed by another line with total counts
"added x, updated y, removed z". For each outgoing change, a "sent
x[/out of y]" was printed.
In addition, these changes were forwarded to the D-Bus server where a
"percent complete" was calculated and broadcasted to clients. All of
that caused a very high overhead for every single change, even if the
actual logging was off. The syncevo-dbus-server was constantly
consuming CPU time during a sync when it should have been mostly idle.
To avoid this overhead, the updated received/sent numbers that come
from the Synthesis engine are now cached and only processed when done
with a SyncML message or some other event happens (whatever happens
first).
To keep the implementation simple, the "added x, updated y, removed z"
information is ignored completely and no longer appears in the output.
* command line: implement --create/remove-database
Creating a database is only possible with a chosen name. The UID is
chosen automatically by the storage. Only implemented in the EDS
backend.
* command line: execute --export and --print-items while the source is still reading
Instead of reading all item IDs, then iterating over them, process
each new ID as soon as it is available. With sources that support
incremental reading (only the PBAP source at the moment) that provides
output sooner and is a bit more memory efficient.
* command line: recover from slow sync with new sync modes
The error message for an unexpected slow sync still mentioned
the old and obsolete "refresh-from-client/server" sync modes.
Better mention "refresh-from-local/remote".
* command line: show backend error when listing databases fails
The command line swallowed errors thrown by the backend while listing
databases. Instead it just showed "<backend name>: backend failed". The goal
was to not distract users who accidentally access a non-functional backend.
But the result is that operations like --configure or --print-databases could
fail without giving the user any hint about the root cause of the issue.
Now the error explanation in all its gory details is included.
For example, not having activesyncd running leads to:
INFO] eas_contact: backend failed: fetching folder list:
GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name
org.meego.activesyncd was not provided by any .service files
And running activesyncd without the necessary gconf keys shows up as:
[INFO] eas_contact: backend failed: fetching folder list:
GDBus.Error:org.meego.activesyncd.Error.AccountNotFound: Failed to find
account [syncevolution@lists.intel.com]
* password handling: fix usage of GNOME Keyring and KWallet (FDO #66110)
When clients like the GTK sync-ui stored a password, it was always
stored as plain text in the config.ini file by the
syncevo-dbus-server. The necessary code for redirecting the password
storage in a keyring (GNOME or KWallet) simply wasn't called in that
case.
The command line tool, even when using the D-Bus server to run the
operation, had the necessary code active and thus was not affected.
Now all SyncEvolution components use the same default: use safe
password storage if either GNOME Keyring or KWallet were enabled
during compilation, don't use it if not.
Fixing this revealed other problems, like not being able to store
certain passwords that lacked the necessary lookup criteria (like
syncURL and/or username). To address this, the lookup criteria where
extended and a new check was added to avoid accidentally removing
other passwords. As a result, it may be possible that SyncEvolution
no longer finds passwords that were stored with older versions of
SyncEvolution. In such a case the passwords must be set again.
* GNOME: clean up keyring access and require libgnome-keyring >= 2.20
The updated error messages now always include information about the
password and libgnome-keyring error texts.
A workaround is used for the "Error communicating with
gnome-keyring-daemon" problem that started to appear fairly
frequently in the automated testing once the keyring was actually
used. The problem shows up with some additional debug messages:
Gkr: received an invalid, unencryptable, or non-utf8 secret
Gkr: call to daemon returned an invalid response: (null).(null)()
It seems that sometimes setting up a session with GNOME keyring
fails such that all further communication leads to decoding problem.
There is an internal method to reset the session, but it cannot be
called directly. As a workaround, fake the death of the GNOME
keyring daemon and thus trigger a reconnect when retrying the GNOME
keyring access. This is done by sending a D-Bus message, which will
also affect other clients of GNOME keyring, but hopefully without
user-visible effects.
* config: enhanced password handling
It is possible to configure a plain username/password combination
once in SyncEvolution and then use references to it in other
configurations, instead of having to set (and update) the
credentials in different places. This is useful in particular with
WebDAV, where credentials had to be repeated several times (target
config, in each database when used as part of SyncML) or when using
a service which requires several configs (Google via SyncML and
CalDAV).
To use this, create a sync config for a normal peer or a dedicated
config just for the credentials, with "username/password/syncURL"
set. The "syncURL" must be set to something identifying the peer if
GNOME Keyring is used for the password storage.
Then set "username", "databaseUser" and "proxyUser" properties to
"id:<name of config with credentials>" and all read and write access
to those properties will be redirected by SyncEvolution into that
other configuration. This even works in the GTK UI.
For user names which contain colons, the new "user:<user name>" format
must be used. Strings without colons are assumed to be normal user
names, so most old configurations should continue to work.
* signon: new backend using libgsignond-glib + libaccounts-glib
The code works with gSSO (https://01.org/gsso) and Ubuntu Online
Accounts.
* GOA: get OAuth2 tokens out of GNOME Online Accounts
"username = goa:..." selects an account in GOA and retrieves the
OAuth2 token from that.
* WebDAV: support OAuth2
If given an authentication configuration which can handle OAuth2,
then OAuth2 is used instead of plain username/password
authentication.
* WebDAV: support Google CardDAV, break Yahoo
Google CardDAV has one peculiarity: it renames new contacts during PUT without
returning the new path to the client. See also
http://lists.calconnect.org/pipermail/caldeveloper-l/2013-July/000524.html
SyncEvolution already had a workaround for that (PROPGET on old path, extract
new path from response) which happened to work. This workaround was originally
added for Yahoo, which sometimes merges contacts into existing ones. In
contrast to Yahoo, Google really seems to create new items.
Without some server specific hacks, the client cannot tell what happened.
Because Google is currently supported and Yahoo is not, let's change the
hard-coded behavior to "renamed items are new".
* WebDAV: started testing with owndrive.com = OwnCloud
* WebDAV: avoid segfault during collection lookup
Avoid referencing pathProps->second when the set of paths that
PROPFINDs returns is empty. Apparently this can happen in combination
with Calypso.
* CalDAV: more workarounds for Google CalDAV + unique IDs
Google became even more strict about checking REV. Tests which
reused a UID after deleting the original item started to fail sometime
since middle of December 2012.
* CalDAV: work around Google server regression (undeclared namespace prefix in XML)
Google CalDAV for a while (December 2012 till January 2013) sent
invalid XML back when asked to include CardDAV properties in a
PROPFIND. This got rejected in the XML parser, which prevents
syncing calendar data:
Neon error code 1: XML parse error at line 55: undeclared namespace prefix
In the meantime Google fixed the issue in response to a bug report
via email. But the workaround, only asking for the properties which
are really needed, still makes sense and thus is kept.
* WebDAV: auto-discovery fix
With Google Contact + CardDAV the auto-discovery failed after
finding the default address book, without reporting that result.
* WebDAV: don't send Basic Auth via http proactively (FDO #57248)
Sending basic authentication headers via http is insecure. Only do
it proactively when the connection is encrypted and thus protects
the information or when the server explicitly asks for it.
* file backend: sub-second mod time stamps
Change tracking in the file backend used to be based on the
modification time in seconds. When running many syncs quickly (as in
testing), that can lead to changes not being detected when they happen
within a second. Now the file backend also includes the sub-second part of the
modification time stamp, if available.
This change is relevant when upgrading SyncEvolution: most of the
items will be considered "updated" once during the first sync after
the upgrade (or a downgrade) because the revision strings get
calculated differently.
* GTK UI: fixed two crashes - running a sync with no service selected
and a 64 bit pointer problem recently discovered by Tino Keitel when
compiling the Debian package with -fPIE.
* packaging: compatible with EDS up to and including 3.10 and both
libical.so.0 and libical.so.1
The binary packages now contain different versions of syncecal.so
and syncebooks.so to cover different combinations of EDS and libical.
* libical: compatibiliy mode for libical.so.0 and libical.so.1
libical 1.0 broke the ABI, leading to libical.so.1. The only relevant change
for SyncEvolution is the renumbering of ICAL_*_PROPERTY enum values. We can
adapt to that change at runtime, which allows us to compile once with
libical.so.0, then patch executables or use dynamic loading to run with the
more recent libical.so.1 if we add 1 to the known constants.
* packaging: fix rpm (FDO #73347)
After installing the syncevolution.org rpm on OpenSUSE,
SyncEvolution was not starting because its shared libraries were not
found unless "ldconfig" was called manually. Now the package does
that automatically.
* packaging: fix description
The syncevolution-bundle description of both rpm and deb
packagesaccidentally used the same description as
syncevolution-evolution.
* glib: fix double-free of source tags
glib 2.39.0 (aka GNOME 3.10) as found in Ubuntu Trusty introduces
warnings when g_source_remove() is passed an unknown tag. SyncEvolution
did this in two cases: in both, the source callback returned false and thus
caused the source to be removed by the caller. In that case, the explicit
g_source_remove() is redundant and must be avoided.
Such a call is faulty and might accidentally remove a new source with the same
tag (unlikely though, given that tags seem to get assigned incrementally).
The only noticable effect were additional error messages with different
numbers:
[ERROR] GLib: Source ID 9 was not found when attempting to remove it
* EDS: fix compile problem with boost and EDS > 3.36
This fixes the following problem, seen with Boost 1.53.0 on altlinux
when compiling for EDS >= 3.6:
/usr/include/boost/smart_ptr/shared_ptr.hpp: In instantiation of 'typename boost::detail::sp_array_access<T>::type boost::shared_ptr<T>::operator[](std::ptrdiff_t) const [with T = char*; typename boost::detail::sp_array_access<T>::type = void; std::ptrdiff_t = long int]':
src/backends/evolution/EvolutionSyncSource.cpp:163:38: required from here
/usr/include/boost/smart_ptr/shared_ptr.hpp:663:22: error: return-statement with a value, in function returning 'void' [-fpermissive]
make[2]: *** [src/backends/evolution/src_backends_evolution_syncecal_la-EvolutionSyncSource.lo]
* EDS contacts: avoid unnecessary DB writes during slow sync
Traditionally, contacts were modified shortly before writing into EDS
to match with Evolution expectations (must have N, only one CELL TEL,
VOICE flag must be set). During a slow sync, the engine compare the
modified contacts with the unmodified, incoming one. This led to
mismatches and/or merge operations which end up not changing anything
in the DB because the only difference would be removed again before
writing.
* EDS contacts: read-ahead cache
Performance is improved by requesting multiple contacts at once and
overlapping reading with processing. On a fast system (SSD, CPU fast
enough to not be the limiting factor), testpim.py's testSync takes 8
seconds for a "match" sync where 1000 contacts get loaded and compared
against the same set of contacts. Read-ahead with only 1 contact per
query speeds that up to 6.7s due to overlapping IO and
processing. Read-ahead with the default 50 contacts per query takes
5.5s. It does not get much faster with larger queries.
* PBAP: add support for obexd 0.47, 0.48 and Bluez 5
obexd 0.48 is almost the same as obexd 0.47, except that it dropped
the SetFilter and SetFormat methods in favor of passing a Bluex 5-style
filter parameter to PullAll.
* PBAP: various enhancements for efficient caching of contacts
* HTTP server: handle message resends
If a client gave up waiting for the server's response and resent its message
while the server was still processing the message, syncing failed with
"protocol error: already processing a message" raised by the
syncevo-dbus-server because it wasn't prepared to handle that situation.
The right place to handle this is inside the syncevo-http-server, because it
depends on the protocol (HTTP in this case) whether resending is valid or
not. It handles that now by tracking the message that is currently in
processing and matching it against each new message. If it matches, the new
request replaces the obsolete one without sending the message again to
syncevo-dbus-server. When syncevo-dbus-server replies to the old message, the
reply is used to finish the newer request.
* engine: prevent timeouts in HTTP server mode
HTTP SyncML clients give up after a certain timeout (SyncEvolution
after RetryDuration = 5 minutes by default, Nokia e51 after 15
minutes) when the server fails to respond.
This can happen with SyncEvolution as server when it uses a slow
storage with many items, for example via WebDAV. In the case of slow
session startup, multithreading is now used to run the storage
initializing in parallel to sending regular "keep-alive" SyncML
replies to the client.
By default, these replies are sent every 2 minutes. This can be
configured with another extensions of the SyncMLVersion property:
SyncMLVersion = REQUESTMAXTIME=5m
Other modes do not use multithreading by default, but it can be
enabled by setting REQUESTMAXTIME explicitly. It can be disabled
by setting the time to zero.
The new feature depends on a libsynthesis with multithreading enabled
and glib >= 2.32.0, which is necessary to make SyncEvolution itself
thread-safe. With an older glib, multithreading is disabled, but can
be enabled as a stop-gap measure by setting REQUESTMAXTIME explicitly.
* Various testing and stability enhancements. SyncEvolution had to
be made thread-safe for the HTTP timeout prevention.
* Nokia: always add TYPE=INTERNET to EMAIL (FDO #61784)
Without the explicit TYPE=INTERNET, email addresses sent to a Nokia
e51 were not shown by the phone and even got lost eventually (when
syncing back).
This commit ensures that the type is set for all emails sent to any
Nokia phone, because there may be other phones which need it and
phones which don't, shouldn't mind. This was spot-checked with a N97
mini, which works fine with and without the INTERNET type.
This behavior can be disabled again for specific Nokia phones by
adding a remote rule which sets the addInternetEmail session variable
to FALSE again.
Non-Nokia phones can enable the feature in a similar way, by setting
the variable to TRUE.
* SyncML: config option for broken peers
Some peers have problems with meta data (CtCap, old Nokia phones)
and the sync mode extensions required for advertising the restart
capability (Oracle Beehive). The default in SyncEvolution is to
advertise the capability, so manual configuration is necessary when
working with a peer that fails in that mode.
Because the problem occurs when SyncEvolution contacts the peers
before it gets the device information from the peer, dynamic rules
based on the peer identifiers cannot be used. Instead the local config
must already disable these extra features in advance.
The "SyncMLVersion" property gets extended for this. Instead of just
"SyncMLVersion = 1.0" (as before) it now becomes possible to say
"SyncMLVersion = 1.0, noctcap, norestart".
"noctcap" disables sending CtCap. "norestart" disables the sync mode
extensions and thus doing multiple sync cycles in the same session
(used between SyncEvolution instances in some cases to get client and
server into sync in one session).
Both keywords are case-insensitive. There's no error checking for
typos, so beware!
The "SyncMLVersion" property was chosen because it was already in use
for configuring SyncML compatibility aspects and adding a new property
would have been harder.
* ActiveSync: added support for specifying folder names
Previously, the database field was interpreted as a Collection ID. This adds
logic to allow the database to be interpreted as a folder path. The logic is:
1) If the database is an empty string, pass it through (this is the most
common case as it is interpreted as "use the default folder for the
source type").
2) If the database matches a Collection ID, use the ID (this is the same as
the previous behaviour).
3) If the database matches a folder path name, with an optional leading "/",
use the Collection ID for the matching folder.
4) Otherwise, force a FolderSync to get the latest folder changes from the
server and repeat steps 2 and 3
5) If still no match, throw an error.
* ActiveSync: support for listing databases
Now --print-databases scans folders on the ActiveSync server and
shows suitable folders for the ActiveSync backends instead of the
previous, hard-coded help text.
Invoking --print-databases can be used as a workaround for
"SyncFolder error: Invalid synchronization key" errors. A better
solution would be to do that automatically, but there was no time
to implement that. See FDO #61869 and "[SyncEvolution] Activesync server losing state"
http://thread.gmane.org/gmane.comp.mobile.syncevolution/4295
* SyncML: workarounds for broken peers
Some peers have problems with meta data (CtCap, old Nokia phones) and
the sync mode extensions required for advertising the restart
capability (Oracle Beehive).
Because the problem occurs when SyncEvolution contacts the peers
before it gets the device information from the peer, dynamic rules
based on the peer identifiers cannot be used. Instead the local config
must already disable these extra features in advance.
The "SyncMLVersion" property gets extended for this. Instead of just
"SyncMLVersion = 1.0" (as before) it now becomes possible to say
"SyncMLVersion = 1.0, noctcap, norestart".
"noctcap" disables sending CtCap. "norestart" disables the sync mode
extensions and thus doing multiple sync cycles in the same session
(used between SyncEvolution instances in some cases to get client and
server into sync in one session).
Both keywords are case-insensitive. There's no error checking for
typos, so beware!
The "SyncMLVersion" property was chosen because it was already in use
for configuring SyncML compatibility aspects and adding a new property
would have been harder.
* engine: local cache sync mode
This patch introduces support for true one-way syncing ("caching"):
the local datastore is meant to be an exact copy of the data on the
remote side. The assumption is that no modifications are ever made
locally outside of syncing. This is different from one-way sync modes,
which allows local changes and only temporarily disables sending them
to the remote side.
Another goal of the new mode is to avoid data writes as much as
possible.
This new mode only works on the server side of a sync, where the
engine has enough control over the data flow. Setting "sync" to:
- "local-cache-incremental" will do an incremental sync (if possible)
or a slow sync (otherwise). This is usually the right mode to use,
and thus has "local-cache" as alias.
- "local-cache-slow" will always do a slow sync. Useful for
debugging or after (accidentally) making changes on the local side.
An incremental sync will ignore such changes because they are not
meant to happen, aren't checked for to improve performance and
thus will leave client and server out-of-sync!
Both modes are recorded in the sync report of the local side. The
target side is the client and records the normal "two-way" or "slow"
sync modes.
With the current SyncEvolution contact field list, first, middle and
last name are used to find matches for contacts. For events, tasks
and memos, time, summary and description are used.
* Minor memory leak fix when using GDBus GIO: GDBusMethodInfo
Also depends on a glib fix, see https://bugzilla.gnome.org/show_bug.cgi?id=695376
* build fixes
Avoid -lrt in make dependencies. Add missing pcre libs to
syncevo-dbus-server. sqlite backend needs "#include <stdio.h>"
(patch from Mario Kicherer).
* autotools: fix temp file vulnerability during compilation (CVE-2014-1639)
We must use the temporary file that was created for us securily, not
a temp file named after that file. This caused a temp file vulnerability
and the real temporary files were not deleted by the script.
* workarounds for warnings from g++ 4.5
Upgrading from releases <= 1.3.99.4:
If the value of "username/databaseUser/proxyUser" contains a colon,
the "user:" prefix must be added to the value, to continue treating it
like a plain user name and not some reference to an unknown identity
provider (like "id:", "goa:", "signon:", etc.).
The lookup of passwords in GNOME Keyring was updated slightly in
1.3.99.5. It may be necessary to set passwords anew if the old one is
no longer found.
Upgrading from release 1.2.x:
The sync format of existing configurations for Mobical (aka Everdroid)
must be updated manually, because the server has encoding problems when
using vCard 3.0 (now the default for Evolution contacts):
syncevolution --configure \
syncFormat=text/x-vcard \
mobical addressbook
The Funambol template explicitly enables usage of the
"refresh-from-server" sync mode to avoid getting throttled with 417
'retry later' errors. The same must be added to existing configs
manually:
syncevolution --configure \
enableRefreshSync=TRUE \
funambol
Upgrading from releases before 1.2:
Old configurations can still be read. But writing, as it happens
during a sync, must migrate the configuration first. Releases >= 1.2
automatically migrates configurations. The old configurations
will still be available (see "syncevolution --print-configs") but must
be renamed manually to use them again under their original names with
older SyncEvolution releases.
SyncEvolution 1.3.99.7 -> 1.4, 16.02.2014
=========================================
Compared to the pre-release, 1.4 mostly just enhanced the testing.
Compatibility with GNOME 3.10 and a glib-related issue that existed
almost forever without causing obvious problems were
fixed. syncevolution.org binaries now finally work with distros using
libical.so.1 (for example, Ubuntu Saucy and Trusty).
Details:
* autotools: fix temp file vulnerability during compilation (CVE-2014-1639)
We must use the temporary file that was created for us securily, not
a temp file named after that file. This caused a temp file vulnerability
and the real temporary files were not deleted by the script.
* glib: fix double-free of source tags
glib 2.39.0 (aka GNOME 3.10) as found in Ubuntu Trusty introduces
warnings when g_source_remove() is passed an unknown tag. SyncEvolution
did this in two cases: in both, the source callback returned false and thus
caused the source to be removed by the caller. In that case, the explicit
g_source_remove() is redundant and must be avoided.
Such a call is faulty and might accidentally remove a new source with the same
tag (unlikely though, given that tags seem to get assigned incrementally).
The only noticable effect were additional error messages with different
numbers:
[ERROR] GLib: Source ID 9 was not found when attempting to remove it
* libical: compatibiliy mode for libical.so.0 and libical.so.1
libical 1.0 broke the ABI, leading to libical.so.1. The only relevant change
for SyncEvolution is the renumbering of ICAL_*_PROPERTY enum values. We can
adapt to that change at runtime, which allows us to compile once with
libical.so.0, then patch executables or use dynamic loading to run with the
more recent libical.so.1 if we add 1 to the known constants.
SyncEvolution 1.3.99.7, 22.01.2014
==================================
Final release candidate for 1.4. No further changes planned unless
new problems are found.
Details:
* SSO: support Ubuntu Online Accounts
When compiling from source on recent Ubuntu it becomes possible to
use Ubuntu Online Accounts for authenticating against Google's
CalDAV and CardDAV servers.
* D-Bus server: fix abort when mixing auto-sync and manual operations (FDO #73562)
When enabling auto-sync for a config and then accessing or syncing the
config manually via the command line tool, the server would abort at
the time when the auto-sync was originally scheduled.
* D-Bus server: accept WBXML with charset in incoming connections
A user reported via email that the Nokia 515 sends
'application/vnd.syncml+wbxml; charset=UTF-8' as type of its messages
this tripped up the syncevo-http-server, leading to:
[ERROR] syncevo-dbus-server: /org/syncevolution/Server: message type 'application/vnd.syncml+wbxml; charset=UT
We need to strip the '; charset=UTF-8' suffix also when checking for
WBXML.
* packaging: compatible with EDS up to and including 3.10
The packages now contain three versions of syncecal.so:
- one for EDS < 3.6
- one for EDS >= 3.6 < 3.10
- one for EDS >= 3.10 with the libecal-1.2 soname patched
The third flavor became necessary because EDS 3.10 accidentally
changed the soname. The API and ABI actually is the same.
Package meta-data was fixed to reflect the extended range of
compatible EDS libraries, so syncevolution-evolution can be installed
again with recent EDS.
* packaging: update syncevolution-kde dependencies
kdebase-runtime became kde-runtime in Debian Wheezy. Accept both
as prerequisite of syncevolution-kde to allow installation on
newer distros without pulling in the transitional kdebase-runtime
package.
* packaging: fix rpm (FDO #73347)
After installing the syncevolution.org rpm on OpenSUSE,
SyncEvolution was not starting because its shared libraries were not
found unless "ldconfig" was called manually. Now the package does
that automatically.
* packaging: fix description
The syncevolution-bundle description of both rpm and deb
packagesaccidentally used the same description as
syncevolution-evolution.
* test improvements, integration of cppcheck and clang's scan-build
SyncEvolution 1.3.99.6, 04.12.2013
==================================
This update focuses on SyncEvolution in IVI again. It adds support for
GENIVI Diagnostic Log and Trace (DLT) and enhances searching in the
unified address book.
The biggest change for normal Linux desktop users is enhanced support
for recent distros. Binaries on syncevolution.org now work with EDS >=
3.6 *and* < 3.6. Distros with libical1 like Ubuntu Saucy are also
supported. Automated testing was updated to cover these newer
platforms more thoroughly.
Details:
* GNOME Online Accounts: fix D-Bus problem in syncevolution.org binaries
Support was included in syncevolution.org binaries, but was not
tested and did not actually work due to some issue accessing
the D-Bus session.
* libsynthesis: partial fix batching of items
The batching of contact writes introduced with SyncEvolution
1.3.99.4 caused problems with non-SyncEvolution SyncML peers when
syncing contacts stored in EDS >= 3.6. EDS < 3.6 was not affected.
That part is fixed. However, even in SyncEvolution<->SyncEvolution
syncs another crash was found. This will require more investigation.
Clearly the feature is not ready yet for general sync, so for now
it is disabled by default and only enabled in the simpler PBAP
sync.
* libsynthesis: avoid redundant (and sometimes slow) getaddrbyname() (FDO #70771)
The network lookup of the hostname can be slow (10 second delay when
not connected) and shouldn't be necessary anyway, so disable it.
* PIM Manager: case-insensitive and transliterated search (FDO #56524)
* PIM: accent-insensitive and transliterated search (FDO #56524)
Accent-insensitive search ignores accents, using the same code as in
EDS. Transliterated search ignores foreign scripts by transliterating
search term and contact properties to Latin first. That one is using
ICU directly in the same way as EDS, but doesn't use the EDS
ETransliterator class to avoid extra string copying.
This commit changes the default behavior such that searching is by
default most permissive (case- and accent-insensitive, does
transliteration). Flags exist to restore more restrictive matching.
* PIM: relax phone number matching
Previously, the current default country was used to turn phone numbers
without an explicit country code into full E164 numbers, which then
had to match the search term when doing a caller ID lookup.
This was inconsistent with EDS, where a weaker
EQUALS_NATIONAL_PHONE_NUMBER was done. The difference is that a
comparison between a number with country code matches one without if
the national number of the same, regardless of the current default
country. This is better because it reduces the influence of the hard
to guess default country on matching.
Another advantage of this change is the lower memory consumption and
faster comparison, because strings are now stored in 4 + 8 byte
numbers instead of strings of varying length.
* PIM: fix incorrect write into pim-manager.ini (FDO #70772)
Removing a peer accidentally wrote the updated list of active
address books into the "sort" property of pim-manager.ini, which
then prevented starting the PIM Manager.
* PIM: ignore broken sort order in config (FDO #70772)
Failure to set the sort order from pim-manager.ini should not
prevent the startup of the PIM Manager because the client cannot
really diagnose and fix the problem. It is better to try again with
the default sort order.
* PIM: adapt to locale changes at runtime (FDO #66618)
Listen to signals from localed D-Bus system service and update all
internal state which depends on the current locale. This state includes:
- pre-computed data in all loaded contacts
- filtering (for example, case sensitivity is locale dependent)
- the sort order
This feature can be controlled by setting the SYNCEVOLUTION_LOCALED
env variable:
- "session" - use a localed instance on the D-Bus session bus instead
of the system instance. This was originally meant for
testing, but might also be useful for per-user setting changes.
- "none" - disables the feature
* PIM: fix sync.py + multiple peers
Due to overwriting a variable, configuring multiple different
peers did not work.
* D-Bus server: support DLT (FDO #66769)
Diagnostic Log and Trace (DLT) manages a sequence of log messages,
with remote controllable level of detail. SyncEvolution optionally
(can be chosen at compile time and again at runtime) uses DLT
instead of its own syncevolution-log.html files. See README-DLT.rst
for more information.
To use the feature, configure SyncEvolution with
"--enable-dbus-server=--dlt --no-syslog"
* EDS: enhanced compatibility mode
SyncEvolution compiled for EDS < 3.6 can now also load EDS backends
compiled for EDS >= 3.6. The packaging for syncevolution.org uses
that to bundle EDS backends compiled on different distros in the
same package.
* EDS: SYNCEVOLUTION_EBOOK_QUERY env variable
Setting the SYNCEVOLUTION_EBOOK_QUERY env variable to a valid EBook
query string limits the results to contacts matching that
query. Useful only in combination with --print-items or
--export. Only implemented for EDS >= 3.6.
* EDS: fix compile problem with boost and EDS > 3.36
This fixes the following problem, seen with Boost 1.53.0 on altlinux
when compiling for EDS >= 3.6:
/usr/include/boost/smart_ptr/shared_ptr.hpp: In instantiation of 'typename boost::detail::sp_array_access<T>::type boost::shared_ptr<T>::operator[](std::ptrdiff_t) const [with T = char*; typename boost::detail::sp_array_access<T>::type = void; std::ptrdiff_t = long int]':
src/backends/evolution/EvolutionSyncSource.cpp:163:38: required from here
/usr/include/boost/smart_ptr/shared_ptr.hpp:663:22: error: return-statement with a value, in function returning 'void' [-fpermissive]
make[2]: *** [src/backends/evolution/src_backends_evolution_syncecal_la-EvolutionSyncSource.lo]
* PBAP: add support for obexd 0.48
obexd 0.48 is almost the same as obexd 0.47, except that it dropped
the SetFilter and SetFormat methods in favor of passing a Bluex 5-style
filter parameter to PullAll.
SyncEvolution now supports 4, in words, four different obexd
APIs. Sigh.
This feature was originally announced for SyncEvolution 1.3.99.5,
but not actually included in the code yet.
SyncEvolution 1.3.99.5, 01.10.2013
==================================
SyncEvolution now supports Google CalDAV/CardDAV with OAuth2
authentication. These are the open protocol that Google currently
supports and thus the recommended way of syncing with Google,
replacing ActiveSync and SyncML (both no longer available to all
Google customers).
Support for Google CardDAV is new. Because of a vCard encoding issue
on the server side, spaces in long notes may get removed. Like
Evolution, SyncEvolution does not yet support some of the advanced
features of the server, in particular custom labels for phone numbers,
emails and addresses. Likewise, some client properties are not
supported by the server: CALURI, CATEGORIES, FBURL, GEO and ROLE are
not supported. Of ORG, only the first two components are supported.
Currently, properties not supported by one side get lost in a full
roundtrip sync.
SyncEvolution depends on external components for OAuth2. It can be
compiled to use gSSO [1] or GNOME Online Accounts. GNOME Online
Accounts >= 3.10 works out of the box for CalDAV and CardDAV, 3.8 only
for CardDAV (but the GNOME Online Accounts binary can be patched to
also support CalDAV, see [2]), anything older than 3.8 does not
work. Support for Ubuntu Online Accounts should not be hard to add,
but is not available yet [3].
[1] https://01.org/gsso and
http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backends/signon/README
[2] http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backends/goa/README
[3] http://thread.gmane.org/gmane.comp.mobile.syncevolution/4353/focus=4490
Details:
* GTK UI: fixed two crashes - running a sync with no service selected
and a 64 bit pointer problem recently discovered by Tino Keitel when
compiling the Debian package with -fPIE.
* password handling: fix usage of GNOME Keyring and KWallet (FDO #66110)
When clients like the GTK sync-ui stored a password, it was always
stored as plain text in the config.ini file by the
syncevo-dbus-server. The necessary code for redirecting the password
storage in a keyring (GNOME or KWallet) simply wasn't called in that
case.
The command line tool, even when using the D-Bus server to run the
operation, had the necessary code active and thus was not affected.
Now all SyncEvolution components use the same default: use safe
password storage if either GNOME Keyring or KWallet were enabled
during compilation, don't use it if not.
Fixing this revealed other problems, like not being able to store
certain passwords that lacked the necessary lookup criteria (like
syncURL and/or username). To address this, the lookup criteria where
extended and a new check was added to avoid accidentally removing
other passwords. As a result, it may be possible that SyncEvolution
no longer finds passwords that were stored with older versions of
SyncEvolution. In such a case the passwords must be set again.
* GNOME: clean up keyring access and require libgnome-keyring >= 2.20
The updated error messages now always include information about the
password and libgnome-keyring error texts.
A workaround is used for the "Error communicating with
gnome-keyring-daemon" problem that started to appear fairly
frequently in the automated testing once the keyring was actually
used. The problem shows up with some additional debug messages:
Gkr: received an invalid, unencryptable, or non-utf8 secret
Gkr: call to daemon returned an invalid response: (null).(null)()
It seems that sometimes setting up a session with GNOME keyring
fails such that all further communication leads to decoding problem.
There is an internal method to reset the session, but it cannot be
called directly. As a workaround, fake the death of the GNOME
keyring daemon and thus trigger a reconnect when retrying the GNOME
keyring access. This is done by sending a D-Bus message, which will
also affect other clients of GNOME keyring, but hopefully without
user-visible effects.
* config: enhanced password handling
It is possible to configure a plain username/password combination
once in SyncEvolution and then use references to it in other
configurations, instead of having to set (and update) the
credentials in different places. This is useful in particular with
WebDAV, where credentials had to be repeated several times (target
config, in each database when used as part of SyncML) or when using
a service which requires several configs (Google via SyncML and
CalDAV).
To use this, create a sync config for a normal peer or a dedicated
config just for the credentials, with "username/password/syncURL"
set. The "syncURL" must be set to something identifying the peer if
GNOME Keyring is used for the password storage.
Then set "username", "databaseUser" and "proxyUser" properties to
"id:<name of config with credentials>" and all read and write access
to those properties will be redirected by SyncEvolution into that
other configuration. This even works in the GTK UI.
For user names which contain colons, the new "user:<user name>" format
must be used. Strings without colons are assumed to be normal user
names, so most old configurations should continue to work.
* signon: new backend using libgsignond-glib + libaccounts-glib
The code works with gSSO (https://01.org/gsso). With some tweaks to
the configure check and some ifdefs it probably could be made to work
with Ubuntu Online Accounts.
The code depends on an account accessible via libaccounts-glib which
has a provider and and (optionally) services enabled for that
provider. It is not necessary that the account already has a signon
identity ID, the backend will create that for the provider (and thus
shared between all services) if necessary.
Therefore it is possible to use the ag-tool to create and enable the
account and services. Provider and service templates are in the next
commit.
* WebDAV: support OAuth2
If given an authentication configuration which can handle OAuth2,
then OAuth2 is used instead of plain username/password
authentication.
* WebDAV: support Google CardDAV, break Yahoo
Google CardDAV has one peculiarity: it renames new contacts during PUT without
returning the new path to the client. See also
http://lists.calconnect.org/pipermail/caldeveloper-l/2013-July/000524.html
SyncEvolution already had a workaround for that (PROPGET on old path, extract
new path from response) which happened to work. This workaround was originally
added for Yahoo, which sometimes merges contacts into existing ones. In
contrast to Yahoo, Google really seems to create new items.
Without some server specific hacks, the client cannot tell what happened.
Because Google is currently supported and Yahoo is not, let's change the
hard-coded behavior to "renamed items are new".
* WebDAV: started testing with owndrive.com = OwnCloud
* GOA: get OAuth2 tokens out of GNOME Online Accounts
"username = goa:..." selects an account in GOA and retrieves the
OAuth2 token from that.
The implementation uses the GOA D-Bus API directly, because our C++
D-Bus bindings are easier to use and this avoids an additional library
dependency.
* PIM: fix UID usage in sync.py example
Using the underscore in the UID has been wrong all along, it only
happened to work because UID sanity checking was missing. After adding
it, the example broke.
Now simply remove the colon. It makes the UID less readable, but it
doesn't have to be, and ensures that file names and database names
contain the UID as-is.
* PIM: if busy, don't shut down
While there are sessions pending or active, the server should not shut down.
It did that while executing a long-running PIM Manager SyncPeer() operations,
by default after 10 minutes.
This was not a problem elsewhere because other operations are associated with
a client, whose presence also prevents shutdowns. Perhaps PIM Manager should
also track the caller and treat it like a client.
* PBAP: do not end Bluez5 transfer prematurely
A transfer was marked as finished prematurely when encountering the
"active" Status value, which can happen for longer transfers.
* updated tests
Upgrading from releases <= 1.3.99.4:
If the value of "username/databaseUser/proxyUser" contains a colon,
the "user:" prefix must be added to the value, to continue treating it
like a plain user name and not some reference to an unknown identity
provider (like "id:", "goa:", "signon:", etc.).
The lookup of passwords in GNOME Keyring was updated slightly in
1.3.99.5. It may be necessary to set passwords anew if the old one is
no longer found.
Upgrading from release 1.2.x:
The sync format of existing configurations for Mobical (aka Everdroid)
must be updated manually, because the server has encoding problems when
using vCard 3.0 (now the default for Evolution contacts):
syncevolution --configure \
syncFormat=text/x-vcard \
mobical addressbook
The Funambol template explicitly enables usage of the
"refresh-from-server" sync mode to avoid getting throttled with 417
'retry later' errors. The same must be added to existing configs
manually:
syncevolution --configure \
enableRefreshSync=TRUE \
funambol
Upgrading from releases before 1.2:
Old configurations can still be read. But writing, as it happens
during a sync, must migrate the configuration first. Releases >= 1.2
automatically migrates configurations. The old configurations
will still be available (see "syncevolution --print-configs") but must
be renamed manually to use them again under their original names with
older SyncEvolution releases.
SyncEvolution 1.3.99.4, 12.07.2013
==================================
The focus of this development snapshot is enhanced performance of
syncing. With EDS, contacts get added, updated or loaded with batch
operations, which led to 4x runtime improvements when importing PBAP
address book for the first time. Removing unnecessary work from any
following PBAP sync led to more than a 6x improvement.
These improvements also benefit non-PBAP syncing and could in theory
work with any SyncML peer. In practice, batching of items is currently
limited to SyncEvolution as peer.
The PBAP backend itself was rewritten such that data gets transferred
from a phone in parallel to processing the already transferred
data. The effect is that on a sufficiently fast system, a sync takes
about the same time as downloading all contacts. To get the text-only
part of the contacts even faster, PBAP syncing can be done such that
it first syncs the text-only parts (without removing existing photos),
then in a second round adds or modifies photos. The PIM Manager uses
this incremental mode by default, in the command line it can be chose
with the SYNCEVOLUTION_PBAP_SYNC env variable.
The HTTP server became better at handling message resends when the
server is slow with processing a message. The server is able to keep a
sync session alive while loading the initial data set by sending
acknowledgement replies before the client times out.
Guido Günther provided some patches addressing problems when compiling
SyncEvolution for Maemo.
Details:
* sync: less verbose output, shorter runtime
For each incoming change, one INFO line with "received x[/out of y]"
was printed, immediately followed by another line with total counts
"added x, updated y, removed z". For each outgoing change, a "sent
x[/out of y]" was printed.
In addition, these changes were forwarded to the D-Bus server where a
"percent complete" was calculated and broadcasted to clients. All of
that caused a very high overhead for every single change, even if the
actual logging was off. The syncevo-dbus-server was constantly
consuming CPU time during a sync when it should have been mostly idle.
To avoid this overhead, the updated received/sent numbers that come
from the Synthesis engine are now cached and only processed when done
with a SyncML message or some other event happens (whatever happens
first).
To keep the implementation simple, the "added x, updated y, removed z"
information is ignored completely and no longer appears in the output.
* HTTP server: handle message resends
If a client gave up waiting for the server's response and resent its message
while the server was still processing the message, syncing failed with
"protocol error: already processing a message" raised by the
syncevo-dbus-server because it wasn't prepared to handle that situation.
The right place to handle this is inside the syncevo-http-server, because it
depends on the protocol (HTTP in this case) whether resending is valid or
not. It handles that now by tracking the message that is currently in
processing and matching it against each new message. If it matches, the new
request replaces the obsolete one without sending the message again to
syncevo-dbus-server. When syncevo-dbus-server replies to the old message, the
reply is used to finish the newer request.
* PBAP: incremental sync (FDO #59551)
Depending on the SYNCEVOLUTION_PBAP_SYNC env variable, syncing reads
all properties as configured ("all"), excludes photos ("text") or
first text, then all ("incremental").
When excluding photos, only known properties get requested. This
avoids issues with phones which reject the request when enabling
properties via the bit flags. This also helps with
"databaseFormat=^PHOTO".
* PIM: use incremental sync for PBAP by default (FDO #59551)
When doing a PBAP sync, PIM manager asks the D-Bus sync helper to set
its SYNCEVOLUTION_PBAP_SYNC to "incremental". If the env variable
is already set, it does not get overwritten, which allows overriding
this default.
* PIM: set debug level in peer configs via env variable
Typically the peer configs get created from scratch, in particular
when testing with testpim.py. In that case the log level cannot be set
in advance and doing it via the D-Bus API is also not supported.
Therefore, for debugging, use SYNCEVOLUTION_LOGLEVEL=<level> to create
peers with a specific log level.
* PIM: include pim-manager-api.txt in source distro (FDO #62516)
The text file must be listed explicitly to be included by "make dist".
* PIM: "full name" -> "fullname" fix in documentation (FDO #62515)
Make the documentation match the code. A single word without
space makes more sense, so let's go with what the code already
used.
* PIM: enhanced searching (search part of FDO #64177)
Search terms now also include 'is/contains/begins-with/ends-with'
and they can be combined with 'and' and 'or', also recursively.
* PIM: Pinyin sorting for zh languages (part of FDO #64173)
Full interleaving of Pinyin transliterations of Chinese names with
Western names can be done by doing an explicit Pinyin transliteration
as part of computing the sort keys.
This is done using ICU's Transliteration("Han-Latin"), which we have
to call directly because boost::locale does not expose that API.
We hard-code this behavior for all "zh" languages (as identified by
boost::locale), because by default, ICU would sort Pinyin separately
from Western names when using the "pinyin" collation.
* PIM: new return value for SyncPeer(), new SyncProgress signal (FDO #63417)
The SyncPeer() result is derived from the sync statistics. To have
them available, the "sync done" signal must include the SyncReport.
Start and end of a sync could already be detected; "modified" signals
while a sync runs depends on a new signal inside the SyncContext when
switching from one cycle to the next and at the end of the last one.
* PIM: allow removal of data together with database removal (part of FDO #64835)
There is a difference in EDS between removing the database definition
from the ESourceRegistry (which makes the data unaccessible via EDS)
and removing the actual database. EDS itself only removes the definition
and leaves the data around to be garbage-collected eventually. This is
not what we want for the PIM Manager API; the API makes a stronger
guarantee that data is really gone.
Fixed by introducing a new mode flag for the deleteDatabase() method
and deleting the directory of the source directly in the EDS backend,
if requested by the caller.
The syncevolution command line tool will use the default mode and thus
keep the data around, while the PIM Manager forces the removal of
data.
* EDS: create new databases by cloning the builtin ones (FDO #64176)
Instead of hard-coding a specific "Backend Summary Setup" in
SyncEvolution, copy the config of the system database. That way
special flags (like the desired "Backend Summary Setup" for local
address books) can be set on a system-wide basis and without having to
modify or configure SyncEvolution.
Because EDS has no APIs to clone an ESource or turn a .source file
into a new ESource, SyncEvolution has to resort to manipulating and
creating the keyfile directly.
* EDS contacts: update PHOTO+GEO during slow sync, avoid rewriting PHOTO file
If PHOTO and/or GEO were the only modified properties during a slow
sync, the updated item was not written into local storage because
they were marked as compare="never" = "not relevant".
For PHOTO this was intentional in the sample config, with the
rationale that local storages often don't store the data exactly as
requested. When that happens, comparing the data would lead to
unnecessary writes. But EDS and probably all other local SyncEvolution
storages (KDE, file) store the photo exactly as requested, so not
considering changes had the undesirable effect of not always writing
new photo data.
For GEO, ignoring it was accidental.
* EDS contacts: avoid unnecessary DB writes during slow sync
Traditionally, contacts were modified shortly before writing into EDS
to match with Evolution expectations (must have N, only one CELL TEL,
VOICE flag must be set). During a slow sync, the engine compare the
modified contacts with the unmodified, incoming one. This led to
mismatches and/or merge operations which end up not changing anything
in the DB because the only difference would be removed again before
writing.
* EDS contacts: read-ahead cache
Performance is improved by requesting multiple contacts at once and
overlapping reading with processing. On a fast system (SSD, CPU fast
enough to not be the limiting factor), testpim.py's testSync takes 8
seconds for a "match" sync where 1000 contacts get loaded and compared
against the same set of contacts. Read-ahead with only 1 contact per
query speeds that up to 6.7s due to overlapping IO and
processing. Read-ahead with the default 50 contacts per query takes
5.5s. It does not get much faster with larger queries.
* command line: execute --export and --print-items while the source is still reading
Instead of reading all item IDs, then iterating over them, process
each new ID as soon as it is available. With sources that support
incremental reading (only the PBAP source at the moment) that provides
output sooner and is a bit more memory efficient.
* WebDAV: avoid segfault during collection lookup
Avoid referencing pathProps->second when the set of paths that
PROPFINDs returns is empty. Apparently this can happen in combination
with Calypso.
* engine: prevent timeouts in HTTP server mode
HTTP SyncML clients give up after a certain timeout (SyncEvolution
after RetryDuration = 5 minutes by default, Nokia e51 after 15
minutes) when the server fails to respond.
This can happen with SyncEvolution as server when it uses a slow
storage with many items, for example via WebDAV. In the case of slow
session startup, multithreading is now used to run the storage
initializing in parallel to sending regular "keep-alive" SyncML
replies to the client.
By default, these replies are sent every 2 minutes. This can be
configured with another extensions of the SyncMLVersion property:
SyncMLVersion = REQUESTMAXTIME=5m
Other modes do not use multithreading by default, but it can be
enabled by setting REQUESTMAXTIME explicitly. It can be disabled
by setting the time to zero.
The new feature depends on a libsynthesis with multithreading enabled
and glib >= 2.32.0, which is necessary to make SyncEvolution itself
thread-safe. With an older glib, multithreading is disabled, but can
be enabled as a stop-gap measure by setting REQUESTMAXTIME explicitly.
* Various testing and stability enhancements. SyncEvolution had to
be made thread-safe for the HTTP timeout prevention.
SyncEvolution 1.3.99.3, 06.03.2013
==================================
Another development snapshot, with a particular focus on enhancing
(and in some cases, fixing) searching in the PIM Manager. The PIM
Manager in this snapshot depends on folks 0.9.x and thus gee
0.8. Support for Bluez 5 was added. The PIM Manager API was extended
by addding CreatePeer and ReplaceSearch. The previous methods,
SetPeer and RefineSearch, are still supported.
Some issues in CalDAV, WebDAV and SyncML were fixed.
Graham R. Cobb contributed several patches for enhancing ActiveSync
support and making it work with Exchange 2010.
Details:
* PIM Manager: add ReplaceSearch, always allow it
The new ReplaceSearch is more flexible than RefineSearch. It can
handle both tightening the search and relaxing it. The downside of it
is the more expensive implementation (must check all contacts again,
then find minimal set of change signals to update view).
Previously, a search which had no filter set at all at the begining
could not be refined. This limitation of the implementation gets
removed by always using a FilteredView, even if the initial filter is
empty.
* PIM Manager: introduce CreateConfig()
That SetPeer() allows modifying and creating a config leads to race
conditions when multiple clients want to create a config. The new
CreateConfig() avoids that by atomically checking that a config does
not exist yet and creating it.
SetPeer() is still available for backwards compatibility. It continues
to be used for modifying an existing config in TestContacts.testSync
to check the effect of the logging settings.
* PIM Manager: fix double entries in filtered search with limit
Stressing the FilteredView by using it in tests originally written
for the FullView showed that the filling up a view may have
used data while it was inconsistent internally, leading to
contacts being present multiple times.
* PIM Manager and sync: support location = GEO property (FDO #60373)
Exposed as "location" -> (lat, long) in the D-Bus bindings.
Reading, writing and updating are supported.
* PIM Manager: support groups = CATEGORIES (FDO #60380)
Allow reading and writing of groups (folks terminology), aka
CATEGORIES in vCard.
* PIM Manager: intelligent phone search in EDS (part of FDO #59571)
If phone number search is enabled in EDS, then the direct search in
EDS now uses the more accurate
E_BOOK_QUERY_EQUALS_NATIONAL_PHONE_NUMBER comparison, with the E164
formatted caller ID as value to compare against. This gives
semantically correct results. The previous solution (now the
fallback) had to use substring searches, which did not match if the
contact's phone number was not formatted according to E164 and
which may have matched the wrong contacts if the trailing numbers
are the same.
* PIM Manager : use pre-computed normalized phone numbers from EDS (part of FDO #59571)
When available, the pre-computed E164 number from EDS will be used
instead of doing one libphonebook parser run for each telephone
number while reading. Benchmarking showed that this parsing was the
number one hotspot, so this is a considerable improvement.
* PIM Manager: fix error messages
Ensure and check that no unnecessary ERROR messages are printed.
libfolks was used slightly incorrectly, leading to several
harmless error messages (glib asserts). libphonenumber printed
its error messages to stdout.
* PIM Manager: fix memory leaks during writing of contacts
Constructing the GValues created additional references instead
of taking over ownership as intended.
* D-Bus server: fix read-after-free bug when using syslog
openlog() expects the string to remain valid. Must ensure that in
LoggerSyslog by making a copy. Found with valgrind.
* PIM Manager: make implementation of some of the D-Bus methods thread-safe
The goal is to make it easier to extend syncevo-dbus-server
with other IPC mechanisms, which then can call the native C++
code directly.
That code was not prepared to handle calls in threads other than the
main one. Now this is checked when entering the methods and work is
shifted to the main thread if necessary. In the meantime the calling
thread waits for completion.
* PIM Manager: check responsiveness (part of FDO #60851)
Enhanced the testActive test so that it can detect when the D-Bus
server stops responding for too long. One major reason for that was
event processing in folks, which got improved as part of
https://bugzilla.gnome.org/show_bug.cgi?id=694385
* PIM Manager: adapt to gee 0.8
Changed the code to compile with gee 0.8, as used by folks 0.9.x.
Older versions of folks are no longer supported.
* PBAP: support Bluez 5
The new Bluez 5 API is the third supported API for doing PBAP
transfers. It gets checked first, then the PBAB backend falls back to
new-style obexd (file based, similar to Bluez 5, but not quite the
same) and finally old-style obexd (data transfer via D-Bus).
In contrast to previous APIs, Bluez 5 does not report the reason for a
failed PBAP transfer. SyncEvolution then throws a generic "transfer
failed" error with "reason unknown" as message.
* command line: recover from slow sync with new sync modes
The error message for an unexpected slow sync still mentioned
the old and obsolete "refresh-from-client/server" sync modes.
Better mention "refresh-from-local/remote".
* CalDAV: more workarounds for Google CalDAV + unique IDs
Google became even more strict about checking REV. Tests which
reused a UID after deleting the original item started to fail sometime
since middle of December 2012.
* CalDAV: work around Google server regression (undeclared namespace prefix in XML)
Google CalDAV for a while (December 2012 till January 2013) sent
invalid XML back when asked to include CardDAV properties in a
PROPFIND. This got rejected in the XML parser, which prevents
syncing calendar data:
Neon error code 1: XML parse error at line 55: undeclared namespace prefix
In the meantime Google fixed the issue in response to a bug report
via email. But the workaround, only asking for the properties which
are really needed, still makes sense and thus is kept.
* WebDAV: don't send Basic Auth via http proactively (FDO #57248)
Sending basic authentication headers via http is insecure. Only do
it proactively when the connection is encrypted and thus protects
the information or when the server explicitly asks for it.
* Nokia: always add TYPE=INTERNET to EMAIL (FDO #61784)
Without the explicit TYPE=INTERNET, email addresses sent to a Nokia
e51 were not shown by the phone and even got lost eventually (when
syncing back).
This commit ensures that the type is set for all emails sent to any
Nokia phone, because there may be other phones which need it and
phones which don't, shouldn't mind. This was spot-checked with a N97
mini, which works fine with and without the INTERNET type.
This behavior can be disabled again for specific Nokia phones by
adding a remote rule which sets the addInternetEmail session variable
to FALSE again.
Non-Nokia phones can enable the feature in a similar way, by setting
the variable to TRUE.
* SyncML: config option for broken peers
Some peers have problems with meta data (CtCap, old Nokia phones)
and the sync mode extensions required for advertising the restart
capability (Oracle Beehive). The default in SyncEvolution is to
advertise the capability, so manual configuration is necessary when
working with a peer that fails in that mode.
Because the problem occurs when SyncEvolution contacts the peers
before it gets the device information from the peer, dynamic rules
based on the peer identifiers cannot be used. Instead the local config
must already disable these extra features in advance.
The "SyncMLVersion" property gets extended for this. Instead of just
"SyncMLVersion = 1.0" (as before) it now becomes possible to say
"SyncMLVersion = 1.0, noctcap, norestart".
"noctcap" disables sending CtCap. "norestart" disables the sync mode
extensions and thus doing multiple sync cycles in the same session
(used between SyncEvolution instances in some cases to get client and
server into sync in one session).
Both keywords are case-insensitive. There's no error checking for
typos, so beware!
The "SyncMLVersion" property was chosen because it was already in use
for configuring SyncML compatibility aspects and adding a new property
would have been harder.
* ActiveSync: added support for specifying folder names
Previously, the database field was interpreted as a Collection ID. This adds
logic to allow the database to be interpreted as a folder path. The logic is:
1) If the database is an empty string, pass it through (this is the most
common case as it is interpreted as "use the default folder for the
source type").
2) If the database matches a Collection ID, use the ID (this is the same as
the previous behaviour).
3) If the database matches a folder path name, with an optional leading "/",
use the Collection ID for the matching folder.
4) Otherwise, force a FolderSync to get the latest folder changes from the
server and repeat steps 2 and 3
5) If still no match, throw an error.
* ActiveSync: support for listing databases
Now --print-databases scans folders on the ActiveSync server and
shows suitable folders for the ActiveSync backends instead of the
previous, hard-coded help text.
Invoking --print-databases can be used as a workaround for
"SyncFolder error: Invalid synchronization key" errors. A better
solution would be to do that automatically, but there was no time
to implement that. See FDO #61869 and "[SyncEvolution] Activesync server losing state"
http://thread.gmane.org/gmane.comp.mobile.syncevolution/4295
* command line: show backend error when listing databases fails
The command line swallowed errors thrown by the backend while listing
databases. Instead it just showed "<backend name>: backend failed". The goal
was to not distract users who accidentally access a non-functional backend.
But the result is that operations like --configure or --print-databases could
fail without giving the user any hint about the root cause of the issue.
Now the error explanation in all its gory details is included.
For example, not having activesyncd running leads to:
INFO] eas_contact: backend failed: fetching folder list:
GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name
org.meego.activesyncd was not provided by any .service files
And running activesyncd without the necessary gconf keys shows up as:
[INFO] eas_contact: backend failed: fetching folder list:
GDBus.Error:org.meego.activesyncd.Error.AccountNotFound: Failed to find
account [syncevolution@lists.intel.com]
* Minor memory leak fix when using GDBus GIO: GDBusMethodInfo
Also depends on a glib fix, see https://bugzilla.gnome.org/show_bug.cgi?id=695376
* build fixes
Avoid -lrt in make dependencies. Add missing pcre libs to
syncevo-dbus-server. sqlite backend needs "#include <stdio.h>"
(patch from Mario Kicherer).
SyncEvolution 1.3.99.2, 13.12.2012
==================================
Another development snapshot. Includes all fixes that went into 1.3.2
and several improvements to the PIM Manager. Documentation was updated
and extended considerably. The pim-manager-api.txt now describes the
abstract API while src/dbus/server/pim/README explains how
SyncEvolution implements it.
Details:
* PIM Manager searches for a caller ID ('phone' search) in EDS
directly while folks still starts up. No unification is done of
these results. Intermediate results are replaced by the final ones
from folks once those are ready.
* PIM Manager: allow configuration of session directories (part of FDO #55921)
Useful for moving the session directories to a temporary file system.
They are essentially just useful for debugging when used as part of
PIM Manager.
- "logdir" - a directory in which directories are created with
debug information about sync session
- "maxsessions" - number of sessions that are allowed to exist
after a sync (>= 0): 0 is special and means unlimited,
1 for just the latest, etc.;
old sessions are pruned heuristically (for example,
keep sessions where something changed instead of
some where nothing changed), so there is no hard
guarantee that the last n sessions are present.
* PIM Manager: write less data to disk (part of FDO #55921)
Avoid writing config file changes to disk by enabling a new
"ephemeral" mode for syncing via the PIM Manager. In this mode,
config file changes are not flushed resp. discarded directly.
This prevents writing to .ini files in ~/.config.
The "synthesis" binfile client files are still written, but they get
redirected into the session directory, which can (and should) be set
to a temp file system and get deleted again quickly.
Data dumps are turned off now in the configs created by the PIM
Manager.
* syncevo-dbus-server: use syslog instead of standard output by default
* syncevo-dbus-server: command line options for controlling
output and startup
-d, --duration=seconds/'unlimited' Shut down automatically
when idle for this duration (default 300 seconds)
-v, --verbosity=level Choose amount of output, 0 = no output,
1 = errors, 2 = info, 3 = debug; default is 1.
-o, --stdout Enable printing to stdout (result of operations)
and stderr (errors/info/debug).
-s, --no-syslog Disable printing to syslog.
-p, --start-pim Activate the PIM Manager (= unified address book)
immediately.
* PIM Manager: store set of active address books persistently (FDO #56334)
Together with storing the sort order persistently, this allows
restarting the daemon and have it create the same unified address book
again.
* PIM Manager: remove colon from valid peer UID character set (FDO #56436)
Using the UID as part of file names gets more problematic when
allowing colons. Remove that character from the API and enforce
the format in the source code.
* PIM Manager API: introduce contact ID and use it for reading
This makes it easier for a client to fully polulate its view with
contact data. Previously it could happen that due to concurrent
changes in the server, a client was returned data for the same
contact multiple times. A client had to detect that and re-issue
read requests.
* PIM Manager API: optional ViewAgent.Quiescent() (FDO #56428)
The callback is guaranteed to be invoked once when a search has
finished sending its initial results, and not sooner. This makes it
possible to check whether the current data contains some contact or
not.
* PIM Manager: limit number of search results (FDO #56142)
A 'limit' search term with a number as parameter (formatted as string)
can be added to a 'phone' or 'any-contains' search term to truncate the
search results after a certain number of contacts. Example:
Search([['any-contains', 'Joe'], ['limit', '10']])
=> return the first 10 Joes.
As with any other search, the resulting view will be updated if
contact data changes.
The limit must not be changed in a RefineSearch(). A 'limit' term may
(but doesn't have to) be given. If it is given, its value must match
the value set when creating the search. This limitation simplifies the
implementation and its testing. The limitation could be removed if
there is sufficient demand.
* PIM Manager: fix refining a search
Due to not mapping the local index in the view to the parent's index,
refining only worked in views where parent and child had the same
index for the contacts in the search view.
* PIM Manager: fix starting when done via search
When the unified address book (= FullView) was not running yet at the
time when a client wanted to search it, the unified address book was
not started and thus the search never returned results.
* PIM Manager: fix writing contact, support photo and notes
folks and EDS do not support writing properties in parallel
(https://bugzilla.gnome.org/show_bug.cgi?id=652659). Must serialize
setting of modified properties.
* PIM Manager: fix incorrect contact removal signals in filtered view
The filtered view did not check whether a parent's removed contact was
really part of the view before sending a removal signal for it.
* D-Bus: missing out parameters in D-Bus introspection XML (FDO #57292)
The problem was in the C++ D-Bus binding. If the method that gets bound
to D-Bus returns a value, that value was ignored in the signature:
int foo() => no out parameter
It works when the method was declared as having a retval:
void foo (int &result) => integer out parameter
This problem existed for both the libdbus and the GIO D-Bus
bindings. In SyncEvolution it affected methods like GetVersions().
* PIM Manager performance: pre-compute normalized telephone numbers
Looking up by phone number spends most of its cycles in normalizing of
the phone numbers in the unified address book. Instead of doing that
work over and over again during the search, do it once while loading.
Looking up a phone number only once does not gain from this change, it
even gets slower (more memory intensive, less cache locality). Only
searching multiple times becomes faster.
Ultimately it would be best to store the normalized strings together
with the telephone number inside EDS when the contact gets
created. Work on that is in progress.
* PIM Manager: improve performance of FullView sorting
This fixes the hotspot during populating the FullView content: moving
contacts around required copying IndividualData and thus copying
complex C++ structs and strings. Storing pointers and moving those
avoids that, with no lack of convenience thanks to boost::ptr_vector.
Reordering also becomes faster, because the intermediate copy only
needs to be of the pointers instead of the full content.
* PIM Manager example: add benchmarking
The new "checkpoints" split up the whole script run into pieces which
are timed separately, with duration printed to stdout. In addition,
tools like "perf" can be started for the duration of one phase.
* SyncML: workarounds for broken peers
Some peers have problems with meta data (CtCap, old Nokia phones) and
the sync mode extensions required for advertising the restart
capability (Oracle Beehive).
Because the problem occurs when SyncEvolution contacts the peers
before it gets the device information from the peer, dynamic rules
based on the peer identifiers cannot be used. Instead the local config
must already disable these extra features in advance.
The "SyncMLVersion" property gets extended for this. Instead of just
"SyncMLVersion = 1.0" (as before) it now becomes possible to say
"SyncMLVersion = 1.0, noctcap, norestart".
"noctcap" disables sending CtCap. "norestart" disables the sync mode
extensions and thus doing multiple sync cycles in the same session
(used between SyncEvolution instances in some cases to get client and
server into sync in one session).
Both keywords are case-insensitive. There's no error checking for
typos, so beware!
The "SyncMLVersion" property was chosen because it was already in use
for configuring SyncML compatibility aspects and adding a new property
would have been harder.
* EDS: fix creating databases
--create-database was broken in combination with the final code in EDS
3.6 because it passed NULL for the UID to e_source_new_with_uid(),
which is considered an error by the implementation of that
method. Must use e_source_new() if we don't have a UID.
* fixed some memory leaks, extended tests to cover new features and bugs
SyncEvolution 1.3.99.1, 25.10.2012
==================================
Development snapshot. The PIM Manager API implementation is fully
implemented, see src/dbus/server/pim/README for an introduction. The
PBAP backend together with a new one-way caching sync mode provides an
efficient way of keeping a local database in sync via Bluetooth with a
phone which does not implement SyncML.
Other changes:
* workarounds for warnings from g++ 4.5
* engine: : local cache sync mode
This patch introduces support for true one-way syncing ("caching"):
the local datastore is meant to be an exact copy of the data on the
remote side. The assumption is that no modifications are ever made
locally outside of syncing. This is different from one-way sync modes,
which allows local changes and only temporarily disables sending them
to the remote side.
Another goal of the new mode is to avoid data writes as much as
possible.
This new mode only works on the server side of a sync, where the
engine has enough control over the data flow. Setting "sync" to:
- "local-cache-incremental" will do an incremental sync (if possible)
or a slow sync (otherwise). This is usually the right mode to use,
and thus has "local-cache" as alias.
- "local-cache-slow" will always do a slow sync. Useful for
debugging or after (accidentally) making changes on the local side.
An incremental sync will ignore such changes because they are not
meant to happen, aren't checked for to improve performance and
thus will leave client and server out-of-sync!
Both modes are recorded in the sync report of the local side. The
target side is the client and records the normal "two-way" or "slow"
sync modes.
With the current SyncEvolution contact field list, first, middle and
last name are used to find matches for contacts. For events, tasks
and memos, time, summary and description are used.
* HTTP proxy: useProxy=0 overrides http_* env variables
Previously, if http_proxy was set, a proxy was used even if
explicitly disabled. This prevented disabling the use of a proxy
which only made sense in some cases, like accessing something
that runs locally. Explicitly telling SyncEvolution to ignore
http_proxy is necessary because it doesn't support no_proxy.
* WebDAV: auto-discovery fix
With Google Contact + CardDAV the auto-discovery failed after
finding the default address book, without reporting that result.
* command line: implement --create/remove-database
Creating a database is only possible with a chosen name. The UID is
chosen automatically by the storage. Only implemented in the EDS
backend.
* file backend: sub-second mod time stamps
Change tracking in the file backend used to be based on the
modification time in seconds. When running many syncs quickly (as in
testing), that can lead to changes not being detected when they happen
within a second. Now the file backend also includes the sub-second part of the
modification time stamp, if available.
This change is relevant when upgrading SyncEvolution: most of the
items will be considered "updated" once during the first sync after
the upgrade (or a downgrade) because the revision strings get
calculated differently.
* D-Bus server: avoid progress outside of 0-100% range
For example in the new TestLocalCache.testItemDelete100, the
percentage value in the ProgressChanged signal become larger
than 100 and then revert to 100 at the end of the sync.
Seems the underlying calculation is faulty or simply inaccurate.
This is not fixed. Instead the result is just clipped to the valid
range.
* code cleanup + improvements in testing
SyncEvolution 1.3.1 -> 1.3.2, 05.11.2012
========================================
Minor (or major, if you depend on auto syncing) bug fix
release. Details:
* auto sync: only synced once (FDO #56667)
A successful sync was incorrectly treated like a sync with a permanent
failure, which prevents further automatic syncing.
* auto sync: notifications were not translated
The code which enabled localization of messages created by the
D-Bus server was incomplete. Localization was only enabled
accidentally through KDE if the KDE platform modules was enabled
during compilation and installed.
* HTTP Proxy: useProxy=0 overrides http_* env variables
Previously, if http_proxy was set, a proxy was used even if
explicitly disabled. This prevented disabling the use of a proxy
which only made sense in some cases, like accessing something
that runs locally. Explicitly telling SyncEvolution to ignore
http_proxy is necessary because it doesn't support no_proxy.
* minor changes in testing and autotools files (missing Boost search path
in gdbus* libs might have caused compile problems)
SyncEvolution 1.3 -> 1.3.1, 05.10.2012
======================================
Minor bug fix release. Details:
* command line: fix output of --import for directories
The running count at the start of the line (#0, #1, ...) was
not incremented when reading individual files from a directory.
* Funambol: work around PHOTO TYPE=image/jpeg, part II
The final version of the fix hadn't made it into the source code.
* vCalendar 1.0 + tasks: DUE date could be shifted by a day (FDO #55238)
Because of incomplete support for time conversion, the due date
could get mixed up when phone and PC were set to something other
than UTC. Reported and fixed by Peter Jan.
* syncevolution.org: syncevolution-evolution had incorrect dependencies
Installation on older Linux distros was not possible because the ebook/ecal
package dependencies were named incorrectly, for example libebook-1.2-10
instead of libebook1.2-10. Only more recent packages have the extra
dash, for example libebook-1.2-12. Reported by Mariusz Sokolowski.
* GTK-3 UI: fixed compile problem
The GTK-3 UI depends on a class from gio-unix-2.0 and failed to
compile on Fedora Core 16 because the configure checks for that lib
(and thus the compiler flags) were missing. Reported by Peter
Robinson.
* Curl: allow using it in the D-Bus server
In the past, using curl as HTTP transport in the syncevo-dbus-server
was prevented, leading to "unsupported transport type is specified in
the configuration". The reason was that using curl would block the
server and make it unresponsive on D-Bus.
This reason has gone away, because now the HTTP traffic happens in a
separate process. Thus now it is allowed to use curl in the
syncevo-dbus-server.
* fix for false negative in syncevo-dbus-server testing
SyncEvolution 1.2.2 -> 1.3, 10.09.2012
======================================
After almost three months of public beta testing the next major
version of SyncEvolution is ready for release. The pre-releases did
have the desired effect of flushing out bugs not found by nightly
testing alone. Thanks everyone for packaging, downloading and testing
them!
New features are KDE/Akonadi and ActiveSync support, not only in the
source code but also in syncevolution.org binaries. ActiveSync is the
recommended way of synchronizing contacts with Google:
https://syncevolution.org/wiki/google-contacts-activesync
The D-Bus server and local sync were rewritten considerably, to make
the code cleaner and more robust. The CalDAV backend now also supports
tasks and memos. CalDAV and CardDAV can be used in combination with a
SyncML peer ("bridging SyncML and WebDAV"), thus allowing a device
which only supports SyncML to talk to a WebDAV service without any
intermediate storage.
1.3 contains bug fixes that were not backported to 1.2.x, so upgrading
is recommended. For example, SyncEvolution 1.3 is required for
Evolution 3.4, otherwise photos are not exported properly. Support for
Evolution >= 3.6 is in the source code, but not in syncevolution.org
binaries. Further workarounds for recent changes in Google CalDAV and
Funambol One Media were added.
Details:
* ActiveSync: updated to work with latest activesyncd and Google, package binaries
Syncing Google contacts was added to the nightly testing. Syncing
contacts and events with Exchange 2012 was already working. Setup
instructions and known issues are described here:
https://syncevolution.org/wiki/google-contacts-activesync
* phone sync: delete<->delete conflict + phone calendar+todo sync (BMC #23744)
When deleting an item on phone and locally, the next sync failed with
ERROR messages about "object not found". This has several reasons:
- libsynthesis super data store attempts to read items
which may or may not exist (triggers ERROR message)
- it checks for 404 but Evolution backends only return a generic
database error (causes sync to fail)
* phone sync: get phone vendor and model from Device ID profile (BMC #736)
In the past we have relied on the user-modifiable device name to be
the fingerprint for matching a phone to a template which is unreliable.
This release changes this in the cases where the phone supports the
Device ID profile (DIP). If support for DIP is detected, then we
extract the vendor and product ids and attempt to associate them
with a product and vendor name by using a newly added lookup table.
This lookup table has to be maintained manually and depends on
contributions by users to cover more devices. See
http://blixtra.org/blog/2011/09/22/syncevolution-needs-you-or-at-least-your-bluetooth-phones/
* vCalendar 1.0: fixed recurring all-day event support
vCalendar 1.0 cannot represent all-day events. The workarounds for
mapping iCalendar 2.0 all-day events into vCalendar 1.0 was
incomplete, leading to effects like shifting EXDATEs and end
times.
* Funambol: ignore UID
Funambol's OneMedia sends UID, but not RECURRENCE-ID. That becomes a
problem when multiple events of the same event series are added to a
backend which follows the iCalendar 2.0 standard (CalDAV, EDS, KDE),
because these events all look like the master event, and there can be
only one of those.
SyncEvolution now strips the UID from all events coming from any
Funambol server (regardless of the version). If a future Funambol
server release adds support for both UID and RECURRENCE-ID, then
SyncEvolution will have to be updated to take advantage of the
improved server. Because the RECURRENCE-ID is also getting
stripped (despite not being set at the moment), SyncEvolution should
continue to work as it does now even if the server changes.
It would have been nice to limit this workaround to affected Funambol
server versions, but an inquiry on the Funambol mailing list didn't
get a reply, therefore SyncEvolution is playing it safe and assumes
that all future Funambol releases will have the same problem.
* Funambol: work around PHOTO TYPE=image/jpeg
A combination of Funambol Android and Funambol server recently led to
the Funambol server sending PHOTO data with TYPE=image/jpeg. This is
invalid and caused EDS to reject the photo (Vladimir Elisseev,
"[SyncEvolution] issues with syncing photos").
Work around the problem by only keeping the part of the type after the
last slash, if there is any. For image/jpeg and similar types that
leads to the desired value and does not affect valid values, because
those do not contain a slash
(http://www.iana.org/assignments/media-types/image/index.html).
* Funambol: avoid slow syncs in refresh from server
libsynthesis has traditionally implemented "refresh-from-server" as
"delete local data" plus "slow" sync. This is more compatible, because
some servers (like Google) do not support "refresh-from-server".
But it has the downside that the server cannot know that the client
won't send any data, and Funambol's OneMedia now only allows one slow
sync before blocking the next one for a certain period of time. This is
done to prevent excessive resource usage by badly behaving clients.
To accomodate both kinds of servers, the new "enableRefreshSync"
sync property can be set set to explicitly allow the usage of
the "refresh-from-server" sync mode. It's off by default. The Funambol
template has it turned on, existing configs must be updated manually
(see upgrading comments below).
* Mobical (aka Everdroid): stopped testing memo syncing
Memos used to work, but now only trigger an unspecific 400 error
on the server side.
* GTK-UI: accept service config with a username again (BMC#23106)
Suppressing configs with empty username had undesired side effects:
modifying configs for direct syncing with a device incorrectly
triggered the same error message, without any means of entering
a username. The faulty check was removed without replacement.
* GTK-UI: added GTK 3 version of UI
When GTK 3 is found during compilation, a GTK 3 version of the
UI is built. The source code of both is different to avoid
excessive use of ifdefs. At the moment, both versions offer
the same features. In the long run, the GTK 3 version will
replace the GTK 2 version.
* command line: added refresh/one-way-from-local/remote (BMC #23537)
The -from-client/server sync modes are confusing because the direction
of the data exchange depends on which side acts as SyncML server or
client.
This release introduces new modes which use -from-local/remote
instead. The statistics and messages also use these variants
now. The old modes are still understood, but are declared as "not
recommended" in the documentation.
* command line: config and source names are optional (BMC #23783)
The need to add "foo" and "bar" pseudo config and source names to the
command line even when all parameters for the operation where
explicitly specified on the command line was confusing.
Now it is possible to invoke item operations without the config and
source name. Names which refer to non-existent configs are still
accepted, as in previous releases. Typos are handled better by
producing a detailed error report which includes (as applicable):
- config doesn't exist
- source doesn't exist or not selected
- backend property not set
Because luids used to be positional arguments after <config> and
<source>, a new --luids keyword is necessary to indicate that the
ensuing parameters are luids and not <config> and <source>.
* command line: introduced --print-databases, supported for CalDAV/CardDAV
Listing databases is now a dedicated operation, instead of being done
whenever syncevolution was invoked without parameters.
Advantages:
- can be combined with property assignments for backends
which do not work without that additional information, for example
CalDAV/CardDAV:
syncevolution --print-databases \
backend=[caldav|carddav] \
syncURL=... \
username=... \
password=...
- can be done for configured sources
* command line: use both stdout and stderr
Traditionally, the "syncevolution" command line tool mixed its
INFO/ERROR/DEBUG messages into the normal stdout. This has the major
drawback that error messages get lost during operations like
syncevolution --export - @default addressbook | grep "John Doe"
Now anything which is not the expected result of the operation is
always sent to stderr. Obviously this includes ERROR messages. INFO
and DEBUG are harder to decide. Because they usually convey meta
information about the running operation, they are also sent to
stderr. The output of running a sync goes to both stdout (summary)
and stderr (progress).
* command line: allow setting empty properties
Due to the way how properties were handled internally, it wasn't
possible to explicitly set a property to its default value. Instead
the property was unset. For example, explicitly setting database= was
not possible.
This is necessary for client-test and ActiveSync, because client-test
needs to know that the testing is expected to run with the default
databases (something which normally is avoided by overwriting empty
database properties).
Now the "is set" state is tracked explicitly in the config storage and
command line property APIs. Unsetting a property via the command line
could be implemented with an explicit command line option, but is not
supported at the moment.
* command line: fixed --export <file name>
When exporting items into a file, the delimiter between items
was missing.
* command line + local sync: fixed erroneous "Comparison impossible" output.
"Comparison impossible" was incorrectly printed after a successful
comparison on the target side of local sync.
* local sync: fix timeout with local sync with libdbus
When using libdbus instead of GIO D-Bus (as done by syncevolution.org
binaries and SyncEvolution on Maemo), local sync may have aborted
after 25 seconds when syncing many items with a D-Bus timeout error:
[ERROR] sending message to child failed: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible ca
Reported by Toke Høiland-Jørgensen for Harmattan. Somehow not encountered
elsewhere.
* synccompare: shorter data dump of PHOTO
A full comparison of the base64 PHOTO data can be very long.
Now some key characteristics of the PHOTO data (number of
characters in base64 encoding, number of bytes in decoded
data, md5sum of decoded data) are printed instead.
That way, unintended changes of the data (different encoding,
different content) should still be found while testing and
added/removed photos are nicely visible in synccompare diffs.
* synccompare: fixed output for byte-identical duplicates
If database dumps contained byte-identical duplicates, they
were treated as a single item on the left side of a comparison.
This caused erroneous "added" entries on the right side.
* secure password storage: usage of GNOME Keyring vs. KDE KWallet configurable
Automatically detecting KDE users is not possible at the
moment. Instead KDE users have to manually set the new "keyring"
global config property to "KDE" (case insensitive) if the
SyncEvolution installation supports both, because GNOME Keyring is the
default to avoid surprises for traditional users. If only KWallet
support is enabled, then this is not necessary.
"GNOME" and "true/false/1/0/yes/no" can also be set. This has the
advantage that keyring usage can be enabled permanently for the
command line in --daemon=no mode; normally keyrings are not used in
that mode because accessing them can bring up UI dialogs.
It also becomes possible to disable keyring usage in syncevo-dbus-server,
something which couldn't be done before.
The --keyring command line option is still supported, as an alias for
"[--sync-property] keyring=<value>". The default value for --keyring
is true, to match the traditional behavior. In contrast to other sync
properties, setting "keyring" does not require an explicit --run
parameter. Again this is done to mirror traditional usage.
* config: improved 'maxlogdirs' documentation
The old explanation made it sound like nothing would get deleted by
default ("If set, ..."). That's not correct, by default only 10
sessions are kept.
Also explain the behavior of deleting intermediate sessions first.
* Evolution: always create databases (PTCOM-113)
Always try to create address book or calendar database, because even
if there is a source there's no guarantee that the actual database
was created already; the original logic for only setting this when
explicitly requesting a new database therefore failed in some cases.
This problem affected users who had never created anything locally
and wanted to use SyncEvolution to migrate their data. Now that
works without having to create dummy entries first.
* Evolution contacts: changed default sync format to vCard 3.0
vCard 3.0 is the better default because it has saner encoding
rules and defines more properties, thus avoiding the need for
non-standard extensions. However, Mobical has problems with
the new default. See upgrade instructions below.
* Evolution: added support for EDS 3.5.x
When compiled against EDS 3.5.x or later, SyncEvolution now uses
the backend code originally written for the EClient API introduced
in EDS 3.2. That code was changed so that it works with the new
include file rules and ESourceRegistry in EDS 3.5.x. Support
for using the EClient API with EDS 3.4 was removed because maintaining
three different flavors of the EDS backend code would be too much
work and not gain much (just the possibility to test the EDSClient
code with 3.4).
At the moment, this is a compile time choice made automatically
by configure. syncevolution.org binaries are compiled against
an older EDS and thus do not work with EDS 3.5.x or later.
EDS 3.5.x handles authentication itself, using a standard system
prompt if necessary. SyncEvolution can no longer provide the password,
and thus the "databaseUser/Password" options have no effect when using
EDS 3.5.x.
* D-Bus server: fixed HTTP presence for recent libdbus
Testing with libdbus 1.6.0 on Debian Testing failed because the lib
changed some behavior: instead of looking up the owner of a certain
bus name immediately, it now does that when invoking a
method. Therefore the check for "have connection" in SyncEvolution
was too simplistic and missed the fact that both were not usable,
causing the server to assume that HTTP was down while in reality it
should have assumed it to be up. This prevented auto-syncing and
manually clicking "Sync" in the GTK UI.
* D-Bus server: made notification verbosity configurable with "notifyLevel"
The new "notifyLevel" per-peer configuration option allows users to
control how many desktop notifications the D-Bus server produces while
executing an automatic sync:
0 - suppress all notifications
1 - show only errors
2 - show information about changes and errors (in practice currently the same as level 3)
3 - show all notifications, including starting a sync (default)
* WebDAV: fixed data corruption issue when uploading item with long UID
In some cases data with a very long UID wasn't handled correctly,
causing the out-going data to be malformed and probably causing a
rejection by the server. The root cause is incorrect string handling.
In releases before 1.2.99.1, only the --import operation of contacts
into CardDAV were affected. In 1.2.99.1, the same code also got used
for calendar items and then could also affect syncing.
* CalDAV: updated Google workarounds
Google started sending empty items (VCALENDAR with no VEVENT inside)
which cannot be removed. SyncEvolution 1.3 ignores such items.
The workaround for a 404 from Google Calendar for a GET (sending a
REPORT request matching the item's UID) was broken: first, processing
the result ended up calling the unset responseEnd boost function
pointer, which caused the request to fail. Second, getting multiple
items wasn't handled (data from all items concatenated together was
used).
That can happen in the somewhat unlike case that some items have a UID
which is a complete superset of the requested UID - not realistic in
real life, but happens during testing.
* Google Calendar: updated URL redirect handling
Google Calendar sometimes returns redirect requests to human-readable
web sites (an "unavailable" page, a login form). This is of course
bogus when the client is an automated CalDAV client.
The "unavailable.html" case was already handled. Made it a bit more
flexible to also catch possible variations of it (additional
parameters, https instead of http).
Added the https://accounts.google.com/ServiceLogin case. Not sure
whether retrying will help in that case, but there's not much else
that SyncEvolution can do.
* WebDAV: bridge with SyncML
Now a peer accessed via SyncML can read/write data stored in a
CalDAV/CardDAV server directly. This can be used to connect a device
which only supports SyncML to a CalDAV/CardDAV server, or sync data
between a SyncML server and a CalDAV/CardDAV server. See "CalDAV and
CardDAV" in the README for details.
* WebDAV: improved --configure
Added INFO output about checking sources. This helps with WebDAV when
the server cannot be contacted (dead, misconfigured) because otherwise
there would be no indication at all why the --configure operation
seems to hang.
Here is some example output, including aborting:
$ syncevolution --configure --template webdav \
syncURL=http://192.168.1.100:9000/ \
username=foo password=bar retryDuration=2s \
target-config@webdav-temp
[INFO] creating configuration target-config@webdav-temp
[INFO] addressbook: looking for databases...
[INFO] addressbook: no database to synchronize
[INFO] calendar: looking for databases...
[INFO] calendar: no database to synchronize
[INFO] memo: looking for databases...
[INFO] memo: no database to synchronize
[INFO] todo: looking for databases...
[INFO] todo: no database to synchronize
It timed out fairly quickly here because of the retryDuration=2s. That
also gets placed in the resulting config, which is probably not desired.
Aborting the operation is now supported:
$ syncevolution --configure \
--template webdav \
syncURL=http://192.168.1.100:9000/ \
username=foo password=bar \
target-config@webdav-temp
[INFO] creating configuration target-config@webdav-temp
[INFO] addressbook: looking for databases...
^C[INFO] Asking to suspend...
[INFO] Press CTRL-C again quickly (within 2s) to stop immediately (can cause problems in the future!)
^C[INFO] Aborting immediately ...
[ERROR] error code from SyncEvolution aborted on behalf of user (local, status 20017): aborting as requested by user
It would be good to make the CTRL-C handling code aware that it can
abort immediately instead of doing the intermediate "asking to suspend"
step, which only makes sense for sync sessions.
* WebDAV: support tasks and memos (BMC #24893)
The new backend property values "CalDAVTodo" and "CalDAVJournal"
select tasks resp. memos stored in a CalDAV collection. "CalDAV"
continues to select events.
Events, tasks and journals can be mixed in the same resource (=
URL). However, this is less efficient than storing them separately.
A good CalDAV server allows filtering items by type, and SyncEvolution
uses that. However, it was found that Radicale 0.7 ignores this
filtering, which could have led to data loss (SyncEvolution asks for
all VTODOs in preparation for a "delete all items" operation in a
"CalDAVTodo" source, gets also VJOURNALs, then deletes them).
Therefore SyncEvolution plays it safe and downloads the VTODO and
VJOURNAL data to double-check that it is working on the right items.
This causes additional traffic for well-behaving servers; currently
it cannot be turned off.
Tasks are exchanged as vCalendar 1.0 or iCalendar 2.0 VJOURNAL.
Memos are exchanged as VTODO or plain text. The logic for storing
incoming plain text is slightly different compared to the way how
the EDS memo backend did it: instead of copying the first line
from the text into the summary, it is now moved. In other words,
the first line gets stripped. The change is primarily technically
motivated; both approaches have pros and cons.
* WebDAV: improved Radicale support
Radicale > 0.7 will return status 200 for delete requests;
is now treated like 204 by SyncEvolution. 412 'Preconditiona Failed'
when asking to delete an already removed item is treated like
the more common 404 'not found'. Same with 410 'gone' instead
of 404 when trying to read a non-existent item.
* CalDAV/CardDAV sync: improved target side output
Added a "target side of local sync ready" INFO message to introduce
the output which has the target context in the [INFO] tag. The sync report
from the target side now has the target context embedded in brackets
after the "Changes applied during synchronization" header, to avoid
ambiguities.
Sometimes the backend has to resend requests because of temporary
issues. If the problem turned out to be permanent, there was a long
period of time, retryDuration=5 minutes to be precice, in which no
visible progress happened.
Now SyncEvolution's WebDAV backend will print a message like this
before going to sleep until it is time to retry:
[INFO @googlecalendar] operation temporarily (?) failed, going to retry in 5.0s before giving up in 18.4s: PROPFIND: Neon error code 1: 401 Unauthorized
The uncertainty comes from several factors. In this example, the 401
might indicate a permanent problem (wrong credentials), or it could be
Google reporting a temporary authorization problem which is (probably)
meant to slow down the client while it asks the user to re-enter the
password. SyncEvolution only asks for passwords once, so it tries
again with the same password if it was successful with it in the
past. Otherwise it gives up immediately.
Another dubious example are name server lookup errors. They can be
permanent (wrong host name) or temporary (name server
down). SyncEvolution errs on the side of retrying, to avoid
interrupting an operation which still has a chance to continue.
Output from the target side of a local sync was passed through stderr
redirection as chunks of text to the frontends. This had several
drawbacks:
- forwarding only happened when the local sync parent was processing
the output redirection, which (due to limitations of the implementation)
only happens when it needs to print something itself
- debug messages were not forwarded
- message boundaries might have been lost
In particular the new INFO messages are relevant while the sync runs
and need to be shown immediately.
* WebDAV: --status for WebDAV source aborted
The command line --status operation did not complete when applied to a
CalDAV/CardDAV source. Instead it aborted because the operation took a
code path where the backend was not fully initialized.
* file backend: more flexible sync support for memos
The databaseFormat=text/calendar for memos did not support
synchronizing as plain text. When using the new
databaseFormat=text/calendar+plain, vCalendar/iCalendar/plain text
are all valid sync formats; the storage is iCalendar 2.0
VJOURNAL in all cases.
* WebDAV: avoid potential crash during database detection
When a server responds to a PROPFIND for a path with results for some
other path, then SyncEvolution crashed during the search for the
default calendar or address book because of a bug in the code which
was meant to handle that kind of response. Apparently Yahoo Calendar
did that. Now seen again in combination with Radicale 0.6.4.
In general, the code was made more robust to cope with bugs in
Radicale 0.6.4. Later Radicale versions fixed these issues and also
worked with SyncEvolution 1.2.2 without client-side workarounds.
* WebDAV: better path normalization
"syncURL" and "database" properties had to end in a trailing slash,
otherwise items were not found (404 errors). Now the necessary slash
is added automatically.
* Curl transport: support SSLServerCertificates=<path>
When the setting refers to a directory, then CURLOPT_CAINFO doesn't
work (must be a file). Check this and use CURLOPT_CAPATH instead.
Caveat: there are some comments in the API documentation about "NSS
enabled libcurl" which supports a directory in
CURLOPT_CAINFO. Hopefully providing an explicit path in CURLOPT_CAPATH
also works in that configuration.
* code cleanup + rewrite: syncing done in separate process
syncevo-dbus-server now runs syncing in a separate process. Local
sync also uses a second helper process. This makes the D-Bus server
more responsive via D-Bus (no more blocking operations) and
minimizes the effect of bugs in code involved with syncing
(backends, system libraries, etc.).
In the long term this restructuring will also allow more advanced
features, like monitoring local or remote storage for changes.
* SyncEvolution <-> SyncEvolution sync: multiple cycles per session
SyncML only allows one send/receive cycle per session. There are cases
(for example, client side merges data that a dumber server failed to
match correctly) where client and server are still out of sync at
the end of a cycle. When SyncEvolution syncs with another SyncEvolution
instance (locally or remotely), both sides detect that the peer
can continue syncing in the same session and start over automatically
when needed. Previously the user had to start another sync session manually.
To the user this is shown as "number of cycles" in a sync session
in the sync report. "Restart" is the process of entering a new cycle.
The cycles are also visible in the command line output as multiple
INFO lines:
[INFO] eds_contact: starting first time sync from client (peer is server)
[INFO] creating complete data backup of source eds_contact before sync (enabled with dumpData and needed for prin
Local data changes to be applied during synchronization:
*** eds_contact ***
no changes
[INFO] eds_contact: sent 1/1
[INFO] eds_contact: started
[INFO] eds_contact: first time sync done successfully
[INFO] eds_contact: starting normal sync from client (peer is server) <===
[INFO] eds_contact: started <===
[INFO] eds_contact: normal sync done successfully <===
[INFO] creating complete data backup after sync (enabled with dumpData and needed for printChanges)
Synchronization successful.
Changes applied during synchronization:
+---------------|-----------------------|-----------------------|-CON-+
| | LOCAL | REMOTE | FLI |
| Source | NEW | MOD | DEL | ERR | NEW | MOD | DEL | ERR | CTS |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| eds_contact | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
| refresh-from-local, 2 cycles, 0 KB sent by client, 0 KB received |
^^^^^^^^
| item(s) in database backup: 1 before sync, 1 after it |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| start Tue Feb 7 17:07:49 2012, duration 0:03min |
| synchronization completed successfully |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
* SyncEvolution <-> SyncEvolution sync: negotiate UID support via SyncCap (BMC #22783)
The semantic of UID/RECURRENCE-ID in calendar data is now tracked
per data store involved in a sync. If full iCalendar 2.0 semantic
(= IDs are globally unique) is guaranteed, then pairs are found
based on these IDs. Otherwise pairs must be found by looking at
item attributes.
Previously a hack was used to detect this kind of support (any kind
of SyncEvolution instance was assumed to support it, although some
backends do not).
* engine: add DTSTAMP+LAST-MODIFIED before writing calendar items
When writing calendar items into a backend storage as iCalendar 2.0 or
vCalendar 1.0, they should have DTSTAMP and LAST-MODIFIED values. DTSTAMP
is expected by some CalDAV servers (like Apple). LAST-MODIFIED is usually
added by the storage, but not always.
In the text/plain -> syncevolution -> text/calendar -> Radicale -> EDS
-> syncevolution chain the LAST-MODIFIED was not added by Radicale, which caused
problems for change tracking in an EDS-based SyncEvolution.
Also necessary when importing from a phone using vCalendar without
DTSTAMP directly into CalDAV.
* autotools: ensure that link lines are complete
As mentioned by Tino Keitel on the mailing list, some libs and
executables were only implicitly linked against libraries that they
called directly. This happened to work by chance because these libraries
ended up in the running executable anyway, due to indirect loading.
Now there is a "make installcheck" test for this kind of defect
and the makefiles were updated to avoid it.
One exception is libsmltk, which depends on the caller providing
SySync logging support.
* syncevolution.org packages: fixed D-Bus server autostart in .deb and .rpm packages
syncevo-dbus-server wasn't started automatically as part of a user
session because /etc/xdg/autostart/syncevo-dbus-server.desktop wasn't
included in the packages. This broke auto syncing after a session
restart (required manually starting SyncEvolution).
* syncevolution.org packages: support KDE
The traditional "syncevolution-evolution" package was
replaced with "syncevolution-bundle". A meta "syncevolution-evolution"
package depends on it, to support seamless updates for users who have
"syncevolution-evolution" installed.
Binary dependencies of the main .deb are ignored for backends
because loading them is optional. The new "syncevolution-kde"
package has the right dependencies for KDE/Akonadi, while
"syncevolution-evolution" mostly just lists standard libs
if the "EDS compatibility" mode is used, where libebook/libecal
are loaded dynamically.
Platform specific code (GNOME keyring, KDE wallet) was moved into
loadable, optional modules, to allow installation of the SyncEvolution
bundle without forcing the installation of unused system components.
* D-Bus: use GIO D-Bus instead of libdbus if available
When compiling from source, the more modern GIO D-Bus is used instead
of libdbus if available and recent enough (>= 2.30). syncevolution.org
binaries still use libdbus, to stay compatible with older Linux
distros.
* several minor bug fixes
syncevo-dbus-server now runs under valgrind in the nightly testing,
plus several more test scenarios were added. This helped to find
and fix various minor memory handling issues.
* developers: backend API changes
beginSync/endSync() (aka m_startDataRead/m_endDataWrite) may now be
called multiple times per SyncSource instance life cycle. SyncSources
derived from TrackingSyncSource should work without changes. Use the
Client::Source::*::testChangesMultiCycles test to check whether your
backend supports this correctly.
Reading and deleting must throw a 404 status exception when an item
is not found. The Client::Source::*::*404 tests cover this.
The special semantic of the former RegisterSyncSource::InactiveSource
(invalid pointer of value 1) caused bugs, like using it in
--print-databases (=> segfault) or not being able to store the result
of a createSource() directly in a smart pointer (=> potential leak in
SyncSource::createSource()).
Obviously a bad idea to start with. Replaced with a
RegisterSyncSource::InactiveSource() method which creates a real,
inactive SyncSource instance which can and must be deleted by the
caller.
This is a SyncSource API change for backend developers. Instead of
RegisterSyncSource::InactiveSource, return
RegisterSyncSource::InactiveSource(). Comparisons against
RegisterSyncSource::InactiveSource needs to be replaced with a call
to the new SyncSource::isInactive().
Long-running backend calls are encouraged to check for events on the
main glib context (either in a loop or with
g_main_context_iteration(NULL)) and abort when
SuspendFlags::getSuspendFlags().getState() returns
SuspendFlags::ABORT.
Implementing the improved local sync output required extending the
D-Bus API. The Server.LogOutput signal now has an additional
"process name" parameter. Normally it is empty. For messages
originating from the target side, it carries that extra target
context string.
This D-Bus API change is backward compatible. Older clients can still
subscribe to and decode the LogOutput messages, they'll simply ignore
the extra parameter. Newer clients expecting that extra parameter
won't work with an older D-Bus daemon: they'll fail to decode the
D-Bus message.
* packagers:
libgdbussyncevo is now installed as a normal library in /usr/lib,
even though SyncEvolution is the only user.
pcrecpp is now a new hard dependency.
Upgrading from release 1.2.x:
The sync format of existing configurations for Mobical (aka Everdroid)
must be updated manually, because the server has encoding problems when
using vCard 3.0 (now the default for Evolution contacts):
syncevolution --configure \
syncFormat=text/x-vcard \
mobical addressbook
The Funambol template explicitly enables usage of the
"refresh-from-server" sync mode to avoid getting throttled with 417
'retry later' errors. The same must be added to existing configs
manually:
syncevolution --configure \
enableRefreshSync=TRUE \
funambol
Upgrading from releases before 1.2:
Old configurations can still be read. But writing, as it happens
during a sync, must migrate the configuration first. Releases >= 1.2
automatically migrates configurations. The old configurations
will still be available (see "syncevolution --print-configs") but must
be renamed manually to use them again under their original names with
older SyncEvolution releases.
SyncEvolution 1.2.99.3 -> 1.3, 10.09.2012
=========================================
Final SyncEvolution 1.3 release. The pre-releases did have the desired
effect of flushing out bugs not found by nightly testing alone. Thanks
everyone for packaging, downloading and testing them! Time to get it
out officially as the next stable release.
* D-Bus server + GIO D-Bus: shutdown fix
When compiled against GIO D-Bus (not the case in syncevolution.org
binaries), the syncevo-dbus-server occasionally shut down before
sending out all pending D-Bus messages. Showed up only in nightly
testing.
* D-Bus server + GIO D-Bus: fix auto-activation (Debian bug #599247)
When syncevo-dbus-server was started on demand by the D-Bus daemon,
then it registered itself with the daemon before it was ready to
serve requests. Only happened in combination with GIO D-Bus and
thus was not a problem before 1.2.99.x.
One user-visible effect was that the GTK UI did not select the default
service when it was started for the first time, because it could not
retrieve that information from syncevo-dbus-server.
* local sync: fix timeout with local sync with libdbus
When using libdbus instead of GIO D-Bus (as done by syncevolution.org
binaries and SyncEvolution on Maemo), local sync may have aborted
after 25 seconds when syncing many items with a D-Bus timeout error:
[ERROR] sending message to child failed: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible ca
Reported by Toke Høiland-Jørgensen for Harmattan. Somehow not encountered
elsewhere.
* KDE: check for D-Bus to avoid crash in KApplication (BMC #25596)
Some unnamed version of KDE crashes in KApplication when invoked
without a D-Bus session. The reporter ran into this when compiling
from source, because the SyncEvolution binary is invoked as part of
the build process, which ran outside of a D-Bus session.
Avoid the crash by checking for a D-Bus session bus before instantiating
KApplication. Instantiating KApplication was added for KWallet support.
Without D-Bus, KWallet does not work either, therefore throw an explicit
error when the lack of D-Bus is detected.
* Funambol: work around PHOTO TYPE=image/jpeg
A combination of Funambol Android and Funambol server recently led to
the Funambol server sending PHOTO data with TYPE=image/jpeg. This is
invalid and caused EDS to reject the photo (Vladimir Elisseev,
"[SyncEvolution] issues with syncing photos").
Work around the problem by only keeping the part of the type after the
last slash, if there is any. For image/jpeg and similar types that
leads to the desired value and does not affect valid values, because
those do not contain a slash
(http://www.iana.org/assignments/media-types/image/index.html).
* syncevo-http-server: fixed printing of server debug output
Python failed to call logSyncEvoOutput() after adding the additional
'process' parameter to LogOutput because it extracts all four
parameters and then cannot pass them to logSyncEvoOutput().
Now logSyncEvoOutput() uses the new process information to instantiate
a logger with the right prefix, using 'sync' as fallback for messages
without that information (as before).
* Some minor code and test cleanup.
SyncEvolution 1.2.99.3 -> 1.2.99.4, 07.08.2012
==============================================
Another release candidate for SyncEvolution 1.3. Lesson learned:
declaring a snapshot as "final" is a good way of luring the hidden bugs
into the light. Of course, then another snapshot is needed...
Details:
* D-Bus server: fix support for autoSyncDelay > 0
Auto syncing was not getting triggered when using an autoSyncDelay > 0;
by default it is 5 minutes. Thanks to Vladimir Elisseev for reporting
this problem.
* command line: fixed --export <file name>
When exporting items into a file, the delimiter between items
was missing.
* config: improved 'maxlogdirs' documentation
The old explanation made it sound like nothing would get deleted by
default ("If set, ..."). That's not correct, by default only 10
sessions are kept.
Also explain the behavior of deleting intermediate sessions first.
* developers: fixed D-Bus interface XML
Reverted to Qt 4.x compatible annotations and changed "templateName"
to "getTemplate" to make it more obvious what the parameter does.
Only relevant for the out-of-tree Qt UI.
Fixed accidental removal of the "template" parameter in
Session.GetNamedConfig(). Was not used in practice, but has to be
correct in case that someone wants to use it.
SyncEvolution 1.2.99.2 -> 1.2.99.3, 24.07.2012
==============================================
Final release candidate for SyncEvolution 1.3 - fingers crossed,
knock on wood, etc.
ActiveSync is now available in binaries from syncevolution.org and
becomes the recommended way of synchronizing contacts with Google. EDS
3.5.x and later are supported when compiling from source;
syncevolution.org binaries continue to support only EDS up to 3.4.
Details:
* EDS: added support for EDS 3.5.x
When compiled against EDS 3.5.x or later, SyncEvolution now uses
the backend code originally written for the EClient API introduced
in EDS 3.2. That code was changed so that it works with the new
include file rules and ESourceRegistry in EDS 3.5.x. Support
for using the EClient API with EDS 3.4 was removed because maintaining
three different flavors of the EDS backend code would be too much
work and not gain much (just the possibility to test the EDSClient
code with 3.4).
At the moment, this is a compile time choice made automatically
by configure. syncevolution.org binaries are compiled against
an older EDS and thus do not work with EDS 3.5.x or later.
EDS 3.5.x handles authentication itself, using a standard system
prompt if necessary. SyncEvolution can no longer provide the password,
and thus the "databaseUser/Password" options have no effect when using
EDS 3.5.x.
* ActiveSync: updated to work with latest activesyncd and Google, package binaries
Syncing Google contacts was added to the nightly testing. Syncing
contacts and events with Exchange 2012 was already working. Setup
instructions and known issues are described here:
https://syncevolution.org/wiki/google-contacts-activesync
* local sync: don't drop data comparison output on target side
synccompare on the target side of a local sync was invoked with its
output being redirected via an unreliable socket to the local sync
parent. When the output was large, some of it might have been lost.
* local sync: fixed crash
When processing stdout from syncevo-local-child in
syncevo-dbus-helper, the LogRedirect class was invoked recursively and
tried to print the same stdout data repeatedly until the
syncevo-dbus-helper crashed due to the infinite recurssion.
* local sync: fixed helper process shutdown in case of parent failure
The helper process only detected that the parent failed when
it tried to log something while the parent had already shut down
the D-Bus connection. Even that did not work reliably and differed
between D-Bus libdbus and GIO.
Added several test cases and fixes for "process died prematurely"
error scenarios.
* Mobical (aka Everdroid): stopped testing memo syncing
Memos used to work, but now only trigger an unspecific 400 error
on the server side.
* autotools: ensure that link lines are complete
As mentioned by Tino Keitel on the mailing list, some libs and
executables were only implicitly linked against libraries that they
called directly. This happened to work by chance because these libraries
ended up in the running executable anyway, due to indirect loading.
Now there is a "make installcheck" test for this kind of defect
and the makefiles were updated to avoid it.
One exception is libsmltk, which depends on the caller providing
SySync logging support.
* D-Bus server: fixed HTTP presence for recent libdbus
Testing with libdbus 1.6.0 on Debian Testing failed because the lib
changed some behavior: instead of looking up the owner of a certain
bus name immediately, it now does that when invoking a
method. Therefore the check for "have connection" in SyncEvolution
was too simplistic and missed the fact that both were not usable,
causing the server to assume that HTTP was down while in reality it
should have assumed it to be up. This prevented auto-syncing and
manually clicking "Sync" in the GTK UI.
* syncevolution.org: declare dependencies on libical and EDS
Let the bundle .deb depend on libical if the lib was enabled during
compilation (for example, for CalDAV). This ensures that it gets
installed on systems which otherwise don't have it.
"syncevolution-evolution" is compatible (and depends on) EDS up to
and including 3.4. The package now declares that dependency and
conflicts with more recent EDS, because even if the older EDS libs
are still installed they won't work when the rest of EDS was
updated.
* CalDAV + syncevolution.org: fixed segfault without libical+libecal
When libical and libecal were not installed, trying to use the CalDAV
backend for VEVENTs segfaulted because it depends on libical and did
not check properly for it. Only affected syncevolution.org binaries.
SyncEvolution 1.2.99.1 -> 1.2.99.2, 04.07.2012
==============================================
Next step towards SyncEvolution 1.3. It adds a workaround for
Funambol's OneMedia and fixes an old bug which became more severe in
1.2.99.1. Also has some usability improvements for
CalDAV/CardDAV. Hopefully it will not take long to stabilize the code,
so test it now while it is still hot :-)
Details:
* Funambol: ignore UID
Funambol's OneMedia sends UID, but not RECURRENCE-ID. That becomes a
problem when multiple events of the same event series are added to a
backend which follows the iCalendar 2.0 standard (CalDAV, EDS, KDE),
because these events all look like the master event, and there can be
only one of those.
SyncEvolution now strips the UID from all events coming from any
Funambol server (regardless of the version). If a future Funambol
server release adds support for both UID and RECURRENCE-ID, then
SyncEvolution will have to be updated to take advantage of the
improved server. Because the RECURRENCE-ID is also getting
stripped (despite not being set at the moment), SyncEvolution should
continue to work as it does now even if the server changes.
It would have been nice to limit this workaround to affected Funambol
server versions, but an inquiry on the Funambol mailing list didn't
get a reply, therefore SyncEvolution is playing it safe and assumes
that all future Funambol releases will have the same problem.
* WebDAV: fixed data corruption issue when uploading item with long UID
In some cases data with a very long UID wasn't handled correctly,
causing the out-going data to be malformed and probably causing a
rejection by the server. The root cause is incorrect string handling.
In releases before 1.2.99.1, only the --import operation of contacts
into CardDAV were affected. In 1.2.99.1, the same code also got used
for calendar items and then could also affect syncing.
* engine: add DTSTAMP+LAST-MODIFIED before writing calendar items
When writing calendar items into a backend storage as iCalendar 2.0 or
vCalendar 1.0, they should have DTSTAMP and LAST-MODIFIED values. DTSTAMP
is expected by some CalDAV servers (like Apple). LAST-MODIFIED is usually
added by the storage, but not always.
In the text/plain -> syncevolution -> text/calendar -> Radicale -> EDS
-> syncevolution chain the LAST-MODIFIED was not added by Radicale, which caused
problems for change tracking in an EDS-based SyncEvolution.
Also necessary when importing from a phone using vCalendar without
DTSTAMP directly into CalDAV.
* Google Calendar: updated URL redirect handling
Google Calendar sometimes returns redirect requests to human-readable
web sites (an "unavailable" page, a login form). This is of course
bogus when the client is an automated CalDAV client.
The "unavailable.html" case was already handled. Made it a bit more
flexible to also catch possible variations of it (additional
parameters, https instead of http).
Added the https://accounts.google.com/ServiceLogin case. Not sure
whether retrying will help in that case, but there's not much else
that SyncEvolution can do.
* CalDAV + VJOURNAL: handle UID conflicts
When asked to insert a VJOURNAL which already existed (= same UID),
CalDAV servers respond with a 412 "Precondition failed" error. This
needs to be detected and translated into an "item needs to be merged"
result so that the engine can load the existing item, merge the data,
and then write back.
* WebDAV: --status for WebDAV source aborted
The command line --status operation did not complete when applied to a
CalDAV/CardDAV source. Instead it aborted because the operation took a
code path where the backend was not fully initialized.
* CalDAV/CardDAV sync: improved target side output
Added a "target side of local sync ready" INFO message to introduce
the output which has the target context in the [INFO] tag. The sync report
from the target side now has the target context embedded in brackets
after the "Changes applied during synchronization" header, to avoid
ambiguities.
Sometimes the backend has to resend requests because of temporary
issues. If the problem turned out to be permanent, there was a long
period of time, retryDuration=5 minutes to be precice, in which no
visible progress happened.
Now SyncEvolution's WebDAV backend will print a message like this
before going to sleep until it is time to retry:
[INFO @googlecalendar] operation temporarily (?) failed, going to retry in 5.0s before giving up in 18.4s: PROPFIND: Neon error code 1: 401 Unauthorized
The uncertainty comes from several factors. In this example, the 401
might indicate a permanent problem (wrong credentials), or it could be
Google reporting a temporary authorization problem which is (probably)
meant to slow down the client while it asks the user to re-enter the
password. SyncEvolution only asks for passwords once, so it tries
again with the same password if it was successful with it in the
past. Otherwise it gives up immediately.
Another dubious example are name server lookup errors. They can be
permanent (wrong host name) or temporary (name server
down). SyncEvolution errs on the side of retrying, to avoid
interrupting an operation which still has a chance to continue.
Output from the target side of a local sync was passed through stderr
redirection as chunks of text to the frontends. This had several
drawbacks:
- forwarding only happened when the local sync parent was processing
the output redirection, which (due to limitations of the implementation)
only happens when it needs to print something itself
- debug messages were not forwarded
- message boundaries might have been lost
In particular the new INFO messages are relevant while the sync runs
and need to be shown immediately.
* command line: fixed password + property lookup during --print-databases
--print-databases for an existing configuration did not look up
passwords stored in a keyring, causing the operation to fail for
backends like CalDAV/CardDAV where credentials are required.
Overriding source properties in that case also only worked when using
the unqualified property name ("databasePassword=foo") but not when
using the source name as prefix ("calendar/databasePassword=foo").
* Developers:
Implementing the improved local sync output required extending the
D-Bus API. The Server.LogOutput signal now has an additional
"process name" parameter. Normally it is empty. For messages
originating from the target side, it carries that extra target
context string.
This D-Bus API change is backward compatible. Older clients can still
subscribe to and decode the LogOutput messages, they'll simply ignore
the extra parameter. Newer clients expecting that extra parameter
won't work with an older D-Bus daemon: they'll fail to decode the
D-Bus message.
SyncEvolution 1.2.2 -> 1.2.99.1, 22.06.2012
===========================================
First pre-release of SyncEvolution 1.3. Contains bug fixes that were
not backported to 1.2.x, so upgrading is recommended. For example,
SyncEvolution 1.3 is required for Evolution 3.4, otherwise photos are
not exported properly. Further workarounds for recent changes in
Google CalDAV were added.
Major new features are KDE/Akonadi support in the syncevolution.org
binaries and ActiveSync support (only in the source code). The D-Bus
server and local sync were rewritten considerably, to make the code
cleaner and more robust. The CalDAV backend now also supports tasks
and memos.
Details:
* phone sync: delete<->delete conflict + phone calendar+todo sync (BMC #23744)
When deleting an item on phone and locally, the next sync failed with
ERROR messages about "object not found". This has several reasons:
- libsynthesis super data store attempts to read items
which may or may not exist (triggers ERROR message)
- it checks for 404 but Evolution backends only return a generic
database error (causes sync to fail)
* phone sync: get phone vendor and model from Device ID profile (BMC #736)
In the past we have relied on the user-modifiable device name to be
the fingerprint for matching a phone to a template which is unreliable.
This release changes this in the cases where the phone supports the
Device ID profile (DIP). If support for DIP is detected, then we
extract the vendor and product ids and attempt to associate them
with a product and vendor name by using a newly added lookup table.
This lookup table has to be maintained manually and depends on
contributions by users to cover more devices. See
http://blixtra.org/blog/2011/09/22/syncevolution-needs-you-or-at-least-your-bluetooth-phones/
* vCalendar 1.0: fixed recurring all-day event support
vCalendar 1.0 cannot represent all-day events. The workarounds for
mapping iCalendar 2.0 all-day events into vCalendar 1.0 was
incomplete, leading to effects like shifting EXDATEs and end
times.
* GTK-UI: accept service config with a username again (BMC#23106)
Suppressing configs with empty username had undesired side effects:
modifying configs for direct syncing with a device incorrectly
triggered the same error message, without any means of entering
a username. The faulty check was removed without replacement.
* GTK-UI: added GTK 3 version of UI
When GTK 3 is found during compilation, a GTK 3 version of the
UI is built. The source code of both is different to avoid
excessive use of ifdefs. At the moment, both versions offer
the same features. In the long run, the GTK 3 version will
replace the GTK 2 version.
* command line: added refresh/one-way-from-local/remote (BMC #23537)
The -from-client/server sync modes are confusing because the direction
of the data exchange depends on which side acts as SyncML server or
client.
This release introduces new modes which use -from-local/remote
instead. The statistics and messages also use these variants
now. The old modes are still understood, but are declared as "not
recommended" in the documentation.
* command line: config and source names are optional (BMC #23783)
The need to add "foo" and "bar" pseudo config and source names to the
command line even when all parameters for the operation where
explicitly specified on the command line was confusing.
Now it is possible to invoke item operations without the config and
source name. Names which refer to non-existent configs are still
accepted, as in previous releases. Typos are handled better by
producing a detailed error report which includes (as applicable):
- config doesn't exist
- source doesn't exist or not selected
- backend property not set
Because luids used to be positional arguments after <config> and
<source>, a new --luids keyword is necessary to indicate that the
ensuing parameters are luids and not <config> and <source>.
* command line: introduced --print-databases, supported for CalDAV/CardDAV
Listing databases is now a dedicated operation, instead of being done
whenever syncevolution was invoked without parameters.
Advantages:
- can be combined with property assignments for backends
which do not work without that additional information, for example
CalDAV/CardDAV:
syncevolution --print-databases \
backend=[caldav|carddav] \
syncURL=... \
username=... \
password=...
- can be done for configured sources
* command line: use both stdout and stderr
Traditionally, the "syncevolution" command line tool mixed its
INFO/ERROR/DEBUG messages into the normal stdout. This has the major
drawback that error messages get lost during operations like
syncevolution --export - @default addressbook | grep "John Doe"
Now anything which not the expected result of the operation is
always sent to stderr. Obviously this includes ERROR messages. INFO
and DEBUG are harder to decide. Because they usually convey meta
information about the running operation, they are also sent to
stderr. The output of running a sync goes to both stdout (summary)
and stderr (progress).
* command line: allow setting empty properties
Due to the way how properties were handled internally, it wasn't
possible to explicitly set a property to its default value. Instead
the property was unset. For example, explicitly setting database= was
not possible.
This is necessary for client-test and ActiveSync, because client-test
needs to know that the testing is expected to run with the default
databases (something which normally is avoided by overwriting empty
database properties).
Now the "is set" state is tracked explicitly in the config storage and
command line property APIs. Unsetting a property via the command line
could be implemented with an explicit command line option, but is not
supported at the moment.
* command line + local sync: fixed erroneous "Comparison impossible" output.
"Comparison impossible" was incorrectly printed after a successful
comparison on the target side of local sync.
* synccompare: shorter data dump of PHOTO
A full comparison of the base64 PHOTO data can be very long.
Now some key characteristics of the PHOTO data (number of
characters in base64 encoding, number of bytes in decoded
data, md5sum of decoded data) are printed instead.
That way, unintended changes of the data (different encoding,
different content) should still be found while testing and
added/removed photos are nicely visible in synccompare diffs.
* synccompare: fixed output for byte-identical duplicates
If database dumps contained byte-identical duplicates, they
were treated as a single item on the left side of a comparison.
This caused erroneous "added" entries on the right side.
* secure password storage: usage of GNOME Keyring vs. KDE KWallet configurable
Automatically detecting KDE users is not possible at the
moment. Instead KDE users have to manually set the new "keyring"
global config property to "KDE" (case insensitive) if the
SyncEvolution installation supports both, because GNOME Keyring is the
default to avoid surprises for traditional users. If only KWallet
support is enabled, then this is not necessary.
"GNOME" and "true/false/1/0/yes/no" can also be set. This has the
advantage that keyring usage can be enabled permanently for the
command line in --daemon=no mode; normally keyrings are not used in
that mode because accessing them can bring up UI dialogs.
It also becomes possible to disable keyring usage in syncevo-dbus-server,
something which couldn't be done before.
The --keyring command line option is still supported, as an alias for
"[--sync-property] keyring=<value>". The default value for --keyring
is true, to match the traditional behavior. In contrast to other sync
properties, setting "keyring" does not require an explicit --run
parameter. Again this is done to mirror traditional usage.
* Evolution: always create databases (PTCOM-113)
Always try to create address book or calendar database, because even
if there is a source there's no guarantee that the actual database
was created already; the original logic for only setting this when
explicitly requesting a new database therefore failed in some cases.
This problem affected users who had never created anything locally
and wanted to use SyncEvolution to migrate their data. Now that
works without having to create dummy entries first.
* Evolution contacts: changed default sync format to vCard 3.0
vCard 3.0 is the better default because it has saner encoding
rules and defines more properties, thus avoiding the need for
non-standard extensions. However, Mobical has problems with
the new default. See upgrade instructions below.
* D-Bus server: made notification verbosity configurable with "notifyLevel"
The new "notifyLevel" per-peer configuration option allows users to
control how many desktop notifications the D-Bus server produces while
executing an automatic sync:
0 - suppress all notifications
1 - show only errors
2 - show information about changes and errors (in practice currently the same as level 3)
3 - show all notifications, including starting a sync (default)
* CalDAV: updated Google workarounds
Google started sending empty items (VCALENDAR with no VEVENT inside)
which cannot be removed. SyncEvolution 1.3 ignores such items.
The workaround for a 404 from Google Calendar for a GET (sending a
REPORT request matching the item's UID) was broken: first, processing
the result ended up calling the unset responseEnd boost function
pointer, which caused the request to fail. Second, getting multiple
items wasn't handled (data from all items concatenated together was
used).
That can happen in the somewhat unlike case that some items have a UID
which is a complete superset of the requested UID - not realistic in
real life, but happens during testing.
* WebDAV: bridge with SyncML
Now a peer accessed via SyncML can read/write data stored in a
CalDAV/CardDAV server directly. This can be used to connect a device
which only supports SyncML to a CalDAV/CardDAV server, or sync data
between a SyncML server and a CalDAV/CardDAV server. See "CalDAV and
CardDAV" in the README for details.
* WebDAV: improved --configure
Added INFO output about checking sources. This helps with WebDAV when
the server cannot be contacted (dead, misconfigured) because otherwise
there would be no indication at all why the --configure operation
seems to hang.
Here is some example output, including aborting:
$ syncevolution --configure --template webdav \
syncURL=http://192.168.1.100:9000/ \
username=foo password=bar retryDuration=2s \
target-config@webdav-temp
[INFO] creating configuration target-config@webdav-temp
[INFO] addressbook: looking for databases...
[INFO] addressbook: no database to synchronize
[INFO] calendar: looking for databases...
[INFO] calendar: no database to synchronize
[INFO] memo: looking for databases...
[INFO] memo: no database to synchronize
[INFO] todo: looking for databases...
[INFO] todo: no database to synchronize
It timed out fairly quickly here because of the retryDuration=2s. That
also gets placed in the resulting config, which is probably not desired.
Aborting the operation is now supported:
$ syncevolution --configure \
--template webdav \
syncURL=http://192.168.1.100:9000/ \
username=foo password=bar \
target-config@webdav-temp
[INFO] creating configuration target-config@webdav-temp
[INFO] addressbook: looking for databases...
^C[INFO] Asking to suspend...
[INFO] Press CTRL-C again quickly (within 2s) to stop immediately (can cause problems in the future!)
^C[INFO] Aborting immediately ...
[ERROR] error code from SyncEvolution aborted on behalf of user (local, status 20017): aborting as requested by user
It would be good to make the CTRL-C handling code aware that it can
abort immediately instead of doing the intermediate "asking to suspend"
step, which only makes sense for sync sessions.
* WebDAV: support tasks and memos (BMC #24893)
The new backend property values "CalDAVTodo" and "CalDAVJournal"
select tasks resp. memos stored in a CalDAV collection. "CalDAV"
continues to select events.
Events, tasks and journals can be mixed in the same resource (=
URL). However, this is less efficient than storing them separately.
A good CalDAV server allows filtering items by type, and SyncEvolution
uses that. However, it was found that Radicale 0.7 ignores this
filtering, which could have led to data loss (SyncEvolution asks for
all VTODOs in preparation for a "delete all items" operation in a
"CalDAVTodo" source, gets also VJOURNALs, then deletes them).
Therefore SyncEvolution plays it safe and downloads the VTODO and
VJOURNAL data to double-check that it is working on the right items.
This causes additional traffic for well-behaving servers; currently
it cannot be turned off.
Tasks are exchanged as vCalendar 1.0 or iCalendar 2.0 VJOURNAL.
Memos are exchanged as VTODO or plain text. The logic for storing
incoming plain text is slightly different compared to the way how
the EDS memo backend did it: instead of copying the first line
from the text into the summary, it is now moved. In other words,
the first line gets stripped. The change is primarily technically
motivated; both approaches have pros and cons.
* WebDAV: improved Radicale support
Radicale > 0.7 will return status 200 for delete requests;
is now treated like 204 by SyncEvolution. 412 'Preconditiona Failed'
when asking to delete an already removed item is treated like
the more common 404 'not found'. Same with 410 'gone' instead
of 404 when trying to read a non-existent item.
* file backend: more flexible sync support for memos
The databaseFormat=text/calendar for memos did not support
synchronizing as plain text. When using the new
databaseFormat=text/calendar+plain, vCalendar/iCalendar/plain text
are all valid sync formats; the storage is iCalendar 2.0
VJOURNAL in all cases.
* WebDAV: avoid potential crash during database detection
When a server responds to a PROPFIND for a path with results for some
other path, then SyncEvolution crashed during the search for the
default calendar or address book because of a bug in the code which
was meant to handle that kind of response. Apparently Yahoo Calendar
did that. Now seen again in combination with Radicale 0.6.4.
In general, the code was made more robust to cope with bugs in
Radicale 0.6.4. Later Radicale versions fixed these issues and also
worked with SyncEvolution 1.2.2 without client-side workarounds.
* WebDAV: better path normalization
"syncURL" and "database" properties had to end in a trailing slash,
otherwise items were not found (404 errors). Now the necessary slash
is added automatically.
* Funambol: avoid slow syncs in refresh from server
libsynthesis has traditionally implemented "refresh-from-server" as
"delete local data" plus "slow" sync. This is more compatible, because
some servers (like Google) do not support "refresh-from-server".
But it has the downside that the server cannot know that the client
won't send any data, and Funambol's OneMedia now only allows one slow
sync before blocking the next one for a certain period of time. This is
done to prevent excessive resource usage by badly behaving clients.
To accomodate both kinds of servers, the new "enableRefreshSync"
sync property can be set set to explicitly allow the usage of
the "refresh-from-server" sync mode. It's off by default. The Funambol
template has it turned on, existing configs must be updated manually
(see upgrading comments below).
* Curl transport: support SSLServerCertificates=<path>
When the setting refers to a directory, then CURLOPT_CAINFO doesn't
work (must be a file). Check this and use CURLOPT_CAPATH instead.
Caveat: there are some comments in the API documentation about "NSS
enabled libcurl" which supports a directory in
CURLOPT_CAINFO. Hopefully providing an explicit path in CURLOPT_CAPATH
also works in that configuration.
* code cleanup + rewrite: syncing done in separate process
syncevo-dbus-server now runs syncing in a separate process. Local
sync also uses a second helper process. This makes the D-Bus server
more responsive via D-Bus (no more blocking operations) and
minimizes the effect of bugs in code involved with syncing
(backends, system libraries, etc.).
In the long term this restructuring will also allow more advanced
features, like monitoring local or remote storage for changes.
* SyncEvolution <-> SyncEvolution sync: multiple cycles per session
SyncML only allows one send/receive cycle per session. There are cases
(for example, client side merges data that a dumber server failed to
match correctly) where client and server are still out of sync at
the end of a cycle. When SyncEvolution syncs with another SyncEvolution
instance (locally or remotely), both sides detect that the peer
can continue syncing in the same session and start over automatically
when needed. Previously the user had to start another sync session manually.
To the user this is shown as "number of cycles" in a sync session
in the sync report. "Restart" is the process of entering a new cycle.
The cycles are also visible in the command line output as multiple
INFO lines:
[INFO] eds_contact: starting first time sync from client (peer is server)
[INFO] creating complete data backup of source eds_contact before sync (enabled with dumpData and needed for prin
Local data changes to be applied during synchronization:
*** eds_contact ***
no changes
[INFO] eds_contact: sent 1/1
[INFO] eds_contact: started
[INFO] eds_contact: first time sync done successfully
[INFO] eds_contact: starting normal sync from client (peer is server) <===
[INFO] eds_contact: started <===
[INFO] eds_contact: normal sync done successfully <===
[INFO] creating complete data backup after sync (enabled with dumpData and needed for printChanges)
Synchronization successful.
Changes applied during synchronization:
+---------------|-----------------------|-----------------------|-CON-+
| | LOCAL | REMOTE | FLI |
| Source | NEW | MOD | DEL | ERR | NEW | MOD | DEL | ERR | CTS |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| eds_contact | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
| refresh-from-local, 2 cycles, 0 KB sent by client, 0 KB received |
^^^^^^^^
| item(s) in database backup: 1 before sync, 1 after it |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| start Tue Feb 7 17:07:49 2012, duration 0:03min |
| synchronization completed successfully |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
* SyncEvolution <-> SyncEvolution sync: negotiate UID support via SyncCap (BMC #22783)
The semantic of UID/RECURRENCE-ID in calendar data is now tracked
per data store involved in a sync. If full iCalendar 2.0 semantic
(= IDs are globally unique) is guaranteed, then pairs are found
based on these IDs. Otherwise pairs must be found by looking at
item attributes.
Previously a hack was used to detect this kind of support (any kind
of SyncEvolution instance was assumed to support it, although some
backends do not).
* syncevolution.org packages: fixed D-Bus server autostart in .deb and .rpm packages
syncevo-dbus-server wasn't started automatically as part of a user
session because /etc/xdg/autostart/syncevo-dbus-server.desktop wasn't
included in the packages. This broke auto syncing after a session
restart (required manually starting SyncEvolution).
* syncevolution.org packages: support KDE
The traditional "syncevolution-evolution" package was
replaced with "syncevolution-bundle". A meta "syncevolution-evolution"
package depends on it, to support seamless updates for users who have
"syncevolution-evolution" installed.
Binary dependencies of the main .deb are ignored for backends
because loading them is optional. The new "syncevolution-kde"
package has the right dependencies for KDE/Akonadi, while
"syncevolution-evolution" mostly just lists standard libs
if the "EDS compatibility" mode is used, where libebook/libecal
are loaded dynamically.
Platform specific code (GNOME keyring, KDE wallet) was moved into
loadable, optional modules, to allow installation of the SyncEvolution
bundle without forcing the installation of unused system components.
* D-Bus: use GIO D-Bus instead of libdbus if available
When compiling from source, the more modern GIO D-Bus is used instead
of libdbus if available and recent enough (>= 2.30). syncevolution.org
binaries still use libdbus, to stay compatible with older Linux
distros.
* several minor bug fixes
syncevo-dbus-server now runs under valgrind in the nightly testing,
plus several more test scenarios were added. This helped to find
and fix various minor memory handling issues.
* developers: backend API changes
beginSync/endSync() (aka m_startDataRead/m_endDataWrite) may now be
called multiple times per SyncSource instance life cycle. SyncSources
derived from TrackingSyncSource should work without changes. Use the
Client::Source::*::testChangesMultiCycles test to check whether your
backend supports this correctly.
Reading and deleting must throw a 404 status exception when an item
is not found. The Client::Source::*::*404 tests cover this.
The special semantic of the former RegisterSyncSource::InactiveSource
(invalid pointer of value 1) caused bugs, like using it in
--print-databases (=> segfault) or not being able to store the result
of a createSource() directly in a smart pointer (=> potential leak in
SyncSource::createSource()).
Obviously a bad idea to start with. Replaced with a
RegisterSyncSource::InactiveSource() method which creates a real,
inactive SyncSource instance which can and must be deleted by the
caller.
This is a SyncSource API change for backend developers. Instead of
RegisterSyncSource::InactiveSource, return
RegisterSyncSource::InactiveSource(). Comparisons against
RegisterSyncSource::InactiveSource needs to be replaced with a call
to the new SyncSource::isInactive().
Long-running backend calls are encouraged to check for events on the
main glib context (either in a loop or with
g_main_context_iteration(NULL)) and abort when
SuspendFlags::getSuspendFlags().getState() returns
SuspendFlags::ABORT.
* packagers:
libgdbussyncevo is now installed as a normal library in /usr/lib,
even though SyncEvolution is the only user.
pcrecpp is now a new hard dependency.
Upgrading from release 1.2.x:
The sync format of existing configurations for Mobical (aka Everdroid)
must be updated manually, because the server has encoding problems when
using vCard 3.0 (now the default for Evolution contacts):
syncevolution --configure \
syncFormat=text/x-vcard \
mobical addressbook
The Funambol template explicitly enables usage of the
"refresh-from-server" sync mode to avoid getting throttled with 417
'retry later' errors. The same must be added to existing configs
manually:
syncevolution --configure \
enableRefreshSync=TRUE \
funambol
Upgrading from releases before 1.2:
Old configurations can still be read. But writing, as it happens
during a sync, must migrate the configuration first. Releases >= 1.2
automatically migrates configurations. The old configurations
will still be available (see "syncevolution --print-configs") but must
be renamed manually to use them again under their original names with
older SyncEvolution releases.
SyncEvolution 1.2.1 -> 1.2.2, 13.01.2012
========================================
Maintenance release with various bug fixes.
* syncevo-dbus-server + ConnMan: fixed "online" detection (BMC #21541, BMC #24587)
SyncEvolution did not recognize any cellular connectivity as
suitable for syncing. The strict check for certain "connected
technology" is unnecessary, anything which makes the computer
"online" should be good enough. So now it just uses the ConnMan
"State" property.
Additional benefit: will continue to work with ConnMan 1.0, which
won't have the "ConnectedTechnologies" property anymore.
The Bluetooth available check was also (incorrectly) using the
ConnMan API. Now asssume that OBEX/Bluetooth is always available.
* automatic backups: added INFO messages and fixed dumpData/printChanges (BMC #24619)
Point out that backups are created (user might be unaware otherwise
and wonder about the delay), explain why (so that users know how to
turn it off).
Turning these backups off with dumpData=0 printChanges=0 had to be
fixed, backups were always written previously.
* EDS compatibility: bumped version check for EDS 3.2
SyncEvolution is known to work with EDS 3.2. Therefore use the
libebook/ecal/edataserver libs from 3.2 if available, without
warnings in the --version output. Also happens with inconsistent
distro setups where the old libs are available and would have been
prefered by SyncEvolution 1.2.1 even though the old libs no longer
work with EDS 3.2.
* GTK-UI: do not accept service config without a username (BMC#23106)
Instead of creating such a config, an error dialog is shown.
* GTK-UI: updated translations
* fixed various compile issues, primarily on Fedora Core 17
(unistd.h/ssize_t, invoking syncevolution during compilation,
missing src/dbus/qt/configure-sub.in)
SyncEvolution 1.2 -> 1.2.1, 25.11.2011
======================================
Maintenance release with various bug fixes.
* GTK UI + config: fix "custom server" setup (BMC #13511)
When the "default" config template (= ScheduleWorld) was downgraded to
"not consumer ready" in SyncEvolution 1.1.0.99.1, setting up a custom
SyncML service in the GTK UI stopped working because the UI wouldn't
show the "not consumer ready" config.
The problem described above is deterministic and fixed now.
Initially the problem seemed to be random. So perhaps there is
also another, related issue.
* phone sync: delete<->delete conflict + phone calendar+todo sync (BMC #23744)
When deleting an item on phone and locally, the next sync failed with
ERROR messages about "object not found". Retrying the sync then worked.
* Nokia: prevent accidental usage of "calendar" or "todo" sources
Nokia phones use a combined "calendar+todo" source for syncing. The
"calendar" and "todo" sources also exist because that is where local
databases are configured.
In such a setup, syncing always has to use "calendar+todo". For example,
to refresh from the Linux desktop to the phone, use:
--sync refresh-from-server <config> calendar+todo
To work with items (restore, show local content), use the underlying sources,
as in:
--print-items <config> calendar
It was possible to accidentally sync with the "calendar". This commit
prevents that by adding an invalid URI setting to the "calendar" and
"todo" sources in the Nokia and Ovi templates. Existing configs are not
touched, so beware when you already have configured your Nokia phone.
* vCard: X- chat extensions were limited to one instance per kind
For example, only one Jabber account could be synchronized. This
was caused by an incomplete definition of the conversion to and from
vCard.
* syncevo-dbus-server + phone sync: catch SIGPIPE to avoid premature exit
Frederik Elwert reported that running a local sync with a phone via
Bluetooth caused the syncevo-dbus-server to shut down during a sync.
Explicitly telling the process to ignore the SIGPIPE signal solved that
problem.
* syncevo-http-server: support chained SSL certificates
So far, the file pointed to by --certificate-file had to
contain the server certificated (signed by a CA known to the client)
and (optionally) a client certificate. Now the file may also contain
additional intermediate certificates which will be sent to the client
(chained certificates).
* documentation: added glossary and command line conventions sections,
improved listing of properties, embedd property definitions in man page,
README and README.html
* EDS compatibility: fixed inconsistency in libecal check
The check for the _r variants in libical still used an older max
version. This might have prevented using them (if not found) or
could have led to a mixture of old and new libecal in the same
process (probably crashed).
* glib: avoid including glib/*.h headers directly
Recent glib deprecates the direct inclusion of some of its headers,
in favor of including glib.h. Doing that here whenever possible, so
perhaps it now compiles on Fedora 17 (untested).
SyncEvolution 1.1.1 -> 1.2, 13.10.2011
======================================
The major new feature of the 1.2 release is support for non-SyncML
protocols in general and CalDAV/CardDAV in particular. ActiveSync
support is in development and will be in 1.3. These protocols are
implemented as backends which are combined with other backends by
SyncEvolution in a so called "local sync". The GTK sync-ui does not
yet support configuring non-SyncML protocols. See the README.rst and
man page for more information on how to use the new feature via the
command line.
Properties not supported by SyncML servers can now be preserved
locally in two-way synchronization (BMC #15030). This depends on
information about what properties a SyncML server supports ("CtCap"),
which is typically not provided by servers. SyncEvolution contains a
copy of that information for Google Contacts (BMC #15029).
Akonadi backend and KWallet support were merged. They are not included
yet in syncevolution.org binaries. To use them compile from source.
The configuration format was updated to solve a conceptual problem
inherited with the legacy property names: the "type" property had
multiple, sometimes conflicting roles. For example, setting the
preferred data format for sync with one peer might have changed the
backend selection for some other peer (BMC #1023). Now
"backend/databaseFormat/syncFormat/forceSyncFormat" replace
"type". "type" is still accepted by the command line as alias.
Upgrading from releases before 1.2:
Old configurations can still be read. But writing, as it happens
during a sync, must migrate the configuration first. Release 1.2
automatically migrates configurations. The old configurations
will still be available (see "syncevolution --print-configs") but must
be renamed manually to use them again under their original names with
older SyncEvolution releases.
Other changes:
* Using the --sync-property and --source-property command line options is
optional, just specifying the property assignment is enough.
* syncevo-http-server was enhanced considerably. See http://syncevolution.org/wiki/http-server-howto
* support NetworkManager API >= 0.9 (BMC #19470)
* syncevolution.org binaries: now compatible with Debian Testing/libnotify.so.4 (BMC #22668)
libnotify is not linked directly into syncevo-dbus-server in the
syncevolution.org binaries. Instead libnotify.so.1 till .so.4
(current Debian Testing) are opened opened dynamically and the
necessary functions are looked up via dlsym(). Not finding the
libraries or the functions silently disables this notification
mechanism.
* Sync mode is recorded when running in SyncML server mode (BMC #2786).
* syncevo-dbus-server automatically stops when some of its libraries
are updated and restarts if auto-syncing is on (BMC #14955).
* Added code for Buteo, mKCal and QtContacts in MeeGo.
Buteo and mKCal were removed again from MeeGo, so the code
is obsolete. The QtContacts backend may be still be useful
to access items via that API, but for syncing on MeeGo
the normal EDS backend is used since MeeGo reverted back
to EDS as PIM storage.
* "databasePassword" source property: lookup failure in keyring (BMC #22937)
The databasePassword also wasn't looked up at all when doing item operations
via the command line.
When configuring sources for an HTTP server, the config name typically
is just the context (@foo). When using the config in the HTTP server,
the config name is the peer inside that context (client@foo). Because
the GNOME keyring lookup keys for the "databasePassword" (more
specifically, the object name) contained the full config name which
was different in both cases, looking up the saved password failed.
The solution is to normalize the config name (to accomodate for
different ways of spelling it) and use only the context, with @ as
before. This will break existing setups where the object name in the
keyring (incorrectly) includes the full config name. In that case just
configure the source again to set the password anew.
* Evolution Calendar: fixed detached recurrence support (BMC #22940)
When manipulating a meeting series with more than one detached
recurrence certain sequences of operations could incorrectly fail
with "UID already exists".
* iCalendar 2.0: must set VALUE in EXDATE (part of BMC #22940)
EXDATE has a VALUE parameter, which wasn't defined in the XML
profile. Didn't seem to matter at all in practice, but wasn't
standard-compliant.
* GTK sync-ui: wrap sync service descriptions (BMC #7199)
Descriptions of different sync services are not fully visible unless
word-wrapping gets enabled.
* CalDAV/CardDAV + local storage: avoid empty properties
The main motivation for this change is that a recent Apple Calendar
server rejects vCards with empty BDAY property. Another reason is that
keeping the data as small as possible is desirable by itself.
Sending an empty property serves as a hint for the peer that the
property is supported. This is not necessary when storing an item in a
backend. Therefore this commit disables empty properties for all
backends which do not themselves set the m_backendRule Synthesis info
value.
* Google Contacts: ensure that first/middle/name are set when storing in EDS (BMC #20864)
Evolution and the MeeGo UX assume that first/middle/last name are set.
That is not the case when a contact is created in the Google Contacts
web interface. Such contacts are sent by Google without the N
property.
SyncEvolution now tries to recreate the name components from the FN
string, by splitting at word boundaries and assuming "<first>
<middle> <last>" or "<last>, <first>" format. Obviously this
heuristic fails for some locales.
* Evolution Calendar: fixed error handling for broken TZIDs
* Sony Ericsson: use ISO-8859-1 for all devices (BMC #14414)
Passing invalid UTF-8 strings into libecal caused glib to
abort syncevo-dbus-server.
* auto sync: show all failed syncs except for temporary network errors (BMC #21888)
Notifications were meant to be shown for all errors except temporary
ones. This has never been implemented correctly since the feature was
introduced: instead of hiding known temporary errors, all errors except
500 (fatal error) were suppressed.
* vCard: inline local photo data (BMC #19661)
Some platforms (Maemo, MeeGo) store photos in separate files. Now SyncEvolution
efficiently includes that photo data in the generated vCard right before sending
it to a peer; previously it sent a useless local file:// URI. The Maemo port
has a less efficient workaround for that which now should be obsolete.
* syncevo-dbus-server: online status wrong without Network Manager or ConnMan (BMC #21543)
When neither Network Manager nor ConnMan are running, network presence was "not
online". This prevented running automatic syncs.
For developers:
* modified backend API
- ClientTestConfig modernized
- InsertItemResult::m_merged turned from boolean to enum
* testing and compilation changes; for example, the minimum version of
libsynthesis is now checked at configure time instead of failing at
runtime due to missing features in the Synthesis engine
SyncEvolution 1.1.99.7 -> 1.2, 13.10.2011
=========================================
Some more bug fixes and testing improvements.
* fixed potential invalid memory access in add<->add conflict handling
* fixed memory leak in workaround for EDS bug
* CalDAV/CardDAV: handle ETags without quotation marks (eGroupware)
* updated README: warning about sync direction moved to --sync option
SyncEvolution 1.1.99.6 -> 1.1.99.7, 15.09.2011
==============================================
Mostly bug fixes again. Some are a bit more intrusive, thus another
pre-release.
* syncevolution.org binaries: now compatible with Debian Testing/libnotify.so.4 (BMC #22668)
libnotify is not linked directly into syncevo-dbus-server in the
syncevolution.org binaries. Instead libnotify.so.1 till .so.4
(current Debian Testing) are opened opened dynamically and the
necessary functions are looked up via dlsym(). Not finding the
libraries or the functions silently disables this notification
mechanism.
* calendar sync: better handling for add<->add conflicts (partly fixes BMC #22783)
When both sides of a sync have added the same event, the sync must
determine which one is more recent instead of blindly overwriting
always the same side. Such conflicts are typically rare except for
enterprise scenarios where meeting invitiations are processed
automatically by a groupware (Exchange, Google Calendar/Mail, ...)
and then the attendee status is updated on one side.
SyncEvolution now does the necessary age comparison and preserves the more
recent data for most properties. In some properties the data from both
sides is preserved by concatenating the text (description, location, ...).
It remains to be seen whether that is really desirable. Also, sync statistics
are slightly off: the incoming item is counted as "added" even though it
gets turned into an update.
* item operations: authentication problem for WebDAV when using keyring (BMC #21311)
The password still wasn't looked up in the keyring when using
--import/export/delete-items.
* "databasePassword" source property: lookup failure in keyring (BMC #22937)
The databasePassword also wasn't looked up at all when doing item operations
via the command line.
When configuring sources for an HTTP server, the config name typically
is just the context (@foo). When using the config in the HTTP server,
the config name is the peer inside that context (client@foo). Because
the GNOME keyring lookup keys for the "databasePassword" (more
specifically, the object name) contained the full config name which
was different in both cases, looking up the saved password failed.
The solution is to normalize the config name (to accomodate for
different ways of spelling it) and use only the context, with @ as
before. This will break existing setups where the object name in the
keyring (incorrectly) includes the full config name. In that case just
configure the source again to set the password anew.
* Evolution Calendar: fixed detached recurrence support (BMC #22940)
When manipulating a meeting series with more than one detached
recurrence certain sequences of operations could incorrectly fail
with "UID already exists".
* iCalendar 2.0: must set VALUE in EXDATE (part of BMC #22940)
EXDATE has a VALUE parameter, which wasn't defined in the XML
profile. Didn't seem to matter at all in practice, but wasn't
standard-compliant.
* GTK sync-ui: wrap sync service descriptions (BMC #7199)
Descriptions of different sync services are not fully visible unless
word-wrapping gets enabled.
* source configs: don't check "backend" unless it is needed
When using a config which has sources with a backend type set which is
not currently available, an error was thrown even if those sources
weren't even part of the current operation (for example, syncing
another source which is currently supported).
* config migration: avoid name conflicts and auto syncing of old configs (BMC #22691)
When (auto-)migrating a config, it was possible that a name for the
peer, say foo.old, was chosen for the renamed config although there
was already such a config, for example foo.old in ~/.sync4j. Besides
being confusing for users, this also led to a bug in the code where it
copied from the older config with the foo.old name.
The main problem fixed is the disabling of auto syncing
in the old config. Otherwise it was still used by syncevo-dbus-server
for syncing, which triggered another auto-migration, ad infinitum...
* auto syncing: must check whether enabled when looking at unknown URLs (part of BMC #22691)
"syncURL = insert your URL here" with "autoSync = 0" did lead to auto
sync attempts although it wasn't enabled. A check for "auto syncing
enabled" was missing for the "unknown transport" case.
* CalDAV/CardDAV + local storage: avoid empty properties
The main motivation for this change is that a recent Apple Calendar
server rejects vCards with empty BDAY property. Another reason is that
keeping the data as small as possible is desirable by itself.
Sending an empty property serves as a hint for the peer that the
property is supported. This is not necessary when storing an item in a
backend. Therefore this commit disables empty properties for all
backends which do not themselves set the m_backendRule Synthesis info
value.
* Apple CardDAV: apply PHOTO import/export scripts by default
A recent Apple Calendar server (correctly) rejects the invalid
PHOTO;TYPE=unknown: property in a vCard. This internal representation
must be cleared before serializing the field list.
* for developers: modified backend API
- ClientTestConfig modernized
- InsertItemResult::m_merged turned from boolean to enum
* testing and compilation changes; for example, the minimum version of
libsynthesis is now checked at configure time instead of failing at
runtime due to missing features in the Synthesis engine
SyncEvolution 1.1.99.5 -> 1.1.99.6, 17.08.2011
==============================================
Mostly bug fixes, some improvements in testing and packaging. This
release was tested successfully with DAViCal 0.9.9.4.
* CalDAV: fixed incorrect change tracking causing "event not found" (BMC #22329)
* CalDAV: handle delete<->delete conflict during local sync (BMC #22327)
If the same event was deleted both locally and in the CalDAV server, syncing
failed with "event not found".
* Google Contacts: ensure that first/middle/name are set when storing in EDS (BMC #20864)
Evolution and the MeeGo UX assume that first/middle/last name are set.
That is not the case when a contact is created in the Google Contacts
web interface. Such contacts are sent by Google without the N
property.
SyncEvolution now tries to recreate the name components from the FN
string, by splitting at word boundaries and assuming "<first>
<middle> <last>" or "<last>, <first>" format. Obviously this
heuristic fails for some locales.
* CalDAV: continue despite Google Calendar access problems (see BMC #19484)
An attempt to work around "403 You don't have access to change that
event" errors, perhaps caused by
http://code.google.com/p/google-caldav-issues/issues/detail?id=38
The problem is now recorded instead of aborting the sync. The sync
then ends in a 22001 = "partial failure" error and the operation
will be retried in the next sync.
* CalDAV: transform UTC RECURRENCE-ID for Evolution (BMC #22594)
Evolution showed a meeting twice on the day of a modified recurrence,
if the meeting series was originally created and modified in Exchange,
then imported into Google Calendar.
* CalDAV syncevolution.org binaries now works when libneon.so.27
or libneon-gnutls.so.27 (Debian) are installed. Previously
libneon.so.27 was required, which is no longer available in
Debian Testing.
* syncevo-dbus-server/gdbus: fixed segfault when asked for properties
when none are available (BMC #22152)
* Evolution Calendar: fixed error handling for broken TZIDs
* Sony Ericsson: use ISO-8859-1 for all devices (BMC #14414)
Passing invalid UTF-8 strings into libecal caused glib to
abort syncevo-dbus-server.
* item operations: authentication problem for WebDAV when using keyring (BMC #21311)
The password wasn't looked up in the keyring when using --print-items/import/export/...
* WebDAV: fixed item operations without configuration (BMC #22164)
Previously failed with "[ERROR] : virtual read-only configuration node, cannot write
property webDAVCredentialsOkay = 1".
* auto sync: show all failed syncs except for temporary network errors (BMC #21888)
Notifications were meant to be shown for all errors except temporary
ones. This has never been implemented correctly since the feature was
introduced: instead of hiding known temporary errors, all errors except
500 (fatal error) were suppressed.
* vCard: inline local photo data (BMC #19661)
Some platforms (Maemo, MeeGo) store photos in separate files. Now SyncEvolution
efficiently includes that photo data in the generated vCard right before sending
it to a peer; previously it sent a useless local file:// URI. The Maemo port
has a less efficient workaround for that which now should be obsolete.
* syncevo-dbus-server: online status wrong without Network Manager or ConnMan (BMC #21543)
When neither Network Manager nor ConnMan are running, network presence was "not
online". This prevented running automatic syncs.
* fixed compile issues with Debian Testing/gcc 4.6.1
Known issues, might still be resolved for the final 1.2:
--------------------------------------------------------
* syncevolution.org binaries: libnotify1 -> libnotify4 incompatibility (BMC #22668)
Newer distros no longer have the libnotify.so.1 that syncevolution.org
binaries depend on. As a workaround it is possible to install the libnotify1
package from older distro releases.
* CalDAV: add<->add conflicts (BMC #22669)
Suppose the same meeting invitation for event UID=FOO is processed in
both Evolution and Google Calendar. This always happens when the meeting
invitation emails is sent to Google Mail, then later viewed in Evolution.
On the Evolution side, the invitation is accepted. In Google Calendar this is
still open.
When syncing in that state the sync engine does not recognize that
both sides have added the same meeting and the "meeting accepted"
information eventually gets lost.
As a workaround, always synchronize the calendar before processing
meeting invitation emails.
SyncEvolution 1.1.99.1 -> 1.1.99.5, 13.07.2011
==============================================
Release 1.1.99.5 is the first release candidate for 1.2. It has gone
through a long stabilization period and thus is suitable for normal users.
The major new feature of the 1.2 release is support for non-SyncML
protocols in general and CalDAV/CardDAV in particular. ActiveSync
support is in development. These protocols are implemented as backends
which are combined with other backends by SyncEvolution in a so called
"local sync". The GTK sync-ui does not yet support configuring
non-SyncML protocols. See the README.rst and man page for more
information on how to use the new feature via the command line.
Properties not supported by SyncML servers can now be preserved
locally in two-way synchronization (BMC #15030). This depends on
information about what properties a SyncML server supports ("CtCap"),
which is typically not provided by servers. SyncEvolution contains a
copy of that information for Google Contacts (BMC #15029).
Akonadi backend and KWallet support were merged. They are not included
yet in syncevolution.org binaries. To use them compile from source.
The configuration format was updated to solve a conceptual problem
inherited with the legacy property names: the "type" property had
multiple, sometimes conflicting roles. For example, setting the
preferred data format for sync with one peer might have changed the
backend selection for some other peer (BMC #1023). Now
"backend/databaseFormat/syncFormat/forceSyncFormat" replace
"type". "type" is still accepted by the command line as alias.
Old configurations can still be read. But writing, as it happens
during a sync, must migrate the configuration first. In contrast to
earlier, more experimental releases in the 1.2 series, 1.1.99.5 and
later automatically migrate configurations. The old configurations
will still be available (see "syncevolution --print-configs") but must
be renamed manually to use them again under their original names with
older SyncEvolution releases.
Other changes:
* syncevo-http-server was enhanced considerably. See http://syncevolution.org/wiki/http-server-howto
* support NetworkManager API >= 0.9 (BMC #19470)
* Sync mode is recorded when running in SyncML server mode (BMC #2786).
* syncevo-dbus-server automatically stops when some of its libraries
are updated and restarts if auto-syncing is on (BMC #14955).
* Using the --sync-property and --source-property command line options is
optional, just specifying the property assignment is enough.
* Added support for Buteo, mKCal and QtContacts in MeeGo.
Buteo and mKCal were removed again from MeeGo, so the code
is obsolete. The QtContacts backend may be still be useful
to access items via that API, but for syncing on MeeGo
the normal EDS backend is used since MeeGo reverted back
to EDS as PIM storage.
* code cleanup and various minor fixes/improvements, see ChangeLog
SyncEvolution 1.1 -> 1.1.1, 26.12.2010
======================================
Maintenance release, in particular improving syncing with phones.
There was a bug that could cause all kinds of weird behavior after
a failed sync with a phone, so updating is highly recommended.
* Synthesis engine: fixed a corruption issue in internal meta data which
caused duplicates and other problems in a pretty indeterminstic way;
apparently caused by failed syncs (BMC #11044).
* Synthesis engine: recurrence rules with end date now sent correctly to phones (BMC #11241).
The RRULE property was not encoded correctly previously during the
iCalendar 2.0 -> vCalendar 1.0 conversion. Events with recurrence count
were okay. Probably also affected SyncML servers without iCalendar 2.0
support.
The fix was confirmed to work with Nokia phones. It also helps with Sony Ericsson
phones, but at least the t700 still has a problem: depending on the phone's
time zone, it repeats the event for one day too long (BMC #10092).
* Synthesis engine: fixed broken time zone information when sending to phone;
previously that broke sending calendar updates to Nokia phones (BMC #9600).
iCalendar 2.0 time zone definitions imported from libical were not
encoded correctly in vCalendar 1.0 items as sent to phones. Nokia
phones accepted such data when part of a new event, but rejected
updates of it.
* Synthesis engine: shorter TZIDs, might help N900 calendar (BMC #6680).
The shorter TZIDs will be included in iCalendar 2.0 data exported
by libsyntesis and thus SyncEvolution. This change is motivated primarily
by the observation that the N900 calendar storage can handle TZID=<location>,
but not TZID=/softwarestudio.org/Tzfile/<location>.
* ScheduleWorld: disable configuration template because service has shut down.
The template is only hidden from the GTK sync-ui, but remains in SyncEvolution
for the time being because it is referenced in several places.
* Evolution CalDAV: added workaround for "must sync twice" (BMC #10265)
The Evolution CalDAV backend seems to update its data when closing the
database, not when opening it. As a result, syncevolution had to be run
twice to see all data changes. The workaround is to open the database
twice at the start of the sync. This is done for all calendar databases,
regardless of which backend they use, in case that some other (yet unknown)
backend needs the same workaround.
* GTK sync-ui: workaround for "Sync Now" button not reacting to online
status changed (BMC #9949).
* Changed slow sync handling. Some users have complained about getting
duplicated contacts (BMC #10081). The exact reason is not known (no
useful logs provided yet), but it might be due to using "duplicate"
as resolution strategy during slow syncs.
This caused slightly different contacts to be duplicated instead of
merging the two copies, reasoning that "no data loss" is better than
"duplicates". This release switches to a mode where the engine
tries harder to avoid duplicates by merging data if modification
time stamps are available for contacts (usually they are). When fields
differ, the more recent data is kept.
* convert absolute alarm back to relative (BMC #11233)
Experiments show that at least Nokia phones (and thus perhaps also
Mobical.com) interpret a fixed alarm as "repeat alarm with the same
relative offset as on first occurrence". The same transformation to
relative alarm times is applied whenever the transformation to
absolute alarm is enabled for a peer.
* Sony Ericsson: enable conversion to absolute alarm times (BMC #10092)
Like Nokia and Mobical.net, Sony Ericsson phones also seem to be unable
to deal with relative alarm times - verified with t700.
* Sony Ericsson C510: workaround for SyncML violation
The phone does not sent identifiers for the target database;
using the source identifier as fallback allows a sync to
run.
* Fixed a regression affecting users who had created a config
with SyncEvolution < 1.0. Using the config worked once, then
failed with "No configuration for ... found". Users must
manually remove the empty "peers" directory inside their
affected configuration, the fix only makes configs without that
directory usable again (BMC #9381).
* Removed obsolete workaround for older mKCal calendar storage.
* Fixed error message in QtContacts backend.
* Same SYNCEVOLUTION_DEBUG code as in master branch.
* Some updates to synccompare, including a workaround for a Perl
bug seen on Debian Testing with Perl 5.10.1-16 (Perl panic).
* Fix compilation of syncevo-dbus-server with libnotify 0.7.0 (BMC #10453).
* Fixed compilation on Debian GNU/Hurd (no MAX_PATH, Mac OS X confusion).
SyncEvolution 1.0.1 -> 1.1, 26.10.2010
======================================
An incremental update, resolving issues where the fixes would have
been too intrusive for a 1.0.x release. In particular compatibility
with Nokia phones was improved. Some new features were also included
(command line options for manipulating items, backends for MeeGo PIM
storages).
Details:
* bug fix in sync-ui: wrong direction of one-way data transfers with devices (BMC #7091)
* bug fix in syncevo-dbus-server: incorrect Presence status after config change (BMC #8453)
Shows up in sync-ui as "'Sync Now' button active after creating a config while offline".
* sync-ui (GTK version): app is now listed as "SyncEvolution (GTK)" under "Office"
* Nokia phones: avoid data loss in two-way sync due to X-EVOLUTION-UI-SLOT (BMC #2566)
* Nokia phones: alarm times in UTC, sending PHOTO (BMC #1657, #5860)
* included all phone templates submitted to syncevolution.org Wiki (BMC #5727)
* syncevo-phone-config: set consumerReady in output, more useful for Wiki (BMC #3803)
* workaround for D-Bus timeouts in EDS libecal/libebook (BMC #4026)
* added generic command line options for importing, exporting, updating, listing
and deleting items in the different backends (http://syncevolution.org/blogs/pohly/2010/manipulate-evolution-kcalextendedmkcal-qtcontacts-pim-items-uniform-command-line)
* added backends for mKCal and QtContacts (MeeGo PIM storage),
meant to be used for manipulating this data on the command line
* enhanced D-Bus interface (BMC #3558, #3559, #3560, #3562, #3563, #7761, #7766)
* the command line tool now warns when running against a different D-Bus daemon (BMC #3563)
* creating and configuring sources in a context (without peer-specific
properties) is now supported
* improved documentation: README.rst, man page, and --help output
* fixed some compile issues (BMC #6367), improved nightly testing
SyncEvolution 1.0 -> 1.0.1, 16.07.2010
======================================
A bug fix release. The main reason for releasing it is that
SyncEvolution 1.0 no longer worked on recent distros (Fedora Core 13,
Debian testing) because of a name clash between the Bluez D-Bus
utility code and recent glib.
Details:
* compile fix for FC 13 (and possibly others): use private copy of gdbus (BMC #3556)
* sync-ui: prevent overwriting device configs by accident (BMC #3566,1194)
Setting up a phone used the template name as config name and overwrote
an existing configuration of another phone that was created using that
same template. Now the code uses the Bluetooth device name as set on the
device and checks for (less likely) collisions. It also sanitizes the
name to avoid complicated config names (only relevant when also using
the command line).
* syncevo-dbus-server: accept 'application/vnd.syncml+xml; charset=UTF-8' for starting an HTTP session (BMC #3554)
The redundant charset specification was set by the Funambol
Thunderbird client. Because of a literal comparison against
'application/vnd.syncml+xml' the messages were rejected.
* config fix: operations on non-peer configs failed (BMC #3157)
When running operations on a non-peer configuration (like --restore @default
addressbook), the operation fails with
[ERROR] <source name>: type 'select backend' not supported
* ZYB.com: service goes away end of June 2010, template removed (BMC #3310)
* some build (BMC #2586, BMC #3557) and language updates
SyncEvolution 0.9.2 -> 1.0, 11.06.2010
======================================
Major new features compared to previous stable release:
* synchronize directly with a phone over Bluetooth/OBEX
* accept Bluetooth/OBEX connections in cooperation with obexd >= 0.19
* run SyncEvolution as a rudimentary HTTP SyncML server
The GTK sync-UI can be used to select a paired phone and create a
configuration for it based on the bundled configuration templates.
Configuration templates are included for Nokia phones; for other
phones see the http://syncevolution.org/development/sync-phone HOWTO
and check out the Wiki there. Some users have already reported success
for Sony Ericsson phones and added setup instructions. New templates
from the Wiki can be dropped into ~/.config/syncevolution-templates
under an arbitrary file name.
Unexpected slow syncs can be detected when running as client (MB
#2416) and unless turned off (see "preventSlowSync"), SyncEvolution
aborts the session so that the situation can be analyzed. A refresh
from client or server might be more suitable. The command line tool
provides instructions at the end of its output. The GTK sync-UI
points towards its recovery dialog.
Automatic synchronization is supported by the syncevo-dbus-server (MB
#6378). When that is installed, it will be started as part of a user
session and keep running to trigger syncs in the
background. Notifications are emitted when syncs start, end or fail
(MB #10000).
Automatic synchronization can be enabled separately for each peer
("autoSync=0/1", off by default), will be done at regular intervals
("autoSyncInterval=30" minutes) when online long enough
("autoSyncDelay=5" minutes). That last option ensures that a) an
automatic sync does not attempt to use a network connection unless it
was already active and b) hopefully is also around long enough to
complete the sync.
The Synthesis XML configuration was split up into different parts
which are assembled from /usr/share/syncevolution/xml. Files in
~/.config/syncevolution-xml override and extend the default files,
which my be useful when adding support for a new phone.
SyncML servers:
* ZYB.com now works thanks to a workaround for anchor handling (MB #2424);
only contacts tested because everything else is considered legacy by ZYB.com
* Horde: avoid confusing the server with a deviceId that starts like the
ones used in old Funambol clients, helps with calendar sync (MB #9347)
* Mobical.net (and other, similar services): fix vCalendar 1.0 alarm
properties before importing them (MB #10458)
* desknow.com works when switching to SyncMLVersion = 1.1
* Funambol, Memotoo (and probably others): preserve meeting series when
receiving update for detached recurrence (BMC #1916)
Evolution:
* addressbook backend: avoid picking CouchDB, second try (MB #7877)
* calendar backend: minor fix for change tracking when deleting
a single instance of a recurring event
* workaround for Evolution 2.30: "timezone cannot be retrieved because it
doesn't exist" is triggered incorrectly when importing non-standard
timezone definitions because libecal changed an error code (MB #9820)
Performance and reliability improvements (MB #7708):
* synccompare much faster
* database dumps consume less disk space
* more intelligent about expiring obsolete session directories
and backups
* database accesses are reduced in several backends
* shorter logs (MB #8092)
* message resending helps under unreliable network connectivity ("RetryInterval")
* full support for suspend&resume in SyncEvolution client to SyncEvolution or
Synthesis server syncs
* better handling of certain third-party time zone definitions (BMC #1332)
Improved GTK sync-UI:
* revised config screen: all in one list where entries can be expanded,
integrated setup of sync with other devices
* recovery support: restore from backup, unexpected slow sync handling
* spinner while network is in use (MB #2229)
* interactive password requests (MB #6376)
* uses new D-Bus API
Command line:
* fixed printing of rejected items (MB #7755)
* consistent logging of added/updated/deleted items with short
description
* improved error reporting (textual descriptions instead of plain
error codes MB #2069, partial success MB #7755, record and show
first ERROR encountered MB #7708)
* can create new sources (MB #8424)
* runs operations inside daemon and thus avoids conflicts with
operations done by other clients; for testing purposes (like
running a client which talks to a local server in the daemon) it is
still possible to ignore the daemon (--daemon=no, MB #5043)
* revised README, now also available as man page (BMC #690)
Redesigned and reimplemented D-Bus API, used by sync-UI and command line:
* central syncevo-dbus-server controls configurations and sync sessions:
http://syncevolution.org/development/direct-synchronization-aka-syncml-server
* accepts incoming SyncML connection requests and messages received by
independent transport stubs (obexd, HTTP server, ...)
* can be used by multiple user interfaces at once
* fully documented, see src/dbus/interfaces and http://api.syncevolution.org
* no longer depends on dbus-glib with hand-written glue code for C++,
instead uses gdbus plus automatic C++ binding generated via C++ templates
Revised configuration layout (MB #8048, design document at
http://syncevolution.org/development/configuration-handling):
* several peer-independent sync and source properties are shared
between multiple peers
* they can be accessed without selecting a specific peer, by using an
empty config name or with the new "@<specific context>" syntax
* user interface of command line unchanged
* old configurations can be read and written, without causing
unwanted slow syncs when moving between stable and unstable
SyncEvolution versions
* old configurations can be migrated with the "--migrate" command
line switch; however, then older SyncEvolution can no longer
access them and migrating more than one old configuration causes
the second or later configuration to loose its "deviceId" property
(which is shared now), causing a slow sync once
* config names may contain characters that are not allowed in the
file names used for the underlying files; will be replaced with
underscores automatically (MB #8350)
Upgrading from 0.9.x:
* Upgrading and downgrading should work seamlessly when using existing
configurations.
* The new configuration layout is only used when creating new
configurations or explicitly invoking "syncevolution --migrate" (see
above). Such configs cannot be used by older SyncEvolution releases.
* The new "RetryInterval" property causes messages to be resent
after 2 minutes (increased from 1 minute in previous 1.0 betas).
At least the Funambol server is known to not handle this correctly
in all cases (http://funzilla.funambol.com/show_bug.cgi?id=7910).
So in the Funambol config template the interval is set to zero,
disabling the feature. Disabling the feature must be done manually
in existing Funambol configurations.
SyncEvolution 1.0 beta 3 -> 1.0 final, 11.06.2010
=================================================
Bug fixes and new features:
* Configuration templates are stored in a single file (BMC #1208).
New templates (like something downloaded from http://syncevolution.org/wiki)
can be dropped into $HOME/.config/syncevolution-templates using an arbitrary
file name.
* Progress and per-source status are now also reported and recorded when
running in server mode (BMC #1359). There are still several limitations
(sync mode not reported, no information about sent/received/processed items
while the sync runs, see BMC #2786).
* Better handling of certain third-party time zone definitions (BMC #1332).
Better logging to track down such problems.
* D-Bus server + command line: return error code when failed (BMC #2193)
* syncevo-phone-config: simplified command line options, several bug fixes
(syntax error, incorrect handling of calendar+todo, BMC #1197)
* Revised README, now also available as man page (BMC #690). Conversion of D-Bus API
documentation into .html page (BMC #1745).
* Funambol, Memotoo (and probably others): preserve meeting series when
receiving update for detached recurrence (BMC #1916)
* Fix for potential out-of-bounds memory access (BMC #1007).
* HTTP server: fix for potential crash when second session was requested while an
older one was still running, initial sync was done without libical time zone
information and thus may have mismatched times (BMC #2435)
* Nokia E55: convert alarm times (BMC #1657). This is done via a new remote rule
in /usr/share/syncevolution/xml/remoterules/server/46_E55.xml
If another phone needs the same treatment, then copy that file to
~/.config/syncevolution-xml/remoterules/server and edit the <model> element.
* GTK GUI: styling fix (BMC #1372), updated toolbar for MeeGo 1.0 (BMC #1970),
avoid duplicating configs when selecting a config created by syncevo-phone-config
or the command line (BMC #1266), scroll bars for emergency window (BMC #1296),
avoid compile problem on Fedora Core 13 due to name collision with system sync()
call, updated translations.
SyncEvolution 1.0 beta 2 -> beta 3, 20.04.2010
==============================================
One more step towards the long awaited 1.0. 0.1 was released over four
years ago and the 1.0 cycle started some time last summer. Beta 3 is
considered feature complete at this point.
Automatic synchronization is supported by the syncevo-dbus-server (MB
#6378). When that is installed, it will be started as part of a user
session and keep running to trigger syncs in the
background. Notifications are emitted when syncs start, end or fail
(MB #10000).
Automatic synchronization can be enabled separately for each peer
("autoSync=0/1", off by default), will be done at regular intervals
("autoSyncInterval=30" minutes) when online long enough
("autoSyncDelay=5" minutes). That last option ensures that a) an
automatic sync does not attempt to use a network connection unless it
was already active and b) hopefully is also around long enough to
complete the sync.
Detecting online status depends on ConnMan. Without it, SyncEvolution
assumes that the network is available. For Bluetooth it is enough to
have a peer paired.
When SyncEvolution is compiled with a backend sync daemon
("syncevo-dbus-server"), then conceptually that daemon controls the
configuration and coordinates manually and automatically started sync
sessions. Previously, the command line tool bypassed the daemon by
running operations itself. Now it can hand over the command line
parameters to the daemon to be executed there ("--daemon=yes", the
default if the daemon is available; MB #5043). Command line parameters
and output of "syncevolution" are the same as before. Note that the
daemon only runs one operation at a time, which delays the command
line client when the daemon is busy. For testing purposes (like
running a client which talks to a local server in the daemon) it is
still possible to ignore the daemon (--daemon=no).
Thanks to fixes and improvements in both Synthesis engine and
SyncEvolution, suspend and resume are fully supported in client and
server (MB #2425). Previously it failed in some cases, as mercilessly
exposed by our automated testing. Now all of these tests pass. The
HTTP server now also handles message resends by clients correctly.
Direct synchronization with older phones (like Sony Ericsson K750i)
can be started now by switching to an older version of the SyncML
standard ("SyncMLVersion" property, MB #9312). No further
interoperability testing with such phones has been done at this
time. When acting as client, that same property allows talking to
older SyncML servers, like desknow.com.
A minor workaround and the right configuration make it possible to
synchronize with Nokia N85 and probably also other S60
devices. Added a template for "Nokia S60". Also made the template
for "Nokia N900" accessible in the GTK GUI.
Because determining which configuration works for a phone involves
a lot of trial-and-error, the new "syncevo-phone-config" script
automates that process.
Other changes:
* Mobical.net (and other, similar services): fix vCalendar 1.0 alarm
specifications before importing them (MB #10458)
* Nokia N900: added a config template for it and disabled the redundant
RespURI when using Bluetooth. Preliminary testing shows that this solves
some of the issues seen before (MB #10224).
* workaround for Evolution 2.30: "timezone cannot be retrieved because it
doesn't exist" is triggered incorrectly when importing non-standard
timezone definitions because libecal change an error code (MB #9820)
* "syncevo-http-server" HTTP server script is included in normal install
* syncevolution.org binaries: finally solved the libbluetooth3
incompatibility (MB #9289). Binaries of beta 2 crashed on more
recent distros because of that.
* SyncML client and Bluetooth: a mobile device running SyncEvolution
creates a configuration automatically (MB #6175). The peer contacting
us has to use the standard SyncEvolution URIs (addressbook, calendar,
todo, memo).
* command line: when dealing with the shared non-peer part of a config,
it checks for properties which are unsuitable only prints
those (MB #8048)
* GTK GUI: improved setup of devices, automatic sync switch,
some fixes for crashes and other tweaks
* Nokia 7210c: send time as UTC instead of relying on time zone
information (MB #9907).
* command line: setting up a configuration for a "SyncEvolution"
server on a client was not possible because the "SyncEvolutionClient"
configuration was picked instead (MB #10004). The latter has to
be used when configuring a SyncEvolution server to talk to a
SyncEvolution client.
* restore: no longer updates the time of the backup (MB #9963)
* various minor improvements and fixes, see ChangeLog
Upgrading:
* The new "RetryInterval" property causes messages to be resent
after 2 minutes (increased from 1 minute in previous 1.0 betas).
At least the Funambol server is known to not handle this correctly
in all cases (http://funzilla.funambol.com/show_bug.cgi?id=7910).
So in the Funambol config template the interval is set to zero,
disabling the feature. Enabling or disabling the feature must
be done manually in existing configurations.
SyncEvolution 1.0 beta 1 -> beta 2, 23.02.2010
==============================================
Several new features and some bug fixing. Despite some open issues
(see below), this release is ready for getting packaged in staging
areas of distros as replacement for 0.9.2.
As before, documentation for 1.0 is only available in the
"Development" section of syncevolution.org, including HOWTOs for
setting up the HTTP SyncML server and phones manually.
Setting up a phone became a bit easier with beta 2, because
SyncEvolution is now integrated with the GNOME Bluetooth panel: once a
device with SyncML client support is paired, a button offers to bring
up the sync-UI and configure or synchronize with that device. We do a
fuzzy match against the Bluetooth device name to find a suitable
template (not manufacturer/model, because that is not readily
available). Still not many (read: hardly any) templates available,
though.
The binaries on syncevolution.org are compiled with Bluetooth support.
libbluetooth2 or libbluetooth3 should be installed, but are not
essential. If there is no suitable version of it, the Bluetooth
channel has to be selected manually as part of the syncURL.
Unexpected slow syncs are prevented by default, in contrast to beta 1
where this feature was available but turned off. When an unexpected
slow sync is detected in a client, users have to follow the
instructions provided by the command line or sync-ui and choose how to
proceed (explicitly request slow sync, refresh from server or client,
restore from backup). SyncEvolution as server currently cannot prevent
slow syncs, even when initiating the sync with a phone.
In preparation for syncing automatically, logdir and database handling
was improved considerably. Backups use less disk space because
identical files share the same file content via hard links. This also
speeds up the synccompare Perl script. Database dumps and the
corresponding comparison are delayed until the session really runs,
which avoids doing needless work a) when the server a client tries to
contact is unreachable or down and b) by only including sources that
are really in use during a sync on the server side.
The Synthesis XML configuration was split up into different parts
which are assembled from /usr/share/syncevolution/xml. Files in
~/.config/syncevolution-xml override and extend the default files,
which my be useful when adding support for a new phone.
Summary of changes since 1.0 beta 1:
* sync-ui: recovery dialog (MB #8050), device setup, config usable with
long strings (MB #9278), fixed displaying of source phases during sync
(MB #9320)
* sync-ui + syncevo-dbus-server: integration with Bluez to detect paired
devices (MB #9216, MB #7089), select template based on device name (MB #7838),
detect network and Bluetooth connectivity (only with ConnMan, MB #7700),
passwords stored in GNOME keyring by syncevo-dbus-server are shown with
dots in sync-ui (MB #9169)
* Evolution addressbook backend: avoid picking CouchDB, second try (MB #7877)
* Evolution calendar backend: minor fix for change tracking when deleting
a single instance of a recurring event
* build fixes: Bluetooth compatibility (MB #9289), use libical _r variant
of calls because 0.43 has issues in the normal version, conflict with
system libsynthesis and libsmltk (MB #9811)
* Horde: avoid confusing the server with a deviceId that starts like the
ones used in old Funambol clients, helps with calendar sync (MB #9347)
* better reporting when SyncEvolution dies during a sync (only happend once
when it wasn't installed properly, but still... MB #9844)
* performance improvements: synccompare much faster/database dumps consume
less disk space/more intelligent about expiring obsolete session directories
and backups/database accesses are reduced in several backends (MB #7708),
shorter logs (MB #8092)
* slow sync detection: now also works in the case where the client detects
an anchor mismatch and enabled by default (MB #2416)
* OBEX transport: some error handling changes and removal of polling, now
also possible via sync-ui + syncevo-dbus-server (MB #9436)
* API changes: SyncSource introduces an "isEmpty" operation which is
needed for the slow sync detection
* SyncML: split up configuration (MB #7712), increased default message size
because the old one might have been too small for large DevInf structures
* several fixes for virtual data sources ("calendar+todo"): now works
on client side, fixed naming on server (MB #9664), fixed error message
for slow sync detection, supported in combination with sync-UI (MB #9535)
* fixes for shared configuration layout: finding sessions of peers in
non-default context, adding sources affected peers in the same context
(MB #9329), wrong context during --configure when using shortcut for peers
in non-default context (MB #9338)
Known gaps for 1.0 final and beyond:
Redesigned and reimplemented D-Bus API, required by sync-UI:
- 'syncevolution' command line tool bypasses D-Bus server and runs
sync sessions itself (MB #5043)
- availability of peers not detected when using NetworkManager
(connected for HTTP, paired for Bluetooth; MB #7700)
SyncML server in general:
- suspend/resume support is untested (MB #2425)
- the progress events and statistics reported for a SyncML client
are not generated when running as SyncML server, will require
a fair amount of refactoring in the Synthesis engine (MB #7709)
HTTP SyncML server:
- a configuration must be created for each peer manually, including
a remoteDeviceId value that contains the peer's SyncML device ID
(MB #7838)
OBEX SyncML server ("sync with phones"):
- does not support phones which require a SAN 1.0 message (MB #9312)
- determining a working configuration for an unknown phone requires
a bit of experimenting, which should be automated (MB #9862)
OBEX SyncML client:
- parsing of SAN message is rudimentary and depends on an existing local
configuration, needs to be refined depending on which SyncML server software
it is meant to work with (MB #6175)
Automatic sync (MB #6378):
- no support for the various server push notification mechanisms
- no intelligent detection of local changes
- no regular background sync, development is in progress
Upgrading from 1.0 beta 1: moving back and forth should work seamlessly
Upgrading from 0.9.x: see under beta 1
SyncEvolution 0.9.2 -> 1.0 beta 1, 26.01.2010
==============================================
Compared to the current stable release, 0.9.2, this beta release can also:
* synchronize directly with a phone over Bluetooth/OBEX
* accept Bluetooth/OBEX connections in cooperation with obexd 0.19
* run SyncEvolution as a rudimentary HTTP SyncML server
These feature were already available in a source-only 1.0 alpha
release. For the beta, we fixed some issues (nothing major)
and in addition to the source, also make binaries available. As
before, we hope to get feedback on where we are going with 1.0 and its
SyncML server and direct synchronization features. If you want to get
involved, now is a good time because a) there is something which works
and b) there is still time to influence the final 1.0, scheduled for
March 2010.
Documentation of the new features can be found in the "Development"
section (http://syncevolution.org/development) for HOWTOs or ask on
the mailing list (http://syncevolution.org/support).
Here is a more complete list of features compared to the stable
release. The full (and up-to-date) list can be retrieved from the
Moblin Bugzilla (MB) issue tracking system with this query:
http://bugzilla.moblin.org/showdependencytree.cgi?id=7892&hide_resolved=0
For changes compared to the 1.0 alpha please consult the
change log.
Implemented features are marked with a plus +, open ones with a minus -.
ZYB.com
+ now works thanks to a workaround for anchor handling (MB #2424)
- only contacts tested because everything is considered legacy
by ZYB.com
Slow sync handling (MB #2416)
+ Unexpected slow syncs can be detected when running as client and
if configured (see "preventSlowSync"), abort the session so that
the situation can be analyzed. A refresh from client or server
might be more suitable. Because this required manual intervention
by the user, the feature is off by default.
- Catching slow syncs does not work yet when running as server and
in one corner case in a client.
Improved sync-UI:
+ settings for HTTP servers are now done inside the list of
all configs and server templates instead of poping up a
separate window
+ uses the new D-Bus API
+ no longer uses private gconf key to select default peer,
replaced by "defaultPeer" in SyncEvolution config
+ added recovery features like handling of unexpected slow syncs (MB #2416)
- restoring from backup only supported by command line (MB #8050)
- spinner to indicate network activity missing (MB #2229)
- interactive password request not implemented yet (MB #6376)
Command line:
+ fixed printing of rejected items (MB #7755)
+ improved error reporting (textual descriptions instead of plain
error codes MB #2069, partial success MB #7755, record and show
first ERROR encountered MB #7708)
+ can create new sources (MB #8424)
Redesigned and reimplemented D-Bus API, required by sync-UI:
+ central syncevo-dbus-server controls configurations and sync sessions:
http://syncevolution.org/development/direct-synchronization-aka-syncml-server
+ accepts incoming SyncML connection requests and messages received by
independent transport stubs (obexd, HTTP server, ...)
+ can be used by multiple user interfaces at once
+ fully documented, see src/dbus/interfaces
+ no longer depends on dbus-glib with hand-written glue code for C++,
instead uses gdbus plus automatic C++ binding generated via C++ templates
- 'syncevolution' command line tool bypasses D-Bus server and runs
sync sessions itself (MB #5043)
- availability of peers not detected (connected for HTTP, paired for
Bluetooth; MB #7700)
- Bluetooth peers can only be configured via command line (MB #9216)
Revised configuration layout (MB #8048, design document at
http://syncevolution.org/development/configuration-handling):
+ several peer-independent sync and source properties are shared
between multiple peers
+ they can be accessed without selecting a specific peer, by using an
empty config name or with the new "@<specific context>" syntax
+ user interface in command line and D-Bus API unchanged
+ old configurations can be read and written, without causing
unwanted slow syncs when moving between stable and unstable
SyncEvolution versions
+ old configurations can be migrated with the "--migrate" command
line switch; however, then older SyncEvolution can no longer
access them and migrating more than one old configuration causes
the second or later configuration to loose its "deviceId" property
(which is shared now), causing a slow sync once
+ config names may contain characters that are not allowed in the
file names used for the underlying files; will be replaced with
underscores automatically (MB #8350)
- users of the sync-ui will not know about the --migrate option,
so if they have only one configuration, it should be migrated
automatically
SyncML server in general:
+ incoming connections are accepted by syncevo-dbus-server via
the D-Bus Connection API; because this is a "personal SyncML
server", all local data is meant to belong to a single user,
and only one sync session can be active at any point in time
+ different users on the same machine can run their own server,
as long as they ensure that listening for incoming connections
does not conflict with each other (different port in HTTP)
+ the session of an HTTP client which stops sending messages expires
after "RetryDuration" seconds instead of blocking the server
forever (MB #7710)
- suspend/resume support is untested (MB #2425)
- automatic backup of server databases is inefficient (done
even when client is not allowed to do a sync; always backs up
all data, including sources which are not active; MB #7708)
- the progress events and statistics reported for a SyncML client
are not generated when running as SyncML server, will require
a fair amount of refactoring in the Synthesis engine (MB #7709)
- the Synthesis server example config contains workarounds for
specific phones, but SyncEvolution does not currently use those;
adding new workarounds should be made very simple (MB #7712)
HTTP SyncML server:
+ test/syncevo-http-server.py provides an experimental HTTP server
based on Python and Twisted
- a configuration must be created for each peer manually, including
a remoteDeviceId value that contains the peer's SyncML device ID
(MB #7838)
OBEX SyncML server ("sync with phones"):
+ peers are contacted via a builtin transport that uses libopenobex (MB #5188)
+ Server Alerted Notification (SAN) message triggers syncs; server ID
and URI are configurable (MB #7871)
- a configuration must be created for each peer manually, including
a syncURL that contains the peer's MAC address (MB #7838)
- should be integrated into the system's Bluetooth pairing (MB #7089)
OBEX SyncML client:
+ obexd 0.19 contains a plugin which passes SyncML messages to syncevo-dbus-server
- parsing of SAN message is rudimentary and depends on an existing local
configuration, needs to be refined depending on which SyncML server software
it is meant to work with (MB #6175)
Automatic sync (MB #6378):
- no support for the various server push notification mechanisms
- no intelligent detection of local changes
- no regular background sync
- depends on safe handling of concurrent editing, which is blocked
by merging of a new Evolution Data Server API (MB #3479)
Upgrading from 0.9.x:
* Upgrading and downgrading should work seamlessly when using existing
configurations. But this being an alpha, better ensure that you have
backups of both your data and your configurations in
~/.config/syncevolution.
* The new configuration layout is only used when creating new
configurations or explicitly invoking "syncevolution --migrate" (see
above). Such configs cannot be used by older SyncEvolution releases.
SyncEvolution 0.9.1 -> 0.9.2, 23.01.2010
========================================
Synthesis SyncML Engine version: see src/synthesis/ChangeLog
New Maemo 5/Nokia N900 calendar backend and packages, brought to you
by Ove Kaaven. These packages are available via the Maemo extras-devel
repository. Bug reports can be submitted both in http://bugs.maemo.org
and http://bugzilla.moblin.org. The latter is the tracker that is
monitored by the SyncEvolution team, which will also incorporate
patches. In general, Ove is the main maintainer of the new backend.
New XMLRPC backend, contributed by Franz Knipp/M-otion. It accesses
data inside a web service via a SOUP API and thus allows synchronizing
it via SyncML. See src/backends/xmlrpc/README for more information.
Added templates for Oracle Beehive and Goosync. Both are not currently
part of the regular testing.
In addition to that, 0.9.2 is an incremental update, with several
updated translations and addressing all of the issues reported by
users for 0.9.1:
- vCard dialects: added "X-GENDER/X-SIP" (used by Maemo) and X-SKYPE
(used by Maemo and recent Evolution, MB #8948)
- Evolution Address Book: avoid picking CouchDB by default (MB #7877, evolution-couchdb #479110)
CouchDB address books are appended at the end of the local database
list, otherwise preserving the order of address books. The initial
release of evolution-couchdb in Ubuntu 9.10 is unusable because it
does not support the REV property.
Reordering the entries ensures that the CouchDB address book is not
used as the default database by SyncEvolution, as it happened in
Ubuntu 9.10. Users can still pick it intentionally via
"evolutionsource".
- installation: templates now in $(datadir)/syncevolution/templates (MB #7808)
This are files used internally, meant to be extended by distributors.
Storing them in /etc is no longer supported, but also unlikely to be
needed. Added warnings that these files cannot simply be copied into
.config because they are not complete configurations.
- installation: "make install" populates $(docdir) (MB #7168)
Previously README, COPYING, NEWS, and server READMEs were copied
into syncevolution.org .tar.gz/.deb/.rpm archives as part of
custom make rules and thus missing in other installations.
- building: --with-boost had no effect (MB#7856), detect incorrect
use of --with-synthesis-src, workaround for lack of --with-docdir
in older autoconf, do not unnecessarily depend on CPPUnit header
files and GNOME/EDS libs (MB#8338), workaround for libtool bug
("cannot install `syncecal.la' to a directory not ending in ..."),
- clarified documentation of properties for file backend (MB#8146)
- stderr redirection: detect "error" messages and show them (MB#7655)
The "GConf Error: Failed to contact configuration server..." error
message was suppressed by the code which catches noise from libraries
invoked by SyncEvolution. Now it is printed as ERROR, making it
easier to detect why running SyncEvolution inside cron needs
additional changes:
http://www.estamos.de/blog/2009/05/08/running-syncevolution-as-cron-job/
- importing contacts from SyncML server without full name (MB#5664):
Evolution expects the name to be set and shows an empty string if
it is missing. Now the name is re-added by appending first, middle and
last name.
- Evolution calendar: work around 'cannot encode item' problem (MB #7879)
Happens when the calendar file contains broken events which reference
a timezone that is not defined. Now the event is treated like one in
the local timezone.
- "http_proxy" env variable is supported regardless which HTTP transport
is used (MB#8177).
- avoid crashes when libecal sets neither error nor pointer (MB#8005)
and when aborting a running sync in the syncevo-dbus-server (MB#8385)
- "--status" output: fixed missing total item counts (MB #9097)
Upgrading from 0.9.1:
* nothing to do, upgrading and downgrading should work seamlessly
SyncEvolution 0.9 -> 0.9.1, 26.10.2009
======================================
Synthesis SyncML Engine version: see src/synthesis/ChangeLog
Mobical and Memotoo are now officially supported.
Memotoo uses vCard 2.1 with several Evolution specific extensions. It
uses iCalendar 2.0, however, without actually supporting the advanced
features of it. Times are converted to UTC and meeting information are
lost.
Mobical uses vCard 2.1 and vCalendar 1.0 as data formats, with the
result that many properties used in Evolution are not supported by the
server. In particular calendar support is very limited (known issues
when events are in time zones different from the one selected locally
and on the server, no support for meetings). For details see
README.mobical.
*** Beware *** that the Mobical SyncML password is *not* the same as
the one for their web site. Log into mobical.net, then go to "my accounts
>> configure new device >> manual settings" to find the SyncML
credentials.
It is now possible to compile database backends outside of
SyncEvolution, install them and have SyncEvolution use them
automatically like any other backend. The backend API has been
enhanced considerably. For example, backend developers have
access to a modular set of utility classes that can be mixed
into a specific implementation. Backends can access the internal
Synthesis representation directly and therefore no longer need
their own vCard/vCalendar/iCalendar parser.
The sqlite demo backend can be enabled and compiled again with
--enable-sqlite. It demonstrates how to map directly from the
Synthesis field list to some internal format (an SQLite database
schema in this case).
Other changes:
* Resend messages to cope with intermittent loss of network
connectivity (Moblin Bugzilla #3427). See the new "ResendDuration"
and "ResendDelay" configuration properties for details.
* SyncEvolution command line uses the GNOME keyring when
the new --keyring option is given.
* The logging of added and updated items was enhanced. Events,
tasks and memos are logged with a short description instead of
just the local ID. The description for contacts was improved.
* Receiving photos from Mobical failed because Mobical
does not quite follow the vCard 2.1 (Moblin Bugzilla #6668).
Sending photos worked, but added a few bytes of garbage
at the end of each photo (typically ignored when showing).
Parser was made more tolerant by Synthesis and encoder bug
was fixed.
* Task priorities used by Mobical and Evolution did not match:
vCalendar 1.0 uses 1-3, iCalendar 2.0 uses 1-9 (MB #6664).
SyncEvolution now translates between the two ranges, with
some information getting lost when talking to a peer which
only supports the smaller range.
* Importing work and home phone numbers from Google into desktop
Evolution works better, because SyncEvolution now adds the "VOICE"
flag expected by Evolution (MB#6501).
* SSL certificate checking with Google is enabled by default
and enabled in Moblin, because libsoup in that distro has
the necessary fix. Without that fix, all connection attempts
fail. The binaries on syncevolution.org are compiled with
--disable-ssl-certificate-check, so users who want the
additional security must enable it.
* .rpms on syncevolution.org no longer specify a dependency
on certain Perl features. This depencency was a problem on
Mandriva. Unwanted hard dependencies on libecal in syncevolution.org
binaries are avoided for real this time (MB#6552).
* Some sync-UI enhancements (describe sync services, avoid crash
with very long input in some of the text boxes (MB#5219), set
application icon, improved some strings).
* sync-UI: now disables sources which are not supported when
setting up a configuration, like memos on Moblin (MB #6672).
Previously the source was enabled, which prevented using
using the configuration as-is on the command line.
* The sync UI allowed to enable calendar and task synchronization
with Google although Google does not support that (MB#5871).
In new installations this is prevented by clearing the URI
for those data categories.
* Trying to remove a non-existent configuration via the command
line now raises an error, to catch typos (MB #6673).
* Improved checks which logs in the logdir belong to the current
server (MB#5215).
* Improved sanity checking of integer configuration parameters
(MB#6500).
* Spelling fix: "aboring" => "aborting"
Known issue:
* Mobical and Memotoo do not have a description in the GUI yet.
* ZYB.com is not supported because of a known anchor handling
problem in the server (MB#2424).
Upgrading from 0.9:
* nothing to do, upgrading and downgrading should work seamlessly
SyncEvolution 0.9.1 beta 2 -> 0.9.1, 26.10.2009
===============================================
Synthesis SyncML Engine version: see src/synthesis/ChangeLog
Minor changes:
* spelling fixes in NEWS file (--source-type => --source-property)
* update to zh_CN
* improved autotools compilation of libsynthesis
SyncEvolution 0.9.1 beta 1 -> 0.9.1 beta 2, 19.10.2009
======================================================
Synthesis SyncML Engine version: see src/synthesis/ChangeLog
Several fixes:
* Receiving photos from Mobical failed because Mobical
does not quite follow the vCard 2.1 (Moblin Bugzilla #6668).
Sending photos worked, but added a few bytes of garbage
at the end of each photo (typically ignored when showing).
Parser was made more tolerant by Synthesis and encoder bug
was fixed.
* Task priorities used by Mobical and Evolution did not match:
vCalendar 1.0 uses 1-3, iCalendar 2.0 uses 1-9 (MB #6664).
SyncEvolution now translates between the two ranges, with
some information getting lost when talking to a peer which
only supports the smaller range.
* The workaround for detecting an endless stream of Alert 222
messages (caused by misbehavior of certain servers when
a specific message has to be resent) aborted certain
valid (albeit somewhat pathologic) sync sessions. Improved
the heuristic so that it still catches the real loop without
aborting in that other case.
* sync-ui: now disables sources which are not supported when
setting up a configuration, like memos on Moblin (MB #6672).
Previously the source was enabled, which prevented using
using the configuration as-is on the command line.
* .rpms on syncevolution.org no longer specify a dependency
on certain Perl features. This depencency was a problem on
Mandriva. Unwanted hard dependencies on libecal in syncevolution.org
binaries are avoided for real this time (MB#6552).
* Trying to remove a non-existent configuration via the command
line now raises an error, to catch typos (MB #6673).
* Message resend options: added sanity checks to catch negative
values, clarified that duration is given in seconds, 0s resend
interval disables resending (MB #6500).
* Spelling fix: "aboring" => "aborting"
SyncEvolution 0.9 -> 0.9.1 beta 1, 06.10.2009
=============================================
Synthesis SyncML Engine version: see src/synthesis/ChangeLog
Mobical and Memotoo are now officially supported.
Memotoo uses vCard 2.1 with several Evolution specific extensions. It
uses iCalendar 2.0, however, without actually supporting the advanced
features of it. Times are converted to UTC and meeting information are
lost.
Mobical uses vCard 2.1 and vCalendar 1.0 as data formats, with the
result that many properties used in Evolution are not supported by the
server. In particular calendar support is very limited (known issues
when events are in time zones different from the one selected locally
and on the server, no support for meetings). For details see
README.mobical.
*** Beware *** that the Mobical SyncML password is *not* the same as
the one for their web site. Log into mobical.net, then go to "my accounts
>> configure new device >> manual settings" to find the SyncML
credentials.
It is now possible to compile database backends outside of
SyncEvolution, install them and have SyncEvolution use them
automatically like any other backend. The backend API has been
enhanced considerably. For example, backend developers have
access to a modular set of utility classes that can be mixed
into a specific implementation. Backends can access the internal
Synthesis representation directly and therefore no longer need
their own vCard/vCalendar/iCalendar parser.
The sqlite demo backend can be enabled and compiled again with
--enable-sqlite. It demonstrates how to map directly from the
Synthesis field list to some internal format (an SQLite database
schema in this case).
Other changes:
* Resend messages to cope with intermittent loss of network
connectivity (Moblin Bugzilla #3427). See the new "ResendDuration"
and "ResendDelay" configuration properties for details.
* The logging of added and updated items was enhanced. Events,
tasks and memos are logged with a short description instead of
just the local ID. The description for contacts was improved.
* The sync UI allowed to enable calendar and task synchronization
with Google although Google does not support that (MB#5871).
In new installations this is prevented by clearing the URI
for those data categories.
* Importing work and home phone numbers from Google into desktop
Evolution works better, because SyncEvolution now adds the "VOICE"
flag expected by Evolution (MB#6501).
* SyncEvolution command line uses the GNOME keyring when
the new --keyring option is given.
* SSL certificate checking with Google is enabled by default
and enabled in Moblin, because libsoup in that distro has
the necessary fix. Without that fix, all connection attempts
fail. The binaries on syncevolution.org are compiled with
--disable-ssl-certificate-check, so users who want the
additional security must enable it.
* syncevolution.org binaries should be compatible with a wider
range of Evolution releases again (MB#6552).
* Some sync UI enhancements (describe sync services, avoid crash
with very long input in some of the text boxes (MB#5219), set
application icon, improved some strings).
* Improved checks which logs in the logdir belong to the current
server (MB#5215).
* Improved sanity checking of integer configuration parameters
(MB#6500).
Known issue:
* Mobical and Memotoo do not have a description in the GUI yet.
* ZYB.com is not supported because of a known anchor handling
problem in the server (MB#2424).
SyncEvolution 0.8.1 -> 0.9, 12.08.2009
--------------------------------------
Synthesis SyncML Engine version: see src/synthesis/ChangeLog
This is a major new release, with first steps towards further improvements.
From this release on, the Synthesis SyncML engine will be the
underlying SyncML and data conversion engine.
A native GTK GUI is now included. The "sync-ui" program depends on a
backend D-Bus service ("synevo-dbus-server") and several auxiliary
files. Therefore, it only runs without hacks after installation in
/usr (possible with .deb, .rpm and binary .tar.gz archives, and
with "sudo make install", after compiling from source). The
normal command line tool still works without being installed.
In this release, the data handling model was changed from "all items
are sent verbatim to the SyncML server" to "parse and convert". The
argument for the former approach was that the SyncML server should be
the only entity in the system which does data conversion. The previous
releases already had to deviate from this approach to accommodate for
minor client/server incompatibilities and for vCard 2.1 support, so the
new approach just takes it one step further.
The main reason for going to full semantic conversion is vCalendar 1.0
support. Support by servers for iCalendar 2.0, the only format
supported by 0.8.1, is often still incomplete or even non-existent. By
doing the conversion on the client side, SyncEvolution is now able to
synchronize events and tasks with a wider variety of servers.
It is still true that properties not supported by a server cannot
be synchronized to other devices, so using a server with full
iCalendar 2.0 support is recommended. But in contrast to 0.8.1,
information that can be stored only locally is no longer lost when
receiving an incomplete update from the SyncML server, thanks to
intelligent merging, provided by the Synthesis engine. This depends on
an accurate description of the server's capabilities, which might not
be provided by all of them. This still needs to be tested in more detail.
Interoperability with servers tested extensively in this release.
The following servers are now supported:
* ScheduleWorld. There is very complete support for Evolution data. The
only known issues are around resuming from an interrupted sync.
* Google contact sync.
Google follows the vCard 2.1 specification, and thus does not support
some of the vCard 3.0 additions, nor some of the common extensions. As
a result, several properties are not synchronized (nickname, birthday,
spouse/manager, URLs, ...). Only one top-level organization seems to
be supported. For details, see README.google.
Regarding Google's SyncML support, refresh-from-client and
one-way-from-client sync modes are not supported. Deleting contacts
moves them out of the main address without deleting them permanently. When
adding such a contact again, the server discards the data sent by the
client and recreates the contact with the data that it remembered.
Because SSL certificate checking for Google works only with libsoup
if the platform has a patched libsoup
(http://bugzilla.gnome.org/show_bug.cgi?id=589323) or libsoup >=
2.28, certificate checking remains turned off by default for
Google. If your platform has a suitable libsoup (like Moblin 2.0),
then enable checking with:
syncevolution --configure \
--sync-property SSLVerifyServer=true \
--sync-property SSLVerifyHost=true \
google
* Funambol, with calendar and task support. Funambol supports iCalendar 2.0
in the current server, so this is enabled in the configuration template.
Not all iCalendar 2.0 features are supported by the server,
most notably support for meetings (drops attendees), meeting
invitations (drops UID), detached recurrences
(drops RECURRENCE-ID). See README.funambol for details.
Interoperability with the Funambol server was improved by adding
support for some vCard extensions (X-MANAGER/ASSISTANT/SPOUSE/ANNIVERSARY,
#2418). Lost ACTION property has a work around (#2422).
To enable that support in an existing configuration so that it
exchanges items in the more suitable iCalendar 2.0 format, use:
syncevolution --configure --source-property sync=two-way \
funambol calendar todo
syncevolution --configure --source-property type='calendar:text/calendar!' \
funambol calendar
syncevolution --configure --source-property type='todo:text/calendar!' \
funambol todo
Without the exclamation mark, format auto-negotiation would pick the
less capable vCalendar 1.0 format because that is marked as preferred
by the server.
*** WARNING ***: After switching from a previous release to the
current one, or vice versa, do a "syncevolution --sync
refresh-from-server" or "--sync refresh-from-client" (depending on
which side has the authoritative copy of the data) once, to get client
and server into a consistent state. Not doing so can result in
applying the same changes to the server multiple times, and thus
duplicates.
Other changes in detail:
* vCalendar 1.0 is now supported.
* Both libcurl and libsoup can be selected at compile time as HTTP(S)
transport mechanism.
* SF #2101015: Expect: 100-continue header results in 417 Error with proxy.
Should no longer occur with the HTTP transports in this release.
* SF #1874805: Syncing with Funambol results in loosing all-day property.
This now works thanks to the Synthesis data conversion rules.
* SF #2586600: Synchronisation with mobical.net fails in 0.8.1.
Works now, but there are some known issues (Bugzilla #3009)
and therefore mobical.net is not officially supported yet.
* SF #2542968: Separator for categories should not be escaped.
Done correctly by the Synthesis vcard conversion.
* bug fix: Evolution notes with only a summary and no description were
not sent correctly to the server. Instead of sending the summary,
an empty text was sent.
* CTRL-C no longer kills SyncEvolution right away. Instead it
asks the server to suspend the session. If that takes too
long, then pressing CTRL-C twice quickly will abort the sync
without waiting for the server (Warning, this may lead to a
slow sync in the next session).
* WBXML is enabled by default now, except for Funambol (#2415).
Using WBXML reduces message sizes and increases parsing
performance.
* New configuration templates can be added to
/etc/default/applications/syncevolution. These templates may contain
icons, which are used by the GUI (no icons shipped right now).
* Information about previous synchronization sessions is now stored in a
machine-readable format and can be accessed using the new
--print-sessions options. The output of this information is more
complete and more nicely formatted.
* --status now shows not only data changes since the last sync, but also
item changes (see README for the difference between the two).
* The new --restore option allows restoring local data to the state as
it was before or after a sync. For this to work, "logdir" must be set
(done by default for new configurations). The format of database dumps
was changed to implement this feature. Instead of in a flat file,
items are now saved as individual files in a directory. To get the
previous format back (for example, to import as one .vcf or .ics file
manually) concatenate these files.
* With –-remove, one can remove configurations. It leaves data files and
the local databases untouched.
Known issues:
* The GUI includes the number of locally deleted items during a
refresh-from-server sync in the number of "received changes"
(#5185), which is a bit misleading. This is a result of #3314,
which introduced changes not "received" from the server.
* When a network error occurs and the client
never notices that the connection to the server was lost,
it will hang forever, waiting for the server's reply (#3427).
* The file backend now works only for data formats understood
by SyncEvolution and the Synthesis engine. Items are parsed
when exchanging them among the backend, engine, and server,
in contrast to 0.8.1, where item content was not touched
locally (#5046).
* The ZYB.com server sends conflicting sync anchors, so
most syncs don't work as expected (#2424).
SyncEvolution 0.9 beta 3 hotfix -> 0.9 final, 12.08.2009
--------------------------------------------------------
Because SSL certificate checking for Google only works with libsoup if
the platform has a patched libsoup
(http://bugzilla.gnome.org/show_bug.cgi?id=589323) or libsoup >= 2.28,
certificate checking remains turned off by default for Google. If your
platform has a suitable libsoup (like Moblin 2.0), then enable
checking with:
syncevolution --configure \
--sync-property SSLVerifyServer=true \
--sync-property SSLVerifyHost=true \
google
Only minor changes:
* updated translations
* refresh-from-server syncs now report how many items were
deleted locally at the start of the sync (Bugzilla #3314).
The GUI includes the number of locally deleted items during a
refresh-from-server sync in the number of "received changes",
which is a bit misleading (#5185).
* fixed build issue on Fedora 11/g++ 4.4 (Bugzilla #5061)
* some build and test improvements
* proper fix for D-Bus error functions (#4919)
* improve sync-ui startup time by avoiding an unnecessary
copying of the sync config into itself (#5021)
* adapted tooltip style (used for SyncML server links) to new
Moblin theme, they weren't visible earlier (#5017)
* notify zone selector in Moblin 2.0 about sync-ui startup (#4752)
* sync-ui: minor layout change for fatal error situation
SyncEvolution 0.9 beta 3 -> 0.9 beta 3 hotfix, 23.07.2009
---------------------------------------------------------
Found additional Google limitation: the server drops photos if they
exceed a certain size. The limit is somewhere between 40KB (okay) and
80KB (dropped).
The last-minute workaround for Google/libsoup/gnutls (using http)
didn't work because apparently Google only supports SyncML over https
(Bugzilla #4551). Now the default configuration template uses https
with all certificate checking disabled. A patch for libsoup was
submitted to upstream.
Some error messages by the "syncevolution" command line tool were not
printed (#4676). The root cause was the intentional interception of
stderr to hide the noise printed by various system libraries
(#1333). Unfortunately remarks about incorrect command line options
were among swallowed messages. No good workaround available short of
disabling the redirection with SYNCEVOLUTION_DEBUG=1, so let's release
an update...
Other changes:
* updated translations
SyncEvolution 0.9 beta 2 -> 0.9 beta 3, 21.07.2009
--------------------------------------------------
Synthesis SyncML Engine version: see src/synthesis/ChangeLog
Enabled calendar and task synchronization for myFunambol.com.
Not all iCalendar 2.0 features are supported by the server,
most notably support for meetings (drops attendees), meeting
invitations (drops UID), detached recurrences
(drops RECURRENCE-ID). See README.funambol for details.
Interoperability with the Funambol server was improved by adding
support for some vCard extensions (X-MANAGER/ASSISTANT/SPOUSE/ANNIVERSARY,
Bugzilla #2418). Lost ACTION property is worked around (#2422).
Synchronization with Google Contacts was enabled and tested.
A configuration template for that server is now provided. Google
follows the vCard 2.1 specification and thus does not support some
of the vCard 3.0 additions, nor some of the common extensions. As
a result, several properties are not synchronized (nickname, birthday,
spouse/manager, URLs, ...). Only one top-level organization seems to
be supported. For details, see README.google.
Regarding Google's SyncML support, refresh-from-client and
one-way-from-client sync modes are not supported. Deleting contacts
moves them out of the main address without deleting them permanently. When
adding such a contact again, the server discards the data sent by the
client and recreates the contact with the data that it remembered.
SSL certificate checking with libsoup (the default transport) is now
supported (#2431). However, libsoup/gnutls are very strict about SSL
certificate checking and reject version 1 certificates, like the one
used by Verisign for Google (#4551). At the moment the only solution
is to fall back to plain http in the Google configuration template.
CTRL-C no longer kills SyncEvolution right away. Instead it
asks the server to suspend the session. If that takes too
long, then pressing CTRL-C twice quickly will abort the sync
without waiting for the server (warning, this may lead to a
slow sync in the next session).
WBXML is enabled by default now, except for Funambol (#2415).
Using WBXML reduces message sizes and increases parsing
performance. It was not enabled initially in the 0.9 releases in order
to test this new feature more thoroughly. Old configs don't have an
explicit enableWBXML setting and therefore will automatically use the
new default.
Various bug fixes and improvements:
* only show servers in GUI which are tested and supported (Bugzilla #3336)
* a single log file is written in .html format (#3474)
* added several translations of the GUI
* lots of testing improvements, build binary packages again
UPGRADING
When enabling calendar and todo synchronization with Funambol
in an existing configuration, set the type so that iCalendar 2.0 is used:
syncevolution --configure --source-property sync=two-way funambol calendar todo
syncevolution --configure --source-property type='calendar:text/calendar!' funambol calendar
syncevolution --configure --source-property type='todo:text/calendar!' funambol todo
When creating a configuration anew, this is not necessary because the
configuration template contains those types.
SyncEvolution 0.9 beta 1 -> 0.9 beta 2, 12.06.2009
--------------------------------------------------
Synthesis SyncML Engine version: see src/synthesis/ChangeLog
Major new feature: a GTK GUI! The "sync-ui" program depends on a
backend D-Bus service ("synevo-dbus-server") and several auxiliary
files. Therefore it only runs without hacks after "sudo make install",
in contrast to the normal command line which can be invoked directly.
New configuration templates can be added to
/etc/default/applications/syncevolution. These templates may contain
icons which are used by the GUI (no icons shipped right now).
Information about previous synchronization sessions is now stored in a
machine-readable format and can be accessed via the new
--print-sessions options. The output of this information is more
complete and nicer formatted.
--status now not only shows data changes since the last sync, but also
the item changes (see README for the difference between the two).
The new --restore option allows restoring local data to the state as
it was before or after a sync. For this to work, "logdir" must be set
(done by default for new configurations). The format of database dumps
was changed to implement this feature: instead of in a flat file,
items are now saved as individual files in a directory. To get the
previous format back (for example, to import as one .vcf or .ics file
manually) concatenate these files.
With --remove one can remove configurations. It leaves data files and
the local databases untouched.
Various bug fixes and improvements:
* compiles and works again on Debian Etch if Boost 1.35 is installed
from www.backports.org (without GUI, see Bugzilla #3358)
* uses XDG_CACHE_HOME (= ~/.cache) for logs and database dumps to
avoid interfering with .desktop search in XDG_DATA_HOME; the
directory there is automatically moved when running syncevolution
(Bugzilla #3309)
* re-enabled certain config options (clientAuthType, maxMsgSize, maxObjSize);
normally it shouldn't be necessary to modify those (Bugzilla #3242, #2784)
* fixed error handling of unexpected server reply in libsoup transport
(Bugzilla #3041)
* message logging is enabled at logLevel 3 (XML translation) and 4 (also
original XML or WBXML message)
* GTK GUI fixes since initial Moblin 2.0 beta: only start it once if
libunique is available (Bugzilla #3154), wrap text in change sync service"
button (Bugzilla #2064), sort sources alphabetically in UI (Bugzilla #2070)
SyncEvolution 0.8.1 -> 0.9 beta 1, 13.05.2009
---------------------------------------------
Synthesis SyncML Engine version: see src/synthesis/ChangeLog
A major new release and the first step towards further improvements:
from this release onwards, the Synthesis SyncML engine is used as the
underlying SyncML and data conversion engine. The focus of this first
beta was to reach the same level of functionality and stability as in
0.8.1. Therefore this beta does not yet bring much new features; this
will be the focus of further beta releases until finally 0.9 will be a
full replacement for 0.8.1.
This release also switches from an "all items are sent verbatim to the
SyncML server" to a "parse and convert" data handling model. The
argument for the former approach was that the SyncML server should be
the only entity in the system which does data conversion. The previous
releases already had to deviate from this approach to accommodate for
minor client/server incompatibilities and for vCard 2.1 support.
The main reason for going to full semantic conversion is vCalendar 1.0
support. Support by servers for iCalendar 2.0, the only format
supported by 0.8.1, is often still incomplete or even non-existent. By
doing the conversion on the client side, SyncEvolution is now able to
synchronize events and tasks with a wider variety of servers.
It is still true that properties not supported by a server cannot
be synchronized to other devices, so using a server with full
iCalendar 2.0 support is recommended. But in contrast to 0.8.1,
information that can only be stored locally is no longer lost when
receiving an incomplete update from the SyncML server thanks to
intelligent merging provided by the Synthesis engine. This depends on
an accurate description of the server's capabilities, which might not
be provided by all of them - still needs to be tested in more detail.
*** WARNING ***: after switching from a previous release to the
current one or vice versa, do a "syncevolution --sync
refresh-from-server" or "--sync refresh-from-client" (depending on
which side has the authoritative copy of the data) once to get client
and server into a consistent state. Not doing so can result in
applying the same changes to the server multiple times and thus
duplicates.
Changes in detail:
* vCalendar 1.0 is now supported. Because this hasn't been tested that
much yet, events and tasks are still disabled in the Funambol default
configuration (SF #2635973).
* Both libcurl and libsoup can be selected at compile time as HTTP(S)
transport mechanism.
* SF #2101015: Expect: 100-continue header results in 417 Error with proxy
Should no longer occur with the HTTP transports in this release.
* SF #1874805: Syncing with Funambol results in loosing all-day property
This now works thanks to the Synthesis data conversion rules.
* SF #2586600: Synchronisation with mobical.net fails
Should work now because of the different SyncML implementation (untested).
* SF #2542968: separator for categories should not be escaped
Done correctly by the Synthesis vcard conversion.
* bug fix: Evolution notes with only a summary and no description were
not sent correctly to the server: an empty text was sent instead of
sending the summary
Known shortcomings in this release which will be fixed before the
final 0.9:
* Verbatim file backups of items on the SyncML server are currently
not possible: the SyncEvolution "file" backend still exists, but
all items are converted by the Synthesis engine and therefore must
be in a format supported by the engine.
* HTTPS can be used with libsoup, but certificate checking is always
disabled. Need to find a portable way to determine where the
certificate file is on various systems.
* Log file handling is not yet unified: the traditional client.log
contains only high-level SyncEvolution log entries. Low-level
SyncML and engine log entries are in sysync_*.html files.
* stdout and stderr messages from system libraries are visible on the
console. 0.8.1 used to redirect those into the client.log to hide
this noise; this will be added again. In the meantime, ignore
messages like "Deadlock potential - avoiding evil bug!". This is
liborbit telling us that it is (hopefully successfully) handling
something nasty.
SyncEvolution 0.8.1 -> 0.8.1a, 15.12.2008
-----------------------------------------
C++ client library: 7.0 plus some patches, see github
repository referenced in configure script.
A minor bug fix release, updating only necessary on Mac OS X.
* #2307976 "Trace/BPT trap - sync failure": occurs randomly in
Mac OS X specific transport layer of the Funambol C++ client
library. Avoided in 0.8.1a by using libcurl as transport,
as in 0.7.
SyncEvolution 0.8 -> 0.8.1, 11.10.2008
--------------------------------------
C++ client library: 7.0 plus some patches, see github
repository referenced in configure script.
A minor bug fix release, updating not really necessary.
The binary packages for Evolution are built now so that
one package works for all compatible Evolution releases,
including the new Evolution 2.24.
* Evolution calendar: regression in 0.8: one-way sync of virtual
birthday calendar (#2095433). "refresh-from-client" works again
for the birthday calendar. Other modes are not supported.
In contrast to previous releases SyncEvolution now does some
sanity checks that the sync mode is right.
* Mac OS X: removing old logdirs failed (#2087389). Fixed.
* SyncML client library: "Expect: 100-continue" header
resulted in 417 error with certain proxies (#2101015).
Now this header is always disabled; it doesn't make
much sense with SyncML anyway.
* The development of the Funambol C++ client library is now
tracked in a git repository on github.com. Modifications
and tags for SyncEvolution are checked in there. The
configure script checks out the right sources from there
automatically; can be controlled via --with-funambol-src
parameter.
* Evolution desktop: the version of the used Evolution libraries
is included in the "--version" output and log files.
* Cleaned up README. Kudos to Martin Wetterstedt for pointing
out mistakes in the README and the web site.
SyncEvolution 0.7 -> 0.8, 29.08.2008
------------------------------------
C++ client library: 7.0 plus compatibility patch for Synthesis
Updating user configuration: this version introduces a new, simplified
configuration layout. Old configurations still work. They can be
converted to the new format via a new "--migrate" command line option.
*** WARNING ***: this version uses a different change tracking for Mac
OS X address book, Evolution calendars, task lists and memos. After
switching from a previous release to the current one or vice versa, do
a "syncevolution --sync refresh-from-server" once to reset the change
tracking. Not doing so can result in applying the same changes to the
server multiple times and thus duplicates.
* New configuration file layout: following the freedesktop.org
recommendation, new configurations are stored in
$XDG_CONFIG_HOME/syncevolution or $HOME/.config/syncevolution if
XDG_CONFIG_HOME is not set. The old layout under
$HOME/.sync4j/evolution is still supported.
* New command line options: new configurations can be created by
syncevolution itself (--configure), including setting of all
configuration properties (--sync-property, --source-property).
The configuration can dumped to stdout (--print-config), with or
without comments explaining each property (--quiet). See the
README for details.
* The "evolutionsource" source property no longer has to be configured.
If left blank, the default client database will be synchronized.
* Selecting which kind of data is to be synchronized under a specific
source name is a lot easier now and the same on all supported
platforms: the SyncEvolution backends can be selected via aliases
(e.g. "contacts") and the format is specified via an optional
MIME type (e.g. "contacts:text/x-vcard"). In the unlikely situation
that multiple backends are active which can synchronize the same
kind of data, then the right one can be selected by the unique
name of the backend (e.g. "Evolution Address Book").
* New configurations automatically get a random client ID string.
Setting it manually is still possible, but no longer necessary.
Disabling unavailable data sources is also done automatically.
SyncEvolution checks that the backend is available and there is
at least one database (the first one will be synchronized unless
explicitly changed). If these checks fail and the sync source was
explicitly requested by the user by listing it after the server
name, then an error is printed and no configuration is written.
If the user wants the default setup, then the source is silently
disabled.
* All passwords can be read from stdin at runtime or an environment
variable (see "--sync-property password=?" or README for details).
Both avoids the less secure storing of plain text passwords in the
configuration files (SF #1832458).
* Detached recurrences: meeting series where some occurrences were
modified are now supported. Previously only the main event was
synchronized. All exceptions got lost when copying back from the
server. Requires a SyncML server which supports this. ScheduleWorld
was extended to do that.
* Fixed segfaults caused by logging certain data. The reason was an
API change in the client library's logging calls which the older
SyncEvolution code hadn't been adapted to. Did not normally occur,
but might have been the reason for SF #1830149 (unconfirmed).
* Time zone support: the time zones of incoming events are mapped
to native time zone definitions whenever possible. Currently
this works if the TZID follows the Olson naming scheme with a
location at the end. Matching the time zone has the advantage of
being able to update the time zone definition without having to
recreate the event. If matching fails and the VTIMEZONE definition
differs from one already imported earlier, then SyncEvolution works
arounds limitation in Evolution by renaming the time zone.
Previously the new event used the old and most likely out-dated
time zone definition.
***WARNING***: Evolution itself does not do either of these steps
itself yet, thus importing meeting invitations via Evolution still
fails in some cases. The code implementing the time zone handling
described above was written with inclusion into Evolution itself in
mind; a discussion with the Evolution developers about that is in
progress.
* On Maemo/Nokia Internet Tablets, calendar synchronization now
works because the new calendar change tracking no longer depends
on some of the backend calls which used to fail (SF #1734977).
* Added SSL configuration options: certificate checking can be
relaxed or disabled completely (SF #1852647).
* Added a new file backend: stores each SyncML item as a separate file
in a directory. The directory has to be specified via the database
name, using [file://]<path> as format. The file:// prefix is
optional, but the directory is only created if it is used.
Change tracking is done via the file systems modification time
stamp: editing a file treats it as modified and then sends it to the
server in the next sync. Removing and adding files also works.
The local unique identifier for each item is its name in the
directory. New files are created using a running count which
initialized based on the initial content of the directory to
"highest existing number + 1" and incremented to avoid collisions.
Although this sync source itself does not care about the content of
each item/file, the server needs to know what each item sent to it
contains and what items the source is able to receive. Therefore
the "type" property for this source must contain a data format
specified, including a version for it. Here are some examples:
- type=file:text/vcard:3.0
- type=file:text/plain:1.0
* Code restructuring: it is now possible to add new backends and thus
write SyncML clients for other kinds of data without touching any
line of code in SyncEvolution itself. All the required interfaces
are documented inside SyncEvolution itself. A HTML documentation can
be built via the new "make doc" target (requires Doxygen and dot).
The SyncEvolution framework itself never depended on GNOME or
Evolution, only the Evolution data sources did. If you want
support for other ways of storing your data, consider writing
a new data source - it is really easy. See EvolutionSyncSource
or TrackingSyncSource for details.
* Messages are printed to the screen immediately. More readable
log file format.
* Maemo: the useless ''list: unable to access calendars:
failure' error message is avoided. It was triggered by not having
memo support in Evolution Data Server. Cleaned up the code so that
it properly distinguishes between 'calendar', 'memo list' and
'task list'.
* added server template for MemoToo; note that the server has not been
tested
* added synchronization of Evolution memo summary
Most devices only synchronize plain text and do not have a
separate summary field. Such an extra summary field was added to
Evolution after memo support was initially implemented in
SyncEvolution, therefore SyncEvolution did not transmit that
field.
Added transmitting the summary by inserting it as first line of
the plain text blob *if* it is not already identical with the
first line. When receiving a memo, the summary is set from the
first line *without* removing the first line because the first
line might have been used as a normal part of the memo.
* Various other minor changes, fixes and lots of code cleanups.
* license cleanup: SyncEvolution is GPL v2 or later
SyncEvolution 0.8 beta 2 -> 0.8 final, 29.08.2008
-------------------------------------------------
C++ client library: 7.0 plus compatibility patch for Synthesis
* license cleanup: SyncEvolution is GPL v2 or later
SyncEvolution 0.8 beta 2 -> 0.8 beta 3, 17.08.2008
--------------------------------------------------
C++ client library: 7.0 plus compatibility patch for Synthesis
* Another revision of updating events in Evolution calendars:
the method introduced in 0.8 beta 1 for dealing with
detached recurrences did not work with the Evolution Exchange
Connector. Now both Exchange and local calendars pass the unit
tests again.
* minor code cleanup (testing, writing additional backends)
SyncEvolution 0.8 beta 1 -> 0,8 beta 2, 03.08.2008
--------------------------------------------------
C++ client library: 7.0 plus compatibility patch for Synthesis
* To prevent accidental sync runs when a configuration change
was intented, a new --run switch must be used when configuration
properties are given on the command line. When neither --run
nor --configure are specified, SyncEvolution prints an
error and refuses to do anything.
* Improved documentation for command line, in particular the synopsis.
* Added a new file backend: stores each SyncML item as a separate file
in a directory. The directory has to be specified via the database
name, using [file://]<path> as format. The file:// prefix is
optional, but the directory is only created if it is used.
Change tracking is done via the file systems modification time
stamp: editing a file treats it as modified and then sends it to the
server in the next sync. Removing and adding files also works.
The local unique identifier for each item is its name in the
directory. New files are created using a running count which
initialized based on the initial content of the directory to
"highest existing number + 1" and incremented to avoid collisions.
Although this sync source itself does not care about the content of
each item/file, the server needs to know what each item sent to it
contains and what items the source is able to receive. Therefore
the "type" property for this source must contain a data format
specified, including a version for it. Here are some examples:
- type=file:text/vcard:3.0
- type=file:text/plain:1.0
* Code restructuring: it is now possible to add new backends and thus
write SyncML clients for other kinds of data without touching any
line of code in SyncEvolution itself. All the required interfaces
are documented inside SyncEvolution itself. A HTML documentation can
be built via the new "make doc" target (requires Doxygen and dot).
SyncEvolution 0.8 alpha 1 -> 0.8 beta 1, 12.07.2008
---------------------------------------------------
C++ client library: the frozen 7.0 code, but before the release
* Added support for detached recurrences (aka modified instances of
a recurring event). Requires a SyncML server which supports this.
ScheduleWorld was extended to do that.
* Fixed segfaults caused by logging certain data. The reason was an
API change in the client library's logging calls which the older
SyncEvolution code hadn't been adapted to. Did not normally occur,
but might have been the reason for SF #1830149 (unconfirmed).
* when creating a config for the first time, only enable sync sources
which can be synchronized (SF #1991286)
The check for that was completely missing. Now SyncEvolution
checks that the backend is available and there is at least one
database (the first one will be synchronized unless explicitly
changed). If these checks fail and the sync source was explicitly
requested by the user by listing it after the server name, then
an error is printed and no configuration is written. If the user
wants the default setup, then the source is silently disabled.
* Fixed incorrect properties in some of the new server templates
(ScheduleWorld syncURL + calender URI, Funambol syncURL, ScheduleWorld
addressbook type)
* Device IDs must start with the "sc-pim-" prefix, otherwise myFUNAMBOL
may treat different devices as the single phone that myFUNAMBOL
supports, leading to unwanted slow syncs.
* Maemo package is build again so that backends are loaded dynamically:
installing Dates application is as it was with the 0.7 release
(SF #1993109). The useless ''list: unable to access calendars:
failure' error message is avoided. It was triggered by not having
memo support in Evolution Data Server. Cleaned up the code so that
it properly distinguishes between 'calendar', 'memo list' and
'task list'.
* added server template for MemoToo; note that the server has not been
tested
* added synchronization of Evolution memo summary
Most devices only synchronize plain text and do not have a
separate summary field. Such an extra summary field was added to
Evolution after memo support was initially implemented in
SyncEvolution, therefore SyncEvolution did not transmit that
field.
Added transmitting the summary by inserting it as first line of
the plain text blob *if* it is not already identical with the
first line. When receiving a memo, the summary is set from the
first line *without* removing the first line because the first
line might have been used as a normal part of the memo.
* removed --properties option: it wasn't implemented yet and won't be in 0.8
* fixed regression in alpha 1: setting sync mode during status query
or sync affected *all* sources, even the disabled ones. Now it only
affects the enabled ones, as intended. To enable disabled sync sources,
list them after the server name.
*** WARNING ***: this version uses a different change tracking for
for Mac OS X AddressBook. After switching from a previous release to the current
one or vice versa, do a "syncevolution --sync refresh-from-server"
once to reset the change tracking. Not doing so can result in applying
the same changes to the server multiple times and thus duplicates.
A similar change was necessary in 0.8 alpha 1 for Evolution calendar,
tasks, and memos. When switching from a version >= 0.8 alpha 1 to an
older version or vice versa also refresh the local databases.
0.8 alpha 1 did not create correct configurations. When you want to continue
using such a configuration, make sure that in addition to the obviously
wrong syncURLs also the less obvious ScheduleWorld config mistakes are fixed:
* calendar: uri=cal2
* addressbook: type=addressbook:text/vcard
* deviceId must start with "sc-pim-" if you synchronize with myFUNAMBOL,
otherwise there may be unwanted slow syncs when multiple devices with
a different deviceId connect. Note that changing the deviceId causes
a slow sync, so you should get client and server in sync before changing
the value, change it, then do a "--sync refresh-from-server".
SyncEvolution 0.7 -> 0.8 alpha 1, 19.04.2008
--------------------------------------------
C++ client library: a snapshot of the development version
Updating user configuration: this version introduces a new, simplified
configuration layout. Old configurations still work. They can be
converted to the new format via a new "--migrate" command line option.
*** WARNING ***: this version uses a different change tracking for
Evolution calendars, task lists and memos. After switching from a previous
release to the current one or vice versa, do a "syncevolution --sync
refresh-from-server" once to reset the change tracking. Not doing so
can result in applying the same changes to the server multiple times
and thus duplicates.
* New configuration file layout: following the freedesktop.org
recommendation, new configurations are stored in
$XDG_CONFIG_HOME/syncevolution or $HOME/.config/syncevolution if
XDG_CONFIG_HOME is not set. The old layout under
$HOME/.sync4j/evolution is still supported.
* New command line options: new configurations can be created by
syncevolution itself (--configure), including setting of all
configuration properties (--sync-property, --source-property).
The configuration can dumped to stdout (--print-config), with or
without comments explaining each property (--quiet). See the
README for details.
* The "evolutionsource" source property no longer has to be configured.
If left blank, the default client database will be synchronized.
* Selecting which kind of data is to be synchronized under a specific
source name is a lot easier now and the same on all supported
platforms: the SyncEvolution backends can be selected via aliases
(e.g. "contacts") and the format is specified via an optional
MIME type (e.g. "contacts:text/x-vcard"). In the unlikely situation
that multiple backends are active which can synchronize the same
kind of data, then the right one can be selected by the unique
name of the backend (e.g. "Evolution Address Book").
* New configurations automatically get a random client ID string.
Setting it manually is still possible, but no longer necessary.
* All passwords can be read from stdin at runtime or an environment
variable (see "--sync-property password=?" or README for details).
Both avoids the less secure storing of plain text passwords in the
configuration files (SF #1832458).
* Detached recurrences: meeting series where some occurrences were
modified are now supported. Previously only the main event was
synchronized. All exceptions got lost when copying back from the
server. ***WARNING***: such events are accepted by ScheduleWorld,
but not propagated to other clients. Under investigation.
* Time zone support: the time zones of incoming events are mapped
to native time zone definitions whenever possible. Currently
this works if the TZID follows the Olson naming scheme with a
location at the end. Matching the time zone has the advantage of
being able to update the time zone definition without having to
recreate the event. If matching fails and the VTIMEZONE definition
differs from one already imported earlier, then SyncEvolution works
arounds limitation in Evolution by renaming the time zone.
Previously the new event used the old and most likely out-dated
time zone definition.
***WARNING***: Evolution itself does not do either of these steps
itself yet, thus importing meeting invitations via Evolution still
fails in some cases. The code implementing the time zone handling
described above was written with inclusion into Evolution itself in
mind; a discussion with the Evolution developers about that is in
progress.
* On Maemo/Nokia Internet Tablets, calendar synchronization now
works because the new calendar change tracking no longer depends
on some of the backend calls which used to fail (SF #1734977).
* Added SSL configuration options: certificate checking can be
relaxed or disabled completely (SF #1852647).
* Adding support for new local data sources is easier now. The
SyncEvolution frame work itself never depended on GNOME or
Evolution, only the Evolution data sources did. If you want
support for other ways of storing your data, consider writing
a new data source - it is really easy. See EvolutionSyncSource
or TrackingSyncSource for details.
* Messages are printed to the screen immediately. More readable
log file format.
* Various other minor changes and fixes.
SyncEvolution 0.6 -> 0.7, 17.12.2007
------------------------------------
C++ client library: r_6_5_3_1
Updating user configuration: no relevant changes in this release. For those
who haven't done so already, enabling large object support is recommended
(see syncml/config.txt sample configs).
* added port for iPhone and Mac OS X Address Book
* fixed Nokia packaging problem which prevented installation
via the package manager unless it was in "red pill" mode
(SF #1781652)
* sync with eGroupware - lost or messed up telephones: SyncEvolution
incorrectly added TYPE=OTHER to phone numbers sent with e.g.
CELL instead of TYPE=CELL (SF #1796086). Another patch was
required for eGroupware itself to correctly map phone numbers
as sent by SyncEvolution, see Compatibility web page.
* added .deb packages
* adapted calendar event insert/update to Evolution 2.12: the UID needs to be
restored, otherwise the Evolution backend crashes (GNOME issue #488881)
* new feature: if the previous log directory is still available,
then local changes made since last sync can be queried
before starting a sync (new option --status) and will be
printed directly before a sync. Setting the "logdir" option
will automatically keep the most recent logs and database
dumps around.
* added command line options:
--sync|-s <mode>
Temporarily synchronize the active sources in that mode. Useful
for a 'refresh-from-server' or 'refresh-from-client' sync which
clears all data at one end and copies all items from the other.
--status|-t
The changes made to local data since the last synchronization are
shown without starting a new one. This can be used to see in advance
whether the local data needs to be synchronized with the server.
--quiet|-q
Suppresses most of the normal output during a synchronization. The
log file still contains all the information.
--help|-h
Prints usage information.
--version
Prints the SyncEvolution version.
* default configurations now reference the normal Evolution databases
("Personal") thus requiring less changes to use. The account information
is now clearly marked as placeholder which needs to be entered.
* bugfix: vCard 3.0 with mixed case were not converted properly to vCard 2.1
by SyncEvolution (must convert to upper case because vCard 2.1 only allows
that), leading to problems with mapping phone numbers in the Funambol server.
Diagnosed and reported by Paul McDermott, thanks a lot!
* support receiving plain text notes with \n and \r\n line breaks;
always send with \r\n
* added explicit error message when syncevolution is invoked
with incorrect names in the list of sources to synchronize:
previously it silently ignored unknown names
* improved output: less verbose ("extracting" items is now
logged at debug level and thus not normally shown) and more
informative printing of changes (table summarizes number of
changes on client and server, heading for comparison changed
to make it clear that it shows changes on the client)
* SyncCap is not generated unless syncModes are configured: added
a comment to example config (SF #1764123)
* improved error handling: catch errors during post-processing and
continue
SyncEvolution 0.7-pre2 -> 0.7, 17.12.2007
-----------------------------------------
C++ client library: r_6_5_3_1
* bugfix: vCard 3.0 with mixed case were not converted properly to vCard 2.1
by SyncEvolution (must convert to upper case because vCard 2.1 only allows
that), leading to problems with mapping phone numbers in the Funambol server.
Diagnosed and reported by Paul McDermott, thanks a lot!
* support receiving plain text notes with \n and \r\n line breaks;
always send with \r\n
* added explicit error message when syncevolution is invoked
with incorrect names in the list of sources to synchronize:
previously it silently ignored unknown names
* added stack dumping in case of premature abort;
removed workaround for lost connection to Evolution Dataserver
again because the workaround itself caused random segfaults inside
glib
SyncEvolution 0.7-pre1 -> 0.7-pre2, 08.11.2007
----------------------------------------------
C++ client library: branch b_v65
Updating user configuration: no relevant changes in this release. For those
who haven't done so already, enabling large object support is recommended
(see syncml/config.txt sample configs). It is required for myFUNAMBOL to
synchronize very large address books and some users have reported segfaults
unless this option was enabled.
* iPhone bug fix: syncing contacts with photos was unreliable (export) and
crashed (import) because the API had not been called correctly
* iPhone + ScheduleWorld: when configured to use vcard3 (recommended!) then
contacts are exchanged as vCard 3.0
* iPhone + ScheduleWorld bugfix: importing vCard 3.0 did not correctly
classify the phone numbers. A sync with the new "--sync refresh-from-server"
option will fix this, assuming that the server has the correct data.
* Evolution: detect a crashed backend and abort SyncEvolution instead of
hanging forever.
* adapted calendar event insert/update to Evolution 2.12: the UID needs to be
restored, otherwise the Evolution backend crashes (GNOME issue #488881)
* new feature: if the previous log directory is still available,
then local changes made since last sync can be queried
before starting a sync (new option --status) and will be
printed directly before a sync. Setting the "logdir" option
will automatically keep the most recent logs and database
dumps around.
* added command line options:
--sync|-s <mode>
Temporarily synchronize the active sources in that mode. Useful
for a 'refresh-from-server' or 'refresh-from-client' sync which
clears all data at one end and copies all items from the other.
--status|-t
The changes made to local data since the last synchronization are
shown without starting a new one. This can be used to see in advance
whether the local data needs to be synchronized with the server.
--quiet|-q
Suppresses most of the normal output during a synchronization. The
log file still contains all the information.
--help|-h
Prints usage information.
--version
Prints the SyncEvolution version.
* default configurations now reference the normal Evolution databases
("Personal") thus requiring less changes to use. The account information
is now clearly marked as placeholder which needs to be entered.
SyncEvolution 0.6 -> 0.7-pre1, 17.10.2007
-----------------------------------------
* C++ client library: tag "sdkcpp_6_0_9_1" (same as before)
* added support for Mac OS X/iPhone address book
* fixed Nokia packaging problem which prevented installation
via the package manager unless it was in "red pill" mode
* improved output: less verbose ("extracting" items is now
logged at debug level and thus not normally shown) and more
informative printing of changes (table summarizes number of
changes on client and server, heading for comparison changed
to make it clear that it shows changes on the client)
* example configs were in share/share directory (SF #1767329)
* Nokia 770/800: uninstallable package fixed by setting category
(SF #1781652)
* sync with eGroupware - lost or messed up telephones: SyncEvolution
incorrectly added TYPE=OTHER to phone numbers sent with e.g.
CELL instead of TYPE=CELL (SF #1796086). Another patch was
required for eGroupware itself to correctly map phone numbers
as sent by SyncEvolution, see Compatibility web page.
* SyncCap is not generated unless syncModes are configured: added
a comment to example config (SF #1764123)
* improved error handling: catch errors during post-processing and
continue
SyncEvolution 0.5 -> 0.6, 13.07.2007
------------------------------------
* C++ client library: tag "sdkcpp_6_0_9_1"
* added support for synchronizing Evolution notes (aka memos) as
plain text where the first line serves as summary; this is the
format understood by ScheduleWorld
* added support for synchronizing Evolution notes (aka memos) as
iCal 2.0 journal; not currently supported by any server and
untested
* revamped example configs and documentation: only one set of
config files for each server is provided, because this is more
likely to be needed by users
* example configs are now installed in share/doc/syncevolution,
enabled message limit and large object support in them
* added support for Nokia 770/800 (aka Maemo):
built with loadable modules so that it works with whatever
backends are installed, improved log handling to accomodate
for limited space on filesystem (see below), some workarounds
* added workaround for Nokia 770:
contacts are not really deleted unless the EDS-Sync with
instant messaging servers is activated; now SyncEvolution
will delete contacts marked as deleted by the GUI before
a sync if it finds any.
WARNING: if you use EDS-Sync and SyncEvolution, then give
EDS-Sync enough time after going online to finish its own
synchronization of modified/deleted contacts before starting
SyncEvolution.
* improved log handling: writing log and database dumps can be
disabled with "logdir=none", verbosity of log is controlled by
"loglevel", better handling of errors during initial database
access
* added workaround for Evolution bug #455274:
the separator for multiple categories in events and tasks
is not generated and interpreted according to iCalendar 2.0
by Evolution; as a consequence of that items sent to the server
had all categories merged into one and items imported into
Evolution only used one of the catories
http://bugzilla.gnome.org/show_bug.cgi?id=455274
* fixed off-by-one counting of months in backup directory names
* fixed error handling: a failed source was not forced into a slow
sync as required; one failed source prevented saving configs of
not-failed ones and thus forced those into an unnecessary slow
sync
* uses the Funambol C++ testing framework (which is based on the
previous SyncEvolution testing); now creates its configs
and (when using CLIENT_TEST_EVOLUTION_PREFIX=file://<path>)
also the Evolution databases automatically
* implemented synccompare as pure Perl script using Algorithm::Diff
instead of external diff tool
* synccompare did not figure out width of shell window as it should have
* better error handling if creating the before/after database dumps
fails (SF #1685637)
* workaround for Funambol 3.0 trailing = parser bug
UPGRADING
Old config files from 0.5 or older continue to work, but it is recommended
to set the following options to enable message size limits:
maxMsgSize = 8192
maxObjSize = 500000
loSupport = 1
SyncEvolution 0.6pre2 -> 0.6, 13.07.2007
----------------------------------------
* improved README/HACKING documents
* fixed the new example configs: use event/task for Funambol 6.0,
name was wrong
* added workaround for Evolution bug #455274:
the separator for multiple categories in events and tasks
is not generated and interpreted according to iCalendar 2.0
by Evolution; as a consequence of that items sent to the server
had all categories merged into one and items imported into
Evolution only used one of the catories
http://bugzilla.gnome.org/show_bug.cgi?id=455274
* added workaround for Nokia 770:
contacts are not really deleted unless the EDS-Sync with
instant messaging servers is activated; now SyncEvolution
will delete contacts marked as deleted by the GUI before
a sync if it finds any.
WARNING: if you use EDS-Sync and SyncEvolution, then give
EDS-Sync enough time after going online to finish its own
synchronization of modified/deleted contacts before starting
SyncEvolution.
SyncEvolution 0.6pre1 -> 0.6pre2, 23.04.2006
--------------------------------------------
* C++ client library: tag "sdkcpp_6_0_7" + revision 1.7 of
build/autotools/test/Makefile.am
* added support for synchronizing Evolution notes (aka memos) as
plain text where the first line serves as summary; this is the
format understood by ScheduleWorld, not the iCal 2.0 format
added in 0.6pre1
* improved log handling: writing log and database dumps can be
disabled with "logdir=none", verbosity of log is controled by
"loglevel", better handling of errors during initial database
access
* fixed off-by-one counting of months in backup directory names
* fixed error handling: a failed source was not forced into a slow
sync as required; one failed source prevented saving configs of
not-failed ones and thus forced those into an unnecessary slow
sync
* revamped example configs: only one set of config files for each
server is provided, because this is more likely to be needed
by users
* uses the Funambol C++ testing framework (which is based on the
previous SyncEvolution testing); now creates its configs
and (when using CLIENT_TEST_EVOLUTION_PREFIX=file://<path>)
also the Evolution databases automatically
SyncEvolution 0.5 -> 0.6pre1, 26.03.2006
----------------------------------------
* C++ client library: CVS snapshot from 26.03.2006
* added support for synchronizing Evolution notes (aka memos) as
iCal 2.0 journal
* added --enable-static-cxa = linking C++ runtime statically:
binaries produced for 0.6 will have less external
dependencies than the 0.5 binaries
* added hacks for Maemo/Nokia 770, including a build
mode with dynamically loadable modules (--enable-shared, --enable-maemo,
--with-patched-dbus)
* implemented synccompare as pure Perl script using Algorithm::Diff
instead of external diff tool
* synccompare did not figure out width of shell window as it should have
* better error handling if creating the before/after database dumps
fails (SF #1685637)
* example configs are now installed in share/doc/syncevolution,
enabled message limit and large object support in them
* workaround for Funambol 3.0 trailing = parser bug
UPGRADING
Old config files continue to work, but it is recommended
to set the following options to enable message size limits:
maxMsgSize = 8192
maxObjSize = 500000
loSupport = 1
SyncEvolution 0.4 -> 0.5, 12.11.2006
------------------------------------
* C++ client library revision "syncevolution-0-5":
- added support for sending changes in smaller chunks
("Large Object Support"): disabled by default, see updated
example configuration
- time is printed with GMT offset so that a server admin in
a different timezone can always figure out how a client log
relates to events on the server
- special item keys as they might be stored in some calendars after
importing non-Evolution events are now properly supported
* bug fix: in 0.4 it was necessary to manually configure the verDTD
or the Funambol 3.0a server would choke on the invalid SyncML during
the second synchronization with SyncEvolution; now this option is set
automatically
* added support and testing of transmitting just the changes
from client to server or vice versa; see "one-way-from-server/client"
in example configuration
* fixed/updated comments in the example configuration
* improved automated testing and fixed the problem that CPPUnit was not
found unless it was part of the system
* Now works on Maemo/Nokia 770: minor changes were necessary so that
the system address book can now be selected under the name "<<system>>.
Copying 300 contacts into the Nokia 770 went fine, but any further
attempt to synchronize suffered from timeouts inside the embedded
Evolution Data Server.
SyncEvolution 0.3 -> 0.4, 11.09.2006
------------------------------------
* C++ client library revision "syncevolution-0-4":
- added support for device information, required by some servers
- fixed incompatibilities with non-Funambol servers
- the user agent string can now be modified in the
spds/syncml/config.txt, but it is recommended to not set
it explicitly. Then SyncEvolution will automatically insert
its current version.
- #305795: for tasks the "text/x-todo" type from the configuration
was sent to servers instead of the correct "text/calendar"
provided by SyncEvolution itself
- sync modes "refresh-client/server" can now be specified as
"refresh-from-client/server" in the config
* updated default syncml/config.txt:
- firstTimeSyncMode has never been implemented in the library,
removed its documentation,
- added documentation for userAgent
- use "refresh-from-client/server"
* SF issue 1511951: support copying changes back from EGroupware
server by not expecting the UID of calendar items to be unmodified
* fixed a bug where after a refresh-from-client sync changes would
be sent to the server again during a two-way sync although the
server already had them
* implemented authentication for Evolution databases
* synccompare was removing too many parts of vCards with
single-value ORG properties
* improved error reporting when selected server is not configured
* changed vCard parser to make it compatible with servers
which send a verbatim semicolon as part of properties where
the semicolon has no special meaning
* If minor errors occur like not being able to insert an
item at the client or server side, then it is reported in the
log and output, but the next synchronization will be a normal
synchronization, not a forced slow one as in previous versions.
The old approach ensured that the problem was noticed and fixed,
but required user assistance. With the new approach synchronization
continues to work, although without fixing the root cause of
the problem.
* Workaround for bug in Evolution 2.0.6 (and perhaps other versions):
for calendars and task lists not all deleted items were reported
at once thus a single synchronization would only tell the server
about a subset of the changes. Repeating the synchronization would
eventually be told of all changes, so now this repetition is built
into the code which queries for changes and a single synchronization
is sufficient as it should be.
SyncEvolution 0.4 pre2 -> 0.4, 11.09.2006
-----------------------------------------
* adapted to C++ client library from CVS head, tagged as syncevolution-0-4:
devinfo.patch patch was merged with several changes to the API
* SF issue 1511951: support copying changes back from EGroupware
server by not expecting the UID of calendar items to be unmodified
SyncEvolution 0.4 pre1 -> pre2, 21.08.2006
------------------------------------------
* C++ client library revision "syncevolution-0-4-pre2":
most patches were merged into CVS head, but .patches/devinfo.patch
still needs to be applied manually when checking out from the
Funambol CVS instead of using the bundled version
* fixed a bug where after a refresh-from-client sync changes would
be sent to the server again during a two-way sync although the
server already had them
* implemented authentication for Evolution databases
* synccompare was removing too many parts of vCards with
single-value ORG properties
* improved error reporting when selected server is not configured
* use 7-bit quoted-printable encoding with explicit UTF-8 charset for
vCard 2.1 to avoid any potential confusion about the content; not
really necessary because SyncML specifies 8-bit UTF-8 as the default
* fix for 0.4 pre 1: sending CHARSET is not allowed (and not
needed) for vCard 3.0, so it was removed again (did not harm
either)
* fix for 0.4 pre 1: sending vCard 2.1 to Synthesis server did
not work because the new device info always mentioned 3.0 as
the preferred format - now the preferred format matches the one
that was configured and that thus will be used.
SyncEvolution 0.3 -> 0.4 pre 1, 2006-08-06
------------------------------------------
* C++ client library revision "funambol30ga" plus the patches
stored in its ".patches" directory:
- the user agent string can now be modified in the
spds/syncml/config.txt, but it is recommended to not set
it explicitly. Then SyncEvolution will automatically insert
its current version.
- now compatible with additional servers (fixed some SyncML
protocol issues, added support for sending device
information)
- revised API of the client library
- #305795: for tasks the "text/x-todo" type from the configuration
was sent to servers instead of the correct "text/calendar"
provided by SyncEvolution itself
- sync modes "refresh-client/server" can now be specified as
"refresh-from-client/server" in the config
* updated default syncml/config.txt:
- firstTimeSyncMode has never been implemented in the library,
removed its documentation,
- added documentation for userAgent
- use "refresh-from-client/server"
* changed vCard parser to make it compatible with servers
which send a verbatim semicolon as part of properties where
the semicolon has no special meaning
* If minor errors occur like not being able to insert an
item at the client or server side, then it is reported in the
log and output, but the next synchronization will be a normal
synchronization, not a forced slow one as in previous versions.
The old approach ensured that the problem was noticed and fixed,
but required user assistance. With the new approach synchronization
continues to work, although without fixing the root cause of
the problem.
* Workaround for bug in Evolution 2.0.6 (and perhaps other versions):
for calendars and task lists not all deleted items were reported
at once thus a single synchronization would only tell the server
about a subset of the changes. Repeating the synchronization would
eventually be told of all changes, so now this repetition is built
into the code which queries for changes and a single synchronization
is sufficient as it should be.
* Made it compile on Maemo 2.0, the Nokia 770 build environment, by
adding "--disable-ecal". Not tested yet, though.
SyncEvolution 0.3, 2006-06-27
-----------------------------
* added syncing of calendars and tasks as iCalendar 2.0
* added syncing of contacts as vCard 3.0
* tested extensively with sync.scheduleworld.com and
added an example configuration for it
* uses C++ client library revision "wmplugin_3_0_20"
which contains several bug fixes, among them proper
support for special characters and memory handling
fixes
* much nicer listing of changes made during a sync,
handled by the improved "synccompare" utility script
(formerly known as "normalize_vcard")
* improved automated testing
SyncEvolution 0.2, 2006-03-19
-----------------------------
* added automatic backup mechanism and log storage,
see "Automatic Backups and Logging".
* output no longer is the original log data, but rather
a human-readable report of errors and synchronization
results.
* "normalize_vcard" can now also compare two .vcf files
directly.
* improved unit tests to catch more errors
* hide certain differences in vcards coming back from
the server: duplication of extended vcard properties,
missing TYPE=OTHER
* fixed client library problems:
see http://forge.objectweb.org/tracker/?group_id=96&atid=100096
#304792, #304829
* added some more problems to the "Known Problems" section
SyncEvolution 0.1, 2006-03-13
-----------------------------
* initial release
Reference in New Issue View Git Blame Copy Permalink