Commit graph

3411 commits

Author SHA1 Message Date
Patrick Ohly
31e286059b eGroupware: added template (BMC #15992)
The problem with eGroupware is that it doesn't have one fixed syncURL.
Users of the template must remember to change it. To make this more
obvious, it is set to http://set.your.domain.here/rpc.php

The failure to fix the URL is leading to fairly obvious network
errors.

Various people have reported success with eGroupware, the latest
one here: http://www.ruinelli.ch/how-to-sync-egroupware-with-a-tablet-n900-with-syncevolution
Therefore it makes sense to also mark this template as "consumerReady"
and have it shown in the sync UI.
2011-04-15 08:15:19 +02:00
Patrick Ohly
29da40384a WebDAV: detect temporary Google error page
When Google calendar has temporary problems, it redirects to
an HTML error page. That defeated our "retry on error" logic.
Now redirects are checked inside Session::check(), with a special
case in place for this Google redirect.
2011-04-14 13:45:56 +02:00
Patrick Ohly
54f03a0905 logging: avoid extra newlines in redirected text
When capturing lines from stdout or stderr, strip the trailing newline
before calling message() because that introduces an additional newline
by itself.
2011-04-14 13:42:18 +02:00
Patrick Ohly
70f7a924cc Logging: add relative time to [INFO/DEBUG ...] prefix at debug level
It is useful to know how long certain operations take. Instead of
adding random time stamps at the log calls, this should better be
done consistently for all lines.

In a normal sync log, such a time stamp is added by the Synthesis
logging code. For stdout logging, this patch adds it if the log
level is >= DEBUG. That way it won't distract normal users but will
be available automatically when producing a log for developers.

Because a full time dump can be long and because the delta between
events is more interesting, only relative time since the start of
logging is printed.
2011-04-14 11:12:08 +02:00
Patrick Ohly
106598bd16 WebDAV: added extensive resending of requests
This is motivated by the observation that the Yahoo server becomes
unresponsive (various 5xx errors) quite often, which used to abort
the sync. Now there are better chances that it'll complete eventually,
although the root cause (server not responsive enough) remains.

PROPFIND, PUT and GET are all tried again. For a failed PUT this is
problematic because it is simply not known whether the command was
already executed. A check for "item does not exist" therefore can only
be done on the first attempt. Likewise, any future "eTag matches"
check (not yet implemented) will be also impossible.

Because of the retry, a PUT is no longer marked as not
idem-potent. The advantage is that an existing connection will be
reused for it.

The implementation is based around the idea that the higher levels in
the stack define a deadline until which the operation must succeed,
and then the lower levels (individual PROPFIND/PUT/GET) retry inside
that allowed time interval. That ensures that the initial service
discovery never takes longer that the configured timeout, which
wouldn't be the case if the timeout was applied to every single
PROPFIND inside the semantic operation.
2011-04-14 10:17:50 +02:00
Patrick Ohly
0d96fa07d3 Akonadi: fixed a compile problem when it was disabled
Without Akonadi support enabled, SyncSource.h was never included
and thus the registration code didn't compile.
2011-04-14 10:17:50 +02:00
Patrick Ohly
707b04f67e KWallet: do not enable by default
This is motivated by the observation that KWallet was unintentionally
enabled just because the header file had been installed. It then
also failed to compile because of missing Qt compile flags (separate
problem), but even if it had successed, the extra library dependency
wasn't wanted in that case, so better request an explicit --enable
before enabling the feature.
2011-04-14 15:58:53 +08:00
Patrick Ohly
8ec73f76ea syncevo-dbus-server: fixed a compile problem with KWallet + libnotify
When both KWallet and libnotify support were enabled, the redefinition
of "signals" in the Qt headers broke the GTK headers that are pulled in
by notify.h, because one of the GTK structs has a "signals" member.

Fixed by asking Qt to not define these special keywords as
preprocessor symbols.
2011-04-13 10:34:14 +02:00
Patrick Ohly
1c9d99414a vcard profile: simplified REV workaround
The scripting code seemed a bit overly complex. There shouldn't be
a need for the temporary variable.

The original patch also had the side-effect of always setting
REV to its normalized version, even if it wasn't empty. That
may or may not influence other syncs, but it is better to not
take the risk.
2011-04-13 10:34:14 +02:00
Dinesh
f531648bb4 Akonadi: preliminary support for KJot
Added preliminary Support for KJots's custom format of storing Akonadi Notes.
Currently it is assumed that all the notes it stores is of Plain Text format
(as happens in the AkoNotes Plasmoid).

Later on will have to dig into implementing support for notes in HTML and other
formats too?
2011-04-13 10:34:14 +02:00
Dinesh
a3d7412972 vCard profile: added preliminary support for KAddressBook extensions
Problems yet to be solved:
  1) Support for KBC's custom fields.

  2) The 1st ; in ADR is causing it to leave out address as a blank....
    ADR;TYPE=dom;TYPE=home;TYPE=intl;TYPE=parcel;TYPE=postal;TYPE=pref;TYPE=work:123;;Planet Earth;Planet Earth;Planet Earth;Planet Earth;Home

  3) "\" being expanded to "\\" and "," ot "\," in X-KADDRESSBOOK-CRYPTOPREF (not creating any problems though)

also URL is being deleted locally , but appearing remotely,
EMAIL is being replaced by FN , when tested against Ovi servers.
2011-04-13 10:34:14 +02:00
Dinesh
0cb2aa6a30 Akonadi: fixed issue with running inside syncevo-dbus-server
KApplication starts of it's own DBus service of the format
org.syncevolution.syncevolution-processId
Sending a patch upstream to stop this needs a lot more work and
should be out by KDE 4.8 (hope so).
Also
cleaned up the ugly
if (getenv("KDE_FULL_SESSION")!= true)
with
if (getenv("KDE_FULL_SESSION"))
2011-04-13 10:34:14 +02:00
Patrick Ohly
3dc7b5fb17 added .gitignore entries for locally created files
This should be enough to keep "git" happy after a compilation inside
the source directory. Kudos to Sascha Peilicke for the original
patch.
2011-04-13 10:34:14 +02:00
Dinesh
b3a9655671 Akonadi: fixed startup issues
KApplication is not meant to be used as a replacement for
QApplication. Hence this patch even fixes some unexpected
segmentation faults.
2011-04-13 10:34:14 +02:00
Dinesh
1bf9c2125a vcard profile: work around REV issues in Akonadi <-> Ovi/Mobical sync
These servers need VCards to have a Revision string. Contacts
created in KABC don't have one by default. So this patch appends a
dummy timestamp to the outgoing VCards if the RevString is empty.
2011-04-13 10:34:14 +02:00
Dinesh
ab25aef728 command line + syncevo-dbus-server: added support for KWallet
Currently not enabled in compilation by default. If KWallet support is
enabled, it will check whether KDE is running and otherwise fall back
to GNOME keyring.
2011-04-13 10:34:13 +02:00
Dinesh
c9458d29bd Akonadi: first functional version
TODOs resolved, adapted to backend API changes.
2011-04-13 10:34:07 +02:00
Patrick Ohly
2ec870e3e5 Akonadi: use X-KADDRESSBOOK vcard extensions
On the backend side, use a new "KDE" rule to enable only
the X-KADDRESSBOOK- variant of ASSISTENT/MANAGER/SPOUSE/ANNIVERSARY.
On the SyncML side, continue to use both X- and X-EVOLUTION-
because that is what SyncML servers understand.
2011-04-13 10:32:55 +02:00
Patrick Ohly
e84e385bb8 rewrite of Akonadi backend for SyncEvolution
This patch adds the necessary boiler-plate to compile
Sascha's Akonadi backend as part of SyncEvolution.

It changes the change tracking so that it is now based on the revision
number maintained by Akonadi. This removes the need to keep the sync
process running all the time to record changes.

Finding local databases (= collections) is implemented inside the
backend, with several TODOs in the code to make this nicer.

Tests were added as part of copying the boiler-plate code from the
Evolution backend. However, this depends on being able to open local
databases following the name pattern <prefix>_<type>_[12], with
<prefix> from CLIENT_TEST_EVOLUTION_PREFIX and <type> one of
ical20/vcard30/itodo20/text. This does not work at the moment.

Because the implementation of isEmpty() always returns "false", the
logic which checks whether a slow sync is acceptable will err on the
side of caution and reject slow sync, even if the local side has no
data.
2011-04-13 10:32:55 +02:00
Sascha Peilicke
6c48f78743 added Akonadi/KDE backend
This is a verbatim copy of svn://anonsvn.kde.org/home/kde/trunk/playground/pim/syncml/clientagent/funambol
revision 1072301, the latest as of January 9th 2010.

The source implements the Funambol C++ client API SyncSource API,
accessing Akonadi as the underlying database. Change tracking is based
on the end time stamp of the last sync.

The source still needs to be adapted to SyncEvolution before it can
be used there.
2011-04-13 10:32:55 +02:00
Patrick Ohly
9f18a93e1c curl transport: fixed compile error
Fixed a compile problem introduced earlier when replacing time_t with
Timespec in m_sendStartTime.
2011-04-13 10:32:55 +02:00
Patrick Ohly
07c65d665c nightly testing: Apple Calendar server has problems with \n
The CardDAV vCard parser in the Apple Calendar server chokes
on \n in ADR. Filed a bug: http://trac.calendarserver.org/ticket/436

In the meantime let's simplify the problematic test case.
2011-04-12 22:55:41 +02:00
Patrick Ohly
4923486be8 nightly testing: use special iCalendar 2.0 test cases for Apple Calendar server
The only difference at this time is that the server requires EXDATE to
be a DATETIME value. Need to check whether Synthesis engine produces
such values (probably only passes through what is generated in Evolution);
in the meantime, use test cases which have DATETIME.
2011-04-12 22:55:41 +02:00
Patrick Ohly
2d62fcb6cf synccompare: ignore some Apple Calendar Server differences
Apple Calendar server adds/removes some additional parameters in
calendar events. Ignore them:
- SCHEDULE-AGENT=NONE and SCHEDULE-STATUS=5.3 for ORGANIZER
- a maximum count of 400 for daily recurrence
- VALUE=DURATION is not set explicitly
- X-VOBJ-FLOATINGTIME-ALLOWED=YES in CREATED
2011-04-12 22:55:41 +02:00
Patrick Ohly
0e999e3551 nightly testing: UNTIL clause fix for Apple Calendar Server
Apple Calendar server complains that the UNTIL clause in one test case
had a DATE value whereas the DTSTART/DTEND had UTC DATE-TIME. Not sure
what the standard says, so let's believe Apple and fix the test case
to have consistent DATE-TIME values.

In fact, a lot of SyncML peers already turned the DATE UNTIL clause
into something with DATE-TIME. Removed all patches which added these
times to the peer-specific test cases.
2011-04-12 22:55:41 +02:00
Patrick Ohly
6ad8835cd5 nightly testing: removed METHOD:PUBLISH, part II
CalDAV requires that calendar items have no METHOD property in the
calendar. Creating an item via the Synthesis engine (and thus local
sync) meet that requirement, but the test data didn't. Removing METHOD
in the test data. Should not affect other uses of the tests.

The previous commit (1f3b6c4fcd) only
addressed test data in ClientTest.cpp. This patch also fixes
ical20.ics.
2011-04-12 22:55:41 +02:00
Patrick Ohly
be95fddac5 nightly testing: added Apple Calendar Server testing
runtests.py now has an entry for Apple Calendar Server,
with both CardDAV and CalDAV. The actual installation
can be local. To use this test, set up source-config@client-test-apple
with the correct syncURL and credentials.
2011-04-12 22:37:09 +08:00
Patrick Ohly
0e24eb7186 nightly testing: fixed names and links for sources with underscore
A previous fix for calendar+todo in the source name introduced
escaping of the special + character. This escaping also replaced _
with __. Unescaping was missing in several places, leading to
google_caldav being shown as google__caldav with links to non-existent
files Client_Source_google_caldav_testOpen.log.
2011-04-12 11:30:40 +02:00
Patrick Ohly
2b9fdd0675 nightly test: include Client::Sync::*::testTimeout in report
This test is a bit special because it is only run for the first source
and doesn't really talk to the server. The logic in resultchecker.py
missed that test because the invocation of "client-test --help" didn't
print that test for "vcard21". Fixed by ensuring that only "vcard21"
is active during the --help call.
2011-04-12 11:25:59 +02:00
Patrick Ohly
f438e5327c nightly testing: show source test results sorted by time
The Client::Source tests were sorted by name. This was confusing
because it hides when a peer like a CalDAV server stops responding
after a certain test.

It is more intuitive to present the results in the order in which they
were run. This is accomplished by sorting by the sub-second modification
time of the corresponding .log files (in resultschecker.py) and keeping
that order (in generate-html.xsl).

A similar approach might also be useful for Client::Sync. Not done yet.
2011-04-12 11:25:09 +02:00
Patrick Ohly
ae2735f2b8 autotools: fixed linking with clang 2.9
clang 2.9 does not support -u<symbol> and ignores it, leading to incomplete
client-test and static syncevolution binaries. Fixed by explicitly passing it
to the linker stage with -Wl,-u.
2011-04-11 21:05:35 +02:00
Patrick Ohly
95b5ce1026 nightly testing: increase timeouts in Client::Sync::*::testTimeout
The timing under valgrind is sufficiently different to cause the
comparison to fail. Increased the total duration and acceptable delta
to make it work under valgrind.
2011-04-11 21:05:35 +02:00
Patrick Ohly
1276a4bd3f syncevo-dbus-server: avoid method name clash
clang 2.9 pointed out that getConnection() is overloaded with
different method signatures. One is about low-level D-Bus, the other
about (HTTP) server stub communication. Distinguish that by calling
the more high-level concept "stub connection".
2011-04-11 21:05:35 +02:00
Patrick Ohly
ae8a09b1ce SyncConfig: "username/password" ambiguous, changed to "sync username/password"
The methods related to sync username/password were just called
get/setUsername/Password(), with additional check/savePassword()
methods. The savePassword() method clashed with a semantically
different method for arbitrary passwords (as pointed out by clang
2.9), so let's make it more clear that one set of methods specifically
is for the sync username/password.
2011-04-11 21:05:35 +02:00
Patrick Ohly
3fe45cb94b SyncConfig: clang 2.9 compiler warning
clang 2.9 warns about comparisons which are always true. For the
range check in a template that cannot be avoided because the type
may or may not be unsigned. Suppress the warning with clang pragmas.
2011-04-11 21:05:35 +02:00
Patrick Ohly
ff89518634 SyncContext testing: wrong signature of virtual method
clang 2.9 pointed out that getLogDir() didn't have the desired effect
anymore after a signature change in the base class. Not sure why the
test passed regardless of that, but it was indeed broken.
2011-04-11 21:05:34 +02:00
Patrick Ohly
077ffa1844 nightly testing: more valgrind suppressions
More aggressively suppress errors in libsoup, libical and glib. Clean
run of sync test on Debian Squeeze.
2011-04-11 21:05:34 +02:00
Patrick Ohly
1f3b6c4fcd nightly testing: removed METHOD:PUBLISH
CalDAV requires that calendar items have no METHOD property in the
calendar. Creating an item via the Synthesis engine (and thus local
sync) meet that requirement, but the test data didn't. Removing METHOD
in the test data. Should not affect other uses of the tests.
2011-04-11 21:05:34 +02:00
Patrick Ohly
54aadcd6d7 runtest.py: "prebuilt" now only allowed once, replaces compiling
Previously, the --prebuilt option could be specified multiple times.
This was never used and just made its implementation unecessarily complex. Now
this option overrides compilation and leaves the rest of the test
running unmodified.
2011-04-05 18:56:04 +08:00
Patrick Ohly
3ad1911705 runtests.py: added Yahoo and Google CalDAV/CardDAV testing
Added separate Client::Source tests for Yahoo CalDAV/CardDAV
and Google CalDAV. Reporting the results needs to know that these
"servers" are local tests.

These tests can only be run if CLIENT_TEST_WEBDAV is set as
expected by WebDAVSourceRegister.cpp.
2011-04-01 18:26:37 +08:00
Patrick Ohly
b64763ee53 runtest.py: --help was broken without --enable
Not setting --enable caused a Python error because the "enable"
member of the config wasn't set at all. Must set a default.
2011-04-01 18:26:37 +08:00
Patrick Ohly
998dac31af WebDAV: fixed tests involving second local source
The callback for "source B" wasn't bound to the creator method,
which was the originall intention. Therefore such sources
were created with the default method and incomplete configs,
breaking some of the Client::Source tests.
2011-04-01 18:11:06 +08:00
Patrick Ohly
9e776311a9 WebDAV: fixed some memory issues
Found in nightly testing with Valgrind. One is a boolean check on an
uninitialized variable which might have caused Basic authorization to
be sent when not desired, the other a harmless end-of run leak.
2011-03-29 08:59:55 +02:00
Patrick Ohly
d3e9177670 WebDAV: more detailed error reporting
Some error scenarios, like "Google user has no calendar => 404 error",
only led to a general 20043 "communication error". This patch
preserves the remote status code by passing it from the request to
Session::check() where it can be passed on as a status exception.
2011-03-28 22:05:50 +02:00
Patrick Ohly
b9cd6e6f3b nightly testing: include webdav backend in PATH
The webdav directory contains syncevo-webdav-lookup, which is
needed for the WebDAV tests.
2011-03-29 03:54:33 +08:00
Patrick Ohly
eaeebb77e7 nightly testing: run setup command only once
The shell in Ubuntu Hardy was seen to not obey || && priorities.
Using explicit brackets solved that.
2011-03-29 03:54:33 +08:00
Patrick Ohly
59ebdb158b nightly testing: run "make clean" in same environment as testing
This was an attempt to avoid issues with the autotools scripts
and rebuilding their files. It did not solve the problem entirely
(--disable-maintainer-mode eventually did), but the change makes
sense nevertheless, for example of the directory inside a chroot
is different than the one in the host.
2011-03-29 03:54:33 +08:00
Patrick Ohly
54c2204d68 nightly testing: include CLIENT_TEST_WEBDAV in log
Rerunning the commands needs to use the same CLIENT_TEST_WEBDAV
env variable. Include it in the log output, similar to the valgrind
env variables.
2011-03-29 03:54:33 +08:00
Patrick Ohly
6fd1cfdfec glib CXX: avoid features not found on Ubuntu Hardy
Ubuntu Hardy's boost does not have reset() in intrusive_ptr.
Can be replaced with constructing a ptr followed by swap().
glib lacks the flag for combining certain change events. We don't
need this (although it reduces overhead a bit), so let's simply drop it.
2011-03-29 03:51:38 +08:00
Patrick Ohly
94de713fa5 glib CXX: fixed compile problem
g++ on Ubuntu Hardy did not agree with clang++ on scoping rules.
Both seem to work with global ref/unref functions (outside of
boost namespace), so let's use that.
2011-03-29 03:49:41 +08:00