The WebDAV backends contained a hack where the UID inside the data was forced
to be identical to the resource name. This is wrong for items created by us
via POST (because the server may choose a resource name != UID) or by some
other entity (where we have no idea how the resource name got chosen).
This commit removes the hack. Testing must be updated to pass correct data
with the same UID as on the server when updating an item, because the backend
will no longer ensure that and changing the UID of a resource gets rejected by
some servers.
The hack was introduced for peers which do not store the UID (for example, a
vCard or iCalendar 1.0 based SyncML client). A better solution must be found,
probably involving the Synthesis engine and its read/update/write cycle.
PUT has the disadvantage that a client needs to choose a name and then
figure out what the real name on the server is. With Google CardDAV that
requires sending another request and only works because the server happens
to remember the original name (which is not guaranteed!).
POST works for new items without a name and happens to be implemented
by Google such that the response already includes all required
information (new name and revision string).
POST is checked for as described in RFC 5995 once before creating a new
item. Servers which don't support it continue to get a PUT.
The 403 status code must be checked to detect UID conflicts when adding
an item that already exists on the server.
The life time and content of the member was not defined. It got used
in multiple places. It's cleaner to bind the openPropCallback() to
a local Props_t instance.
The "addressbook" source was written for the original iPhone and
Mac OS X. It hasn't been in use for a long time, keeping the code
around without actually compiling it makes no sense anymore.
The "--update <dir name>" operation was supposed to take the
item luids from the file names inside the directory. That part
had not been implemented, turning the operation accidentally
into an "--import".
Also missing was the escaping/unescaping of luids. Now the
same escaping is done as in command line output and command
line parsing to make the luids safe for use as file name.
When running Akonadi with the reference home dir, it creates
a socket symlink that we must not copy. Otherwise all Akonadi
instances end up sharing the same Unix domain socket name and
thus Akonadi instance.
When testing for specific output of shell tools, we need to ensure that the
system locale (typically engineering English) is active, otherwise we end
up with mismatches when the error messages get translated.
The macro was supposed to help Clang scan-build detect that error
paths are not taken. However, it caused the expression to be
evaluated twice, which is not okay for some more recent checks.
Instead we now enable either the real assert or our simplified check,
depending on a custom CT_ASSERT_SIMPLE that has to be set when running with
scan-build.
The same applies to "no throw" asserts. To keep the wrapper macros
simple, we pass the same statement twice.
The --shell value used for Clang build-scan builds includes
scan-build and an additional wrapper script. These extra commands
break result checking. Instead of trying to filter them out,
use the new, optional --simple-shell.
When aborting, our AbortHandler gets called to close down logging.
This may involve memory allocation, which is unsafe. In FDO #76375, a
deadlock on a libc mutex was seen.
To ensure that the process shuts down anyway, install an alarm and give
the process five seconds to shut down before the SIGALRM signal will kill
it.
The information that the sync is freezing is now also handed down to
the transport and all sources. In the case of PBAP caching, the local
transport notifies the child where the PBAP source then uses Bluez
5.15 Transfer1.Suspend/Resume to freeze/thaw the actual OBEX transfer.
If that fails (for example, not implemented because Bluez is too old
or the transfer is still queueing), then the transfer gets cancelled
and the entire sync fails. This is desirable for PBAP caching and
Bluetooth because a failed sync can easily be recovered from (just
start it again) and the overall goal is to free up Bluetooth bandwidth
quickly.
When using GIO, it is possible to avoid the DBusServer listening on a
publicly accessible address. Connection setup becomes more reliable,
too, because the D-Bus server side can detect that a child died
because the connection will be closed.
When using libdbus, the traditional server/listen and client/connect
model is still used. GDBus GIO mimicks the same API to minimize
changes in ForkExec. The API gets simplified to support both
approaches:
- Choosing an address is no longer possible (was only used in the
test example anyway)
- The new connection callback must be specified already when starting
the server.
The GDBus GIO implementation uses SyncEvolution utility code. Strictly
speaking, this creates a circular dependency
libsyncevolution<->libgdbus. This is solved by not linking libgdbus
against libsyncevolution and expecting executables to do that instead.
The GDBus GIO example no longer linked because of that; it gets removed
because it was of little value.
Now that we started using CLOEXEC, we might as well use it for the
permanent file descriptors created for each ForkExec instance.
g_clear_object() unnecessarily creates a dependency on a more recent glib
version. It probably isn't used correctly here anyway (g_clear_object() checks
for NULL itself, so the code around it doesn't need and should do that), so we
can use the older g_object_unref() instead.
The code is a verbatim copy of the corresponding files from gSSO. The
only difference is that we compile the implementation as C++ code,
because that is what we set compile flags for in libsyncevolution.
This helper class will be used for D-Bus over plain files in
combination with g_dbus_connection_new().
Currently implements the new API by stopping to consume data on the
local side of the sync. The Bluetooth transfer on the remote side
continues to run and isn't even notified about the suspend; need a new
obexd API to also suspend that and then notify the remote side when we
want it to do that.
This adds GetPeerStatus() and "progress" events.
To detect DB changes as they happen, the SyncSource operations are
monitored. Upon entry, a counter gets increased and transmitted
through to the PIM manager in syncevo-dbus-server using extended
SourceProgress structs (only visible internally - public API must not
be extended!). This will count operations which fail and count those
twice which get resumed, so the counts will be too high
occasionally. That is in line with the API definition; because it is
not exact, the API only exposes a "modified" flag.
Progress is reported based on the "item received" Synthesis event and
the total item count. A modified libsynthesis is needed where the
SyncML binfile client on the target side of the local sync actually
sends the total item count (via NumberOfChanges). This cannot be done
yet right at the start of the sync, only the second SyncML message
will have it. That is acceptable, because completion is reached very
quickly anyway for syncs involving only one message.
At the moment, SyncContext::displaySourceProgress() holds back "item
received" events until a different event needs to be emitted. Progress
reporting might get more fine-grained when adding allowing held back
events to be emitted at a fixed rate, every 0.1s. This is not done yet
because it seems to work well enough already.
For testing and demonstration purposes, sync.py gets command line
arguments for setting progress frequency and showing progress either
via listening to signals or polling.
The code was meant to check whether a contact really has contact
data; because the if check was missing, the sync result checking
in testSync failed for test data where one or more contacts had
not photos.
The function now keeps the event loop running while waiting for
completion. That ensures that events get received immediately.
testpim.py now matches the po scanning process because of the _(' text
pattern. We need to exclude it explicitly.
The is new API and flag grant control over the PBAP sync mode. It is
implemented by setting the SYNCEVOLUTION_PBAP_SYNC env variable in the
forked helper process.
The main advantage is that processed data can be discarded
immediately. When using a plain file, the entire address book must be
stored in it.
It also enables suspending a transfer by stopping to read from the
pipe, either via some internal API or simply freezing the
syncevo-local-sync process with SIGSTOP.
The drawback is that obexd does not react well to a full pipe. It
simply gets stuck in a blocking write(); in other words, all obexd
operations get frozen and obexd stops responding on D-Bus.
SyncEvolution was meant to load the syncecal or syncebook shared object
which uses the most recent libraries (libical, libecal/libebook) on
the system and then stop loading further backends. Due to a string
handling bug the check for already backends always found nothing,
leading to multiple conflicting backends loaded on some systems (for
example, those with libical0 and libical1 installed).
If that happened, the backend became unusable.
In testAutoSyncNoNetworkManager, syncs keep happening while the test
terminates. The processes get killed correctly, but that ocassionally triggers
notifictions which the post-test D-Bus log checking treats as test failures.
To avoid this, truncate the D-Bus log before shutting down. Then all extra
events are available for debugging, but will not be seen by the check code.
Check for an already running instance. Otherwise the new one
will fail to start up, which will be hard to diagnose. Happened
recently when a test was broken and failed to kill its daemon
instance.
Sometimes time stamps are necessary to understand timing-dependent
issues. A separate flag for timestamps on/off might be useful.
For now, enable them in debug mode.
Occasionally in the nightly testing reading the existing akonadi.db
produced incomplete output which could not be used to create the
modified akonadi.db. This patch works around that by using pre-generated
akonadi.db.dump files as input if found.
"evolution" tests used to be a traditional name for all Client::Source
and SyncEvolution unit tests. This caused problems when adding
Akonadi (crash in Akonadi client lib when EDS was used before) and
was not nice in the test reporting (multiple different tests all mixed
into one summary line). It also prevented running these tests in parallel.
All of this gets fixed by splitting "evolution" into "eds", "kde",
"file" and "unittests". To avoid updating the nightly test run scripts,
--enable evolution=.... is still understood.
More recent distros (for example, Ubuntu Saucy) rely on
XDG_RUNTIME_DIR. Each time dbus-session.sh runs, it must
ensure that the runtime dir exists and is empty.
Not doing so may have caused random concurrency issues in
parallel nightly testing.
It also was a problem when trying to run activesyncd + SyncEvolution
on a headless Ubuntu Saucy server (see FDO #76273).
Better assume that the program may change directories. In that
case we need an absolute path for the output file.
Nightly testing failed once because valgrind was not able to
write its output file in the new directory. It remained unclear
why the directory was changed and to what, though. But that is
a different problem.
Nightly testing uses a "Client_Sync_Current" symlink to point
the server to a location where it shall write its per-test logs
without having to reconfigure the server.
When the server-side session runs longer than a specific test, it ends up
using the "Client_Sync_Current" link of the next test. "permission denied"
errors were also seen, although it is less certain whether that had the same
root cause.
To avoid such issues, the sync session now tries to resolve the log dir
path to an absolute path at the start of the session.
