This reverts commit 7327b23a4dd31abdc9596916743892402bcffe0c.
Depends on 273527 "ActiveSync: added getDatabases support for fetching
folder list" which has to be reverted.
The new ReplaceSearch is more flexible than RefineSearch. It can
handle both tightening the search and relaxing it. The downside of it
is the more expensive implementation (must check all contacts again,
then find minimal set of change signals to update view).
Previously, a search which had no filter set at all at the begining
could not be refined. This limitation of the implementation gets
removed by always using a FilteredView, even if the initial filter is
empty.
When the parent of a FilteredView emits signals, it is not necessarily
always in a consistent state and the FilteredView must not invoke
methods in the parent. Stressing the FilteredView by using it in tests
originally written for the FullView showed that the filling up a view
violated that rule and led to contacts being present multiple
times. Fixed by delaying the reading from the parent into either an
idle callback or the parent's quiescence signal processing, whichever
comes first.
Added priorities and proper tracking of "activated" state. The latter
depends on having the Timeout instance valid when the callback
returns, therefore the Server must delay the deletion of the instance
until the callback is done executing.
Priorities are needed to order different tasks in the PIM manager
correctly.
The "activated" state is something that the PIM manager needs to track
reliably, which had to be done by every callback. If a callback forgot
to do that, in the worst case this might have removed an entirely
unrelated source when the tag got reused in the meantime.
Previously, the database field was interpreted as a Collection ID. This adds
logic to allow the database to be interpreted as a folder path. The logic is:
1) If the database is an empty string, pass it through (this is the most
common case as it is interpreted as "use the default folder for the
source type").
2) If the database matches a Collection ID, use the ID (this is the same as
the previous behaviour).
3) If the database matches a folder path name, with an optional leading "/",
use the Collection ID for the matching folder.
4) Otherwise, force a FolderSync to get the latest folder changes from the
server and repeat steps 2 and 3
5) If still no match, throw an error.
Steps 2 and 3 are in the new function lookupFolder. The remaining logic has
been added to the open function. Note that the result is that m_folder (and
hence getFolder()) are always either empty or a Collection ID -- that is as
before so the sync logic itself is unchanged.
A new method, findCollections, fetches the folder list from the server and
creates two maps:
m_collections - store all the information about each collection (folder),
indexed by server collection ID
m_folderPaths - map full folder paths to collection IDs
getDatabases uses this data to returns the folder path, collection ID and a
flag indicating if the folder is the default for that type.
Note 1: getDatabases always asks activesyncd to update the folder list from the
server in order to return up to date information.
Note 2: this depends on a new libeasclient routine:
eas_sync_handler_get_folder_list
If phone number search is possible, then the direct search in EDS now
uses the more accurate E_BOOK_QUERY_EQUALS_NATIONAL_PHONE_NUMBER
comparison, with the E164 formatted caller ID as value to compare
against. That value includes the current country code.
For testing purposes, setting SYNCEVOLUTION_PIM_EDS_SUBSTRING forces
the usage of the traditional suffix match. This is used to test both
the old and new flavor of testFilterStartupRefine.
FilterStartupRefineSmart is the one which uses phone number matching.
When available, the pre-computed E164 number from EDS will be used
instead of doing one libphonebook parser run for each telephone number
while reading. Benchmarking showed that this parsing was the number
one hotspot, so this is a considerable improvement.
Google became even more strict about checking REV. Tests which
reused a UID after deleting the original item started to fail sometime
since middle of December 2012.
To fix this, tests must differentiate reuse of an item by adding a suffix
("-A", "-B", etc.). If CLIENT_TEST_UNIQUE_UID has a value >= 2, that suffix
will be used when mangling the input item. Otherwise the suffix is ignored
and nothing changes.
For testing Google, CLIENT_TEST_UNIQUE_UID=2 is used.
The backends had SYNCEVOLUTION_LIBS in their _DEPENDENCIES entries,
which is wrong because SYNCEVOLUTION_LIBS must include -lrt (which
can't be a dependency).
Fixed by depending on libsyncevolution.la directly.
The library might be used, for example when the PIM Manager is part of
the server, so better add it. Avoids hard linker errors on some
systems when it is used and not specified, which is better than given
one extraneous library that -as-needed can take care of.
The code which removed a photo from a contact did it by creating a
g_file_icon() with a NULL GFile. This is invalid and caused warnings
from glib. Eventually it must have passed NULL to folks, which is what
the new code does directly.
Don't let libphonenumber write to stdout. Instead redirect into
SyncEvolution logging and manage severity level. For example,
previously parsing errors of phone numbers were logged as [ERROR] by
libphonenumber. Now that appears as "phonenumber error: ...".
The previous macros assumed to be used inside the SyncEvo namespace
and had problems when used in a context where "Logger" was resolved to
something other than SyncEvo::Logger. Making that more explicit solved
that problem.
Use e_book_client_connect_direct_sync(), the official API, when
available. Support for e_book_client_new_direct() is still in the
code; it can be removed onces the 3.6 openismus-work branch adapts the
official API.
The new Bluez 5 API is the third supported API for doing PBAP
transfers. It gets checked first, then the PBAB backend falls back to
new-style obexd (file based, similar to Bluez 5, but not quite the
same) and finally old-style obexd (data transfer via D-Bus).
In contrast to previous APIs, Bluez 5 does not report the reason for a
failed PBAP transfer. SyncEvolution then throws a generic "transfer
failed" error with "reason unknown" as message.
When using more than 10 contacts (TESTPIM_TEST_ACTIVE_NUM), logging
and polling are turned towards causing less work by testpim.py itself.
TESTPIM_TEST_ACTIVE_RESPONSE can be used to give a maximum response
time in seconds, which gets checked by watchdog by calling
GetAllPeers().
This is relevant in two cases, data loaded before folks starts reading
from EDS (TESTPIM_TEST_ACTIVE_LOAD set) and folks getting change
notifications as SyncEvolution imports data into EDS (not set).
Related to FDO #60851. Currently SyncEvolution+Folks do not pass the
performance test.
The server must remain responsive at all times. The new Watchdog
class tests that by calling GetAllPeers() at regular intervals. It
can only be used in tests which do not block the event processing
in the Python script itself.
Once the watchdog runs, it must be removed when a test stops. The new
cleanup test member can be used to add cleanup code at runtime which
will be called when tearing down the test.
The D-Bus server has to respond to the entire request at once, which
makes it unresponsive. Needs to be avoided in the client, because
doing it on the server side is too complicated. The server would have
to start gathering results, but those results may become invalid again
in response to processing events before it can send them out.
Related to BGO #60851.
The \n in the testRead vCard was interpreted by Python instead of
being written into the file. Must mark the string as raw. Did not
affect the test itself, but the libebook vCard parser reported
warnings.
Track which changes need to be stored and how that gets processed. Was
useful for finding that modifying groups failed when the groups hadn't
actually changed.
Setting TESTPIM_TEST_ACTIVE_NUM to a value larger than 10 enables a
mode with less logging and more efficient event handling in
runUntil. Useful for performance testing, because otherwise testpim.py
itself consumes considerable CPU cycles.
Mark pending reads by adding the start time. Avoid requesting data
again which is still pending. Relevant when dealing with many contacts
in testActive, helps performance by avoiding redundant operations.
Sending DEBUG messages via the LogOutput signal is expensive and often
not necessary. Ideally LogOutput should only send data that someone is
interested in. The problem is that the SyncEvolution D-Bus API has no
way of specifying that and some clients, like dbus-monitor, would not be
able to use it.
Therefore this commit makes the default level of detail configurable
when syncevo-dbus-server is started, via a separate --dbus-verbosity
option.
This gets applied to output generated from syncevo-dbus-server itself
as well as output from sync sessions.
test-dbus.py exposes that via a new TEST_DBUS_QUIET option. The
default is to include DEBUG output. When TEST_DBUS_QUIET is set to a
non-empty string, only ERROR and INFO messages are sent.
That SetPeer() allows modifying and creating a config leads to race
conditions when multiple clients want to create a config. The new
CreateConfig() avoids that by atomically checking that a config does
not exist yet and creating it.
SetPeer() is still available for backwards compatibility. It continues
to be used for modifying an existing config in TestContacts.testSync
to check the effect of the logging settings.
Exposed as "location" -> (lat, long) in the D-Bus bindings.
Reading, writing and updating are supported. Depends on a folks
release which has the patch which adds FolksLocationDetails.
Default EDS test data now includes GEO. Several peers do not support
it, which gets ignored by synccompare.
Was lost earlier during syncing. Must be defined in field list and
vCard profile. Still not supported by PIM Manager, because folks doesn't
support it (see FDO #60373).
Constructing the GValues created additional references instead
of taking over ownership as intended. For refcounted GObject,
the solution is to pass the instance and let GValue take the
reference.
For other cases, ownership of a new instance is transfered to the
GValue using the new constructors with a boolean flag.
The existing constructors with native data pointers
always copied data or increased the reference counter.
It is useful to store the result of creation methods
directly in a GValue, so allow that with a second
variation of the constructors which explicitly take
an "addRef" parameter.
The test must be activated by compiling manager.cpp with
PIM_MANAGER_TEST_THREADING and then runs start() in a new
thread which gets created during startup.
The first attempt was buggy: the idle callback is always active and
prevents sleeping => 100% CPU load. It also seems to prevent proper
shutdown somehow.
This version is much simpler. It removes the task queue and the
manager's idle callback. Instead the non-main thread adds an idle
callback itself which gets run once. To have the main thread notice
the new idle callback, g_main_context_wakeup() is used, as before.
The reason for not using this approach before was that the glib
documentation only mentioned GAsyncQueue as one of its structures
which are thread-safe. A look at the code shows that GMainContext and
GMainLoop are also thread-safe.
This adds the infrastructure for shifting the work of the D-Bus
methods into the main thread if called by another thread, which may
happen when we add other bindings for them.
Work is shifted to the main thread via a GAsyncQueue +
g_main_context_wakeup(). The calling thread then blocks waiting on a
condition variable until signaled by the main thread. Results are
stored for the calling thread as part of the operation that it
provides. Exceptions in the main thread are caught+serialized as
string and deserialized+rethrown in the calling thread - a bit crude,
but should work and reuses existing code.
Google CalDAV currently sends invalid XML back when asked to include
CardDAV properties in a PROPFIND. This gets rejected in the XML
parser, which prevents syncing calendar data:
Neon error code 1: XML parse error at line 55: undeclared namespace prefix
The incorrect XML is this:
<D:propstat>
<D:status>HTTP/1.1 404 Not Found</D:status>
<D:prop>
...
<caldav:max-attendees-per-instance/>
<ns1:addressbook-home-set xmlns:ns1="urn:ietf:params:xml:ns:carddav"/>
==> <ns1:principal-address/>
<ns1:addressbook-description/>
<ns1:supported-address-data/>
<ns1:max-resource-size/>
</D:prop>
</D:propstat>
This was introduced on the server side sometime after December 12nd
2012 (tests run at that time showed a different response) and does not
affect SyncEvolution 1.2 because it did not yet ask for CardDAV
properties.
The workaround on the client side is to ask for only the properties
which are really needed.
Don't try to list non-existent directories, that causes an uncaught
Python exception. Happens when using testpim.py for the first time
in a clean directory.
The error message for an unexpected slow sync still mentioned
the old and obsolete "refresh-from-client/server" sync modes.
Better mention "refresh-from-local/remote".
This reverts commit 3a71a2bf53 and
commit a0375e0160.
Need to back out the workaround for broken peers because it breaks
SyncEvolution<->SyncEvolution syncing.
When compiled with PIM Manager, syncevo-dbus-server still should
shut down automatically when idle. Only when the PIM Manager was
started and has (or will have) a unified address book, then keep
the process running.
