The word "source" implies reading, while in fact access is read/write.
"datastore" avoids that misconception. Writing it in one word emphasizes
that it is single entity.
While renaming, also remove references to explicit --*-property
parameters. The only necessary use today is "--sync-property ?"
and "--datastore-property ?".
--datastore-property was used instead of the short --store-property
because "store" might be mistaken for the verb. It doesn't matter
that it is longer because it doesn't get typed often.
--source-property must remain valid for backward compatility.
As many user-visible instances of "source" as possible got replaced in
text strings by the newer term "datastore". Debug messages were left
unchanged unless some regex happened to match it.
The source code will continue to use the old variable and class names
based on "source".
Various documentation enhancements:
Better explain what local sync is and how it involves two sync
configs. "originating config" gets introduces instead of just
"sync config".
Better explain the relationship between contexts, sync configs,
and source configs ("a sync config can use the datastore configs in
the same context").
An entire section on config properties in the terminology
section. "item" added (Todd Wilson correctly pointed out that it was
missing).
Less focus on conflict resolution, as suggested by Graham Cobb.
Fix examples that became invalid when fixing the password
storage/lookup mechanism for GNOME keyring in 1.4.
The "command line conventions", "Synchronization beyond SyncML" and
"CalDAV and CardDAV" sections were updated. It's possible that the
other sections also contain slightly incorrect usage of the
terminology or are simply out-dated.
The real username is only relevant when running a sync. When looking
at a config with a D-Bus client like the GTK UI, the username should
always be "id:<config>", to avoid accidentally removing the
indirection, while the password should be the real one, to allow the
user to edit like he normally would with passwords stored in a
keyring.
To achive this, overriding the username must be suppressed when
resolving as part of the D-Bus config API. While at it, move the
entire "iterate over properties" into a common utility function in
PasswordConfigProperty.
In practice, the methods are always called for a specific SyncConfig.
Passing that allows removing several other parameters and, more
importantly, also grants access to the config and through that other
configs. This will be needed for the indirect credential lookup.
The real username is only relevant when running a sync. When looking
at a config with a D-Bus client like the GTK UI, the username should
always be "id:<config>", to avoid accidentally removing the
indirection, while the password should be the real one, to allow the
user to edit like he normally would with passwords stored in a
keyring.
To achive this, overriding the username must be suppressed when
resolving as part of the D-Bus config API. While at it, move the
entire "iterate over properties" into a common utility function in
PasswordConfigProperty.
In practice, the methods are always called for a specific SyncConfig.
Passing that allows removing several other parameters and, more
importantly, also grants access to the config and through that other
configs. This will be needed for the indirect credential lookup.
Commit 006bcf26 introduced a more versatile replacement for
FileConfigNode, but didn't switch over old code to the replacement
because of a code freeze. Switching now.
Also use the hash variant in VolatileConfigNode, because it is more
efficient by dropping the (in that case irrelevant) order of
properties.
KWallet support was broken: syncevo-dbus-server checked
KDE_FULL_SESSION to determine whether it should use KWallet instead of
GNOME Keyring. That did not work, because the env variable was not set
for D-Bus daemons.
Automatically detecting KDE users is not possible at the
moment. Instead KDE users have to manually set the new "keyring"
global config property to "KDE" (case insensitive) if the
SyncEvolution installation supports both, because GNOME Keyring is the
default to avoid surprises for traditional users. If only KWallet
support is enabled, then this is not necessary.
"GNOME" and "true/false/1/0/yes/no" can also be set. This has the
advantage that keyring usage can be enabled permanently for the
command line in --daemon=no mode; normally keyrings are not used in
that mode because accessing them can bring up UI dialogs.
It also becomes possible to disable keyring usage in syncevo-dbus-server,
something which couldn't be done before.
The --keyring command line option is still supported, as an alias for
"[--sync-property] keyring=<value>". The default value for --keyring
is true, to match the traditional behavior. In contrast to other sync
properties, setting "keyring" does not require an explicit --run
parameter. Again this is done to mirror traditional usage.
Reading a password also (unintentionally) checked all supported
storages while searching for the password. Now it uses exactly
one storage and falls back to asking for the password directly.
The commit itself also cleans up the code a bit (reformatted, fixed
comments). Choosing the right slot in the password signals is done via
a new InitStateTri parameter which contains the "keyring" setting.
Error checking (unsupported keyring string, --keyring=yes and no
keyring enabled) is done in additional slots which run after all the
regular ones.
Parameter parsing for --sync and --keyring were unified. However,
there is the difference that --keyring has an implicit default value
("yes") and never has an additional parameter, in contrast to --sync,
which always is followed by one.
The new CmdlineTest::testKeyring covers different ways of using
--keyring. It relies on actually invoking keyring backends, something
not done by the default SyncContext UI. Therefore
CmdlineSyncClient+KeyringSyncCmdline were moved into libsyncevolution,
to be used by CmdlineTest.
This commit moves the blocking syncing, database restore and command
line execution into a separate, short-lived process executing the
syncevo-dbus-helper. The advantage is that the main
syncevo-dbus-server remains responsive under all circumstances (fully
asynchronous now) and suffers less from memory leaks and/or crashes
during a sync.
The core idea behind the new architecture is that Session remains the
D-Bus facing side of a session. It continues to run inside
syncevo-dbus-server and uses the syncevo-dbus-helper transparently via
a custom D-Bus interface between the two processes. State changes of
the helper are mirrored in the server.
Later the helper might also be used multiple times in a Session. For
example, anything related to loading backends should be moved into the
helper (currently the "is config usable" check still runs in the
syncevo-dbus-server and needs to load/initialize backends). The
startup code of the helper already handles that (see boolean result of
operation callback), but it is not used yet in practice.
At the moment, only the helper provides a D-Bus API. It sends out
signals when it needs information from the server. The server watches
those and replies when ready. The helper monitors the connection to
the parent and detects that it won't get an answer if that connection
goes down.
The problem of "helper died unexpectedly" is also handled, by not
returning a D-Bus method reply until the requested operation is
completed (different from the way how the public D-Bus API is
defined!).
The Connection class continues to use such a Session, as before. It's
now fully asynchronous and exchanges messages with the helper via the
Session class.
Inside syncevo-dbus-server, boost::signals2 and the dbus-callbacks
infrastructure for asynchronous methods execution are used heavily
now. The glib event loop is entered exactly once and only left to shut
down.
Inside syncevo-dbus-helper, the event loop is entered only as
needed. Password requests sent from syncevo-local-sync to
syncevo-dbus-helper are handled asynchronously inside the event loop
driven by the local transport.
syncevo-dbus-helper and syncevo-local-sync are conceptually very
similar. Should investigate whether a single executable can serve both
functions.
The AutoSyncManager was completely rewritten. The data structure is a
lot simpler now (basically just a cache of transient information about
a sync config and the relevant config properties that define auto
syncing). The main work happens inside the schedule() call, which
verifies whether a session can run and, if not possible for some
reasons, ensures that it gets invoked again when that blocker is
gone (timeout over, server idle, etc.). The new code also uses
signals/slots instead of explicit coupling between the different
classes.
All code still lives inside the src/dbus/server directory. This
simplifies checking differences in partly modified files like
dbus-sync.cpp. A future commit will move the helper files.
The syslog logger code is referenced by the server, but never used.
This functionality needs further thought:
- Make usage depend on command line option? Beware that test-dbus.py
looks for the "ready to run" output and thus startup breaks when
all output goes to syslog instead of stdout.
- Redirect glib messages into syslog (done by LogRedirect, disabled when
using LoggerSyslog)?
The syncevo-dbus-server now sends the final "Session.StatusChanged
done" signal immediately. The old implementation accidentally delayed
sending that for 100 seconds. The revised test-dbus.py checks for
more "session done" quit events to cover this fix.
Only user-visible messages should have the INFO level in any of the
helpers. Messages about starting and stopping processes are related to
implementation details and thus should only have DEBUG level.
The user also doesn't care about where the operation eventually
runs. All messages related to it should be in INFO/DEBUG/ERROR
messages without a process name. Therefore now syncevo-dbus-server
logs with a process name (also makes some explicit argv[0] logging
redundant; requires changes in test-dbus.py) and syncevo-dbus-helper
doesn't.
syncevo-local-sync is different from syncevo-dbus-helper: it produces
user-relevant output (the other half of the local sync). It's output
is carefully chosen so that the process name is something the user
understands (target context) and output can be clearly related to one
side or the other (for example, context names are included in the sync
table).
Output handling is based on the same idea as output handling in the
syncevo-dbus-server:
- Session registers itself as the top-most logger and sends
SyncEvolution logging via D-Bus to the parent, which re-sends
it with the right D-Bus object path as output of the session.
- Output redirection catches all other output and feeds it back
to the Session log handler, from where it goes via D-Bus
to the parent.
The advantage of this approach is that level information is made
available directly to the parent and that message boundaries are
preserved properly.
stderr and stdout are redirected into the parent and logged there as
error. Normally the child should not print anything. While it runs,
LogRedirect inside it will capture output and log it
internally. Anything reaching the parent thus must be from early
process startup or shutdown.
Almost all communication from syncevo-dbus-helper to
syncevo-dbus-server is purely information for the syncevo-dbus-server;
syncevo-dbus-helper doesn't care whether the signal can be
delivered. The only exception is the information request, which must
succeed.
Instead of catching exceptions everywhere, the optional signals are
declared as such in the EmitSignal template parameterization and
no longer throw exceptions when something goes wrong. They also don't
log anything, because that could lead to quite a lof of output.
All user-facing functions (password handling, reading from
stdin) are now in a dedicated "UserInterface" class, instead of
spreading that between ConfigUserInterface and SyncContext.
SyncContext no longer has the misleading "is a" [Config]UserInterface]
relationship. Instead it "has a" UserInterface instance, which is set
at runtime. Long term the plan is remove the need to subclass
SyncContext. In the local sync that was already possible
(LocalTransportContext->LocalTransportUI).
The guarantee that there always is a usable instance that can be used
without checking for NULL is provided by the getUserInterfaceNonNull()
method, which falls back to a dummy instance if necessary.
Reading data and password from stdin is moved out of the core
libsyncevolution into the syncevolution binary. That way the D-Bus
server and client-test do not accidentally attempt to read from stdin
(has happened when setting up testing incorrectly).
The special semantic of the former RegisterSyncSource::InactiveSource
(invalid pointer of value 1) caused bugs, like using it in
--print-databases (=> segfault) or not being able to store the result
of a createSource() directly in a smart pointer (=> potential leak in
SyncSource::createSource()).
Obviously a bad idea to start with. Replaced with a
RegisterSyncSource::InactiveSource() method which creates a real,
inactive SyncSource instance which can and must be deleted by the
caller.
This is a SyncSource API change for backend developers.
Instead of RegisterSyncSource::InactiveSource, return
RegisterSyncSource::InactiveSource().
Comparisons against RegisterSyncSource::InactiveSource needs
to be replaced with a call to the new SyncSource::isInactive().
User visible fixes:
* --print-databases: no longer crashes when EDS or KDE backends
are not usable. Instead it prints "not enabled during compilation or
not usable in the current environment".
* --print-databases: continues with other backends even if
one backend throws an exception, as the KDE backend does
when it cannot find Akonadi. Error messages are printed.
Forcing D-Bus API users to start a new session for each config that
they want to modify adds considerable complexity to clients, in
particular for local sync.
Relaxing that requirements is easy because the current implementation
already only runs one session at a time. It's also only a minor
restriction for future improvements: sessions which request access to
all configs cannot run in parallel with other sessions.
Thus this patch adds Session.SetNamedConfig() and (for the sake of
completeness) Session.GetNamedConfig().
Temporary config changes via SetNamedConfig() are harder to implement
and not currently supported. They are part of the API though, just in
case.
At the property level, the isDefault retval exposed whether the
property value was set explicitly in the config or taken from the
property default. That information got lost at the
SyncConfig/SyncSourceConfig level although there are cases where that
is relevant (like providing better error messages, BMC #23783).
Now that level uses the new InitState classes instead of plain
int/bool/std::string return values. Code which assigns these return
values to local variables doesn't need to be adapted. Directly using
the return value in an expression might need some work (typically
adding a get() if the compiler cannot infer the desired
type). Overriding the virtual methods always needs to be adapted.
The previously suggested (and implemented) approach of reusing the
peerName property to convey hardware information didn't pass the smell
test (too complex to explain that the same property has different
meanings depending on the context).
So now a new "hardwareName" property is used instead. Also renamed
the TemplateDescription member accordingly, updated the API docs
and implemented the "hardwareName only set if known" part of it.
Previously the DeviceDescription deviceName was being stored in the
TemplateDescription peerName.
This patch defines devicename in both DeviceDescription &
TemplateDescription as storing the user-modifiable device name, thus
unifiying the meaning. The peerName is defined to hold the vendor
and/or model information discovered using the Bluetooth Device ID
profile (DIP) or, in the case that DIP is not supported, peerName will
be an empty string. Whether the peerName includes the model of the
device is dependent on whether a matching entry was found in the
product lookup table.
In the past we have relied on the user-modifiable device name to be
the fingerprint for matching a phone to a template which is unreliable.
This commit changes this in the cases where the phone supports the
Device ID profile (DIP). If support DIP is detected then we extract
the vendor and product id's from the PnPInformation service
record. Then, we attempt to associate the ids with a product and
vendor name by using a newly added lookup table.
The preference for which value to use as the fingerprint is as
follows:
1) If the product is found in the lookup table we use
that. (The product lookup table is manually updated and currently
only has a few entries.)
2) Otherwise, we use the vendor name. (All vendors should be in the
lookup table, so this should always be used if the DIP is supported.
3) If the DIP is not supported by this device we fall back to using
the user-modifiable device name as before.
Additionaly, because the user-modifiable device name is still
desirable to expose through the dbus interface, it is now made
avaiable through the peerName property. This is intended to be used in
GUIs, for example.
Patrick Ohly
Chris Kühl