Just saying "foo = bar" is enough now. --sync-property and
--source-property are still accepted, in case that "foo" ever becomes
ambiguous (not the case right now).
The meaning of "type" was horribly complex and had effects on the
backend and the peer. It was impossible to specify the sync format to
be used for a specific peer independently of the local backend and its
format, so adding a peer to a context broke the context configuration
(BMC #1023).
This is now fixed by splitting "type" into four independent properties:
- backend = plugin which interfaces with the data
- databaseFormat = data format used inside backend, only relevant for file backend
- syncFormat = data format preferred when talking to peer
- forceSyncFormat = disable format auto-negotiation, use preferred format
With that split, it is now possible to specify the format in which the
file backend stores items independently of the format in which they
are exchanged with the peer.
Old configurations with "type" can still be read. The values specified
inside it are transparently mapped to the new properties. Command line
and D-Bus API users will only see the new properties.
The command line tool still accepts "type" as an alias for the four new
properties. Using that has the same disadvantage as before: it will modify
the context even if only modifying the peer was intended.
The D-Bus API accepts only the new properties. Clients using "type"
must be adapted to the new property names. Clients not using that
continue to run unchanged.
Writing into the configuration requires a migration of the peer config
*and* the context in which it is defined. That is necessary because
the new semantic (independent database format) cannot be stored in the
old format. The migration is handled by rewriting first the context,
then all peers defined inside it.
Other user-visible changes:
- updated help texts
- the canonical "backend" value for the file backend is just "file"
instead of the long "Files in one directory", which is now an alias
(used to be the other way around); done because "type = file"
was expanded to the long name, which was a bit unexpected and showed
how unintuitive the long name is
Internal changes:
- getMimeVersion() is still present, although it hasn't been used
for a long time; FileSyncSource::getMimeVersion() now derives
the version from the supported Mime types, in case that the
function will be needed again in the future
- setSourceType() with string as argument was replaced with one
taking a SourceType instance; to emulate the old behavior if
desired, construct SourceType from an old-style string
- ConfigProperty methods need to be virtual so that derived classes
like SourceBackendConfigProperty can generate content at runtime
(a recent commit broke that feature)
- File templates were stripped down to the essential properties,
with "type" replaced by the per-peer "syncFormat". "type" would
still have been accepted (so it is not necessary to adapt
syncevo-phone-config right away), but has the original
disadvantage of modifying "backend" and "databaseFormat".
Because of the shared source property name changes
(evolutionsource/user/password -> database*), mixing old and new
configs inside the same context is not possible. Bumped
CONFIG_CONTEXT_MIN_VERSION and CONFIG_CONTEXT_CUR_VERSION to reflect
that and added the necessary code for migrating a context and all
peers inside it.
This is triggered by a) an automatic config migration in release mode
and b) an explicit --migrate of *any peer inside the context* or of
the context itself.
When creating a config from scratch, the command line source properties
applying to the default source set in the template were ignored. That's
because createFilters() needs to know about all active sources, otherwise
it won't create the filters for them.
When running a command line failed, an assertion was raised without
ever showing the error output of the failed command line. Now it is
included as message text in the CPPUNIT_ASSERT_MESSAGE() exception.
If --source-property was set while syncing, the intention was to only
apply (and thus potentially enable) the filter to active sources. If
no sources where specified on the command line, then the code looked
at the config to build that list.
That code must have been broken for a long time, because it had a ==
"disabled" check instead of != "disabled", thus enabling exactly the
wrong sources. Fixed.
Probably not relevant for old releases because it only applied to cases
where no source filter was set. That additional shortcut check was
removed when rewriting the property filters, so now it was found.
Now the @context or @peer@context suffix in sync and source properties
on the command line can be used to configure both sides of the sync
differently.
The new format of the property name in --sync-property is:
<name>[@<context>|@<peer>@<context>]
--source-property also allows a source name:
[<source>/]<name>[@<context>|@<peer>@<context>]
This allows to set source properties differently for different
sources in the same command line invocation. The @<context> or
@<peer>@<context> will be used to set properties differently for
main and target context in a local sync (not used yet).
The advantage of this grammar is that a string can be split purely based
on the syntax in PropertySpecifier::StringToPropSpec().
The patch itself is based on the idea of first collecting all of these
config property filters in a new case-insensitive hash structure,
FullProps in ConfigFilter.cpp/h, as part of parsing command line
parameters.
Then once specific filters for sync or sources are needed, they are
generated from FullProps by collecting all that apply, starting with
the ones with lowest priority and overwriting them with more important
(= more specific) ones. This also covers additional filters, like the
shared properties of the target context when printing a template.
Currently FullProps may contain arbitrary source and config
names. Typos are not detected, which is both hard to implement (which
names and configs are valid in the current invocation?) and also
forces users to be very specific (can't apply one set of filters to
different configs) - this is the same conflict of interest as in
"configure", which allows unknown --enable/disable parameters because
they might be relevant in a sub-configure script.
SyncConfig itself still only stores the filters which apply to it, not
the full set of overrides that the Cmdline has in its m_props. The
advantage is that the API remains the same (no change needed or done
in the syncevo-dbus-server). The disadvantage is that in a local
sync, no information is available about the properties applying to the
target context - probably needs to change.
This renames the legacy "evolutionsource/user/password" properties to
"database" and "databaseUser/Password". The justification is that
these properties are also used outside of the Evolution backends.
The change can be done now because the on-disk format has to be
changed anyway in a backward-incompatible way.
The CmdlineTest::testOldConfigure test covers reading an old-style
config.
The goal is to allow multiple names for properties. This will be used
to replace legacy names like "evolutionsource", but it could also be
used to allow abbreviations on the command line.
First the patch replaces the single ConfigProperty::m_name with a list
of names, of which the first one is the current name of the
property.
The name that is to be used depends on the node which is to be
manipulated: if it already has a property under an alias, that alias
is read and also written, so existing configuration continue to use
the old config name (avoids the problem of having to remove the old
name and insert the new one at the same spot in a .ini file). Old
configs continue to parse okay. Writing into node which has none of
the names set, as in migrating into a fresh config, uses the new
names.
Most of the patch deals with the removal of a single, unique name,
either by picking the name so that it matches a node, using the
default name or simply hard-coding a property name ("sync").
The command line accepts all aliases, then uses the main name for
overriding configuration options.
SyncConfig inherited "const char *" from the Funambol C++ API and some
other methods used the same approach for efficient access to plain
strings. However, this has the disadvantage that dynamically generated
strings cannot be returned. SyncConfig had to use an awkward
workaround with a local string cache.
This patch converts most of that code to a normal std::string return
value and removes the string cache.
Out-of-tree backends must be adapted, otherwise they won't compile.
SyncEvolution < 1.2 considered all existing configurations
user-visible in UIs, therefore the ConsumerReady flags was not always
set (done by sync-ui, not done by command line).
In order to keep such old configs visible even when the updated
sync-ui (and soon Genesis) check the flag, the syncevo-dbus-server
transparently adds the flag when it finds a config written by
SyncEvolution < 1.2.
The Cmdline.cpp migration code does the same when rewriting such and
old config for read/write use.
The "*.old" configs get their ConsumerReady flag removed so that UIs
can decide whether they still need to be shown to the user. The
command line will show them, the GTK sync-ui is meant to hide them
(but doesn't to it yet).
Migrating a config in a context wasn't necessary so far, but also
wasn't explicitly forbidden or prevented. It did not work correctly
when the config was identified without explicit context on the command
line, because that short-hand was only mapped correctly before
renaming the config, but not anymore afterward. As a result, the
config was moved into the @default context or possibly some other
context which had a peer with the same name.
Cleaned up the m_server parsing (duplicate code?!) and fixed this
problem by setting m_server to the fully-qualified configuration name,
including a @default suffix. This is user-visible in error messages,
which actually may be a good thing.
The goal is to release SyncEvolution 1.2 with a more recent
libsynthesis, which rewrites its on-disk binfiles such that
older releases are forced to do slow syncs.
This versioning change forces old configs to be migrated. This
affects both the old .sync4j style config (as represented by the
testOldConfigure test) as well as unshared
~/.config/syncevolution/<peer> configs.
gen-autotools.sh now detects pre-releases: have "99" in the version or
the source isn't cleanly tagged. "syncevolution --version" includes
"(pre-release)" if in pre-release mode, otherwise just the version (as
before).
This patch adds version numbers to the on-disk config and files. The
goals are:
1. refuse to use a config which was written by a SyncEvolution release
in a format that is too recent to be handled correctly by the
current release ("detect invalid downgrades")
2. refuse to modify a config in such a way that the previous release
using that config will not be able to use it anymore ("prevent
unintentional upgrade of config")
In the first case the user is told:
SyncEvolution <version> is too old to read configuration '<config>',
please upgrade SyncEvolution.
In the second case the user is told to migrate the configuration manually:
Proceeding would modify config '<config>' such that the
previous SyncEvolution release will not be able to use it.
Stopping now. Please explicitly acknowledge this step by
running the following command on the command line:
syncevolution --migrate '<config>'
These are printed as [ERROR] messages on the command line and shown as
error codes 22004 resp. 22005 in front-ends which don't know about these
scenarios.
The first problem should be rare, so presenting a nicer error messages
in UIs is not essential. But the second problem will occur. The plan
(not implemented yet) is to automatically migrate in stable releases
without asking.
"--migrate @<context name>" would have happily migrated the non-peer
properties, but would not copy the peers inside that context. Wasn't
needed so far, but shouldn't have been allowed either.
Conflicts:
configure-pre.in
src/syncevo/SyncContext.cpp
src/syncevo/configs/datatypes/10calendar-fieldlist.xml
test/ClientTest.cpp
test/client-test-main.cpp
test/synccompare.pl
Conflicts all resolved by using the code from "master" branch. Caused
by similar changes on release branch.
The ScheduleWorld service shut down end of November 2010.
Removing it completely from SyncEvolution is a bit
intrusive (it is used as reference configuration and in
unit tests), so this patch only makes it invisible to
sync-ui users and replaces it in the examples with
Memotoo.
Database backups used to be created unconditionally, even if not
needed for comparison. Now the new "dumpData" variable is
checked. If change printing is enabled, dumping data is implied.
Passing a NULL terminated list of parameters is sometimes a convenient
way of executing certain complex operations. Cmdline testing already did
it, now this is also in the Cmdline class itself.
Backends like XMLRPC need information about URL, proxy and SSL
settings, etc. This can be done via source specific properties, like
evolutionsource, but this is not how this is normally done. It would
be nicer if the exising per-peer properties could be used. The goal is
that a normal peer configuration can be created from a template with
the necessary information to enable sources using that information.
This patch makes this possible by adding a context parameter to
SyncSourceParams:
* @param context Additional non-source config settings.
* When running as part of a normal sync, these are the
* settings for the peer. When running in a local sync,
* these settings come from the "source-config" peer
* config inside the config context of the source.
* Testing uses "source-config@client-test". On the
* command line, this is the config chosen by the
* user, which may or may not have peer-specific settings!
Note that this still doesn't solve the problem for XMLRPC to SyncML
peer sync, because in that case ("normal sync") the context will be
the one describing the peer. SyncURL is already used and proxy
settings might not match.
The XMLRPC backends therefore was not changed and continues to use
evolutionsource.
There was no obvious way of deleting all items. Added * as wildcard
for that. It can be specified in addition to other luids, which
will be ignored if a star is given.
The --import operation was specified as splitting at a blank line and
was meant to work for vCard 3.0 and iCalendar 2.0 files. However, if
the blank line between items used DOS line ends (\r\n) like the items
themselves, then splitting them didn't work because of the literal
match with "\n\n".
This patch introduces a special semantic for the "\n\n" delimiter:
it also matches "\n\r\n" when splitting. This is implemented by
the custom FindDelimiter class.
A context holds a set of source settings, without any peer-specific
properties like "sync mode". The code for configuring such a context
tried to set the sync mode and thus failed.
This patch unifies the check for "configuring context" in one place.
It also slightly modifies the set of sources which are configured:
traditionally, all sources were configured, but only the listed ones
were enabled. This makes sense because users then can enable the source
selectively, either during --run or with --configure without access to
the original template.
For a context, there is no concept of a "disabled" source. Adding
those makes no sense and thus is not done anymore.
Configuring a single source in a new context, adding a second source
and adding a peer is now tested in CmdlineTest::testConfigureSources.
Note that this contains a hack to reflect the broken behavior in BMC
fixed.
Using std::string for both local IDs as respresented by the backend
and encoded as printed by the command line was confusing. It didn't
help that the m_luids member contained encoded strings, instead of
normal luids.
This patch introduces a dedicated class which holds an encoded luid,
and provides the necessary conversion functions. m_luids now really
contains plain luids, so its values can be passed to the backends
directly.
Watching for configuration changes (e.g., for refreshing the list of
servers in the GUI) currently is a bit cumbersome. It requires
watching for SessionChanged, request GetConfigs, and look if
the configs have changed. This is not really a problem, but it
might be more convenient to get notifications about config
changes directly.
This patch introduces a Server.ConfigChanged signal which is at the
end of a session if and only if it modified (removed, updated, added)
a config. This includes the Session.SetConfig() as well as command
line operations executed inside the server.
Testing the signal can only proof that it is emitted. Checking that it
is not emitted would have to block waiting for something, and it is
uncertain what that should be. SessionChanged may be emitted before
the ConfigChange signal and thus isn't suitable.
When configuring a context for local sync, no peer is needed
inside that context. Therefore "--configure" without "--template"
is valid, because the templates only differ in peer-specific
properties.
This patch uses the "SyncEvolution" template as automatic fallback
for context configs. A change to FileConfigTree ensures that the
"peers" directory is created, as expected by the code which checks
for new-style configs.
The --delete-items/--import options did not work with the KCalExtended
backend. The reason was that saving changes is only done in
KCalExtendedSource::endWrite(), which wasn't called because the command
line did not invoke endDataWrite().
That used to be intentional to avoid updating change tracking state in
other backends, but clearly was a hack. Because the sources are now
instantiated without change tracking, endDataWrite() is safe to call,
which fixes the issue with KCalExtended.
'isPropAssignment' checks each line and match whether it's
a property assignment according to match '<word> = value'.
Comments for 'KCalExtended' and 'QtContacts' match this pattern.
Obviously, they should be ignored like other backends comments.
This patch adds an utility script (readme2c.pl) which extracts the
SYNOPSIS and OPTIONS part of README.rst and puts it into C
strings. These are then used inside the SyncEvolution command line for
short help (just the synopsis) and --help (also all options).
This patch also cleans up README.rst so that the OPTIONS part really
documents most options in sufficient detail for --help to be useful.
The USAGE section remains part of the README and the man page where
the different operations are introduced in a more task-oriented way.
This separation is not always easy to make. The --restore operation
and related parameters are only described in the USAGE part because I
couldn't find a way to introduce them briefly and then add a thorough
explanation under OPTIONS.
Some unit tests of CmdlineTest are failed due to new backends
which were newly implemented, including KCalExtended and QtContacts.
Change the reference data.
ZYB.com will be replaced with Vodafone 360, which we don't support (yet).
This patch removes the obsolete template for ZYB.com. The sync-UI still
has some strings for it, just in case that someone has an old template
around or runs it against an old syncevo-dbus-server.
Local IDs were not decoded before invoking the underlying SyncSource
operation. As a result, it failed with "item not found" errors for
those items which really had special characters in their IDs.
Besides the issue with reading the "type" (MBC #3157),
operations like "--print-items @some-context addressbook"
also failed because the TrackingSyncSource would try to write
updated id/rev pairs into a DevNullConfigNode, set because
there is no tracking node in @some-context.
DevNullConfigNode does not allow writing. This patch uses
a set of config nodes where the tracking node is volatile and
allows writing.
The additional benefit also for peer configs is that the SyncSource
can no longer overwrite the change tracking for that peer
accidentally because it never gets access to that node.
This complements --import/update/export/list-items. --remove was
already taken, for deleting a configuration.
Because the removal is done via the normal m_deleteItem operation, we
inherit the normal SyncML behavior (logging, no error return code for
non-existent items). It would have been nice to let the user choose
how to handle that situation (like --force in rm), but that can't be
done without SyncSource API changes.
These operations provide a common interface for manipulating
databases supported by SyncEvolution backends. --delete is
an obvious gap, will be added soon.
The implementation supports all backends offering the SyncSourceRaw
interface (Evolution, Akonadi, XMLRPC, file, but not the sqlite demo),
which is determined via a dynamic cast. SyncSourceRaw is necessary
because it allows direct access to the native data format without
going through the Synthesis engine.
Because the Synthesis engine is not activated, printing the short
description (SyncSourceLogging interface) mentioned in the README.rst
is not possible yet. Having such an engine would be useful also for
importing/exporting data which is not in the native format of the
backend; additional command line parameters will be needed to define
that format.
All of the new operations have in common that they work on one source
and multiple LUIDs. This is a slight deviation from other command line
synopsises where all arguments after the server config refer to sources.
The new m_accessItems flag is set for these special operations; they
also share much of the setup code.
The implementation itself tries to use the generic SyncSource
interface as much as possible. It only falls back to more specialized
implementations where necessary (SyncSourceRaw).
The builtin synopsis and usage was intentionally not updated. The
expection is that before this patch lands in the "master" branch, the
builtin text will come directly from README.rst (which was updated).
Previously, it was possible to list several different operations
on the command line ("--version --configure --run ..."). SyncEvolution
then picked one of them in (for users) unpredictable ways.
Now all requested operations are gathered while parsing the command
line and the tool aborts with an error message listing all of them
if more than one is given.
With this patch, template files are also found in
.config/syncevolution-templates (or the corresponding directory
pointed to by the XDG env variables).
The tests had to be adapted a) to not find templates installed
by the user in his own home and b) to cover finding templates
via XDG_CONFIG_HOME.
A user reported that the default 1 minute resend interval did not work
for him with his own Funambol server installation. The Funambol
developers are notified, but won't be able to do much about this
in the near future.
Therefore this patch disables message resend in the Funambol server
template. Existing configurations have to be updated manually, both
to suppress the resending and to enable it again later.
The default resend interval is also increased from 1 minute to 2
minutes, to help other slow servers which might have problems handling
resends when they arrive to quickly.
There is a regression caused by a string exception
"basic_string::_S_construct NULL not valid". This is because
I push back a non-existing string into vector when collecting
converted command line arguments.
The criteria for not invoking Cmdline::run() was a bit too broad.
For "--template ?", "--help", "--version", "?", "--print-servers" no
output was produced in --daemon mode.
Perhaps the whole m_dontrun logic should be updated. In order to
minimize changes, this patch only duplicates the Cmdline::run()
logic.
argv[0] is skipped by Cmdline::parse(), therefore wasn't added to
the updated argument list for the D-Bus, which then basically
ignored the first command line parameter.
Convert relative paths into absolute paths before passing arguments
to dbus server because command line and dbus may have different working
directory.
Leave this kind of task in Command line for it knows the meaning of
parameters and convert them if necessary.
Add a new function 'relToAbs' to convert a relative path to a
canonicalized absolute path.
Added "--status without config name" comment to help output.
Changed the "No running session(s) just now" into a full sentence,
with a slightly more "positive" twist to it:
"Background sync daemon is idle."
This patch changes what is shown to the user and how the user
interacts with the command line. Details below.
"--use-daemon [yes/no]" implies that yes/no is optional (square
brackets!), with "yes" being the option that could be expected for a
plain "--use-daemon" parameter. But the implementation always expects
a "yes" or "no" parameter. The original format suggested was
"--use-daemon[=yes/no]"
This patch switches to that format, changes --use-daemon into --daemon
(to be consistent with --keyring) and enables the same format for
--keyring. Although not documented, 0/f/false and 1/t/true are also
accepted. Because the value becomes part of the parameter, m_argc checks
had to be adapted.
The documentation for "--use-daemon" was inserted in the middle of the
"--keyring documentation".
"Parameter not set" has to be available to the Cmdline caller in the
command line too, in addition to true/false. This was done originally
with a string which is empty, "yes" or "no". Using a tri-state
Cmdline::Bool class makes this a bit more explicit and removes the
ambiguity around what useDaemon() returns (values weren't documented).
When running without --daemon and no daemon available, the
first lines of output were:
ERROR: org.<cryptic error>
[INFO] itodo20: inactive
....
=> The command line should fall back *silently* to running in-process.
=> Error messages should be formatted and logged as such, using SE_LOG_ERROR().
Old code might not have done that and we need to preserve that for compatibility
with frontends, but new code should use [ERROR] as output.
=> D-Bus error messages (as in the case above) must have some user (and developer!)
comprehensible explanation what went wrong. The D-Bus error class itself
is no enough.
Although I haven't tested it, I suspect that the code also would have
re-run the operation in-process after the D-Bus server already
executed it and failed.
I rewrote this so that a check for "daemon available" without error messages
is done first before committing to using the daemon. Once that decision is made,
the command line will not fall back to in-process execution.
Rewrote several error messages. Telling a user of a distro's binary to
"re-configure" is misleading (he didn't configure himself).
"can't" => "cannot", punctuation changes. Not sure whether is always an
improvement, comments welcome.
Comment on coding style: I've used "if ()" instead of "if()" because that is
the GNU recommendation.
Implement cmdline with support of dbus server. To enable cmdline
with dbus server, use the option '--use-daemon yes/no' in case that
you enable dbus service when configuration.
In a typical scenario, a new session is created for the purpose of
execution of arguments. It is scheduled with other sessions but with
a highest priority. Once it becomes active, command line call
'Session.Execute', a newly added method to execute command line
arguments.
The config name of a session should be known for dbus clients like
command line. A new property 'configName' is added in the properties
when calling 'Session.GetConfig'.
CTRL-C handling are processed once executing a real sync to dbus
server. It is mapped to invoke 'Session.Suspend' and 'Session.Abort'.
The meaning of '--enable-dbus-service' is expanded accordingly.
'--status' without server means printing all running session in the
dbus server.
'--monitor' could accept an optional config name. If one is given,
only attach to a session of that config, otherwise print an error.
If none is given, pick the first.
For cases where the Cmdline instance uses a non-standard output stream,
the SyncContext should also use that one. Needs to be tested as
part of MB #5042/5043.
Cmdline and SyncContext standard output should always go
through a variable that can be pointed towards the actual
output channel.
Cmdline already had m_out for that, but it wasn't used
everywhere. Fixed.
SyncContext now has setOutput()/getOutput() and that is
used in SyncContext.cpp. It is not set anywhere yet.
To catch incorrect use of cout or cerr inside SyncEvolution,
the SyncEvo namespace defines its own cout and cerr which cannot
be used like std::cout/cerr, thus triggering compiler
errors. Use "std::cout/cerr" when necessary.
'templateName' property is introduced to present the 'device class'
for a matching device.
The property is optional. There might be templates which only
match one specific model or where the class is so obscure that
showing it is useless.
There are 'template id' and 'template name' concepts inside the core,
'template id' is the handle of the template, user can later use it
to uniquely retrieve the corresponding template, 'template name' on
the other hand is an explict property 'templateName' inside the
template metadata, describing the class of the devices for UI.
'template id' is only used by Cmdline and it use the first entry of
the fingerprint.
We decided against "abusing" the first entry in the
fingerPrint list as the name of the device family/class.
The reason is that there might be templates which have
no such class, which would be impossible to determine
if only "fingerPrint" is available.
Instead the plan is to introduce a separate "templateName"
property.
Note that the sync-ui currently still assumes that the first
fingerPrint entry can be shown to the user. To avoid confusion
(device is called "Nokia N85", shown as "Nokia 7210c") we have
to keep one template per model.
The first entry in the fingerprint is used to uniqely identify the template
(a.k.a template name), it was formulated like 'nokia_7210c' which is not
friendly to user. This patch changed the naming to 'Nokia 7210c', other existing
templates is also updated accordingly.
All duration config properties now use the same parsing code and store
their values as seconds. This changes the interpretation of the auto
sync durations, which used minutes before, but because they were only
introduced very recently, this change is still acceptable.
It would be nice to store values in config files with a hint about the
unit, but we cannot do that because it would break compatibility with
older software expecting plain number of seconds.
The format accepted for all of these options is following ISO 8601,
with some relaxed rules:
/**
* Time interval >= 0. Values are formatted as number of seconds
* and accepted in a variety of formats, following ISO 8601:
* - x = x seconds
* - d = x[YyWwDdHhMmSs] = x years/weeks/days/hours/minutes/seconds
* - d[+]d... = combination of the previous durations
*
* As an extension of ISO 8601, white spaces are silently ignored,
* suffix checks are case-insensitive and s (or S) for seconds
* can be omitted.
*/
class SecondsConfigProperty ...
'peerIsClient' was duplicated in config.ini and template.ini, this
patch reused this field.
The patch also removed the 'default' template for server side, that is not
needed and broken.
This is required when creating the configuration with virtual sources,
otherwise a virtual SyncSource will always be marked as 'disabled' because
it thought there is "no database to synchronize".
Implement blob support. Blobs are stored in the per-peer source
directory in a ".cache" sub-directory. We could have relied on the
peer name and device name to make file names unique when sharing
a common directory. Perhaps that is indeed the better solution:
~/.cache/syncevolution/blobs with logdir specifying the root?
Blob support is only needed when running as server. When running as
client, the binfile layer provides the necessary implementation.
Support for suspend was off by default because <resumesupport> was not
set. Adding it, because we should have whatever is
needed. <resumeitemsupport> depends on the ability to store blobs.
Not sure whether this is relevant on the client side, where these
services are provided by the binfile layer.
When using --print-config or --configure with a context as config name
(like @default), then per-peer properties do not have any effect. This
might be confusing for users, so the command line should better report
this as error and refuse to execute the operation.
As usual, "type" is a bit special because it is both shared and
per-peer.
Instead of a 1-5 valued rank, convert to percentage.
./syncevolution -l \?nokia_7210c
Available configuration templates:
template name = template description matching score in percent (100% = exact match)
Nokia_7210c = Template for Nokia 7210c phone and as default template for all Nokia phones 100%
SyncEvolutionClient = SyncEvolution server side template 40%
ServerDefault = server side default template 20%
Support generating a legacy version SAN package leverating the new
API in libsynthesis.
Added a new configuration option 'SyncMLVersion'(1.0|1.1|1.2), on
the server side, this is used to select the SAN version to send to
the client (default to empty which means it will first try SAN 1.2
and fallback to SAN 1.1 automatically). On the client side, this is
used to select the SyncML version talking to remote server, default
to empty which means it will try the latest commonly supported SyncML
version by the server, when setting a value the client will only try
the specifid version.
The "your SyncML server account name/password" strings inserted into
configs by the command line only make sense when really talking to a
server. In configs on the server side they are harmful, because clients
like phones are then expected to authorize themselves accordingly.
This patch removes the strings from configs where peerIsClient is true.
First, "syncevolution --configure --template scheduleworld@context ..." should
trigger an error. The context, if any, must be specified as part of the config
name.
Second, "syncevolution --configure template scheduleworld foo@bar", when
foo@bar did not exist yet, copied the shared properties from the default
context into foo@bar and the context @bar.
Determining the context had to distinguish between "--template scheduleworld"
and "scheduleworld@bar", which implies the "scheduleworld" template.
The "type" property is special because it must be set in the per-peer
config node (backend and data format) and in the shared config node
(only backend relevant).
The MultiplexConfigNode picks the right node, but that was based on
the (invalid) assumption that the peer nodes would be NULL when not in
use. What really happens is that dummy instances are used, to allow
writing into these nodes. Because of this false assumption, the "type"
for the context was always printed as the default "select backend".
Fixed by telling the MultiplexConfigNode that its peer nodes are
dummies.
In addition, setting the "type" property for a context didn't work
because copyProperties() treated "type" as a per-peer property which
had to be skipped. Fixed by teaching it about the special case.
The D-Bus and Cmdline tests were extended to cover these cases. They
failed (as expected) before fixing the implementation.
The syncevolutionclient template was changed to only match with
'syncevolutionclient' exactly to avoid collision with the server
side template 'syncevolution'.
The cmdline test has to adapt accordingly, added a header line
to describe the meaning of each output column.
1) change the registration order of the properties: before
'preventSlowSync'.
2) newly added 3 properties break unit tests in CmdlineTest.
Fix them.
3) prevent automatic termination when there is any auto sync
task enabled by configs.
4) don't regularly checking auto sync if a config's 'autoSyncInterval'
is set as value '0'. Only do auto sync when changes are detected
in this scenario.
The SyncEvo::CmdlineTest failed because the default values of
preventSlowSync and maxMsgSize were changed recently. The deviceId
prefix change broke another part of the test.
1) Add description for how to implement this feature and give
comments of each class for bluetooth devices detection
2) Move sync device list from BluezAdapter to DBusServer. Thus
operations on sync device list are also moved to DBusServer.
So 'TemplatesChanged' signal could track device list in a more
reasonable way
3) Once default adapter is changed, remove devices in the sync device
list that belongs to this adapter
4) Other minor changes
The Synthesis engine requires that each <datastore> has a unique
integer <dbtypeid>. This is part of the API for progress events
and configuring the <datastore>s.
SyncEvolution uses the source name as unique key and mapped that
to an integer via Hash(), without checking for hash collisions. This
patch fixes that by assigning an ID the same way the first time a
source is used. Hash collisions are checked at that point. Without
hash collisions (the assumption which had to hold already anyway),
the same IDs will be used as before.
The ID is recorded in the new internal "synthesisID", which means that
it won't change even when there are collisions and sources are added
and removed.
Another benefit of this patch is that the source lookup can be done
easily via the unique ID, which is useful for an upcoming patch (which
will rename some sources inside the Synthesis engine).
Because SourceList assigns the unique ID, it has to prevent that users
of it add sources, normal or virtual, without going through
SourceList. Therefore the m_virtualDS and vector base class were made
private.
This patch further clarifies what exactly the role of SourceList is
and that normal sources and virtual sources are handled differently.
This is important so that code can iterate over one or the other set.
GetConfigs returns related templates to the paired bluetooth
devices with specific scores:
1)Query against the underlying template system via device names
got from Bluez and create a list of templates for each device. Since
there might be many devices and one template from system might be
for more than one device. Thus the template names returned to clients
have to be changed. So the rule to change these templates is:
Bluetooth_<mac address of device>_<seq number>, where seq number enumerates
the templates created for this device.
2)GetConfig would get the template based the cached templates info.
Each template matched for devices is added 4 properties:
description - the description for the template
score - the calculated score based on the device name and template
deviceName - the device name that the template is for
fingerPrint - the fingerPrint of the template, used for dbus clients
to re-match with user input device info
Also, the 'syncURL' will be replaced with bluetooth mac address for
these templates.
When adding a source "foo" in peer "xyz" which doesn't exist in peer
"abc" using the same context, the new source appeared as new source in
"abc" (okay, by design because of shared properties) and was enabled
(not okay).
It was meant to be disabled via the "sync = disabled" default value,
except that this default had not been changed when implementing MB
Changed that default now. Because "sync" is set explicitly when
configuring sources, this should have no side effects.
The class turned out to be necessary for SyncConfig testing, once that
started to look for configs in the normal locations in $HOME and
$XDG_CONFIG_HOME (separate patch for that).
1) .template.ini renamed to template.ini
2) removed duplicated configuration templates in the test directory, we will use
the real templates for testing.
3) change default template for Nokia phones to Nokia_7210c
Mostly replaced <server> with <config> and added new text about the
new config layout. Did some general updating of the documentation to
match the improvements found in 1.0.
Because sharing of properties is not documented, I added code which
prints that as part of "--print-config". Lists of sync properties are
printed beneath the main sync section in a comment, for source
properties once after all sources.
Added --print-configs and --print-peers as alias for --print-servers.
Somehow after merging, src/templates/Funambol and src/templates/ScheduleWorld
came back. The WebURL change for ScheduleWorld did not make it into all
the right places on master.
This property is meant to be used by the sync-UI to store the
intended spelling of the peer (mixed case) and store characters
which are not valid for config names. Therefore the value needs
to use escaping rules.
Introduced TemplateConfig to abstracting the template configuration structure,
the template metadata used for matching is also parsed here.
The fields introduced in the metadata are:
PeerIsClient: identify whether this is a server side configuration or a client
side configuration.
Fingerprint: the matching string for this template, it is a comma separated string
with each string modeled as: "Manufacture_Model". The first substring is also
used as the name to identify this template so that user can select the template
by this name.
eg:
Nokia 7210c: Nokia_7210c
SyncEvolution server: SyncEvolutionServer, SyncEvolution
ScheduleWorld: ScheduleWorld,default
SyncEvolution client: SyncEvolutionClient, SyncEvolution
Description: this is a just a descriptive string not used for matching.
GetServerTemplates is changed to add another "devices" parameter to identify
it is asking for templates for a list of "devices". Each device is a tuple
<matchstring (devicename), matchMode (server/client/all)>.
TemplateList as the return type, which is a list of class TemplateDescription
so that we can also return enough information for corresponding templates. This
list is sorted by the 3-tuple <finger, rank, name>.
Add MatchServerTemplates method which will iterating all templates inside the
folder and match against the input parameter and finally return a sorted
list of matched templates.
The atcually fuzzy match algorithm is based on a LCS (added in the following
commit).
Cmdline interface is changed accordingly:
--template ? is changed to --template ?[string], so that user use the former
case to match all templates for a tradiontial SyncML client and the latter case
to match templates related to an input string.
SyncConfig API is also renamed (Server -> Peer) because both server/client
configuration/template are handled.
The original configuration template (Funambol and ScheduleWorld) has been moved
to the new template structure (under servers), they also have a .template.ini
file added so that they can be matched and picked up. All templates for
supported servers still have built-in template support in the code as before.
Templates for SyncEvolution based server is also added.
Server side templates are added (Nokia default, Nokia_7210c and SyncEvolutionServer).
Add unit test for the new template match use case.
As requested by Mark, the webURL was changed from
sync.scheduleworld.com to www.scheduleworld.com, which is the
official entry point for browerser. IMHO sync.scheduleworld.com
has always worked, though, and currently still does.
Goosync provides access to Google calendar, tasks and contacts
via SyncML. Google itself only supports contacts. Memos are not
supported by either of them.
This patch only adds the template with the necessary configuration
parameters. We have done no testing with the service, so it is
not marked as "consumer ready" at this time.
The template uses an unspecific "https://your.company/mobilesync/server"
syncURL. The hope is that users just have to replace "your.company"
in many cases.
https is chosen because apparently, many installations are restricted
to that.
"syncevolution --configure" works by copying the config with modified
properties set as filters. This did not work for adding sources because
SyncContext::copy() only worked on *existing* sources.
Changed the semantic of the sourceFilter parameter in
SyncContext::copy() so that it specifies *exactly* which sources are
to be copied instead of acting as a filter.
If the given set is a subset of the existing sources, this does not
change anything (SyncContext.cpp dummy.copy(*this, &activeSources)).
If the given set is a superset, then now the additional sources will
be copied where they were ignored before (Cmdline.cpp). There are no
other calls to SyncConfig::copy(), so this change is safe.
Added a new CmdlineTest to cover this situation.
The command line did not read the global defaultPeer property unless
it found an existing context. This is an unnecessary optimization,
reading from an non-existant context yields no properties except
perhaps for defaultPeer, so we can do that and get the correct
result with and without a context.
Jussi mentioned that he didn't get defaultPeer when reading a
template. Added TestMultipleConfigs.testSharedTemplate to verify
that, but it passes without issues.
Broke while adding support for shared configs. We didn't have a test
for it, therefore this went unnoticed. I have added a test for it
(CmdlineTest::printFileTemplates) and fixed the issue (using the wrong
file layout for templates).
The test depends on having access to installed templates in a known
location, which previously was a problem due to the hard-coded
TEMPLATE_DIR. Now SyncContext::createServerTemplate() checks an env
variable (SYNCEVOLUTION_TEMPLATE_DIR) which is set by most of the
command line tests to prevent reading from there unintentionally and
once to "./templates" to ensure reading from there.
This depends on peer/context awareness in the code which looks for
the old config and renames it. If the old config is not in the same
context as the new one, it is searched by peer name alone. This
works for all old-style configs as well as rewriting a peer inside
the same context.
Without copying this directory, using the migrated configuration will
lead to a slow sync.
SyncEvo::CmdlineTest::testMigrate was extended to cover this problem.
Manually testing shows that the slow sync is indeed avoided with
this change.
remoteDevId must be user-visible, because it is necessary to configure
it in advance when the peer is an HTTP SyncML client.
Also updated several property documentation strings. We have to
explain how they apply when the local side acts as SyncML server
and/or OBEX is involved.
When adding a new peer, existing shared properties must be
preserved. The same applies to migrating (= copying) and old
configuration.
To be consistent, printing templates also needs to take existing
shared properties into account.
Special treatment is necessary for the "type" property: it must be
taken from the peer, not from the existing config, so that the peer
can choose a data format for himself. The peer config must be
consistent with regards to the backend for this to work.
The code is similar to syncevo-dbus-server, but not identical because
temporarily overriding properties is done differently. Therefore
SyncConfig::createServerTemplate() has been left as it is (= unaware
of shared properties).
The client-test SyncEvo::CmdlineTest was updated to cover some
of these case.
Root of the tested configuration files is the "default" context. The
global property "defaultPeer" is thus not covered and must be ignored
in the --print-config output (where it is included).
Removing and scanning for properties of a specific peer must take the
"peers" hierarchy into account. Some tests work with more than one
peer configuration at a time.
The "type" property is duplicated. This is only visible in a file
system dump, not in the --print-config output.
The reference ScheduleWorld config dump uses the order of properties
as in the registry so that it can be converted into the --print-config
.ini format automatically. When the order produced by scanning files is
required, the lines have to be sorted first.
The old sync4j-style config is no longer derived from the reference
ScheduleWorld config. Instead it is stored as plain text; this was
easier than updating the transformation.
One problem found in testMigrate is that obsolete shared properties
are not removed. Removal of obsolete properties was a side effect of
rewriting a config under a new name as part of --migrate; it still
works for per-peer properties. This is a minor feature, so I don't
intend to reimplement this for shared properties.
The tests do not yet cover proper creation of templates when there
are already existing shared properties. Needs to be added together
with a fix for the problem.
A global property is not useful yet at the moment because there is
no way of sharing or reading it outside of a specific peer
configuration. This property will be useful once shared properties
truly work. Adding it now simplifies testing.
The property uses an escape mechanism to store unusual characters,
although that shouldn't be necessary in practice. The motivation is
that it is typically read and written by frontends, which benefit
from the escape mechanism (in contrast to normal users editing the
config file directly).
The property is user-visible, just in case that someone is interested.
That SafeConfigProperty automatically sets the "hidden" flag was
unexpected. Removed that behaviour, in particular given that all
other such nodes already set the flag explicitly anyway.
This patch introduces code changes for the new layout without actually
using it yet. Therefore all existing tests for the older layout
still pass. The new meaning of the former "server name" is introduced:
- A plain string now refers to a peer configuration (can be client
or server).
- The @ sign allows selecting a specific context. Different contexts
have independent sets of local sources and peer definitions.
- An empty peer name selects a view on the configuration which contains
no peer-specific properties. Not fully implemented yet.
The FileConfigTree is instantiated with a root which is one level high
up compare to before this patch (for example,
"~/.config/syncevolution" instead of
"./config/syncevolution/scheduleworld") and then config files include
that dropped level in their relative path name
("scheduleworld/config.ini" instead of "config.ini"). This allows
accessing the global properties in
"~/.config/syncevolution/config.ini" and will be used to move peers
further down the hierarchy
("~/.config/syncevolution/peers/scheduleworld/config.ini").
To keep the output of "--print-servers" consistent, the FileConfigTree
gets another parameter which identifies the subset of the larger tree
that is referenced by this FileConfigTree instance.
One side effect of this change is that FileConfigTree instances are no
longer completely separate. Something to keep in mind when
instantiating SyncContext multiple times (MB#8006).
Code may no longer make assumptions in which config node a property is
stored. This is determined by the new getNode() calls based on the
property attributes (hidden, sharing). The new layout is represented as
a set of config nodes. Older layouts use the same set of nodes with
identical instances assigned to them, if they don't really have separate
files for each of them.
SyncSourceNodes no longer grants direct access to the nodes, to catch
code which incorrectly access a specific node directly. For the same
reason the name of the nodes changed.
Code which needs access to all hidden or all visible properties now
does this via a config node returned by getProperties(). Currently
this is identical to the underlying nodes. Once the new layout is
active, this node will act as a multiplexer which gathers properties
from all underlying nodes when reading and picks the right one when
writing.
The "change ID" parameter for sources has been obsolete for a while
and was removed.
Reorganized the property registration so that it is a bit easier
to see which properties are hidden and which use non-default sharing.
The default sharing is "no sharing".
Some other code was improved while touching it:
- removed useless visibility[] array in favor of a i != 0 check in
SyncConfig::copy()
- added default parameters to save/checkPassword() methods
- some constness definition changes
- Property::getProperty() is a virtual call which could only be
overloaded in one case because the constness was wrong; now
getProperty() always returns the string value and getPropertyValue()
some other kind of representation of it, depending on the
class.
- ConstSyncSourceNodes is based on SyncSourceNodes instead of duplicating
it, which simplifies the implementation.
The simplified SyncSourceAdmin API changed slightly: instead of passing
a pointer to the source's SyncSourceNodes, the default nodes are now
found via the SyncSource pointer. For callers this is a bit less
work and it is more general.
The hidden "last" property hasn't been used since moving to Synthesis
and also is not needed for the new server mode. Some virtual methods
could be removed because they no longer are called. Some comments
referred to an obsolete distinction between properties defined by
libfunambol and SyncEvolution.
We used to have the strings "your SyncML server account name" and
"your SyncML server password" hard-coded in the code which generated
templates. As Jussi pointed out, this is a problem for GUIs which have
to hide these strings from their users.
The purpose of these strings was to have them in configs in cases
where the user forgot to set them, as a reminder.
This path moves the strings into the command line config setup,
thus achieving both of these goals.
Use "peerIsClient" configuration property to decide whether this should be a
Server Alerted Sync Session or a client initated.
Use "remoteIdentifier" as the string sending to the remote peer during SAN so
that remote peer can identifier the server properly.
If this should be, will first generate and send SAN in SyncContext::doSync,
get the client request and inialize the session as server with the received
request.
The Cmdline tests hadn't been updated together with adding the
new options and therefore failed. New properties must be added
to the reference config m_scheduleWorldConfig and new internal
properties also to the props list in testOldConfigure.
While checking the test failures, the following problems were found
and fixed:
- "adminData" property name was used for both sync and source property.
The sync property stores Synthesis device information and was
renamed to "deviceData" to avoid confusion. SyncConfig API calls
renamed accordingly. Property instance now follows syncProp* naming
convention.
- The source "adminData" property was not declared as "hidden"
and therefore showed up in the user-visible config.ini files
and --print-config output. The real value set/get in SyncSourceAdmin
was done correctly via the m_hiddenNode.
The order of the servers dependend on the order in which the
filesystem returned the directory names, in other words, it
was pretty random. Now the list is sorted alphabetically by
the server name.
This was found in the testPrintServers when the order of
the servers started to deviate from the (also unsorted) order
in the reference output.
Previously, sync failed because the datastore configuration specified
no means of storing the admin data (single chunk of text and local ID/remote ID
mapping). This could have been added by configuring the SDK_textdb as
<plugin_admin_module>.
Instead this patch implements the functionality inside SyncEvolution,
using a new ".server.ini" config node for the mapping and an
internal "adminData" property for the admin data text chunk. This is
more natural because it keeps the data under our control.
Authentication is configured via <simpleauthuser/pw>. In contrast to
using the SDK_textdb, this allows choosing the username and password.
Ultimately this should also be done inside SyncEvolution, to avoid
writing the username/password into the XML (unsolved encoding issue)
and into log files (privacy issue).
Selecting active sources during a sync or status check was done with a
combination of setting a sync mode via a source config filter and
setting a list of active sources. Now the SyncConfig supports a source
filter which is applied to all sources and source filters for each
source. The latter override the former.
This is powerful enough to start syncs with full control over which
sources are active in which mode, as described in the new D-Bus API.
As part of this patch, the command line semantic is implemented
entirely using a combination of different source filters.
The intention is to get rid of the historic and inconsistent
naming of some classes and their corresponding files:
* EvolutionSyncClient = class derived from Funambol's SyncClient,
* SyncEvolutionConfig = SyncEvolution's config
With the strict 'namespace SyncEvo' and the syncevo/ path prefix for
most header files it is no longer necessary to have "SyncEvolution" or
"Evolution" in the names. This patch thus renames as follows:
EvolutionSyncClient => SyncContext
EvolutionSmartPtr => SmartPtr
SyncEvolutionCmdline => Cmdline
SyncEvolutionConfig => SyncConfig
SyncEvolutionUtil => util
The former EvolutionSyncClient always had a role that went beyond just
running a sync, for example it also provided config access. With the
upcoming server support it also won't be just a client. Thus the new
name "SyncContext".
The 'syncevo/' prefix is used throughout the code now.
removed whenever the prefix made it clear that the file belongs
to SyncEvolution. This helps finding incorrect include paths.
Quotes should be used exclusively for SyncEvolution files which don't
have a specific prefix yet (test.h, config.h) to help identifying
them.
