Recent g++ on Fedora Rawhide warns that the dynamic cast result
is used without NULL check. We know that the cast must succeed,
so a static cast is more appropriate.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
std::unique_ptr usually can be used instead. std::vector also works
for arrays and even has a data() method as part of the official API in
C++11.
For historic reasons, the functions creating SyncSources returned
plain pointers. We are breaking the API now, so we might as well fix
that.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
We can use std::shared_ptr and std::function instead now.
Lambdas are usually a better alternative to boost/std::bind. The
downside is the need to explicitly specify parameters completely. When
inlining callbacks entirely with lambdas, duplication of that
parameter list can be avoided.
Whenever possible, use std::make_shared to construct objects that are
tracked by std::shared_ptr.
Some objects need a std::weak_ptr during object destruction. For that
we have to use our own implementation of std::enable_shared_from_this,
with a matching creator function. The additional benefit is that we
can get rid of explicit static "create" methods by making that create
function a friend.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
boost/foreach.hpp is no longer needed, range-based loops work
the same. With some helpers, even reverse iteration and
boost::make_split_iterator() can be handled the same way.
"auto" makes it possible to avoid explicitly spelling out the
expected type.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Running with no sources leads to exception later on and isn't
intended; it makes more sense to treat an empty env var like an unset
one and use the default source list.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
auto_ptr has been deprecated for a while now. unique_ptr can
be taken for granted now, so use that instead.
GDBusMessage requires a custom deleter. Not sure how auto_ptr
handled that before.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
"username", "proxyUsername" and "databaseUser" used to be simply a
string containing the name of the respective user or (in the case of
the ActiveSync backend) the account ID in gconf.
Now it is also possible to provide credentials (username + password)
indirectly: when any of these properties is set to "id:<config name>",
then the "username/password" properties in that config are used
instead. This is useful in particular with WebDAV, where credentials
had to be repeated several times (target config, in each database when
used as part of SyncML) or when using a service which requires several
configs (Google via SyncML and CalDAV).
For user names which contain colons, the new "user:<user name>" format
must be used. Strings without colons are assumed to be normal user
names.
This commit changes the SyncConfig APIs for this extension. More work
is needed to make the indirect lookup via "id" functional.
"username", "proxyUsername" and "databaseUser" used to be simply a
string containing the name of the respective user or (in the case of
the ActiveSync backend) the account ID in gconf.
Now it is also possible to provide credentials (username + password)
indirectly: when any of these properties is set to "id:<config name>",
then the "username/password" properties in that config are used
instead. This is useful in particular with WebDAV, where credentials
had to be repeated several times (target config, in each database when
used as part of SyncML) or when using a service which requires several
configs (Google via SyncML and CalDAV).
For user names which contain colons, the new "user:<user name>" format
must be used. Strings without colons are assumed to be normal user
names.
This commit changes the SyncConfig APIs for this extension. More work
is needed to make the indirect lookup via "id" functional.
Since the introduction of SuspendFlags, the ony remaining user
of the virtual aspect of checkForSuspend/checkForAbort() was the
testing code. By using the suspend/abort request mechanism in
SuspendFlags, it becomes possible to move checkForSuspend/Abort()
into SuspendFlags itself.
This will be useful to use it outside of a SyncContext member.
Backends are now allowed to create RegisterSyncSourceTest instances
with empty name. They will be ignored except that their init() method
will be called after main() and before the list of source tests is
used.
This can be used to create additional RegisterSyncSourceTest instances
at a time when the test of libsyncevolution is initialized. This is
needed by the WebDAV backend, which has to instantiate sources and use
the config system. That crashed when done before the ConfigProperty
instances were initialized.
The WebDAV backend must support different kinds of items in the same
collection. The new testLinkedSources covers this by adding, updating
and deleting an item of one kind and checking that other sources
referencing the same database do not see these changes.
This test must be activated for a specific source by adding links to
other sources using the same database. A separate commit will do that
for WebDAV.
Due to the way how properties were handled internally, it wasn't
possible to explicitly set a property to its default value. Instead
the property was unset. For example, explicitly setting database= was
not possible.
This is necessary for client-test and ActiveSync, because client-test
needs to know that the testing is expected to run with the default
databases (something which normally is avoided by overwriting empty
database properties).
Now the "is set" state is tracked explicitly in the config storage and
command line property APIs. Unsetting a property via the command line
could be implemented with an explicit command line option, but is not
supported at the moment.
Tests were extended to cover the new functionality and adapted to the
change behavior for "type" migration: syncFormat was empty
already (because the empty string matched the default), but
forceSyncFormat was unnecessarily set explicitly. Now it is not.
"database", "databaseUser", "databasePassword" must not be set by
client-test when explicitly set to empty. That is necessary for
ActiveSync, where database= means "use default database" and
explicitly selecting that database via --print-database is not
currently supported.
KWallet support was broken: syncevo-dbus-server checked
KDE_FULL_SESSION to determine whether it should use KWallet instead of
GNOME Keyring. That did not work, because the env variable was not set
for D-Bus daemons.
Automatically detecting KDE users is not possible at the
moment. Instead KDE users have to manually set the new "keyring"
global config property to "KDE" (case insensitive) if the
SyncEvolution installation supports both, because GNOME Keyring is the
default to avoid surprises for traditional users. If only KWallet
support is enabled, then this is not necessary.
"GNOME" and "true/false/1/0/yes/no" can also be set. This has the
advantage that keyring usage can be enabled permanently for the
command line in --daemon=no mode; normally keyrings are not used in
that mode because accessing them can bring up UI dialogs.
It also becomes possible to disable keyring usage in syncevo-dbus-server,
something which couldn't be done before.
The --keyring command line option is still supported, as an alias for
"[--sync-property] keyring=<value>". The default value for --keyring
is true, to match the traditional behavior. In contrast to other sync
properties, setting "keyring" does not require an explicit --run
parameter. Again this is done to mirror traditional usage.
Reading a password also (unintentionally) checked all supported
storages while searching for the password. Now it uses exactly
one storage and falls back to asking for the password directly.
The commit itself also cleans up the code a bit (reformatted, fixed
comments). Choosing the right slot in the password signals is done via
a new InitStateTri parameter which contains the "keyring" setting.
Error checking (unsupported keyring string, --keyring=yes and no
keyring enabled) is done in additional slots which run after all the
regular ones.
Parameter parsing for --sync and --keyring were unified. However,
there is the difference that --keyring has an implicit default value
("yes") and never has an additional parameter, in contrast to --sync,
which always is followed by one.
The new CmdlineTest::testKeyring covers different ways of using
--keyring. It relies on actually invoking keyring backends, something
not done by the default SyncContext UI. Therefore
CmdlineSyncClient+KeyringSyncCmdline were moved into libsyncevolution,
to be used by CmdlineTest.
The SyncContext::initMain() must be called in main(), to give
all global instances a chance to influence the operation (like
registering platform init code). This failed for KDE when
the order of global instance instantiation happened to be wrong.
Setting signal handlers there also makes more sense.
The reason for doing process initialization in src/client-test-app.cpp
was that the rest of the code was meant to be SyncEvolution and Unix
independent. That's less of a concern today.
A SyncEvolution HTTP server running locally can be configured (outside
of client-test!) to use "Client_Sync_Current" as logdir. That symlink
will be pointed to a new, clean directory each time a sync runs. That
way it becomes possible to associate the server side of a
SyncEvolution<->SyncEvolution test with the specific test logs on the
client side.
The main goal is to test CalDAV/CardDAV sources as part
of a SyncML client and/or server. A test involving syncevo-http-server
is now named "<client storage><server storage>":
- edsfile = EDS in client, file in server (used to be syncevohttp)
- davfile = CalDAV/CardDAV in client, file in server (new)
- edsdav = EDS in client, CalDAV/CardDAV in server (new)
For this, WebDAVSourceRegister.cpp must be able to create test sources
which match the client 1/2 sync configs. The client "1" or "2" strings
are passed through the abstract ClientTest into the source A/B create
callbacks. WebDAVSourceRegister.cpp cannot get this directly from
ClientTest because it lives in a plugin which is not necessarily
linked against ClientTest.
A conceptual change is that CLIENT_TEST_EVOLUTION_PREFIX/USER/PASSWORD
no longer override existing config properties. That is necessary
because the shared prefix is too simplistic for WebDAV (needs full URL
in "database"); also helps KDE (needs resource URI). The env variables
and the default "SyncEvolution_Test_" value for the database prefix are
still used if the config does not exist. That is useful to prevent
accidentally running client-test against the default databases.
The nightly setup script might (should!?) be made public to simplify
configuring a server.
Another change is the user-configurable part of client-test now lives
entirely in the _1/_2 client sync configs and contexts. From there the
source properties are copied into the Client::Source context each time
client-test runs.
client-test now uses the same process initialization as all other
SyncEvolution binaries and thus uses the new glib log handler.
Also removes some code duplication.
The glib output redirection test became a bit redundant, but let's
keep it. For that we need to restore the default glib log handler
while that test runs.
The memset/memcpy of the embedded boost::function instances inside the
old ClientTestConfig was causing segfaults at the end of a client-test
run if compiled with optimization.
Therefore this commit turns ClientTestConfig into a proper class
containing members which initialize themselves (Bool wrapper class,
std::string), thus memset is no longer needed and used. Also added the
standard m_ prefix.
m_numItems is gone, was never set by any backend anyway and even
expected to be consistent in one test. Now CLIENT_TEST_NUM_ITEMS is
read by defNumItems() each time it is needed.
Removed "const char *" strings from method parameters. This revealed
that config.itemType (a const char *) was incorrectly passed to
insert() where the boolean "relax" parameter should have been given.
Replaced by "false" (= strict checking) even though the old code
must have run with an implicit "true" (= relaxed checking). Let's see
whether any tests fail now.
With Google Calendar, having a VALARM leads to different communication
compared to the case without VALARM (requires resending). Test both
cases by providing multiple sets of linked items.
Client::Source now contains LinkedItems_1 and LinkedItems_2
sub-groups. This required changes in resultchecker.py, to keep these
tests as listed in the same table as the other source tests.
ClientTestConfig becomes more complicated: it used to be a plain C
struct which could be copied/cleaned with memcpy/memset. This approach
is kept by adding a pointer to a std::vectore. A nicer solution would
be to turn all "const char *" into std::string and int/bool values
with wrapper classes which initialize them.
For testing sources instantiated via SyncSource.cpp (for example, EDS)
and client-test-app.cpp (ActiveSync) append the name of the server set
via the CLIENT_TEST_SERVER env variable to target-config@client-test,
separated by a hyphen.
This allows configuring testing differently for different servers.
Was already done for CalDAV sources, inside WebDAVSourceRegister.cpp.
More global initialization/destruction ordering problems appeared
on current Debian Testing (gcc 4.6.1, binutils 2.21.52.20110606-2):
- WebDAVCredentialsOkay registered and visibility modified before
constructed, which later resets visibility
- test setup uses config before config properties are registered
- logger stack deallocated before later destructor attempt to use it
Fixed with:
- WebDAVCredentialsOkay() singleton
- delay all work in test setup until really needed
- never deallocate logger stack, use singleton
As discussed on the mailing list, "source-config" is ambiguous because
the "addressbook/calendar/..." configs are also called "source
configs".
Now the naming is "sync" config (for the config with syncURL=local://,
because it is used for syncing) and "target" config (because it is
used as target in a sync config's syncURL).
Rejected:
"local" config - because the databases are not necessarily local
"source" config - see above
"client" or "server" config - because both sides might use local data
and/or client/server could refer to the role
of the peer or the SyncML client/server model
used internally
The distinction between vcard21 and vcard30 became mute in the Evolution
backend a while ago. Both tests ended up using the vCard 3.0 Evolution
tests data and the default uri for each server. This patch removes
the vCard 2.1 special case.
It also renames the tests and test data to reflect that they always
were Evolution specific. The new naming convention, also applied
to file, QtContacts, KCalExtended, XMLRPC, Maemo and Akonadi backends, is
now <backend>_contact/event/task/memo, with eds/file/qt/kcal/maemo/kde
as backend names.
The reasoning is:
- results in unique string (in particular no overlap with
backend type names), easier to search for
- underscore already used before (in contrast to hyphen)
- no plural-s to keep the name shorter
The Akonadi backend should be using its own test data instead of
the Evolution ones.
The intention is that backends shall be able to bind additional
parameters to their implementation of the callbacks. The default
implementation used by most (all?) backends was adapted, so no
changes there are necessary.
Doing setSourceType() unconditionally unintentionally also removed a
configured syncFormat and forceSyncFormat setting for an existing
configuration. Only setBackend() (as with this patch) is necessary
and valid in all cases (never run a test with the wrong backend!).
An incomplete source config might already exist. Better set the source
type in all cases, to ensure that we really run with the right backend
and data format.
The meaning of "type" was horribly complex and had effects on the
backend and the peer. It was impossible to specify the sync format to
be used for a specific peer independently of the local backend and its
format, so adding a peer to a context broke the context configuration
(BMC #1023).
This is now fixed by splitting "type" into four independent properties:
- backend = plugin which interfaces with the data
- databaseFormat = data format used inside backend, only relevant for file backend
- syncFormat = data format preferred when talking to peer
- forceSyncFormat = disable format auto-negotiation, use preferred format
With that split, it is now possible to specify the format in which the
file backend stores items independently of the format in which they
are exchanged with the peer.
Old configurations with "type" can still be read. The values specified
inside it are transparently mapped to the new properties. Command line
and D-Bus API users will only see the new properties.
The command line tool still accepts "type" as an alias for the four new
properties. Using that has the same disadvantage as before: it will modify
the context even if only modifying the peer was intended.
The D-Bus API accepts only the new properties. Clients using "type"
must be adapted to the new property names. Clients not using that
continue to run unchanged.
Writing into the configuration requires a migration of the peer config
*and* the context in which it is defined. That is necessary because
the new semantic (independent database format) cannot be stored in the
old format. The migration is handled by rewriting first the context,
then all peers defined inside it.
Other user-visible changes:
- updated help texts
- the canonical "backend" value for the file backend is just "file"
instead of the long "Files in one directory", which is now an alias
(used to be the other way around); done because "type = file"
was expanded to the long name, which was a bit unexpected and showed
how unintuitive the long name is
Internal changes:
- getMimeVersion() is still present, although it hasn't been used
for a long time; FileSyncSource::getMimeVersion() now derives
the version from the supported Mime types, in case that the
function will be needed again in the future
- setSourceType() with string as argument was replaced with one
taking a SourceType instance; to emulate the old behavior if
desired, construct SourceType from an old-style string
- ConfigProperty methods need to be virtual so that derived classes
like SourceBackendConfigProperty can generate content at runtime
(a recent commit broke that feature)
- File templates were stripped down to the essential properties,
with "type" replaced by the per-peer "syncFormat". "type" would
still have been accepted (so it is not necessary to adapt
syncevo-phone-config right away), but has the original
disadvantage of modifying "backend" and "databaseFormat".
The goal is to allow multiple names for properties. This will be used
to replace legacy names like "evolutionsource", but it could also be
used to allow abbreviations on the command line.
First the patch replaces the single ConfigProperty::m_name with a list
of names, of which the first one is the current name of the
property.
The name that is to be used depends on the node which is to be
manipulated: if it already has a property under an alias, that alias
is read and also written, so existing configuration continue to use
the old config name (avoids the problem of having to remove the old
name and insert the new one at the same spot in a .ini file). Old
configs continue to parse okay. Writing into node which has none of
the names set, as in migrating into a fresh config, uses the new
names.
Most of the patch deals with the removal of a single, unique name,
either by picking the name so that it matches a node, using the
default name or simply hard-coding a property name ("sync").
The command line accepts all aliases, then uses the main name for
overriding configuration options.
Conflicts:
configure-pre.in
src/syncevo/SyncContext.cpp
src/syncevo/configs/datatypes/10calendar-fieldlist.xml
test/ClientTest.cpp
test/client-test-main.cpp
test/synccompare.pl
Conflicts all resolved by using the code from "master" branch. Caused
by similar changes on release branch.
To enable it in configuration, use '--enable-buteo-tests'.
To test client-test with buteo to do sync, make sure the
environment variable 'CLIENT_TEST_BUTEO=1' is set.
Three main steps to support buteo testing in client-test,
1) pre-run: storage preparation, clean up buteo logging
files and set corresponding keys in meego-sync-conf.xml.
For qtcontacts, switch backend database files for each
client; For calendar/todo/note, specify different notebook
names for each client.
2) run: re-implement ClientTest.doSync function, which
dbus-calls 'msyncd', the dbus daemon of buteo to do sync.
Listen to needed signals to track its status.
3) post-run: collect its status and summarize its statistics
and convert to internal sync report.
Buteo doesn't support explicit slow-sync mode and refresh-
from-server mode. Add two workarounds for them:
For slow-sync mode, wipe out anchors;
For refresh-from-server, wipe out anchors and clean all local
data and do two-way sync.
To enable buteo testing in your environment, you have to:
1) Add read/write privilege for the directory /etc/sync for
current testing user
2) To test one remote server, make sure the profile xml is put
in $HOME/.sync/profiles/sync. The name of the profile xml should
be set as the server name.
When running testTwoWaySync, there's no guarantee that the previous sync
succeeded. As a result, the sync may turn into a slow sync. That's okay,
the test was meant to allow that. The problem is that preventSlowSync
then turns that into a sync error, which breaks the test. Seen once in
nightly testing and then reproduced manually.
This patch avoids the problem by disabling preventSlowSync temporarily for
each test which may run in two-way (requested sync mode) and is allowed
to end up with different modes (checked sync mode). It could be extended
to other requested sync modes, but that wasn't necessary so far and isn't
done to minimize changes.
Backends like XMLRPC need information about URL, proxy and SSL
settings, etc. This can be done via source specific properties, like
evolutionsource, but this is not how this is normally done. It would
be nicer if the exising per-peer properties could be used. The goal is
that a normal peer configuration can be created from a template with
the necessary information to enable sources using that information.
This patch makes this possible by adding a context parameter to
SyncSourceParams:
* @param context Additional non-source config settings.
* When running as part of a normal sync, these are the
* settings for the peer. When running in a local sync,
* these settings come from the "source-config" peer
* config inside the config context of the source.
* Testing uses "source-config@client-test". On the
* command line, this is the config chosen by the
* user, which may or may not have peer-specific settings!
Note that this still doesn't solve the problem for XMLRPC to SyncML
peer sync, because in that case ("normal sync") the context will be
the one describing the peer. SyncURL is already used and proxy
settings might not match.
The XMLRPC backends therefore was not changed and continues to use
evolutionsource.
"text" stands for Evolution memos and depends on
EvolutionCalendarSource. Added.
The name lookup differs for local and for sync indices, must do
the name lookup before calling checkEvolutionSource(). Fixed.
Destroying ORBit based Evolution backends at the very end of the
process life time caused assertions in ORBit. Added explicit cleanup
code which is called before triggering the normal library shutdown by
leaving main().
The D-Bus based Evolution Data Server calendar and contact daemons
(e-calendar-factory and e-contact-factory) shut down after 10 seconds of
inactivity.
Normally libecal/libebook detect that, but not when the client only uses the
synchronous API ("destroyed" signal not delivered, see e-cal.c). That affects
SyncEvolution.
This patch ensures that a EvolutionContactSource resp. EvolutionCalendarSource
remains open throughout the lifetime of the client-test app. This prevents
the storages from shutting down.
This instances are created on demand and deleted when the process quits.
This might be too late for ORBit based Evolution; there were error messages
after all tests succeeded and an assertion failure in ORBit, leading to
a nonzero return code.
do checking to avoid duplicate before pushing back a source
into m_localSource2Config for TestEvolution.
Virtual sources may contain many sub-sources, which might duplicate
other sources.
When glib logging is invoked, it check the application name and complains
if not set:
** (process:10130): WARNING **: g_set_application_name not set.
With output redirection we hide this in the command line, but it
showed up in client-test when the system was in a state were another
glib WARNING was triggered.
This patch sets the unlocalized program name instead of using a
localized application name, because we don't have and don't need
localization for these two programs. That seems to satisfy glib.
With "syncevolution" using the D-Bus server and thus the keyring
by default, the client-test executable should better also use
the keyring. Otherwise setting the password via the command line
will render a test config unusable for client-test.
The patch turns out to be very easy: reusing the code in
CmdlineSyncClient via changing the inheritance of the sync engine used
by client-test-app.cpp plus adapting compile flags is enough.
Since we support both evolution and file backend for supported test
case in client-test.
It might be useful running client-test with calendar+todo test cases
using file backend. (It supports evolution backend only without this
patch).
Patrick Ohly
Salvatore Iovene
Gabriel Schulhof
Zhu, Yongsheng
Chen Congwu