The goal is to allow multiple names for properties. This will be used
to replace legacy names like "evolutionsource", but it could also be
used to allow abbreviations on the command line.
First the patch replaces the single ConfigProperty::m_name with a list
of names, of which the first one is the current name of the
property.
The name that is to be used depends on the node which is to be
manipulated: if it already has a property under an alias, that alias
is read and also written, so existing configuration continue to use
the old config name (avoids the problem of having to remove the old
name and insert the new one at the same spot in a .ini file). Old
configs continue to parse okay. Writing into node which has none of
the names set, as in migrating into a fresh config, uses the new
names.
Most of the patch deals with the removal of a single, unique name,
either by picking the name so that it matches a node, using the
default name or simply hard-coding a property name ("sync").
The command line accepts all aliases, then uses the main name for
overriding configuration options.
Conflicts:
configure-pre.in
src/syncevo/SyncContext.cpp
src/syncevo/configs/datatypes/10calendar-fieldlist.xml
test/ClientTest.cpp
test/client-test-main.cpp
test/synccompare.pl
Conflicts all resolved by using the code from "master" branch. Caused
by similar changes on release branch.
To enable it in configuration, use '--enable-buteo-tests'.
To test client-test with buteo to do sync, make sure the
environment variable 'CLIENT_TEST_BUTEO=1' is set.
Three main steps to support buteo testing in client-test,
1) pre-run: storage preparation, clean up buteo logging
files and set corresponding keys in meego-sync-conf.xml.
For qtcontacts, switch backend database files for each
client; For calendar/todo/note, specify different notebook
names for each client.
2) run: re-implement ClientTest.doSync function, which
dbus-calls 'msyncd', the dbus daemon of buteo to do sync.
Listen to needed signals to track its status.
3) post-run: collect its status and summarize its statistics
and convert to internal sync report.
Buteo doesn't support explicit slow-sync mode and refresh-
from-server mode. Add two workarounds for them:
For slow-sync mode, wipe out anchors;
For refresh-from-server, wipe out anchors and clean all local
data and do two-way sync.
To enable buteo testing in your environment, you have to:
1) Add read/write privilege for the directory /etc/sync for
current testing user
2) To test one remote server, make sure the profile xml is put
in $HOME/.sync/profiles/sync. The name of the profile xml should
be set as the server name.
When running testTwoWaySync, there's no guarantee that the previous sync
succeeded. As a result, the sync may turn into a slow sync. That's okay,
the test was meant to allow that. The problem is that preventSlowSync
then turns that into a sync error, which breaks the test. Seen once in
nightly testing and then reproduced manually.
This patch avoids the problem by disabling preventSlowSync temporarily for
each test which may run in two-way (requested sync mode) and is allowed
to end up with different modes (checked sync mode). It could be extended
to other requested sync modes, but that wasn't necessary so far and isn't
done to minimize changes.
Backends like XMLRPC need information about URL, proxy and SSL
settings, etc. This can be done via source specific properties, like
evolutionsource, but this is not how this is normally done. It would
be nicer if the exising per-peer properties could be used. The goal is
that a normal peer configuration can be created from a template with
the necessary information to enable sources using that information.
This patch makes this possible by adding a context parameter to
SyncSourceParams:
* @param context Additional non-source config settings.
* When running as part of a normal sync, these are the
* settings for the peer. When running in a local sync,
* these settings come from the "source-config" peer
* config inside the config context of the source.
* Testing uses "source-config@client-test". On the
* command line, this is the config chosen by the
* user, which may or may not have peer-specific settings!
Note that this still doesn't solve the problem for XMLRPC to SyncML
peer sync, because in that case ("normal sync") the context will be
the one describing the peer. SyncURL is already used and proxy
settings might not match.
The XMLRPC backends therefore was not changed and continues to use
evolutionsource.
"text" stands for Evolution memos and depends on
EvolutionCalendarSource. Added.
The name lookup differs for local and for sync indices, must do
the name lookup before calling checkEvolutionSource(). Fixed.
Destroying ORBit based Evolution backends at the very end of the
process life time caused assertions in ORBit. Added explicit cleanup
code which is called before triggering the normal library shutdown by
leaving main().
The D-Bus based Evolution Data Server calendar and contact daemons
(e-calendar-factory and e-contact-factory) shut down after 10 seconds of
inactivity.
Normally libecal/libebook detect that, but not when the client only uses the
synchronous API ("destroyed" signal not delivered, see e-cal.c). That affects
SyncEvolution.
This patch ensures that a EvolutionContactSource resp. EvolutionCalendarSource
remains open throughout the lifetime of the client-test app. This prevents
the storages from shutting down.
This instances are created on demand and deleted when the process quits.
This might be too late for ORBit based Evolution; there were error messages
after all tests succeeded and an assertion failure in ORBit, leading to
a nonzero return code.
do checking to avoid duplicate before pushing back a source
into m_localSource2Config for TestEvolution.
Virtual sources may contain many sub-sources, which might duplicate
other sources.
When glib logging is invoked, it check the application name and complains
if not set:
** (process:10130): WARNING **: g_set_application_name not set.
With output redirection we hide this in the command line, but it
showed up in client-test when the system was in a state were another
glib WARNING was triggered.
This patch sets the unlocalized program name instead of using a
localized application name, because we don't have and don't need
localization for these two programs. That seems to satisfy glib.
With "syncevolution" using the D-Bus server and thus the keyring
by default, the client-test executable should better also use
the keyring. Otherwise setting the password via the command line
will render a test config unusable for client-test.
The patch turns out to be very easy: reusing the code in
CmdlineSyncClient via changing the inheritance of the sync engine used
by client-test-app.cpp plus adapting compile flags is enough.
Since we support both evolution and file backend for supported test
case in client-test.
It might be useful running client-test with calendar+todo test cases
using file backend. (It supports evolution backend only without this
patch).
The testInterrupt* tests simulate a hard transport failure. Each time
that happened, SyncContext would retry at least once after the default
retry period (1 minute) before giving up, slowing down the testing
considerably.
With the new SyncOptions "retryInterval" set to zero, retrying is
disabled. This must not be done when testing the resending itself.
Introduced TemplateConfig to abstracting the template configuration structure,
the template metadata used for matching is also parsed here.
The fields introduced in the metadata are:
PeerIsClient: identify whether this is a server side configuration or a client
side configuration.
Fingerprint: the matching string for this template, it is a comma separated string
with each string modeled as: "Manufacture_Model". The first substring is also
used as the name to identify this template so that user can select the template
by this name.
eg:
Nokia 7210c: Nokia_7210c
SyncEvolution server: SyncEvolutionServer, SyncEvolution
ScheduleWorld: ScheduleWorld,default
SyncEvolution client: SyncEvolutionClient, SyncEvolution
Description: this is a just a descriptive string not used for matching.
GetServerTemplates is changed to add another "devices" parameter to identify
it is asking for templates for a list of "devices". Each device is a tuple
<matchstring (devicename), matchMode (server/client/all)>.
TemplateList as the return type, which is a list of class TemplateDescription
so that we can also return enough information for corresponding templates. This
list is sorted by the 3-tuple <finger, rank, name>.
Add MatchServerTemplates method which will iterating all templates inside the
folder and match against the input parameter and finally return a sorted
list of matched templates.
The atcually fuzzy match algorithm is based on a LCS (added in the following
commit).
Cmdline interface is changed accordingly:
--template ? is changed to --template ?[string], so that user use the former
case to match all templates for a tradiontial SyncML client and the latter case
to match templates related to an input string.
SyncConfig API is also renamed (Server -> Peer) because both server/client
configuration/template are handled.
The original configuration template (Funambol and ScheduleWorld) has been moved
to the new template structure (under servers), they also have a .template.ini
file added so that they can be matched and picked up. All templates for
supported servers still have built-in template support in the code as before.
Templates for SyncEvolution based server is also added.
Server side templates are added (Nokia default, Nokia_7210c and SyncEvolutionServer).
Add unit test for the new template match use case.
Enabling only the super datastore to SyncContext.
Avoid changing the databaseid automatically for super source as it has special
meaning.
There are a later commit fixing the LocalTest problem for super datastore.
This is necessary because the two virtual clients need their own
deviceId and evolutionsource properties. Existing old-style
configs are used automatically because the configs are searched
first without the explicit context.
This patch introduces code changes for the new layout without actually
using it yet. Therefore all existing tests for the older layout
still pass. The new meaning of the former "server name" is introduced:
- A plain string now refers to a peer configuration (can be client
or server).
- The @ sign allows selecting a specific context. Different contexts
have independent sets of local sources and peer definitions.
- An empty peer name selects a view on the configuration which contains
no peer-specific properties. Not fully implemented yet.
The FileConfigTree is instantiated with a root which is one level high
up compare to before this patch (for example,
"~/.config/syncevolution" instead of
"./config/syncevolution/scheduleworld") and then config files include
that dropped level in their relative path name
("scheduleworld/config.ini" instead of "config.ini"). This allows
accessing the global properties in
"~/.config/syncevolution/config.ini" and will be used to move peers
further down the hierarchy
("~/.config/syncevolution/peers/scheduleworld/config.ini").
To keep the output of "--print-servers" consistent, the FileConfigTree
gets another parameter which identifies the subset of the larger tree
that is referenced by this FileConfigTree instance.
One side effect of this change is that FileConfigTree instances are no
longer completely separate. Something to keep in mind when
instantiating SyncContext multiple times (MB#8006).
Code may no longer make assumptions in which config node a property is
stored. This is determined by the new getNode() calls based on the
property attributes (hidden, sharing). The new layout is represented as
a set of config nodes. Older layouts use the same set of nodes with
identical instances assigned to them, if they don't really have separate
files for each of them.
SyncSourceNodes no longer grants direct access to the nodes, to catch
code which incorrectly access a specific node directly. For the same
reason the name of the nodes changed.
Code which needs access to all hidden or all visible properties now
does this via a config node returned by getProperties(). Currently
this is identical to the underlying nodes. Once the new layout is
active, this node will act as a multiplexer which gathers properties
from all underlying nodes when reading and picks the right one when
writing.
The "change ID" parameter for sources has been obsolete for a while
and was removed.
Reorganized the property registration so that it is a bit easier
to see which properties are hidden and which use non-default sharing.
The default sharing is "no sharing".
Some other code was improved while touching it:
- removed useless visibility[] array in favor of a i != 0 check in
SyncConfig::copy()
- added default parameters to save/checkPassword() methods
- some constness definition changes
- Property::getProperty() is a virtual call which could only be
overloaded in one case because the constness was wrong; now
getProperty() always returns the string value and getPropertyValue()
some other kind of representation of it, depending on the
class.
- ConstSyncSourceNodes is based on SyncSourceNodes instead of duplicating
it, which simplifies the implementation.
The simplified SyncSourceAdmin API changed slightly: instead of passing
a pointer to the source's SyncSourceNodes, the default nodes are now
found via the SyncSource pointer. For callers this is a bit less
work and it is more general.
The active sources must be set before calling sync(), doing it in
prepare() is too late. The right way to do it with the revised
API is via source filters: disable all sources with a filter that
applies to all sources, then enable the desired one(s) with the
right mode with a more specific filter.
Selecting active sources during a sync or status check was done with a
combination of setting a sync mode via a source config filter and
setting a list of active sources. Now the SyncConfig supports a source
filter which is applied to all sources and source filters for each
source. The latter override the former.
This is powerful enough to start syncs with full control over which
sources are active in which mode, as described in the new D-Bus API.
As part of this patch, the command line semantic is implemented
entirely using a combination of different source filters.
The intention is to get rid of the historic and inconsistent
naming of some classes and their corresponding files:
* EvolutionSyncClient = class derived from Funambol's SyncClient,
* SyncEvolutionConfig = SyncEvolution's config
With the strict 'namespace SyncEvo' and the syncevo/ path prefix for
most header files it is no longer necessary to have "SyncEvolution" or
"Evolution" in the names. This patch thus renames as follows:
EvolutionSyncClient => SyncContext
EvolutionSmartPtr => SmartPtr
SyncEvolutionCmdline => Cmdline
SyncEvolutionConfig => SyncConfig
SyncEvolutionUtil => util
The former EvolutionSyncClient always had a role that went beyond just
running a sync, for example it also provided config access. With the
upcoming server support it also won't be just a client. Thus the new
name "SyncContext".
The 'syncevo/' prefix is used throughout the code now.
removed whenever the prefix made it clear that the file belongs
to SyncEvolution. This helps finding incorrect include paths.
Quotes should be used exclusively for SyncEvolution files which don't
have a specific prefix yet (test.h, config.h) to help identifying
them.
Added syncevo/declarations.h, which has
This is now used for all SyncEvolution source files, except
for the GTK UI, which is written in plain C. In the library
it helps to avoid name clashes.
The reason for using defines instead of spelling out "namespace SyncEvo"
is twofold:
1. if that should ever become necessary, it is easier to
rename the namespace via configure options by changing
the define
2. editors don't indent the whole file content
The main motivation for this change is that it allows the implementor
of a backend to choose the implementations for the different aspects
of a datasource (change tracking, item import/export, logging, ...)
independently of each other. For example, change tracking via revision
strings can now be combined with exchanging data with the Synthesis
engine via a single string (the traditional method in SyncEvolution)
and with direct access to the Synthesis field list (now possible for
the first time).
The new backend API is based on the concept of providing
implementations for certain functionality via function objects instead
of implementing certain virtual methods. The advantage is that
implementors can define their own, custom interfaces and mix and match
implementations of the different groups of functionality.
Logging (see SyncSourceLogging in a later commit) can be done by
wrapping some arbitrary other item import/export function objects
(decorator design pattern).
The class hierarchy is now this:
- SyncSourceBase: interface for common utility code, all other
classes are derived from it and thus can use that code
- SyncSource: base class which implements SyncSourceBase and
hooks a datasource into the SyncEvolution core;
its "struct Operations" holds the function objects which
can be implemented in different ways
- TestingSyncSource: combines some of the following classes
into an interface that is expected by the client-test
program; backends only have to derive from (and implement this)
if they want to use the automated testing
- TrackingSyncSource: provides the same functionality as
before (change tracking via revision strings, item import/export
as string) in a single interface; the description of the pure
virtual methods are duplicated so that developers can go through
this class and find everything they need to know to implement
it
The following classes contain the code that was previously
found in the EvolutionSyncSource base class. Implementors
can derive from them and call the init() methods to inherit
and activate the functionality:
- SyncSourceSession: binds Synthesis session callbacks to
virtual methods beginSync(), endSync()
- SyncSourceChanges: implements Synthesis item tracking callbacks
with set of LUIDs that the user of the class has to fill
- SyncSourceDelete: binds Synthesis delete callback to
virtual method
- SyncSourceRaw: read and write items in the backends format,
used for testing and backup/restore
- SyncSourceSerialize: exchanges items with Synthesis engine
using a string representation of the data; this is how
EvolutionSyncSource has traditionally worked, so much of the
same virtual methods are now in this class
- SyncSourceRevisions: utility class which does change tracking
via some kind of "revision" string which changes each time
an item is modified; this code was previously in the
TrackingSyncSource
The only way that I found not to execute a test was not to register
it in CPPUnit. FilterTest() does this by replacing a valid test
or test group with a dummy one, SkipTest, which just prints the
test name and that it is skipped.
Registering tests has to be intercepted at multiple levels:
- CPPUNIT_TEST in test suites
- ADD_TEST in ClientTest
- addTest in ClientTest
Not currently intercepted are complete test suites (CPPUNIT_TEST_SUITE).
The main purpose of this patch is to avoid running the time consuming
suspend and interrupt tests, but this feature might also be useful for
other tests, which is why it was implemented in a more general way.
Both EvolutionSyncSource::backupData() (implemented for the memo source
in the TrackingSyncSource base class) and Client::Source::text::testImport
must dump items in the native format, the one which is used for restore
and for the test cases. See Bugzilla #3967 and #3929.
This broke during the 0.9 development cycle, but wasn't detected during
automated testing because the necessary tests weren't enabled for
the "text" source until recently.
Now createItem() is passed a hint what the desired format is. "raw"
is used for backup and testing.
Also removed obsolete exportData() call in EvolutionCalendarSource.
This was used before introducing the newer backupData() call.
The two size options are now copied into the Synthesis XML configuration.
Large object support is always on, so the setting was removed.
Note that the Synthesis engine allocates twice the amount specified in
the configuration and then uses that amount if the server allows it.
That explains why outgoing messages where twice as large as specified.
Incoming messages where inside the size limit.
Config::testcases_server had no useful purpose. The extra dump()
method became redundant because the EvolutionSyncSource API *always*
returns "native" items and thus the generic method works fine.
As with the previous change from GPL to LGPL v2.1, this is covered
by copyright ownership and/or the Funambol contributor agreement.
The motivation for adding v3 is greater flexibility regarding
reusing the code in other places and relicensing it.
Also, now the license and disclaimer are mentioned at the start
of the source files. That is not strictly necessary, but more
explicit.
EvolutionSyncClient::getChanges() can be used to query whether
items were changed locally. Added/updated/deleted/total number
of items are reported, plus the time needed to collect that information.
The information is stored in a SyncReport, with ITEM_ANY used
for the total item count.
--status now also does this check and prints the information.
Printing a SyncReport as an ASCII table was considerably
improved for this: columns which are known to be irrelevant
can be suppressed. The layout is a lot more dynamic with
(so I hope) nicer alignment: text related to a source name
is aligned with that name, which itself is flushed to the
right of the name column.
--status was broken by the last commit: XDG_DATA_DIR was not
expanded. Fixed it as part of the improvements for printing
changes in --restore. Also made some other minor changes
to utility classes as part of these improvements.
The new --restore option can restore data from a data dump,
identified by the directory and a before/after flag. The
corresponding API is EvolutionSyncClient::restore().
Using the directory format will make restoring items easier because
there is no need to break one file with unknown content into single
items.
The SyncSourceReport now contains information about the backups
made before and after a sync; the only useful information right
now is the number of items.
Comparison was missing altogether. Items were not imported as text/calendar
because the type wasn't set. Creating new items didn't work because <<REVISION>>
was in the wrong place: it needs to be in the DESCRIPTION, so that SUMMARY and first
line of DESCRIPTION remain identical (expected by some peers and our own sync source).
update-copyright.sh can be used to add copyright remarks for the current
year. It finds the authors who made a change in each file and adds/updates
their copyright remark. Intel employees are grouped under "Intel Corporation".
Removed the "encoding" source option. It was of dubious usefulness
with Funambol and has meaning with Synthesis at all.
Removed dead code in EvolutionSyncConfig.
Added "enableWBXML" sync option. WBXML is used by default, but for
debugging disabling it might be useful: dumping WBXML doesn't seem
to work reliably.
Most Client::Sync tests use the default encoding, usually WBXML unless
changed via CLIENT_TEST_XML=1. Client::Sync::*::testItemsXML always
uses XML and Client::Sync::*::testItems always WBXML.
Use the ClientTest::compare() helper function and change it so that
it invokes "synccompare", as needed when using it in SyncEvolution.
Improved it so that it prints the synccompare command after a failure
and adds a line break before it.
The conversion test checks whether information is lost and/or modified
when converting the test items into the field list defined in the Synthesis
XML config. For that it converts into the field list and back, then compares
against the original test item file with synccompare.
Because a valid Synthesis session is necessary, this test hooks into
running a sync session via a callback that is triggered by the first
Synthesis progress event. It then aborts the session without contacting
the server.
boost::bind() with a reference should have worked (?), but didn't:
the bound function did not update the variable it was bound to.
Using a pointer worked.
The logdir is now overridden so that all files (client.log, message
dumps, Synthesis logs) are written in a subdirectory names after
the current test. A <test>.log file captures all output written
outside of the sync itself.
