maryjane
/
syncevolution
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

- improved logging: log file and backup can be avoided with logdir=none, 

loglevel is configurable
- fixed off-by-one counting of months in backup directory names
- only one example configuration per server, users typically do not
  need the _1/2 suffices and the testing system no longer has to be
  configured manually either
- documented the switch from the original SyncEvolution testing to its
  successor, the generic Funambol C++ client testing framework


git-svn-id: https://zeitsenke.de/svn/SyncEvolution/trunk@362 15ad00c4-1369-45f4-8270-35d70d36bdcd
This commit is contained in:
Patrick Ohly 2007-04-21 12:33:23 +00:00
parent 910a1b593d
commit 614d3b5210
6 changed files with 159 additions and 166 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

45
HACKING
View File

@ -55,38 +55,23 @@ which call the real functions which then may throw exceptions.
The EvolutionSyncSource::handleException() function deals with
logging the exception.
"make check" in the src directory compiles and runs unit
tests. Those tests were written inside the CppUnit
[http://cppunit.sourceforge.net/cppunit-wiki] framework
and so they (and just they) require that "cppunit-config"
is in the shell search path. Before running them, setup
your Evolution and ~/.sync4j configuration as described
in the README section 'Exchanging Data' (create address
books 'SyncEvolution test #1/2', etc).
SyncEvolution uses the Funambol C++ testing framework (which is
based on the previous SyncEvolution testing). Configure with
--enable-integration-tests and (optionally) --enable-unit-tests,
then run "src/client-test" as described in the C++ client libraries
test/README.txt.
Running the test with the environment variable TEST_EVOLUTION_LOG
set to the server's log file ensures that for each
individual sync session both the client and the corresponding
server log are stored in individual files in the src
directory. The filenames are derived from the tests and
the steps executed for each of them.
It understands the same environment variables as the C++ client
libraries "client-test", among them:
- CLIENT_TEST_SERVER = choose config (will be created with _1 and _2
suffix if those do not exist yet; for remote servers the syncURI
and username/password must be added)
- CLIENT_TEST_LOG = name of server log file
The basic configuration is chosen via TEST_EVOLUTION_SERVER;
the default is "localhost".
If the server needs a certain delay between syncs, that
can be set via TEST_EVOLUTION_DELAY=<seconds>. The Sync4j
2.3 server needs a delay of around 20 seconds to work
correctly.
When invoking TestEvolution without parameters all
tests will be run. Otherwise each test listed on the
command line will be run:
./TestEvolution ContactSync::testItems \
ContactSync::testCopy
For a list of available tests see the TestEvolution.cpp
source code.
In addition it supports:
- CLIENT_TEST_EVOLUTION_PREFIX=[name|file://<path>] overrides
the evolutionsource setting, if file:// is used then these
database will be created automatically
"make valgrind" runs the same tests inside valgrind
[http://www.valgrind.org]. A suppression file is

110
README
View File

@ -140,11 +140,6 @@ server compare against the ones it already knows. This is avoided
whenever possible because matching items during a slow synchronization
can lead to duplicate entries.
Due to a limitation in the client library implementation it might
force a slow synchronization of all databases even if only one of them
failed. Therefore it might be less risky to invoke SyncEvolution for
each data source separately.
After a successful synchronization the server's configuration file is
updated so that the next run can be done incrementally. If the
configuration file has to be recreated e.g. because it was lost, the
@ -154,6 +149,25 @@ it already has stored previously which then would lead to duplication
of items.
Getting Started with ScheduleWorld
----------------------------------
- get a ScheduleWorld account
- create a configuration:
mkdir -p ~/.sync4j/evolution
cp -r <path to example configs>/scheduleworld ~/.sync4j/evolution/
- edit ~/.sync4j/evolution/spds/syncml/config.txt,
set "username" and "password" (beware, avoid extra spaces at the line ends)
and choose a unique string for "deviceId";
optional: create a directory where SyncEvolution can store
backup files and a log during each synchronization, see
documentation of "logdir/maxlogdirs" for details
- edit ~/.sync4j/evolution/spds/sources/*/config.txt,
set "evolutionsource" to the name or uri of your Evolution
database ("syncevolution" without parameter prints both for
each available database)
- synchronize with "syncevolution scheduleworld"
Configuration
-------------
@ -165,23 +179,21 @@ The format is a simple list of
pairs with one pair per line. Leading spaces and space around the
equals character are skipped. <value> then runs until the the end of
the line. In other words, it cannot start with spaces nor contain line
breaks. Do not put quotation marks around <value>, they would be
treated as part of the value itself. Lines starting with a hash (#)
after optional leading spaces are treated as comments and skipped.
breaks BUT spaces at the end of the line are part of the value! Be
careful not to add extraneous spaces. Do not put quotation marks around
<value>, they would be treated as part of the value itself. Lines
starting with a hash (#) after optional leading spaces are treated as
comments and skipped.
Each data source is configured in
$HOME/.sync4j/evolution/<server>/spds/sources/<source>/config.txt
See "etc/localhost_1/spds/syncml/config.txt" for options in the server
configuration and "etc/localhost_1/spds/sources/addressbook_1/config.txt"
See "scheduleworld/spds/syncml/config.txt" for options in the server
configuration and "scheduleworld/spds/sources/addressbook/config.txt"
for options in the data source configuration. In packages of SyncEvolution
these files might be installed in [/usr/local]/shared/doc/syncevolution
and use the name of specific SyncML servers instead of the generic
"localhost".
Without changes this example configuration can be used immediately
with a local Funambol installation for testing the operation of
SyncEvolution, see "Exchanging Data" below.
these files might be installed in [/usr/local]/shared/doc/syncevolution.
Another example configuration "funambol" is provided for use with a
local Funambol installation.
Normally at least the following configuration options need to be adapted:
spds/syncml
@ -210,9 +222,11 @@ One can synchronize with multiple server databases in one run, but the
same server database can only be accessed once. To synchronize the
same server database with multiple local Evolution data sources, one
has to setup two independent configurations with different "deviceId"
settings and synchronize them separately. Such a setup is provided
with the "localhost_1/2" configs and using it for testing is explained
in more detail in "Exchanging Data" below.
settings and synchronize them separately. To create such a setup simply
copy the whole configuration tree of the server, e.g.:
cp -r ~/.sync4j/evolution/localhost ~/.sync4j/evolution/localhost_copy
and then edit ~/.sync4j/evolution/localhost_copy/spds/syncml/config.txt
to update the "deviceId".
If the Evolution data source requires authentication, the
"evolutionuser" and "evolutionpassword" are used as credentials.
@ -232,7 +246,7 @@ Automatic Backups and Logging
To support recovery from a synchronization which damaged the
local data or modified it in an unexpected way, SyncEvolution
always creates the following files during a synchronization:
can create the following files during a synchronization:
- a dump of the data in a format which can be imported
back into Evolution, e.g. .vcf for address books
- a full log file with debug information
@ -261,8 +275,8 @@ and is limited to directories starting with the
prefix, so it is safe to put other files or directories
into the configured log directory.
If that option is not set, then the directory will be
created as
If that option is not set (as in the example configurations),
then the directory will be created as
$TMPDIR/SyncEvolution-<username>-<server>
with access allowed for the user only. Files from a
previous synchronization will be overwritten. This is
@ -270,6 +284,12 @@ a lot less useful because the data will usually
be lost during the next reboot and each synchronization run
overwrites the data of the previous one.
To avoid writing any additional log file or database dumps during
a synchronization the "logdir" can be set to "none". To reduce
the verbosity of the log set "logLevel". If not set or 0, then
the verbosity is set to 3 = DEBUG when writing to a log file and
2 = INFO when writing to the console directly.
Configuration with ScheduleWorld
--------------------------------
@ -282,7 +302,7 @@ much Evolution data as possible. The "note" URI and "text/x-journal" type
can be used to synchronize memos.
SyncEvolution is primarily tested against ScheduleWorld. The
"scheduleworld_1" example configuration is ready to be used with these
"scheduleworld" example configuration is ready to be used with these
URIs, one only has to fill in the real username and password.
@ -369,46 +389,22 @@ tasks):
same URI on the SyncML server
4. synchronize again, this time using the other data source
The "etc/localhost_1" directory contains a configuration for a default
Sync4j 2.3 installation on the local host and Evolution address book,
calendar and tasks all called "SyncEvolution test #1". The
"etc/localhost_2" is the counterpart with the configuration of a
second client which synchronizes against the same server, but Evolution
databases called "SyncEvolution test #2". Both configurations can be
copied directly to ".sync4j/evolution":
mkdir -p ~/.sync4j/evolution
cp -a etc/localhost* ~/.sync4j/evolution
The default example configurations already reference Evolution databases
under the fictional names "SyncEvolution test #1". After copying them into
.sync4j/evolution under two different names, changing "#1" into "#2" and
the deviceId in the second copy and creating the "SyncEvolution test #1/2"
databases "syncevolution" can be invoked first for one configuration, then
the other to test how data is copied via the server.
Note that the second client pretends to be a "sc-pim-ppc" client to
avoid the need to reconfigure the default Sync4j installation. This
implies that you cannot use this predefined Evolution config if you
actually synchronize against a PocketPC client.
The same configuration also exists for ScheduleWorld, but beware
that username/password must be adapted in the spds/syncml/config.txt
files.
For them to work, also create the two address books/calendars/tasks
SyncEvolution test #1
SyncEvolution test #2
inside Evolution. SyncEvolution never creates automatically what is
referenced in a source configuration to avoid surprises.
Steps 1 above then becomes an invocation of
syncevolution localhost_1 addressbook_1
and step 4
syncevolution localhost_2 addressbook_2
This copies all contacts into the server and from there into the new
address book. Now one can either compare the address books in
Evolution or do that automatically:
Now one can either compare the address books in Evolution or do that
automatically, described here for contacts:
- save the complete address books: mark all entries, save as vCard
- invoke synccompare with two file names as arguments and it will
normalize and compare them automatically
Normalizing is necessary because the order of cards and their
properties as well as other minor formatting aspects may be
different. The output comes from a "diff --side-by-side", but
different. The output comes from a side-by-side comparison, but
is augmented by the script so that the context of each change
is always the complete item that was modified. Lines or items
following a ">" on the right side were added, those on the

59
etc/Makefile.am
View File

@ -1,20 +1,15 @@
# the original source files suitable for Funambol running on localhost,
# will be manipulated as necessary for other example configs
localhost_source = localhost_1/spds/syncml/config.txt \
localhost_1/spds/sources/addressbook_1/config.txt
localhost_source = source-config.txt \
syncml-config.txt
# copy for Funambol:
# everything but addressbook disabled
funambol_copy = funambol_2/spds/syncml/config.txt \
funambol_2/spds/sources/addressbook_2/config.txt \
funambol_2/spds/sources/calendar_2/config.txt \
funambol_2/spds/sources/todo_2/config.txt \
funambol_2/spds/sources/memo_2/config.txt \
funambol_1/spds/syncml/config.txt \
funambol_1/spds/sources/addressbook_1/config.txt \
funambol_1/spds/sources/calendar_1/config.txt \
funambol_1/spds/sources/memo_1/config.txt \
funambol_1/spds/sources/todo_1/config.txt
funambol_copy = funambol/spds/syncml/config.txt \
funambol/spds/sources/addressbook/config.txt \
funambol/spds/sources/calendar/config.txt \
funambol/spds/sources/memo/config.txt \
funambol/spds/sources/todo/config.txt
# copy for ScheduleWorld:
# everything enabled
@ -29,6 +24,8 @@ EXTRA_DIST = $(localhost_source)
CLEANFILES = $(all_copy)
MAINTAINERCLEANFILES = Makefile.in
all: $(all_copy)
# apparently only very recent versions of automake know about $(docdir):
# this is a custom rule to install the etc example files there
install-data-local: $(all_copy)
@ -48,52 +45,32 @@ uninstall-local:
done
funambol_1/spds/syncml/config.txt : localhost_1/spds/syncml/config.txt
funambol/spds/syncml/config.txt : syncml-config.txt
mkdir -p $(@D)
cp $< $@
funambol_1/spds/sources/addressbook_1/config.txt : localhost_1/spds/sources/addressbook_1/config.txt
funambol/spds/sources/addressbook/config.txt : source-config.txt
mkdir -p $(@D)
cp $< $@
funambol_2/spds/syncml/config.txt : funambol_1/spds/syncml/config.txt
funambol/spds/sources/calendar/config.txt : source-config.txt
mkdir -p $(@D)
sed -e s/sc-api-nat/sc-pim-ppc/ $< >$@
sed -e s/addressbook/calendar/ -e 's:type = text/x-vcard:type = text/calendar:' -e 's/uri = card/uri = cal/' -e 's/sync = two-way/sync = none/' $< >$@
funambol_2/spds/sources/addressbook_2/config.txt : funambol_1/spds/sources/addressbook_1/config.txt
funambol/spds/sources/todo/config.txt : funambol/spds/sources/addressbook/config.txt
mkdir -p $(@D)
sed -e s/addressbook_1/addressbook_2/ -e s/#1/#2/ $< >$@
sed -e s/addressbook/todo/ -e 's:type = text/x-vcard:type = text/x-todo:' -e 's/uri = card/uri = todo/' -e 's/sync = two-way/sync = none/' $< >$@
funambol_2/spds/sources/calendar_2/config.txt : funambol_1/spds/sources/addressbook_1/config.txt
funambol/spds/sources/memo/config.txt : funambol/spds/sources/addressbook/config.txt
mkdir -p $(@D)
sed -e s/addressbook_1/calendar_2/ -e s/#1/#2/ -e 's:type = text/x-vcard:type = text/calendar:' -e 's/uri = card/uri = cal/' $< >$@
funambol_2/spds/sources/todo_2/config.txt : funambol_1/spds/sources/addressbook_1/config.txt
mkdir -p $(@D)
sed -e s/addressbook_1/todo_2/ -e s/#1/#2/ -e 's:type = text/x-vcard:type = text/x-todo:' -e 's/uri = card/uri = todo/' $< >$@
funambol_2/spds/sources/memo_2/config.txt : funambol_1/spds/sources/addressbook_1/config.txt
mkdir -p $(@D)
sed -e s/addressbook_1/memo_2/ -e s/#1/#2/ -e 's:type = text/x-vcard:type = text/plain:' -e 's/uri = card/uri = note/' -e 's/sync = two-way/sync = none/' $< >$@
funambol_1/spds/sources/calendar_1/config.txt : funambol_1/spds/sources/addressbook_1/config.txt
mkdir -p $(@D)
sed -e s/addressbook_1/calendar_1/ -e 's:type = text/x-vcard:type = text/calendar:' -e 's/uri = card/uri = cal/' $< >$@
funambol_1/spds/sources/todo_1/config.txt : funambol_1/spds/sources/addressbook_1/config.txt
mkdir -p $(@D)
sed -e s/addressbook_1/todo_1/ -e 's:type = text/x-vcard:type = text/x-todo:' -e 's/uri = card/uri = todo/' $< >$@
funambol_1/spds/sources/memo_1/config.txt : funambol_1/spds/sources/addressbook_1/config.txt
mkdir -p $(@D)
sed -e s/addressbook_1/memo_1/ -e 's:type = text/x-vcard:type = text/plain:' -e 's/uri = card/uri = note/' -e 's/sync = two-way/sync = none/' $< >$@
sed -e s/addressbook/memo/ -e 's:type = text/x-vcard:type = text/plain:' -e 's/uri = card/uri = note/' -e 's/sync = two-way/sync = none/' $< >$@
# www.scheduleworld.com configuration:
# - different URL
# - RFC compliant types
# - standard sync sources, not the custom "todo" one
$(scheduleworld_copy) : scheduleworld_% : funambol_%
$(scheduleworld_copy) : scheduleworld% : funambol%
mkdir -p $(@D)
sed -e 's;^syncURL = http://localhost;#syncURL = http://localhost;' \
-e 's;^#syncURL = http://sync.scheduleworld;syncURL = http://sync.scheduleworld;' \

4
etc/localhost_1/spds/sources/addressbook_1/config.txt → etc/source-config.txt
View File

@ -9,11 +9,13 @@ name = addressbook_1
# the items on the client
# refresh-from-server = discard all local items and replace with
# the items on the server
# one-way-from-client = transmit changes from client
# one-way-from-server = transmit changes from server
# none = synchronization disabled
sync = two-way
# overrides the supported synchronization modes
syncModes = slow,two-way,refresh-from-client,refresh-from-server
syncModes = slow,two-way,refresh-from-client,refresh-from-server,one-way-from-client,one-way-from-server
# specifies the format of the data
#

15
etc/localhost_1/spds/syncml/config.txt → etc/syncml-config.txt
View File

@ -26,9 +26,22 @@ userAgent = SyncEvolution
# full path to directory where automatic backups and logs
# are stored for all synchronizations; if empty, the temporary
# directory "$TMPDIR/SyncEvolution-<username>-<server>" will
# be used to keep the data of just the latest synchronization run
# be used to keep the data of just the latest synchronization run;
# if "none", then no backups of the databases are made and any
# output is printed directly instead of writing it into a log
#
# When writing into a log nothing will be shown during the
# synchronization. Instead the important messages are extracted
# automatically from the log at the end.
logdir =
# level of detail for log messages:
# - 0 (or unset) = INFO messages without log file, DEBUG with log file
# - 1 = only ERROR messages
# - 2 = also INFO messages
# - 3 = also DEBUG messages
loglevel = 0
# Unless this option is set, SyncEvolution will never delete
# anything in the "logdir". If set, the oldest directories and
# all their content will be removed after a successful sync

92
src/EvolutionSyncClient.cpp
View File

@ -94,7 +94,7 @@ class LogDir {
int m_maxlogdirs; /**< number of backup dirs to preserve, 0 if unlimited */
string m_prefix; /**< common prefix of backup dirs */
string m_path; /**< path to current logging and backup dir */
string m_logfile; /**< path to log file there */
string m_logfile; /**< path to log file there, empty if not writing one */
const string &m_server; /**< name of the server for this synchronization */
LogLevel m_oldLogLevel; /**< logging level to restore */
bool m_restoreLog; /**< false if nothing needs to be restored because setLogdir() was never called */
@ -105,11 +105,14 @@ public:
{}
// setup log directory and redirect logging into it
// @param path path to configured backup directy, NULL if defaulting to /tmp
// @param path path to configured backup directy, NULL if defaulting to /tmp, "none" if not creating log file
// @param maxlogdirs number of backup dirs to preserve in path, 0 if unlimited
void setLogdir(const char *path, int maxlogdirs) {
// @param logLevel 0 = default, 1 = ERROR, 2 = INFO, 3 = DEBUG
void setLogdir(const char *path, int maxlogdirs, int logLevel = 0) {
m_maxlogdirs = maxlogdirs;
if (path && path[0]) {
if (path && !strcasecmp(path, "none")) {
m_logfile = "";
} else if (path && path[0]) {
m_logdir = path;
// create unique directory name in the given directory
@ -124,7 +127,7 @@ public:
<< "-"
<< setfill('0')
<< setw(4) << tm->tm_year + 1900 << "-"
<< setw(2) << tm->tm_mon << "-"
<< setw(2) << tm->tm_mon + 1 << "-"
<< setw(2) << tm->tm_mday << "-"
<< setw(2) << tm->tm_hour << "-"
<< setw(2) << tm->tm_min;
@ -144,6 +147,7 @@ public:
}
seq++;
}
m_logfile = m_path + "/client.log";
} else {
// create temporary directory: $TMPDIR/SyncEvolution-<username>
stringstream path;
@ -168,18 +172,22 @@ public:
throw runtime_error(m_path + ": " + strerror(errno));
}
}
m_logfile = m_path + "/client.log";
}
// redirect logging into that directory, including stderr,
// after truncating it
m_logfile = m_path + "/client.log";
ofstream out;
out.exceptions(ios_base::badbit|ios_base::failbit|ios_base::eofbit);
out.open(m_logfile.c_str());
out.close();
setLogFile(m_logfile.c_str(), true);
if (m_logfile.size()) {
// redirect logging into that directory, including stderr,
// after truncating it
ofstream out;
out.exceptions(ios_base::badbit|ios_base::failbit|ios_base::eofbit);
out.open(m_logfile.c_str());
out.close();
setLogFile(m_logfile.c_str(), true);
}
m_oldLogLevel = LOG.getLevel();
LOG.setLevel(LOG_LEVEL_DEBUG);
LOG.setLevel(logLevel > 0 ? (LogLevel)(logLevel - 1) /* fixed level */ :
m_logfile.size() ? LOG_LEVEL_DEBUG /* default for log file */ :
LOG_LEVEL_INFO /* default for console output */ );
m_restoreLog = true;
}
@ -231,10 +239,14 @@ public:
}
if (all) {
setLogFile("-", false);
if (m_logfile.size()) {
setLogFile("-", false);
}
LOG.setLevel(m_oldLogLevel);
} else {
setLogFile(m_logfile.c_str(), false);
if (m_logfile.size()) {
setLogFile(m_logfile.c_str(), false);
}
}
}
@ -248,7 +260,7 @@ public:
// as the final report (
class SourceList : public list<EvolutionSyncSource *> {
LogDir m_logdir; /**< our logging directory */
bool m_prepared; /**< remember whether syncPrepare() completed successfully */
bool m_prepared; /**< remember whether syncPrepare() dumped databases successfully */
bool m_doLogging; /**< true iff additional files are to be written during sync */
bool m_reportTodo; /**< true if syncDone() shall print a final report */
arrayptr<SyncSource *> m_sourceArray; /** owns the array that is expected by SyncClient::sync() */
@ -293,9 +305,9 @@ public:
}
// call as soon as logdir settings are known
void setLogdir(const char *logDirPath, int maxlogdirs) {
void setLogdir(const char *logDirPath, int maxlogdirs, int logLevel) {
if (m_doLogging) {
m_logdir.setLogdir(logDirPath, maxlogdirs);
m_logdir.setLogdir(logDirPath, maxlogdirs, logLevel);
} else {
// at least increase log level
LOG.setLevel(LOG_LEVEL_DEBUG);
@ -305,11 +317,12 @@ public:
// call when all sync sources are ready to dump
// pre-sync databases
void syncPrepare() {
if (m_doLogging) {
if (m_logdir.getLogfile().size() &&
m_doLogging) {
// dump initial databases
dumpDatabases("before", "after");
m_prepared = true;
}
m_prepared = true;
}
// call at the end of a sync with success == true
@ -330,28 +343,34 @@ public:
}
// scan for error messages
ifstream in;
in.open(m_logdir.getLogfile().c_str());
while (in.good()) {
string line;
getline(in, line);
if (line.find("[ERROR]") != line.npos) {
success = false;
cout << line << "\n";
} else if (line.find("[INFO]") != line.npos) {
cout << line << "\n";
string logfile = m_logdir.getLogfile();
if (logfile.size()) {
ifstream in;
in.open(m_logdir.getLogfile().c_str());
while (in.good()) {
string line;
getline(in, line);
if (line.find("[ERROR]") != line.npos) {
success = false;
cout << line << "\n";
} else if (line.find("[INFO]") != line.npos) {
cout << line << "\n";
}
}
in.close();
}
in.close();
cout << flush;
cout << flush;
cerr << flush;
cout << "\n";
if (success) {
cout << "Synchronization successful.\n";
} else {
} else if (logfile.size()) {
cout << "Synchronization failed, see "
<< m_logdir.getLogdir()
<< logfile
<< " for details.\n";
} else {
cout << "Synchronization failed.\n";
}
// compare databases?
@ -433,7 +452,8 @@ int EvolutionSyncClient::sync()
try {
arrayptr<char> logdir(config.getSyncMLNode()->readPropertyValue("logdir"));
arrayptr<char> maxlogdirs(config.getSyncMLNode()->readPropertyValue("maxlogdirs"));
sourceList.setLogdir(logdir, atoi(maxlogdirs));
arrayptr<char> loglevel(config.getSyncMLNode()->readPropertyValue("logLevel"));
sourceList.setLogdir(logdir, atoi(maxlogdirs), atoi(loglevel));
SyncSourceConfig *sourceconfigs = config.getSyncSourceConfigs();
for (int index = 0; index < config.getNumSources(); index++) {