maryjane
/
syncevolution
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

command line: specify properties per source and config 

The new format of the property name in --sync-property is:
  <name>[@<context>|@<peer>@<context>]

--source-property also allows a source name:
  [<source>/]<name>[@<context>|@<peer>@<context>]

This allows to set source properties differently for different
sources in the same command line invocation. The @<context> or
@<peer>@<context> will be used to set properties differently for
main and target context in a local sync (not used yet).

The advantage of this grammar is that a string can be split purely based
on the syntax in PropertySpecifier::StringToPropSpec().

The patch itself is based on the idea of first collecting all of these
config property filters in a new case-insensitive hash structure,
FullProps in ConfigFilter.cpp/h, as part of parsing command line
parameters.

Then once specific filters for sync or sources are needed, they are
generated from FullProps by collecting all that apply, starting with
the ones with lowest priority and overwriting them with more important
(= more specific) ones. This also covers additional filters, like the
shared properties of the target context when printing a template.

Currently FullProps may contain arbitrary source and config
names. Typos are not detected, which is both hard to implement (which
names and configs are valid in the current invocation?) and also
forces users to be very specific (can't apply one set of filters to
different configs) - this is the same conflict of interest as in
"configure", which allows unknown --enable/disable parameters because
they might be relevant in a sub-configure script.

SyncConfig itself still only stores the filters which apply to it, not
the full set of overrides that the Cmdline has in its m_props. The
advantage is that the API remains the same (no change needed or done
in the syncevo-dbus-server). The disadvantage is that in a local
sync, no information is available about the properties applying to the
target context - probably needs to change.
This commit is contained in:
Patrick Ohly 2011-01-25 11:11:53 +01:00
parent 8fe00eed75
commit 16bf21f53e
10 changed files with 759 additions and 197 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

50
README.rst
View File

@ -385,6 +385,26 @@ a list of valid values.
to update the configuration. Can be used multiple times. Specifying
an unused property will trigger an error message.
The <property> has the following format: ``<name>[@<context>|@<peer>@<context>]``
The optional <context> or <peer>@<context> suffix limits the scope
of the value to that particular configuration. This is currently
only useful for a local sync, which involves a source and a target
configuration.
A string without a second @ sign inside is always interpreted as a
context name, so in contrast to the <server> string, "foo" cannot be
used to reference the "foo@default" configuration. Use the full name
including the context for that.
When no config or context is specified explicitly, a value is
changed in all active configs, typically the one given with
``<server>``. The priority of multiple values for the same config
is `more specific definition wins`, so ``<peer>@<context>``
overrides ``@<context>``, which overrides `no suffix given`.
Specifying some suffix which does not apply to the current operation
does not trigger an error, so beware of typos.
When using the configuration layout introduced with 1.0, some of the
sync properties are shared between peers, for example the directory
where sessions are logged. Permanently changing such a shared
@ -396,15 +416,39 @@ a list of valid values.
--source-property|-z <property>=<value>|<property>=?|?
Same as --sync-property, but applies to the configuration of all active
sources. `--sync <mode>` is a shortcut for `--source-property sync=<mode>`.
The <property> has the following format: ``[<source>/]<name>[@<context>|@<peer>@<context>]``
In it's simplest form without <source>, <context> or <config>,
the name specifies one of the know properties.
When combined with `--configure`, the configuration of all sources
is modified. The value is applied to all sources unless sources are
listed explicitly on the command line. So if you want to change a
source property of just one specific sync source, then use
`--configure --source-property ... <server> <source>`.
Adding the <source>/ prefix makes it possible to set the same
property differently for different sources in one command::
--configure --source-property addressbook/sync=two-way \
--source-property calendar/sync=one-way-from-server \
<server>
If the same property is set both with and without a <source>/ prefix,
then the more specific value with that prefix is used for that source,
regardless of the order on the command line. The following command
disables all sources except for the addressbook::
--configure --source-property addressbook/sync=none \
--source-property sync=two-way \
<server>
As with sync properties, some properties are shared between peers,
in particular the selection of which local data to synchronize.
in particular the selection of which local data to synchronize. The
optional configuration suffix in ``<property>`` also has the same
meaning as for sync properties. That suffix is checked first, so
"sync@foo@default" overrides "addressbook/sync", even though
"addressbook/sync" normally overrides "sync".
--template|-l <peer name>|default|?<device>
Can be used to select from one of the built-in default configurations

328
src/syncevo/Cmdline.cpp
View File

@ -130,7 +130,7 @@ bool Cmdline::parse(vector<string> &parsed)
opt++;
string param;
string cmdopt(m_argv[opt - 1]);
if (!parseProp(m_validSourceProps, m_sourceProps,
if (!parseProp(SOURCE_PROPERTY_TYPE,
m_argv[opt - 1], opt == m_argc ? NULL : m_argv[opt],
"sync")) {
return false;
@ -143,7 +143,7 @@ bool Cmdline::parse(vector<string> &parsed)
} else if(boost::iequals(m_argv[opt], "--sync-property") ||
boost::iequals(m_argv[opt], "-y")) {
opt++;
if (!parseProp(m_validSyncProps, m_syncProps,
if (!parseProp(SYNC_PROPERTY_TYPE,
m_argv[opt - 1], opt == m_argc ? NULL : m_argv[opt])) {
return false;
}
@ -151,7 +151,7 @@ bool Cmdline::parse(vector<string> &parsed)
} else if(boost::iequals(m_argv[opt], "--source-property") ||
boost::iequals(m_argv[opt], "-z")) {
opt++;
if (!parseProp(m_validSourceProps, m_sourceProps,
if (!parseProp(SOURCE_PROPERTY_TYPE,
m_argv[opt - 1], opt == m_argc ? NULL : m_argv[opt])) {
return false;
}
@ -382,7 +382,7 @@ bool Cmdline::isSync()
!m_restore.empty() ||
m_accessItems ||
m_dryrun ||
(!m_run && (m_syncProps.size() || m_sourceProps.size()))) {
(!m_run && m_props.hasProperties())) {
return false;
} else {
return true;
@ -443,6 +443,9 @@ bool Cmdline::run() {
// potentially harmful operations, otherwise users might
// expect it to have an effect when it doesn't.
// TODO: check filter properties for invalid config and source
// names
if (m_usage) {
usage(true);
} else if (m_version) {
@ -495,7 +498,7 @@ bool Cmdline::run() {
} else if (m_printConfig) {
boost::shared_ptr<SyncConfig> config;
ConfigProps syncFilter;
SourceFilters_t sourceFilters;
SourceProps sourceFilters;
if (m_template.empty()) {
if (m_server.empty()) {
@ -508,8 +511,10 @@ bool Cmdline::run() {
return false;
}
syncFilter = m_syncProps;
sourceFilters[""] = m_sourceProps;
// No need to include a context or additional sources,
// because reading the m_server config already includes
// the right information.
m_props.createFilters("", m_server, NULL, syncFilter, sourceFilters);
} else {
string peer, context;
SyncConfig::splitConfigString(SyncConfig::normalizeConfigString(m_template), peer, context);
@ -520,7 +525,15 @@ bool Cmdline::run() {
return false;
}
getFilters(context, syncFilter, sourceFilters);
// When instantiating a template, include the properties
// of the target context as filter to preserve shared
// properties, the final name inside that context as
// peer config name, and the sources defined in the template.
list<string> sourcelist = config->getSyncSources();
set<string> sourceset(sourcelist.begin(), sourcelist.end());
m_props.createFilters(std::string("@") + context, "",
&sourceset,
syncFilter, sourceFilters);
}
// determine whether we dump a peer or a context
@ -547,12 +560,7 @@ bool Cmdline::run() {
m_out << endl << "[" << name << "]" << endl;
SyncSourceNodes nodes = config->getSyncSourceNodes(name);
boost::shared_ptr<FilterConfigNode> sourceProps = nodes.getProperties();
SourceFilters_t::const_iterator it = sourceFilters.find(name);
if (it != sourceFilters.end()) {
sourceProps->setFilter(it->second);
} else {
sourceProps->setFilter(sourceFilters[""]);
}
sourceProps->setFilter(sourceFilters.createSourceFilter(name));
dumpProperties(*sourceProps, SyncSourceConfig::getRegistry(),
flags | ((name != *(--sources.end())) ? HIDE_LEGEND : DUMP_PROPS_NORMAL));
}
@ -708,22 +716,6 @@ bool Cmdline::run() {
}
}
// Apply config changes on-the-fly. Regardless what we do
// (changing an existing config, migrating, creating from
// a template), existing shared properties in the desired
// context must be preserved unless explicitly overwritten.
// Therefore read those, update with command line properties,
// then set as filter.
ConfigProps syncFilter;
SourceFilters_t sourceFilters;
getFilters(context, syncFilter, sourceFilters);
from->setConfigFilter(true, "", syncFilter);
BOOST_FOREACH(const SourceFilters_t::value_type &entry, sourceFilters) {
from->setConfigFilter(false, entry.first, entry.second);
}
// Write into the requested configuration, creating it if necessary.
//
// Which sources are configured is determined as follows:
// - all sources in the template by default, except when
// - sources are listed explicitly, and either
@ -740,6 +732,22 @@ bool Cmdline::run() {
(!fromScratch || configureContext)) {
sources = &m_sources;
}
// Apply config changes on-the-fly. Regardless what we do
// (changing an existing config, migrating, creating from
// a template), existing shared properties in the desired
// context must be preserved unless explicitly overwritten.
// Therefore read those, update with command line properties,
// then set as filter.
ConfigProps syncFilter;
SourceProps sourceFilters;
m_props.createFilters(string("@") + context, m_server, sources, syncFilter, sourceFilters);
from->setConfigFilter(true, "", syncFilter);
BOOST_FOREACH(const SourceProps::value_type &entry, sourceFilters) {
from->setConfigFilter(false, entry.first, entry.second);
}
// Write into the requested configuration, creating it if necessary.
boost::shared_ptr<SyncContext> to(createSyncClient());
to->prepareConfigForWrite();
to->copy(*from, sources);
@ -798,9 +806,9 @@ bool Cmdline::run() {
syncMode = "disabled";
} else if (selected) {
// user absolutely wants it: enable even if off by default
FilterConfigNode::ConfigFilter::const_iterator sync =
m_sourceProps.find("sync");
syncMode = sync == m_sourceProps.end() ? "two-way" : sync->second;
ConfigProps filter = m_props.createSourceFilter(m_server, source);
ConfigProps::const_iterator sync = filter.find("sync");
syncMode = sync == filter.end() ? "two-way" : sync->second;
}
if (!syncMode.empty() &&
!configureContext) {
@ -861,8 +869,7 @@ bool Cmdline::run() {
// extra sanity check
if (!m_sources.empty() ||
!m_syncProps.empty() ||
!m_sourceProps.empty()) {
!m_props.hasProperties()) {
usage(true, "too many parameters for --remove");
return false;
} else {
@ -881,11 +888,13 @@ bool Cmdline::run() {
context.reset(createSyncClient());
context->setOutput(&m_out);
// apply filters
context->setConfigFilter(true, "", m_syncProps);
context->setConfigFilter(false, "", m_sourceProps);
// operating on exactly one source
string sourceName = *m_sources.begin();
// apply filters
context->setConfigFilter(true, "", m_props.createSyncFilter(m_server));
context->setConfigFilter(false, "", m_props.createSourceFilter(m_server, sourceName));
SyncSourceNodes sourceNodes = context->getSyncSourceNodesNoTracking(sourceName);
SyncSourceParams params(sourceName, sourceNodes, context);
cxxptr<SyncSource> source(SyncSource::createSource(params, true));
@ -1114,26 +1123,21 @@ bool Cmdline::run() {
context.reset(createSyncClient());
context->setQuiet(m_quiet);
context->setDryRun(m_dryrun);
context->setConfigFilter(true, "", m_syncProps);
context->setConfigFilter(true, "", m_props.createSyncFilter(m_server));
context->setOutput(&m_out);
if (m_sources.empty()) {
if (m_sourceProps.empty()) {
// empty source list, empty source filter => run with
// existing configuration without filtering it
} else {
// Special semantic of 'no source selected': apply
// filter only to sources which are
// *active*. Configuration of inactive sources is left
// unchanged. This way we don't activate sync sources
// accidentally when the sync mode is modified
// temporarily.
BOOST_FOREACH(const std::string &source,
context->getSyncSources()) {
boost::shared_ptr<PersistentSyncSourceConfig> source_config =
context->getSyncSourceConfig(source);
if (source_config->getSync() == "disabled") {
context->setConfigFilter(false, source, m_sourceProps);
}
// Special semantic of 'no source selected': apply
// filter (if any exists) only to sources which are
// *active*. Configuration of inactive sources is left
// unchanged. This way we don't activate sync sources
// accidentally when the sync mode is modified
// temporarily.
BOOST_FOREACH(const std::string &source,
context->getSyncSources()) {
boost::shared_ptr<PersistentSyncSourceConfig> source_config =
context->getSyncSourceConfig(source);
if (source_config->getSync() == "disabled") {
context->setConfigFilter(false, source, m_props.createSourceFilter(m_server, source));
}
}
} else {
@ -1142,19 +1146,18 @@ bool Cmdline::run() {
m_sources) {
boost::shared_ptr<PersistentSyncSourceConfig> source_config =
context->getSyncSourceConfig(source);
ConfigProps filter = m_props.createSourceFilter(m_server, source);
if (!source_config || !source_config->exists()) {
// invalid source name in m_sources, remember and
// report this below
unmatchedSources.insert(source);
} else if (m_sourceProps.find("sync") ==
m_sourceProps.end()) {
} else if (filter.find("sync") == filter.end()) {
// Sync mode is not set, must override the
// "sync=disabled" set below with the original
// sync mode for the source or (if that is also
// "disabled") with "two-way". The latter is part
// of the command line semantic that listing a
// source activates it.
FilterConfigNode::ConfigFilter filter = m_sourceProps;
string sync = source_config->getSync();
filter["sync"] =
sync == "disabled" ? "two-way" : sync;
@ -1162,7 +1165,7 @@ bool Cmdline::run() {
} else {
// sync mode is set, can use m_sourceProps
// directly to apply it
context->setConfigFilter(false, source, m_sourceProps);
context->setConfigFilter(false, source, filter);
}
}
@ -1219,7 +1222,7 @@ bool Cmdline::run() {
// safety catch: if props are given, then --run
// is required
if (!m_run &&
(m_syncProps.size() || m_sourceProps.size())) {
(m_props.hasProperties())) {
usage(false, "Properties specified, but neither '--configure' nor '--run' - what did you want?");
return false;
}
@ -1265,56 +1268,104 @@ string Cmdline::cmdOpt(const char *opt, const char *param)
return res;
}
bool Cmdline::parseProp(const ConfigPropertyRegistry &validProps,
FilterConfigNode::ConfigFilter &props,
const char *opt,
const char *param,
const char *propname)
bool Cmdline::parseProp(PropertyType propertyType,
const char *opt,
const char *param,
const char *propname)
{
std::string args = cmdOpt(opt, param);
if (!param) {
usage(true, string("missing parameter for ") + cmdOpt(opt, param));
usage(true, string("missing parameter for ") + args);
return false;
}
// determine property name and parameter for it
string propstr;
string paramstr;
if (propname) {
propstr = propname;
paramstr = param;
} else if (boost::trim_copy(string(param)) == "?") {
m_dontrun = true;
if (propname) {
return listPropValues(validProps, propname, opt);
} else {
return listProperties(validProps, opt);
}
paramstr = param;
} else {
string propstr;
string paramstr;
if (propname) {
propstr = propname;
paramstr = param;
} else {
const char *equal = strchr(param, '=');
if (!equal) {
usage(true, string("the '=<value>' part is missing in: ") + cmdOpt(opt, param));
const char *equal = strchr(param, '=');
if (!equal) {
usage(true, string("the '=<value>' part is missing in: ") + args);
return false;
}
propstr.assign(param, equal - param);
paramstr.assign(equal + 1);
}
boost::trim(propstr);
boost::trim_left(paramstr);
// parse full property string
PropertySpecifier spec = PropertySpecifier::StringToPropSpec(propstr);
// determine property type and registry
const ConfigPropertyRegistry *validProps = NULL;
switch (propertyType) {
case SYNC_PROPERTY_TYPE:
validProps = &m_validSyncProps;
break;
case SOURCE_PROPERTY_TYPE:
validProps = &m_validSourceProps;
break;
case UNKNOWN_PROPERTY_TYPE:
// must guess based on both registries
if (!propstr.empty()) {
bool isSyncProp = m_validSyncProps.find(spec.m_property) != NULL;
bool isSourceProp = m_validSourceProps.find(spec.m_property) != NULL;
if (isSyncProp) {
if (isSourceProp) {
usage(true, StringPrintf("property '%s' in %s could be both a sync and a source property, use --sync-property or --source-property to disambiguate it", propname, args.c_str()));
return false;
} else {
validProps = &m_validSyncProps;
}
} else if (isSourceProp) {
validProps = &m_validSourceProps;
} else {
usage(true, StringPrintf("unrecognized property '%s' in %s", propname, args.c_str()));
return false;
}
propstr.assign(param, equal - param);
paramstr.assign(equal + 1);
} else {
usage(true, StringPrintf("a property name must be given in '%s'", args.c_str()));
}
}
boost::trim(propstr);
boost::trim_left(paramstr);
// TODO: complain if sync property includes source prefix
if (boost::trim_copy(string(param)) == "?") {
m_dontrun = true;
if (propname) {
return listPropValues(*validProps, spec.m_property, opt);
} else {
return listProperties(*validProps, opt);
}
} else {
if (boost::trim_copy(paramstr) == "?") {
m_dontrun = true;
return listPropValues(validProps, propstr, cmdOpt(opt, param));
return listPropValues(*validProps, spec.m_property, cmdOpt(opt, param));
} else {
const ConfigProperty *prop = validProps.find(propstr);
const ConfigProperty *prop = validProps->find(spec.m_property);
if (!prop) {
m_err << "ERROR: " << cmdOpt(opt, param) << ": no such property" << endl;
m_err << "ERROR: " << args << ": no such property" << endl;
return false;
} else {
string error;
if (!prop->checkValue(paramstr, error)) {
m_err << "ERROR: " << cmdOpt(opt, param) << ": " << error << endl;
m_err << "ERROR: " << args << ": " << error << endl;
return false;
} else {
props[propstr] = paramstr;
ContextProps &props = m_props[spec.m_config];
if (validProps == &m_validSyncProps) {
props.m_syncProps[spec.m_property] = paramstr;
} else {
props.m_sourceProps[spec.m_source][spec.m_property] = paramstr;
}
return true;
}
}
@ -1372,57 +1423,32 @@ bool Cmdline::listProperties(const ConfigPropertyRegistry &validProps,
return true;
}
void Cmdline::getFilters(const string &context,
ConfigProps &syncFilter,
map<string, ConfigProps> &sourceFilters)
{
// Read from context. If it does not exist, we simply set no properties
// as filter. Previously there was a check for existance, but that was
// flawed because it ignored the global property "defaultPeer".
boost::shared_ptr<SyncConfig> shared(new SyncConfig(string("@") + context));
shared->getProperties()->readProperties(syncFilter);
BOOST_FOREACH(StringPair entry, m_syncProps) {
syncFilter[entry.first] = entry.second;
}
BOOST_FOREACH(std::string source, shared->getSyncSources()) {
SyncSourceNodes nodes = shared->getSyncSourceNodes(source, "");
ConfigProps &props = sourceFilters[source];
nodes.getProperties()->readProperties(props);
// Special case "type" property: the value in the context
// is not preserved. Every new peer must ensure that
// its own value is compatible (= same backend) with
// the other peers.
props.erase("type");
BOOST_FOREACH(StringPair entry, m_sourceProps) {
props[entry.first] = entry.second;
}
}
sourceFilters[""] = m_sourceProps;
}
static void findPeerProps(FilterConfigNode::ConfigFilter &filter,
ConfigPropertyRegistry &registry,
list<string> &peerProps)
set<string> &peerProps)
{
BOOST_FOREACH(StringPair entry, filter) {
const ConfigProperty *prop = registry.find(entry.first);
if (prop &&
prop->getSharing() == ConfigProperty::NO_SHARING &&
!(prop->getFlags() & ConfigProperty::SHARED_AND_UNSHARED)) {
peerProps.push_back(entry.first);
peerProps.insert(entry.first);
}
}
}
void Cmdline::checkForPeerProps()
{
list<string> peerProps;
set<string> peerProps;
findPeerProps(m_syncProps, SyncConfig::getRegistry(), peerProps);
findPeerProps(m_sourceProps, SyncSourceConfig::getRegistry(), peerProps);
BOOST_FOREACH(FullProps::value_type &entry, m_props) {
ContextProps &props = entry.second;
findPeerProps(props.m_syncProps, SyncConfig::getRegistry(), peerProps);
BOOST_FOREACH(SourceProps::value_type &entry, props.m_sourceProps) {
findPeerProps(entry.second, SyncSourceConfig::getRegistry(), peerProps);
}
}
if (!peerProps.empty()) {
SyncContext::throwError(string("per-peer (unshared) properties not allowed: ") +
boost::join(peerProps, ", "));
@ -2684,18 +2710,17 @@ protected:
CPPUNIT_ASSERT(!filter.m_cmdline->run());
CPPUNIT_ASSERT_EQUAL_DIFF("", filter.m_out.str());
CPPUNIT_ASSERT_EQUAL_DIFF("sync = refresh-from-server",
string(filter.m_cmdline->m_sourceProps));
CPPUNIT_ASSERT_EQUAL_DIFF("",
string(filter.m_cmdline->m_syncProps));
string(filter.m_cmdline->m_props[""].m_sourceProps[""]));
CPPUNIT_ASSERT_EQUAL_DIFF("", string(filter.m_cmdline->m_props[""].m_syncProps));
TestCmdline filter2("--source-property", "sync=refresh", NULL);
CPPUNIT_ASSERT(filter2.m_cmdline->parse());
CPPUNIT_ASSERT(!filter2.m_cmdline->run());
CPPUNIT_ASSERT_EQUAL_DIFF("", filter2.m_out.str());
CPPUNIT_ASSERT_EQUAL_DIFF("sync = refresh",
string(filter2.m_cmdline->m_sourceProps));
string(filter2.m_cmdline->m_props[""].m_sourceProps[""]));
CPPUNIT_ASSERT_EQUAL_DIFF("",
string(filter2.m_cmdline->m_syncProps));
string(filter2.m_cmdline->m_props[""].m_syncProps));
}
void testConfigure() {
@ -2710,8 +2735,8 @@ protected:
{
// updating type for peer must also update type for context
TestCmdline cmdline("--configure",
"--source-property", "type=file:text/vcard:3.0",
"scheduleworld", "addressbook",
"--source-property", "addressbook/type=file:text/vcard:3.0",
"scheduleworld",
NULL);
cmdline.doit();
CPPUNIT_ASSERT_EQUAL_DIFF("", cmdline.m_err.str());
@ -2858,6 +2883,15 @@ protected:
CPPUNIT_ASSERT_EQUAL_DIFF(syncProperties + sourceProperties,
filterIndented(cmdline.m_out.str()));
}
{
TestCmdline cmdline("--source-property", "sync=?",
NULL);
cmdline.doit();
CPPUNIT_ASSERT_EQUAL_DIFF("", cmdline.m_err.str());
CPPUNIT_ASSERT_EQUAL_DIFF("'--source-property sync=?'\n",
filterIndented(cmdline.m_out.str()));
}
}
void testConfigureSources() {
@ -2898,7 +2932,7 @@ protected:
// add calendar
{
TestCmdline cmdline("--configure",
"--source-property", "database = file://tmp/test2",
"--source-property", "database@foobar = file://tmp/test2",
"--source-property", "type = calendar",
"@foobar",
"calendar",
@ -2943,6 +2977,21 @@ protected:
"peers/scheduleworld/sources/addressbook/config.ini:type = file:text/vcard:3.0",
"peers/scheduleworld/sources/addressbook/config.ini:type = addressbook:text/vcard");
CPPUNIT_ASSERT_EQUAL_DIFF(expected, res);
// disable all sources except for addressbook
{
TestCmdline cmdline("--configure",
"--source-property", "addressbook/sync=two-way",
"--source-property", "sync=none",
"scheduleworld@foobar",
NULL);
cmdline.doit();
}
res = scanFiles(root);
removeRandomUUID(res);
boost::replace_all(expected, "sync = two-way", "sync = disabled");
boost::replace_first(expected, "sync = disabled", "sync = two-way");
CPPUNIT_ASSERT_EQUAL_DIFF(expected, res);
}
void testOldConfigure() {
@ -3039,8 +3088,11 @@ protected:
TestCmdline cmdline("--configure",
"--sync", "two-way",
"-z", "database=source",
"--sync-property", "maxlogdirs=20",
"-y", "LOGDIR=logdir",
// note priority of suffix: most specific wins
"--sync-property", "maxlogdirs@scheduleworld@default=20",
"--sync-property", "maxlogdirs@default=10",
"--sync-property", "maxlogdirs=5",
"-y", "LOGDIR@default=logdir",
"scheduleworld",
NULL);
cmdline.doit();

25
src/syncevo/Cmdline.h
View File

@ -167,7 +167,7 @@ protected:
Bool m_keyring;
Bool m_monitor;
Bool m_useDaemon;
FilterConfigNode::ConfigFilter m_syncProps, m_sourceProps;
FullProps m_props;
const ConfigPropertyRegistry &m_validSyncProps;
const ConfigPropertyRegistry &m_validSourceProps;
@ -193,14 +193,13 @@ protected:
/**
* parse sync or source property
*
* @param validProps list of valid properties
* @retval props add property name/value pair here
* @param propertyType sync, source, or unknown (in which case the property name must be given and must be unique)
* @param opt command line option as it appeard in argv (e.g. --sync|--sync-property|-z)
* @param param the parameter following the opt, may be NULL if none given (error!)
* @param propname if given, then this is the property name and param contains the param value (--sync <param>)
*/
bool parseProp(const ConfigPropertyRegistry &validProps,
FilterConfigNode::ConfigFilter &props,
bool parseProp(PropertyType propertyType,
const char *opt,
const char *param,
const char *propname = NULL);
@ -212,22 +211,8 @@ protected:
bool listProperties(const ConfigPropertyRegistry &validProps,
const string &opt);
typedef map<string, ConfigProps> SourceFilters_t;
/**
* read properties from context, then update with
* command line properties
*
* @param context context name, without @ sign
* @retval syncFilter global sync properties
* @retval sourceFilters entries for specific sources, key "" as fallback
*/
void getFilters(const string &context,
ConfigProps &syncFilter,
SourceFilters_t &sourceFilters);
/**
* check that m_syncProps and m_sourceProps don't contain
* check that m_props don't contain
* properties which only apply to peers, throw error
* if found
*/

196
src/syncevo/ConfigFilter.cpp Normal file
View File

@ -0,0 +1,196 @@
/*
* Copyright (C) 2008-2009 Patrick Ohly <patrick.ohly@gmx.de>
* Copyright (C) 2009 Intel Corporation
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) version 3.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
* 02110-1301 USA
*/
#include "config.h"
#include <syncevo/ConfigFilter.h>
#include <syncevo/SyncConfig.h>
SE_BEGIN_CXX
void ConfigProps::add(const ConfigProps &other)
{
BOOST_FOREACH(const StringPair &entry, other) {
std::pair<iterator, bool> res = insert(entry);
if (!res.second) {
res.first->second = entry.second;
}
}
}
ConfigProps SourceProps::createSourceFilter(const std::string &source) const
{
const_iterator it = find("");
ConfigProps filter;
if (it != end()) {
filter = it->second;
}
if (!source.empty()) {
it = find(source);
if (it != end()) {
filter.add(it->second);
}
}
return filter;
}
ConfigProps FullProps::createSyncFilter(const std::string &config) const
{
const_iterator it = find("");
ConfigProps filter;
if (it != end()) {
// first unset context
filter = it->second.m_syncProps;
}
if (!config.empty()) {
std::string normal = SyncConfig::normalizeConfigString(config, false);
std::string peer, context;
SyncConfig::splitConfigString(normal, peer, context);
// then overwrite with context config
it = find(std::string("@") + context);
if (it != end()) {
filter.add(it->second.m_syncProps);
}
// finally peer config, if we have one
if (!peer.empty()) {
it = find(normal);
if (it != end()) {
filter.add(it->second.m_syncProps);
}
}
}
return filter;
}
ConfigProps FullProps::createSourceFilter(const std::string &config,
const std::string &source) const
{
const_iterator it = find("");
ConfigProps filter;
if (it != end()) {
// first unset context
filter = it->second.m_sourceProps.createSourceFilter(source);
}
if (!config.empty()) {
std::string normal = SyncConfig::normalizeConfigString(config, false);
std::string peer, context;
SyncConfig::splitConfigString(normal, peer, context);
// then overwrite with context config
it = find(std::string("@") + context);
if (it != end()) {
filter.add(it->second.m_sourceProps.createSourceFilter(source));
}
// finally peer config, if we have one
if (!peer.empty()) {
it = find(normal);
if (it != end()) {
filter.add(it->second.m_sourceProps.createSourceFilter(source));
}
}
}
return filter;
}
bool FullProps::hasProperties() const
{
BOOST_FOREACH(const value_type &context, *this) {
if (!context.second.m_syncProps.empty()) {
return true;
}
BOOST_FOREACH(const SourceProps::value_type &source, context.second.m_sourceProps) {
if (!source.second.empty()) {
return true;
}
}
}
return false;
}
void FullProps::createFilters(const string &context,
const string &config,
const set<string> *sources,
ConfigProps &syncFilter,
SourceProps &sourceFilters)
{
boost::shared_ptr<SyncConfig> shared;
if (!context.empty()) {
// Read from context. If it does not exist, we simply set no properties
// as filter. Previously there was a check for existance, but that was
// flawed because it ignored the global property "defaultPeer".
shared.reset(new SyncConfig(context));
shared->getProperties()->readProperties(syncFilter);
}
// add command line filters for context or config?
if (!context.empty()) {
syncFilter.add(createSyncFilter(context));
// default for (so far) unknown sources which might be created
sourceFilters[""].add(createSourceFilter(context, ""));
}
if (!config.empty()) {
syncFilter.add(createSyncFilter(config));
sourceFilters[""].add(createSourceFilter(config, ""));
}
// build full set of all sources
set<string> allSources;
if (sources) {
allSources = *sources;
}
if (shared) {
std::list<std::string> tmp = shared->getSyncSources();
allSources.insert(tmp.begin(), tmp.end());
}
if (!config.empty()) {
std::list<std::string> tmp = SyncConfig(config).getSyncSources();
allSources.insert(tmp.begin(), tmp.end());
}
// explicit filter for all known sources
BOOST_FOREACH(std::string source, allSources) {
ConfigProps &props = sourceFilters[source];
if (shared) {
// combine existing properties from context and command line
// filter
SyncSourceNodes nodes = shared->getSyncSourceNodes(source, "");
nodes.getProperties()->readProperties(props);
// Special case "type" property: the value in the context
// is not preserved. Every new peer must ensure that
// its own value is compatible (= same backend) with
// the other peers.
props.erase("type");
props.add(createSourceFilter(context, source));
}
if (!config.empty()) {
props.add(createSourceFilter(config, source));
}
}
}
SE_END_CXX

131
src/syncevo/ConfigFilter.h Normal file
View File

@ -0,0 +1,131 @@
/*
* Copyright (C) 2011 Intel Corporation
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) version 3.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
* 02110-1301 USA
*/
#ifndef INCL_SYNC_EVOLUTION_CONFIG_FILTER
# define INCL_SYNC_EVOLUTION_CONFIG_FILTER
#include <syncevo/util.h>
#include <string>
#include <map>
#include <set>
#include <syncevo/declarations.h>
SE_BEGIN_CXX
using namespace std;
/** a case-insensitive string to string mapping */
class ConfigProps : public map<string, string, Nocase<string> > {
public:
/** format as <key> = <value> lines */
operator string () const;
/**
* Add all entries from the second set of properties,
* overwriting existing ones (in contrast to map::insert(),
* which does not overwrite).
*/
void add(const ConfigProps &other);
};
/**
* Properties for different sources.
*
* Source and property names are case-insensitive.
*/
class SourceProps : public map<string, ConfigProps, Nocase<string> >
{
public:
/**
* Combine per-source property filters with filter for
* all sources: per-source filter values always win.
*/
ConfigProps createSourceFilter(const std::string &source) const;
};
/**
* A pair of sync and source properties. Source properties are
* reached via "" for "all sources", and "<source name>" for a
* specific source.
*/
struct ContextProps
{
ConfigProps m_syncProps;
SourceProps m_sourceProps;
};
/**
* A collection of sync and source settings, including different contexts.
*
* Primary index is by configuration:
* "" for unset, "@<context>" for explicit context, "foo@bar" for peer config
*
* Index is case-insensitive.
*/
class FullProps : public map<string, ContextProps, Nocase<string> >
{
public:
/** any of the contained ConfigProps has entries */
bool hasProperties() const;
/**
* Combines sync properties into one filter, giving "config"
* priority over "context of config" and over "no specific context".
* Contexts which do not apply to the config are silently ignored.
* Error checking for invalid contexts in the FullProps instance
* must be done separately.
*
* @param config empty string (unknown config) or valid peer or context name
*/
ConfigProps createSyncFilter(const std::string &config) const;
/**
* Combines source properties into one filter. Same priority rules
* as for sync properties apply. Priorities inside each context
* are resolved via SourceProps::createSourceFilter(). The context
* is checked first, so "sync@foo@default" overrides "addressbook/sync".
*
* @param config valid peer or context name
* @param source empty string (only pick properties applying to all sources) or source name
*/
ConfigProps createSourceFilter(const std::string &config,
const std::string &source) const;
/**
* read properties from context, then update with command line
* properties for a) that context and b) the given config
*
* @param context context name, including @ sign, empty if not needed
* @param config possibly non-normalized configuration name which determines
* additional filters, can be empty
* @param sources additional sources for which sourceFilters need to be set
* @retval syncFilter global sync properties
* @retval sourceFilters entries for sources known in either context, config, or
* listed explicitly,
* key "" as fallback for unknown sources
*/
void createFilters(const string &context,
const string &config,
const set<string> *sources,
ConfigProps &syncFilter,
SourceProps &sourceFilters);
};
SE_END_CXX
#endif

7
src/syncevo/ConfigNode.cpp
View File

@ -40,4 +40,11 @@ boost::shared_ptr<ConfigNode> ConfigNode::createFileNode(const string &filename)
return savenode;
}
void ConfigNode::writeProperties(const ConfigProps &props)
{
BOOST_FOREACH(const ConfigProps::value_type &entry, props) {
setProperty(entry.first, entry.second);
}
}
SE_END_CXX

16
src/syncevo/ConfigNode.h
View File

@ -32,14 +32,11 @@ using namespace std;
#include <syncevo/declarations.h>
#include <syncevo/util.h>
#include <syncevo/ConfigFilter.h>
SE_BEGIN_CXX
/** a case-insensitive string to string mapping */
class ConfigProps : public map<string, string, Nocase<string> > {
public:
/** format as <key> = <value> lines */
operator string () const;
};
// see ConfigFilter.h
class ConfigProps;
/**
* This class corresponds to the Funambol C++ client
@ -176,12 +173,7 @@ class ConfigNode {
* Add the given properties. To replace the content of the
* node, call clear() first.
*/
virtual void writeProperties(const ConfigProps &props)
{
BOOST_FOREACH(const ConfigProps::value_type &entry, props) {
setProperty(entry.first, entry.second);
}
}
virtual void writeProperties(const ConfigProps &props);
/**
* Remove a certain property.

3
src/syncevo/Makefile.am
View File

@ -19,6 +19,8 @@ lib_LTLIBRARIES = libsyncevolution.la
SYNCEVOLUTION_SOURCES = \
ConfigTree.h \
ConfigFilter.h \
ConfigFilter.cpp \
ConfigNode.h \
ConfigNode.cpp \
HashConfigNode.h \
@ -115,6 +117,7 @@ libsyncevolution_includedir= $(includedir)/syncevo
libsyncevolution_include_HEADERS = \
declarations.h \
Cmdline.h \
ConfigFilter.h \
TrackingSyncSource.h \
MapSyncSource.h \
FileConfigNode.h \

160
src/syncevo/SyncConfig.cpp
View File

@ -77,6 +77,58 @@ std::string ConfigLevel2String(ConfigLevel level)
}
}
PropertySpecifier PropertySpecifier::StringToPropSpec(const std::string &spec, int flags)
{
PropertySpecifier res;
size_t slash = spec.find('/');
if (slash != spec.npos) {
// no normalization needed at the moment
res.m_source = spec.substr(0, slash);
slash++;
} else {
slash = 0;
}
size_t at = spec.find('@', slash);
if (at != spec.npos) {
// Context or config?
if (spec.find('@', at + 1) != spec.npos) {
// has a second @ sign, must be config name
res.m_config = spec.substr(at + 1);
} else {
// context, include leading @ sign
res.m_config = spec.substr(at);
}
if (flags & NORMALIZE_CONFIG) {
res.m_config = SyncConfig::normalizeConfigString(res.m_config, false);
}
} else {
at = spec.size();
}
res.m_property = spec.substr(slash, at - slash);
return res;
}
std::string PropertySpecifier::toString()
{
std::string res;
res.reserve(m_source.size() + 1 + m_property.size() + 1 + m_config.size());
res += m_source;
if (!m_source.empty()) {
res += '/';
}
res += m_property;
if (!m_config.empty()) {
if (m_config[0] != '@') {
res += '@';
}
res += m_config;
}
return res;
}
string ConfigProperty::getName(const ConfigNode &node) const
{
if (m_names.empty()) {
@ -1120,9 +1172,14 @@ list<string> SyncConfig::getSyncSources() const
"/sources"));
}
// get sources from filter and union them into returned sources
BOOST_FOREACH(const SourceFilters_t::value_type &value, m_sourceFilters) {
BOOST_FOREACH(const SourceProps::value_type &value, m_sourceFilters) {
if (value.first.empty()) {
// ignore filter for all sources
continue;
}
list<string>::iterator it = std::find(sources.begin(), sources.end(), value.first);
if ( it == sources.end()) {
if (it == sources.end()) {
// found a filter for a source which does not exist yet
sources.push_back(value.first);
}
}
@ -1181,12 +1238,7 @@ SyncSourceNodes SyncConfig::getSyncSourceNodes(const string &name,
cacheDir = m_tree->getRootPath() + "/" + peerPath + "/.cache";
node = m_tree->open(peerPath, ConfigTree::visible);
peerNode.reset(new FilterConfigNode(node, m_sourceFilter));
SourceFilters_t::const_iterator filter = m_sourceFilters.find(name);
if (filter != m_sourceFilters.end()) {
peerNode =
boost::shared_ptr<FilterConfigNode>(new FilterConfigNode(boost::shared_ptr<ConfigNode>(peerNode), filter->second));
}
peerNode.reset(new FilterConfigNode(node, m_sourceFilters.createSourceFilter(name)));
hiddenPeerNode = m_tree->open(peerPath, ConfigTree::hidden);
trackingNode = m_tree->open(peerPath, ConfigTree::other, changeId);
serverNode = m_tree->open(peerPath, ConfigTree::server, changeId);
@ -1216,12 +1268,7 @@ SyncSourceNodes SyncConfig::getSyncSourceNodes(const string &name,
sharedNode = peerNode;
} else {
node = m_tree->open(sharedPath, ConfigTree::visible);
sharedNode.reset(new FilterConfigNode(node, m_sourceFilter));
SourceFilters_t::const_iterator filter = m_sourceFilters.find(name);
if (filter != m_sourceFilters.end()) {
sharedNode =
boost::shared_ptr<FilterConfigNode>(new FilterConfigNode(boost::shared_ptr<ConfigNode>(sharedNode), filter->second));
}
sharedNode.reset(new FilterConfigNode(node, m_sourceFilters.createSourceFilter(name)));
}
SyncSourceNodes nodes(!peerPath.empty(), sharedNode, peerNode, hiddenPeerNode, trackingNode, serverNode, cacheDir);
@ -2014,9 +2061,6 @@ void SyncConfig::setConfigFilter(bool sync,
if (m_globalNode != m_contextNode) {
m_globalNode->setFilter(filter);
}
} else if (source.empty()) {
m_nodeCache.clear();
m_sourceFilter = filter;
} else {
m_nodeCache.clear();
m_sourceFilters[source] = filter;
@ -2815,13 +2859,17 @@ class SyncConfigTest : public CppUnit::TestFixture {
CPPUNIT_TEST_SUITE(SyncConfigTest);
CPPUNIT_TEST(normalize);
CPPUNIT_TEST(parseDuration);
CPPUNIT_TEST(propertySpec);
CPPUNIT_TEST_SUITE_END();
private:
void normalize()
{
ScopedEnvChange xdg("XDG_CONFIG_HOME", "/dev/null");
ScopedEnvChange home("HOME", "/dev/null");
// use same dir as CmdlineTest...
ScopedEnvChange xdg("XDG_CONFIG_HOME", "CmdlineTest");
ScopedEnvChange home("HOME", "CmdlineTest");
rm_r("CmdlineTest");
CPPUNIT_ASSERT_EQUAL(std::string("@default"),
SyncConfig::normalizeConfigString(""));
@ -2835,6 +2883,31 @@ private:
SyncConfig::normalizeConfigString("FooBar@Something"));
CPPUNIT_ASSERT_EQUAL(std::string("foo_bar_x_y_z"),
SyncConfig::normalizeConfigString("Foo/bar\\x:y:z"));
// keep @default if explicitly requested
CPPUNIT_ASSERT_EQUAL(std::string("foobar@default"),
SyncConfig::normalizeConfigString("FooBar", false));
// test config lookup
SyncConfig foo_default("foo"), foo_other("foo@other"), bar("bar@other");
foo_default.flush();
foo_other.flush();
bar.flush();
CPPUNIT_ASSERT_EQUAL(std::string("foo"),
SyncConfig::normalizeConfigString("foo"));
CPPUNIT_ASSERT_EQUAL(std::string("foo"),
SyncConfig::normalizeConfigString("foo@default"));
CPPUNIT_ASSERT_EQUAL(std::string("foo@default"),
SyncConfig::normalizeConfigString("foo", false));
CPPUNIT_ASSERT_EQUAL(std::string("foo@default"),
SyncConfig::normalizeConfigString("foo@default", false));
CPPUNIT_ASSERT_EQUAL(std::string("foo@other"),
SyncConfig::normalizeConfigString("foo@other"));
foo_default.remove();
CPPUNIT_ASSERT_EQUAL(std::string("foo@other"),
SyncConfig::normalizeConfigString("foo"));
CPPUNIT_ASSERT_EQUAL(std::string("foo@other"),
SyncConfig::normalizeConfigString("foo", false));
}
void parseDuration()
@ -2868,6 +2941,55 @@ private:
CPPUNIT_ASSERT(!SecondsConfigProperty::parseDuration("m", error, seconds));
}
void propertySpec()
{
ScopedEnvChange xdg("XDG_CONFIG_HOME", "/dev/null");
ScopedEnvChange home("HOME", "/dev/null");
PropertySpecifier spec;
spec = PropertySpecifier::StringToPropSpec("foo");
CPPUNIT_ASSERT_EQUAL(string(""), spec.m_source);
CPPUNIT_ASSERT_EQUAL(string("foo"), spec.m_property);
CPPUNIT_ASSERT_EQUAL(string(""), spec.m_config);
CPPUNIT_ASSERT_EQUAL(string("foo"), spec.toString());
spec = PropertySpecifier::StringToPropSpec("source/foo@ContEXT");
CPPUNIT_ASSERT_EQUAL(string("source"), spec.m_source);
CPPUNIT_ASSERT_EQUAL(string("foo"), spec.m_property);
CPPUNIT_ASSERT_EQUAL(string("@context"), spec.m_config);
CPPUNIT_ASSERT_EQUAL(string("source/foo@context"), spec.toString());
spec = PropertySpecifier::StringToPropSpec("source/foo@ContEXT", PropertySpecifier::NO_NORMALIZATION);
CPPUNIT_ASSERT_EQUAL(string("source"), spec.m_source);
CPPUNIT_ASSERT_EQUAL(string("foo"), spec.m_property);
CPPUNIT_ASSERT_EQUAL(string("@ContEXT"), spec.m_config);
CPPUNIT_ASSERT_EQUAL(string("source/foo@ContEXT"), spec.toString());
spec = PropertySpecifier::StringToPropSpec("foo@peer@context");
CPPUNIT_ASSERT_EQUAL(string(""), spec.m_source);
CPPUNIT_ASSERT_EQUAL(string("foo"), spec.m_property);
CPPUNIT_ASSERT_EQUAL(string("peer@context"), spec.m_config);
CPPUNIT_ASSERT_EQUAL(string("foo@peer@context"), spec.toString());
spec = PropertySpecifier::StringToPropSpec("foo@context");
CPPUNIT_ASSERT_EQUAL(string(""), spec.m_source);
CPPUNIT_ASSERT_EQUAL(string("foo"), spec.m_property);
CPPUNIT_ASSERT_EQUAL(string("@context"), spec.m_config);
CPPUNIT_ASSERT_EQUAL(string("foo@context"), spec.toString());
spec = PropertySpecifier::StringToPropSpec("source/foo");
CPPUNIT_ASSERT_EQUAL(string("source"), spec.m_source);
CPPUNIT_ASSERT_EQUAL(string("foo"), spec.m_property);
CPPUNIT_ASSERT_EQUAL(string(""), spec.m_config);
CPPUNIT_ASSERT_EQUAL(string("source/foo"), spec.toString());
spec = PropertySpecifier::StringToPropSpec("");
CPPUNIT_ASSERT_EQUAL(string(""), spec.m_source);
CPPUNIT_ASSERT_EQUAL(string(""), spec.m_property);
CPPUNIT_ASSERT_EQUAL(string(""), spec.m_config);
CPPUNIT_ASSERT_EQUAL(string(""), spec.toString());
}
};
SYNCEVOLUTION_TEST_SUITE_REGISTRATION(SyncConfigTest);

40
src/syncevo/SyncConfig.h
View File

@ -152,6 +152,39 @@ template<class T> class InitList : public list<T> {
typedef InitList<string> Aliases;
typedef InitList<Aliases> Values;
enum PropertyType {
/** sync properties occur once per config */
SYNC_PROPERTY_TYPE,
/** source properties occur once per source in each config */
SOURCE_PROPERTY_TYPE,
/** exact type is unknown */
UNKNOWN_PROPERTY_TYPE
};
/**
* A property name with optional source and context.
* String format is [<source>/]<property>[@<context>|@<peer>@<context>]
*
* Note that the part after the @ sign without another @ is always
* a context. The normal shorthand of just <peer> without context
* does not work here.
*/
class PropertySpecifier {
public:
std::string m_source; /**< source name, empty if applicable to all or sync property */
std::string m_property; /**< property name, must not be empty */
std::string m_config; /**< config name, empty if none, otherwise @<context> or <peer>@<context> */
enum {
NO_NORMALIZATION = 0,
NORMALIZE_SOURCE = 1,
NORMALIZE_CONFIG = 2
};
/** parse, optionally also normalize source and config */
static PropertySpecifier StringToPropSpec(const std::string &spec, int flags = NORMALIZE_SOURCE|NORMALIZE_CONFIG);
std::string toString();
};
/**
* A property has a name and a comment. Derived classes might have
@ -1624,12 +1657,9 @@ private:
/**
* temporary override for all sync source settings
* ("" as key) or specific sources (source name as key)
*/
FilterConfigNode::ConfigFilter m_sourceFilter;
/** temporary override for settings of specific sources */
typedef std::map<std::string, FilterConfigNode::ConfigFilter> SourceFilters_t;
SourceFilters_t m_sourceFilters;
SourceProps m_sourceFilters;
static string getOldRoot() {
return getHome() + "/.sync4j/evolution";