syncevolution/src/dbus/server/read-operations.cpp

336 lines
14 KiB
C++
Raw Normal View History

/*
* Copyright (C) 2011 Intel Corporation
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) version 3.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
* 02110-1301 USA
*/
#include "read-operations.h"
D-Bus server: fork/exec for sync, command line and restore operations This commit moves the blocking syncing, database restore and command line execution into a separate, short-lived process executing the syncevo-dbus-helper. The advantage is that the main syncevo-dbus-server remains responsive under all circumstances (fully asynchronous now) and suffers less from memory leaks and/or crashes during a sync. The core idea behind the new architecture is that Session remains the D-Bus facing side of a session. It continues to run inside syncevo-dbus-server and uses the syncevo-dbus-helper transparently via a custom D-Bus interface between the two processes. State changes of the helper are mirrored in the server. Later the helper might also be used multiple times in a Session. For example, anything related to loading backends should be moved into the helper (currently the "is config usable" check still runs in the syncevo-dbus-server and needs to load/initialize backends). The startup code of the helper already handles that (see boolean result of operation callback), but it is not used yet in practice. At the moment, only the helper provides a D-Bus API. It sends out signals when it needs information from the server. The server watches those and replies when ready. The helper monitors the connection to the parent and detects that it won't get an answer if that connection goes down. The problem of "helper died unexpectedly" is also handled, by not returning a D-Bus method reply until the requested operation is completed (different from the way how the public D-Bus API is defined!). The Connection class continues to use such a Session, as before. It's now fully asynchronous and exchanges messages with the helper via the Session class. Inside syncevo-dbus-server, boost::signals2 and the dbus-callbacks infrastructure for asynchronous methods execution are used heavily now. The glib event loop is entered exactly once and only left to shut down. Inside syncevo-dbus-helper, the event loop is entered only as needed. Password requests sent from syncevo-local-sync to syncevo-dbus-helper are handled asynchronously inside the event loop driven by the local transport. syncevo-dbus-helper and syncevo-local-sync are conceptually very similar. Should investigate whether a single executable can serve both functions. The AutoSyncManager was completely rewritten. The data structure is a lot simpler now (basically just a cache of transient information about a sync config and the relevant config properties that define auto syncing). The main work happens inside the schedule() call, which verifies whether a session can run and, if not possible for some reasons, ensures that it gets invoked again when that blocker is gone (timeout over, server idle, etc.). The new code also uses signals/slots instead of explicit coupling between the different classes. All code still lives inside the src/dbus/server directory. This simplifies checking differences in partly modified files like dbus-sync.cpp. A future commit will move the helper files. The syslog logger code is referenced by the server, but never used. This functionality needs further thought: - Make usage depend on command line option? Beware that test-dbus.py looks for the "ready to run" output and thus startup breaks when all output goes to syslog instead of stdout. - Redirect glib messages into syslog (done by LogRedirect, disabled when using LoggerSyslog)? The syncevo-dbus-server now sends the final "Session.StatusChanged done" signal immediately. The old implementation accidentally delayed sending that for 100 seconds. The revised test-dbus.py checks for more "session done" quit events to cover this fix. Only user-visible messages should have the INFO level in any of the helpers. Messages about starting and stopping processes are related to implementation details and thus should only have DEBUG level. The user also doesn't care about where the operation eventually runs. All messages related to it should be in INFO/DEBUG/ERROR messages without a process name. Therefore now syncevo-dbus-server logs with a process name (also makes some explicit argv[0] logging redundant; requires changes in test-dbus.py) and syncevo-dbus-helper doesn't. syncevo-local-sync is different from syncevo-dbus-helper: it produces user-relevant output (the other half of the local sync). It's output is carefully chosen so that the process name is something the user understands (target context) and output can be clearly related to one side or the other (for example, context names are included in the sync table). Output handling is based on the same idea as output handling in the syncevo-dbus-server: - Session registers itself as the top-most logger and sends SyncEvolution logging via D-Bus to the parent, which re-sends it with the right D-Bus object path as output of the session. - Output redirection catches all other output and feeds it back to the Session log handler, from where it goes via D-Bus to the parent. The advantage of this approach is that level information is made available directly to the parent and that message boundaries are preserved properly. stderr and stdout are redirected into the parent and logged there as error. Normally the child should not print anything. While it runs, LogRedirect inside it will capture output and log it internally. Anything reaching the parent thus must be from early process startup or shutdown. Almost all communication from syncevo-dbus-helper to syncevo-dbus-server is purely information for the syncevo-dbus-server; syncevo-dbus-helper doesn't care whether the signal can be delivered. The only exception is the information request, which must succeed. Instead of catching exceptions everywhere, the optional signals are declared as such in the EmitSignal template parameterization and no longer throw exceptions when something goes wrong. They also don't log anything, because that could lead to quite a lof of output.
2012-03-26 17:19:25 +02:00
#include "dbus-user-interface.h"
#include "server.h"
#include "dbus-sync.h"
#include <syncevo/IniConfigNode.h>
SE_BEGIN_CXX
ReadOperations::ReadOperations(const std::string &config_name, Server &server) :
m_configName(config_name), m_server(server)
{}
void ReadOperations::getConfigs(bool getTemplates, std::vector<std::string> &configNames)
{
if (getTemplates) {
SyncConfig::DeviceList devices;
// get device list from dbus server, currently only bluetooth devices
m_server.getDeviceList(devices);
// also include server templates in search
devices.push_back(SyncConfig::DeviceDescription("", "", SyncConfig::MATCH_FOR_CLIENT_MODE));
//clear existing templates in dbus server
m_server.clearPeerTempls();
SyncConfig::TemplateList list = SyncConfig::getPeerTemplates(devices);
std::map<std::string, int> numbers;
BOOST_FOREACH(const boost::shared_ptr<SyncConfig::TemplateDescription> peer, list) {
//if it is not a template for device
if(peer->m_deviceName.empty()) {
configNames.push_back(peer->m_templateId);
} else {
string templName = "Bluetooth_";
templName += peer->m_deviceId;
templName += "_";
std::map<std::string, int>::iterator it = numbers.find(peer->m_deviceId);
if(it == numbers.end()) {
numbers.insert(std::make_pair(peer->m_deviceId, 1));
templName += "1";
} else {
it->second++;
stringstream seq;
seq << it->second;
templName += seq.str();
}
configNames.push_back(templName);
m_server.addPeerTempl(templName, peer);
}
}
} else {
SyncConfig::ConfigList list = SyncConfig::getConfigs();
BOOST_FOREACH(const SyncConfig::ConfigList::value_type &server, list) {
configNames.push_back(server.first);
}
}
}
boost::shared_ptr<SyncConfig> ReadOperations::getLocalConfig(const string &configName, bool mustExist)
{
string peer, context;
SyncConfig::splitConfigString(SyncConfig::normalizeConfigString(configName),
peer, context);
boost::shared_ptr<SyncConfig> syncConfig(new SyncConfig(configName));
/** if config was not set temporarily */
if (!setFilters(*syncConfig)) {
// the default configuration can always be opened for reading,
// everything else must exist
if ((context != "default" || peer != "") &&
mustExist &&
!syncConfig->exists()) {
SE_THROW_EXCEPTION(NoSuchConfig, "No configuration '" + configName + "' found");
}
}
return syncConfig;
}
void ReadOperations::getConfig(bool getTemplate,
Config_t &config)
{
getNamedConfig(m_configName, getTemplate, config);
}
void ReadOperations::getNamedConfig(const std::string &configName,
bool getTemplate,
Config_t &config)
{
map<string, string> localConfigs;
boost::shared_ptr<SyncConfig> dbusConfig;
SyncConfig *syncConfig;
string syncURL;
/** get server template */
if(getTemplate) {
string peer, context;
boost::shared_ptr<SyncConfig::TemplateDescription> peerTemplate =
m_server.getPeerTempl(configName);
if(peerTemplate) {
SyncConfig::splitConfigString(SyncConfig::normalizeConfigString(peerTemplate->m_templateId),
peer, context);
dbusConfig = SyncConfig::createPeerTemplate(peerTemplate->m_path);
// if we have cached template information, add match information for it
localConfigs.insert(pair<string, string>("description", peerTemplate->m_description));
stringstream score;
score << peerTemplate->m_rank;
localConfigs.insert(pair<string, string>("score", score.str()));
// Actually this fingerprint is transferred by getConfigs, which refers to device name
localConfigs.insert(pair<string, string>("deviceName", peerTemplate->m_deviceName));
// This is the reliable device info obtained from the bluetooth
// device id profile (DIP) or emtpy if DIP not supported.
if (!peerTemplate->m_hardwareName.empty()) {
localConfigs.insert(pair<string, string>("hardwareName", peerTemplate->m_hardwareName));
}
// This is the fingerprint of the template
localConfigs.insert(pair<string, string>("fingerPrint", peerTemplate->m_matchedModel));
// This is the template name presented to UI (or device class)
if (!peerTemplate->m_templateName.empty()) {
localConfigs.insert(pair<string,string>("templateName", peerTemplate->m_templateName));
}
// if the peer is client, then replace syncURL with bluetooth
// MAC address
syncURL = "obex-bt://";
syncURL += peerTemplate->m_deviceId;
} else {
SyncConfig::splitConfigString(SyncConfig::normalizeConfigString(configName),
peer, context);
dbusConfig = SyncConfig::createPeerTemplate(peer);
}
if(!dbusConfig.get()) {
SE_THROW_EXCEPTION(NoSuchConfig, "No template '" + configName + "' found");
}
// use the shared properties from the right context as filter
// so that the returned template preserves existing properties
boost::shared_ptr<SyncConfig> shared = getLocalConfig(string("@") + context, false);
ConfigProps props;
shared->getProperties()->readProperties(props);
dbusConfig->setConfigFilter(true, "", props);
BOOST_FOREACH(std::string source, shared->getSyncSources()) {
SyncSourceNodes nodes = shared->getSyncSourceNodes(source, "");
props.clear();
nodes.getProperties()->readProperties(props);
// Special case "type" property: the value in the context
// is not preserved. Every new peer must ensure that
// its own value is compatible (= same backend) with
// the other peers.
props.erase("type");
dbusConfig->setConfigFilter(false, source, props);
}
syncConfig = dbusConfig.get();
} else {
dbusConfig = getLocalConfig(configName);
password handling: fixed KWallet support, global configuration option KWallet support was broken: syncevo-dbus-server checked KDE_FULL_SESSION to determine whether it should use KWallet instead of GNOME Keyring. That did not work, because the env variable was not set for D-Bus daemons. Automatically detecting KDE users is not possible at the moment. Instead KDE users have to manually set the new "keyring" global config property to "KDE" (case insensitive) if the SyncEvolution installation supports both, because GNOME Keyring is the default to avoid surprises for traditional users. If only KWallet support is enabled, then this is not necessary. "GNOME" and "true/false/1/0/yes/no" can also be set. This has the advantage that keyring usage can be enabled permanently for the command line in --daemon=no mode; normally keyrings are not used in that mode because accessing them can bring up UI dialogs. It also becomes possible to disable keyring usage in syncevo-dbus-server, something which couldn't be done before. The --keyring command line option is still supported, as an alias for "[--sync-property] keyring=<value>". The default value for --keyring is true, to match the traditional behavior. In contrast to other sync properties, setting "keyring" does not require an explicit --run parameter. Again this is done to mirror traditional usage. Reading a password also (unintentionally) checked all supported storages while searching for the password. Now it uses exactly one storage and falls back to asking for the password directly. The commit itself also cleans up the code a bit (reformatted, fixed comments). Choosing the right slot in the password signals is done via a new InitStateTri parameter which contains the "keyring" setting. Error checking (unsupported keyring string, --keyring=yes and no keyring enabled) is done in additional slots which run after all the regular ones. Parameter parsing for --sync and --keyring were unified. However, there is the difference that --keyring has an implicit default value ("yes") and never has an additional parameter, in contrast to --sync, which always is followed by one. The new CmdlineTest::testKeyring covers different ways of using --keyring. It relies on actually invoking keyring backends, something not done by the default SyncContext UI. Therefore CmdlineSyncClient+KeyringSyncCmdline were moved into libsyncevolution, to be used by CmdlineTest.
2012-05-29 18:14:13 +02:00
DBusUserInterface ui(dbusConfig->getKeyring());
//try to check password and read password from gnome keyring if possible
PasswordConfigProperty::checkPasswords(ui, *dbusConfig,
// Keep usernames as they are, but retrieve passwords for the D-Bus client.
PasswordConfigProperty::CHECK_PASSWORD_ALL & ~PasswordConfigProperty::CHECK_PASSWORD_RESOLVE_USERNAME,
dbusConfig->getSyncSources());
syncConfig = dbusConfig.get();
}
/** get sync properties and their values */
ConfigPropertyRegistry &syncRegistry = SyncConfig::getRegistry();
BOOST_FOREACH(const ConfigProperty *prop, syncRegistry) {
InitStateString value = prop->getProperty(*syncConfig->getProperties());
if (boost::iequals(prop->getMainName(), "syncURL") && !syncURL.empty() ) {
localConfigs.insert(pair<string, string>(prop->getMainName(), syncURL));
} else if (value.wasSet()) {
localConfigs.insert(pair<string, string>(prop->getMainName(), value));
}
}
// Set ConsumerReady for existing SyncEvolution < 1.2 configs
// if not set explicitly,
// because in older releases all existing configurations where
// shown. SyncEvolution 1.2 is more strict and assumes that
// ConsumerReady must be set explicitly. The sync-ui always has
// set the flag for configs created or modified with it, but the
// command line did not. Matches similar code in the Cmdline.cpp
// migration code.
//
// This does not apply to templates which always have ConsumerReady
// set explicitly (to on or off) or not set (same as off).
if (!getTemplate &&
syncConfig->getConfigVersion(CONFIG_LEVEL_PEER, CONFIG_CUR_VERSION) == 0 /* SyncEvolution < 1.2 */) {
localConfigs.insert(make_pair("ConsumerReady", "1"));
}
// insert 'configName' of the chosen config (configName is not normalized)
localConfigs.insert(pair<string, string>("configName", syncConfig->getConfigName()));
config.insert(pair<string,map<string, string> >("", localConfigs));
/* get configurations from sources */
list<string> sources = syncConfig->getSyncSources();
BOOST_FOREACH(const string &name, sources) {
localConfigs.clear();
SyncSourceNodes sourceNodes = syncConfig->getSyncSourceNodes(name);
ConfigPropertyRegistry &sourceRegistry = SyncSourceConfig::getRegistry();
BOOST_FOREACH(const ConfigProperty *prop, sourceRegistry) {
InitStateString value = prop->getProperty(*sourceNodes.getProperties());
if (value.wasSet()) {
localConfigs.insert(pair<string, string>(prop->getMainName(), value));
}
}
config.insert(pair<string, map<string, string> >( "source/" + name, localConfigs));
}
}
void ReadOperations::getReports(uint32_t start, uint32_t count,
Reports_t &reports)
{
SyncContext client(m_configName, false);
std::vector<string> dirs;
client.getSessions(dirs);
uint32_t index = 0;
// newest report firstly
for( int i = dirs.size() - 1; i >= 0; --i) {
/** if start plus count is bigger than actual size, then return actual - size reports */
if(index >= start && index - start < count) {
const string &dir = dirs[i];
std::map<string, string> aReport;
// insert a 'dir' as an ID for the current report
aReport.insert(pair<string, string>("dir", dir));
SyncReport report;
// peerName is also extracted from the dir
string peerName = client.readSessionInfo(dir,report);
boost::shared_ptr<SyncConfig> config(new SyncConfig(m_configName));
string storedPeerName = config->getPeerName();
//if can't find peer name, use the peer name from the log dir
if(!storedPeerName.empty()) {
peerName = storedPeerName;
}
/** serialize report to ConfigProps and then copy them to reports */
IniHashConfigNode node("/dev/null","",true);
node << report;
ConfigProps props;
node.readProperties(props);
BOOST_FOREACH(const ConfigProps::value_type &entry, props) {
aReport.insert(entry);
}
// a new key-value pair <"peer", [peer name]> is transferred
aReport.insert(pair<string, string>("peer", peerName));
reports.push_back(aReport);
}
index++;
}
}
void ReadOperations::checkSource(const std::string &sourceName)
{
boost::shared_ptr<SyncConfig> config(new SyncConfig(m_configName));
setFilters(*config);
list<std::string> sourceNames = config->getSyncSources();
list<std::string>::iterator it;
for(it = sourceNames.begin(); it != sourceNames.end(); ++it) {
if(*it == sourceName) {
break;
}
}
if(it == sourceNames.end()) {
source -> datastore rename, improved terminology The word "source" implies reading, while in fact access is read/write. "datastore" avoids that misconception. Writing it in one word emphasizes that it is single entity. While renaming, also remove references to explicit --*-property parameters. The only necessary use today is "--sync-property ?" and "--datastore-property ?". --datastore-property was used instead of the short --store-property because "store" might be mistaken for the verb. It doesn't matter that it is longer because it doesn't get typed often. --source-property must remain valid for backward compatility. As many user-visible instances of "source" as possible got replaced in text strings by the newer term "datastore". Debug messages were left unchanged unless some regex happened to match it. The source code will continue to use the old variable and class names based on "source". Various documentation enhancements: Better explain what local sync is and how it involves two sync configs. "originating config" gets introduces instead of just "sync config". Better explain the relationship between contexts, sync configs, and source configs ("a sync config can use the datastore configs in the same context"). An entire section on config properties in the terminology section. "item" added (Todd Wilson correctly pointed out that it was missing). Less focus on conflict resolution, as suggested by Graham Cobb. Fix examples that became invalid when fixing the password storage/lookup mechanism for GNOME keyring in 1.4. The "command line conventions", "Synchronization beyond SyncML" and "CalDAV and CardDAV" sections were updated. It's possible that the other sections also contain slightly incorrect usage of the terminology or are simply out-dated.
2014-07-28 15:29:41 +02:00
SE_THROW_EXCEPTION(NoSuchSource, "'" + m_configName + "' has no '" + sourceName + "' datastore");
}
bool checked = false;
try {
// this can already throw exceptions when the config is invalid
SyncSourceParams params(sourceName, config->getSyncSourceNodes(sourceName), config);
auto_ptr<SyncSource> syncSource(SyncSource::createSource(params, false, config.get()));
if (syncSource.get()) {
syncSource->open();
// success!
checked = true;
}
} catch (...) {
Exception::handle();
}
if (!checked) {
source -> datastore rename, improved terminology The word "source" implies reading, while in fact access is read/write. "datastore" avoids that misconception. Writing it in one word emphasizes that it is single entity. While renaming, also remove references to explicit --*-property parameters. The only necessary use today is "--sync-property ?" and "--datastore-property ?". --datastore-property was used instead of the short --store-property because "store" might be mistaken for the verb. It doesn't matter that it is longer because it doesn't get typed often. --source-property must remain valid for backward compatility. As many user-visible instances of "source" as possible got replaced in text strings by the newer term "datastore". Debug messages were left unchanged unless some regex happened to match it. The source code will continue to use the old variable and class names based on "source". Various documentation enhancements: Better explain what local sync is and how it involves two sync configs. "originating config" gets introduces instead of just "sync config". Better explain the relationship between contexts, sync configs, and source configs ("a sync config can use the datastore configs in the same context"). An entire section on config properties in the terminology section. "item" added (Todd Wilson correctly pointed out that it was missing). Less focus on conflict resolution, as suggested by Graham Cobb. Fix examples that became invalid when fixing the password storage/lookup mechanism for GNOME keyring in 1.4. The "command line conventions", "Synchronization beyond SyncML" and "CalDAV and CardDAV" sections were updated. It's possible that the other sections also contain slightly incorrect usage of the terminology or are simply out-dated.
2014-07-28 15:29:41 +02:00
SE_THROW_EXCEPTION(SourceUnusable, "The datastore '" + sourceName + "' is not usable");
}
}
void ReadOperations::getDatabases(const string &sourceName, SourceDatabases_t &databases)
{
boost::shared_ptr<SyncConfig> config(new SyncConfig(m_configName));
setFilters(*config);
SyncSourceParams params(sourceName, config->getSyncSourceNodes(sourceName), config);
const SourceRegistry &registry(SyncSource::getSourceRegistry());
BOOST_FOREACH(const RegisterSyncSource *sourceInfo, registry) {
auto_ptr<SyncSource> source(sourceInfo->m_create(params));
if (!source.get()) {
continue;
} else if (source->isInactive()) {
source -> datastore rename, improved terminology The word "source" implies reading, while in fact access is read/write. "datastore" avoids that misconception. Writing it in one word emphasizes that it is single entity. While renaming, also remove references to explicit --*-property parameters. The only necessary use today is "--sync-property ?" and "--datastore-property ?". --datastore-property was used instead of the short --store-property because "store" might be mistaken for the verb. It doesn't matter that it is longer because it doesn't get typed often. --source-property must remain valid for backward compatility. As many user-visible instances of "source" as possible got replaced in text strings by the newer term "datastore". Debug messages were left unchanged unless some regex happened to match it. The source code will continue to use the old variable and class names based on "source". Various documentation enhancements: Better explain what local sync is and how it involves two sync configs. "originating config" gets introduces instead of just "sync config". Better explain the relationship between contexts, sync configs, and source configs ("a sync config can use the datastore configs in the same context"). An entire section on config properties in the terminology section. "item" added (Todd Wilson correctly pointed out that it was missing). Less focus on conflict resolution, as suggested by Graham Cobb. Fix examples that became invalid when fixing the password storage/lookup mechanism for GNOME keyring in 1.4. The "command line conventions", "Synchronization beyond SyncML" and "CalDAV and CardDAV" sections were updated. It's possible that the other sections also contain slightly incorrect usage of the terminology or are simply out-dated.
2014-07-28 15:29:41 +02:00
SE_THROW_EXCEPTION(NoSuchSource, "'" + m_configName + "' backend of datastore '" + sourceName + "' is not supported");
} else {
databases = source->getDatabases();
return;
}
}
source -> datastore rename, improved terminology The word "source" implies reading, while in fact access is read/write. "datastore" avoids that misconception. Writing it in one word emphasizes that it is single entity. While renaming, also remove references to explicit --*-property parameters. The only necessary use today is "--sync-property ?" and "--datastore-property ?". --datastore-property was used instead of the short --store-property because "store" might be mistaken for the verb. It doesn't matter that it is longer because it doesn't get typed often. --source-property must remain valid for backward compatility. As many user-visible instances of "source" as possible got replaced in text strings by the newer term "datastore". Debug messages were left unchanged unless some regex happened to match it. The source code will continue to use the old variable and class names based on "source". Various documentation enhancements: Better explain what local sync is and how it involves two sync configs. "originating config" gets introduces instead of just "sync config". Better explain the relationship between contexts, sync configs, and source configs ("a sync config can use the datastore configs in the same context"). An entire section on config properties in the terminology section. "item" added (Todd Wilson correctly pointed out that it was missing). Less focus on conflict resolution, as suggested by Graham Cobb. Fix examples that became invalid when fixing the password storage/lookup mechanism for GNOME keyring in 1.4. The "command line conventions", "Synchronization beyond SyncML" and "CalDAV and CardDAV" sections were updated. It's possible that the other sections also contain slightly incorrect usage of the terminology or are simply out-dated.
2014-07-28 15:29:41 +02:00
SE_THROW_EXCEPTION(NoSuchSource, "'" + m_configName + "' has no '" + sourceName + "' datastore");
}
SE_END_CXX