Error codes >= sysync::LOCAL_STATUS_CODE stand for problems detected
locally. This wasn't handled correctly by SyncEvolution:
- by default, unknown local exceptions led to the non-local STATUS_FATAL = 500
- Synthesis error codes reported via BadSynthesisResult got lost
in Exception::handle() because that exception class had no
special catch clause
When the overall source and thus sync session status is "OK" despite
non-zero item failures, the status is set to 22001 =
STATUS_PARTIAL_FAILURE. The command line then reports "some changes
could not be transferred (local, status 22001)".
The syncevolution command line return code indicates a failure just as
for all the other non-ok statuses.
For QA: it should be possible to reproduce this problem by setting the
backend of a source to the file backend and then pointing it towards
an existing directory where the current user has no write
permissions. Then do a "refresh-from-server" with a non-empty server.
The formatting of a sync report assumed that the number of failed or
rejected items was recorded per NEW/MOD/DEL operation. This was wrong,
only a summary of local and remote failures is available. This patch
changes the formatting so that these summaries are shown as additional
"ERR" column.
The "on server/client" strings were replaced with "REMOTE/LOCAL";
previously the report was labelled incorrectly in a SyncML server.
However, currently we don't have these statistics there anyway (MB #7709).
Here's an example:
+---------------|-----------------------|-----------------------|-CON-+
| | LOCAL | REMOTE | FLI |
| Source | NEW | MOD | DEL | ERR | NEW | MOD | DEL | ERR | CTS |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| addressbook | 32 | 0 | 32 | 2 | 0 | 0 | 0 | 0 | 0 |
| refresh-from-server, 0 KB sent by client, 33 KB received |
| item(s) in database backup: 32 before sync, 30 after it |
| some changes could not be transferred (local, status 22001) |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| calendar | 13 | 0 | 13 | 0 | 0 | 0 | 0 | 0 | 0 |
| refresh-from-server, 0 KB sent by client, 7 KB received |
| item(s) in database backup: 13 before sync, 13 after it |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| start Fri Jan 15 15:54:18 2010, duration 0:09min |
| some changes could not be transferred (local, status 22001) |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+
First ERROR encountered: addressbook: intentional error
The --status output is unchanged:
+---------------------------------------------|-----------------------+
| Source | NEW | MOD | DEL |TOTAL|
+---------------------------------------------+-----+-----+-----+-----+
| addressbook | 0 | 0 | 0 | 0 |
+---------------------------------------------+-----+-----+-----+-----+
| calendar | 0 | 0 | 0 | 0 |
+---------------------------------------------+-----+-----+-----+-----+
| memo | 0 | 0 | 0 | 0 |
+---------------------------------------------+-----+-----+-----+-----+
| todo | 0 | 0 | 0 | 0 |
+---------------------------------------------+-----+-----+-----+-----+
| start Fri Jan 15 16:25:13 2010, duration 0:00min |
+---------------------------------------------+-----+-----+-----+-----+
The statistics array entries are left unchanged by this patch
intentionally. First, we might have these failure statistics per
operation at some point. Second, on-disk .ini dump parsing would need
backward compatibility code if the internal array was changed.
When we can be sure that we have no local data in a source,
then the check for an unexpected slow sync can be suppressed
because there is no risk of duplicates or lost data.
The check depends on having done a backup before. If that was
disabled, the check is enabled because there is no other API
to query the content of the local database.
It would be nice if we had the same information about the server,
but that is not available a) without running the sync and b) not
sent by all servers.
This solution depends on ABORTDATASTORE() in an <alertscript>,
which is supported by libsynthesis only with an additional patch.
ABORTDATASTORE() is done for each source where the sync mode
requested by the server does not match the expected one.
The check is only added when a slow sync would not be acceptable.
Explicit "slow" and "resfresh-from-server" (which is done under
the hood as a slow sync?!) don't need the check. Distinguishing
unexpected "slow" from "refresh-from-client" has to be done
via ALERTCODE(), SLOWSYNC() returns true in both cases.
Note that currently, "no local data" and "first time sync" are
ignored when checking for acceptable slow syncs. This might have
to be added.
In addition, the code in SyncContext::doSync() is notified via the
"delayedabort" session variable that it should abort after processing
the current message and before sending the reply. This is necessary to
get the <alertscript> executed for all sources. Calling ABORTSESSION()
inside the script was tried first but aborted right away, so that an
unexpected slow sync could only be detected for one source.
Abort handling didn't work for aborting before sending a reply:
- SessionStep would return STEPCMD_SENDDATA
- we would start sending
- abort was checked later
A good point to abort a sync is after the engine has processed
a message and before sending out the reply. This patch adds that
check for STEPCMD_SENDDATA and checkForAbort() directly after
the SessionStep() call.
Because the whole suspend/abort handling is somewhat tricky, this
patch adds debug logging around those state changes.
The code which detects the request to abort also analyzes the sitution
and prints some guidance for the user how to recover from the problem:
[ERROR] Aborting because of unexpected slow sync for source(s): calendar
[INFO] Doing a slow synchronization may lead to duplicated items or
lost data when the server merges items incorrectly. Choosing
a different synchronization mode may be the better alternative.
Restart synchronization of affected source(s) with one of the
following sync modes to recover from this problem:
slow, refresh-from-server, refresh-from-client
Analyzing the current state:
syncevolution --status syncevolution_client calendar
Running with one of the three modes:
syncevolution --sync [slow|refresh-from-server|refresh-from-client] syncevolution_client calendar
On the server, doing this check in <alertscript> is not sufficient
because the server does the anchor check after executing the
<alertscript> and thus switches to an unexpected slow sync later on.
Need a different solution for server initiated syncs with phones.
This patch adds a plain text explanation of the Synthesis and SyncML
error codes. It explains whether the error was found locally or
remotely and includes the original status code, because sometimes
this is more helpful than the text.
The ERROR logging uses these explanations and the sync report
formatter.
This patch also adds some more status codes:
- 418 "already exists"
- 22000 "unexpected slow sync"
The latter is in the range reserved for application codes. We'll
use that for MB #2416.
Printing of per-source status codes was avoided, both at the point
where they were encountered as well as in the final sync report. The
reason for that was that often, the overall sync status is duplicated
for each source.
But for those cases were each source has a different status or no
error at all, printing that piece of information is useful, so this
patch adds it.
Note that now "no status code or error" for a source means that it
synchronized successfully.
Introduced TemplateConfig to abstracting the template configuration structure,
the template metadata used for matching is also parsed here.
The fields introduced in the metadata are:
PeerIsClient: identify whether this is a server side configuration or a client
side configuration.
Fingerprint: the matching string for this template, it is a comma separated string
with each string modeled as: "Manufacture_Model". The first substring is also
used as the name to identify this template so that user can select the template
by this name.
eg:
Nokia 7210c: Nokia_7210c
SyncEvolution server: SyncEvolutionServer, SyncEvolution
ScheduleWorld: ScheduleWorld,default
SyncEvolution client: SyncEvolutionClient, SyncEvolution
Description: this is a just a descriptive string not used for matching.
GetServerTemplates is changed to add another "devices" parameter to identify
it is asking for templates for a list of "devices". Each device is a tuple
<matchstring (devicename), matchMode (server/client/all)>.
TemplateList as the return type, which is a list of class TemplateDescription
so that we can also return enough information for corresponding templates. This
list is sorted by the 3-tuple <finger, rank, name>.
Add MatchServerTemplates method which will iterating all templates inside the
folder and match against the input parameter and finally return a sorted
list of matched templates.
The atcually fuzzy match algorithm is based on a LCS (added in the following
commit).
Cmdline interface is changed accordingly:
--template ? is changed to --template ?[string], so that user use the former
case to match all templates for a tradiontial SyncML client and the latter case
to match templates related to an input string.
SyncConfig API is also renamed (Server -> Peer) because both server/client
configuration/template are handled.
The original configuration template (Funambol and ScheduleWorld) has been moved
to the new template structure (under servers), they also have a .template.ini
file added so that they can be matched and picked up. All templates for
supported servers still have built-in template support in the code as before.
Templates for SyncEvolution based server is also added.
Server side templates are added (Nokia default, Nokia_7210c and SyncEvolutionServer).
Add unit test for the new template match use case.
The code was originally written for an improved diff to be used in
synccompare. It improves upon the normal LCS by taking into account
that the some subsequences might be "nicer" than others of the same
length, for example because they cross less "boundaries" inside the
sequences.
The reason for adding it now is for use in a fuzzy manufacturer/model
matching.
Changing the CPPUnit defines broke linking of syncevolution in Moblin
(--enable-integration-tests): test.cpp then depends on CPPUnit libs
and was linked into syncevolution and syncevo-dbus-server, causing link
failures.
Made inclusion of test.cpp conditional on actually having unit tests
inside the library. Only then do syncevolution and syncevo-dbus-server
have to link against CPPUnit.
As requested by Mark, the webURL was changed from
sync.scheduleworld.com to www.scheduleworld.com, which is the
official entry point for browerser. IMHO sync.scheduleworld.com
has always worked, though, and currently still does.
Goosync provides access to Google calendar, tasks and contacts
via SyncML. Google itself only supports contacts. Memos are not
supported by either of them.
This patch only adds the template with the necessary configuration
parameters. We have done no testing with the service, so it is
not marked as "consumer ready" at this time.
The template uses an unspecific "https://your.company/mobilesync/server"
syncURL. The hope is that users just have to replace "your.company"
in many cases.
https is chosen because apparently, many installations are restricted
to that.
The recent fix for extracting broken calendar events introduced
icalproperty_remove_parameter_by_kind(), which has to be added to
the EDS wrapper to avoid hard library dependencies.
When an instance was deleted while it had a pending message and
active timeout/abort events, those event sources were not removed.
Using the new smart pointer type for GLib event handles avoids that.
According to the documentation, a handle is always > 0, so initializing
it with 0 as GLibEvent is more appropriate than the -1 used previously
in the code here.
There might still be one problem left: what if wait() is left by
throwing an exception? Should the two event sources be cleared in that
case?
A SyncML server cannot resend a message. With HTTP, a reply simply
cannot be sent, for other transports the D-Bus API doesn't support it.
Therefore RetryInterval is ignored and only RetryDuration is used as
the final timeout after which the session is aborted.
The same transport callback is used for client and server. It was
changed to log the timeout duration (sent in via its parameter). The
rest of the logic is in the caller of TransportAgent::wait().
While touching the timeout code in initSAN(), the resending of the SAN
message was removed. This makes the code consistent with the HTTP
SyncML client case, where the initial message is also not resent.
The new test-dbus.py TestConnection.testTimeoutSync covers thus timeout
handling. It depends on setting RetryDuration to 10 in the server-side
config of the sc-api-nat device.
The timeout will be needed for connections to clients which stop
sending messages or died without telling the server stub (HTTP
server). A test will be added once the server mode properly supports
such timeouts.
When the timeout is triggered, we need to get out of the
g_main_loop_run() called indirectly by DBusTransportAgent::wait(). We don't
want to interfere with other calls to that function, so g_main_loop_quit()
is only called when waiting in the agent. Otherwise the timeout is
remembered and reported the next time wait() is called.
The "cancel on false from callback" feature is not implemented. It is not
used and thus would be impossible to test. Remove this feature?
A GLib guint event handle can be compared to 0 (the invalid handle),
just like a pointer. Previously, eptr didn't support this because the
template was parameterized with a class and then used pointers to it.
This patch changes the base template so that both pointers and
integral types can be tracked. The base template was renamed to
SmartPtr. eptr became a source compatibility class with the previous
"track pointers" semantic.
GLibEvent is a specialization which uses g_source_remove() to free
the event associated with a guint handle.
I just noticed that the statistics table had "0" for all cells, despite having
local changes and items.
This broke during the backend API migration towards Boost function pointers.
The implementation in TrackingSyncSource did not have enough parameters and did
not fill the SyncSourceReport passed in as parameter with the values.
I'm not sure why the old code compiled, I would have expected type errors
because boost::bind() with only "this" as parameter should not have matched
m_checkStatus. Anyway, now the item counts are copied into the report.
There might be exceptions or other problems making sync completed
unnormally without setting 'waiting' as false. So we always set
the 'waiting' as false since sync is done and no longer waiting
any I/O events anymore.
It scanned "peers" inside the top .config/syncevolution directory
instead of looking inside the current context. This showed up as
failure in test-dbus.py testSourceRemovalGlobal.
SyncSource::createTestingSource() has to create sources using a
default configuration. It did that by using the "testing" config name.
Since introducing shared configs, that config happened to use shared
source properties from the "default" context.
This broke testing on my laptop after setting "evolutionsource" for a
specific peer in that context. Now "testing@testing" is used as config
name, which should be unique again.
libsoup does not use http_proxy set in environment. SyncConfig reads
http_proxy from environment and returns it. If it is not avilable in
the environment, then configured value is returned.
"http_proxy" set to an empty value disables the use
of a proxy.
Older automake doesn't have --with-docdir and thus no
way of installing our docs in that dir. As a workaround,
check right before generating Makefiles whether $docdir
is empty and set it if necessary.
This happens for corrupt calendar data where a VEVENT refers to a
VTIMEZONE that is not available. Work around this by removing the
broken TZID in case of failure, then retrying the encoding.
This treats such events as if they were using local time without
timezone information, which is also how the Evolution GUI deals
with such events. It is correct as long as users only deal with
such broken events in their own timezone.
Mobical has many regressions due to 'priority' property.
Mobical doesn't send 'priority' when its value is '0'. This has
been recorded in the README.mobical. The root cause of regressions
is due to changes in synccompare for nokia_7210c.
'=~' is miswritten as '= ~' so $nokia_7210c is always true, thus
it changes all priority values into '[...]'. As a result, removal
of 'priority:0' won't happen. This causes the comparisons failed.
The reason is that the update config contains 'password', which
save passwords in keyring. Usually, it pops up a dialog which
prevents testing. Change update the property from 'password' to
'username'.
In Session.GetConfig, when checking whether a given config exists,
configs set temporarily should be concerned. Otherwise, the given
config will be treated as a non-existing one though it has been
set temporarily.
Previous implementation only applies temporary configs when the config
exists on disk. This is not expected.
Also write a test case(testGetConfigWithTempConfig) for this scenario:
first set up a config with temporary configs, and then call GetConfig
to get configs from it. This will be failed in the previous version and
successful now.
The test case will check the password and ask it from dbus clients.
The test follows the working flow to send response to dbus server.
There might be a gnome-keyring-ask process popped out for dbus server
will ask password from gnome-keyring first. To remove this problem,
a timeout function is regularly called and kill that process to force
dbus server sending request to dbus clients.
