When compiling as C++14 (the default under gcc 6), template
expansion no longer picks the right instantiation of push_back.
We have to disambiguate which of the different push_back
alternatives we want.
Fixes:
Cmdline.cpp:1734:85: error: no matching function for call to 'bind(<unresolved
overloaded function type>, const boost::reference_wrapper<std::__cxx11::list<std::__cxx11::basic_string<char> >
>, const boost::arg<1>&)'
processLUIDs(source, boost::bind(&list<string>::push_back, boost::ref(luids), _1));
The implicit conversion of boost::shared_ptr->bool is no
longer supported when compiling as C++14 (the default under
gcc 6). We have to cast explicitly.
Fixes errors like this:
Logging.h:258:41: error: cannot convert 'const boost::shared_ptr<SyncEvo::Logger>' to 'bool' in return
operator bool () const { return m_logger; }
libical v2 provides support for interoperable timezone definitions
again, so the copy of the original code that was added for libical v1
is not needed anymore and also wouldn't compile.
Therefore SyncEvolution can drop it when being compiled for v2. In that
case, a corresponding change in libsynthesis calls
icaltzutil_set_exact_vtimezones_support(0) to enable interoperable
timezones.
This addresses compilation and running with libical v2, i.e. what
normal distros are doing. Compiling with libical v1 and running
with v2 needs to be addressed separately, if possible at all.
The previous mkdir_p() walked down top to bottom and checked each path
entry as it went along. That approach failed unnecessarily when some
existing parent directory could not be read (non-readable /home, for
example).
The new version tries to create the lowest directories first
(brute-force) and only walks up when that is necessary to avoid an
ENOENT.
twisted.web.error.NoResource was replaced by
twisted.web.resource.NoResource. This has become a real problem for
example on Fedora 22 where the old name is no longer available.
As reported by Canonical, syncing fails if data items contain
text which is not correct UTF-8 in one of the fields that
SyncEvolution logs in the command line output (like SUMMARY of
a calendar event).
That is because the byte string coming from the item is passed
unchecked to the D-Bus implementation for transmission via D-Bus. But
D-Bus strings must be correct UTF-8, so depending on the D-Bus library
in use, one gets a segfault (GIO D-Bus, due to an unchecked NULL
pointer access) or an "out of memory" error (libdbus, which checks for
NULL).
What the D-Bus bindings now do is checking the string in advance (to
avoid error messages inside the D-Bus implementations) and then
replacing all invalid bytes with question marks. The rest of the
string is preserved.
Handling this inside the D-Bus binding layer is not the "correct"
solution (which would be to check for UTF-8 in the higher layers were
such bad data might get into SyncEvolution), but it is the less
intrusive and more complete one. Changing the bindings such that all
strings must be declared explicitly as "UTF-8 string" would have been
a way to find all places where such checks are missing, but it turned
out to be too complex and requiring too many changes.
Extracting a meaningful description of each item from the Synthesis
engine when updating and adding items is easy to do for items of
certain known types (contacts and calendar items) and arguably an
improvement; in particular it makes tests like
TestCmdline.testSyncOutput more realistic.
In some cases, the prefix which was supposed to be embedded
in the log messages from the target side of a local sync got
lost on the way to the command line tool.
Primarily this affected the added/updated/deleted messages, as in:
[INFO remote@client] @client/addressbook: started
[INFO remote@client] updating "Joan Doe"
[INFO remote@client] @client/addressbook: received 1/1
This was not caught by automated testing because the corresponding
test (TestCmdline.testSyncOutput and TestCmdline.testSyncOutput2) is
using the file backend, which does not generate such messages at the
moment (to be changed separately).
When sending an access token with insufficient scope (for example,
because the Ubuntu Online Accounts service definition was incomplete,
as documented in FDO #86824), Google responds with a 403 "service
denied" error.
Neon (arguably correctly) treats this as a permanent error and not
as a transient authentication error. Google should better send
a 401 error.
To activate the 401 error handling in SyncEvolution, detect this
special case and turn the general SE_ERROR error into SE_AUTH.
The Ubuntu Online Accounts signon backend complained about
invalid "username" content with an error message containing "sso",
the string used for gSSO.
Define these magic strings once and then use only the defines.
Use the information contained in the AgAuthData object to decide which
authentication method is being used.
Move the common authentication code into a private authenticate()
method.
[PO: fixed g_variant_builder_end() mem leak]
[PO: avoid issue found via cppcheck: returning std::string::c_str() in
a method returning a std::string forces a string copy, which is
inefficient.]
[PO: UiPolicy was not set because of an invalid g_variant_builder_add()
"sv" parameter.]
Instead of passing the session data and the mechanism, pass the full
AgAuthData object to the SignonAuthProvider constructor.
This object can be used to build the session data without converting all
dictionaries to and from GHashTable.
[PO: fixed g_variant_builder_end() mem leak]
[PO: ForceTokenRefresh was not set because of an invalid g_variant_builder_add()
"sv" parameter.]
It's not sync-evolution task to complete the account creation; in those
platforms where this backend is deployed (Ubuntu, Elementary, KDE) the
account should already be created with a proper SignonIdentity attached
to it.
Remove the failure count from the getOAuth2Bearer() method, and add an
invalidateCachedSecrets() method instead.
This moves the logic of how to deal with failures back to the
AuthProvider backend and simplifies the session code, which only needs
to call invalidateCachedSecrets() when the token is wrong.
This will help implementing a similar logic for the getCredentials()
method, where authentication errors could lead to requesting a new
password.
[PO: avoid issue found via cppcheck: returning std::string::c_str() in
a method returning a std::string forces a string copy, which is
inefficient.]
The libaccounts-based backends ("gsso" and "uoa") are going to use more
of libaccounts-glib, making sharing of code with the plain "signon"
plugin much harder to maintain.
Since 1.4.99.4, syncing WebDAV collections always checks first
whether there are items in the collections. This was partly done for
slow sync prevention (which is not necessary for empty collections),
partly for the "is the datastore usable" check.
However, this did not take into account that for CalDAV collections,
the entire content gets downloaded for this check. That is because
filtering by item type (VEVENT vs. VJOURNAL) is not implemented
correctly by all servers. So now all CalDAV syncs, whether incremental
or slow, always transfered all items, which is not the
intention (incremental syncs should be fast and efficient).
This commit adds a more efficient isEmpty() check: for simple CardDAV
collections, only luid and etag get transferred, as in
listAllItems(). This is the behavior from 1.5.
For CalDAV, a report with a filter for the content type is used and
the transfer gets aborted after the first item, without actually
double-checking the content of the item. This is different from
listAllItems(), which really transfers the content. This extra content
check would only be needed for some old servers (Radical 0.7) and is
not essential, because reporting "not empty" even when empty is safe.
Capture the item status and pass it to the response handler.
Response handlers are allowed to return a non-zero integer when using
the initAbortingReportParser(), which then aborts processing of the
response.
This leads to errors being returned by
ne_xml_dispatch_request(). Session::run() needs to be told if the
request was aborted, in which case all errors are ignored.
It turned out that finding databases on an Apple Calendar server accessed via
http depends on sending Basic Auth even when the server does not ask for it:
without authentication, there is no information about the current principal,
which is necessary for finding the user's databases.
To make this work again, sending the authentication header is now forced for
plain http if (and only if) the request which should have returned the
principal URL fails to include it. This implies sending the same request
twice, but as this scenario should be rare in practise (was only done for
testing), this is acceptable.
Occasionally the script shutdown got stuck on Ubuntu Vivid because killing
the background daemon failed although it was still runnning, thus causing the
wait to hang forever.
Not exactly sure what caused this. The enhancement tries to fall back to
killing the process instead of the process group (in case that there is
a race condition, which shouldn't be the case when waiting for the daemon),
preserves stderr from the kill commands and adds ps output when there is
an unexpected failure.
The test started to fail in 2015 because the VTIMEZONE generated by
the code copied from libical has a minor dependency on the current
time: it finds the transitions for the current year, then pretends
that the same rule applied since 1970. While doing that, it uses this
years transition date and just replaces the year. Because the exact
date varies between years, the result depends on the current year.
The test now simply ignores the day of the month, while still checking
year and month. Those should always be the same.
When storing a pointer with PlainGStr::reset(), the default delete []
was used to free the memory. Should have used g_free(), as in the
constructor. Affects the signon backend.
cppcheck warned about the non-portable initialization of a float
via memset. It is okay on all current architectures, but better
assign a real float 0 value anyway, while keeping the memset for
the rest of the elements.
The second memset() had already become redundant when adding the
PullParams constructor.
When the SuspendFlags::handleSignal() signal handler got called while
the program was allocating memory in libc, a deadlock occurred when
the mutex locking code in RecMutex::lock() tried to allocate the guard
instance (see backtrace below).
This could be fixed with new mutex code which gives up the guard
concept, a guard concept using C11 mechanisms (copy semantic of a
stack-allocated guard) or not locking at all in getSuspendFlags().
This commit uses the latter. It is simpler and can still be done
correctly because locking is not necessary except when the singleton
has not been allocated yet. That part gets moved into the process
startup phase via SyncContext::initMain().
Thread 1 (Thread 0x7fec389e6840 (LWP 16592)):
at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
from /lib/x86_64-linux-gnu/libc.so.6
from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
d=<optimized out>, p=0x7fec38196f60 <SyncEvo::suspendRecMutex>,
this=<synthetic pointer>)
at /usr/include/boost/smart_ptr/detail/shared_count.hpp:167
p=0x7fec38196f60 <SyncEvo::suspendRecMutex>, this=<synthetic pointer>)
at /usr/include/boost/smart_ptr/shared_ptr.hpp:363
this=<synthetic pointer>)
at /data/runtests/work/sources/syncevolution/src/syncevo/ThreadSupport.h:60
at /data/runtests/work/sources/syncevolution/src/syncevo/ThreadSupport.h:81
at /data/runtests/work/sources/syncevolution/src/syncevo/SuspendFlags.cpp:58
at /data/runtests/work/sources/syncevolution/src/syncevo/SuspendFlags.cpp:280
oldp=0x115b7d0, oldsize=80, nb=48) at malloc.c:4208
at malloc.c:3029
n_bytes=n_bytes@entry=32)
Because of the confustion around the libical 1.0 ABI, Debian ended
up renaming the package for Debian Jessie while keeping the libical.so.1
soname. Pre-built binaries from older distros work, so just adapt
the package meta data.
The class is used for non-GObject types, so we must use a
type-specific method to increase the
ref-count. intrusive_ptr_add_ref() does that automatically for us.
Calling g_object_ref() was a cut-and-paste error.
The previous commit for the problem caused a compilation error
in the PBAP backend, because Bluez uses a 64 bit type that no longer
had a type trait on 32 bit architectures.
Adding "[unsigned] long long" traits fixes that. It seems to work
also on 64 bit architectures, so I am not adding any ifdefs and just
enable it unconditionally.
size_t is defined as unsigned long on s390 in 32bit mode, which is is not the
same type as uin32_t = unsigned int, which caused a compile failure due to the
missing type trait.
The solution is to define traits for all the standard language types instead
of the fixed-size types. This removes the 64 bit types on 32 bit platforms but
adds the types that were missing on s390.
All the types that we use in the D-Bus bindings are among these standard
types, so that is okay.
This fixes a crash/memory corruption issue in the plain signon backend
as used in Tizen.
The real bug was that HashTable2Variant() creates a GHashTableCXX by
stealing a floating GVariant without actually acquiring the reference,
leading to various glib errors caused by double free and/or invalid
memory access.
The code in signon.cpp receiving the GHashTableCXX was correct, albeit
overly complicated (creating a new reference via g_variant_ref_sink()
explicitly instead of relying on GHashTableCXX semantic).
A local sync or Bluetooth sync do not need the 'username' property.
When it is set despite that, issue a warning.
Previously, the value was checked even when not needed, which
caused such syncs to fail when set to something other than a plain
username.
Useful for correlating events in the daemon with events in testing.
We need to use a process group to deliver SIGINT/SIGTERM, otherwise
we cannot be sure that we catch all processes created by the daemon.
The return code of the daemon was not checked before (accidentally?!)
and this patch does not change that. Might be fixed in the future.
The daemon log gets reused when starting daemons multiple times.
When starting a second time, the simple grep used to find the
output of the previous and did not wait properly. We need to count
matching lines and continue once a new one appears.
Multiple different wrappercheck instances are running concurrently
in the nightly testing, so better include a prefix with unique PID
in the "set -x" output to identify where it came from.
Occassionally, syncevo-dbus-server locked up after receiving
a CTRL-C. This primarily affected nightly testing, in particular (?)
on Ubuntu Lucid.
The reason was two-fold:
- indirectly allocating memory when getSuspendFlags() had to
allocated the guard instance of the mutex
- glib mutex held by some other thread when trying to do
g_main_loop_quit().
SuspendFlags already handles signals without these problems. So
instead of overriding that, let SuspendFlags signal the receival
of a signal by writing to its FD and monitor that in our main
event loop, then quit the main event loop from inside the main
thread itself.
The issue with malloc being called in niam() in syncevo-dbus-server
and deadlocking was only found after adding gdb's output of the
syncevo-dbus-server stack backtrace.
Patrick Ohly
frdsktp
Niels Ole Salscheider
Alberto Mardegan
Tino Mettler