This is a monthly bugfix release with numerous minor fixes including:
* clone: fix URL too long problem with many heads
* commands: revised documentation of 'default' and 'default-push'
* convert/cvs: skip bad tags
* convert/git: check status when reading output stream
* convert/subversion: fix default URL checker prototype
* copies: properly visit file context ancestors on working file contexts
* diffstat: use ui.plain() instead of ui.interactive()
* dirstate: fix in memory dirstate entries for 1-second race
* dispatch: don't mangle ImportError abort messages
* filemerge: use working dir parent as ancestor for backward wdir merge
* hgrc.5: describe form of config values and mention lists as well
* hgweb: fix attribute error in error response (issue2060)
* log: document the new xml style
* merge: correctly compute the flag for noexec filesystems
* mq: rewrite strip docstrings
* mq: use util.unlink instead of os.unlink and os.removedirs
* patch: don't look for headers in diff lines
* push: fix bug in warning message selection
* revlog: fix lazyparser.iter() to return all revisions (issue2137)
* schemes: fix // breakage with Python 2.6.5 (issue2111)
* static-http: allow clone -r (issue2164)
* subrepo: fix repo root path handling in svn subrepo
* subrepo: propagate and catch push failures
* templates: document missing template variables
* util: fix default termwidth() under Windows
* util: use an explicit prefix for checkexec/checklink temporary files
Whoosh is a fast, featureful full-text indexing and searching
library implemented in pure Python.
Some of Whoosh's features include:
* Pythonic API.
* Pure-Python. No compilation or binary packages needed, no
mysterious crashes.
* Fielded indexing and search.
* Fast indexing and retrieval -- much faster than any other
pure-Python solution.
* Pluggable scoring algorithm (including BM25F), text analysis,
storage, posting format, etc.
* Powerful query language parsed by pyparsing.
* Pure Python spell-checker (as far as I know, the only one).
Whoosh takes much inspiration (and sometimes translates code) from
other open-source search engines. The fundamental design is similar
to (but does not entirely work like and is not compatible with)
Lucene, but uses KinoSearch's indexing algorithm, some scoring
algorithms from Terrier, and the English morphological variation
generator from Minion.
The Easy Publish and Consume library provides an easy method to:
* publish data per HTTPS
* announce that information via DNS-SD
* find that information
* and finally consume it
You can use this library as key/value store published to the network,
using encryption, authentication and service discovery.
pycassa is a Cassandra library with the following features:
1. Auto-failover single or thread-local connections
2. A simplified version of the thrift interface
3. A method to map an existing class to a Cassandra ColumnFamily.
4. Support for SuperColumns
Changes since 0.0.8a:
- Decoder can now treat values of unknown types as opaque OctetString.
- Fix to Set/SetOf type decoder to handle uninitialized scalar SetOf
components correctly.
- API versioning mechanics retired (pyasn1.v1 -> pyasn1) what makes
it possible to zip-import pyasn1 sources (used by egg and py2exe).
- Allow any non-zero values in Boolean type BER decoder, as it's in
accordnance with the standard.
changes in sbcl-1.0.38 relative to sbcl-1.0.37:
* incompatible change: Thread names are now restricted to SIMPLE-STRINGs
like for any other thread-related datastructure, MUTEX, etc. (lp#547095)
* deprecation: the SB-QUEUE contrib was merged into the SB-CONCURRENCY
contrib module. New code should depend on SB-CONCURRENCY, not SB-QUEUE.
* deprecation: SB-THEAD:GET-MUTEX was deprecated in favor of
SB-THREAD:GRAB-MUTEX.
* new contributed module: SB-CONCURRENCY is a new contrib; it's supposed to
contain additional data structures and tools for concurrent programming;
at the moment it contains a lock-free queue, and a lock-free mailbox
implementation.
* new feature: added SB-THREAD:GRAB-MUTEX; it's like the now deprecated
GET-MUTEX but takes &key rather than &optional parameters. Also added
:TIMEOUT argument to GRAB-MUTEX on non-sb-lutex platforms like Linux and
BSD.
* new feature: added SB-THREAD:TRY-SEMAPHORE, a non-blocking variant of
SB-THREAD:WAIT-ON-SEMAPHORE.
* new feature: SB-EXT:ATOMIC-DECF has been added as a companion to
SB-EXT:ATOMIC-INCF.
* new feature: a CANCEL-DEADLINE is associated with DEADLINE-TIMEOUT
conditions to defer the deadline for forever.
* enhancement: *STANDARD-OUTPUT*, *STANDARD-INPUT*, and *ERROR-OUTPUT* are
now bivalent.
* enhancement: errors from NO-APPLICABLE-METHOD and
NO-PRIMARY-METHOD now have a RETRY restart available to retry the
generic function call.
* enhancement: SB-BSD-SOCKET improvements
** sockets and socket streams now have a more informative printed
representation based on the corresponding SOCKET-NAME and
SOCKET-PEERNAME.
** SOCKET-MAKE-STREAM once more supports the :AUTO-CLOSE option.
(lp#540413)
** SOCKET-CLOSE now accepts :ABORT argument, which is passed on to
CL:CLOSE when appropriate, and no longer disassociates the stream
from the socket if close failed. (lp#543951)
* improvements to the instrumenting profiler
** new feature: report per-function GC overhead. (thanks to John Fremlin)
** optimization: counters no longer use locks for the overflow mode.
** bug fix: whenever a profiling counter wrapped into overflow mode, it
incurred an off-by-one miscount.
* enhancement: improved MAKE-HASH-TABLE documentation (lp#543473)
* enhancement: improved DEFMETHOD pretty-printing.
* enhancement: perform range reduction when arguments are too large for
x87's transcendentals (instead of returning 0). (lp#327192)
* enhancement: eliminate some spurious TYPE-WARNINGs. Should help with
some of CL-PPCRE's macros. (lp#570079)
* enhancement: our machine code is slightly less hostile to valgrind on
x86-64.
* enhancement: up-to-date versions of NetBSD-current are supported. (Thanks
to Robert Swindells and Aleksej Saushev)
* bug fix: correct restart text for the continuable error in MAKE-PACKAGE.
* bug fix: a rare case of startup-time page table corruption.
* bug fix: a semaphore with multiple waiters and some of them unwinding due
to timeouts could be left in an inconsistent state.
* bug fix: fix typo in "Reporting Bugs" section of the manual (lp#520366)
* bug fix: misoptimization of multiplication by one in
(SB-C::FLOAT-ACCURACY 0) policies.
* bug fix: miscounts in SB-PROFILE.
* bug fix: Fix lost wakeup bug between SB-THREAD:CONDITION-WAIT and
CONDITION-NOTIFY on Linux. See threads "lost wakeup in condition-wait /
condition-notify" (Feb 2010) and "Condition-Wait, Deadline handler, waking
up itself" (March 2010) for further details.
* bug fix: allow forward FIND and POSITION on lists to elide checking :END
against length of the list if the element is found before the specified
END is reached. (thanks to Alec Berryman, lp#554385)
* bug fix: errors signalled during package graph modification no longer
block FIND-SYMBOL and FIND-PACKAGE in other threads. (lp#511072)
* bug fix: SB-POSIX build was broken when SBCL was compiled without the
:SB-DOC feature. (lp#552564)
* bug fix: SB-INTROSPECT build issues on GENGC/PPC. (lp#490490)
* bug fix: more robust runtime executable path detection. (lp#375549)
* bug fix: GCD always returns positive values. (lp#413680)
* bug fix: Converting division to multiplication by reciprocal handles
denormals.
* bug fix: We were too eager in eliding range reduction tests on x87.
The maximal magnitude is 2^63, not 2^64.
* bug fix: Transforms for TRUNCATE don't die when the result is completely
ignored anymore.
* bug fix: Maybe restore buildability on Alpha.
* bug fix: READ-BYTE isn't inline anymore, fixing weird streams failures.
(lp#569404)
* bug fix: RANDOM-STATE can be printed readably again.
* bug fix: Unreadable objects were sometimes printed like #<\nFoo>.
* bug fix: Using EQL with non-constant values of constant type (e.g. EQL
types) could result in type mismatches during compilation.
changes in sbcl-1.0.37 relative to sbcl-1.0.36:
* enhancement: Backtrace from THROW to uncaught tag on x86oids now shows
stack frame thrown from.
* enhancement: WITH-COMPILATION-UNIT :POLICY allows restricting changes to
compiler optimization qualities inside dynamic extent of its body.
* enhancement: LOAD-LOGICAL-PATHNAME-TRANSLATIONS can be used to load
translations from SYS:SITE;<HOST>.TRANSLATIONS.NEWEST (thanks to Michael
Weber)
* optimization: SLOT-VALUE and (SETF SLOT-VALUE) take advantage of
constraint propgation, allowing better compilation eg. when used to
access structures with WITH-SLOTS. (lp#520366)
* optimization: the compiler is now more aware of the type of the underlying
storage vector for multidimensional simple arrays resulting in better code
for accessing such arrays.
* optimization: passing NIL as the environment argument to TYPEP no longer
inhibits optimizing it. (lp#309788)
* optimization: more efficient register usage when handling single-float
arguments on x86-64. (thanks to Lutz Euler)
* optimization: ADJUST-ARRAY and STABLE-SORT on vectors no longer use
pre-allocated temporary vectors. (lp#496249)
* bug fix: Fix compiler error involving MAKE-ARRAY and IF forms
in :INITIAL-CONTENTS. (lp#523612)
* bug fix: FUNCTION-LAMBDA-EXPRESSION lost declarations from interpreted
functions. (lp#524707)
* bug fix: bogus style warnings from certain (SETF SLOT-VALUE) and
WITH-SLOTS usages during compilation.
* bug fix: SB-C::CLASS-INFO now prints correctly. (lp#514762)
* enhancement: Can now build with ud2 instead of int3 as trap instruction on
all x86oid platforms with :UD2-BREAKPOINTS target feature.
* bug fix: Breakpoints now work when using ud2 instead of int3 as trap
instruction (tested on x86oid linux with ud2-breakpoints).
* bug fix: slam.sh now works on win32.
* bug fix: better differences of numeric types. (lp#309124)
* bug fix: arrays declared intersection and union types can have their
upgraded element type derived. (lp#316078)
* bug fix: SB-SPROF allocation profiling for all threads failed to profile
threads started during profiling. (lp#472499)
* bug fix: SB-INTROSPECT test failure when building without SB-EVAL feature.
(lp#535658)
* bug fix: SB-CLTL2:DECLARATION-INFORMATION did not take
SB-EXT:RESTRICT-COMPILER-POLICY into account. (lp#313337)
* bug fix: Comma inside a backquoted array or structure resulted in nonsense
values instead of signaling an error. (lp#309093)
* bug fix: Spurious unused variable warning in a DEFSTRUCT edge case.
(lp#528807)
* bug fix: More consistent warnings and notes for ignored DYNAMIC-EXTENT
declarations (lp#497321)
* bug fix: FIND and POSITION on lists did not check sequence bounds properly
and failed to detect circular lists (lp#452008)
* bug fix: leakage from ~/.asdf-install into the ASDf-INSTALL contrib build
(lp#538974)
* bug fix: LOOP OF-TYPE VECTOR compile-time error. (lp#540186)
* bug fix: SIGNAL SB-SYS:INTERACTIVE-INTERRUPT before entering the debugger
due to it, so that handlers can run.
* bug fix: reparsing undefined types if they have become defined since
parsing. (lp#309128)
* bug fix: missing &REST type in a proclamation for a function with both
&REST and &KEY in lambda-list caused miscompilation (lp#458354)
* bug fix: WHO-CALLS information for source-transformed and compiler-macro
expanded calls (lp#542174)
* bug fix: more accurate WHO-MACROEXPANDS information; point into rather
than just at toplevel form.
Changes since 1.0.9:
* Mark Forwarding and DirectOnly options as being experimental.
* Don't redefine MAX if it already exists.
* Fixes for definitions under Windows.
* Ensure subnet-up/down scripts are called after HUP when necessary.
* Fix reloading Subnets when StrictSubnets is set.
* Reload Subnets when getting a HUP signal and StrictSubnets is used.
* Ensure ICMP_NET_ANO is defined.
* Convert Port to numeric form before sending it to other nodes.
If one uses a symbolic name for the Port option, tinc will send that name
literally to other nodes. However, it is not guaranteed that all nodes have
the same contents in /etc/services, or have such a file at all.
* Never delete Subnets when StrictSubnets is set
If a node is unreachable, and not connected to an edge anymore, it gets
deleted. When this happens its subnets are also removed, which should
not happen with StrictSubnets=yes.
Solution:
- do not remove subnets in src/net.c::purge(), we know that all subnets
in the list came from our hosts files.
I think here you got the check wrong by looking at the tunnelserver
code below it - with strictsubnets we still inform others but do not
remove the subnet from our data.
- do not remove nodes in net.c::purge() that still have subnets
attached.
* Log unauthorized Subnets when StrictSubnets is set.
* ConnectTo does not mean tinc does not listen for incoming connections anymore.
* Fixes for the Forwarding option.
* Add the DirectOnly option.
When this option is enabled, packets that cannot be sent directly to the destination node,
but which would have to be forwarded by an intermediate node, are dropped instead.
When combined with the IndirectData option,
packets for nodes for which we do not have a meta connection with are also dropped.
* Add the Forwarding option.
This determines if and how incoming packets that are not meant for the local
node are forwarded. It can either be off, internal (tinc forwards them itself,
as in previous versions), or kernel (packets are always sent to the TUN/TAP
device, letting the kernel sort them out).
* Add the StrictSubnets option.
When this option is enabled, tinc will not accept dynamic updates of Subnets
from other nodes, but will only use Subnets read from local host config files
to build its routing table.
* Preload all Subnets in TunnelServer mode.
This simplifies the logic in protocol_subnet.c.
* Check for dirent.h.
* Simplify reading lines from configuration files.
Instead of allocating storage for each line read, we now read into fixed-size
buffers on the stack. This fixes a case where a malformed configuration file
could crash tinc.
* Clamp MSS to miminum MTU in both directions.
Clamp MSS of both incoming and outgoing packets, and use the minimum of the
PMTU of both directions when clamping.
* Add --disable-zlib configure option
* Add --disable-lzo configure option
* Ensure peers with a meta connection always have our key.
This keeps UDP probes going, which in turn keeps NAT mappings alive.
* Update copyright notices.
* Try to set DF bit on BSDs as well.
Every operating system seems to have its own, slightly different way to disable
packet fragmentation. Emit a compiler warning when no suitable way is found.
On OpenBSD, it seems impossible to do it for IPv4.
* Immediately exchange keys when establishing a meta connection.
This in turn will trigger PMTU discovery, and ensures nodes know each others
reflexive UDP address and port.
* Determine peer's reflexive address and port when exchanging keys.
To help peers that are behind NAT connect to each other directly via UDP, they
need to know the exact external address and port that they use. Keys exchanged
between NATted peers necessarily go via a third node, which knows this address
and port, and can append this information to the keys, which is in turned used
by the peers.
Since PMTU discovery will immediately trigger UDP communication from both sides
to each other, this should allow direct communication between peers behind
full, address-restricted and port-restricted cone NAT.
* Be liberal in accepting KEY_CHANGED/REQ_KEY/ANS_KEY requests.
When we got a key request for or from a node we don't know, we disconnected the
node that forwarded us that request. However, especially in TunnelServer mode,
disconnecting does not help. We now ignore such requests, but since there is no
way of telling the original sender that the request was dropped, we now retry
sending REQ_KEY requests when we don't get an ANS_KEY back.
* Run subnet-up/down scripts for local MAC addresses as well.
* Fix subnet-up/down scripts being called with an empty SUBNET.
Commit 052ff8b2c598358d1c5febaa9f9f5fc5d384cfd3 contained a bug that causes
scripts to be called with an empty, or possibly corrupted SUBNET variable when
a Subnet is added or removed while the owner is still online. In router mode,
this normally does not happen, but in switch mode this is normal.
* Make MSS clamping configurable, but enabled by default.
It can either be set globally in tinc.conf, or per-node in host config files.
* Also clamp MSS of TCP over IPv6 packets.
* Optimise handling of select() returning <= 0.
Before, we immediately retried select() if it returned -1 and errno is EAGAIN
or EINTR, and if it returned 0 it would check for network events even if we
know there are none. Now, if -1 or 0 is returned we skip checking network
events, but we do check for timer and signal events.
* Ping nodes immediately when receiving SIGALRM.
One reason to send the ALRM signal is to let tinc immediately try to connect to
outgoing nodes, for example when PPP or DHCP configuration of the outgoing
interface finished. Conversely, when the outgoing interface goes down one can
now send this signal to let tinc quickly detect that links are down too.
* Clamp MSS of IPv4 SYN packets.
Some ISPs block the ICMP Fragmentation Needed packets that tinc sends. We
clamp the MSS of IPv4 SYN packets to prevent hosts behind those ISPs from
sending too large packets.
* Allow Port and PMTUDiscovery options in tinc.conf, always enable PMTUDiscovery by default.
* Use xstrdup() instead of xasprintf() to copy static strings.
* Allow port to be specified in Address statements.
This allows one to connect to use more than one port number to connect to
another node. The syntax is now:
Address = <hostname> [<port>]
* Do not fragment packets smaller than RFC defined minimum MTUs.
For IPv6, the minimum MTU is 1280 (RFC 2460), for IPv4 the minimum is actually
68, but this is such a low limit that it will probably hurt performance, so we
do as if it is 576 (the minimum packet size hosts should be able to handle, RFC
791). If we detect a path MTU smaller than those minima, and we have to handle
a packet that is bigger than the PMTU but smaller than those minima, we forward
them via TCP instead of fragmenting or returning ICMP packets.
* Forget addresses of unreachable nodes.
We clear the cached address used for UDP connections when a node becomes
unreachable. This also prevents host-up scripts from passing the old, cached
address from when the host becomes reachable again from a different address.
* Remove unused variable in lookup_subnet_*() functions.
* When learning MAC addresses, only check our own Subnets for previous entries.
Before it would check all addresses, and not learn an address if another node
already claimed that address. This caused fast roaming to fail, the code from
commit 6f6f426b353596edca77829c0477268fc2fc1925 was never triggered.
* Start a tinc service if it already exists.
* Fast handoff of roaming MAC addresses.
In switch mode, if a known MAC address is claimed by a second node before it
expired at the first node, it is likely that this is because a computer has
roamed from the LAN of the first node to that of the second node. To ensure
packets for that computer are routed to the second node, the first node should
delete its corresponding Subnet as soon as possible, without waiting for the
normal expiry timeout.
* Move socket error interpretation to utils.h.
* Use WSAGetLastError() to determine cause of network errors on Windows.
This reduces log spam and lets path MTU discovery work faster.
* Remove localedir leftovers.
* Use IP_DONTFRAGMENT instead of IP_MTU_DISCOVER on Windows.
This ensures the DF bit on outgoing UDP packets gets set on Windows when path
MTU discovery is enabled, reducing fragmentation.
* Forward packets to not directly reachable hosts via UDP if possible.
If MTU probing discovered a node was not reachable via UDP, packets for it were
forwarded to the next hop, but always via TCP, even if the next hop was
reachable via UDP. This is now fixed by retrying to send the packet using
send_packet() if the destination is not the same as the nexthop.
* Make maxmtu equal to minmtu when fixing the path MTU to a node.
This ensures MTU probes used to ping nodes are not too large, and prevents
restarting MTU probing unnecessarily.
* Always reply to MTU probes via UDP.
It could sometime happen that a node would return MTU probes via TCP, which
does not make a lot of sense.
* Allow UDP packets with an address different from the corresponding TCP connection.
* Use uint32_t instead of long int for connection options.
Options should have a fixed width anyway, but this also fixes a possible MinGW
compiler bug where %lx tries to print a 64 bit value, even though a long int is
only 32 bits.
* Add dummy device.
* Clarify and increase level of log message about MTU probes to unreachable nodes.
* Handle weighted Subnets in switch and hub modes.
We now handle MAC Subnets in exactly the same way as IPv4 and IPv6 Subnets.
This also fixes a problem that causes unncessary broadcasting of unicast
packets in VPNs where some daemons run 1.0.10 and some run other versions.
* Fix a possible crash when sending the HUP signal.
When the HUP signal is sent while some outgoing connections have not been made
yet, or are being retried, a NULL pointer could be dereferenced resulting in
tinc crashing. We fix this by more careful handling of outgoing_ts, and by
deleting all connections that have not been fully activated yet at the HUP
signal is received.
* Fix description of the WEIGHT environment variable.
* Include missing header.
* Remove debugging message when reading packets from a BSD device.
* Allow the cloning /dev/tap interface to be used on FreeBSD and NetBSD.
This device works like /dev/tun on Linux, automatically creating a new tap
interface when a program opens it. We now pass the actual name of the newly
created interface in $INTERFACE.
* Use MTU probes to regularly ping other nodes over UDP.
This keeps NAT mappings for UDP alive, and will also detect when a node is not
reachable via UDP anymore or if the path MTU is decreasing. Tinc will fall back
to TCP if the node has become unreachable.
If UDP communication is impossible, we stop sending probes, but we retry if it
changes its keys.
We also decouple the UDP and TCP ping mechanisms completely, to ensure tinc
properly detects failure of either method.
* Small updates to the documentation.
Mention that TCPOnly is not necessary anymore since tinc will autodetect
whether it can send via UDP or not. Also mention the WEIGHT environment
variable and the new default value (2048 bits) of RSA keys.
* Ensure that the texinfo manual can be converted to HTML.
The top node was made conditional with the @iftex command, since it should not
appear in PostScript and PDF output. However, it is still necessary for
texi2html, so we have to use @ifnottex instead.
Texi2html also complains about the use of @cindex in the copyright statement,
so we remove that.
* Revert "Raise default crypto algorithms to AES256 and SHA256."
Although it would be better to have the new defaults, only the most recent
releases of most of the platforms supported by tinc come with a version of
OpenSSL that supports SHA256. To ensure people can compile tinc and that nodes
can interact with each other, we revert the default back to Blowfish and SHA1.
* Remove code duplication when checking ADD_EDGE/DEL_EDGE messages.
* Don't disconnect clients in TunnelServer mode who send unauthorised ADD_SUBNETs.
So that we are liberal in what we accept.
* Removed last gettext function.
* Remove autogenerated files from EXTRA_DIST.
Apparently they were once necessary, but autoconf now includes them
automatically. Some of them are not used anymore, and this caused make dist to
fail.
* Update the NEWS.
* Add more authors to the copyright headers.
Git's log and blame tools were used to find out which files had significant
contributions from authors who sent in patches that were applied before we used
git.
* Drop support for localisation.
Localised messages don't make much sense for a daemon, and there is only the
Dutch translation which costs time to maintain.
* Remove checkpoint tracing.
This feature is not necessary anymore since we have tools like valgrind today
that can catch stack overflow errors before they make a backtrace in gdb
impossible.
* K&R style braces.
* Update the address of the Free Software Foundation in all copyright headers.
* Remove Ivo's old email addresses.
* Remove all occurences of $Id$.
* Update copyright information.
- Update year numbers in copyright headers.
- Add copyright information for Michael Tokarev and Florian Forster to the
copyright headers of files to which they have contributed significantly.
- Mention Michael and Florian in AUTHORS.
- Mention that tinc is GPLv3 or later if compiled with the --enable-tunemu
flag.
* Send large packets we cannot handle properly via TCP.
During the path MTU discovery phase, we might not know the maximum MTU yet, but
we do know a safe minimum. If we encounter a packet that is larger than that
the minimum, we now send it via TCP instead to ensure it arrives. We also
allow large packets that we cannot fragment or create ICMP replies for to be
sent via TCP.
* Raise default RSA key length to 2048 bits.
* Use a mutex to allow the TAP reader to process packets faster on Windows.
The TAP-Win32 device is not a socket, and select() under Windows only works
with sockets. Tinc used a separate thread to read from the TAP-Win32 device,
and passed this via a local socket to the main thread which could then select()
from it. We now use a global mutex, which is only unlocked when the main thread
is waiting for select(), to allow the TAP reader thread to process packets
directly.
* Remove extra {.
* Raise default crypto algorithms to AES256 and SHA256.
In light of the recent improvements of attacks on SHA1, the default hash
algorithm in tinc is now SHA256. At the same time, the default symmetric
encryption algorithm has been changed to AES256.
* Use access() instead of stat() for checking whether scripts exist.
* Remove dropin random() function, as it is not used anymore.
* Allow compiling for Windows XP and higher.
This allows us to use getaddrinfo(), getnameinfo() and related functions, which
allow tinc to make connections over existing IPv6 networks. These functions are
not available on Windows 2000 however. By default, support is enabled, but when
compiling for Windows 2000 the configure switch --with-windows2000 should be
used.
Since getaddrinfo() et al. are not functions but macros on Windows, we have to
use AC_CHECK_DECLS() instead of AC_CHECK_FUNCS() in configure.in.
* Also do not use drand48(), it is not available on Windows.
* Use only rand(), not random().
We used both rand() and random() in our code. Since it returns an int, we have
to use %x in our format strings instead of %lx. This fixes a crash under
Windows when cross-compiling tinc with a recent version of MinGW.
* Apparently it's impolite to ask GCC to subtract two pointers.
If two pointers do not belong to the same array, pointer subtraction gives
nonsensical results, depending on the level of optimisation and the
architecture one is compiling for. It is apparently not just subtracting the
pointer values and dividing by the size of the object, but uses some kind of
higher magic not intended for mere mortals. GCC will not warn about this at
all. Casting to void * is also a no-no, because then GCC does warn that strict
aliasing rules are being broken. The only safe way to query the ordering of two
pointers is to use the (in)equality operators.
The unsafe implementation of connection_compare() has probably caused the "old
connection_t for ... still lingering" messages. Our implementation of AVL trees
is augmented with a doubly linked list, which is normally what is traversed.
Only when deleting an old connection the tree itself is traversed.
* Remove superfluous call to avl_delete().
* Handle unicast packets larger than PMTU in switch mode.
If PMTUDiscovery is enabled, and we see a unicast packet that is larger than
the path MTU in switch mode, treat it just like we would do in router mode.
* Allow PMTUDiscovery in switch and hub modes again.
PMTUDiscovery was disabled in commit d5b56bbba56480b5565ffb38496175a7c1df60ac
because tinc did not handle packets larger than the path MTU in switch and hub
modes. We now allow it again in preparation of proper support, but default to
off.
* Put Subnet weight in a separate environment variable.
Commit 5674bba5c54c1aee3a4ac5b3aba6b3ebded91bbc introduced weighted Subnets,
but the weight was included in the SUBNET variable passed to subnet-up/down
scripts. This makes it harder to use in those scripts. The weight is now
stripped from the SUBNET variable and put in the WEIGHT variabel.
* Don't stat() on iPhone/iPod.
Grzegorz Dymarek noted that tinc segfaults at the stat() call in
execute_script() on the iPhone. We can omit the stat() call for the moment,
the subsequent call to system() will fail with just a warning.
* Add support for iPhones and recent iPods.
This is a slightly modified patch from Grzegorz Dymarek that allows tinc to use
the tunemu device, which allows tinc to be compiled for iPhones and recent
iPods. To enable support for tunemu, the --enable-tunemu option has to be used
when running the configure script.
* Another safe bitfield conversion.
* Add the GPL license to the repository.
Tinc is licensed under the GPL version 2 or later. To ensure autoconf does not
install the wrong license if COPYING is missing, we have to put the right one
in place.
* Convert bitfields to integers in a safe way.
This is commit eb391c52eed46f3f03b404553df417851fc0cb90 redone, but without the
non-standard anonymous union.
* Ensure tinc compiles with gcc -std=c99.
We use a lot of C99 features already, but also some extensions which are not in
the standard.
* UNIX signal numbers start at 1.
* Replace asprintf() by xasprintf().
* Check the return value of fscanf() when reading a PID file.
* Add xasprintf() and xvasprintf().
These functions wrap asprintf() and vasprintf(), and check the return value. If
the function failed, tinc will exit with an error message, similar to xmalloc()
and friends.
* Remove extra semicolon in my definition of setpriority()
* Always remove a node from the UDP tree before freeing it.
Valgrind caught tinc reading free'd memory during a purge(). This was caused by
first removing it from the main node tree, which will already call free_node(),
and then removing it from the UDP tree. This might cause spurious segmentation
faults.
* Change level of some debug messages, zero pointer after freeing hostname.
* Do not log errors when recvfrom() returns EAGAIN or EINTR.
Although we select() before we call recvfrom(), it sometimes happens that
select() tells us we can read but a subsequent read fails anyway. This is
harmless.
* Remove pending MTU probe events when a node's reachability status changes.
* Don't try to send MTU probes to unreachable nodes.
If there is an outstanding MTU probe event for a node which is not reachable
anymore, a UDP packet would be sent to that node, which caused a key request to
be sent to that node, which triggered a NULL pointer dereference. Probes and
other UDP packets to unreachable nodes are now dropped.
* Properly set HMAC length for incoming packets.
* try outgoing connections before chroot/drop_privs
When chrooted, we either need to force-initialize resolver
and/or nsswitch somehow (no clean way) or resolve all the
names we want before entering chroot jail. The latter
looks cleaner, easier and it is actually safe because
we still don't talk with the remote nodes there, only
initiating outgoing connections.
* cleanup setpriority thing to make it readable
* Add some const where appropriate.
* Add ProcessPriority option.
This option can be set to low, normal or high. On UNIX flavours, this changes
the nice value of the process by +10, 0 and -10 respectively. On Windows, it
sets the priority to BELOW_NORMAL_PRIORITY_CLASS, NORMAL_PRIORITY_CLASS and
HIGH_PRIORITY_CLASS respectively.
A high priority might help to reduce latency and packet loss on the VPN.
* src/net_socket.c: Bind outgoing TCP sockets to `BindToAddress'.
If a host has multiple addresses on an interface, the source address of the TCP
connection(s) was picked by the operating system while the UDP packets used a
bound socket, i. e. the source address was the address specified by the user.
This caused problems because the receiving code requires the TCP connection and
the UDP connection to originate from the same IP address.
This patch adds support for the `BindToInterface' and `BindToAddress' options
to the setup of outgoing TCP connections.
Tested with Debian Etch on x86 and Debian Lenny on x86_64.
Signed-off-by: Florian Forster <octo@verplant.org>
* src/linux/device.c: Fix segfault when running without `--net'.
If running without `--net', the (global) variable `netname' is NULL. This
creates a segmentation fault because this NULL-pointer is passed to strdup:
Program terminated with signal 11, Segmentation fault.
#0 0xb7d30463 in strlen () from /lib/tls/i686/cmov/libc.so.6
(gdb) bt
#0 0xb7d30463 in strlen () from /lib/tls/i686/cmov/libc.so.6
#1 0xb7d30175 in strdup () from /lib/tls/i686/cmov/libc.so.6
#2 0x0805bf47 in xstrdup (s=0x0) at xmalloc.c:118 <---
#3 0x0805be33 in setup_device () at device.c:66
#4 0x0805072e in setup_myself () at net_setup.c:432
#5 0x08050db2 in setup_network () at net_setup.c:536
#6 0x0805b27f in main (argc=Cannot access memory at address 0x0) at tincd.c:580
This patch fixes this by checking `netname' in `setup_device'. An alternative
would be to check for NULL-pointers in `xstrdup' and return NULL in this case.
Signed-off-by: Florian Forster <octo@verplant.org>
* tunnelserver: log which ADD_SUBNET was refused
Add some logging about refused ADD_SUBNET
(it causes subsequent client disconnect so it's
important to know which subnet was at fault).
Maybe we should just ignore it completely.
* Do not forward broadcast packets when TunnelServer is enabled.
First of all, the idea behind the TunnelServer option is to hide all other
nodes from each other, so we shouldn't forward broadcast packets from them
anyway. The other reason is that since edges from other nodes are ignored, the
calculated minimum spanning tree might not be correct, which can result in
routing loops.
* Use packet size before decompression to calculate path MTU.
Since compression can either grow or shrink a packet, the size of an MTU probe
after decompression might not reflect the real path MTU. Now we use the size
before decompression, which is independent of the compression algorithm, and
substract a safety margin such that the calculated path MTU will be safe even
for packets which grow as much as possible after compression.
* Add declaration for sockaddrcmp_noport().
* Fix ans_key exchange in recent changes
send_ans_key() was using the wrong in vs. outkeylength to
terminate the key being sent, so it was always empty.
* Use xrealloc instead of if(ptr) ptr = xmalloc().
* Fix initialisation of packet decryption context broken by commit 3308d13e7e3bf20cfeaf6f2ab17228a9820cea66.
Instead of a single, global decryption context, each node has its own context.
However, in send_ans_key(), the global context was initialised. This commit
fixes that and removes the global context completely.
Also only set status.validkey after all checks have been evaluated.
* don't log every strange packet coming to the UDP port
it's a sure way to fill up syslog. Only log those if
debug level is up to PROTOCOL
* Fix link to Mattias Nissler's tun/tap driver for MacOS/X.
* If PMTUDiscovery is not set, do not forward packets via TCP unnecessarily.
* ignore indirect edge registrations in tunnelserver mode
In tunnelserver mode we're not interested to hear about
our client edges, just like in case of subnets. Just
ignore all requests which are not about our node or the
client node.
The fix is very similar to what was done for subnets.
Note that we don't need to add the "unknown" nodes to
the list in tunnelserver mode too, so move allocation
of new nodes down the line.
* TunnelServer: Don't disconnect client on DEL_SUBNET too
Similar changes as was in 2327d3f6eb5982bcc922ff1ab1ec436ba6aeffdc
but for del_subnet_h().
Before, we vere returning false (and causing disconnect of the
client) in case of tunnelserver and the client sending DEL_SUBNET
for non-his subnet or for subnet which owner isn't in our connection
list.
After the mentioned change to add_subnet_h() that routine does not
add such indirect owners to the connection list anymore, so that
was ok (owner == NULL and we return true).
But if we too has a connection with the node about which the client
is sending DEL_SUBNET notification, say, because that client lost
connection with that other node, we'll disconnect this client from
us too, returning false for indirect DEL_SUBNET.
Fix that by allowing and ignoring indirect DEL_SUBNET in tunnelserver
mode.
Also rearranged the function a bit, to match add_subnet_h() (in
particular, syntax-check everything first, see if we've seen this
request before).
And also fix some comments.
* format 'not supported on this platform' error message
Format it in a similar way in all places, to make translation happier.
No functional changes.
* change error messages in droppriv code to match the rest
Change formatting of error messages about failed syscalls
to be the same as in other places in tincd.
Also suggest a change in "$foo not supported on this platform"
message as it's now used more than once.
* bugfix: chdir(/) after chroot
Fix the famous chdir(".") vs chdir("/") after chroot(something).
* bugfix: move mlock to after detach() so it works for child, not parent
mlock()/mlockall() are not persistent across fork(), and it's
done in parent process before daemon() which does fork(). So
basically, current --mlock does nothing useful.
Move mlock() to after detach() so it works for child process
instead of parent.
Also, check if the platform supports mlock right when processing
options (since else we'll have to die after startup, not at
startup, the error message will be in log only).
* bugfix: initialize pid (as read from pidfile) to zero
If we didn't read any number from a pid file, we'll return
an unitialized variable to the caller, and it will treat
that garbage as a pid of a process (possible to kill).
Fix that.
* Implement privilege dropping
Add two options, -R/--chroot and -U/--user=user, to chroot to the
config directory (where tinc.conf is located) and to perform
setuid to the user specified, after all the initialization is done.
What's left is handling of pid file since we can't remove it anymore.
* Rename setup_network_connections() and split out try_outgoing_connections()
In preparation of chroot/setuid operations, split out call to
try_outgoing_connections() from setup_network_connections()
(which was the last call in setup_network_connections()).
This is because dropping privileges should be done in-between
setup_network_connections() and try_outgoing_connections().
This patch renames setup_network_connections() to setup_network()
and moves call to try_outgoing_connections() into main routine.
No functional changes.
* Handle UDP packets from different and ports than advertised.
Previously, tinc used a fixed address and port for each node for UDP packet
exchange. The port was the one advertised by that node as its listening port.
However, due to NAT the port might be different. Now, tinc sends a different
session key to each node. This way, the sending node can be determined from
incoming packets by checking the MAC against all session keys. If a match is
found, the address and port for that node are updated.
* Use a simple Random Early Drop algorithm in send_tcppacket().
* Disable PMTUDiscovery in switch and hub modes.
In switch and hub modes, tinc does not generate ICMP packets in response to
packets that are larger than the path MTU. However, if PMTUDiscovery is
enabled, the IP_MTU_DISCOVER and IPV6_MTU_DISCOVER option is set on the UDP
sockets, which causes all UDP packets to be sent with the DF bit set, causing
large packets to be dropped, even if they would otherwise be routed fine.
* Update THANKS and copyright information.
* Allow weight to be assigned to Subnets.
Tinc allows multiple nodes to own the same Subnet, but did not have a sensible
way to decide which one to send packets to. Tinc also did not check the
reachability of nodes when deciding where to route packets to, so it would not
automatically fail over to a reachable node.
Tinc now assigns a weight to each Subnet. The default weight is 10, with lower
weights having higher priority. The Subnets are now internally sorted in the
same way as the kernel's routing table, and the Subnets are search linearly,
skipping those of unreachable nodes. A small cache of recently used addresses
is used to speed up the lookup functions.
* Enable PMTUDiscovery only if BOTH sides wants it.
Don't enable PMTUDiscovery if at least one side does not support it.
Before it was enabled if at least one side supported it, now both are required.
* Handle neighbor solicitation requests without link layer addresses.
Apparently FreeBSD likes to send out neighbor solicitation requests, even on a
tun interface where this is completely pointless. These requests do not have an
option header containing a link layer address, so the proxy-neighborsol code
was treating these requests as invalid. We now handle such requests, and send
back equally pointless replies, also without a link layer address. This seems
to satisfy FreeBSD.
* Allow tunnelserver to work with clients that have other peers.
In TunnelServer mode, tinc server disconnects any client if it announces
indirect subnets -- subnets that are not theirs (e.g. subnets for nodes
the CLIENT has connections now, even if those nodes are known to the server
too). Fix that by ignoring such (indirect) announces instead.
While we're at it, move check for such indirect subnet registration to
before allocating new node structure, as in TunnelServer mode we don't
really need to know that other node.
* Disable old RSA keys when generating new ones.
When generating an RSA keypair, the new public and private keys are appended to
files. However, when OpenSSL reads keys it only reads the first in a file, not
the last. Instead of printing an easily ignored warning, tinc now disables old
keys when appending new ones.
* Validate Name before using it in a filename when generating a keypair.
* Allow reading config files with CRLF endings on Unix systems.
* Remove unused definitions from net.h.
* Use a global list to track outgoing connections.
Previously an outgoing_t was maintained for each outgoing connection,
but the pointer to it was either stored in a connection_t or in an event_t.
This made it very hard to keep track of and to clean up.
Now a list is created when tinc starts and reads all the ConnectTo variables,
and which is recreated when tinc receives a HUP signal.
* Add missing cleanup functions in close_network_connections().
* Change flush_events() to expire_events().
The former function made a totally bogus shallow copy of the event_tree, called
the handler of each event and then deleted the whole tree. This should've
caused tinc to crash when an ALARM signal was sent more than once, but for some
reason it didn't. It also behaved incorrectly when a handler added a new event.
The new function just moves the expiration time of all events to the past.
* Move free()s at the end om main() to the proper destructor functions.
* Only send packets via UDP if UDP communication is possible.
When no session key is known for a node, or when it is doing PMTU discovery but
no MTU probes have returned yet, packets are sent via TCP. Some logic is added
to make sure intermediate nodes continue forwarding via TCP. The per-node
packet queue is now no longer necessary and has been removed.
* Consistently allocate device and iface variables on the heap.
This fixes a segfault when no Device has been specified and tinc exits, and it
would try to free() a static string. Thanks to Borg for spottin.
* Update documentation for git.
Changes since 0.5.6:
0.6.0
=====
- orm
- Unit of work internals have been rewritten. Units of work
with large numbers of objects interdependent objects
can now be flushed without recursion overflows
as there is no longer reliance upon recursive calls
[ticket:1081]. The number of internal structures now stays
constant for a particular session state, regardless of
how many relationships are present on mappings. The flow
of events now corresponds to a linear list of steps,
generated by the mappers and relationships based on actual
work to be done, filtered through a single topological sort
for correct ordering. Flush actions are assembled using
far fewer steps and less memory. [ticket:1742]
- Along with the UOW rewrite, this also removes an issue
introduced in 0.6beta3 regarding topological cycle detection
for units of work with long dependency cycles. We now use
an algorithm written by Guido (thanks Guido!).
- one-to-many relationships now maintain a list of positive
parent-child associations within the flush, preventing
previous parents marked as deleted from cascading a
delete or NULL foreign key set on those child objects,
despite the end-user not removing the child from the old
association. [ticket:1764]
- A collection lazy load will switch off default
eagerloading on the reverse many-to-one side, since
that loading is by definition unnecessary. [ticket:1495]
- Session.refresh() now does an equivalent expire()
on the given instance first, so that the "refresh-expire"
cascade is propagated. Previously, refresh() was
not affected in any way by the presence of "refresh-expire"
cascade. This is a change in behavior versus that
of 0.6beta2, where the "lockmode" flag passed to refresh()
would cause a version check to occur. Since the instance
is first expired, refresh() always upgrades the object
to the most recent version.
- The 'refresh-expire' cascade, when reaching a pending object,
will expunge the object if the cascade also includes
"delete-orphan", or will simply detach it otherwise.
[ticket:1754]
- id(obj) is no longer used internally within topological.py,
as the sorting functions now require hashable objects
only. [ticket:1756]
- The ORM will set the docstring of all generated descriptors
to None by default. This can be overridden using 'doc'
(or if using Sphinx, attribute docstrings work too).
- Added kw argument 'doc' to all mapper property callables
as well as Column(). Will assemble the string 'doc' as
the '__doc__' attribute on the descriptor.
- Usage of version_id_col on a backend that supports
cursor.rowcount for execute() but not executemany() now works
when a delete is issued (already worked for saves, since those
don't use executemany()). For a backend that doesn't support
cursor.rowcount at all, a warning is emitted the same
as with saves. [ticket:1761]
- The ORM now short-term caches the "compiled" form of
insert() and update() constructs when flushing lists of
objects of all the same class, thereby avoiding redundant
compilation per individual INSERT/UPDATE within an
individual flush() call.
- internal getattr(), setattr(), getcommitted() methods
on ColumnProperty, CompositeProperty, RelationshipProperty
have been underscored (i.e. are private), signature has
changed.
- engines
- The C extension now also works with DBAPIs which use custom
sequences as row (and not only tuples). [ticket:1757]
- sql
- Restored some bind-labeling logic from 0.5 which ensures
that tables with column names that overlap another column
of the form "<tablename>_<columnname>" won't produce
errors if column._label is used as a bind name during
an UPDATE. Test coverage which wasn't present in 0.5
has been added. [ticket:1755]
- somejoin.select(fold_equivalents=True) is no longer
deprecated, and will eventually be rolled into a more
comprehensive version of the feature for [ticket:1729].
- the Numeric type raises an *enormous* warning when expected
to convert floats to Decimal from a DBAPI that returns floats.
This includes SQLite, Sybase, MS-SQL. [ticket:1759]
- Fixed an error in expression typing which caused an endless
loop for expressions with two NULL types.
- Fixed bug in execution_options() feature whereby the existing
Transaction and other state information from the parent
connection would not be propagated to the sub-connection.
- Added new 'compiled_cache' execution option. A dictionary
where Compiled objects will be cached when the Connection
compiles a clause expression into a dialect- and parameter-
specific Compiled object. It is the user's responsibility to
manage the size of this dictionary, which will have keys
corresponding to the dialect, clause element, the column
names within the VALUES or SET clause of an INSERT or UPDATE,
as well as the "batch" mode for an INSERT or UPDATE statement.
- Added get_pk_constraint() to reflection.Inspector, similar
to get_primary_keys() except returns a dict that includes the
name of the constraint, for supported backends (PG so far).
[ticket:1769]
- Table.create() and Table.drop() no longer apply metadata-
level create/drop events. [ticket:1771]
- ext
- the compiler extension now allows @compiles decorators
on base classes that extend to child classes, @compiles
decorators on child classes that aren't broken by a
@compiles decorator on the base class.
- Declarative will raise an informative error message
if a non-mapped class attribute is referenced in the
string-based relationship() arguments.
- Further reworked the "mixin" logic in declarative to
additionally allow __mapper_args__ as a @classproperty
on a mixin, such as to dynamically assign polymorphic_identity.
- postgresql
- Postgresql now reflects sequence names associated with
SERIAL columns correctly, after the name of of the sequence
has been changed. Thanks to Kumar McMillan for the patch.
[ticket:1071]
- Repaired missing import in psycopg2._PGNumeric type when
unknown numeric is received.
- psycopg2/pg8000 dialects now aware of REAL[], FLOAT[],
DOUBLE_PRECISION[], NUMERIC[] return types without
raising an exception.
- Postgresql reflects the name of primary key constraints,
if one exists. [ticket:1769]
- oracle
- Now using cx_oracle output converters so that the
DBAPI returns natively the kinds of values we prefer:
- NUMBER values with positive precision + scale convert
to cx_oracle.STRING and then to Decimal. This
allows perfect precision for the Numeric type when
using cx_oracle. [ticket:1759]
- STRING/FIXED_CHAR now convert to unicode natively.
SQLAlchemy's String types then don't need to
apply any kind of conversions.
- firebird
- The functionality of result.rowcount can be disabled on a
per-engine basis by setting 'enable_rowcount=False'
on create_engine(). Normally, cursor.rowcount is called
after any UPDATE or DELETE statement unconditionally,
because the cursor is then closed and Firebird requires
an open cursor in order to get a rowcount. This
call is slightly expensive however so it can be disabled.
To re-enable on a per-execution basis, the
'enable_rowcount=True' execution option may be used.
- examples
- Updated attribute_shard.py example to use a more robust
method of searching a Query for binary expressions which
compare columns against literal values.
0.6beta3
========
- orm
- Major feature: Added new "subquery" loading capability to
relationship(). This is an eager loading option which
generates a second SELECT for each collection represented
in a query, across all parents at once. The query
re-issues the original end-user query wrapped in a subquery,
applies joins out to the target collection, and loads
all those collections fully in one result, similar to
"joined" eager loading but using all inner joins and not
re-fetching full parent rows repeatedly (as most DBAPIs seem
to do, even if columns are skipped). Subquery loading is
available at mapper config level using "lazy='subquery'" and
at the query options level using "subqueryload(props..)",
"subqueryload_all(props...)". [ticket:1675]
- To accomodate the fact that there are now two kinds of eager
loading available, the new names for eagerload() and
eagerload_all() are joinedload() and joinedload_all(). The
old names will remain as synonyms for the foreseeable future.
- The "lazy" flag on the relationship() function now accepts
a string argument for all kinds of loading: "select", "joined",
"subquery", "noload" and "dynamic", where the default is now
"select". The old values of True/
False/None still retain their usual meanings and will remain
as synonyms for the foreseeable future.
- Added with_hint() method to Query() construct. This calls
directly down to select().with_hint() and also accepts
entities as well as tables and aliases. See with_hint() in the
SQL section below. [ticket:921]
- Fixed bug in Query whereby calling q.join(prop).from_self(...).
join(prop) would fail to render the second join outside the
subquery, when joining on the same criterion as was on the
inside.
- Fixed bug in Query whereby the usage of aliased() constructs
would fail if the underlying table (but not the actual alias)
were referenced inside the subquery generated by
q.from_self() or q.select_from().
- Fixed bug which affected all eagerload() and similar options
such that "remote" eager loads, i.e. eagerloads off of a lazy
load such as query(A).options(eagerload(A.b, B.c))
wouldn't eagerload anything, but using eagerload("b.c") would
work fine.
- Query gains an add_columns(*columns) method which is a multi-
version of add_column(col). add_column(col) is future
deprecated.
- Query.join() will detect if the end result will be
"FROM A JOIN A", and will raise an error if so.
- Query.join(Cls.propname, from_joinpoint=True) will check more
carefully that "Cls" is compatible with the current joinpoint,
and act the same way as Query.join("propname", from_joinpoint=True)
in that regard.
- sql
- Added with_hint() method to select() construct. Specify
a table/alias, hint text, and optional dialect name, and
"hints" will be rendered in the appropriate place in the
statement. Works for Oracle, Sybase, MySQL. [ticket:921]
- Fixed bug introduced in 0.6beta2 where column labels would
render inside of column expressions already assigned a label.
[ticket:1747]
- postgresql
- The psycopg2 dialect will log NOTICE messages via the
"sqlalchemy.dialects.postgresql" logger name.
[ticket:877]
- the TIME and TIMESTAMP types are now availble from the
postgresql dialect directly, which add the PG-specific
argument 'precision' to both. 'precision' and
'timezone' are correctly reflected for both TIME and
TIMEZONE types. [ticket:997]
- mysql
- No longer guessing that TINYINT(1) should be BOOLEAN
when reflecting - TINYINT(1) is returned. Use Boolean/
BOOLEAN in table definition to get boolean conversion
behavior. [ticket:1752]
- oracle
- The Oracle dialect will issue VARCHAR type definitions
using character counts, i.e. VARCHAR2(50 CHAR), so that
the column is sized in terms of characters and not bytes.
Column reflection of character types will also use
ALL_TAB_COLUMNS.CHAR_LENGTH instead of
ALL_TAB_COLUMNS.DATA_LENGTH. Both of these behaviors take
effect when the server version is 9 or higher - for
version 8, the old behaviors are used. [ticket:1744]
- declarative
- Using a mixin won't break if the mixin implements an
unpredictable __getattribute__(), i.e. Zope interfaces.
[ticket:1746]
- Using @classdecorator and similar on mixins to define
__tablename__, __table_args__, etc. now works if
the method references attributes on the ultimate
subclass. [ticket:1749]
- relationships and columns with foreign keys aren't
allowed on declarative mixins, sorry. [ticket:1751]
- ext
- The sqlalchemy.orm.shard module now becomes an extension,
sqlalchemy.ext.horizontal_shard. The old import
works with a deprecation warning.
0.6beta2
========
- py3k
- Improved the installation/test setup regarding Python 3,
now that Distribute runs on Py3k. distribute_setup.py
is now included. See README.py3k for Python 3 installation/
testing instructions.
- orm
- The official name for the relation() function is now
relationship(), to eliminate confusion over the relational
algebra term. relation() however will remain available
in equal capacity for the foreseeable future. [ticket:1740]
- Added "version_id_generator" argument to Mapper, this is a
callable that, given the current value of the "version_id_col",
returns the next version number. Can be used for alternate
versioning schemes such as uuid, timestamps. [ticket:1692]
- added "lockmode" kw argument to Session.refresh(), will
pass through the string value to Query the same as
in with_lockmode(), will also do version check for a
version_id_col-enabled mapping.
- Fixed bug whereby calling query(A).join(A.bs).add_entity(B)
in a joined inheritance scenario would double-add B as a
target and produce an invalid query. [ticket:1188]
- Fixed bug in session.rollback() which involved not removing
formerly "pending" objects from the session before
re-integrating "deleted" objects, typically occured with
natural primary keys. If there was a primary key conflict
between them, the attach of the deleted would fail
internally. The formerly "pending" objects are now expunged
first. [ticket:1674]
- Removed a lot of logging that nobody really cares about,
logging that remains will respond to live changes in the
log level. No significant overhead is added. [ticket:1719]
- Fixed bug in session.merge() which prevented dict-like
collections from merging.
- session.merge() works with relations that specifically
don't include "merge" in their cascade options - the target
is ignored completely.
- session.merge() will not expire existing scalar attributes
on an existing target if the target has a value for that
attribute, even if the incoming merged doesn't have
a value for the attribute. This prevents unnecessary loads
on existing items. Will still mark the attr as expired
if the destination doesn't have the attr, though, which
fulfills some contracts of deferred cols. [ticket:1681]
- The "allow_null_pks" flag is now called "allow_partial_pks",
defaults to True, acts like it did in 0.5 again. Except,
it also is implemented within merge() such that a SELECT
won't be issued for an incoming instance with partially
NULL primary key if the flag is False. [ticket:1680]
- Fixed bug in 0.6-reworked "many-to-one" optimizations
such that a many-to-one that is against a non-primary key
column on the remote table (i.e. foreign key against a
UNIQUE column) will pull the "old" value in from the
database during a change, since if it's in the session
we will need it for proper history/backref accounting,
and we can't pull from the local identity map on a
non-primary key column. [ticket:1737]
- fixed internal error which would occur if calling has()
or similar complex expression on a single-table inheritance
relation(). [ticket:1731]
- query.one() no longer applies LIMIT to the query, this to
ensure that it fully counts all object identities present
in the result, even in the case where joins may conceal
multiple identities for two or more rows. As a bonus,
one() can now also be called with a query that issued
from_statement() to start with since it no longer modifies
the query. [ticket:1688]
- query.get() now returns None if queried for an identifier
that is present in the identity map with a different class
than the one requested, i.e. when using polymorphic loading.
[ticket:1727]
- A major fix in query.join(), when the "on" clause is an
attribute of an aliased() construct, but there is already
an existing join made out to a compatible target, query properly
joins to the right aliased() construct instead of sticking
onto the right side of the existing join. [ticket:1706]
- Slight improvement to the fix for [ticket:1362] to not issue
needless updates of the primary key column during a so-called
"row switch" operation, i.e. add + delete of two objects
with the same PK.
- Now uses sqlalchemy.orm.exc.DetachedInstanceError when an
attribute load or refresh action fails due to object
being detached from any Session. UnboundExecutionError
is specific to engines bound to sessions and statements.
- Query called in the context of an expression will render
disambiguating labels in all cases. Note that this does
not apply to the existing .statement and .subquery()
accessor/method, which still honors the .with_labels()
setting that defaults to False.
- Query.union() retains disambiguating labels within the
returned statement, thus avoiding various SQL composition
errors which can result from column name conflicts.
[ticket:1676]
- Fixed bug in attribute history that inadvertently invoked
__eq__ on mapped instances.
- Some internal streamlining of object loading grants a
small speedup for large results, estimates are around
10-15%. Gave the "state" internals a good solid
cleanup with less complexity, datamembers,
method calls, blank dictionary creates.
- Documentation clarification for query.delete()
[ticket:1689]
- Fixed cascade bug in many-to-one relation() when attribute
was set to None, introduced in r6711 (cascade deleted
items into session during add()).
- Calling query.order_by() or query.distinct() before calling
query.select_from(), query.with_polymorphic(), or
query.from_statement() raises an exception now instead of
silently dropping those criterion. [ticket:1736]
- query.scalar() now raises an exception if more than one
row is returned. All other behavior remains the same.
[ticket:1735]
- Fixed bug which caused "row switch" logic, that is an
INSERT and DELETE replaced by an UPDATE, to fail when
version_id_col was in use. [ticket:1692]
- sql
- join() will now simulate a NATURAL JOIN by default. Meaning,
if the left side is a join, it will attempt to join the right
side to the rightmost side of the left first, and not raise
any exceptions about ambiguous join conditions if successful
even if there are further join targets across the rest of
the left. [ticket:1714]
- The most common result processors conversion function were
moved to the new "processors" module. Dialect authors are
encouraged to use those functions whenever they correspond
to their needs instead of implementing custom ones.
- SchemaType and subclasses Boolean, Enum are now serializable,
including their ddl listener and other event callables.
[ticket:1694] [ticket:1698]
- Some platforms will now interpret certain literal values
as non-bind parameters, rendered literally into the SQL
statement. This to support strict SQL-92 rules that are
enforced by some platforms including MS-SQL and Sybase.
In this model, bind parameters aren't allowed in the
columns clause of a SELECT, nor are certain ambiguous
expressions like "?=?". When this mode is enabled, the base
compiler will render the binds as inline literals, but only across
strings and numeric values. Other types such as dates
will raise an error, unless the dialect subclass defines
a literal rendering function for those. The bind parameter
must have an embedded literal value already or an error
is raised (i.e. won't work with straight bindparam('x')).
Dialects can also expand upon the areas where binds are not
accepted, such as within argument lists of functions
(which don't work on MS-SQL when native SQL binding is used).
- Added "unicode_errors" parameter to String, Unicode, etc.
Behaves like the 'errors' keyword argument to
the standard library's string.decode() functions. This flag
requires that `convert_unicode` is set to `"force"` - otherwise,
SQLAlchemy is not guaranteed to handle the task of unicode
conversion. Note that this flag adds significant performance
overhead to row-fetching operations for backends that already
return unicode objects natively (which most DBAPIs do). This
flag should only be used as an absolute last resort for reading
strings from a column with varied or corrupted encodings,
which only applies to databases that accept invalid encodings
in the first place (i.e. MySQL. *not* PG, Sqlite, etc.)
- Added math negation operator support, -x.
- FunctionElement subclasses are now directly executable the
same way any func.foo() construct is, with automatic
SELECT being applied when passed to execute().
- The "type" and "bind" keyword arguments of a func.foo()
construct are now local to "func." constructs and are
not part of the FunctionElement base class, allowing
a "type" to be handled in a custom constructor or
class-level variable.
- Restored the keys() method to ResultProxy.
- The type/expression system now does a more complete job
of determining the return type from an expression
as well as the adaptation of the Python operator into
a SQL operator, based on the full left/right/operator
of the given expression. In particular
the date/time/interval system created for Postgresql
EXTRACT in [ticket:1647] has now been generalized into
the type system. The previous behavior which often
occured of an expression "column + literal" forcing
the type of "literal" to be the same as that of "column"
will now usually not occur - the type of
"literal" is first derived from the Python type of the
literal, assuming standard native Python types + date
types, before falling back to that of the known type
on the other side of the expression. If the
"fallback" type is compatible (i.e. CHAR from String),
the literal side will use that. TypeDecorator
types override this by default to coerce the "literal"
side unconditionally, which can be changed by implementing
the coerce_compared_value() method. Also part of
[ticket:1683].
- Made sqlalchemy.sql.expressions.Executable part of public
API, used for any expression construct that can be sent to
execute(). FunctionElement now inherits Executable so that
it gains execution_options(), which are also propagated
to the select() that's generated within execute().
Executable in turn subclasses _Generative which marks
any ClauseElement that supports the @_generative
decorator - these may also become "public" for the benefit
of the compiler extension at some point.
- A change to the solution for [ticket:1579] - an end-user
defined bind parameter name that directly conflicts with
a column-named bind generated directly from the SET or
VALUES clause of an update/insert generates a compile error.
This reduces call counts and eliminates some cases where
undesirable name conflicts could still occur.
- Column() requires a type if it has no foreign keys (this is
not new). An error is now raised if a Column() has no type
and no foreign keys. [ticket:1705]
- the "scale" argument of the Numeric() type is honored when
coercing a returned floating point value into a string
on its way to Decimal - this allows accuracy to function
on SQLite, MySQL. [ticket:1717]
- the copy() method of Column now copies over uninitialized
"on table attach" events. Helps with the new declarative
"mixin" capability.
- engines
- Added an optional C extension to speed up the sql layer by
reimplementing RowProxy and the most common result processors.
The actual speedups will depend heavily on your DBAPI and
the mix of datatypes used in your tables, and can vary from
a 30% improvement to more than 200%. It also provides a modest
(~15-20%) indirect improvement to ORM speed for large queries.
Note that it is *not* built/installed by default.
See README for installation instructions.
- the execution sequence pulls all rowcount/last inserted ID
info from the cursor before commit() is called on the
DBAPI connection in an "autocommit" scenario. This helps
mxodbc with rowcount and is probably a good idea overall.
- Opened up logging a bit such that isEnabledFor() is called
more often, so that changes to the log level for engine/pool
will be reflected on next connect. This adds a small
amount of method call overhead. It's negligible and will make
life a lot easier for all those situations when logging
just happens to be configured after create_engine() is called.
[ticket:1719]
- The assert_unicode flag is deprecated. SQLAlchemy will raise
a warning in all cases where it is asked to encode a non-unicode
Python string, as well as when a Unicode or UnicodeType type
is explicitly passed a bytestring. The String type will do nothing
for DBAPIs that already accept Python unicode objects.
- Bind parameters are sent as a tuple instead of a list. Some
backend drivers will not accept bind parameters as a list.
- threadlocal engine wasn't properly closing the connection
upon close() - fixed that.
- Transaction object doesn't rollback or commit if it isn't
"active", allows more accurate nesting of begin/rollback/commit.
- Python unicode objects as binds result in the Unicode type,
not string, thus eliminating a certain class of unicode errors
on drivers that don't support unicode binds.
- Added "logging_name" argument to create_engine(), Pool() constructor
as well as "pool_logging_name" argument to create_engine() which
filters down to that of Pool. Issues the given string name
within the "name" field of logging messages instead of the default
hex identifier string. [ticket:1555]
- The visit_pool() method of Dialect is removed, and replaced with
on_connect(). This method returns a callable which receives
the raw DBAPI connection after each one is created. The callable
is assembled into a first_connect/connect pool listener by the
connection strategy if non-None. Provides a simpler interface
for dialects.
- StaticPool now initializes, disposes and recreates without
opening a new connection - the connection is only opened when
first requested. dispose() also works on AssertionPool now.
[ticket:1728]
- metadata
- Added the ability to strip schema information when using
"tometadata" by passing "schema=None" as an argument. If schema
is not specified then the table's schema is retained.
[ticket: 1673]
- declarative
- DeclarativeMeta exclusively uses cls.__dict__ (not dict_)
as the source of class information; _as_declarative exclusively
uses the dict_ passed to it as the source of class information
(which when using DeclarativeMeta is cls.__dict__). This should
in theory make it easier for custom metaclasses to modify
the state passed into _as_declarative.
- declarative now accepts mixin classes directly, as a means
to provide common functional and column-based elements on
all subclasses, as well as a means to propagate a fixed
set of __table_args__ or __mapper_args__ to subclasses.
For custom combinations of __table_args__/__mapper_args__ from
an inherited mixin to local, descriptors can now be used.
New details are all up in the Declarative documentation.
Thanks to Chris Withers for putting up with my strife
on this. [ticket:1707]
- the __mapper_args__ dict is copied when propagating to a subclass,
and is taken straight off the class __dict__ to avoid any
propagation from the parent. mapper inheritance already
propagates the things you want from the parent mapper.
[ticket:1393]
- An exception is raised when a single-table subclass specifies
a column that is already present on the base class.
[ticket:1732]
- mysql
- Fixed reflection bug whereby when COLLATE was present,
nullable flag and server defaults would not be reflected.
[ticket:1655]
- Fixed reflection of TINYINT(1) "boolean" columns defined with
integer flags like UNSIGNED.
- Further fixes for the mysql-connector dialect. [ticket:1668]
- Composite PK table on InnoDB where the "autoincrement" column
isn't first will emit an explicit "KEY" phrase within
CREATE TABLE thereby avoiding errors, [ticket:1496]
- Added reflection/create table support for a wide range
of MySQL keywords. [ticket:1634]
- Fixed import error which could occur reflecting tables on
a Windows host [ticket:1580]
- mssql
- Re-established support for the pymssql dialect.
- Various fixes for implicit returning, reflection,
etc. - the MS-SQL dialects aren't quite complete
in 0.6 yet (but are close)
- Added basic support for mxODBC [ticket:1710].
- Removed the text_as_varchar option.
- oracle
- "out" parameters require a type that is supported by
cx_oracle. An error will be raised if no cx_oracle
type can be found.
- Oracle 'DATE' now does not perform any result processing,
as the DATE type in Oracle stores full date+time objects,
that's what you'll get. Note that the generic types.Date
type *will* still call value.date() on incoming values,
however. When reflecting a table, the reflected type
will be 'DATE'.
- Added preliminary support for Oracle's WITH_UNICODE
mode. At the very least this establishes initial
support for cx_Oracle with Python 3. When WITH_UNICODE
mode is used in Python 2.xx, a large and scary warning
is emitted asking that the user seriously consider
the usage of this difficult mode of operation.
[ticket:1670]
- The except_() method now renders as MINUS on Oracle,
which is more or less equivalent on that platform.
[ticket:1712]
- Added support for rendering and reflecting
TIMESTAMP WITH TIME ZONE, i.e. TIMESTAMP(timezone=True).
[ticket:651]
- Oracle INTERVAL type can now be reflected.
- sqlite
- Added "native_datetime=True" flag to create_engine().
This will cause the DATE and TIMESTAMP types to skip
all bind parameter and result row processing, under
the assumption that PARSE_DECLTYPES has been enabled
on the connection. Note that this is not entirely
compatible with the "func.current_date()", which
will be returned as a string. [ticket:1685]
- sybase
- Implemented a preliminary working dialect for Sybase,
with sub-implementations for Python-Sybase as well
as Pyodbc. Handles table
creates/drops and basic round trip functionality.
Does not yet include reflection or comprehensive
support of unicode/special expressions/etc.
- examples
- Changed the beaker cache example a bit to have a separate
RelationCache option for lazyload caching. This object
does a lookup among any number of potential attributes
more efficiently by grouping several into a common structure.
Both FromCache and RelationCache are simpler individually.
- documentation
- Major cleanup work in the docs to link class, function, and
method names into the API docs. [ticket:1700/1702/1703]
0.6beta1
========
- Major Release
- For the full set of feature descriptions, see
http://www.sqlalchemy.org/trac/wiki/06Migration .
This document is a work in progress.
- All bug fixes and feature enhancements from the most
recent 0.5 version and below are also included within 0.6.
- Platforms targeted now include Python 2.4/2.5/2.6, Python
3.1, Jython2.5.
- orm
- Changes to query.update() and query.delete():
- the 'expire' option on query.update() has been renamed to
'fetch', thus matching that of query.delete().
'expire' is deprecated and issues a warning.
- query.update() and query.delete() both default to
'evaluate' for the synchronize strategy.
- the 'synchronize' strategy for update() and delete()
raises an error on failure. There is no implicit fallback
onto "fetch". Failure of evaluation is based on the
structure of criteria, so success/failure is deterministic
based on code structure.
- Enhancements on many-to-one relations:
- many-to-one relations now fire off a lazyload in fewer
cases, including in most cases will not fetch the "old"
value when a new one is replaced.
- many-to-one relation to a joined-table subclass now uses
get() for a simple load (known as the "use_get"
condition), i.e. Related->Sub(Base), without the need to
redefine the primaryjoin condition in terms of the base
table. [ticket:1186]
- specifying a foreign key with a declarative column, i.e.
ForeignKey(MyRelatedClass.id) doesn't break the "use_get"
condition from taking place [ticket:1492]
- relation(), eagerload(), and eagerload_all() now feature
an option called "innerjoin". Specify `True` or `False` to
control whether an eager join is constructed as an INNER
or OUTER join. Default is `False` as always. The mapper
options will override whichever setting is specified on
relation(). Should generally be set for many-to-one, not
nullable foreign key relations to allow improved join
performance. [ticket:1544]
- the behavior of eagerloading such that the main query is
wrapped in a subquery when LIMIT/OFFSET are present now
makes an exception for the case when all eager loads are
many-to-one joins. In those cases, the eager joins are
against the parent table directly along with the
limit/offset without the extra overhead of a subquery,
since a many-to-one join does not add rows to the result.
- Enhancements / Changes on Session.merge():
- the "dont_load=True" flag on Session.merge() is deprecated
and is now "load=False".
- Session.merge() is performance optimized, using half the
call counts for "load=False" mode compared to 0.5 and
significantly fewer SQL queries in the case of collections
for "load=True" mode.
- merge() will not issue a needless merge of attributes if the
given instance is the same instance which is already present.
- merge() now also merges the "options" associated with a given
state, i.e. those passed through query.options() which follow
along with an instance, such as options to eagerly- or
lazyily- load various attributes. This is essential for
the construction of highly integrated caching schemes. This
is a subtle behavioral change vs. 0.5.
- A bug was fixed regarding the serialization of the "loader
path" present on an instance's state, which is also necessary
when combining the usage of merge() with serialized state
and associated options that should be preserved.
- The all new merge() is showcased in a new comprehensive
example of how to integrate Beaker with SQLAlchemy. See
the notes in the "examples" note below.
- Primary key values can now be changed on a joined-table inheritance
object, and ON UPDATE CASCADE will be taken into account when
the flush happens. Set the new "passive_updates" flag to False
on mapper() when using SQLite or MySQL/MyISAM. [ticket:1362]
- flush() now detects when a primary key column was updated by
an ON UPDATE CASCADE operation from another primary key, and
can then locate the row for a subsequent UPDATE on the new PK
value. This occurs when a relation() is there to establish
the relationship as well as passive_updates=True. [ticket:1671]
- the "save-update" cascade will now cascade the pending *removed*
values from a scalar or collection attribute into the new session
during an add() operation. This so that the flush() operation
will also delete or modify rows of those disconnected items.
- Using a "dynamic" loader with a "secondary" table now produces
a query where the "secondary" table is *not* aliased. This
allows the secondary Table object to be used in the "order_by"
attribute of the relation(), and also allows it to be used
in filter criterion against the dynamic relation.
[ticket:1531]
- relation() with uselist=False will emit a warning when
an eager or lazy load locates more than one valid value for
the row. This may be due to primaryjoin/secondaryjoin
conditions which aren't appropriate for an eager LEFT OUTER
JOIN or for other conditions. [ticket:1643]
- an explicit check occurs when a synonym() is used with
map_column=True, when a ColumnProperty (deferred or otherwise)
exists separately in the properties dictionary sent to mapper
with the same keyname. Instead of silently replacing
the existing property (and possible options on that property),
an error is raised. [ticket:1633]
- a "dynamic" loader sets up its query criterion at construction
time so that the actual query is returned from non-cloning
accessors like "statement".
- the "named tuple" objects returned when iterating a
Query() are now pickleable.
- mapping to a select() construct now requires that you
make an alias() out of it distinctly. This to eliminate
confusion over such issues as [ticket:1542]
- query.join() has been reworked to provide more consistent
behavior and more flexibility (includes [ticket:1537])
- query.select_from() accepts multiple clauses to produce
multiple comma separated entries within the FROM clause.
Useful when selecting from multiple-homed join() clauses.
- query.select_from() also accepts mapped classes, aliased()
constructs, and mappers as arguments. In particular this
helps when querying from multiple joined-table classes to ensure
the full join gets rendered.
- query.get() can be used with a mapping to an outer join
where one or more of the primary key values are None.
[ticket:1135]
- query.from_self(), query.union(), others which do a
"SELECT * from (SELECT...)" type of nesting will do
a better job translating column expressions within the subquery
to the columns clause of the outer query. This is
potentially backwards incompatible with 0.5, in that this
may break queries with literal expressions that do not have labels
applied (i.e. literal('foo'), etc.)
[ticket:1568]
- relation primaryjoin and secondaryjoin now check that they
are column-expressions, not just clause elements. this prohibits
things like FROM expressions being placed there directly.
[ticket:1622]
- `expression.null()` is fully understood the same way
None is when comparing an object/collection-referencing
attribute within query.filter(), filter_by(), etc.
[ticket:1415]
- added "make_transient()" helper function which transforms a
persistent/ detached instance into a transient one (i.e.
deletes the instance_key and removes from any session.)
[ticket:1052]
- the allow_null_pks flag on mapper() is deprecated, and
the feature is turned "on" by default. This means that
a row which has a non-null value for any of its primary key
columns will be considered an identity. The need for this
scenario typically only occurs when mapping to an outer join.
[ticket:1339]
- the mechanics of "backref" have been fully merged into the
finer grained "back_populates" system, and take place entirely
within the _generate_backref() method of RelationProperty. This
makes the initialization procedure of RelationProperty
simpler and allows easier propagation of settings (such as from
subclasses of RelationProperty) into the reverse reference.
The internal BackRef() is gone and backref() returns a plain
tuple that is understood by RelationProperty.
- The version_id_col feature on mapper() will raise a warning when
used with dialects that don't support "rowcount" adequately.
[ticket:1569]
- added "execution_options()" to Query, to so options can be
passed to the resulting statement. Currently only
Select-statements have these options, and the only option
used is "stream_results", and the only dialect which knows
"stream_results" is psycopg2.
- Query.yield_per() will set the "stream_results" statement
option automatically.
- Deprecated or removed:
* 'allow_null_pks' flag on mapper() is deprecated. It does
nothing now and the setting is "on" in all cases.
* 'transactional' flag on sessionmaker() and others is
removed. Use 'autocommit=True' to indicate 'transactional=False'.
* 'polymorphic_fetch' argument on mapper() is removed.
Loading can be controlled using the 'with_polymorphic'
option.
* 'select_table' argument on mapper() is removed. Use
'with_polymorphic=("*", <some selectable>)' for this
functionality.
* 'proxy' argument on synonym() is removed. This flag
did nothing throughout 0.5, as the "proxy generation"
behavior is now automatic.
* Passing a single list of elements to eagerload(),
eagerload_all(), contains_eager(), lazyload(),
defer(), and undefer() instead of multiple positional
*args is deprecated.
* Passing a single list of elements to query.order_by(),
query.group_by(), query.join(), or query.outerjoin()
instead of multiple positional *args is deprecated.
* query.iterate_instances() is removed. Use query.instances().
* Query.query_from_parent() is removed. Use the
sqlalchemy.orm.with_parent() function to produce a
"parent" clause, or alternatively query.with_parent().
* query._from_self() is removed, use query.from_self()
instead.
* the "comparator" argument to composite() is removed.
Use "comparator_factory".
* RelationProperty._get_join() is removed.
* the 'echo_uow' flag on Session is removed. Use
logging on the "sqlalchemy.orm.unitofwork" name.
* session.clear() is removed. use session.expunge_all().
* session.save(), session.update(), session.save_or_update()
are removed. Use session.add() and session.add_all().
* the "objects" flag on session.flush() remains deprecated.
* the "dont_load=True" flag on session.merge() is deprecated
in favor of "load=False".
* ScopedSession.mapper remains deprecated. See the
usage recipe at
http://www.sqlalchemy.org/trac/wiki/UsageRecipes/SessionAwareMapper
* passing an InstanceState (internal SQLAlchemy state object) to
attributes.init_collection() or attributes.get_history() is
deprecated. These functions are public API and normally
expect a regular mapped object instance.
* the 'engine' parameter to declarative_base() is removed.
Use the 'bind' keyword argument.
- sql
- the "autocommit" flag on select() and text() as well
as select().autocommit() are deprecated - now call
.execution_options(autocommit=True) on either of those
constructs, also available directly on Connection and orm.Query.
- the autoincrement flag on column now indicates the column
which should be linked to cursor.lastrowid, if that method
is used. See the API docs for details.
- an executemany() now requires that all bound parameter
sets require that all keys are present which are
present in the first bound parameter set. The structure
and behavior of an insert/update statement is very much
determined by the first parameter set, including which
defaults are going to fire off, and a minimum of
guesswork is performed with all the rest so that performance
is not impacted. For this reason defaults would otherwise
silently "fail" for missing parameters, so this is now guarded
against. [ticket:1566]
- returning() support is native to insert(), update(),
delete(). Implementations of varying levels of
functionality exist for Postgresql, Firebird, MSSQL and
Oracle. returning() can be called explicitly with column
expressions which are then returned in the resultset,
usually via fetchone() or first().
insert() constructs will also use RETURNING implicitly to
get newly generated primary key values, if the database
version in use supports it (a version number check is
performed). This occurs if no end-user returning() was
specified.
- union(), intersect(), except() and other "compound" types
of statements have more consistent behavior w.r.t.
parenthesizing. Each compound element embedded within
another will now be grouped with parenthesis - previously,
the first compound element in the list would not be grouped,
as SQLite doesn't like a statement to start with
parenthesis. However, Postgresql in particular has
precedence rules regarding INTERSECT, and it is
more consistent for parenthesis to be applied equally
to all sub-elements. So now, the workaround for SQLite
is also what the workaround for PG was previously -
when nesting compound elements, the first one usually needs
".alias().select()" called on it to wrap it inside
of a subquery. [ticket:1665]
- insert() and update() constructs can now embed bindparam()
objects using names that match the keys of columns. These
bind parameters will circumvent the usual route to those
keys showing up in the VALUES or SET clause of the generated
SQL. [ticket:1579]
- the Binary type now returns data as a Python string
(or a "bytes" type in Python 3), instead of the built-
in "buffer" type. This allows symmetric round trips
of binary data. [ticket:1524]
- Added a tuple_() construct, allows sets of expressions
to be compared to another set, typically with IN against
composite primary keys or similar. Also accepts an
IN with multiple columns. The "scalar select can
have only one column" error message is removed - will
rely upon the database to report problems with
col mismatch.
- User-defined "default" and "onupdate" callables which
accept a context should now call upon
"context.current_parameters" to get at the dictionary
of bind parameters currently being processed. This
dict is available in the same way regardless of
single-execute or executemany-style statement execution.
- multi-part schema names, i.e. with dots such as
"dbo.master", are now rendered in select() labels
with underscores for dots, i.e. "dbo_master_table_column".
This is a "friendly" label that behaves better
in result sets. [ticket:1428]
- removed needless "counter" behavior with select()
labelnames that match a column name in the table,
i.e. generates "tablename_id" for "id", instead of
"tablename_id_1" in an attempt to avoid naming
conflicts, when the table has a column actually
named "tablename_id" - this is because
the labeling logic is always applied to all columns
so a naming conflict will never occur.
- calling expr.in_([]), i.e. with an empty list, emits a warning
before issuing the usual "expr != expr" clause. The
"expr != expr" can be very expensive, and it's preferred
that the user not issue in_() if the list is empty,
instead simply not querying, or modifying the criterion
as appropriate for more complex situations.
[ticket:1628]
- Added "execution_options()" to select()/text(), which set the
default options for the Connection. See the note in "engines".
- Deprecated or removed:
* "scalar" flag on select() is removed, use
select.as_scalar().
* "shortname" attribute on bindparam() is removed.
* postgres_returning, firebird_returning flags on
insert(), update(), delete() are deprecated, use
the new returning() method.
* fold_equivalents flag on join is deprecated (will remain
until [ticket:1131] is implemented)
- engines
- transaction isolation level may be specified with
create_engine(... isolation_level="..."); available on
postgresql and sqlite. [ticket:443]
- Connection has execution_options(), generative method
which accepts keywords that affect how the statement
is executed w.r.t. the DBAPI. Currently supports
"stream_results", causes psycopg2 to use a server
side cursor for that statement, as well as
"autocommit", which is the new location for the "autocommit"
option from select() and text(). select() and
text() also have .execution_options() as well as
ORM Query().
- fixed the import for entrypoint-driven dialects to
not rely upon silly tb_info trick to determine import
error status. [ticket:1630]
- added first() method to ResultProxy, returns first row and
closes result set immediately.
- RowProxy objects are now pickleable, i.e. the object returned
by result.fetchone(), result.fetchall() etc.
- RowProxy no longer has a close() method, as the row no longer
maintains a reference to the parent. Call close() on
the parent ResultProxy instead, or use autoclose.
- ResultProxy internals have been overhauled to greatly reduce
method call counts when fetching columns. Can provide a large
speed improvement (up to more than 100%) when fetching large
result sets. The improvement is larger when fetching columns
that have no type-level processing applied and when using
results as tuples (instead of as dictionaries). Many
thanks to Elixir's Gaëtan de Menten for this dramatic
improvement ! [ticket:1586]
- Databases which rely upon postfetch of "last inserted id"
to get at a generated sequence value (i.e. MySQL, MS-SQL)
now work correctly when there is a composite primary key
where the "autoincrement" column is not the first primary
key column in the table.
- the last_inserted_ids() method has been renamed to the
descriptor "inserted_primary_key".
- setting echo=False on create_engine() now sets the loglevel
to WARN instead of NOTSET. This so that logging can be
disabled for a particular engine even if logging
for "sqlalchemy.engine" is enabled overall. Note that the
default setting of "echo" is `None`. [ticket:1554]
- ConnectionProxy now has wrapper methods for all transaction
lifecycle events, including begin(), rollback(), commit()
begin_nested(), begin_prepared(), prepare(), release_savepoint(),
etc.
- Connection pool logging now uses both INFO and DEBUG
log levels for logging. INFO is for major events such
as invalidated connections, DEBUG for all the acquire/return
logging. `echo_pool` can be False, None, True or "debug"
the same way as `echo` works.
- All pyodbc-dialects now support extra pyodbc-specific
kw arguments 'ansi', 'unicode_results', 'autocommit'.
[ticket:1621]
- the "threadlocal" engine has been rewritten and simplified
and now supports SAVEPOINT operations.
- deprecated or removed
* result.last_inserted_ids() is deprecated. Use
result.inserted_primary_key
* dialect.get_default_schema_name(connection) is now
public via dialect.default_schema_name.
* the "connection" argument from engine.transaction() and
engine.run_callable() is removed - Connection itself
now has those methods. All four methods accept
*args and **kwargs which are passed to the given callable,
as well as the operating connection.
- schema
- the `__contains__()` method of `MetaData` now accepts
strings or `Table` objects as arguments. If given
a `Table`, the argument is converted to `table.key` first,
i.e. "[schemaname.]<tablename>" [ticket:1541]
- deprecated MetaData.connect() and
ThreadLocalMetaData.connect() have been removed - send
the "bind" attribute to bind a metadata.
- deprecated metadata.table_iterator() method removed (use
sorted_tables)
- deprecated PassiveDefault - use DefaultClause.
- the "metadata" argument is removed from DefaultGenerator
and subclasses, but remains locally present on Sequence,
which is a standalone construct in DDL.
- Removed public mutability from Index and Constraint
objects:
- ForeignKeyConstraint.append_element()
- Index.append_column()
- UniqueConstraint.append_column()
- PrimaryKeyConstraint.add()
- PrimaryKeyConstraint.remove()
These should be constructed declaratively (i.e. in one
construction).
- The "start" and "increment" attributes on Sequence now
generate "START WITH" and "INCREMENT BY" by default,
on Oracle and Postgresql. Firebird doesn't support
these keywords right now. [ticket:1545]
- UniqueConstraint, Index, PrimaryKeyConstraint all accept
lists of column names or column objects as arguments.
- Other removed things:
- Table.key (no idea what this was for)
- Table.primary_key is not assignable - use
table.append_constraint(PrimaryKeyConstraint(...))
- Column.bind (get via column.table.bind)
- Column.metadata (get via column.table.metadata)
- Column.sequence (use column.default)
- ForeignKey(constraint=some_parent) (is now private _constraint)
- The use_alter flag on ForeignKey is now a shortcut option
for operations that can be hand-constructed using the
DDL() event system. A side effect of this refactor is
that ForeignKeyConstraint objects with use_alter=True
will *not* be emitted on SQLite, which does not support
ALTER for foreign keys.
- ForeignKey and ForeignKeyConstraint objects now correctly
copy() all their public keyword arguments. [ticket:1605]
- Reflection/Inspection
- Table reflection has been expanded and generalized into
a new API called "sqlalchemy.engine.reflection.Inspector".
The Inspector object provides fine-grained information about
a wide variety of schema information, with room for expansion,
including table names, column names, view definitions, sequences,
indexes, etc.
- Views are now reflectable as ordinary Table objects. The same
Table constructor is used, with the caveat that "effective"
primary and foreign key constraints aren't part of the reflection
results; these have to be specified explicitly if desired.
- The existing autoload=True system now uses Inspector underneath
so that each dialect need only return "raw" data about tables
and other objects - Inspector is the single place that information
is compiled into Table objects so that consistency is at a maximum.
- DDL
- the DDL system has been greatly expanded. the DDL() class
now extends the more generic DDLElement(), which forms the basis
of many new constructs:
- CreateTable()
- DropTable()
- AddConstraint()
- DropConstraint()
- CreateIndex()
- DropIndex()
- CreateSequence()
- DropSequence()
These support "on" and "execute-at()" just like plain DDL()
does. User-defined DDLElement subclasses can be created and
linked to a compiler using the sqlalchemy.ext.compiler extension.
- The signature of the "on" callable passed to DDL() and
DDLElement() is revised as follows:
"ddl" - the DDLElement object itself.
"event" - the string event name.
"target" - previously "schema_item", the Table or
MetaData object triggering the event.
"connection" - the Connection object in use for the operation.
**kw - keyword arguments. In the case of MetaData before/after
create/drop, the list of Table objects for which
CREATE/DROP DDL is to be issued is passed as the kw
argument "tables". This is necessary for metadata-level
DDL that is dependent on the presence of specific tables.
- the "schema_item" attribute of DDL has been renamed to
"target".
- dialect refactor
- Dialect modules are now broken into database dialects
plus DBAPI implementations. Connect URLs are now
preferred to be specified using dialect+driver://...,
i.e. "mysql+mysqldb://scott:tiger@localhost/test". See
the 0.6 documentation for examples.
- the setuptools entrypoint for external dialects is now
called "sqlalchemy.dialects".
- the "owner" keyword argument is removed from Table. Use
"schema" to represent any namespaces to be prepended to
the table name.
- server_version_info becomes a static attribute.
- dialects receive an initialize() event on initial
connection to determine connection properties.
- dialects receive a visit_pool event have an opportunity
to establish pool listeners.
- cached TypeEngine classes are cached per-dialect class
instead of per-dialect.
- new UserDefinedType should be used as a base class for
new types, which preserves the 0.5 behavior of
get_col_spec().
- The result_processor() method of all type classes now
accepts a second argument "coltype", which is the DBAPI
type argument from cursor.description. This argument
can help some types decide on the most efficient processing
of result values.
- Deprecated Dialect.get_params() removed.
- Dialect.get_rowcount() has been renamed to a descriptor
"rowcount", and calls cursor.rowcount directly. Dialects
which need to hardwire a rowcount in for certain calls
should override the method to provide different behavior.
- DefaultRunner and subclasses have been removed. The job
of this object has been simplified and moved into
ExecutionContext. Dialects which support sequences should
add a `fire_sequence()` method to their execution context
implementation. [ticket:1566]
- Functions and operators generated by the compiler now use
(almost) regular dispatch functions of the form
"visit_<opname>" and "visit_<funcname>_fn" to provide
customed processing. This replaces the need to copy the
"functions" and "operators" dictionaries in compiler
subclasses with straightforward visitor methods, and also
allows compiler subclasses complete control over
rendering, as the full _Function or _BinaryExpression
object is passed in.
- postgresql
- New dialects: pg8000, zxjdbc, and pypostgresql
on py3k.
- The "postgres" dialect is now named "postgresql" !
Connection strings look like:
postgresql://scott:tiger@localhost/test
postgresql+pg8000://scott:tiger@localhost/test
The "postgres" name remains for backwards compatiblity
in the following ways:
- There is a "postgres.py" dummy dialect which
allows old URLs to work, i.e.
postgres://scott:tiger@localhost/test
- The "postgres" name can be imported from the old
"databases" module, i.e. "from
sqlalchemy.databases import postgres" as well as
"dialects", "from sqlalchemy.dialects.postgres
import base as pg", will send a deprecation
warning.
- Special expression arguments are now named
"postgresql_returning" and "postgresql_where", but
the older "postgres_returning" and
"postgres_where" names still work with a
deprecation warning.
- "postgresql_where" now accepts SQL expressions which
can also include literals, which will be quoted as needed.
- The psycopg2 dialect now uses psycopg2's "unicode extension"
on all new connections, which allows all String/Text/etc.
types to skip the need to post-process bytestrings into
unicode (an expensive step due to its volume). Other
dialects which return unicode natively (pg8000, zxjdbc)
also skip unicode post-processing.
- Added new ENUM type, which exists as a schema-level
construct and extends the generic Enum type. Automatically
associates itself with tables and their parent metadata
to issue the appropriate CREATE TYPE/DROP TYPE
commands as needed, supports unicode labels, supports
reflection. [ticket:1511]
- INTERVAL supports an optional "precision" argument
corresponding to the argument that PG accepts.
- using new dialect.initialize() feature to set up
version-dependent behavior.
- somewhat better support for % signs in table/column names;
psycopg2 can't handle a bind parameter name of
%(foobar)s however and SQLA doesn't want to add overhead
just to treat that one non-existent use case.
[ticket:1279]
- Inserting NULL into a primary key + foreign key column
will allow the "not null constraint" error to raise,
not an attempt to execute a nonexistent "col_id_seq"
sequence. [ticket:1516]
- autoincrement SELECT statements, i.e. those which
select from a procedure that modifies rows, now work
with server-side cursor mode (the named cursor isn't
used for such statements.)
- postgresql dialect can properly detect pg "devel" version
strings, i.e. "8.5devel" [ticket:1636]
- The psycopg2 now respects the statement option
"stream_results". This option overrides the connection setting
"server_side_cursors". If true, server side cursors will be
used for the statement. If false, they will not be used, even
if "server_side_cursors" is true on the
connection. [ticket:1619]
- mysql
- New dialects: oursql, a new native dialect,
MySQL Connector/Python, a native Python port of MySQLdb,
and of course zxjdbc on Jython.
- VARCHAR/NVARCHAR will not render without a length, raises
an error before passing to MySQL. Doesn't impact
CAST since VARCHAR is not allowed in MySQL CAST anyway,
the dialect renders CHAR/NCHAR in those cases.
- all the _detect_XXX() functions now run once underneath
dialect.initialize()
- somewhat better support for % signs in table/column names;
MySQLdb can't handle % signs in SQL when executemany() is used,
and SQLA doesn't want to add overhead just to treat that one
non-existent use case. [ticket:1279]
- the BINARY and MSBinary types now generate "BINARY" in all
cases. Omitting the "length" parameter will generate
"BINARY" with no length. Use BLOB to generate an unlengthed
binary column.
- the "quoting='quoted'" argument to MSEnum/ENUM is deprecated.
It's best to rely upon the automatic quoting.
- ENUM now subclasses the new generic Enum type, and also handles
unicode values implicitly, if the given labelnames are unicode
objects.
- a column of type TIMESTAMP now defaults to NULL if
"nullable=False" is not passed to Column(), and no default
is present. This is now consistent with all other types,
and in the case of TIMESTAMP explictly renders "NULL"
due to MySQL's "switching" of default nullability
for TIMESTAMP columns. [ticket:1539]
- oracle
- unit tests pass 100% with cx_oracle !
- support for cx_Oracle's "native unicode" mode which does
not require NLS_LANG to be set. Use the latest 5.0.2 or
later of cx_oracle.
- an NCLOB type is added to the base types.
- use_ansi=False won't leak into the FROM/WHERE clause of
a statement that's selecting from a subquery that also
uses JOIN/OUTERJOIN.
- added native INTERVAL type to the dialect. This supports
only the DAY TO SECOND interval type so far due to lack
of support in cx_oracle for YEAR TO MONTH. [ticket:1467]
- usage of the CHAR type results in cx_oracle's
FIXED_CHAR dbapi type being bound to statements.
- the Oracle dialect now features NUMBER which intends
to act justlike Oracle's NUMBER type. It is the primary
numeric type returned by table reflection and attempts
to return Decimal()/float/int based on the precision/scale
parameters. [ticket:885]
- func.char_length is a generic function for LENGTH
- ForeignKey() which includes onupdate=<value> will emit a
warning, not emit ON UPDATE CASCADE which is unsupported
by oracle
- the keys() method of RowProxy() now returns the result
column names *normalized* to be SQLAlchemy case
insensitive names. This means they will be lower case for
case insensitive names, whereas the DBAPI would normally
return them as UPPERCASE names. This allows row keys() to
be compatible with further SQLAlchemy operations.
- using new dialect.initialize() feature to set up
version-dependent behavior.
- using types.BigInteger with Oracle will generate
NUMBER(19) [ticket:1125]
- "case sensitivity" feature will detect an all-lowercase
case-sensitive column name during reflect and add
"quote=True" to the generated Column, so that proper
quoting is maintained.
- firebird
- the keys() method of RowProxy() now returns the result
column names *normalized* to be SQLAlchemy case
insensitive names. This means they will be lower case for
case insensitive names, whereas the DBAPI would normally
return them as UPPERCASE names. This allows row keys() to
be compatible with further SQLAlchemy operations.
- using new dialect.initialize() feature to set up
version-dependent behavior.
- "case sensitivity" feature will detect an all-lowercase
case-sensitive column name during reflect and add
"quote=True" to the generated Column, so that proper
quoting is maintained.
- mssql
- MSSQL + Pyodbc + FreeTDS now works for the most part,
with possible exceptions regarding binary data as well as
unicode schema identifiers.
- the "has_window_funcs" flag is removed. LIMIT/OFFSET
usage will use ROW NUMBER as always, and if on an older
version of SQL Server, the operation fails. The behavior
is exactly the same except the error is raised by SQL
server instead of the dialect, and no flag setting is
required to enable it.
- the "auto_identity_insert" flag is removed. This feature
always takes effect when an INSERT statement overrides a
column that is known to have a sequence on it. As with
"has_window_funcs", if the underlying driver doesn't
support this, then you can't do this operation in any
case, so there's no point in having a flag.
- using new dialect.initialize() feature to set up
version-dependent behavior.
- removed references to sequence which is no longer used.
implicit identities in mssql work the same as implicit
sequences on any other dialects. Explicit sequences are
enabled through the use of "default=Sequence()". See
the MSSQL dialect documentation for more information.
- sqlite
- DATE, TIME and DATETIME types can now take optional storage_format
and regexp argument. storage_format can be used to store those types
using a custom string format. regexp allows to use a custom regular
expression to match string values from the database.
- Time and DateTime types now use by a default a stricter regular
expression to match strings from the database. Use the regexp
argument if you are using data stored in a legacy format.
- __legacy_microseconds__ on SQLite Time and DateTime types is not
supported anymore. You should use the storage_format argument
instead.
- Date, Time and DateTime types are now stricter in what they accept as
bind parameters: Date type only accepts date objects (and datetime
ones, because they inherit from date), Time only accepts time
objects, and DateTime only accepts date and datetime objects.
- Table() supports a keyword argument "sqlite_autoincrement", which
applies the SQLite keyword "AUTOINCREMENT" to the single integer
primary key column when generating DDL. Will prevent generation of
a separate PRIMARY KEY constraint. [ticket:1016]
- new dialects
- postgresql+pg8000
- postgresql+pypostgresql (partial)
- postgresql+zxjdbc
- mysql+pyodbc
- mysql+zxjdbc
- types
- The construction of types within dialects has been totally
overhauled. Dialects now define publically available types
as UPPERCASE names exclusively, and internal implementation
types using underscore identifiers (i.e. are private).
The system by which types are expressed in SQL and DDL
has been moved to the compiler system. This has the
effect that there are much fewer type objects within
most dialects. A detailed document on this architecture
for dialect authors is in
lib/sqlalchemy/dialects/type_migration_guidelines.txt .
- Types no longer make any guesses as to default
parameters. In particular, Numeric, Float, NUMERIC,
FLOAT, DECIMAL don't generate any length or scale unless
specified.
- types.Binary is renamed to types.LargeBinary, it only
produces BLOB, BYTEA, or a similar "long binary" type.
New base BINARY and VARBINARY
types have been added to access these MySQL/MS-SQL specific
types in an agnostic way [ticket:1664].
- String/Text/Unicode types now skip the unicode() check
on each result column value if the dialect has
detected the DBAPI as returning Python unicode objects
natively. This check is issued on first connect
using "SELECT CAST 'some text' AS VARCHAR(10)" or
equivalent, then checking if the returned object
is a Python unicode. This allows vast performance
increases for native-unicode DBAPIs, including
pysqlite/sqlite3, psycopg2, and pg8000.
- Most types result processors have been checked for possible speed
improvements. Specifically, the following generic types have been
optimized, resulting in varying speed improvements:
Unicode, PickleType, Interval, TypeDecorator, Binary.
Also the following dbapi-specific implementations have been improved:
Time, Date and DateTime on Sqlite, ARRAY on Postgresql,
Time on MySQL, Numeric(as_decimal=False) on MySQL, oursql and
pypostgresql, DateTime on cx_oracle and LOB-based types on cx_oracle.
- Reflection of types now returns the exact UPPERCASE
type within types.py, or the UPPERCASE type within
the dialect itself if the type is not a standard SQL
type. This means reflection now returns more accurate
information about reflected types.
- Added a new Enum generic type. Enum is a schema-aware object
to support databases which require specific DDL in order to
use enum or equivalent; in the case of PG it handles the
details of `CREATE TYPE`, and on other databases without
native enum support will by generate VARCHAR + an inline CHECK
constraint to enforce the enum.
[ticket:1109] [ticket:1511]
- The Interval type includes a "native" flag which controls
if native INTERVAL types (postgresql + oracle) are selected
if available, or not. "day_precision" and "second_precision"
arguments are also added which propagate as appropriately
to these native types. Related to [ticket:1467].
- The Boolean type, when used on a backend that doesn't
have native boolean support, will generate a CHECK
constraint "col IN (0, 1)" along with the int/smallint-
based column type. This can be switched off if
desired with create_constraint=False.
Note that MySQL has no native boolean *or* CHECK constraint
support so this feature isn't available on that platform.
[ticket:1589]
- PickleType now uses == for comparison of values when
mutable=True, unless the "comparator" argument with a
comparsion function is specified to the type. Objects
being pickled will be compared based on identity (which
defeats the purpose of mutable=True) if __eq__() is not
overridden or a comparison function is not provided.
- The default "precision" and "scale" arguments of Numeric
and Float have been removed and now default to None.
NUMERIC and FLOAT will be rendered with no numeric
arguments by default unless these values are provided.
- AbstractType.get_search_list() is removed - the games
that was used for are no longer necessary.
- Added a generic BigInteger type, compiles to
BIGINT or NUMBER(19). [ticket:1125]
-ext
- sqlsoup has been overhauled to explicitly support an 0.5 style
session, using autocommit=False, autoflush=True. Default
behavior of SQLSoup now requires the usual usage of commit()
and rollback(), which have been added to its interface. An
explcit Session or scoped_session can be passed to the
constructor, allowing these arguments to be overridden.
- sqlsoup db.<sometable>.update() and delete() now call
query(cls).update() and delete(), respectively.
- sqlsoup now has execute() and connection(), which call upon
the Session methods of those names, ensuring that the bind is
in terms of the SqlSoup object's bind.
- sqlsoup objects no longer have the 'query' attribute - it's
not needed for sqlsoup's usage paradigm and it gets in the
way of a column that is actually named 'query'.
- The signature of the proxy_factory callable passed to
association_proxy is now (lazy_collection, creator,
value_attr, association_proxy), adding a fourth argument
that is the parent AssociationProxy argument. Allows
serializability and subclassing of the built in collections.
[ticket:1259]
- association_proxy now has basic comparator methods .any(),
.has(), .contains(), ==, !=, thanks to Scott Torborg.
[ticket:1372]
- examples
- The "query_cache" examples have been removed, and are replaced
with a fully comprehensive approach that combines the usage of
Beaker with SQLAlchemy. New query options are used to indicate
the caching characteristics of a particular Query, which
can also be invoked deep within an object graph when lazily
loading related objects. See /examples/beaker_caching/README.
0.5.9
=====
- sql
- Fixed erroneous self_group() call in expression package.
[ticket:1661]
0.5.8
=====
- sql
- The copy() method on Column now supports uninitialized,
unnamed Column objects. This allows easy creation of
declarative helpers which place common columns on multiple
subclasses.
- Default generators like Sequence() translate correctly
across a copy() operation.
- Sequence() and other DefaultGenerator objects are accepted
as the value for the "default" and "onupdate" keyword
arguments of Column, in addition to being accepted
positionally.
- Fixed a column arithmetic bug that affected column
correspondence for cloned selectables which contain
free-standing column expressions. This bug is
generally only noticeable when exercising newer
ORM behavior only availble in 0.6 via [ticket:1568],
but is more correct at the SQL expression level
as well. [ticket:1617]
- postgresql
- The extract() function, which was slightly improved in
0.5.7, needed a lot more work to generate the correct
typecast (the typecasts appear to be necessary in PG's
EXTRACT quite a lot of the time). The typecast is
now generated using a rule dictionary based
on PG's documentation for date/time/interval arithmetic.
It also accepts text() constructs again, which was broken
in 0.5.7. [ticket:1647]
- firebird
- Recognize more errors as disconnections. [ticket:1646]
0.5.7
=====
- orm
- contains_eager() now works with the automatically
generated subquery that results when you say
"query(Parent).join(Parent.somejoinedsubclass)", i.e.
when Parent joins to a joined-table-inheritance subclass.
Previously contains_eager() would erroneously add the
subclass table to the query separately producing a
cartesian product. An example is in the ticket
description. [ticket:1543]
- query.options() now only propagate to loaded objects
for potential further sub-loads only for options where
such behavior is relevant, keeping
various unserializable options like those generated
by contains_eager() out of individual instance states.
[ticket:1553]
- Session.execute() now locates table- and
mapper-specific binds based on a passed
in expression which is an insert()/update()/delete()
construct. [ticket:1054]
- Session.merge() now properly overwrites a many-to-one or
uselist=False attribute to None if the attribute
is also None in the given object to be merged.
- Fixed a needless select which would occur when merging
transient objects that contained a null primary key
identifier. [ticket:1618]
- Mutable collection passed to the "extension" attribute
of relation(), column_property() etc. will not be mutated
or shared among multiple instrumentation calls, preventing
duplicate extensions, such as backref populators,
from being inserted into the list.
[ticket:1585]
- Fixed the call to get_committed_value() on CompositeProperty.
[ticket:1504]
- Fixed bug where Query would crash if a join() with no clear
"left" side were called when a non-mapped column entity
appeared in the columns list. [ticket:1602]
- Fixed bug whereby composite columns wouldn't load properly
when configured on a joined-table subclass, introduced in
version 0.5.6 as a result of the fix for [ticket:1480].
[ticket:1616] thx to Scott Torborg.
- The "use get" behavior of many-to-one relations, i.e. that a
lazy load will fallback to the possibly cached query.get()
value, now works across join conditions where the two compared
types are not exactly the same class, but share the same
"affinity" - i.e. Integer and SmallInteger. Also allows
combinations of reflected and non-reflected types to work
with 0.5 style type reflection, such as PGText/Text (note 0.6
reflects types as their generic versions). [ticket:1556]
- Fixed bug in query.update() when passing Cls.attribute
as keys in the value dict and using synchronize_session='expire'
('fetch' in 0.6). [ticket:1436]
- sql
- Fixed bug in two-phase transaction whereby commit() method
didn't set the full state which allows subsequent close()
call to succeed. [ticket:1603]
- Fixed the "numeric" paramstyle, which apparently is the
default paramstyle used by Informixdb.
- Repeat expressions in the columns clause of a select
are deduped based on the identity of each clause element,
not the actual string. This allows positional
elements to render correctly even if they all render
identically, such as "qmark" style bind parameters.
[ticket:1574]
- The cursor associated with connection pool connections
(i.e. _CursorFairy) now proxies `__iter__()` to the
underlying cursor correctly. [ticket:1632]
- types now support an "affinity comparison" operation, i.e.
that an Integer/SmallInteger are "compatible", or
a Text/String, PickleType/Binary, etc. Part of
[ticket:1556].
- Fixed bug preventing alias() of an alias() from being
cloned or adapted (occurs frequently in ORM operations).
[ticket:1641]
- sqlite
- sqlite dialect properly generates CREATE INDEX for a table
that is in an alternate schema. [ticket:1439]
- postgresql
- Added support for reflecting the DOUBLE PRECISION type,
via a new postgres.PGDoublePrecision object.
This is postgresql.DOUBLE_PRECISION in 0.6.
[ticket:1085]
- Added support for reflecting the INTERVAL YEAR TO MONTH
and INTERVAL DAY TO SECOND syntaxes of the INTERVAL
type. [ticket:460]
- Corrected the "has_sequence" query to take current schema,
or explicit sequence-stated schema, into account.
[ticket:1576]
- Fixed the behavior of extract() to apply operator
precedence rules to the "::" operator when applying
the "timestamp" cast - ensures proper parenthesization.
[ticket:1611]
- mssql
- Changed the name of TrustedConnection to
Trusted_Connection when constructing pyodbc connect
arguments [ticket:1561]
- oracle
- The "table_names" dialect function, used by MetaData
.reflect(), omits "index overflow tables", a system
table generated by Oracle when "index only tables"
with overflow are used. These tables aren't accessible
via SQL and can't be reflected. [ticket:1637]
- ext
- A column can be added to a joined-table declarative
superclass after the class has been constructed
(i.e. via class-level attribute assignment), and
the column will be propagated down to
subclasses. [ticket:1570] This is the reverse
situation as that of [ticket:1523], fixed in 0.5.6.
- Fixed a slight inaccuracy in the sharding example.
Comparing equivalence of columns in the ORM is best
accomplished using col1.shares_lineage(col2).
[ticket:1491]
- Removed unused `load()` method from ShardedQuery.
[ticket:1606]
libcassandra is a C++ client API for Apache Cassandra. The standard
front-end (in the sense of "language binding") for writing C++ programs
that use Cassandra.
Thrift is a software framework for scalable cross-language services
development. It combines a software stack with a code generation engine to
build services that work efficiently and seamlessly between C++, Java,
Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, Smalltalk, and OCaml.
This module provides an interface the to Cassandra distributed database.
It uses the Thrift interface. This is changing rapidly and supports
version 0.5.0 of Cassandra.
- change to how getmail counts messages in an IMAP mailbox; prevents problems
where getmail would only see the first 500 messages in a mailbox with some
IMAP servers that return oddball responses to SELECT. Thanks: David Damerell.
1.) Add missing modules "mod_proxy_scgi.so" and "mod_reqtimeout.so"
if the package is built with shared modules enabled.
This fixes PR pkg/43229 by Ryo HAYASAKA.
2.) Get rid of "PLIST.worker" and use "PLIST_VARS" instead.
3.) Use an option group instead of the "APACHE_MPM" configuration variable
to configure the worker model.
4.) Enable the "apache-shared-modules" options by default. This provides
more flexibility and matches the behaviour of a lot of other
platforms e.g. Solaris or Linux distributions like Ubuntu.
Bump the package revision as the binary package will change by default.
taca
wiz
tonnerre
asau
abs
schmonz
minskim
tron
ghen