With pkgsrc built packages, the goal is to have the user's compiler
optimization flags set by the user using CFLAGS, etc. To that end,
the cmake Makefile has subst.mk code to "sed -e 's,-O.,,'" out opt.
flags in Modules/Compiler/*.cmake for cmake compiles with the
CMAKE_BUILD_TYPE set (e.g. to Release).
Unfortunately, this sed-based approach has two drawbacks: First, it
gets applied to packages build with cmake outside of pkgsrc (changing
the semantics of -DCMAKE_BUILD_TYPE=Release in an unexpected way by
remove -O flags from it). Second, the sed expression damages some
of the Modules/Compiler/*.cmake as not all opt flags match the "-O."
regular expression.
To address this we:
1. remove the subst.mk sed operation on Modules/Compiler/*.cmake
2. add a new config variable CMAKE_PKGSRC_BUILD_FLAGS that should
be set for pkgsrc-based cmake builds (e.g. in mk/configure/cmake.mk)
3. we add a patch for Modules/Compiler/GNU.cmake to look for
CMAKE_PKGSRC_BUILD_FLAGS and choose a set of non-opt compiler
flags when this is set (to provide the desired behavior under pksrc).
note that the GNU.cmake file is also used by clang. We can
make the corresponding changes to other compilers if desired.
The result is that cmake compiles under pkgsrc set CMAKE_PKGSRC_BUILD_FLAGS
and follow CFLAGS, etc. (bypassing CMAKE_BUILD_TYPE), but cmake compiles
outside of pkgsrc follow the standard cmake semantics for CMAKE_BUILD_TYPE.
Also, we no longer damage Modules/Compiler/*.cmake with sed.
Abseil Python Common Libraries are a collection of Python library code
for building Python applications. The code is collected from Google's
own Python code base, and has been extensively tested and used in
production.
1.2.0:
This is a minor release that essentially updates internal C-Blosc sources to 1.14.2. This is important for forward compatibility reasons. Because of this an update is highly recommended.
Also, support for Python 2.6 and 3.4 has been dropped.
Upstream changes:
0.010 2018-03-25 10:15:40-07:00 America/Los_Angeles
- Undo the previous "fix" for the minimum perl version.
The basic requirement is for 5.004 but the tests require
5.006.
Upstream changes:
1.119 Mon Apr 2 2018
[TESTS]
- Various test fixes for Win32 and "perl in space" (ETJ)
[DOCUMENTATION]
- document the 'input_record_separator' option to loop_on()
Upstream changes:
[1.04] Released on 2018-03-26
Sped up critic_ok() by 4x by not recreating a Perl::Critic object
over and over. Thanks, Jan Hol膷apek. (GH#10)
Now requires Test::Builder~0.88 or later to support the done_testing()
method. (GH#2)
Now requires MCE 1.827 to deal with problems running under taint
mode. Thanks, Alexandr Ciornii. (GH #6)
r1.14 of src/external/gpl3/binutils/dist/bfd/elflink.c may fix the
underlying problem but r1.55 of pkgsrc/x11/qt5-qtbase/Makefile means
we no longer tickle it so remove.
Upstream changes:
1.302135 2018-03-29 22:53:00-07:00 America/Los_Angeles
- No changes since last trial
1.302134 2018-03-19 21:20:08-07:00 America/Los_Angeles (TRIAL RELEASE)
- Make sure all hubs, events, and contexts get a unique (per run) id.
- Use a common generator for unique(enough) id's (not UUIDs)
IPython 6.3.1
This is a bugfix release to switch the default completions back to IPython's own completion machinery. We discovered some problems with the completions from Jedi, including completing column names on pandas data frames.
You can switch the completions source with the config option :configtrait:Completer.use_jedi.
ccache 3.4.2
Bug fixes:
The cleanup algorithm has been fixed to not misbehave when files are removed by another process while the cleanup process is running. Previously, too many files could be removed from the cache if multiple cleanup processes were triggered at the same time, in extreme cases trimming the cache to a much smaller size than the configured limits.
Correctly hash preprocessed headers located in a “.gch directory”. Previously, ccache would not pick up changes to such precompiled headers, risking false positive cache hits.
Fixed build failure when using the bundled zlib sources.
ccache 3.3.5 added a workaround for not triggering Clang errors when a precompiled header’s dependency has an updated timestamp (but identical content). That workaround is now only applied when the compiler is Clang.
Made it possible to perform out-of-source builds in dev mode again.
Notable changes since 1.12.1:
- Serious optimization of memory copy functions (see new
blosc/fastcopy.c). This benefits the speed of all the codecs, but
specially the BloscLZ one.
- As a result of the above, the BloscLZ codec received a new
adjustment of knobs so that you should expect better compression
ratios with it too.
- Fixed a buffer overrun that happens when compressing small buffers
and len(destination_buffer) < (len(source_buffer) +
BLOSC_MAX_OVERHEAD).
- New split mode that favors forward compatibility. That means that,
from now on, all the buffers created starting with blosc 1.14.0 will
be forward compatible with any previous versions of the library --at
least until 1.3.0, when support for multi-codec was introduced.
3.55.0:
This release includes several improvements to the handling of the :obj:~hypothesis.settings.database setting.
The :obj:~hypothesis.settings.database_file setting was a historical artefact, and you should just use :obj:~hypothesis.settings.database directly.
The :envvar:HYPOTHESIS_DATABASE_FILE environment variable is deprecated, in favor of :meth:~hypothesis.settings.load_profile and the :obj:~hypothesis.settings.database setting.
If you have not configured the example database at all and the default location is not usable (due to e.g. permissions issues), Hypothesis will fall back to an in-memory database. This is not persisted between sessions, but means that the defaults work on read-only filesystems.
3.54.0:
This release improves the :func:~hypotheses.strategies.complex_numbers strategy, which now supports min_magnitude and max_magnitude arguments, along with allow_nan and allow_infinity like for :func:~hypotheses.strategies.floats.
3.53.0:
This release removes support for Django 1.8, which reached end of life on 2018-04-01.
5.2.3:
- Fix hang on close in :class:.ThreadedKernelClient (used in QtConsole)
when using tornado with asyncio
- Fix errors when using deprecated :attr:.KernelManager.kernel_cmd
0.31.0
- Fixed displaying of errors on Python 3
- Fixed single digit versions in wheel files not being properly recognized
- Fixed wrong character encodings being used (instead of UTF-8) to read and
write RECORD (this sometimes crashed bdist_wheel too)
- Enabled Zip64 support in wheels by default
- Metadata-Version is now 2.1
- Dropped DESCRIPTION.rst and metadata.json from the list of generated files
- Dropped support for the non-standard, undocumented provides-extra and
requires-dist keywords in setup.cfg metadata
- Deprecated all wheel signing and signature verification commands
- Removed the (already defunct) tool extras from setup.py
Default changes
- socket.max.fails changed to 1 to provide same functionality (fail
request immediately on error) now when retries are working properly
again.
- fetch.max.bytes (new config property) is automatically adjusted to
be >= message.max.bytes, and receive.message.max.bytes is
automatically adjusted to be > fetch.max.bytes.
New features
- Message Headers support
- Java-compatible Murmur2 partitioners
- Add PKCS#12 Keystore support - ssl.keystore.location
Noteworthy fixes
- Formalise and fix Producer retries and retry-ordering
- Ordering is now retained despite retries if max.in.flight=1.
- Behaviour is now documented
- Fix timeouts for retried requests and properly handle retries for
all request types
- Add and use fetch.max.bytes to limit total Fetch response size.
Fixes "Invalid response size" issues.
Enhancements
- Added sasl.mechanism and compression.type configuration property
aliases for conformance with Java client.
- Improved Producer performance
- C++: add c_ptr() to Handle,Topic,Message classes to expose
underlying librdkafka object
- Honour per-message partition in produce_batch() if MSG_F_PARTITION
set
- Added on_request_sent() interceptor
- Added experimental flexible producer queuing.strategy=fifo|lifo
- Broker address DNS record round-robin: try to maintain round-robin
position across resolve calls.
- Set system thread name for internal librdkafka threads
- Added more concise and user-friendly 'consumer' debug context
- Add partitioner (string) topic configuration property to set the
builtin partitioners
- Generate rdkafka-static.pc (pkg-config) for static linking
Fixes
- Fix producer memory leak on <0.11 brokers when compressed messageset
is below copy threshold
- CRC32C - fix unaligned access on ARM
- Fix read after free in buf_write_seek
- Fix broker wake up
- Fix consumer hang when rebalancing during commit
- CMake fixes for Windows
- LeaveGroup was not sent on close when doing final offset commits
- Fix for consumer slowdown/stall on compacted topics where actual
last offset < MsgSet.LastOffset
- Fix global->topic conf fallthru in C++ API
- Fix infinite loop on LeaveGroup failure
- Fix possible crash on OffsetFetch retry
- Incorporate compressed message count when deciding on fetch backoff
- Fix debug-only crash on Solaris (%s NULL)
- Drain broker ops queue on termination to avoid hang
- cmake: Allow build static library
- Don't store invalid offset as next one when pausing
- use #if instead of #ifdef / defined() for atomics
- fixed .lib paths in nuget packaging
- Fixes strerror_r crash on alpine
- Allow arbitrary lengthed (>255) SASL PLAIN user/pass
- Trigger ApiVersionRequest on reconnect if broker.version.fallback
supports it
- Read Fetch MsgAttributes as int8
- Portability: stop using typeof in rdkafka_transport.c
- Portability: replace use of #pragma once with header guards
- mklove: add LIBS in reverse order to maintain dependency order
- Fix build when python is not available #1358
New Feature
- automatic migration of log dirs to new locations
- KIP-145 - Expose Record Headers in Kafka Connect
- Add the AdminClient in Streams' KafkaClientSupplier
- Support dynamic updates of frequently updated broker configs
Improvement
- KafkaConnect should support regular expression for topics
- Move kafka-streams test fixtures into a published package
- SSL support for Connect REST API
- Grow default heap settings for distributed Connect from 256M to 1G
- Enable access to key in ValueTransformer
- Add "getAllKeys" API for querying windowed KTable stores
- Unify StreamsKafkaClient instances
- Revisit Streams DSL JavaDocs
- Extend Consumer Group Reset Offset tool for Stream Applications
- KIP-175: ConsumerGroupCommand no longer shows output for consumer
groups which have not committed offsets
- Add a broker metric specifying the number of consumer group
rebalances in progress
- Use Jackson for serialising to JSON
- KafkaShortnamer should allow for case-insensitive matches
- Improve Util classes
- Gradle 3.0+ is needed on the build
- Adding records deletion operation to the new Admin Client API
- Kafka metrics templates used in document generation should maintain
order of tags
- Provide for custom error handling when Kafka Streams fails to
produce
- Make Repartition Topics Transient
- Connect Schema comparison is slow for large schemas
- Add a Validator for NonNull configurations and remove redundant null
checks on lists
- Have State Stores Restore Before Initializing Toplogy
- Optimize condition in if statement to reduce the number of
comparisons
- Removed unnecessary null check
- Introduce Incremental FetchRequests to Increase Partition
Scalability
- SSLTransportLayer should keep reading from socket until either the
buffer is full or the socket has no more data
- Improve KTable Source state store auto-generated names
- Extend consumer offset reset tool to support deletion (KIP-229)
- Expose Kafka cluster ID in Connect REST API
- Maven artifact for kafka should not depend on log4j
- ConsumerGroupCommand should use the new consumer to query the log
end offsets.
- Change LogSegment.delete to deleteIfExists and harden log recovery
- Make ProducerConfig and ConsumerConfig constructors public
- Improve synchronization in CachingKeyValueStore methods
- Improve Kafka GZip compression performance
- Improve JavaDoc of SourceTask#poll() to discourage indefinite
blocking
- Avoid creating dummy checkpoint files with no state stores
- Change log level from ERROR to WARN for not leader for this
partition exception
- Delay initiating the txn on producers until initializeTopology with
EOS turned on
Bug
- change log4j to slf4j
- Use RollingFileAppender by default in log4j.properties
- Cached zkVersion not equal to that in zookeeper, broker not
recovering.
- FileRecords.read doesn't handle size > sizeInBytes when start is not
zero
- a soft failure in controller may leave a topic partition in an
inconsistent state
- Cannot truncate to a negative offset (-1) exception at broker
startup
- automated leader rebalance causes replication downtime for clusters
with too many partitions
- kafka-run-class has potential to add a leading colon to classpath
- QueryableStateIntegrationTest.concurrentAccess is failing
occasionally in jenkins builds
- FileStreamSource Connector not working for large files (~ 1GB)
- KeyValueIterator returns null values
- KafkaProducer is not joining its IO thread properly
- Kafka connect: error with special characters in connector name
- Replace StreamsKafkaClient with AdminClient in Kafka Streams
- LogCleaner#cleanSegments should not ignore failures to delete files
- Connect Rest API allows creating connectors with an empty name -
KIP-212
- KerberosLogin#login should probably be synchronized
- Support replicas movement between log directories (KIP-113)
- Consumer ListOffsets request can starve group heartbeats
- Struct.put() should include the field name if validation fails
- Clarify handling of connector name in config
- Allow user to specify relative path as log directory
- KafkaConsumer should validate topics/TopicPartitions on
subscribe/assign
- Controller should only update reassignment znode if there is change
in the reassignment data
- records.lag should use tags for topic and partition rather than
using metric name.
- KafkaProducer should not wrap InterruptedException in close() with
KafkaException
- Connect classloader isolation may be broken for JDBC drivers
- JsonConverter generates "Mismatching schema" DataException
- NoSuchElementException in markErrorMeter during
TransactionsBounceTest
- Make KafkaFuture.Function java 8 lambda compatible
- ThreadCache#sizeBytes() should check overflow
- KafkaFuture timeout fails to fire if a narrow race condition is hit
- DeleteRecordsRequest to a non-leader
- ReplicaFetcherThread should close the ReplicaFetcherBlockingSend
earlier on shutdown
- NoSuchMethodError when creating ProducerRecord in upgrade system
tests
- Running tools on Windows fail due to typo in JVM config
- Streams metrics tagged incorrectly
- ducker-ak: add ipaddress and enum34 dependencies to docker image
- Kafka cannot recover after an unclean shutdown on Windows
- Scanning plugin.path needs to support relative symlinks
- Reconnecting to broker does not exponentially backoff
- TaskManager should be type aware
- Major performance issue due to excessive logging during leader
election
- RecordCollectorImpl should not retry sending
- Restore and global consumer should not use auto.offset.reset
- Global Consumer should handle TimeoutException
- Reduce rebalance time by not checking if created topics are
available
- VerifiableConsumer with --max-messages doesn't exit
- Transaction markers are sometimes discarded if txns complete
concurrently
- Simplify StreamsBuilder#addGlobalStore
- JmxReporter can't handle windows style directory paths
- CONSUMER-ID and HOST values are concatenated if the CONSUMER-ID is >
50 chars
- ClientQuotaManager threads prevent shutdown when encountering an
error loading logs
- Streams configuration requires consumer. and producer. in order to
be read
- Timestamp on streams directory contains a colon, which is an illegal
character
- Add methods in Options classes to keep binary compatibility with
0.11
- RecordQueue.clear() does not clear MinTimestampTracker's maintained
list
- Selector memory leak with high likelihood of OOM in case of down
conversion
- GlobalKTable never finishes restoring when consuming transactional
messages
- Server crash while deleting segments
- IllegalArgumentException if 1.0.0 is used for
inter.broker.protocol.version or log.message.format.version
- Using standby replicas with an in memory state store causes Streams
to crash
- Issues with protocol version when applying a rolling upgrade to
1.0.0
- Fix system test dependency issues
- Kafka Connect requires permission to create internal topics even if
they exist
- A metric named 'XX' already exists, can't register another one.
- Improve sink connector topic regex validation
- Flaky Unit test:
KStreamKTableJoinIntegrationTest.shouldCountClicksPerRegionWithNonZeroByteCache
- Make KafkaStreams.cleanup() clean global state directory
- AbstractCoordinator not clearly handles NULL Exception
- Request logging throws exception if acks=0
- GlobalKTable missing #queryableStoreName()
- KTable state restore fails after rebalance
- Make loadClass thread-safe for class loaders of Connect plugins
- System Test failed: ConnectRestApiTest
- Broken symlink interrupts scanning the plugin path
- NetworkClient should not return internal failed api version
responses from poll
- Transient failure in
NetworkClientTest.testConnectionDelayDisconnected
- Line numbers on log messages are incorrect
- Topic can not be recreated after it is deleted
- mBeanName should be removed before returning from
JmxReporter#removeAttribute()
- Kafka Core should have explicit SLF4J API dependency
- StreamsResetter should return non-zero return code on error
- kafka-acls regression for comma characters (and maybe other
characters as well)
- Error deleting log for topic, all log dirs failed.
- punctuate with WALL_CLOCK_TIME triggered immediately
- Exclude node groups belonging to global stores in
InternalTopologyBuilder#makeNodeGroups
- Transient failure in
kafka.api.AdminClientIntegrationTest.testLogStartOffsetCheckpointkafka.api.AdminClientIntegrationTest.testAlterReplicaLogDirs
- NetworkClient.inFlightRequestCount() is not thread safe, causing
ConcurrentModificationExceptions when sensors are read
- ConcurrentModificationException during streams state restoration
- Update KStream JavaDoc with regard to KIP-182
- RocksDB segments not removed when store is closed causes
re-initialization to fail
- auto commit not work since coordinatorUnknown() is always true.
- Fix StateRestoreListener To Use Correct Batch Ending Offset
- NullPointerException on KStream-GlobalKTable leftJoin when
KeyValueMapper returns null
- StreamThread.shutdown doesn't clean up completely when called before
StreamThread.start
- Update ZooKeeper to 3.4.11, Gradle and other minor updates
- output from ensure copartitioning is not used for Cluster metadata,
resulting in partitions without tasks working on them
- Consumer should not block setting initial positions of unavailable
partitions
- Non-aggregation KTable generation operator does not construct value
getter correctly
- AdminClient should handle empty or null topic names better
- When enable trace level log in mirror maker, it will throw null
pointer exception and the mirror maker will shutdown
- Simplify KStreamReduce
- Base64URL encoding under JRE 1.7 is broken due to incorrect padding
assumption
- ChangeLoggingKeyValueBytesStore.all() returns null
- Fetcher.retrieveOffsetsByTimes() should add all the topics to the
metadata refresh topics set.
- LogSemgent.truncateTo() should always resize the index file
- Connect: Plugin scan is very slow
- Connect: Some per-task-metrics not working
- Connect header parser incorrectly parses arrays
- Java Producer: Excessive memory usage with compression enabled
- New Connect header support doesn't define 'converter.type' property
correctly
- ZooKeeperClient holds a lock while waiting for responses, blocking
shutdown
- Transient failure in
DynamicBrokerReconfigurationTest.testThreadPoolResize
- Broker leaks memory and file descriptors after sudden client
disconnects
- Delegation token internals should not impact public interfaces
- Streams quickstart pom.xml is missing versions for a bunch of
plugins
- Deadlock while processing Controller Events
- Broker doesn't reject Produce request with inconsistent state
- LogCleanerManager.doneDeleting() should check the partition state
before deleting the in progress partition
- KafkaController.brokerInfo not updated on dynamic update
- Connect standalone SASL file source and sink test fails without
explanation
- Connect distributed and standalone worker 'main()' methods should
catch and log all exceptions
- Consumer bytes-fetched and records-fetched metrics are not
aggregated correctly
- Coordinator disconnect in heartbeat thread can cause commitSync to
block indefinitely
- Regression in consumer auto-commit backoff behavior
- GroupMetadataManager.loadGroupsAndOffsets decompresses record batch
needlessly
- log segment deletion could cause a disk to be marked offline
incorrectly
- Delayed operations may not be completed when there is lock
contention
- Expression for GlobalKTable is not correct
- System tests do not handle ZK chroot properly with SCRAM
- Fix config initialization in DynamicBrokerConfig
- ReplicaFetcher crashes with "Attempted to complete a transaction
which was not started"
Test
- Add concurrent tests to exercise all paths in group/transaction
managers
- Add unit tests for ClusterConnectionStates
- Only delete reassign_partitions znode after reassignment is complete
- KafkaStreamsTest fails in trunk
- SelectorTest may fail with ConcurrentModificationException
Sub-task
- Add capability to create delegation token
- Add authentication based on delegation token.
- Add capability to renew/expire delegation tokens.
- always leave the last surviving member of the ISR in ZK
- handle ZK session expiration properly when a new session can't be
established
- Streams should not re-throw if suspending/closing tasks fails
- Use async ZookeeperClient in Controller
- Use async ZookeeperClient in SimpleAclAuthorizer
- Use async ZookeeperClient for DynamicConfigManager
- Use async ZookeeperClient for Admin operations
- Trogdor should handle injecting disk faults
- Add process stop faults, round trip workload, partitioned
produce-consume test
- add the notion of max inflight requests to async ZookeeperClient
- Add workload generation capabilities to Trogdor
- Add ZooKeeperRequestLatencyMs to KafkaZkClient
- Use ZookeeperClient in LogManager
- Use ZookeeperClient in GroupCoordinator and TransactionCoordinator
- Use ZookeeperClient in KafkaApis
- Use ZookeeperClient in ReplicaManager and Partition
- Tests for KafkaZkClient
- Transient failure in
kafka.api.SaslScramSslEndToEndAuthorizationTest.testTwoConsumersWithDifferentSaslCredentials
- minimize the number of triggers enqueuing
PreferredReplicaLeaderElection events
- Enable dynamic reconfiguration of SSL keystores
- Enable resizing various broker thread pools
- Enable reconfiguration of metrics reporters and their custom configs
- Enable dynamic reconfiguration of log cleaners
- Enable reconfiguration of default topic configs used by brokers
- Enable reconfiguration of listeners and security configs
- Add ProduceBench to Trogdor
- move ZK metrics in KafkaHealthCheck to ZookeeperClient
- Add documentation for delegation token authentication mechanism
- Document dynamic config update
- Extend ConfigCommand to update broker config using new AdminClient
- Add test to verify markPartitionsForTruncation after fetcher thread
pool resize
