* Common Locale Data Repository (CLDR) 1.9.1
* Update timezone data support to Olson 2011c
* 8271 UCOL_RUNTIME_VERSION should be updated for 4.6
* 8277 Collation Reordering Use Of USCRIPT_UNKNOWN
* 8290 Can't find Hangul with search coll (usearch doesn't handle CE iter
behavior)
* 8303 ULocale#toLanguageTag() should not supply "und" as language when the
locale has only private use
* 8341 USpoof uses NFKD, should be NFD
major changes:
Locale Data: ICU uses and supports data from Common Locale Data Repository
(CLDR) 1.7 , which includes data for 146 languages, 159 territories,
468 locales- 21% more locale data than the previous release.
Number system support and the number keyword.
Number system override in DateFormat
Numerics used by Hebrew Calendar date in Hebrew locale
BCP47 (language tag) / Locale transformation
BCP47 mapping of LDML keywords
Encoding selector: Return a list of charsets that can handle the input text
Simple duration: Implementation of CLDR duration format
Available/Preferred keywords for a locale (Calendar, Collation, and Currency)
StringPrep standard profiles: RFC3491 NAMEPREP, RFC3530 NFS4, RFC3722 iSCSI,
RFC3920 NodePrep/ResourcePrep, RFC4011 MIB, RFC4013 SASLprep, RFC4505 trace
and RFC4518 LDAPprep
Miscellaneous Arabic shaping enhancements
UTF-8 friendly internal data structure for Unicode data lookup
API to get CLDR version used by ICU
ISCII charset converter updates (added Gurumukhi, other updates)
Performance improvements in Time Zone Name format/parse, and in
DateIntervalFormat construction
Pkgsrc changes:
o New MASTER_SITE
o Adjust PLIST
o Remove no-longer-needed patches, since corresponding changes
have been adopted upstream
o BUILDLINK_ABI_DEPENDS bumped to >=4.0, since a new shared library
version is installed
o Fixes security vulnerability, ref. below.
Dependent pkgsrc packages will have their revisions bumped shortly
due to the (possibly/probably) changed ABI.
Upstream changes:
4.0.1:
ICU4C 4.0.1 is a maintenance release of ICU4J 4.0. The primary
changes of this release were:
* Updated time zone data to 2008i
* Technical preview of string search implementation using
Boyer-Moore algorithm (#6286). For detail information, please
see the tech note here.
* #5691 Conversion: consistent illegal sequences
* #6435 Bad @stable ICU4.0 tags
* #6597 TestDisplayNamesMeta failure
* #6670 Test failure in format/TimeZoneTest/TestShortZoneIDs
4.0:
Major changes in ICU 4.0 include the following:
* Common Changes
o Unicode 5.1 (#5696)
o Locale Data: ICU uses and supports data from Common
Locale Data Repository (CLDR) 1.6 , which includes many
improvements in quality and quantity of data.
o add/removeLikelySubtags (#6124)
o Charset converter file size improvement (#5987)
o Date Interval Formatting (#6157) Note: Calendar type
supported by this feature is Gregorian only in this
release.
o Improved Plural support
* ICU4C Specific Changes
Additional Calendars
+ Chinese (#4081)
+ Coptic/Ethiopic (#4571)
* ICU4J Specific Changes
o Charset
+ Graduated from Technology Preview status
+ ICU2022 Converter (#5791)
+ HZ Converter (#6128)
+ SCSU/BOCU-1 Converter (#2147)
+ Charset Converter Callback (#6144)
o Thai Dictionary break iterator (#5385)
o JDK TimeZone support (#5975)
o Locale Service Provider (#5976)
o More convenient formatting of year+month, day+month,
and other combinations (#6304)
o Simple Duration Formatting (#6303)
* ICU4C Security Fixes
ICU4C 4.0 resolves the vulnerabilities CVE-2007-4770 and
CVE-2007-4771 which were found in earlier versions of ICU.
The standard ICU tests verify that these have been corrected,
however, the updated versions of the previous tests may be
run by applying the following patch to ICU 4.0: r24324. As
well, ICU4C and ICU4J 4.0 resolve the issue underlying
CVE-2008-1036.
Major changes in ICU 3.6 include the following:
- Unicode: ICU uses and supports Unicode 5.0, which is the latest major release of Unicode. Unicode 5.0 will be used in many operating systems and applications, and this version of ICU is important maintain interoperability with these new operating systems and applications. More information about Unicode 5.0 can be found in the Unicode press release.
- Locale Data: ICU uses and supports data from Common Locale Data Repository (CLDR) 1.4, which includes many improvements in quality and quantity of data. There is 25% more CLDR locale data in 245 locales in ICU.
- ICU4C Specific Changes
- Charset Detection: A charset detection framework was added, which provides heuristics for detecting the charset for unlabeled sequences of bytes.
- Layout: The font layout engine has support added for Tibetan, Sinhala and Old Hangul.
- BiDi: The BiDi algorithm was enhanced to be more flexible and efficient
- ICU Data Management: The new icupkg tool provides an easier way to manage ICU's data library. This tool allows you to add, update or remove data from ICU's data archive.
- Time Zones The time zone data is modularized to allow easier building and updating of the data.
- Word Boundaries: The Thai word break iteration was improved to be more accurate. Also dictionary based detection of Thai word boundaries is now active for all locales.
- UText
- The BreakIterator uses UText for abstract text processing.
- 64-bit indexing is now used to allow access to larger chunks of text.
- API for read-only locking for security and robustness was added.
- Performance
- The u_sprintf/u_sscanf performance from the icuio library has been improved for number formatting/parsing.
- Constructing a DateFormat is significantly faster for many locales.
- Opening and closing a charset converter is significantly faster.
- The UTF-8 transformation functions and macros are faster.
- The UText API was improved for performance.
- The collation open and close functions have a small performance improvement.
being here, update to 3.4.1
changes:
-Updated timezone data
-Improved portability
-Improved default codepage and default locale detection.
-A number of collation bug fixes.
New Features:
Major changes in ICU 3.4 include the following:
Updates to conform to Unicode 4.1, including new characters
properties and values, text segmentation, plus collation
updated for Unicode Technical Standard #10 (UCA) and regex
updated for Unicode Technical Standard #18.
* Updates to conform to the Common Locale Data
Repository (CLDR), Version 1.3 for the latest locale
data. This includes:
* New data to support localization of timezones,
United Nations M.49 regions (including continents
and regions), mappings from language to script
and territory.
* Consolidation of inherited data and improved
resource aliasing for smaller data footprint
* Additional locales, and many other fixes and
additions of locale data.
* POSIX migration support: direct API support for all
POSIX character classes, implemented according to
Unicode recommendations
performance/build/porting enhancements. The following list summarizes
the main new features in this release.sion.
CLDR 1.2.
This is the main new feature in the release. ICU locale data is now completely
built from the CLDR 1.2 data, which contains data for 232 locales, covering 72
languages and 108 territories. Many translated names for languages,
territories, and scripts have been added, as well as for time zones,
calendars, and other named items such as collation. For more information,
see http://www.unicode.org/press/pr-cldr1.2.html.
Miscellaneous
Universal Timescale conversions. ICU now provides mechanisms for quickly and
reliably converting between the different binary representations of date/time
used on different platforms.
Accept-Language. ICU provides a mechanism for matching Accept-Language against
a list of locales.
DateFormat and Calendar Performance. Object construction performance has been
significantly improved.
Footprint. The size of executables that statically link to ICU has been
reduced.
Stdin. The icuio library can now read from stdin.
UnicodeSet C API. More uset_* C API were added.
i5/OS (os/400). Building ICU has been simplified to allow more configure
options to work.
POSIX. Default codepage determination has been fixed.
major changes:
ICU 3.0 includes the latest bug fixes, locale/charset updates, and
performance/build/porting enhancements.
- Collation
Collation data is in a separate data tree, allowing for easier
modularization and maintenance.
getFunctionalEquivalent API allows for better caching and UI support.
- Unicode 4.0.1
ICU is updated to the latest version of Unicode standard, which had
significant property changes.
- CLDR 1.1
Updates to CLDR 1.1, with many updates to locale data, and special
emphasis on collation data.
- Formatting
As an aid to migration of traditional C (stdio) and C++ (iostream)
formatting, the POSIX-like input/output library, icuio, is officially
supported.
Significant digits now supported in DecimalFormat, for general use and
%g support.
- RFC822 time zone format support in DateFormat for compatibility.
- Currency formatting/parsing improvements
Allows parsing multiple currencies with one formatter, without knowing the
currency in advance. Much cleaner design allowing extensibility to other
measurement units in the future.
- Regular expressions (C)
The regular expressions framework now features a C API, instead of just C++.
- Locales
Locale canonicalization spec defined and implemented. Provides
interoperability with POSIX and .NET locale IDs, more RFC 3066 support.
- Layout engine
Layout engine now supports using different canonically-equivalent Unicode
forms of the same text: e.g. a + ´ or á. This is especially important for
non-Latin scripts.
- Build Environment
ICU can now build its data library much faster on most platforms.
For a complete list see:
http://oss.software.ibm.com/cvs/icu/~checkout~/icu/readme.html?tag=release-3-0
Lot's of changes and fixes.
For example:
# Number Formatting
ICU4C adds support for formatting and parsing of 64-bit integers.
# Text Analysis (Break Iterators)
Full conformance with Unicode Consortium UAX 29 and UAX 14 definitions for
text boundary positions. Significantly improved performance for reverse
direction iteration and isBoundary tests of arbitrary string positions.
# StringPrep
ICU 2.8 adds APIs and a tool for generic support of StringPrep profiles such
as those used in NFS 4.
For a complete list see:
http://oss.software.ibm.com/icu/download/2.8/index.html
features and new and modified APIs from version 2.4:
* Added support for Unicode 4.0
* Added support for Unicode regular expressions
* Enhanced sorting
* Added support for international domain names
* Added service registration for pluggable ICU modules
* Added layout engine API for language-specific glyphs
* Separated currencies from locales
* Added POSIX-like API for message catalogs
* Added new charset converters
Based on a PR pkg/20825 by Hiramatsu Yoshifumi, modified by me.
- follow PKG_SYSCONFDIR
List of major changes for this release:
* Regular Expressions Phase 1
ICU 2.4 introduces a Regular Expression C++ API that is modeled after
the JDK 1.4 API. ICU 2.4's Regular Expression API supports Unicode
level 1 regular expressions (see Unicode Regular Expression
Guidelines) but not all pattern metacharacters and features are
supported yet. Regular expressions leverage all of the UnicodeSet
support, including all Unicode 3.2 property names and property value
names. Future ICU releases will complete the pattern support, add
support for higher Unicode regex levels, and improve performance. For
more details see the API References and the User Guide.
* Modularized ICU library building
ICU 2.4 provides build-time switches to prune parts of the library
code, for smaller custom distributions. For details see the readme
file.
* Character set alias management support
Additional APIs map alias+standard to a unique charset name (e.g.,
"Shift-JIS"+"IANA"->"ibm-943_P14A-2000") and enumerate all charset
names in the alias table, not just the installed ones. See
convrtrs.txt and ucnv.h.
These APIs allow programmers to avoid data corruption problems when
different platforms use the same names for different character
conversion mappings.
* EBCDIC-z/OS converter option
The EBCDIC converter now handles swapped LF/NL mappings
algorithmically instead of with modified .ucm/.cnv conversion table
files. This makes this behavior available for all supported EBCDIC
conversions without adding to the data package size. See "swaplfnl" in
convrtrs.txt.
* Additional converter
A new converter implementation has been added for the encoding of IMAP
mailbox names. See RFC 2060/5.1.3. Mailbox International Naming
Convention and "IMAP-mailbox-name" in convrtrs.txt.
* Customizable break iteration
ICU 2.4 allows registration of a BreakIterator with a locale ID. This
allows applications to provide more sophisticated word/sentence break
engines and use them seamlessly with the ICU APIs. In future releases,
this registration mechanism will be extended to all relevant ICU
services. If you are interested in ICU customization, please try out
this feature.
* Collation performance
ICU 2.4 collation was improved in several areas, with an emphasis on
performance:
* Latin-1: Improved performance of u_strcoll().
* Russian/Cyrillic: Improved performance by tailoring collation for
cyrillic-script languages, removing UCA contractions that are not
used for modern Russian (this uses the [suppressContractions]
tailoring option).
* Korean: Improved performance by resolving collation elements for
modern Hangul syllables at build time (this uses the [optimize]
tailoring option).
* Japanese: The default strength for Japanese was reduced from
quaternary to tertiary as in all other locales.
* UnicodeSet performance
UnicodeSet performance is significantly improved, especially for
add(codePoint) and contains(codePoint).
* Unicode property aliases ICU 2.4 introduces APIs for mapping between
all appropriate Unicode property aliases and property value aliases
and ICU property enumeration constants. See u_getPropertyName() etc.
in uchar.h.
* Unicode string functions
* There are new C functions for searching for last occurrences of
characters and partial strings. See u_strrstr(), u_strrchr32()
etc.
* New C/C++/Java functions for efficient checking if a string
contains more than a certain number of code points. See
hasMoreChar32Than().
* Copying UnicodeStrings via the standard assignment operator and
copy constructor does not preserve readonly aliasing any more
because this can sometimes have unexpected and dangerous effects.
A new fastCopyFrom() member function provides the old copy
semantics. See Jitterbug 1794 for more details.
* UTF macros simplified
The low-level C macros for handling code points in 8-bit and 16-bit
Unicode strings have been replaced by a simpler, more consistent set
with more concise names. For details see utf_old.h and utf.h.
Similarly, ICU 2.4 defines the UChar32 consistently (now always as
int32_t) and adds a U_SENTINEL non-code point value for new APIs.
* Performance tests
ICU 2.4 has a new performance test framework and additional
performance tests using this framework. This is not currently
documented, but it is available as part of the source distribution at
source/test/perf/.
