what is new for perl v5.30.0
Core Enhancements
Limited variable length lookbehind in regular expression pattern matching is now experimentally supported
Using a lookbehind assertion (like "(?<=foo?)" or "(?<!ba{1,9}r)" previously would generate an error and
refuse to compile. Now it compiles (if the maximum lookbehind is at most 255 characters), but raises a
warning in the new "experimental::vlb" warnings category. This is to caution you that the precise behavior
is subject to change based on feedback from use in the field.
See "(?<=pattern)" in perlre and "(?<!pattern)" in perlre.
The upper limit "n" specifiable in a regular expression quantifier of the form "{m,n}" has been doubled to 65534
The meaning of an unbounded upper quantifier "{m,}" remains unchanged. It matches 2**31 - 1 times on most
platforms, and more on ones where a C language short variable is more than 4 bytes long.
Unicode 12.1 is supported
Because of a change in Unicode release cycles, Perl jumps from Unicode 10.0 in Perl 5.28 to Unicode 12.1 in
Perl 5.30.
For details on the Unicode changes, see <https://www.unicode.org/versions/Unicode11.0.0/> for 11.0;
<https://www.unicode.org/versions/Unicode12.0.0/> for 12.0; and
<https://www.unicode.org/versions/Unicode12.1.0/> for 12.1. (Unicode 12.1 differs from 12.0 only in the
addition of a single character, that for the new Japanese era name.)
The Word_Break property, as in past Perl releases, remains tailored to behave more in line with expectations
of Perl users. This means that sequential runs of horizontal white space characters are not broken apart,
but kept as a single run. Unicode 11 changed from past versions to be more in line with Perl, but it left
several white space characters as causing breaks: TAB, NO BREAK SPACE, and FIGURE SPACE (U+2007). We have
decided to continue to use the previous Perl tailoring with regards to these.
Wildcards in Unicode property value specifications are now partially supported
You can now do something like this in a regular expression pattern
qr! \p{nv= /(?x) \A [0-5] \z / }!
which matches all Unicode code points whose numeric value is between 0 and 5 inclusive. So, it could match
the Thai or Bengali digits whose numeric values are 0, 1, 2, 3, 4, or 5.
This marks another step in implementing the regular expression features the Unicode Consortium suggests.
Most properties are supported, with the remainder planned for 5.32. Details are in "Wildcards in Property
Values" in perlunicode.
qr'\N{name}' is now supported
Previously it was an error to evaluate a named character "\N{...}" within a single quoted regular expression
pattern (whose evaluation is deferred from the normal place). This restriction is now removed.
Turkic UTF-8 locales are now seamlessly supported
Turkic languages have different casing rules than other languages for the characters "i" and "I". The
uppercase of "i" is LATIN CAPITAL LETTER I WITH DOT ABOVE (U+0130); and the lowercase of "I" is LATIN SMALL
LETTER DOTLESS I (U+0131). Unicode furnishes alternate casing rules for use with Turkic languages.
Previously, Perl ignored these, but now, it uses them when it detects that it is operating under a Turkic
UTF-8 locale.
It is now possible to compile perl to always use thread-safe locale operations.
Previously, these calls were only used when the perl was compiled to be multi-threaded. To always enable
them, add
-Accflags='-DUSE_THREAD_SAFE_LOCALE'
to your Configure flags.
Eliminate opASSIGN macro usage from core
This macro is still defined but no longer used in core
"-Drv" now means something on "-DDEBUGGING" builds
Now, adding the verbose flag ("-Dv") to the "-Dr" flag turns on all possible regular expression debugging.
Incompatible Changes
Assigning non-zero to $[ is fatal
Setting $[ to a non-zero value has been deprecated since Perl 5.12 and now throws a fatal error. See
"Assigning non-zero to $[ is fatal" in perldeprecation.
Delimiters must now be graphemes
See "Use of unassigned code point or non-standalone grapheme for a delimiter." in perldeprecation
Some formerly deprecated uses of an unescaped left brace "{" in regular expression patterns are now illegal
But to avoid breaking code unnecessarily, most instances that issued a deprecation warning, remain legal and
now have a non-deprecation warning raised. See "Unescaped left braces in regular expressions" in
perldeprecation.
Previously deprecated sysread()/syswrite() on :utf8 handles is now fatal
Calling sysread(), syswrite(), send() or recv() on a ":utf8" handle, whether applied explicitly or
implicitly, is now fatal. This was deprecated in perl 5.24.
There were two problems with calling these functions on ":utf8" handles:
o All four functions only paid attention to the ":utf8" flag. Other layers were completely ignored, so a
handle with ":encoding(UTF-16LE)" layer would be treated as UTF-8. Other layers, such as compression
are completely ignored with or without the ":utf8" flag.
o sysread() and recv() would read from the handle, skipping any validation by the layers, and do no
validation of their own. This could lead to invalidly encoded perl scalars.
my() in false conditional prohibited
Declarations such as "my $x if 0" are no longer permitted.
Fatalize $* and $#
These special variables, long deprecated, now throw exceptions when used.
Fatalize unqualified use of dump()
The "dump()" function, long discouraged, may no longer be used unless it is fully qualified, i.e.,
"CORE::dump()".
Remove File::Glob::glob()
The "File::Glob::glob()" function, long deprecated, has been removed and now throws an exception which
advises use of "File::Glob::bsd_glob()" instead.
"pack()" no longer can return malformed UTF-8
It croaks if it would otherwise return a UTF-8 string that contains malformed UTF-8. This protects against
potential security threats. This is considered a bug fix as well.
Any set of digits in the Common script are legal in a script run of another script
There are several sets of digits in the Common script. "[0-9]" is the most familiar. But there are also
"[\x{FF10}-\x{FF19}]" (FULLWIDTH DIGIT ZERO - FULLWIDTH DIGIT NINE), and several sets for use in
mathematical notation, such as the MATHEMATICAL DOUBLE-STRUCK DIGITs. Any of these sets should be able to
appear in script runs of, say, Greek. But the design of 5.30 overlooked all but the ASCII digits "[0-9]",
so the design was flawed. This has been fixed, so is both a bug fix and an incompatibility.
All digits in a run still have to come from the same set of ten digits.
JSON::PP enables allow_nonref by default
As JSON::XS 4.0 changed its policy and enabled allow_nonref by default, JSON::PP also enabled allow_nonref
by default.
Deprecations
In XS code, use of various macros dealing with UTF-8.
This deprecation was scheduled to become fatal in 5.30, but has been delayed to 5.32 due to problems that
showed up with some CPAN modules. For details of what's affected, see perldeprecation.
Performance Enhancements
o Translating from UTF-8 into the code point it represents now is done via a deterministic finite
automaton, speeding it up. As a typical example, "ord("\x7fff")" now requires 12% fewer instructions
than before. The performance of checking that a sequence of bytes is valid UTF-8 is similarly improved,
again by using a DFA.
o Eliminate recursion from finalize_op().
o A handful of small optimizations related to character folding and character classes in regular
expressions.
o Optimization of "IV" to "UV" conversions.
o Speed up of the integer stringification algorithm by processing two digits at a time instead of one.
o Improvements based on LGTM analysis and recommendation.
o Code optimizations in regcomp.c, regcomp.h, regexec.c.
o Regular expression pattern matching of things like "qr/[^a]/" is significantly sped up, where a is any
ASCII character. Other classes can get this speed up, but which ones is complicated and depends on the
underlying bit patterns of those characters, so differs between ASCII and EBCDIC platforms, but all case
pairs, like "qr/[Gg]/" are included, as is "[^01]".
Pkgsrc changes:
* Add candidate fix from https://rt.cpan.org/Public/Bug/Display.html?id=72467
* Remove patches which have been integrated upstream
* Rename and re-mould some patches which required adjustments
http://perlnews.org/2016/05/perl-5-24-released/ has pointer to
more details and says:
May 9 2016
Perl 5.24.0 has been released.
You can read about the changes which include:
Postfix dereferencing is no longer experimental
Unicode 8.0 is now supported
The autoderef feature has been removed
Perl 5.24.0 represents approximately 11 months of development since
Perl 5.22.0 and contains approximately 360,000 lines of changes
across 1,800 files from 77 authors.
The find-prefix infrastructure was required in a pkgviews world where
packages installed from pkgsrc could have different installation
prefixes, and this was a way for a dependency prefix to be determined.
Now that pkgviews has been removed there is no longer any need for the
overhead of this infrastructure. Instead we use BUILDLINK_PREFIX.pkg
for dependencies pulled in via buildlink, or LOCALBASE/PREFIX where the
dependency is coming from pkgsrc.
Provides a reasonable performance win due to the reduction of `pkg_info
-qp` calls, some of which were redundant anyway as they were duplicating
the same information provided by BUILDLINK_PREFIX.pkg.
* A safer ARGV
* CGI.pm and Module::Build disappear from core
* Hexadecimal floating point values
* Variable aliases
* Repetition in list assignment
* List pipe opens on Win32
* Various small fixes
Experimental Subroutine signatures
subs now take a prototype attribute
More consistent prototype parsing
rand now uses a consistent random number generator
New slice syntax
Experimental Postfix Dereferencing
Unicode 6.3 now supported
New \p{Unicode} regular expression pattern property
Better 64-bit support
use locale now works on UTF-8 locales
use locale now compiles on systems without locale ability
More locale initialization fallback options
-DL runtime option now added for tracing locale setting
-F now implies -a and -a implies -n
$a and $b warnings exemption
* In module.mk, pthread is marked as direct dependency unconditionally for
packages using module.mk. It is unwanted behavior.
* Include pthread.buildlink3.mk in buildlink3.mk only for the case perl5 is
built with "threads".
Fixes potential unwanted dependeny on pthread using perl5/module.mk.
18 of Perl 5.
You can find a full list of changes in the file "perldelta.pod" located in
the "pod" directory inside the release and on the web.
Perl v5.18.0 represents approximately 12 months of development since Perl
v5.16.0 and contains approximately 400,000 lines of changes across 2,100
files from 113 authors.
Enhancements were made to code features such as "use", which now explicitly enables feature bundles, a more consistent "eval" operator, and a revamped "substr" function. The "__SUB__" sequence now returns a reference to the current subroutine, allowing for recursive subroutines to be written more simply. Support for Unicode 6.1 was improved. Many other minor improvements and bugfixes, along with several deprecations, were made
* Several modules were updated to fix minor bugs.
* Tests were fixed.
* Documentation was improved.
Changes 5.14.0:
* Unicode version 6.0 is now supported.
* Regular expressions can now be reset to the default modifiers.
* Four new regular expression modifiers, "/d", "/l" , "/u", and "/a", were
introduced.
* The substitution operators now support the modifier "/r" to work
non-destructively, leaving the original unmodified.
* Larger octals can now be specified.
* Array and hash operators can now experimentally operate directly on hard
references.
* printf-like functions now understand post-1980 size modifiers.
* Exception handling was made more reliable and consistent.
* Other enhancements and many bug fixes were made
PkgSrc:
* Fixed configuring on Mac OS X 10.7.
pkgsrc changes:
- preparing for databases/p5-gdbm (using Makefile.common)
- supporting external access to perl install dirs (dirs.mk)
- fixing some errors/warnings from pkglint
- adding conflict to packages of CPAN modules which are in Perl5 code
on the one hand and are broken with perl-5.10+
Bumping PKGREVISION (dependencies changed, rebuild highly recommended).
Recursive rebuild is not required.
Pkgsrc changes:
- Adjust some patches
- Remove patches integrated upstream
- Use Perl's "Policy.sh" framework (along the way remove some patches)
- Link in an enterprise friendly way on HPUX
- in buildlink3.mk bump minimum required pkgsrc Perl version to
5.12.0
- placate some pkglint warnings
Upstream changes:
- From version 5.12.1 announce
This release contains minor bug fixes and updates of several core
modules, as well as minor documentation updates. It should be fully
backward compatible with Perl 5.12.0.
- From version 5.12.0 announce:
Perl 5.12.0 represents approximately two years of development since
version 5.10.0 and contains over 750,000 lines of changes across over
3,000 files from over 200 authors and committers.
[...]
Notable changes in Perl 5.12 include:
* Perl now conforms much more closely to the Unicode standard.
Additionally, this release includes an upgrade to version
5.2 of the standard.
* New experimental APIs allow developers to extend Perl with
"pluggable" keywords and syntax.
* Perl now has a better sense of time and will be able to keep
accurate time well past the "Y2038" barrier.
* New syntax allows developers to specify package version numbers
directly in "package" statements
* Perl now warns the user about the use of deprecated features
by default.
- For a complete list of changes see included perl5120delta(1) and
perl5121delta(1) or http://perldoc.perl.org/perl5120delta.html and
http://perldoc.perl.org/perl5121delta.html
This changes the buildlink3.mk files to use an include guard for the
recursive include. The use of BUILDLINK_DEPTH, BUILDLINK_DEPENDS,
BUILDLINK_PACKAGES and BUILDLINK_ORDER is handled by a single new
variable BUILDLINK_TREE. Each buildlink3.mk file adds a pair of
enter/exit marker, which can be used to reconstruct the tree and
to determine first level includes. Avoiding := for large variables
(BUILDLINK_ORDER) speeds up parse time as += has linear complexity.
The include guard reduces system time by avoiding reading files over and
over again. For complex packages this reduces both %user and %sys time to
half of the former time.
explaining that this should prevent a repeat of the current problem where
binary perl packages built for 5.8.8 can be installed together with
perl 5.10.0 without any warning/error. By the perl numbering scheme,
perl 5.12.0 will be the next maintenance version which installs modules
in a different path than what's used for 5.10.0.
The contents of perl itself doesn't change as a result of this change,
so no revision bump, but a revision bump for all packages which depend
directly on perl is forthcoming, as a workaround for the 5.8.8 -> 5.10.0
transition.
at using p5-* packages built with the new perl from being installed
together with an older version of perl. The p5-* packages will not
work because the new and old perls install modules in different
directories. As a consequence, bump package revision.
and add a new helper target and script, "show-buildlink3", that outputs
a listing of the buildlink3.mk files included as well as the depth at
which they are included.
For example, "make show-buildlink3" in fonts/Xft2 displays:
zlib
fontconfig
iconv
zlib
freetype2
expat
freetype2
Xrender
renderproto
the pkglint warning:
As {INSTALL,DEINSTALL}_TEMPLATE is modified using "+=", its name
should indicate plural.
This does make the variables a bit more suggestive of the fact that they
hold lists of values.
RECOMMENDED is removed. It becomes ABI_DEPENDS.
BUILDLINK_RECOMMENDED.foo becomes BUILDLINK_ABI_DEPENDS.foo.
BUILDLINK_DEPENDS.foo becomes BUILDLINK_API_DEPENDS.foo.
BUILDLINK_DEPENDS does not change.
IGNORE_RECOMMENDED (which defaulted to "no") becomes USE_ABI_DEPENDS
which defaults to "yes".
Added to obsolete.mk checking for IGNORE_RECOMMENDED.
I did not manually go through and fix any aesthetic tab/spacing issues.
I have tested the above patch on DragonFly building and packaging
subversion and pkglint and their many dependencies.
I have also tested USE_ABI_DEPENDS=no on my NetBSD workstation (where I
have used IGNORE_RECOMMENDED for a long time). I have been an active user
of IGNORE_RECOMMENDED since it was available.
As suggested, I removed the documentation sentences suggesting bumping for
"security" issues.
As discussed on tech-pkg.
I will commit to revbump, pkglint, pkg_install, createbuildlink separately.
Note that if you use wip, it will fail! I will commit to pkgsrc-wip
later (within day).
INSTALL/DEINSTALL script creation within pkgsrc.
If an INSTALL or DEINSTALL script is found in the package directory,
it is automatically used as a template for the pkginstall-generated
scripts. If instead, they should be used simply as the full scripts,
then the package Makefile should set INSTALL_SRC or DEINSTALL_SRC
explicitly, e.g.:
INSTALL_SRC= ${PKGDIR}/INSTALL
DEINSTALL_SRC= # emtpy
As part of the restructuring of the pkginstall framework internals,
we now *always* generate temporary INSTALL or DEINSTALL scripts. By
comparing these temporary scripts with minimal INSTALL/DEINSTALL
scripts formed from only the base templates, we determine whether or
not the INSTALL/DEINSTALL scripts are actually needed by the package
(see the generate-install-scripts target in bsd.pkginstall.mk).
In addition, more variables in the framework have been made private.
The *_EXTRA_TMPL variables have been renamed to *_TEMPLATE, which are
more sensible names given the very few exported variables in this
framework. The only public variables relating to the templates are:
INSTALL_SRC INSTALL_TEMPLATE
DEINSTALL_SRC DEINSTALL_TEMPLATE
HEADER_TEMPLATE
The packages in pkgsrc have been modified to reflect the changes in
the pkginstall framework.
because some Perl modules make some (bad) assumptions about the
structure of a MakeMaker-generated Makefile. Instead, remove the
perllocal.pod file whenever a p5-* module or perl itself is removed.
While here, rename some of the install/deinstall templates to more
descriptive names.
Bump the PKGREVISION to 3.
for many "core" modules, UTF-8 and Unicode bugfixes, and ithreads
bugfixes.
The major changes are in the pkgsrc infrastructure to handle Perl and
Perl modules. All pkgsrc-installed Perl modules are now installed in
"vendor" directories, and the perl interpreter has been modifed to
search for libraries in the following order: site, vendor, perl. The
Perl library is stored in a directory that is named for the Perl ABI
version associated with the Perl release, so any updates of Perl to
newer versions can be done "in-place" as long as Perl ABI version
remains the same. All Perl scripts and man pages are stored in
locations that won't conflict between site, vendor, and perl modules,
and a new utility perllink(1) now manages symlinks to those scripts
and man pages under the usual ${LOCALBASE}/bin and ${LOCALBASE}/man/man1.
PERL5_SITEPREFIX may be set to the prefix where local, site-specific
modules will be installed, e.g. PERL5_SITEPREFIX=/usr/local. Note
that modules installed here are completely unmanaged by pkgsrc.
Update the buildlink and tool dependencies on perl to require perl>=5.8.7
to reflect the new locations for Perl modules and the Perl shared
library.
around at either build-time or at run-time is:
USE_TOOLS+= perl # build-time
USE_TOOLS+= perl:run # run-time
Also remove some places where perl5/buildlink3.mk was being included
by a package Makefile, but all that the package wanted was the Perl
executable.
run-time dependency (DEPENDS) on a tool is to append a ":run" modifier
to the tool name, e.g.,
USE_TOOLS+= perl:run
Tools without modifiers or with an explicit ":build" modifier will
cause build dependencies (BUILD_DEPENDS) on those tools to be added.
This makes the notation a bit more compact.
probably has something to do with how make deals with short-circuit
evaluation of conditionals, but it's a simple matter to split the
conditional since USE_PERL5 is eventually going away.
Plan:
(1) Change USE_PERL5=build into USE_TOOLS+=perl.
(2) Change all other USE_PERL5 into including perl5/buildlink3.mk.
Possibly, for packages that don't actually build anything with perl,
but merely require it for the perl interpreter, we can instead do:
USE_TOOLS+= perl
TOOLS_DEPMETHOD.perl= DEPENDS
but this is more verbose than simply including the perl5/buildlink3.mk
file.
Move the PERL5_REQD computation into a lang/perl5/version.mk file,
and only do the USE_PERL5 logic in bsd.pkg.use.mk if we're not using
the new tools framework. This consolidates all of the perl-handling
into two places -- lang/perl5 and mk/tools/perl.mk.
using them, as these values are referenced till later when the variables
will definitely have the correct value. This removes the need to
include vars.mk here.
does. This allows us to use dynamic PLISTs for Perl modules that are
built using Module::Build. Bump the PKGREVISION of p5-Module-Build
to 1.
* Drop the use of PERL5_USES_MODULE_BUILD and introduce a new variable
PERL5_MODULE_TYPE that is either "MakeMaker" or "Module::Build" that
names the framework used to build/install the module.
* Split out the variables set in perl5/buildlink3.mk that are also used
by perl5/module.mk into a new file perl5/vars.mk. Move some PERL5_*
variable definitions from pkgsrc/mk/bsd.pkg.use.mk into perl5/vars.mk.
This just centralizes the common PERL5_* definitions into a single
file location.
* Convert the known packages that use Module::Build to set
PERL5_MODULE_TYPE and PERL5_PACKLIST:
devel/p5-Class-Container
devel/p5-Exception-Class
devel/p5-Log-Dispatch
devel/p5-Array-Compare
textproc/p5-Pod-Coverage
www/p5-Apache-Session-Wrapper
www/p5-MasonX-Request-WithApacheSession
module directory has changed (eg. "darwin-2level" vs.
"darwin-thread-multi-2level").
binary packages of perl modules need to be distinguishable between
being built against threaded perl and unthreaded perl, so bump the
PKGREVISION of all perl module packages and introduce
BUILDLINK_RECOMMENDED for perl as perl>=5.8.5nb5 so the correct
dependencies are registered and the binary packages are distinct.
addresses PR pkg/28619 from H. Todd Fujinaka.
installation needed when including perl5/buildlink3.mk. The only
option currently supported is "threads", which implies that a perl
that supports threads is required. The requirement is checked at
pre-install time using an INSTALL script template by both the package
build and the binary package.
Add PERL5_OPTIONS+=threads to both devel/p5-SDL and graphics/p5-GD
since those modules require a perl that supports threads.
