paexec:
- add new option -0. It works just like in "xargs -0".
- add new option -J.
- add new option -mw=.
- fix help message display by -h.
- -md= now allows no delimiter mode in -g mode.
- -c and -C override each other if one is implied after another.
Add new tool "paargs". It is a wrapper over paexec(1) that
simplifies use of paexec.
Fix transport_broken_rnd test script.
This fixes regression test on Solaris.
Update man page for paexec(1).
SLURM is an open-source resource manager designed for Linux clusters of
all sizes. It provides three key functions. First it allocates exclusive
and/or non-exclusive access to resources (computer nodes) to users for some
duration of time so they can perform work. Second, it provides a framework
for starting, executing, and monitoring work (typically a parallel job) on
a set of allocated nodes. Finally, it arbitrates contention for resources
by managing a queue of pending work.
Renamed from parallel/slurm.
OK wiz@
Changes since 2.6.4:
Adds additional capabilities such as SQL accounting and job profiling
Change maintainer to bacon@NetBSD.org
Install example Linux init scripts
The actual fix as been done by "pkglint -F */*/buildlink3.mk", and was
reviewed manually.
There are some .include lines that still are indented with zero spaces
although the surrounding .if is indented. This is existing practice.
1.10.7:
- Fix bug in TCP BTL that impacted performance on 10GbE (and faster)
networks by not adjusting the TCP send/recv buffer sizes and using
system default values
- Add missing MPI_AINT_ADD and MPI_AINT_DIFF function delcarations in
mpif.h
- Fixed time reported by MPI_WTIME; it was previously reported as
dependent upon the CPU frequency.
- Fix platform detection on FreeBSD
- Fix a bug in the handling of MPI_TYPE_CREATE_DARRAY in
MPI_(R)(GET_)ACCUMULATE
- Fix openib memory registration limit calculation
- Add missing MPI_T_PVAR_SESSION_NULL in mpi.h
- Fix "make distcheck" when using external hwloc and/or libevent packages
- Add latest ConnectX-5 vendor part id to OpenIB device params
- Fix race condition in the UCX PML
- Fix signal handling for rsh launcher
- Fix Fortran compilation errors by removing MPI_SIZEOF in the Fortran
interfaces when the compiler does not support it
- Fixes for the pre-ignore-TKR "mpi" Fortran module implementation
(i.e., for older Fortran compilers -- these problems did not exist
in the "mpi" module implementation for modern Fortran compilers):
- Add PMPI_* interfaces
- Fix typo in MPI_FILE_WRITE_AT_ALL_BEGIN interface name
- Fix typo in MPI_FILE_READ_ORDERED_BEGIN interface name
- Fixed the type of MPI_DISPLACEMENT_CURRENT in all Fortran interfaces
to be an INTEGER(KIND=MPI_OFFSET_KIND).
- Fixed typos in MPI_INFO_GET_* man pages. Thanks to Nicolas Joly for
the patch
- Fix typo bugs in wrapper compiler script
Unsorted entries in PLIST files have generated a pkglint warning for at
least 12 years. Somewhat more recently, pkglint has learned to sort
PLIST files automatically. Since pkglint 5.4.23, the sorting is only
done in obvious, simple cases. These have been applied by running:
pkglint -Cnone,PLIST -Wnone,plist-sort -r -F
This has been a pkglint warning for several years now, and pkglint can even
fix it automatically. And it did for this commit.
Only in lang/mercury, two passes of autofixing were necessary because there
were nested variables.
- Adds Process._authkey alias to .authkey for 2.7 compat.
- Remove superfluous else clause from max_memory_per_child_check.
- Document and test all supported Python versions.
- Extend 'Process' to be compatible with < Py3.5.
- Use a properly initialized logger in pool.py error logging.
- _trywaitkill can now kill a whole process group if the worker process declares itself as a group leader.
- Fix cpython issue 14881 (See http://bugs.python.org/issue14881).
- Fix for a crash on windows.
- Fix messaging in case of worker exceeds max memory.
* Added support for MPI-3.1 features including nonblocking collective I/O,
address manipulation routines, thread-safety for MPI initialization,
pre-init functionality, and new MPI_T routines to look up variables
by name.
* Fortran 2008 bindings are enabled by default and fully supported.
* Added support for the Mellanox MXM InfiniBand interface. (thanks
to Mellanox for the code contribution).
* Added support for the Mellanox HCOLL interface for collectives.
(thanks to Mellanox for the code contribution).
* Significant stability improvements to the MPICH/portals4
implementation.
* Completely revamped RMA infrastructure including several
scalability improvements, performance improvements, and bug fixes.
* Added experimental support for Open Fabrics Interfaces (OFI) version 1.0.0.
https://github.com/ofiwg/libfabric (thanks to Intel for code contribution)
* The Myrinet MX network module, which had a life cyle from 1.1 till
3.1.2, has now been deleted.
* Several other minor bug fixes, memory leak fixes, and code cleanup.
--------------
* Fix hwloc-bind --membind for CPU-less NUMA nodes (again).
Thanks to Gilles Gouaillardet for reporting the issue.
* Fix a memory leak on IBM S/390 platforms running Linux.
* Fix a memory leak when forcing the x86 backend first on amd64/topoext
platforms running Linux.
* Command-line tools now support "hbm" instead "numanode" for filtering
only high-bandwidth memory nodes when selecting locations.
+ hwloc-bind also support --hbm and --no-hbm for filtering only or
no HBM nodes.
* Add --children and --descendants to hwloc-info for listing object
children or object descendants of a specific type.
* Add --no-index, --index, --no-attrs, --attrs to disable/enable display
of index numbers or attributes in the graphical lstopo output.
* Try to gather hwloc-dump-hwdata output from all possible locations
in hwloc-gather-topology.
* Updates to the documentation of locations in hwloc(7) and
command-line tools manpages.
- max_memory_per_child was measured in kilobytes on Linux, but bytes on
*BSD/MacOS, it's now always kilobytes.
- Windows: Adds support for max_memory_per_child, but requires the
``psutil`` package to be installed.
- Fixed bug in ForkingPickler.loadbuf, where it tried to pass
a BytesIO instance directly to ``pickle.loads`` on Python 2.7.
MASTER_SITES= site1 \
site2
style continuation lines to be simple repeated
MASTER_SITES+= site1
MASTER_SITES+= site2
lines. As previewed on tech-pkg. With thanks to rillig for fixing pkglint
accordingly.
Prompted by Nicolas Joly in private mail.
1.10.4 - 01 Sept 2016
------
- Fix assembler support for MIPS
- Improve memory handling for temp buffers in collectives
- Fix [all]reduce with non-zero lower bound datatypes
Thanks Hristo Iliev for the report
- Fix non-standard ddt handling. Thanks Yuki Matsumoto for the report
- Various libnbc fixes. Thanks Yuki Matsumoto for the report
- Fix typos in request RMA bindings for Fortran. Thanks to @alazzaro
and @vondele for the assist
- Various bug fixes and enhancements to collective support
- Fix predefined types mapping in hcoll
- Revive the coll/sync component to resolve unexpected message issues
during tight loops across collectives
- Fix typo in wrapper compiler for Fortran static builds
1.10.3 - 15 June 2016
------
- Fix zero-length datatypes. Thanks to Wei-keng Liao for reporting
the issue.
- Minor manpage cleanups
- Implement atomic support in OSHMEM/UCX
- Fix support of MPI_COMBINER_RESIZED. Thanks to James Ramsey
for the report
- Fix computation of #cpus when --use-hwthread-cpus is used
- Add entry points for Allgatherv, iAllgatherv, Reduce, and iReduce
for the HCOLL library
- Fix an HCOLL integration bug that could signal completion of request
while still being worked
- Fix computation of cores when SMT is enabled. Thanks to Ben Menadue
for the report
- Various USNIC fixes
- Create a datafile in the per-proc directory in order to make it
unique per communicator. Thanks to Peter Wind for the report
- Fix zero-size malloc in one-sided pt-to-pt code. Thanks to Lisandro
Dalcin for the report
- Fix MPI_Get_address when passed MPI_BOTTOM to not return an error.
Thanks to Lisandro Dalcin for the report
- Fix MPI_TYPE_SET_ATTR with NULL value. Thanks to Lisandro Dalcin for
the report
- Fix various Fortran08 binding issues
- Fix memchecker no-data case. Thanks to Clinton Stimpson for the report
- Fix CUDA support under OS-X
- Fix various OFI/MTL integration issues
- Add MPI_T man pages
- Fix one-sided pt-to-pt issue by preventing communication from happening
before a target enters a fence, even in the no-precede case
- Fix a bug that disabled Totalview for MPMD use-case
- Correctly support MPI_UNWEIGHTED in topo-graph-neighbors. Thanks to
Jun Kudo for the report
- Fix singleton operations under SLURM when PMI2 is enabled
- Do not use MPI_IN_PLACE in neighborhood collectives for non-blocking
collectives (libnbc). Thanks to Jun Kudo for the report
- Silence autogen deprecation warnings for newer versions of Perl
- Do not return MPI_ERR_PENDING from collectives
- Use type int* for MPI_WIN_DISP_UNIT, MPI_WIN_CREATE_FLAVOR, and MPI_WIN_MODEL.
Thanks to Alastair McKinstry for the report
- Fix register_datarep stub function in IO/OMPIO. Thanks to Eric
Chamberland for the report
- Fix a bus error on MPI_WIN_[POST,START] in the shared memory one-sided component
- Add several missing MPI_WIN_FLAVOR constants to the Fortran support
- Enable connecting processes from different subnets using the openib BTL
- Fix bug in basic/barrier algorithm in OSHMEM
- Correct process binding for the --map-by node case
- Include support for subnet-to-subnet routing over InfiniBand networks
- Fix usnic resource check
- AUTHORS: Fix an errant reference to Subversion IDs
- Fix affinity for MPMD jobs running under LSF
- Fix many Fortran binding bugs
- Fix `MPI_IN_PLACE`-related bugs
- Fix PSM/PSM2 support for singleton operations
- Ensure MPI transports continue to progress during RTE barriers
- Update HWLOC to 1.9.1 end-of-series
- Fix a bug in the Java command line parser when the
-Djava.library.path options was given by the user
- Update the MTL/OFI provider selection behavior
- Add support for clock_gettime on Linux.
- Correctly detect and configure for Solaris Studio 12.5
beta compilers
- Correctly compute #slots when -host is used for MPMD case
- Fix a bug in the hcoll collectives due to an uninitialized field
- Do not set a binding policy when oversubscribing a node
- Fix hang in intercommunicator operations when oversubscribed
- Speed up process termination during MPI_Abort
- Disable backtrace support by default in the PSM/PSM2 libraries to
prevent unintentional conflicting behavior.
1.10.2: 26 Jan 2016
-------------------
**********************************************************************
* OSHMEM is now 1.2 compliant
**********************************************************************
- Fix NBC_Copy for legitimate zero-size messages
- Fix multiple bugs in OSHMEM
- Correctly handle mpirun --host <user>@<ip-address>
- Centralize two MCA params to avoid duplication between OMPI and
OSHMEM layers: opal_abort_delay and opal_abort_print_stack
- Add support for Fujitsu compilers
- Add UCX support for OMPI and OSHMEM
- Correctly handle oversubscription when not given directives
to permit it. Thanks to @ammore1 for reporting it
- Fix rpm spec file to not include the /usr directory
- Add Intel HFI1 default parameters for the openib BTL
- Resolve symbol conflicts in the PSM2 library
- Add ability to empty the rgpusm cache when full if requested
- Fix another libtool bug when -L requires a space between it
and the path. Thanks to Eric Schnetter for the patch.
- Add support for OSHMEM v1.2 APIs
- Improve efficiency of oshmem_preconnect_all algorithm
- Fix bug in buffered sends support
- Fix double free in edge case of mpirun. Thanks to @jsharpe for
the patch
- Multiple one-sided support fixes
- Fix integer overflow in the tuned "reduce" collective when
using buffers larger than INT_MAX in size
- Fix parse of user environment variables in mpirun. Thanks to
Stefano Garzarella for the patch
- Performance improvements in PSM2 support
- Fix NBS iBarrier for inter-communicators
- Fix bug in vader BTL during finalize
- Improved configure support for Fortran compilers
- Fix rank_file mapper to support default --slot-set. Thanks
to Matt Thompson for reporting it
- Update MPI_Testsome man page. Thanks to Eric Schnetter for
the suggestion
- Fix missing resize of the returned type for subarray and
darray types. Thanks to Keith Bennett and Dan Garmann for
reporting it
- Fix Java support on OSX 10.11. Thanks to Alexander Daryin
for reporting the problem
- Fix some compilation issues on Solaris 11.2. Thanks to
Paul Hargrove for his continued help in such areas
Version 1.11.4
--------------
* Add MemoryMode and ClusterMode attributes in the Machine object on KNL.
Add doc/examples/get-knl-modes.c for an example of retrieving them.
Thanks to Grzegorz Andrejczuk.
* Fix Linux build with -m32 with respect to libudev.
Thanks to Paul Hargrove for reporting the issue.
* Fix build with Visual Studio 2015, thanks to Eloi Gaudry for reporting
the issue and providing the patch.
* Don't forget to display OS device children in the graphical lstopo.
* Fix a memory leak on Solaris, thanks to Bryon Gloden for the patch.
* Properly handle realloc() failures, thanks to Bryon Gloden for reporting
the issue.
* Fix lstopo crash in ascii/fig/windows outputs when some objects have a
lstopoStyle info attribute.
Version 1.11.3
--------------
* Bug fixes
+ Fix a memory leak on Linux S/390 hosts with books.
+ Fix /proc/mounts parsing on Linux by using mntent.h.
Thanks to Nathan Hjelm for reporting the issue.
+ Fix a x86 infinite loop on VMware due to the x2APIC feature being
advertised without actually being fully supported.
Thanks to Jianjun Wen for reporting the problem and testing the patch.
+ Fix the return value of hwloc_alloc() on mmap() failure.
Thanks to Hugo Brunie for reporting the issue.
+ Fix the return value of command-line tools in some error cases.
+ Do not break individual thread bindings during x86 backend discovery in a
multithreaded process. Thanks to Farouk Mansouri for the report.
+ Fix hwloc-bind --membind for CPU-less NUMA nodes.
+ Fix some corner cases in the XML export/import of application userdata.
* API Improvements
+ Add HWLOC_MEMBIND_BYNODESET flag so that membind() functions accept
either cpusets or nodesets.
+ Add hwloc_get_area_memlocation() to check where pages are actually
allocated. Only implemented on Linux for now.
- There's no _nodeset() variant, but the new flag HWLOC_MEMBIND_BYNODESET
is supported.
+ Make hwloc_obj_type_sscanf() parse back everything that may be outputted
by hwloc_obj_type_snprintf().
* Detection Improvements
+ Allow the x86 backend to add missing cache levels, so that it completes
what the Solaris backend lacks.
Thanks to Ryan Zezeski for reporting the issue.
+ Do not filter-out FibreChannel PCI adapters by default anymore.
Thanks to Matt Muggeridge for the report.
+ Add support for CUDA compute capability 6.x.
* Tools
+ Add --support to hwloc-info to list supported features, just like with
hwloc_topology_get_support().
- Also add --objects and --topology to explicitly switch between the
default modes.
+ Add --tid to let hwloc-bind operate on individual threads on Linux.
+ Add --nodeset to let hwloc-bind report memory binding as NUMA node sets.
+ hwloc-annotate and lstopo don't drop application userdata from XMLs anymore.
- Add --cu to hwloc-annotate to drop these application userdata.
+ Make the hwloc-dump-hwdata dump directory configurable through configure
options such as --runstatedir or --localstatedir.
* Misc Improvements
+ Add systemd service template contrib/systemd/hwloc-dump-hwdata.service
for launching hwloc-dump-hwdata at boot on Linux.
Thanks to Grzegorz Andrejczuk.
+ Add HWLOC_PLUGINS_BLACKLIST environment variable to prevent some plugins
from being loaded. Thanks to Alexandre Denis for the suggestion.
+ Small improvements for various Windows build systems,
thanks to Jonathan L Peyton and Marco Atzeri.
Version 1.11.2
--------------
* Improve support for Intel Knights Landing Xeon Phi on Linux:
+ Group local NUMA nodes of normal memory (DDR) and high-bandwidth memory
(MCDRAM) together through "Cluster" groups so that the local MCDRAM is
easy to find.
- See "How do I find the local MCDRAM NUMA node on Intel Knights
Landing Xeon Phi?" in the documentation.
- For uniformity across all KNL configurations, always have a NUMA node
object even if the host is UMA.
+ Fix the detection of the memory-side cache:
- Add the hwloc-dump-hwdata superuser utility to dump SMBIOS information
into /var/run/hwloc/ as root during boot, and load this dumped
information from the hwloc library at runtime.
- See "Why do I need hwloc-dump-hwdata for caches on Intel Knights
Landing Xeon Phi?" in the documentation.
Thanks to Grzegorz Andrejczuk for the patches and for the help.
* The x86 and linux backends may now be combined for discovering CPUs
through x86 CPUID and memory from the Linux kernel.
This is useful for working around buggy CPU information reported by Linux
(for instance the AMD Bulldozer/Piledriver bug below).
Combination is enabled by passing HWLOC_COMPONENTS=x86 in the environment.
* Fix L3 cache sharing on AMD Opteron 63xx (Piledriver) and 62xx (Bulldozer)
in the x86 backend. Thanks to many users who helped.
* Fix the overzealous L3 cache sharing fix added to the x86 backend in 1.11.1
for AMD Opteron 61xx (Magny-Cours) processors.
* The x86 backend may now add the info attribute Inclusive=0 or 1 to caches
it discovers, or to caches discovered by other backends earlier.
Thanks to Guillaume Beauchamp for the patch.
* Fix the management on alloc_membind() allocation failures on AIX, HP-UX
and OSF/Tru64.
* Fix spurious failures to load with ENOMEM on AIX in case of Misc objects
below PUs.
* lstopo improvements in X11 and Windows graphical mode:
+ Add + - f 1 shortcuts to manually zoom-in, zoom-out, reset the scale,
or fit the entire window.
+ Display all keyboard shortcuts in the console.
* Debug messages may be disabled at runtime by passing HWLOC_DEBUG_VERBOSE=0
in the environment when --enable-debug was passed to configure.
* Add a FAQ entry "What are these Group objects in my topology?".
* Changes in pdsh-2.31 (2013-11-07)
===================================
-- Fix issue 56: slurm: Allow mixed use of -P, -w and -j options.
-- Fix issue 59: pdsh very slow when using a few thousand hosts
and genders.
-- testsuite: Expanded tests for genders module (Pythagoras
Watson)
* Changes in pdsh-2.30 (2013-03-02)
===================================
-- Fix issue 55: genders -X option removes more hosts than
expected.
(This was a generic fix for hostname matching, so it probably
affected -x and other options as well.)
-- testsuite: Add test for issue 55.
Upstream changes:
20160822
- Bug fixes and man page updates.
20160722
- env_parallel is now ready for wider testing. It is still beta
quality.
- env_parallel is heavily modified for all shells and testing has been
increased.
- Selectively choosing what to export using --env now works for
env_parallel (bash, csh, fish, ksh, pdksh, tcsh, zsh).
- --round-robin now gives more work to a job that processes faster
instead of same amount to all jobs.
- --pipepart works on block devices on GNU/Linux.
- Bug fixes and man page updates.
20160622
- $PATH can now be exported using --env PATH. Useful if GNU Parallel
is not in your path on remote machines.
- If --block is left out, --pipepart will use a block size that will
result in 10 jobs per jobslot.
- Bug fixes and man page updates.
20160522
- niceload --net pauses the program if the internet connection is
overloaded.
- Bug fixes and man page updates.
20160422
- :::+ and ::::+ work like ::: and :::: but links this input source to
the previous input source in a --xapply fashion. Contrary to
--xapply values do not wrap: The shortest input source determines
the length.
- --line-buffer --keep-order now outputs continously from the oldest
job still running. This is more what you would expect than the
earlier behaviour where --keep-order had no effect with
--line-buffer.
- env_parallel supports tcsh, csh, pdksh. In fish it now supports
arrays. In csh/tcsh it now supports variables, aliases, and arrays
with no special chars. In pdksh it supports aliases, functions,
variables, and arrays.
- Function exporting on Mac OS X works around old Bash version.
- Better CPU detection on OpenIndiana.
- Bug fixes and man page updates.
20160322
- env_parallel is a function that exports the environment (functions,
aliases, variables, and arrays) to GNU Parallel. Run 'man
env_parallel' for details.
- niceload --prg now searches for substrings if no process with the
name is found.
- Bug fixes and man page updates.
20160222
- Bug fixes and man page updates.
20160122
- --sql DBURL uses DBURL as storage for jobs and output. It does not
run any jobs so it requires at least one --sqlworker. DBURL must
point to a table.
- --sqlworker DBURL gets jobs from DBURL and stores the result back to
DBURL.
- --sqlandworker is a shorthand for --sql and --sqlworker.
- --sqlworker requires the output of a single job to fit in memory.
- --results now also saves a file called 'seq' containing the sequence
number.
- If $PARALLEL_ENV is a file, then that file will be read into
$PARALLEL_ENV.
- man parallel_tutorial has been given an overhaul.
- --workdir now accepts replacementstrings.
- Bug fixes and man page updates.
20151222
- --transfer is now an alias for --transferfile {}.
- --transferfile works like --transfer, but takes an argument like
--return. This makes it possible to combine transferring files
with multiple input sources: parallel -S server --tf {1} wc {2}
{1} ::: * ::: -l -w -c
- total_jobs() can now be used in {= =}: parallel echo job {#} of '{=
$_=total_jobs() =}' ::: {1..50}
- Bug fixes and man page updates.
20151122
- Bug fixes and man page updates.
20151022
- --plus makes it possible to use {##} as a shorthand for
{= $_=$Global::JobQueue->total_jobs() =} which gives the the number
of jobs to run in total.
- {= $_=$Global::JobQueue->total_jobs() =} is incompatible with -X,
-m, and --xargs.
- GNU Parallel is now mostly compatible with lsh
(http://www.lysator.liu.se/~nisse/lsh/) and somewhat compatible with
autossh (http://www.harding.motd.ca/autossh/).
- --workdir ... now also works when run locally.
- Bug fixes and man page updates.
20150922
- Bug fixes and man page updates.
20150822
- If $job->skip() is called in {= =} the job will not be run.
- @arg can now be used in {= =}: parallel echo '{= $arg[2] < 5 and
$_="j" =}' ::: 1 2 3 ::: 4 5 6
- Bug fixes and man page updates.
20150722
- If a record fits in a block, --block only reads upto the size of
--block. If not: --block reads a full --block more.
- Due to widely spread distributions still using --tollef even after a
year of being retired following a year of being obsolete, --tollef
now gives an error unless used with --gnu.
- --nice now uses a perl wrapper instead of the nice command.
- Bug fixes and man page updates.
20150622
- --halt has been rewritten completely. You can now combine
percentages with success or fail. See the man page.
- Exit values 102..254 have been removed. 101 means more than 100 jobs
failed.
- Killing through --timeout, --memfree, or --halt is now done as a
process group.
- --termseq determines which signals are sent when a job is killed.
- An empty argument would previously cause no string to be
inserted. This is now changed to '' being inserted, thus prepending
a space to the output of: parallel echo {} b ::: ''
- $PARALLEL_ENV can now be set to an environment prepending the
command. Used in env_parallel as mentioned in the manpage.
- --retry-failed will retry all failed jobs in a joblog. It will
ignore any command given.
- --ssh and $PARALLEL_SSH can be used to set the command used for
ssh. The command is assumed to behave the same way as ssh.
- --fifo now works in csh, too.
- Q(...) can be used in {= =} to shell quote a string.
- Bug fixes and man page updates.
20150522
- Security: The security issue for --sshlogin + --fifo/--cat has been
fixed. Thereby all issues with
http://lists.gnu.org/archive/html/parallel/2015-04/msg00045.html
have been fixed.
- Security: After further security analysis the issue fixed in
20150422 also fixed the problem for --tmux.
- Bug fixes and man page updates.
20150422
New in this release:
- Security fix. An attacker on the local system could make you
overwrite one of your own files with a single byte. The problem
exist when you use --compress or --tmux or --pipe or --cat or
--fifo. The attacker must figure out the randomly chosen file name
and create a symlink within a time window of 15 ms.
- --shuf will generate all jobs, and shuffle them before running
them. This is useful to get a quick preview of the results before
running the full batch.
- Bug fixes and man page updates.
20150322
New in this release:
- --number-of-cores respects 'taskset' on GNU/Linux.
- --joblog --pipe gives the data send and received in the log.
- Bug fixes and man page updates.
20150222
- --tmux has gotten a major overhaul.
- Bug fixes and man page updates.
20150122
- Remote jobs now send stderr (standard error) to stderr (standard
error) instead of stdout (standard output).
- Remote execution command is now packed using base64 encoded
bzip2. This means that big environments (app. 100 KB) can be
transferred. This change makes remote execution alpha quality.
- --semaphoretimeout now takes a negative value. This means give up if
the semaphore is not gotten within the timeout.
- --halt -1 and -2 now means halt if a job succeeds (so the opposite
of 1 and 2).
- --no-keep-order will reverse --keep-order.
- Bash's second fix of shellshock caused --env to break again when
exporting functions. This has been fixed again.
- A semibig refactoring of big functions. All non-trivial functions
are now less than 100 lines. The refactoring makes this release beta
quality.
- A description of the design decisions for GNU Parallel can be found
in 'man parallel_design'.
- A bug fix in replacement strings caused rewrite of the replacement
function. This makes use of replacement strings alpha quality.
- Bug fixes and man page updates.
20141122
- Remote systems can be divided into hostgroups (e.g. web and db) by
prepending '@groupname/' to the sshlogin. Multiple groups can be
given by separating groups with '+'. E.g. @web/www1 @web+db/www2
@db/mariadb
- Remote execution can be restricted to servers that are part of one
or more groups by '@groupname' as an sshlogin. Multiple groups can
be given by separating groups with '+'. E.g. -S @web or -S @db+web
- With --hostgroup you can restrict arguments to certain hostgroups by
appending '@groupname' to the argument. Multiple groups can be
given by separating groups with '+'. E.g. my_web_arg@web
db-or-web-arg@db+web db-only-arg@db Thanks to Michel Courtine for
developing a prototype for this.
- Bug fixes and man page updates.
20141022
- --env was changed to deal with Bash's new way of exporting a function.
- Bug fixes and man page updates.
20140922
- If the file give as --sshloginfile is changed it will be re-read
when a job finishes though at most once per second. This makes it
possible to add and remove hosts while running.
20140822
- GNU Parallel now uses the same shell it was started from as the
command shell for local jobs. So if GNU Parallel is started from
tcsh it will use tcsh as its shell even if the login $SHELL is
different. For remote jobs the login $SHELL will be used.
- The whole current environment in bash can be copied by using a shell
wrapper function (Search manual for env_parallel).
- --plus adds the replacement strings {+/} {+.} {+..} {+...} {..}
{...} {/..} {/...}. The idea being that '+foo' matches the opposite
of 'foo' and {} = {+/}/{/} = {.}.{+.} = {+/}/{/.}.{+.} = {..}.{+..}
= {+/}/{/..}.{+..} = {...}.{+...} = {+/}/{/...}.{+...}
- GNU Parallel now deals correctly with the combination
rsync-3.1.X-client and rsync-2.5.7-server
- Bug fixes and man page updates.
20140722
- {= perl expression =} can be used as replacement string. The
expression should modify $_. E.g. {= s/\.gz$// =} to remove .gz from
the string. This makes replacement strings extremely flexible.
- Positional perl expressions (similar to {2}) are given as {=2 perl
expression=} where 2 is the position.
- One small backwards incompatibility: {1}_{2} will replace {2} with
the empty string if there is only one argument. Previously {2} would
have been left untouched.
- Replacement strings can be defined using --rpl. E.g. parallel --rpl
'{.gz} s/\.gz$//' echo {.gz} ::: *.gz
- The parenthesis around {= perl expression =} can be changed with
--parens.
- --tmux will direct the output to a tmux session instead of
files. Each running jobs will be in its own window.
- --halt 10% will stop spawning new jobs if 10% failed so far.
- Bug fixes and man page updates.
20140622
- --shebang and --shebang-wrap now works on FreeBSD.
- --pipepart works with --header, --regexp, --cat and --fifo.
- ./configure --disable-documentation will not build documentation.
- {%} works as job slot.
- Bug fixes and man page updates.
20140522
- Use --semaphoretimeout to ignore the semaphore lock after a while.
- {%} introduced as job slot replacement string. It has known bugs.
- --slotreplace changes slot replacement string.
- Bug fixes and man page updates.
20140422
- --pipepart is a highly efficient alternative to --pipe if the input
is a real file and not a pipe.
- If using --cat or --fifo with --pipe the {} in the command will be
replaced with the name of a physical file and a fifo respectively
containing the block from --pipe. Useful for commands that cannot
read from standard input (stdin).
- --controlmaster has gotten an overhaul and is no longer
experimental.
- --env is now copied when determining CPUs on remote system. Useful
for copying $PATH if parallel is not in the normal path.
- --results now chops the argument if the argument is longer than the
allowed path length.
- Build now survives if pod2* are not installed.
- Bug fixes and man page updates.
20140322
- Bug fixes and man page updates.
20140222
- --tollef has been retired.
- --compress has be redesigned due to bugs.
- Format of --eta and --joblog has changed slightly.
- Bug fixes and man page updates.
20140122
- Bug fixes and man page updates.
20131222
- GNU Parallel now has a motto: For people who live life in the
parallel lane.
- Detect if the buffer dir $TMPDIR runs out of space. This changes a
central piece of code, making this release alpha quality.
- --keep-order --(n)onall will sort according to the sshlogin.
- Detect the number of CPUs on DEC Tru64.
- GNU sql: --list-tables for sqlite3.
- Bug fixes and man page updates.
20131122
- A citation notice is printed on stderr only if stderr is a terminal,
the user has not specified --no-notice and the user has not run
--bibtex once. This makes the release alpha quality.
- --compress will compress temporary files. If the output is big and
very compressible this will take up less disk space in $TMPDIR and
possibly be faster due to less disk I/O.
- --compress-program comtrols which program to use for compressing
temporary files.
- --bar show progress as a progress bar compatible with zenity.
- --resume can now be used with --result: Jobs already run will be
skipped.
- --transfer and --basefile support paths relative to the --workdir by
inserting /./ into the path.
- Bug fixes and man page updates.
20131022
- --transfer files with /./ in the path will copy the files relative
to the --workdir.
- The maximal command length is now cached in a file halfing the
startup time.
- Bug fixes and man page updates.
20130922
- PDF-files of documentation is now included.
- Bug fixes and man page updates.
20130822
- A tutorial showing much of GNU Parallel's functionality. Spend an
hour walking through the tutorial. Your commandline will love you
for it. man parallel_tutorial
- --line-buffer will buffer output on line basis. --group keeps the
output together for a whole job. --ungroup allows output to mixup
with half a line coming from one job and half a line coming from
another job. --line-buffer fits between these two; it prints a full
line, but will allow for mixing lines of different jobs.
- --record-env records all variables and functions set. Useful to
record the normal environment before using --env _.
- --env _ exports all functions and variables to the remote server
except for the ones recorded.
- New signing key. Due to recommendations from NIST
http://www.keylength.com/en/4/ the signing key was changed from
1024D/ID:FFFFFFF1 to 9888R/ID:88888888.
- Bug fixes and man page updates.
20130722
- --round-robin with --pipe will write all blocks to already running
jobs.
- --env can now transfer Bash function for remote execution. That is
pretty cool!
- Quite a few bug fixes and man page updates.
20130622
- --xapply now recycles arguments if an input source has more
arguments than others.
- The sleep time between jobs is now both increased and decreased
exponentially.
- 10 seconds installation check the signature using GnuPG if GnuPG is
installed.
- Bug fixes and man page updates.
20130522
- --ctrlc is now default if --pipe is not used; thus making it
possible to kill remotely started jobs simply by pressing Ctrl-C.
- --timeout 200% now means kill jobs that take more than twice the
time of the median run time of a job.
- Jobs are now distributed round robin when having mulitiple
--sshlogin instead of filling up one --sshlogin at a time.
- niceload: darwin version of --io --mem --swap. Thanks to Anders F
Bjorklund.
- Bug fixes and man page updates.
20130422
- 10 seconds installation: wget -O - pi.dk/3 | sh
- HPUX CPU/core detection. Thanks to Javier Tarradas.
- CPU detection for HURD, IRIX, SCO OpenServer and (old) HPUX.
- --ctrlc will send SIGINT to tasks running on remote computers and
thus killing them.
- --load now uses 'ps' to see immediately see the number of running
processes instead of 'uptime' thus making --load react much faster.
- Testing on Centos 3.9 and FreeBSD 7 revealed a lot of compability
bugs. Some of these required quite extensive changes making this
release beta quality.
- --retries works with --onall.
- The new --load computation now works on FreeBSD 7.
- --nice works under tcsh.
- Bug fixes and man page updates.
20130222
- --resume works with --pipe.
- --resume-failed will go through --joblog, redo the failed jobs and
then continue like --resume.
- Negative positional arguments count from the end: {-1} means the
last argument, {-2} the second to last.
- NetBSD CPU detection.
- --blocksize increases exponentially if it smaller than a full
record.
- Processing n-line records (--pipe -L n) is now much faster.
- --tollef is obsoleted after discussion on the mailing list.
- Bug fixes and man page updates.
