pkgsrc/textproc/miller/distinfo

$NetBSD: distinfo,v 1.16 2020/03/17 14:38:25 fcambus Exp $

SHA1 (mlr-5.7.0.tar.gz) = 50b18bb10bbf8a5e03f58b2ecd7809844834ed43
RMD160 (mlr-5.7.0.tar.gz) = 3b3454bdd5ff771c8521a07ccd66e5932fc67af8
SHA512 (mlr-5.7.0.tar.gz) = a801c9e6e10422cd0cc79c117ce7d49268f6d2bbe7c625fed0dd9144fcb7f5d2382a4450772f468f3d31b61912206bd8686f9c78fca2dbd372ee1e003f9cff88
Size (mlr-5.7.0.tar.gz) = 1266826 bytes
miller: update to 5.7.0. ChangeLog: Features The new remove-empty-columns and skip-trivial-records are keystroke-savers for things which would other require DSL syntax, as tracked in #274. Bugfixes A bug regarding optional regex-pattern groups was fixed in #277. As of #294 you can now specify --implicit-csv-header for the join-file in mlr join. A bug with spaces in XTAB-file values was fixed on #296. A bug with missing final newline for XTAB-formatted files using MMAP files was fixed on #301. Documentation Look-and-feel at http://johnkerl.org/miller/doc/ is (hopefully) improved, including clearer visual indication of which section/page you're currently looking at. Note that this change has been live for a few weeks, as look-and-feel-related doc-mods from post-5.6.2 were backported to http://johnkerl.org/miller/doc/. #282 improves DSL-function documentation at http://johnkerl.org/miller/doc/reference-dsl.html#Built-in_functions_for_filter_and_put,_summary Note Support for mmap mode has been entirely discontinued. This is an invisible change and should not affect you at all. For anyone interested in lower-level details, though, the summary is as follows: For an incremental performance gain (perhaps 10-20% run time at most, but see below), within the C source code one can use the mmap system call to access input files via pointer arithmetic rather than malloc-and-memcopy using stdio. However mmap is not available when reading from standard input -- it cannot be memory-mapped. This means all file-format readers are implemented twice within the Miller source code. While I try to regression-test Miller thoroughly, running all canned tests through mmap and stdio mode, I've nonetheless found my mmap implementations liable to corner-cases which I miss but users find: for example #29, #102, and #296. As tracked on #160, various operating systems do not release mmapped pages after use as one might intuit, meaning that for large files and/or large numbers of files, I've for a long time now needed to have Miller opt out of mmap usage for precisely those cases which most need the performance gain: see #160, #181, and #256. Additionally, mmap is not used at all for Windows/MSYS2 so there is nothing to lose there. For these reasons, keeping mmap mode isn't worth the development overhead. As of release 5.6.3, the mlr executable will still accept the --mmap and --no-mmap command-line flags as no-ops, for backward compatibility. The caveat for you is that for everyday small files, the default was previously mmap mode and is now stdio (except mlr ... < filename or ... \| mlr ... which have always used stdio). There is the off chance that this will newly reveal an old, latent bug or two somewhere. I've re-run regressions in valgrind mode to aggressively catch any errors, but, please let me know ASAP via GitHub issue of any unexpected behavior in 5.7.0. 2020-03-17 15:38:25 +01:00			$NetBSD: distinfo,v 1.16 2020/03/17 14:38:25 fcambus Exp $
Import miller-2.0.0 as textproc/miller. Miller is like sed, awk, cut, join, and sort for name-indexed data such as CSV. With Miller, you get to use named fields without needing to count positional indices. This is something the Unix toolkit always could have done, and arguably always should have done. It operates on key-value-pair data while the familiar Unix tools operate on integer-indexed fields: if the natural data structure for the latter is the array, then Miller's natural data structure is the insertion-ordered hash map. This encompasses a variety of data formats, including but not limited to the familiar CSV. (Miller can handle positionally-indexed data as a special case.) 2015-08-28 11:27:10 +02:00
miller: update to 5.7.0. ChangeLog: Features The new remove-empty-columns and skip-trivial-records are keystroke-savers for things which would other require DSL syntax, as tracked in #274. Bugfixes A bug regarding optional regex-pattern groups was fixed in #277. As of #294 you can now specify --implicit-csv-header for the join-file in mlr join. A bug with spaces in XTAB-file values was fixed on #296. A bug with missing final newline for XTAB-formatted files using MMAP files was fixed on #301. Documentation Look-and-feel at http://johnkerl.org/miller/doc/ is (hopefully) improved, including clearer visual indication of which section/page you're currently looking at. Note that this change has been live for a few weeks, as look-and-feel-related doc-mods from post-5.6.2 were backported to http://johnkerl.org/miller/doc/. #282 improves DSL-function documentation at http://johnkerl.org/miller/doc/reference-dsl.html#Built-in_functions_for_filter_and_put,_summary Note Support for mmap mode has been entirely discontinued. This is an invisible change and should not affect you at all. For anyone interested in lower-level details, though, the summary is as follows: For an incremental performance gain (perhaps 10-20% run time at most, but see below), within the C source code one can use the mmap system call to access input files via pointer arithmetic rather than malloc-and-memcopy using stdio. However mmap is not available when reading from standard input -- it cannot be memory-mapped. This means all file-format readers are implemented twice within the Miller source code. While I try to regression-test Miller thoroughly, running all canned tests through mmap and stdio mode, I've nonetheless found my mmap implementations liable to corner-cases which I miss but users find: for example #29, #102, and #296. As tracked on #160, various operating systems do not release mmapped pages after use as one might intuit, meaning that for large files and/or large numbers of files, I've for a long time now needed to have Miller opt out of mmap usage for precisely those cases which most need the performance gain: see #160, #181, and #256. Additionally, mmap is not used at all for Windows/MSYS2 so there is nothing to lose there. For these reasons, keeping mmap mode isn't worth the development overhead. As of release 5.6.3, the mlr executable will still accept the --mmap and --no-mmap command-line flags as no-ops, for backward compatibility. The caveat for you is that for everyday small files, the default was previously mmap mode and is now stdio (except mlr ... < filename or ... \| mlr ... which have always used stdio). There is the off chance that this will newly reveal an old, latent bug or two somewhere. I've re-run regressions in valgrind mode to aggressively catch any errors, but, please let me know ASAP via GitHub issue of any unexpected behavior in 5.7.0. 2020-03-17 15:38:25 +01:00			`SHA1 (mlr-5.7.0.tar.gz) = 50b18bb10bbf8a5e03f58b2ecd7809844834ed43`
			`RMD160 (mlr-5.7.0.tar.gz) = 3b3454bdd5ff771c8521a07ccd66e5932fc67af8`
			`SHA512 (mlr-5.7.0.tar.gz) = a801c9e6e10422cd0cc79c117ce7d49268f6d2bbe7c625fed0dd9144fcb7f5d2382a4450772f468f3d31b61912206bd8686f9c78fca2dbd372ee1e003f9cff88`
			`Size (mlr-5.7.0.tar.gz) = 1266826 bytes`