Version 0.11.2, 2022-03-27
Support native roff escapes instead of using Unicode escapes. (This is a
work in progress, as there are many.) This makes special characters usable
in sections and so on.
Fix a bogus free when running lowdown-diff(1). While here, handle differences
between totally-different files, where before we would error out.
Document how differences are displayed in each output mode, as well as how
metadata is a special case. Fix -tterm ANSI escape filtering for background
colour, allowing changes to be viewed again.
Fix a sneaky -tterm bug found by AFL where having the number of footnotes
change digit length (e.g., 9 to 10) within a table would cause an assertion.
Another big thanks to Michael Dexter in providing hardware for continuously
running AFL and finding some of these corner case bugs!
Version 0.11.1, 2022-03-05
- Have -tlatex process header identifiers, both automatically generated
and from PHP extra attributes. Allow linking to these within the document
body.
- Have -tms print that an image happened instead of just omitting it
entirely. It now behaves like -tman in this regard.
- Fix where -tterm would print double-spaces in certain conditions.
- Have extended attribute classes apply as OpenDocument classes in -tfodt.
Version 0.11.0, 2022-02-21
Add class and identifier extended attributes to headers. These are
stipulated in PHP Markdown Extra. Documentation in lowdown(5).
When using header identifiers (the default) and the names collide, use
pandoc's method of creating unique header names. Also handle the case of
markup in headers and the identifier created from those to also be like
pandoc. This isn't a fix, but rather make output consistent with other
tools.
Fix output of how links are handled in -Tms when part of headers. The same
applies to other block-level macros with link children such as in
definition lists. This also cleans up how headers are shown in the PDF
TOC. While here, make header identifiers the same as in -Thtml so that
intra-document links are consistent, and enable linking within a document
using the PDF markers.
Fix footnotes to be actual footnotes in -Tlatex and -Tms instead of
endnotes. The other formats (-Tman, etc.) all continue to use endnotes.
The note system has internally been cleaned up and now allows for more
flexible note placement, e.g., side-notes.
Allow footnotes to properly exist in -Tlatex tables. Also allow for table
alignment.
Bump the minor number as the ABI has been tidied up with respect to
footnotes: they're now all in their LOWDOWN_FOOTNOTE blocks instead of
split into definitions and references. The LOWDOWN_DOC_FOOTER element,
which was not used, has also been removed.
The system in general has spent many, many computer-hours being fuzzed
with AFL thanks to hardware donated by Michael Dexter. Thank you!
Finally, make -T be a synonym for the new -t, which is used by pandoc.
Version 0.10.0, 2021-10-08
Add "flat" OpenDocument output with -Tfodt. This implements the
OpenDocument v1.3 standard, and is readable at least on LibreOffice.
This further implements -Tfodt change tracking in lowdown-diff(1).
It does not support images or definition lists yet.
All checksums have been double-checked against existing RMD160 and
SHA512 hashes
Unfetchable distfiles (fetched conditionally?):
./textproc/convertlit/distinfo clit18src.zip
Version 0.9.2, 2021-10-08
Significantly improve -Tterm output, both in terms of styles and layout.
Terminal output styles are easy to set as compile-time constants by editing
term.h. If you're going to edit this for a downstream installation of
lowdown, please let me know and I can stash it in a styles directory!
Add --term-no-ansi to disable all ANSI escapes in output. This produces a
clean, undecorated terminal-formatted document.
Version 0.9.1, 2021-10-04
Expand support for images having PHP Extra attributes to both links and
images supporting classes, images, and the existing image attributes.
These applies to both reference and inline links.
This deprecates --parse-no-img-ext in favour of the more general
--parse-no-ext-attrs. The same applies to the library's LOWDOWN_IMG_EXT
deprecated by LOWDOWN_ATTRS.
Version 0.9.0, 2021-09-23
Metadata consistency. Make sure that all standalone document metadata
(e.g., css for HTML, volume for manpages, etc.) is properly escaped
for its output context.
The -X option (metadata extraction) now extracts the unformatted
version of the given key. This is because output media don't
necessarily have a single way of formatting data: in -Thtml, for
instance, metadata may be formatted differently if it appears in
links, attributes, or within HTML content. It's now the caller's
responsibility to escape the results of -X.
This generates API changes, because metadata is now recorded during
the parse phase, not during the rendering phase. While here, metadata
extraction for "diff" mode is disabled (as it should be).
While here, the "base level header" metadata key has been replaced
with "shift-heading-level-by". This is because the existing "base
header level" never actually worked properly, and instead behaved
like "shift-heading-level-by".
Add GFM task lists to all output modes. While doing this, clean up
list output for -Tgemini and -Tterm.
Version 0.8.6, 2021-08-06
Bug-fix: the --parse-no-intraemph flag would set the condition (i.e.,
intra-emphasis was not parse) instead of unsetting it. Now it does the
correct thing.
Version 0.8.5, 2021-07-09
Add support for NO_COLOR and the flag version, --term-nocolour. This only
applies to -Tterm output when no in difference mode.
Version 0.8.4, 2021-04-03
Fix a crash in -Tgemini with tables. Allow for multi-page tables in -Tms.
Start to expand on regression suite. Clean up documentation, when possible,
and add some minor code polish.
Version 0.8.3, 2021-03-13
Several fixes to -Tlatex smart typography (single quotes and dashes). The
smart typography now handles single quotes within words. Lastly, fix a bug
where footnotes started at two instead of one.
Version 0.8.2, 2021-02-19
Fix tables as processed by the difference engine. Tables are now
fully opaque, which means that any changes will result in the
deletion and re-addition of the table. This isn't a good fix, but
it does mean that any tables run through the difference engine will
be sane for output.
Fix metadata to also be properly handled by both the difference
engine and conforming front-ends. This is a bit unusual since
metadata is both processed during parse and also affects document
output, such as in document title. For now use the same rule that
front-ends with metadata differences affecting document layout
(e.g., title) use the new form, if changed.
Lastly, fix footnote reference. When they're emitted in the new
document, the reference definitions are re-ordered in the correct
way to allow -Tms and such to work properly. While here, make sure
that all printed footnote numbers start at one and colours are
properly represented in output.
Split lowdown(1) into lowdown-diff(1) for easier reading.
Properly render tables for -Tgemini as fixed-width displays.
By default, render Gemini link labels using "Excel" format
(hexavigesimal) with the option of using Roman numerals
(--gemini-link-roman) or without labels at all (--gemini-link-noref).
This choice of default may change in later versions, hwoever.
ChangeLog:
Version 0.8.1, 2021-02-09
Add --term-nolinks to strip URLs out of terminal output (when alternative
text is available). Then add --nroff-nolinks and --nroff-shortlinks, just
like those for -Tterm, for use with -Tman or --nroff-no-groff.
Fix long-standing kinda-bug where www autolinks were being reported as
regular links instead of autolinks.
Introduce -m and -M, which allow metadata to be provided on the command
line. Metadata keys are first looked for in -m, overriden by what's in
the document, and those overridden by what's in -M.
Remove the deprecated -D, -d, -E, and -e, which were long ago replaced by
long options.
Inhibit printing of metadata in -Tgemini unless --gemini-metadata is given.
ChangeLog:
Version 0.8.0, 2021-01-31
Recognise the volume, source, and section metadata. These are currently
only used by -Tman.
Convert all internal functions to return an error code on memory allocation
failure. Prior to this, these functions had a chance of exiting and
printing failure to stderr. Now, this is left as the responsibility of
the front-end. There's no significant API change except that all renderers
return a value.
Fix the difference engine in several subtle ways, improving the produced
scripts, and also fix crashes where similar text would match multiple
parts of the parse tree, resulting in assertions.
Re-write the -Tms and -Tman generator to use a completely different
internal algorithm. This algorithm, instead of formatting directly into
output, converts the AST into an array of output blocks marked either as
text, literal, macro, or font/colour change. An assembler for this array
manages newlines and spacing between blocks. This fixes all known instances
of unexpected line breaks and allows for significantly simplified handling
of text interspersed with macros (e.g., links, etc.). An API result of
this is that the tree passed to lowdown_nroff_rndr(3) is now const.
Recognise non-block and block lists for -Tlatex output.
Emit a UTF-8 preconv header to all -Tms and -Tman so that -Kutf8 need not
be passed to the formatter.
Remove the --nroff-hardwrap option, which needlessly complicates code
without benefit.
ChangeLog:
Version 0.7.9, 2021-01-11
- Fix a crash introduced with the new UTF-8 handling in -Tterm.
- Allow the "section" metadata key to set the manpage section when used
with -Tman.
- Significantly clean up lists (definition, ordered, unordered) and how
indenting and compacting is handled in -Tman and -Tms.
- Use EX and EE for block code when in -Tman. These only apply in
LOWDOWN_NROFF_GROFF mode. While looking at block codes, fix how much
trailing space is emitted and make sure that subsequent paragraphs
are indented.
- Don't have sections in -Tman be followed by superfluous paragraph
macros (PP, etc.).
Version 0.7.7, 2021-01-06
Have the -Tterm output correctly adjust columns for UTF-8 wide characters.
This makes both ASCII and UTF-8 (e.g, Russian) correctly align on terminal
output.
Version 0.7.5, 2020-12-09
- Add support for the Gemini protocol I've been hearing so much about.
This may now be used from lowdown(1) or as a library in lowdown(3).
This is a very simple output: it basically does no formatting at
all, but does have some logic to shuffle around links. Version
Version 0.7.4, 2020-08-29
- Bug-fix: don't emit stray .PP macros when in -Tman or -Tms mode.
This bug was introduced in version 0.7.3.
- The library is now usable from C++ applications as-is, without
needing extra compilation flags.
- Add missing lowdown_diff(3) and lowdown_metaq_free(3) manpages.
Version 0.7.3, 2020-08-20
- Fix some smart typography bugs, specifically with ending single or
double quotes. While doing so, create a regression suite for smart
typography and update the manpage. Bug submitted by Larry Kollar,
thanks!
- Make base header level metadata key work properly. Another bug
submitted by Larry Kollar, thanks!
- Fix --nroff-hard-wrap to emit trailing newlines. Yet another bug
submitted by Larry Kollar, thanks!
Larry further suggested using typographic conventions in both the
-Tlatex and -Tms output to have initial paragraphs not indent, but
subsequent ones should. This is the default mode in LaTeX, which
I was disabling. I've now removed my override to be more like
default LaTeX, and Larry proposed an initial patch for having -Tms
do the same. Thanks yet again!
Version 0.7.1, 2020-06-21
- For -Tms, make numbered output the default. This is because earlier,
it was not possible to make numbered output at all with --nroff-groff.
Also clarify the role of --no-groff while at it.
- Add --latex-no-numbered to -Tlatex for symmetry with -Tms.
- Initial support for extended image attributes as in PHP Markdown
Extra. For the time being, these only allow for non-pixel widths
and heights for images. Spend time in the documentation explaining
image support, as it's a little complicated: LaTeX and HTML support
any images, but ms only supports EPS and PS, and only in block
mode. (Among other complexities...)
- Changes to internal API: the hdoc and hbuf structures have been
renamed to struct lowdown_doc and struct lowdown_buf. First, I
dislike typedefing away the fact that a variable is a structure
very much. Second, the names were a holdover from the old system
and shouldn't be part of the exported API.
- Add lowdown_buf_new(3) and lowdown_buf_free(3) to make the low-level
library usable, where before it was pointing to private functions.
Version 0.7.0, 2020-06-04
- Simple support for LaTeX documents with -Tlatex. This will be
expanded in future versions, but for now gets most documents having
maths and graphics, too.
- Significantly improve metadata handling. Until now, metadata was
handled inconsistently across output formats, and not always in
line with the documented behaviour with respect to overriding. Now,
all of the basic metadata keys are properly handled, RCS-specified
keys properly override, and all of these is documented more rigorously
in lowdown(1).
- Support base header level metadata on all outputs.
ChangeLog:
Version 0.6.5, 2020-05-20
- Fix an off-by-one where smarty-pants would be applied to some terms not
on a word boundary. For example, 1/40 was erroneously construed as 1/4
then a zero.
- Use pkg-config(1) for the installed library and improve the documentation.
pkgsrc changes:
- Fix LICENSE marker, lowdown is ISC licensed
- Enable test suite
- Remove forgotten checksum for patch-Makefile in distinfo
ChangeLog:
Version 0.6.4, 2020-03-29
- Enable and enforce portability by also running regression tests over
all operating systems and architectures. Updates oconfigure.
ChangeLog:
lowdown 0.6.1:
- Quick fix: we never actually... printed for the diff.
lowdown 0.6.0:
- Pull in newest oconfigure.
- Add common usage.
- Emit surrounding <p> for a list item if it's in a block and not already
containing block element(s).
- Bring docs up to date with code.
- Allow blank-separated definition key/value pairs to trigger block mode.
- Record whether a paragraph ends with a blank line or not.
- Proper styling of line prefixes.
- Get proper spacing for definition lists with -Tterm.
- Paragraphs in definition values don't get two newlines.
- Don't needlessly break up paragraphs that have a dli prefix in them when
within a list context.
- Show in alpha order.
- Document definition lists.
- Make line breaking around definition lists consistent.
- Merge multiple definition lists.
- Report definition list status.
- Complete parsing support for single-title description lists. There's
still work to be done, but this covers most uses.
- Recognise two newlines before a dli prefix as starting a block dli.
- Parse multiple definition list titles, don't parse zero-length trailing
definition list data, and properly form nroff definition lists w/multiple
datas.
- Add definition lists to -Tterm, documentation to nroff, forgotten header
change.
- Add nroff support for definition lists and clean up regular lists, too.
- Parse definition lists by default and add disabling documentation for them.
- Support for definition lists.
- const-ify and make sure formatters don't puke on definition lists.
- Add initial support for PHP Markdown Extra (and the many others inheriting
this functionality) definition lists. So far very simple and not enabled
by default.
ChangeLog:
lowdown 0.4.6:
- Make sure stdint is everywhere.
lowdown 0.4.5:
- Use BSD.lv's mandoc.css.
- Use BSD.lv's style and clean up.
- Newest sblg.
- Add stdint.
lowdown 0.4.4:
- Use Makefile.configure's macros.
- Bring up to speed w/other bsd.lv sites.
- Newest oconfigure.
- Document list-start for commonmark.
- All outputs process start list item for commonmark.
- Capture list start iff commonmark.
lowdown 0.4.3:
- Add atom version feed.
- Use new oconfigure.
- Bound reading into the value buffer.
lowdown 0.4.2:
- Document maths.
- Remove MATHEXP ("mathexp") facility.
- Disable MATHEXP mode. This offers nothing, but ends up complicating
the user experience. Stick with only one "math mode" and document it
properly.
- Have math blocks reported in -Ttree and accumulate the content in a
text node.
- Begin incorporating better math mode.
- Fix pdfhref links in lists.
- In nroff mode, stretch table to fit width and embolden header.
- Add table documentation.
- Typo found by Anton Lindqvist.
Push diff implementation (from libdiff) directly into the code instead of using the external library. From a patch by Anton Lindqvist as suggested on the OpenBSD ports mailing lits. Thank you!
Significantly update the diffing algorithm. First, make some general fixes to the algorithm. Second, improve the "optimisations" phase by adding top-down analysis that matches un-matched, non-terminal adjacent children. This helps with text changes in text-only paragraphs. Third, add a SES (shortest edit script) computation for matched adjacent text nodes. Lastly, add the new diff function manpages.
Portability: don't use %F for date formatting. This doesn't work with some libc versions. Also some documentation readability improvements.
Strip leading white-space from metadata extracted using -X. Sync with newest oconfigure.
Document the metadata functionality in lowdown(5), thanks to Christina Sophonpanich (thanks!). Also sync with newest oconfigure.
Add a "diff" tool, lowdown-diff(1). This utility uses an algorithm adapted from Detecting Changes in XML Documents to compute the semantic difference between two parse trees. It is fully documented. While there, also add more inter-paragraph spacing to -Tms output, producing more elegant documents, and continue fleshing out lowdown(5). Also add some more metadata recognition in -s output for all modes (copyright, affiliation, etc.).
Re-wrote escape parser to -Tms and -Tman to respect roff special characters. Have e-mail autolinks respect the mailto: in pdfhref'd output, and have links with mailto: omit the schema in display just like in -Thtml. Make block-list-items render properly in -Tms and -Tman. Also introduce lowdown(5), a work-in-progress to document the Markdown formatting accepting by this system. The first were noted, and the last contributed in full, by Christina Sophonpanich — thanks!
Fixed compilation on Linux and Mac OS X by adding memrchr compatibility. Noted by Christina Sophonpanich — thanks!
Considerable clean-up of -Tms and -Tman, with the aim of much higher PDF output quality: proper nested list support, hyperlinks, PS/PDF TOC, and even some images (PS/EPS only—experimental!). Also, after some pointers on the groff mailing list, use the correct invocation for generating PDF output. Fix up footnote printing to use automatic -ms macros and registers, if applicable.
Also added support for the "affiliation" metadata keyword.
Added some CommonMark support, initially just escaped newlines, supported only when the commonmark input flag is specified. Removed the sphd input flag in favour of commonmark. Also fixed raw HTML block outputting and setext-style level-two headers.
lowdown is just another Markdown translator. It can output traditional
HTML or a document for your troff type-setter of choice, such as
groff(1), Heirloom troff, or even mandoc(1). lowdown doesn't require
XSLT, Python, or even Perl - it's just clean, secure, open source
C code with no dependencies.