mail/rspamd: Update to 1.7.8

1.7.8: 12 Jul 2018
- [Feature] Add more extended statistics about fuzzy updates
- [Feature] Add more non-conformant Received headers support
- [Feature] Add preliminary function to get fuzzy hashes from text in
  Lua
- [Feature] Allow to configure AV module rejection message
- [Feature] Implement fuzzy hashes extraction in mime tool
- [Feature] Improve WHITE_ON_WHITE rule
- [Feature] Improve integer -> string conversion
- [Feature] Reuse maps in multimap module more aggressively
- [Fix] Avoid race condition in skip map as pool lifetime is not
  enough
- [Fix] Eliminate all specific C plugins pools
- [Fix] Fix DKIM check rule if DNS is unavailable
- [Fix] Fix build where ucontext is defined in ucontext.h
- [Fix] Fix crash in base url handling
- [Fix] Fix descriptors leak in sqlite3 locking code
- [Fix] Fix messages quarantine
- [Fix] Fix padded numbers printing
- [Fix] Fix race condition on maps reinit
- [Fix] Fix regexp functions when no data is passed
- [Fix] Fix specific urls extraction
- [Fix] Fix styles propagation
- [Fix] Improve resetting of the limit buckets
- [Fix] Initialize sqlite3 properly
- [Fix] Work with broken resolvers in resolv.conf
- [Project] Implement HTTP maps caching
- [Project] Refresh fuzzy hashes when matched
- [Project] Add logic to deduplicate fuzzy updates queue
- [WebUI] Add missed declarations
- [WebUI] Avoid using "undefined" property
- [WebUI] Do not accept passwords containing control characters
- [WebUI] Do not redeclare variables
- [WebUI] Enable strict mode,
- [WebUI] Fix variable assignment
- [WebUI] Initialize variables at declaration
- [WebUI] Remove duplicated path from RequireJS config
- [WebUI] Remove unused block
- [WebUI] Remove unused variable
- [WebUI] Remove unused variables
- [WebUI] Use self-explanatory notation
- [WebUI] Use type-safe equality operators

1.7.7: 02 Jul 2018
- [CritFix] Check NM part of pubkey to match it with rotating keypairs
- [CritFix] Do not overwrite PID of the main process
- [CritFix] Fix maps after reload
- [CritFix] Fix maps race conditions on reload
- [CritFix] Fix shmem leak in encrypting proxy mode
- [Feature] Add a concept of ignored symbols to avoid race conditions
- [Feature] Add ability to print bayes tokens in rspamadm mime
- [Feature] Add method to get statistical tokens in Lua API
- [Feature] Add preliminary mime stat command
- [Feature] Add rspamadm mime tool
- [Feature] Add urls extraction tool
- [Feature] Address ZeroFont exploit
- [Feature] Allow rspamadm mime to process multiple files
- [Feature] Allow to extract words in `rspamadm mime`
- [Feature] Allow to print mime part data
- [Feature] Allow to show HTML structure on extraction
- [Feature] Distinguish IP failures from connection failures
- [Feature] Improve output for mime command
- [Feature] Improve styles propagation
- [Feature] Main process crash will now cleanup all children
- [Feature] Preload file and static maps in main process
- [Feature] Print stack trace on crash
- [Feature] Process font size in HTML parser
- [Feature] Propagate content length of invisible tags
- [Feature] Read ordinary file maps in chunks to be more safe on
  rewrites
- [Feature] Support base tag in HTML
- [Feature] Support more size suffixes when parsing HTML styles
- [Feature] Support opacity style
- [Fix] Another fix for nested composites
- [Fix] Fill nm id in keypairs cache code
- [Fix] Fix colors alpha channel handling
- [Fix] Fix destruction logic
- [Fix] Fix double free
- [Fix] Fix maps preload logic
- [Fix] Fix nested composites process
- [Fix] Fix proxying of Exim connections
- [Fix] Fix reload crash
- [Fix] Fix rspamadm -l command
- [Fix] Update ed25519 signing schema
- [WebUI] Stop using "const" declaration
- [WebUI] Update RequireJS to 2.3.5

1.7.6: 15 Jun 2018
- [CritFix] Fix multiple neural networks support
- [Feature] Add decryption function to keypair command
- [Feature] Add gzip compression for HTTP requests in elastic module
- [Feature] Add gzip methods to lua util
- [Feature] Add maps based on Top Level Domains
- [Feature] Add pubkey checks for dkim_signing
- [Feature] Add support of fake DNS records
- [Feature] Add tool to encrypt files
- [Feature] Allow to add symbols using settings directly
- [Feature] Allow to match private and public keys for DKIM signatures
- [Feature] Allow to set task flags via settings
- [Feature] Allow to specify fake DNS address from the config
- [Feature] Implement signatures verification using rspamadm keypair
- [Feature] Implement signing using `rspamadm keypair`
- [Feature] Improve error reporting for DKIM key access issues
- [Feature] Provide $HOSTNAME variable in UCL
- [Feature] Rework levenshtein distance computation
- [Feature] Split message parsing and processing
- [Feature] Support ED25519 DKIM signatures
- [Feature] Support encrypted configs in UCL
- [Feature] Suppress duplicate warning on very large radix tries
- [Feature] Use OSB to combine header names
- [Fix] Cleanup maps data on shutdown
- [Fix] Fix '~' behaviour in composites
- [Fix] Fix HTTP maps updates
- [Fix] Fix NIST signatures
- [Fix] Fix RFC822 comments when processing a mime address
- [Fix] Fix double free
- [Fix] Fix dynamic settings application
- [Fix] Fix for CommuniGate Pro maillist
- [Fix] Fix keypair creation method to actually create keypair...
- [Fix] Fix matching patterns with no paths
- [Fix] Fix memory leak in parsing comments
- [Fix] Fix parsing of urls with numeric password
- [Fix] Fix plugins intialisation in configwizard
- [Fix] Fix potential crash on reload
- [Fix] Fix potential race condition for a finished HTTP connections
- [Fix] Fix race-condition leak on processes reload
- [Fix] Fix signing in openssl mode
- [Fix] Free language detector structures
- [Fix] Relax alignment requirements
- [Fix] Send DMARC reports compressed
- [Fix] Try to fix leak in dmarc module
- [Fix] Try to plug memory leak in metric exporter
- [Project] Convert rspamadm subcommands to Lua
- [WebUI] Display smtp sender/recipient in history
- [WebUI] Fix elements disabling in "Symbols" tab
- [WebUI] Limit recipients list in history column to 3
- [WebUI] Match envelope and mime addresses following in arbitrary
  order
- [WebUI] Update column header
- [WebUI] Wrap addresses in history

1.7.5: 18 May 2018
- [Conf] Add MSBL proposed return codes
- [Conf] Add additional groups for policies
- [CritFix] Do not use volatile Lua strings as UCL keys
- [Feature] Add ability to add fuzzy hashes to headers
- [Feature] Add function to extract most meaningful urls
- [Feature] Add rule to block mixed text and encrypted parts
- [Feature] Allow multiple groups for symbols
- [Feature] Allow to disable lua squeezing logic
- [Feature] Allow to get multipart children in Lua
- [Feature] Allow to insert multiple headers from milter headers
- [Feature] Allow to print scores in subject and further extensions
- [Feature] Be more error-prone in squeezed rules
- [Feature] Support multiple return codes in emails module
- [Feature] Use EMA for calculating averages
- [Feature] Use common jit cache for all regexps
- [Feature] support for CommuniGate Pro self-generated messages
- [Fix] Allow to have multiple values for headers as arrays
- [Fix] Do not open sockets for disabled workers
- [Fix] Fix AuthservId
- [Fix] Fix base64 folding in Lua API
- [Fix] Fix build on non-x86 platforms
- [Fix] Fix cached maps logic
- [Fix] Fix compatibility with old maps query logic
- [Fix] Fix crash if skip_map is used
- [Fix] Fix importing static maps from UCL
- [Fix] Fix parsing of unix sockets
- [Fix] Fix raw_mime regexp on HTML part with no text content
- [Fix] Fix tables logging
- [Fix] Fix vertical tab handling in libucl
- [Fix] Try to fix frequency counters
- [Fix] Use better sharding for ip_score
- [Fix] Use multiple results from SURBL DNS reply
- [Fix] When doing AV scan select a different server for retransmit

mail/rspamd/Makefile

@@ -1,7 +1,6 @@
-# $NetBSD: Makefile,v 1.43 2018/07/20 03:34:19 ryoon Exp $
+# $NetBSD: Makefile,v 1.44 2018/07/31 12:27:02 fhajny Exp $
 
-DISTNAME=	rspamd-1.7.4
-PKGREVISION=	1
+DISTNAME=	rspamd-1.7.8
 CATEGORIES=	mail
 MASTER_SITES=	${MASTER_SITE_GITHUB:=vstakhov/}
 

mail/rspamd/PLIST

@@ -1,4 +1,4 @@
-@comment $NetBSD: PLIST,v 1.12 2018/05/16 16:49:44 fhajny Exp $
+@comment $NetBSD: PLIST,v 1.13 2018/07/31 12:27:02 fhajny Exp $
 bin/rspamadm
 bin/rspamadm-${PKGVERSION}
 bin/rspamc
@@ -12,6 +12,7 @@ man/man1/rspamc.1
 man/man8/rspamd.8
 share/examples/rspamd/2tld.inc
 share/examples/rspamd/actions.conf
+share/examples/rspamd/cgp.inc
 share/examples/rspamd/common.conf
 share/examples/rspamd/composites.conf
 share/examples/rspamd/dmarc_whitelist.inc
@@ -150,6 +151,8 @@ share/rspamd/languages/ur.json
 share/rspamd/languages/vi.json
 share/rspamd/languages/yi.json
 share/rspamd/languages/zh-CN.json
+share/rspamd/lib/ansicolors.lua
+share/rspamd/lib/argparse.lua
 share/rspamd/lib/fun.lua
 share/rspamd/lib/global_functions.lua
 share/rspamd/lib/lua_auth_results.lua
@@ -162,17 +165,17 @@ share/rspamd/lib/lua_redis.lua
 share/rspamd/lib/lua_squeeze_rules.lua
 share/rspamd/lib/lua_stat.lua
 share/rspamd/lib/lua_util.lua
-share/rspamd/lib/rspamadm/ansicolors.lua
+share/rspamd/lib/plugins_stats.lua
+share/rspamd/lib/rescore_utility.lua
 share/rspamd/lib/rspamadm/confighelp.lua
 share/rspamd/lib/rspamadm/configwizard.lua
 share/rspamd/lib/rspamadm/corpus_test.lua
 share/rspamd/lib/rspamadm/fuzzy_convert.lua
 share/rspamd/lib/rspamadm/fuzzy_stat.lua
-share/rspamd/lib/rspamadm/getopt.lua
 share/rspamd/lib/rspamadm/grep.lua
-share/rspamd/lib/rspamadm/plugins_stats.lua
+share/rspamd/lib/rspamadm/keypair.lua
+share/rspamd/lib/rspamadm/mime.lua
 share/rspamd/lib/rspamadm/rescore.lua
-share/rspamd/lib/rspamadm/rescore_utility.lua
 share/rspamd/lib/rspamadm/stat_convert.lua
 share/rspamd/lua/antivirus.lua
 share/rspamd/lua/arc.lua
@@ -258,9 +261,9 @@ share/rspamd/www/js/lib/domReady.js
 share/rspamd/www/js/lib/footable.min.js
 share/rspamd/www/js/lib/humanize.min.js
 share/rspamd/www/js/lib/jquery-3.3.1.min.js
+share/rspamd/www/js/lib/require.min.js
 share/rspamd/www/js/lib/visibility.min.js
 share/rspamd/www/js/main.js
-share/rspamd/www/js/require.js
 share/rspamd/www/plugins.txt
 share/rspamd/www/react-index.html
 @pkgdir etc/rspamd

mail/rspamd/distinfo

@@ -1,9 +1,8 @@
-$NetBSD: distinfo,v 1.24 2018/05/16 16:49:44 fhajny Exp $
+$NetBSD: distinfo,v 1.25 2018/07/31 12:27:02 fhajny Exp $
 
-SHA1 (rspamd-1.7.4.tar.gz) = 6087573227f7a737e69ae854d879b251631720fa
-RMD160 (rspamd-1.7.4.tar.gz) = 931514920f052b251713a5539e87abc19caa7fbe
-SHA512 (rspamd-1.7.4.tar.gz) = 625f7bffe75e81866f5eefabab4fb41bddb823397f6a930ab4c452e2219b4a33a5c4e1628a72551f55452a05c82c7c20d7be2097db9d59a68e66f6d348279191
-Size (rspamd-1.7.4.tar.gz) = 4485331 bytes
+SHA1 (rspamd-1.7.8.tar.gz) = 8213d37209d0050d7f94eb26ad22ca85455913df
+RMD160 (rspamd-1.7.8.tar.gz) = 13de5b2fe6b8349b7731991787d5be1b111ad1d9
+SHA512 (rspamd-1.7.8.tar.gz) = 43f2350dfa4e0369cd53480bd4c8d0aa3898d7c471540281e7460a264207032aea226c4fd519a26397b9e9e8c8de3951e5a9752e010eabf1b3733a0c3e3086c2
+Size (rspamd-1.7.8.tar.gz) = 4542955 bytes
 SHA1 (patch-CMakeLists.txt) = 1d3e66b52ef38638674baf4c10b68c97b2ca38ad
-SHA1 (patch-contrib_t1ha_t1ha__bits.h) = 79255739468384adb099b1e7a0ae41549151e660
 SHA1 (patch-src_CMakeLists.txt) = c80015bb275ba8b9ad31491311275bf353f8f30a

mail/rspamd/patches/patch-contrib_t1ha_t1ha__bits.h

@@ -1,17 +0,0 @@
-$NetBSD: patch-contrib_t1ha_t1ha__bits.h,v 1.1 2018/05/16 16:49:44 fhajny Exp $
-
-Backport SunOS fix.
-
-https://github.com/vstakhov/rspamd/commit/7f596814c14a5844d49c50cad68df568e6417793
-
---- contrib/t1ha/t1ha_bits.h.orig	2018-05-01 15:52:02.000000000 +0000
-+++ contrib/t1ha/t1ha_bits.h
-@@ -390,7 +390,7 @@ static __always_inline uint16_t fetch16_
- #endif /* fetch16_le */
- 
- #if T1HA_USE_FAST_ONESHOT_READ && UNALIGNED_OK && defined(PAGESIZE) &&         \
--    PAGESIZE > 0 && !defined(__SANITIZE_ADDRESS__)
-+    !defined(__SANITIZE_ADDRESS__) && !defined(__sun)
- #define can_read_underside(ptr, size)                                          \
-   ((size) <= sizeof(uintptr_t) && ((PAGESIZE - (size)) & (uintptr_t)(ptr)) != 0)
- #endif /* can_fast_read */