patch provided by Ove Soerensen in PR 26792
changes compared to 2.7f:
wwwoffle 2.8c contains various fixes and new features. For example the
removal of some memory-leaks, fixes for race-conditions and it's noew
CSS aware and has support for chunked encoding.
the "ChangeLog" file:
* src/configrdwr.c:
When allowing '=' signs in URL-SPECIFICATIONS ignore trailing spaces.
* doc/FAQ: Added a question about startup error/warning messages.
* src/wwwoffles.c:
Don't attempt to modify HTML if the page is compressed.
* src/headbody.c: Fix the Set-Cookie problem again.
* doc/scripts/FAQ-html.pl: New file.
* doc/scripts/README.CONF-html.pl, doc/scripts/README.CONF-msg.pl:
Convert characters with umlauts into proper HTML entities.
* src/headbody.c: A fix for the stupid bug in the previous version.
* src/io.c:
Fix for partial pages when server uses compression (suggested by
Paul Rombouts).
* src/wwwoffled.c: Close the client socket if SocketRemoteName() fails.
Bug Fixes:
Correct some invalid HTML in the message files. Fix some memory leaks, free
some unfreed memory, check some pointers, close some files. Don't send the
extra arguments to POST requests that WWWOFFLE uses internally. Removed
segmentation fault potential when password not used in config file. Another
fix for the cookie problem. Warn if running as root. Try both IPv6 and IPv4
socket binding (IPv6 may not accept IPv4). Potential bug fix for page
corruption. Potential fix for IPv6 configuration on Solaris.
New Features:
Add an option to have case-insensitive matching for URL-SPEC path and args.
Added the option to only fetch images on the same host (automatic fetching).
Allow URL-SPECs to contain an '=' sign embedded in them (long time bug fix).
The monitor options page now accepts ranges of hours or days (e.g. '1-5n').
[Note: The change to IPv6 binding to not require IPv4-mapped addresses was
already present in the package as "patch-ac".]
[Note: TODO: Have the package create a user, and let the "rc.d" script run
wwwoffle as that user.]
Also, reorder ${MASTER_SITES} to comply with the request on the download
page, and work around an install target corner case for the benefit of bulk
build machines (only applies for hosts on which wwwoffle has never run).
Bug Fixes:
Discard POST/PUT requests that have negative content-lengths. Make the
CanonicaliseHost() function robust to bad IP addresses. Fix some memory leaks,
free some unfreed memory. Don't give socket error using '-f' option. Fix
compilation on Cygwin. Fix IPv6 compilation on Solaris 9. Bug fix for v2.7c
Cookie change. Don't replace '//' in a URL path with '/'.
"file" is in "path"'s allocation. This fixes a bug where the server task
would randomly try to acces a file as a directory, and fail. Bump
PKGREVISION, as this was a fairly serious bug.
Bug Fixes:
Install two DLLs for the Win32 version. Don't crash for HTTP servers that send
headers prefixed with whitespace. Make the "edit selected entry" option work.
Don't write uncompressed data to the cache with a header saying it is
compressed. Be more lenient in detecting spiders that cannot make requests.
The wwwoffle-tools programs now handle dir names as if they had http:// in
front. Disallow wwwoffle requests for protocols that WWWOFFLE does not
handle. Use the command line config filename in error messages. Fix to allow
compilation on SGI IRIX. Handle XHTML style tags when modifying HTML. Updated
setuid/setgid code. Some memory leaks removed and potential crashes removed
due to using lint).
New Features:
Split up Set-Cookie headers since browsers can't handle them.
Don't request deflated data since WWWOFFLE and servers don't agree on format.
Added a form on the monitor options page to stop monitoring a URL.
The confirm-requests option now asks for confirmation for page reloads.
Documentation:
Update FAQ to reference privoxy as well as JunkBuster.
Describe how to modify htdig templates to work with WWWOFFLE.
Bug Fixes:
Ensure that only one argument is given to wwwoffle -o or -O. Some more version
2.7 documentation updates. Fix crash using 'wwwoffle -O|-o|-put|-post <URL>'.
Put refresh URLs in the outgoing directory with correct URL. Delete
auto-generated files in Makefile before re-generating. Choosing 'edit selected
item' in configuration pages shows current values. Fix potential crash with
wwwoffle-hash. Put a DESTDIR variable in Makefile for easier installation.
Fix parsing IPv6 addresses in audit-usage.pl script. Makefile change for
FreeBSD make. Bug fix for the Alias section of the config file. Fix Makefile
for Win32.
New Features:
Allow CGI scripts to be used with the built-in web server.
Bug Fixes:
Ensure that the -put or -post options to wwwoffle have one URL. Fix IPv6
checking (configure fails if IPv6 not available). Fix conditional request
problem (304 reply for non-conditional requests). Make the socket binding
errors less confusing. Fix requesting of compressed data. Handle NULL strings
in FTP code and parsing requests. Speed up wildcard matching of '/*' paths.
When search script fails give an error not a blank page. The content-length
header is not removed unless compression is being used. Fix core dump with
configuration page adding first item to DontGet/DontCache section. Preserve
cache file timestamps when compressing them. Handle relative URLs that start
with '//'. Fix Solaris compilation problem with statfs/statvfs. Bug fix for
failure to censor some headers. Remove the 'alt' attribute from disabled
images when modifying HTML.
New Features:
Re-instate the old configuration editing web pages due to user demand.
Allow wildcards to have more than two '*' in them.
The upgrade-config.pl script warns about URL-SPECs with path='/' not '/*'.
dealing better with javascript pages, and fixes for old ftp servers.
Lots of new features and configure options, too. Also, new
translations for French, Dutch, Polish, German; docs and notes in
a few other languages; and translations for local pages are now
selected by browser settings!
rather than "wwwoffle.conf.default", so that wwwoffle-upgrade-config*.pl
can use it to populate the new config file with comments (and to minimize
gratuitious changes).
Bug Fixes:
Fix pagination problem in wwwoffled manual page. Fix core dump with missing
DontCompress section. Fix error with convert-cache and uncompress-cache
programs not recognising valid configuration items. Fix harmless buffer
overrun. Improve the URL decoded strings displayed in indexes. Don't complain
about empty directory when installing. Canonicalise the pathname in URLs.
Compile on __bsdi__ systems. Convert decimal IP addresses to dotted-quad.
Add support for compilation on Apple OS X. Enable HTML modifications on URLs
with error status. Use a case-insensitive check when censoring headers. Fix
up HTML to that it validates with an SGML checker.
New Features:
Added IPv6 support.
Added bind-ipv4 and bind-ipv6 options to specify local IP address to bind to.
Added 'random' sort order option for indexes.
Made index sorting use alphabetical as a secondary sort.
Added HTTP/1.1 'Cache-Control: max-age=..' header handling (same as 'Expires').
Translations:
Updated the Russian translations of the WWWOFFLE messages.
Bug Fixes:
Changed the HTML message pages to be HTML 4 compliant (use ';' instead of '&'
for URL arguments). Fixed meta-refresh-self problem. Don't get confused by
scripts when parsing HTML. Fix crash in convert-cache program. Made
compilation work on SGI machines. Don't truncate partial cached file. Fix the
request-changed option.
New Features:
Add comments where WWWOFFLE modifies the original page's HTML.
Add the option to remove stylesheets, external links and embedded styles.
Add the option to remove Java applets.
Add options to remove links and replace images listed in the DontGet section.
Add the option to not fetch webbugs (1x1 pixel images) when parsing HTML.
Add the option to replace webbugs (1x1 pixel images) when modifying HTML.
Translations:
Added Dutch translations of WWWOFFLE messages.
Version 2.6b of WWWOFFLE released : Sat Mar 24 14:30:00 2001
------------------------------------------------------------
Bug Fixes:
HTML parsing optimisations. HTML parser memory leak fixed. Fix stylesheet
link parser. Stop cached pages containing trailing junk. Fixed wwwoffle
manual page quote character bug. Fix problems parsing parameter strings in
URLs. Fix ssl-allow-port config file parsing.
Win32 Bug Fixes:
Fixed the socket closing code.
Documentation:
Updated the README.win32 file.
Updated FAQ to version 2.6.
Updated French translated pages.
Added a README.compress that describes the compression problems and solutions.
New Features:
Request data from servers is sent compressed, config option (see zlib note).
Reply to client with compressed data if asked, config option (see zlib note).
Compress the files in the cache when purging, based on age (see zlib note).
Allow fetching in autodial mode as well as online mode.
*NOTE* The use of zlib to enable compression should be considered a beta quality
feature in this version. It requires the zlib library to compile it.
*NOTE* If upgrading from a version < 2.6a then see the notes for version 2.6a.
Bug Fixes:
Fix crash with invalid entries in config file. Handle some invalid URL
encodings that servers/clients use. Stop crashes when monitoring illegal
refresh URLs. Handle malformed headers better. Check the DontGet section of
the configuration file when doing SSL. Print error messages when starting on
stderr. Read the configuration file if using wwwoffle-write. Fix bug with
form argument parsing. Set fetch-stylesheets to true in the default config
file. Bug fixes for modifying HTML with meta refresh tags. Stop duplicated
items in the configuration file.
Win32 Bug Fixes:
Fixed logfile opening problem. Added a fchdir() function for cygwin.
Workaround cygwin socket closing problem. Change the makefile for cygwin
installation (don't auto install .dll or .bat).
Documentation
Updated the Welcome.html page to include more links to useful information.
Include an HTML version of README.CONF in the html directory.
Updated several README files and INSTALL file.
*NOTE* The default location of the replacement .gif file for the DontGet section
has changed, upgrade-config.pl should fix wwwoffle.conf automatically.
*NOTE* If upgrading from a version < 2.6 then see the notes for version 2.6.
NetBSD package changes:
- - Install upgrade-config-2.x-2.5.pl (as wwwoffle-upgrade-config-2.x-2.5.pl).
- - Protect convert-cache the same way as upgrade-config -- prefix with wwwoffle-.
- - Update MESSAGE file accordingly.
- - Add a new-style rc.d script.
- - Fix the included "htdig" scripts to work with packaged "htdig".
("htdig" remains optional.)
- - Install the convert-cache and upgrade-config-pl tools to make
upgrading less painful. Run convert-cache automatically for binary
package install, but only suggest to for source install.
- - Expand the MESSAGE text to explain how to use all this.
Convert most MESSAGE files to new syntax (${VARIABLE} gets replaced,
not @VARIABLE@, nor @@VARIABLE@@).
By default, substitutions are done for LOCALBASE, PKGNAME, PREFIX,
X11BASE, X11PREFIX; additional patterns can be added via MESSAGE_SUBST.
Clean up some packages while I'm there; add RCS tags to most MESSAGEs.
Remove some uninteresting MESSAGEs.
Version 2.6 of WWWOFFLE released : Sat Nov 18 19:15:00 2000
-----------------------------------------------------------
Bug Fixes:
Improve HTML modification for unterminated tags. Allow passworded pages to be
fetched. Improve compilation on non-Linux systems. Fix bug with proxy config
file entry. Fix an error with not truncating files. Fix an error with
dir-perm and file-perm. Fix problem when getting pages with passwords. Fix
problem deleting pages with passwords.
Documentation
Added a note to the FAQ about DoS attacks and ipchains.
*NOTE* If upgrading from version 2.[2345] then you will need to convert your
cache to the new format, see the file CONVERT for details.
*NOTE* If upgrading from version 2.4[abc] the max-size option in the Purge
section of the configuration file has changed. See CHANGES.CONF.
*NOTE* If upgrading from version 2.[123] the setting of the times of monitoring
URLs has changed, check the monitor index and correct where needed.
*NOTE* If upgrading from version 1.x or 2.[01] then you will need to delete
your cache since conversion from those formats is no longer provided.
Version 2.6-beta of WWWOFFLE released : Sun Oct 22 10:30:00 2000
----------------------------------------------------------------
Bug Fixes:
Handle usernames specified in URLs including the '@' character. Fix problems
deleting URLs with arguments. Fix bug with recursive fetching in same dir.
Retry the select system call if it is interrupted.
Win32 Bug Fixes:
Fix for local web-pages not being opened in binary mode. Compilation fixes.
Internal Changes:
Re-examined all URL-encoding and URL-decoding issues (small cache change).
Ensure that the canonical form of the URL is used throughout.
Changed the URLs in the indexes for monitor, delete & refresh.
Documentation
Re-written the README.CONF file with new layout and more information.
Added three more questions to the FAQ and updated several others.
Configuration File
Allow many of the configuration file options be selectable on a URL by URL basis.
Move some configuration file options around and create some new sections.
Allow purge ages to be specified in larger units (weeks, months, years).
Allow re-request times to be specified in larger units (minutes, hours, days).
New Configuration Options
Add the ability to demoronise HTML (replace bogus characters with real ones).
Add the ability to remove meta refresh tags that redirect browsers.
Added the option to convert redirections to DontGet pages to errors.
Allow the HTML modifications to happen to pages viewed when online.
Add timeouts to DNS lookups to stop WWWOFFLE servers hanging up.
Add the option to enable the use of lock files (defaults to disabled).
New Features
Remove the index of the latest pages (was slow on big caches).
Add an index of the pages that were in the outgoing directory last time.
Change the don't cache option so that pages are not requested when offline.
Allow password protected URLs to be deleted.
Aliased pages now use a redirect rather than re-writing the URL.
Make it safe to have symlinks in the cache.
Searching
Changed the ht://Dig search URLs in WWWOFFLE from /htdig/* to /search/htdig/*.
Allow the use of UdmSearch instead of ht://Dig.
Contrib
Improved the audit-usage.pl script to show cache hit/miss status for requests.
Related changes:
- Fix the permissions for the installed files
(this fix is the same one that Frederick Bruckman did for 2.5c).
- Remove the extra newline added to POST requests.
- Only clear the supplementary group list if running as root.
- Stopped overflow of 32 bit integer when purging.
- Allow wrong reply to CWD from broken FTP server.
- Don't get confused by badly nested script or blink tags.
- Disallow the Accept-Encoding header on outgoing requests.
- Make the URL-SPECIFICATION in the config file have lower case
for protocol and host.
- Modified the Meta refresh tag handling.
- Fix FTP directory listings for files with ':' in them.
- and other bug fixes.
- Added Russian and Polish translations of WWWOFFLE messages.
- Updated FAQ.
WWWOFFLE is functional without ht://Dig (search function is not available
of course) and htdig pkg may be added/removed AFTER addition of wwwoffle pkg.
(Another reason is that ht://Dig is not so useful for Asian languages. :)
Add reference to www/htdig pkg.
- New, optional Makefile variable HOMEPAGE, specifies a URL for
the home page of the software if it has one.
- The value of HOMEPAGE is used to add a link from the
README.html files.
- pkglint updated to know about it. The "correct" location for
HOMEPAGE in the Makefile is after MAINTAINER, in that same
section.
