pkglint -r --network --only "migrate"
As a side-effect of migrating the homepages, pkglint also fixed a few
indentations in unrelated lines. These and the new homepages have been
checked manually.
Performing substitutions during post-patch breaks tools such as mkpatches,
making it very difficult to regenerate correct patches after making changes,
and often leading to substituted string replacements being committed.
This version finally fixes the old problem for primo-installation when the
sympa wizard would not start to generate the sympa.conf... because the
sympa.conf file was empty.
In addition, Soji fixed a timeout problem when Sympa closed a list, because
Sympa spent ages checking whether the list was included or not in another one.
He simply added a table registering the lists inclusions to speed up the
checking process. And because fixing the problem was not enough to him, he
also added a new page in the Sympa web interface to visualize lists inclusions.
In summary, what you have now is:
"including_lists" page to get lists including particular lists. This page
is accessible only by list owners and listmasters.
Now looping by include_list settings can be detected. Inclusion is skipped
if a setting will cause looping.
the new 'include_sympa_list' parameter obsoletes 'include_list' in list
config and named data sources. Older one may still be used as it is
mapped to newer one internally.
The subindex page did not allow to actually accept/reject subscriptions. This
was because of a badly named parameter being passed to the templates.
WWSympa does no longer respect HTTP request header fields noticing the request
was forwarded ("X-Forwarded-Host:" etc.). In particular cases, you may require
adjusting configuration.
Now wwsympa_url parameter defines the URL publicly accessed by users.
Contrarily "http_host" and "cookie_domain" parameters indicate the URI and/or
domain which are locally recognized by wwsympa, i.e taken form "http_host"
("Host:" field) and "request_uri" CGI environments.
"Location:" response field sent by wwsympa to cause redirection also gives
local URL.
If requests sent from users to wwsympa are rewritten by intermediate
server(s), you should make sure that wwsympa server and rewriting server(s)
are appropriately configured.
a new parameter, use_tls, appears to replace obsoleted "use_ssl" and
"use_start_tls" parameters; note that these obsoleted parameters can stil be
used. But the new one is way more convenient: "use_tls" takes one of following
values corresponding to old ones:
"starttls" : use_ssl=0, use_start_tls=1.
"ldaps" : use_ssl=1, use_start_tls=0.
"none" : TLS/SSL is disabled.
Now STARTTLS may be enabled in LDAP datasource configuration of lists
along with LDAPS.
The "Feedback" link will now be visible by listmasters only! We received
enough mails from end users mis-interpreting this link with their local
helpdesk. :-|
Now [% path_cgi %] template variable respects "wwsympa_url" configuration
parameter. Previously it was taken from CGI environment variable.
Soji also improved Sympa daemons to capture the SIGCHLD signal, so that
defunct children will be closed quicker.
As we tend to change the CSS regularly according to the different feedback you
send us, we added the Sympa version number in the CSS file URL, so that it will
be automatically reloaded, ignoring cache, when your users arrive on the web
interface after an upgrade.
Changing web_archive_spam_protection did not update spam protection mode on
past archives. Fixed by applying spam protection at the time of display, not
when archives were created / rebuilt.
base element in web output should be eliminated, because it will mess reverse
proxy and so on. Fixed by making partial URIs to be redirected to base URI.
WWSympa: Cannot view attachments in bounce messages and moderated messages.
Fixed by correcting inappropriate relative URL paths.
MASTER_SITES= site1 \
site2
style continuation lines to be simple repeated
MASTER_SITES+= site1
MASTER_SITES+= site2
lines. As previewed on tech-pkg. With thanks to rillig for fixing pkglint
accordingly.
for all pkgsrc dir/file ownership rules. Fixes unprivileged
user/group names from leaking into binary packages, manifest as
non-fatal chown/chgrp failure messages at pkg_add time.
Bump respective packages' PKGREVISION.
http://www.sympa.org/distribution/latest-stable/NEWS
for complete list.
Main changes are:
Tracking feature
The tracking feature is a way to request DSN or DSN + MDN when sending
a message to each subscribers. In that case, Sympa (bounced.pl) collect both
DSN and MDN and store them in a new table "notification_table".
Then, for each message, the list owner can display which subscribers has
displayed, received or not received the message. This can be used for some
important lists where list owner need to collect the proof of reception or
display of each message. This page is accessible via archive
Spools back to filesystem
Because of stability and scalability issues, spool for bulk sending no
longer depend on database tables: It is based on filesystem.
Stats
Now Sympa stores data whenever one of the following event occurs:
a message is sent to a list;
a user subscribed to a list;
a user unsubscribed from a list;
a user is added to a list by another user;
a user is removed from a list by another user;
a user is removed from a list by the automatic bounce management;
a file is uploaded to the shared directory;
a file is deleted from the shared;
a message to a list is rejected in moderation;
a user logs in to the web interface;
a user logs out;
a list is created;
a list is deleted;
a list is restored;
a human user (not a harvester) hits a page.
These data are regularly aggregated by the task_manager.
The aggregated data are available to users, owners and listmaster in
their respective interfaces.
Web interface
The most visible is probably the new web interface skin.
It is nicer, simpler, responsive and retains all the customization
capabilities that make Sympa adaptable to your needs.
Exclusion table was just a display of the users excluded. This page is now
a form, similar to the review page, which allows to restore users subscriptions.
Added "Add subscriber" button to modindex page if the sender is not subscribed.
It may be useful because users need not know about SUB command and sympa
command address. The buttons appear only when Web user is the list owner.
The "Customizing" link was moved from main list admin to list config sub
section as list related text files are in a way list configuration parameters
as well.
Better password validation. When the user requests change of password via
WWSympa or SympaSOAP, new password may be checked its strength.
New parameter password_validation may be used to customize policy of password
validation. See help text of the parameter for more details.
Web services
Full VOOT 2.0 protocol support. VOOT is an opensocial subset to manage group
membership. full details on this IETF draft here: http://openvoot.org.
This feature enables Sympa to be both group provider and consumer in the VOOT
protocol.
The soap server now exposes full review with owners, editors and gecos to list masters and list owners.
Plug, plug, plug...
We added or improved several ways to plug new functionnalities in Sympa.
Template plugins
This feature allows to package code to be used in templates. It enables
foreign data integration in Sympa's UI and emails, user specific
UI enhancement and much more! Integrating user targeted data retreived through
a call to a webservice becomes possible in a simple way, automatically adding
info from a database to outgoing emails as well !
Further details here : https://www.sympa.org/manual/templates_plugins
Enhanced custom actions functionality
Custom actions are used to run specific code and/or display user defined
templates. They can be executed in list or global context (it is up to you
to decide what to do in both cases). Previously, a custom action was a
simple TT2 template added to the web interface. It could only display data,
not process them. They were improved to allow greater expressiveness.
You can now develop a perl module to process the data passed to your
custom action.
Warning: AFTER UPGRADING TO 6.2, ANY PRE-EXISTING CUSTOM ACTION MUST BE
MOVED TO THE RELEVANT CUSTOM_ACTION DIRECTORY TO KEEP WORKING.
Mail hooks
Sympa::Message::Plugin provides hook mechanism to intervene in processing by
Sympa. Each hook may modify messages or may break ordinary processing.
This functionnality is quite new and still experimental. Please read the
perldoc of Sympa::Message::Plugin for more details about how to create and enable hooks in message handling.
Custom attributes provisionning
Custom attributes can now be provisionned using external data sources,
the same way as email addresses. For now, only SQL or LDAP datasources are
supported. To use this feature, you need first to define the custom attributes
as previously. This attribute must have the same name as the fields used in
your queries.
Synchronization disabling
In several organizations, nightly maintenances make the datasources
unavailable during some period of times. a new sub-parameter in data sources
definition allow to prevent Sympa from trying to synchronize list members
with these sources during a defined time range.
A recurrent question regarding list inclusion was to be able to extract not the whole list, but only a subset. It is now possible.
include_list parameter value can now look like this:
<listname> filter <filter_condition>
ODBC is now supported for email data sources.
sql, ldap and ldap2 user inclusions now handles gecos. It uses 2nd returned
column for sql and 2nd given attribute for ldap (comma separated).
Now "host" parameter of the LDAP datasource in list config may include URLs
(ldap, ldaps and ldapi) along with hostnames. So "port" and "use_ssl"
parameters
Scenarios
Now CustomCondition can set the action to take (do_it, reject ...) by
setting $_, this allows for complex, single-module CustomConditions.
"[msg_header#field][index]" returns the value of particular field.
Index may be negative. - "[msg_header#field]" still returns list of field
values, additionally, ordering will be preserved. Fixes:
Conditions is_listmaster, is_owner, is_editor and is_subscriber can
handle multiple values.
They also parse arguments as header field values to get address parts.
New parameters
New list parameter: 'member_include'. This parameters allows to defined
.incl files to include list members, the same way it is done for list admins.
This grants extended flexibility in datasources, allowing list owners to tweak
their own data sources without accessing to the actual details of the query
made to gather email addresses.
New sympa.conf config parameter "sender_headers" to specify header fields by
which message sender is detected. This is a enhancement to
S. Shipway's improvement.
RFC: What is the reasonable default for this parameter?
Behavior of one time ticket lock-out is configurable.
Two new robot parameters were introduced:
one_time_ticket_lockout:
"one_time" won't allow access again. This is previous behavior.
"remote_addr" will lock the ticket on remote address accessed at first time.
"open" will never lock-out tickets.
one_time_ticket_lifetime: Duration before ticket expires. Default is "2d", previous hardcoded value.
Changes of list config parameters on archiving. New parameter
"process_archive" controls archiving. The default is "off":
To enable archiving, it must be set to "on" explicitly. OTOH even if
archiving is disabled by setting it "off", past archives will be kept
and accessible according to access settings.
Config files
wwsympa.conf will no longer be used: it would be migrated to sympa.conf.
Alias management uniformization
Now alias maintenance utilities other than newaliases may be used without
special configure options nor patch to alias_manager.pl. Changes:
aliaswrapper and virtualwrapper were deprecated and replaced with sympa_newaliases-wrapper.
New alias management program sympa_newaliases.pl which will typically be
called by alias_manager.pl via sympa_newaliases-wrapper.
New site configuration parameters aliases_db_type and aliases_program will
control behaviour of alias database maintenance.
openssl dependency dropped
openssl(1) utility is no longer required. Instead, a few CPAN modules are required to use S/MIME or SSL features:
Crypt::SMIME
Crypt::OpenSSL::X509
The parameter "openssl" in sympa.conf was obsoleted. Note that p12topem.pl still needs openssl: To make it work, PATH environment variable should be set appropriately.
Internationalization
i18n titles in configuration.
List families scenari
Now you can define a "scenari" directory in the lists family directory.
These scenarii will be available for lists instantiated from this family.
List families addistional files
Until now, only the config.tt2 file was parsed when the family was
instantiated, thus producing the lists' config file. As a lot of other files
can be used to configure a list (footer, header, etc.), we added the
possibility to instantiate these files as well. The list of files parsed
(in addition to config.tt2) when instantiating a family are defined in
a new (sympa.conf and robot.conf) parameter called 'parsed_family_files'.
This parameter must contain a comma-separated list of file names.
If these files exist, with the '.tt2' extension added to their name,
in a family, they will be parsed and added to the list directory.
The default value of the parameter is: 'message.footer,message.header,message.footer.mime,message.header.mime,info'.
These files are updated in existing lists when a family is instantiated.
note that it will overwrite any other customization.
On the web automatic lists
Administration
Widely extended logs when the log_module parameter is activated with the
value 'scenario': all scenario results are evaluated. Remember that extended
logs can be limited by the IP or the email of the user, using the
'log_condition' parameter. log_module and log_condition are now declared in
sympa.conf instead of wwsympa.conf. Remember these parameters syntax:
log_module: can take the value "scenario" - only.
log condition: takes a character string as value. The form is
"ip=<an.ip>,email=<an_email>". You can put ip alone, email alone or both,
with the comma-separated form.
Listmaster notifications can now be grouped. If, for a specific robot,
several notifications are issued within a short period of time with the
same operation code only the first ones are sent, the next ones are stacked.
The stacking ends when no notifications are issued anymore, or when the first
one is too old. All the stacked notifications are then sent as attachments of
a single message. Internal settings : Stack if more than 3 notifications with
the same operation code are issued for the same robot - Send stacked
notifications if no new one satisfying the condition above was received for
more than 30 seconds - Send stacked notifications if the oldest one was issued
more than 60 seconds ago
sympa.pl -health_check checks config files, database connection and structure,
and data structure version.
Daemons such as sympa.pl no longer check database structure nor data version.
File extension of lock files are changed from ".lock" to ".LOCK", because lowercase name can conflict with domain part in the future.
{perl>=5.16.6,p5-ExtUtils-ParseXS>=3.15}:../../devel/p5-ExtUtils-ParseXS
since pkgsrc enforces the newest perl version anyway, so they
should always pick perl, but sometimes (pkg_add) don't due to the
design of the {,} syntax.
No effective change for the above reason.
Ok joerg
- wwsympa/wwsympa.fcgi.in: Preventing faulty newsletter sending (security fix)
- src/sympa.pl.in: [Submitted by B. Marchal, univ. Lorraine] the
loop_prevention_regex parameter was never taken into account.
- New scenarios to check DKIM signature
- When the user requests change of password via WWSympa or SympaSOAP, new
password may be checked its strength.
- src/lib/Scenario.pm: Now CustomCondition can set the action to
take (do_it, reject ...) by setting $_, this allows for complex,
single-module CustomConditions.
and various bug fixes
[10541] src/lib/List.pm: [Submitted by S. Shipway, Univ. of Auckland]
Workaround for aggressive DMARC policy such as yahoo.com. The patch
adds option #3 of this DMARC FAQ: http://dmarc.org/faq.html#s_3
- New list config paragraph "dmarc_protection" to munge "From:" header
and put original header content erc. into comment.
[10540] src/lib/Bulk.pm: New parameters for merged messages.
"part.description", "part.disposition", "part.encoding" and "part.type"
may be used for each part of input messages. These are all-lowercase
(except "part.description").
***** [10207] src/etc/script/create_db.Oracle, src/etc/script/create_db.Pg,
***** src/etc/script/create_db.SQLite, src/etc/script/create_db.Sybase,
***** src/etc/script/create_db.mysql: Two new database fields appeared in
***** this version and a field was modified.
***** - The new fields are prev_id_session (varchar(30)) and
***** refresh_date_session (int(11)). they are located in the session_table
***** table.
***** - The modified field is dkim_privatekey_bulkspool and is located in the
***** bulkspool_table table. Its length went from varchar(1000) to
***** varchar(2000).
***** Sympa install using MySQL and SQLite backends will have no trouble at
***** all, as the database structure is updated by Sympa.
***** However, if you use Postgres, Oracle or Sybase, please have a look
***** (respectively) at the create_db.Pg, create_db.Oracle or
***** create_db.Sybase to check the definition of those fields. Please update
***** your database structure before running Sympa.
[10206] src/lib/Sympa/DatabaseDescription.pm: Changing length of DKIM
private key in database to ensure database creation scripts will be
updated.
[10074] mail_tt2/command_report.tt2, mail_tt2/info_report.tt2,
mail_tt2/review.tt2, src/lib/Commands.pm, src/lib/List.pm,
src/lib/tt2.pm, web_tt2/edit_list_request.tt2,
web_tt2/review_family.tt2, web_tt2/search_user.tt2,
web_tt2/suboptions.tt2, web_tt2/subscriber_table.tt2,
web_tt2/suspend_request.tt2, wwsympa/wwslib.pm,
wwsympa/wwsympa.fcgi.in: i18n of options for list parameters and
subscriber options.
- Options on edit_list page are shown by i18n'ed titles.
- Only listmasters can view real config values.
- Subscriber options on review pages, command results, subscriber
option pages and so on are shown by i17n'ed titles (along with real
option values).
***** [10051] src/lib/tt2.pm, web_tt2/Makefile.am, web_tt2/css.tt2,
***** web_tt2/ja_JP, web_tt2/ja_JP/css.tt2, web_tt2/ko_KR,
***** web_tt2/ko_KR/css.tt2, web_tt2/main.tt2, web_tt2/zh_CN,
***** web_tt2/zh_CN/css.tt2, web_tt2/zh_TW, web_tt2/zh_TW/css.tt2,
***** wwsympa/wwsympa.fcgi.in: Per-language css.tt2 will override any
***** portion of main css, not fully replacing it. So they may be used for
***** locale-specific customization.
***** Background: Default css.tt2 specifies the font families covering
***** Western scripts (Latin, Cyrillic, ...). East Asian users may prefer
***** consistent font family supporting Western along with Eastern scripts
***** (Han, Hangul, ...).
[9966] src/lib/Message.pm, src/lib/confdef.pm: New site config
parameter "sender_headers" to specify header fields by which message
sender is detected.
This is a enhancement to S. Shipway's improvement.
[9963] web_tt2/review.tt2, web_tt2/show_exclude.tt2,
wwsympa/wwsympa.fcgi.in: [Reported by so many listmasters we lost the
count] Exclusion table was just a display of the users excluded. list
owners could not do anything to restore subscriptions;
This page is now a form, similar to the review page, which allows to
restore users subscriptions.
[9951] src/lib/List.pm: Now you can define a "scenari" directory in
the lists family directory. These scenarii will be available for lists
instantiated from this family.
The "scenari" directory must be put directly in the family directory,
not in the overall "families" directory. For example, if you want to
define scenarii specific to the "staff" family, you must define a
scenari directory in the /home/sympa/etc/families/staff/ directory. Not
in /home/sympa/etc/families/.
***** [9989] configure.ac, src/Makefile.am, src/alias_manager.pl.in,
***** src/etc/script/ldap_alias_manager.pl.in,
***** src/etc/script/mysql_alias_manager.pl.in, src/lib/confdef.pm,
***** src/sympa_newaliases-wrapper.c, src/sympa_newaliases.pl.in: Now alias
***** maintenance utilities other than newaliases may be used without special
***** configure options nor patch to alias_manager.pl.
***** Changes:
***** - aliaswrapper and virtualwrapper were deprecated and replaced with
***** sympa_newaliases-wrapper.
***** - New alias management program sympa_newaliases.pl which will typically be
***** called by alias_manager.pl via sympa_newaliases-wrapper.
***** - New site configuration parameters aliases_db_type and aliases_program
***** will control behaviour of alias database maintenance.
***** - configure script:
***** - Options --with-sendmail_aliases and --with-virtual_aliases were
***** deprecated. Use --with-aliases_file instead.
***** - New options --with-makemap and --with-postalias, along with
***** options --with-newaliases and --with-postmap are available.
***** - Option --with-postmap_arg was removed.
***** - Alias managers can handle postmap/makemap style maps (delimited by
***** whitespace), not only newaliases style maps (delimited by colon).
[9953] wwsympa/wwsympa.fcgi.in: [Submitted by S. Shipway, univ.
Auckland] several changes in privilegs to ease everyday lists
moderation:
- Owners and lismasters can moderate messages and shared repository
- Editors can moderate subscriptions
- 'del' and 'add' sceanrios are evaluated to make their result
available in each page.
[8451] src/etc/Makefile.am, src/etc/create_list_templates/confidential,
src/etc/create_list_templates/confidential/comment.tt2,
src/etc/create_list_templates/confidential/config.tt2,
src/etc/scenari/send.confidential: New "confidential" list model.
These lists are used for groups who don't want any publicity around
their activities; All possible restrictions are applied to prevent
unauthorized users to know these lists exist and to learn anything
about them.
[8454] web_tt2/footer.tt2, web_tt2/tt2_error.tt2: Removing references
to the Sympa version in web pages to avoid pages to be searched by bad
guys willing to exploit known vulnerabilities on out of date servers.
For complete list of changes, see
http://www.sympa.org/distribution/latest-stable/NEWS
Do it for all packages that
* mention perl, or
* have a directory name starting with p5-*, or
* depend on a package starting with p5-
like last time, for 5.18, where this didn't lead to complaints.
Let me know if you have any this time.
a) refer 'perl' in their Makefile, or
b) have a directory name of p5-*, or
c) have any dependency on any p5-* package
Like last time, where this caused no complaints.
series, users are encouraged to read /usr/pkg/share/doc/sympa/NEWS for
details. Summary of new features:
*** New bulk.pl daemon installed with Sympa. This daemon is dedicated to
mail distribution and allows to parallelize this process on a single
server as well as on multiple servers.
*** [Olivier Lumineau, CRU] Fresh new web CSS for the web interface.
*** Replace storage of password with encryption by md5 fingerprint. This
make remind password impossible. So now a one time ticket table is created
ticket are sent by email as an authentication token. Ticket can be used
for lost password, create account, moderation request . It should be
generalized to all operation that need a email chalenge.
*** New propertie in object message : spam_status . This feature is used in
modindex (listing of message waiting for moderation) to show message
tagued as spam.
4 new parameters :
- antispam_feature default off
- antispam_tag_header_name default X-Spam-Status
- antispam_tag_header_spam_regexp default ^\s*Yes
- antispam_tag_header_ham_regexp default ^\s*No
*** DKIM : Sympa now supports DKIM for message diffusion and control.
*** web_tt2/Makefile.am, web_tt2/ca.tt2, web_tt2/lca.tt2,
wwsympa/wwsympa.fcgi.in: It is now possible to create Custom actions
at the list or robot level. These custom actions allow you to create
new pages in the Sympa web interface. for now, you can only display
informations using this method. any post treatment (such as form
submission) must be handled outside of Sympa. See
https://www.sympa.org/manual_6.1/customizing#custom_actions for more
details.
*** [Submitted by J. jourdan] "suspension of membership." The user can suspend
his subscription to the lists
that he subscribes. For a finite length or not. Added a calendar in
javascript to select a date.
Also, lots of translastion updates, and bug fixes (including security ones)
to trigger/signal a rebuild for the transition 5.10.1 -> 5.12.1.
The list of packages is computed by finding all packages which end
up having either of PERL5_USE_PACKLIST, BUILDLINK_API_DEPENDS.perl,
or PERL5_PACKLIST defined in their make setup (tested via
"make show-vars VARNAMES=..."), minus the packages updated after
the perl package update.
sno@ was right after all, obache@ kindly asked and he@ led the
way. Thanks!
escalation vulnerabilities) and updated translations:
* Sympa was not fully compliant to the RFC 2616, leading for example
to possible unwanted list deletion by administrators using prefetching
tools. This was fixed by replacing all the threatening GET requests
by POST requests;
* Use of sprint() function for creating SQL queries lead to possible
SQL injection through cookie manipulation;
* The use of files in /tmp lead to vulnerabilities.
Features:
po/ja.po, po/web_help_ja.po: update Japanese translation of the user
interface, add Japanese translation of online help
po/ru.po: Updated Russian translation.
src/Commands.pm: [#3990][Submitted by A. Berstein, electricembers.net] The
quiet option has been reactivated for the "reject" mail command.
Bug fixes:
wwsympa/archived.pl: [Reported by M. Kretchner, INRIA] It was impossible
to remove a message from web archives or rebuild these archives.
check_perl_modules.pl: [Reported by M. Gorecka-Wolniewicz,
Nicolaus Copernicus univ., Torun] In some cases, CAS logout didn't work.
src/task_manager.pl, wwsympa/archived.pl, wwsympa/bounced.pl: [#3957]
[Reported by O. Berger, Telecom & Management SudParis] When launching
Sympa daemons (other than sympa.pl) with an unknown option, the daemon
was still launched instead of failing to launch.
to trigger/signal a rebuild for the transition 5.8.8 -> 5.10.0.
The list of packages is computed by finding all packages which end
up having either of PERL5_USE_PACKLIST, BUILDLINK_API_DEPENDS.perl,
or PERL5_PACKLIST defined in their make setup (tested via
"make show-vars VARNAMES=...").
