Behaviour changes:
- classesmatching(): order of classes changed
- Suppress standard services noise on SUSE
Fixes:
- Reduce verbosity of yum package module
- Reduce verbosity of apt_get package module
- Upgrade dependencies to latest patch versions.
Upgraded libraries:
- curl 7.47.0
- libxml2 2.9.3
- LMDB 0.9.18
- MySQL 5.1.72
- OpenLDAP 2.4.44
- OpenSSL 1.0.2g
- PCRE 8.38
- PostgreSQL 9.3.11
- Redis 2.8.24
- rsync 3.1.2
PHP was kept at 5.6.17 because of problems with the 5.6.19 version.
- parse def.json vars, classes in C
- Namespaced classes can now be specified on the command line.
- getvalues() will now return a list also for data containers,
and will descend recursively into the containers.
- @if minimum_version now correctly ignores lines starting with '@'
- Fix definition of classes from augments file
- Don't follow symbolic links when copying extended attributes.
- Fix ps options for FreeBSD to check processes only in current
host and not in jails
- Fix cf-serverd error messages with classic protocol clients
- The isvariable() function call now correctly accepts all
array variables when specified inline. Previously it would not
accept certain special characters, even though they could be
specified indirectly by using a variable to hold it.
- Show errors regarding failure to copy extended attributes
when doing a local file copy. Errors could happen when copying
across two different mount points where the support for extended
attributes is different between the mount points.
- Fix bad option nlwp to vzps on Proxmox / OpenVZ.
- Fix file descriptor leak when there are network errors.
- Fix a regression which would sometimes cause "Permission
denied" errors on files inside directories with very restricted
permissions.
- Check for empty server response in RemoteDirList after
decryption
- Allow def.json up to 5MB instead of 4K.
- Add guard for binary upgrade during bootstrap
- Fix HP-UX specific bug that caused a lot of log output to
disappear.
- Fix a bug which sometimes caused package promises to be
skipped with "XX Another cf-agent seems to have done this since
I started" messages in the log, most notably in long running
cf-agent runs (longer than one minute).
- Define (bootstrap|failsafe)_mode during update.cf when triggerd
from failsafe.cf
- Fix two cases where action_policy warn still produces errors
- Fix classes being set because of hash collision in the
implementation.
- fix build failure on FreeBSD 7.1
- Installing packages containing version numbers using yum
now works correctly.
Patch based largely on a pull request by bahamat@:
https://github.com/joyent/pkgsrc/pull/307
3.7.2:
Bug fixes:
- readfile() and read*list() should print an error if they fail to read file.
(Redmine #7702)
- Fix 'AIX_PREINSTALL_ALREADY_DONE.txt: cannot create' error
message on AIX.
- If there is an error saving a mustache template file
it is now logged with log-level error (was inform).
- Change: Clarify bootstrap/failsafe reports
- Fixed several bugs which prevented CFEngine from loading
libraries from the correct location. This affected several platforms.
(Redmine #6708)
- If file_select.file_types is set to symlink and there
are regular files in the scanned directory, CFEngine no longer
produces an unneccessary error message. (Redmine #6996)
- Fix: Solaris packages no longer contain duplicate library
files, but instead symlinks to them. (Redmine #7591)
- cf-agent, cf-execd, cf-promises, cf-runagent and cf-serverd honor
multiple -D, -N and -s arguments (Redmine #7191)
- Fix "@endif" keyword sometimes being improperly processed
by policy parser. (Redmine #7413)
- It is possible to edit the same value in multiple regions
of one file. (Redmine #7460)
- Fix select_class not setting class when used in common bundle with slist.
(Redmine #7482)
- Fix broken HA policy for 3rd disaster-recovery node.
- Directories should no more be changed randomly
into files. (Redmine #6027)
- Include latest security updates for 3.7.
- Reduce malloc() thread contention on heavily loaded
cf-serverd, by not exiting early in the logging function, if no message
is to be printed. (Redmine #7624)
- Improve cf-serverd's lock contention because of getpwnam()
call. (Redmine #7643)
- action_policy "warn" now correctly produces warnings instead
of various other verbosity levels. (Redmine #7274)
- Change: Improve efficiency and debug reports (Redmine #7527)
- Change package modules permissions on hub package so that
hub can execute package promises. (Redmine #7602)
- No longer hang when changing permissions/ownership on fifos
(Redmine #7030)
- Fix exporting CSV reports through HTTPS. (Redmine #7267)
- failsafe.cf will be created when needed. (Redmine #7634)
- Mustache templates: Fix key when value is not a
primitive. The old behavior, when iterating across a map or array of
maps, was to abort if the key was requested with . The new
behavior is to always replace with either the key name or the
iteration position in the array. An error is printed if is used
outside of a Mustache iteration section.
- Legacy package promise: Result classes are now defined if
the package being promised is already up to date. (Redmine #7399)
- TTY detection should be more reliable. (Redmine #7606)
Masterfiles:
- Add: Path to svcprop in stdlib
- Add: New `results` classes body [] (Redmine #7418, #7481)
- Remove: Support for email settings from augments_file (Redmine #7682)
3.7.1:
Bug fixes:
- Fix daemons not restarting correctly on upgrade on AIX. (Redmine #7550)
- Fix upgrade causing error message under systemd because of open ports.
- Fix build with musl libc. (Redmine #7455)
- Long promiser strings with multiple lines are now
abbreviated in logs. (Redmine #3964)
- Fixed a bug which could cause daemons to not to be killed
correctly when upgrading or manually running "service cfengine3 stop".
(Redmine #7193)
- Package promise: Fix inability to install certain packages
with numbers.
- Fix package promise not removing dependant packages. (Redmine #7424)
- Fix warning "Failed to parse csv file entry" with certain very long
commands promises. (Redmine #7400)
- Fix misaligned help output in cf-hub. (Redmine #7273)
- Augmenting inputs from the augments_file (Redmine #7420)
- Add support for failover to 3rd HA node located outside cluster.
- Upgrade all dependencies for patch release.
- Fix a bug which caused daemons not to be restarted on
upgrade. (Redmine #7528)
3.7.0:
New features:
- New package promise implementation.
- Full systemd support for all relevant platforms
- New classes to determine whether certain features are enabled:
* feature_yaml
* feature_xml
For the official CFEngine packages, these are always enabled, but
packages from other sources may be built without the support.
- New readdata() support for generic data input (CSV, YAML, JSON, or auto)
- YAML support: new readyaml() function and in readdata()
- CSV support: new readcsv() function and in readdata()
- New string_mustache() function
- New data_regextract() function
- eval() can now be called with "class" as the "mode" argument, which
will cause it to return true ("any") if the calculated result is
non-zero, and false ("!any") if it is zero.
- New list_ifelse() function
- New mapjson() function as well as JSON support in maparray().
- filestat() function now supports "xattr" argument for extended
attributes.
- "ifvarclass" now has "if" as an alias, and "unless" as an inverse
alias.
- Ability to expand JSON variables directory in Mustache templates:
Prefix the name with '%' for multiline expansion, '$' for compact
expansion.
- Ability to expand the iteration *key* in Mustache templates with @
- Canonical JSON output: JSON output has reliably sorted keys so the
same data structure will produce the same JSON every time.
- New "@if minimum_version(x.x)" syntax in order to hide future language
improvements from versions that don't understand them.
- compile time option (--with-statedir) to
override the default state/ directory path.
- Fix error messages/ handling in process signalling which no longer
allowed any signals to fail silently
- Also enable shortcut keyword for cf-serverd classic protocol, eg to
simplify the bootstrap process for clients that have different
sys.masterdir settings (Redmine #3697)
- methods promises now accepts the bundle name in the promiser string,
as long as it doesn't have any parameters.
- In a services promise, if the service_method bundle is not specified,
it defaults to the promiser string (canonified) with "service_" as a
prefix. The bundle must be in the same namespace as the promise.
- inline JSON in policy files: surrounding with parsejson() is now
optional *when creating a new data container*.
- New data_expand() function to interpolate variables in a data container.
- Add configurable network bandwidth limit for all outgoing
connections ("bwlimit" attribute in "body common control") . To
enforce it in both directions, make sure the attribute is set on both
sides of the connection.
- Secure bootstrap has been facilitated by use of
"cf-agent --boostrap HUB_ADDRESS --trust-server=no"
- Implement new TLS-relevant options (Redmine #6883):
- body common control: tls_min_version
- body server control: allowtlsversion
- body common control: tls_ciphers
- body server control: allowciphers (preexisting)
Changes:
- Improved output format, less verbose, and messages are grouped.
- cf-execd: agent_expireafter default was changed to 120 minutes
(Redmine #7113)
- All embedded databases are now rooted in the state/ directory.
- TLS used as default for all outgoing connections.
- process promise now reports kept status instead of repaired if a
signal is not sent, even if the restart_class is set. The old
behavior was to set the repaired status whenever the process was not
running. (Redmine#7216).
- Bootstrapping requires keys to be generated in advance using cf-key.
- Disable class set on reverse lookup of interfaces IP addresses.
(Redmine #3993, Redmine #6870)
- Define a hard class with just the OS major version on FreeBSD.
- Abort cf-agent if OpenSSL's random number generator can't
be seeded securely.
- Masterfiles source tarball now installs using the usual commands
"./configure; make install".
- Updated Emacs syntax highlighting template to support the latest
syntax enhancements in 3.7.
Deprecations:
- Arbitrary arguments to cfruncommand (using "cf-runagent -o") are
not acceptable any more. (Redmine #6978)
- 3.4 is no longer supported in masterfiles.
Bug fixes:
- Fix server common bundles evaluation order (Redmine#7211).
- Limit LMDB disk usage by preserving sparse areas in LMDB files
(Redmine#7242).
- Fixed LMDB corruption on HP-UX 11.23. (Redmine #6994)
- Fixed insert_lines failing to converge if preserve_block was used.
(Redmine #7094)
- Fixed init script failing to stop/restart daemons on openvz/lxc
hosts. (Redmine #3394)
- rm_rf_depth now deletes base directory as advertised. (Redmine #7009)
- Refactored cf-agent's connection cache to properly differentiate
hosts using all needed attributes like host and port.
(Redmine #4646)
- Refactored lastseen database handling to avoid inconsistencies.
(Redmine #6660)
- cf-key --trust-key now supports new syntax to also update the
lastseen database, so that clients using old protocol will trust
the server correctly.
- Fixed a bug which sometimes caused an agent or daemon to kill or stop
itself. (Redmine #7075, #7244)
- Fixed a bug which made it difficult to kill CFEngine daemons,
particularly cf-execd. (Redmine #6659, #7193)
- Fixed a bug causing systemd not to be detected correctly on Debian.
(Redmine #7297)
- "cf-promises -T" will now correctly report the checked out commit,
even if you haven't checked out a Git branch. (Redmine #7332)
- Reduce verbosity of harmless errors related to socket timeouts and
missing thermal zone files. (Redmine #6486 and #7238)
- Fix process_result logic to match the purpose of body process_select
days_older_than (Redmine #3009)
Masterfiles:
Added:
- Support for user specified overring of framework defaults without
modifying policy supplied by the framework itself (see
example_def.json)
- Support for def.json class augmentation in update policy
- Run vacuum operation on postgresql every night as a part of
maintenance.
- Add measure_promise_time action body to lib (3.5, 3.6, 3.7, 3.8)
- New negative class guard `cfengine_internal_disable_agent_email` so
that agent email can be easily disabled by augmenting def.json
Changed:
- Relocate def.cf to controls/VER/
- Relocate update_def to controls/VER
- Relocate all controls to controls/VER
- Only load cf_hub and reports.cf on CFEngine Enterprise installs
- Relocate acls related to report collection from bundle server
access_rules to controls/VER/reports.cf into bundle server
report_access_rules
- Re-organize cfe_internal splitting core from enterprise specific
policies and loading the appropriate inputs only when necessary
- Moved update directory into cfe_internal as it is not generally
intended to be modified
- services/autorun.cf moved to lib/VER/ as it is not generally intended
to be modified
- To improve predictibility autorun bundles are activated in
lexicographical order
- Relocate services/file_change.cf to cfe_internal/enterprise. This
policy is most useful for a good OOTB experience with CFEngine
Enterprise Mission Portal.
- Relocate service_catalogue from promsies.cf to services/main.cf. It is
intended to be a user entry. This name change correlates with the main
bundle being activated by default if there is no bundlesequence
specified.
- Reduce benchmarks sample history to 1 day.
- Update policy no longer generates a keypair if one is not found.
(Redmine: #7167)
- Relocate cfe_internal_postgresql_maintenance bundle to lib/VER/
- Set postgresql_monitoring_maintenance only for versions 3.6.0 and
3.6.1
- Move hub specific bundles from lib/VER/cfe_internal.cf into
lib/VER/cfe_internal_hub.cf and load them only if policy_server policy
if set.
- Re-organize lib/VER/stdlib.cf from lists into classic array for use
with getvalues
Removed:
- Diff reporting on /etc/shadow (Enterprise)
- Update policy from promise.cf inputs. There is no reason to include
the update policy into promsies.cf, update.cf is the entry for the
update policy
- _not_repaired outcome from classes_generic and scoped_classes generic
(Redmine: # 7022)
Fixes:
- standard_services now restarts the service if it was not already
running when using service_policy => restart with chkconfig (Redmine
#7258)
Problems found with existing digests:
Package memconf distfile memconf-2.16/memconf.gz
b6f4b736cac388dddc5070670351cf7262aba048 [recorded]
95748686a5ad8144232f4d4abc9bf052721a196f [calculated]
Problems found locating distfiles:
Package dc-tools: missing distfile dc-tools/abs0-dc-burn-netbsd-1.5-0-gae55ec9
Package ipw-firmware: missing distfile ipw2100-fw-1.2.tgz
Package iwi-firmware: missing distfile ipw2200-fw-2.3.tgz
Package nvnet: missing distfile nvnet-netbsd-src-20050620.tgz
Package syslog-ng: missing distfile syslog-ng-3.7.2.tar.gz
Otherwise, existing SHA1 digests verified and found to be the same on
the machine holding the existing distfiles (morden). All existing
SHA1 digests retained for now as an audit trail.
Upstream changelog follows.
3.6.5
Features:
- Introduced "systemd" hard class.
- Added paths to dtrace, zfs and zpool on FreeBSD in masterfiles.
Bug fixes:
- Fixed build error on certain RHEL5 and SLES10 setups.
- Fixed a bug which caused dangling symlinks not to be removed.
- Fixed data_readstringarrayidx function not preserving the order of the
array it's producing.
- Fixed a bug which sometimes caused CFEngine to kill the wrong daemon
if both the host and a container inside the host were running
CFEngine.
- Made sure the rm_rf_depth bundle also deletes the base directory.
- Fixed monitord reporting wrongly on open ports.
- Skip adding the class when its name is longer than 1024 characters.
Fixed core dump when the name is too large.
- Fixed detection of stopped process on Solaris.
- Fixed infinite loop plus a couple more minor
bugs in edit_xml promises.
3.6.4
Features:
- Introduced users promises support on HP-UX platform.
- Introduced process promises support on HP-UX platform.
Bug fixes:
- Fixed bug on FreeBSD which sometimes led to the wrong process being killed
- Fixed package version comparison sometimes failing with rpm package manager
- Fixed a bug in users promises which would sometimes set the wrong
password hash if the user would also be unlocked at the same time.
- Fixed a bug on AIX which would occationally kill the wrong process.
- Improved error message for functions that require an absolute path.
- Fixed some spelling errors in examples.
- Fixed error in out-of-tree builds when building cf-upgrade.
- Fixed a bug which would make cf-agent exit with an error if it was
built with a custom log directory, and that directory did not exist.
- Fixed ordering of evaluating promises when depends_on is used.
- Skip non-empty directories silently when recursively deleting.
- Fix memory exhaustion with list larger than 4994 items.
- Fix cf-execd segfault on IP address detection
- Fix hard class detection of RHEL6 ComputeNode
3.6.3
New features:
- support for HP-UX 11.23 and later
- experimental support for Red Hat Enterprise Linux 7
Bug fixes:
- fix getindices on multi-dimensional arrays
- fix mustache template method to run in dryrun mode
- set mailto and mailfrom settings for execd in def.cf
- fix conflation of multi-index entries in arrays
- fix promise locking when transferring using update.cf
- update JSON parser to return an error on truncation
- fix sys.hardware_addresses not expanded
- fix opening database txn /var/cfengine/cf_lastseen.lmdb:
MDB_READERS_FULL when running cf-keys --show-hosts
- fix segfault (Null pointer dereference) when select_end in
delete_lines never matches
- fix max_file_size => "0" not disabling or allowing any size
- fix ifvarclass, with iteration over list, failing when deleting
files with time condition
- fix classes defined with "or" constraint are never set if any value
doesn't evaluate to a scalar
- update "mailfrom" default in default policy
- fix logrotate ambiguity of filename
- fix parsing JSON files
- reduce write count activity to /var partition
- fix files delete attribute incorrectly triggering promise_kept
- update services bundle output related to chkconfig when run in
inform mode.
- fix Solaris serverd tests
- fix broken bechaviour of merging arrays with readstringarray
- fix ifelapsed bug with bundle nesting
- fix handling cf_null in bundlesequence
- fix maparray reading whole input array when using subarray
- fix directories being randomly changed to files
- update defaults promise type to work with classes
- systemd integration in services promises
- fix touch attribute ignoring action = warn_only
- fix 4KB string limit in functions readfile, string_downcase,
string_head, string_reverse, string_length, string_tail,
string_upcase
3.6.2
Bug fixes:
- don't regenerate software_packages.csv every time
- improve verbose message for package_list_command
- fix missing log output on AIX
- assorted fixes to dirname() esp on Windows
- fix package manager detection
- fix build issues on FreeBSD
- allow copying of dead symbolic links
- preserve order in readstringarrayidx
- fix passing of unexpanded variable references to arrays
- use entries for new {admin,deny}_{ips,hostnames} constraints in
the relevant legacy lists
- cope with ps's numeric fields overflowing to the right
- interpret failing function calls in ifvarclass as class not set
- remove unexpanded lists when extending lists
- infer start-time of a process from elapsed when needed
- fix input range definition for laterthan() function
- don't add trailing delimiter when join()'ing lists ending with a
null-value
- 9999999999 (ten 9s) or higher has been historically used as an upper
bound in CFEngine code and policy but because of overflow on 32-bit
platforms it caused problems with big numbers. Fixed in two ways:
first change all existing policy uses to 999999999 (nine 9s instead
of eleven 9s), second fix the C code to not wrap-around in case of
overflow, but use the LONG_MAX value
- cf-serverd and other daemons no longer reload their configuration
every minute if CFEngine is built with an inputs directory outside
of the work directory (not the default).
3.6.1
New features:
- Introduced Solaris and AIX support into the 3.6 series, with many associated
build and bug fixes.
Changes:
- Short-circuit evaluation of classes promises if class is already set
- fix to assume all non-specified return codes are failed in commands
promises
- cf-serverd logs reconfiguration message to NOTICE (was INFO) so that it's
always logged in syslog
Bug fixes:
- File monitoring has been completely rewritten (changes attribute in files
promise), which eliminates many bugs, particularly regarding files that
are deleted. Upgrading will keep all monitoring data, but downgrading again
will reinitialize the DB, so all files will be reported as if they were new.
- $(this.promiser) expands in files promises for 'transformer',
'edit_template', 'copy_from.source', 'file_select.exec_program', 'classes'
and 'action' bodies
- 'body changes' notifies about disappeared files in file monitoring
- Set not-kept classes when files or commands promise should be repaired,
but is warn-only
- Fixed CFEngine template producing a zero-sized file
- Add 0-9 A-Z _ to allowed context of module protocol
- Extend ps command column width on Solaris and filter on zone
rather than adding it to the ps output.
- Fixed strftime() function on Solaris when called with certain specifiers.
- Fixed users promise bug regarding password hashes in a NIS/NSS setup.
- Fixed $(sys.uptime), $(sys.systime) and $(sys.sysday) in AIX.
#5148, #5206)
- Fixed processes_select complaining about "Unacceptable model uncertainty
examining processes"
- ps command for linux has been changed to cope with big rss values
- Address ps -axo shift on FreeBSD 10 and later
- methods and services promises respect action_policy => "warn"
- LMDB should no longer deadlock if an agent is killed on the hub while
holding the DB lock. Note that the change only affects binary packages
shipped by CFEngine, since the upstream LMDB project has not yet integrated
the change.
3.6.0
Changes:
- Changes to logging output
- add process name and pid in syslog message (GitHub #789)
- cf-serverd logging levels are now more standardised:
- INFO logs only failures
- VERBOSE logs successful requests as well
- DEBUG logs actual protocol traffic.
- cf-serverd now logs the relevant client IP address on
each message.
- Logging contexts to local database (cf_classes.tcdb) has been deprecated.
- 'usebundle' promisees are logged for all the bundle promises
- output from 'reports' promises has nothing prefixed except 'R: '
- a log line with stack path is generated when the promise type evaluated
changes
- LMDB (symas.com/mdb) is the default database for local data storage : use
version 0.9.9 or later. cf-agent --self-diagnostics (-x) is only
implemented for TCDB, not for LMDB
- port argument in readtcp() and selectservers() may be a
service name (e.g. "http", "pop3").
- Enable source file in agent copy_from promises to be a relative path.
- file "changes" reporting now reports with log level "notice", instead
of "error".
- process_results default to AND'ing of set attributes if not specified
- interface is now canonified in sys.hardware_mac[interface] to align with
sys.ipv4[interface]
- cf-promises no longer errors on missing bodies when run without
--full-check (-c)
- Linux flavor "SUSE" now correctly spelled with all uppercase in variables
and class names. The "suse" lowercase version is also
provided for convenience.
- $(this.promise_filename) and $(..._dirname) variables are now absolute
paths.
- including the same file multiple times in 'body control inputs' is
not an error
- portnumber in body copy_from now supports service names like "cfengine",
"pop3" etc, check /etc/services for more.
- The failsafe.cf policy, run on bootstrap and in some other
unusual cases, has been extracted from C code into libpromises/failsafe.cf
- masterfiles
- cf_promises_validated is now in JSON format
- timestamp key is timestamp (sec since unix epoch) of last time validated
- the masterfiles now come from https://github.com/cfengine/masterfiles
and are not in the core repository
- cf-serverd calls cf-agent with -Dcfruncommand when executing cf-runagent
requests
Mark as removed: promise_notkept_log_include, promise_notkept_log_exclude,
promise_repaired_log_include, promise_repaired_log_exclude, classes_include,
classes_exclude, variables_include, variables_exclude attributes from
report_data_select body (syntax is valid but not functional). They have been
replaced by the following attributes: promise_handle_include,
promise_handle_exclude, metatags_include, metatags_exclude.
New features:
- New promise type "users" for managing local user accounts.
- TLS authentication and fully encrypted network protocol.
Additions specific to the new type of connections:
- New attribute "allowlegacyconnects" in body server control,
which enables serving policy via non-latest cfengine protocol,
to the given list of hosts. If the option is absent, it
defaults to allow all hosts. To refuse non-TLS connections,
specify an empty list.
- New attribute "protocol_version" in body copy_from, and body
common control, which defines the preferred protocol for
outgoing connections.. Allowed values at the moment: "0" or
"undefined", "classic" or "1", "latest" or "2". By leaving the
copy_from option as undefined the common control option is
used, and if both are undefined then classic protocol is used
by default.
- The new networking protocol uses TLS for authentication,
after which all dialog is encrypted within the established
TLS session. cf-serverd is still able to speak the legacy
protocol with old agents.
- The 'skipverify' option in 'body server control' is
deprecated and only left for compatibility; it does
nothing
- cf-serverd does not hang up the connection if some request
fails, so that the client can add more requests.
- For the connections using the new protocol, all of the
paths in bundle server access_rules now differentiate
between a directory and a file using the trailing
slash. If the path exists then this is auto-detected and
trailing slash appended automatically. You have to append
a trailing slash manually to an inexistent or symbolic
path (e.g. "/path/to/$(connection.ip)/") to force
recursive access.
- New in 'access' promises for 'bundle server access_rules'
- Attributes "admit_ips", "admit_hostnames", "admit_keys",
"deny_ips", "deny_hostnames", "deny_keys"
- "admit_keys" and "deny_keys" add the new functionality
of controlling access according to host identity,
regardless of the connecting IP.
- For these new attributes, regular expressions
are not allowed, only CIDR notation for "admit/deny_ips", exact
"SHA=..." strings for "admit/deny_keys", and exact hostnames
(e.g. "cfengine.com") or subdomains (starting with dot,
e.g. ".cfengine.com") for "admit/deny"_hostnames. Same rules
apply to 'deny_*' attributes.
- These new constraints and the paths in access_rules, can contain
special variables "$(connection.ip)", "$(connection.hostname)",
"$(connection.key)", which are expanded dynamically for every
received connection.
- For connections using the new protocol, "admit" and "deny"
constraints in bundle server access_rules are being phased
out, preferred attributes are now "admit_ips", "deny_ips",
"admit_hostnames", "deny_hostnames", "admit_keys",
"deny_keys".
- New "shortcut" attribute in bundle server access_rules used to
dynamically expand non-absolute request paths.
- masterfiles
- standard library split: lib/3.5 (compatibility) and lib/3.6
(mainline)
- many standard library bundles and bodies, especially packages-
and file-related, were revised and fixed
- supports both Community and Enterprise
- new 'inventory/' structure to provide OS, dmidecode, LSB, etc.
system inventory (configured mainly in def.cf)
- cf_promises_release_id contains the policy release ID which is
the GIT HEAD SHA if available or hash of tree
- a bunch'o'bundles to make starting with CFEngine easier:
- file-related: file_mustache, file_mustache_jsonstring, file_tidy,
dir_sync, file_copy, file_link, file_hardlink, file_empty, file_make
- packages-related: package_absent, package_present, package_latest,
package_specific_present, package_specific_absent,
package_specific_latest, package_specific
- XML-related: xml_insert_tree_nopath, xml_insert_tree, xml_set_value,
xml_set_attribute
- VCS-related: git_init, git_add, git_checkout,
git_checkout_new_branch, git_clean, git_stash, git_stash_and_clean,
git_commit, git
- process-related: process_kill
- other: cmerge, url_ping, logrotate, prunedir
- New command line options for agent binaries
- New options to cf-promises
- '--show-classes' and '--show-vars'
- '--eval-functions' controls whether cf-promises should evaluate
functions
- Colorized output for agent binaries with command line option '--color'
(auto-enabled if you set CFENGINE_COLOR=1)
- New language features
- New variable type 'data' for handling of structured data (ie JSON),
including supporting functions:
- 'data_readstringarray' - read a delimited file into a data map
- 'data_readstringarrayidx' - read a delimited file into a data array
- 'datastate' - create a data variable with currently set classes
and variables
- 'datatype' - determine the type of the top element of a container
- 'format' - %S can be used to serialize 'data' containers into
a string
- 'mergedata' - merge two data containers, slists/ilists/rlists,
or "classic" arrays into a data container
- 'parsejson' - create a data container from a JSON string
- 'readjson' - create a data container from a file that contains JSON
- 'storejson' - serialize a data container into a string
- Most functions operating on lists can also operate on data containers
- pass a data container to a bundle with the @(container) notation
- the module protocol accepts JSON for data containers with
the '%' sigil
- Tagging of classes and variables allows annotating of language
construct with meta data; supporting functionality:
- The module protocol in 'commands' promises has been extended
to allow setting of tags of created variables and classes,
and the context of created variables
- 'getclassmetatags' - returns list of meta tags for a class
- 'getvariablemetatags' - returns list of meta tags for a variable
- 'body file control' has an 'inputs' attribute to include library files
and other dependencies
- bundlesequences can be built with bundlesmatching() based on bundle
name and tags
- New attributes in existing promise types and bodies
- New option 'preserve_all_lines' for insert_type in insert_lines promises
- Caching of expensive system functions to avoid multiple executions of
execresult() etc, can be controlled via cache_system_functions
attribute in body common control
- New option 'mailsubject' in body executor control allows defining
the subject in emails sent by CFEngine
- Support for Mustache templates in 'files' promises; use
'template_method' and 'template_data' attributes. Without
'template_data' specified, uses datastate().
- New and improved functions
- 'bundlesmatching' - returns list of defined bundles matching
a regex and tags
- 'canonifyuniquely' - converts a string into a unique, legal class name
- 'classesmatching' - returns list of set classes matching a regex
and tags
- 'eval' - evaluates mathematical expressions; knows SI k, m, g
quantifiers, e.g. "100k"
- 'findfiles' - list files matching a search pattern; use "**" for
recursive searches
- 'makerule' - evaluates whether a target file needs to be rebuilt
from sources
- 'max', 'min' - returns maximum and minimum of the numbers in
a container or list (sorted by a 'sort' method)
- 'mean' - returns the mean of the numbers in a container or list
- 'nth' - learned to look up by key in a data container holding a map
- 'packagesmatching' - returns a filtered list of installed packages.
- 'readfile' - learned to read system files of unknown size like those
in /proc
- 'sort' - can sort lexicographically, numerically (int or real), by IP,
or by MAC
- 'string_downcase', 'string_upcase' - returns the lower-/upper-case
version of a string
- 'string_head', 'string_tail' - returns the beginning/end of a string
- 'string_length' - returns the length of a string
- 'string_reverse' - reverses a string
- 'string_split' - improved implementation, deprecates 'splitstring'
- 'variablesmatching' - returns a list of variables matching a regex
and tags
- 'variance' - returns the variance of numbers in a list or container
- New hard classes
- Introduced alias 'policy_server' for context 'am_policy_hub'
(the latter will be deprecated)
- all the time-based classes have GMT equivalents
- New variables
- 'sys.bindir' - the location of the CFEngine binaries
- 'sys.failsafe_policy_path' - the location of the failsafe policy file
- 'sys.inputdir' - the directory where CFEngine searches for policy files
- 'sys.key_digest' - the digest of the host's cryptographic key
- 'sys.libdir', 'sys.local_libdir' - the location of the CFEngine libraries
- 'sys.logdir' - the directory where the CFEngine log files are saved
- 'sys.masterdir' - the location of masterfiles on the policy server
- 'sys.piddir' - the directory where the daemon pid files are saved
- 'sys.sysday' - the number of days since the beginning of the UNIX epoch
- 'sys.systime' - the number of seconds since the beginning of the UNIX epoch
- 'sys.update_policy_path' - the name of the update policy file
- 'sys.uptime' - the number of minutes the host has been online
- 'this.promise_dirname' - the name of the file in which the current promise
is defined
- 'this.promiser_uid' - the ID of the user running cf-agent
- 'this.promiser_gid' - the group ID of the user running cf-agent
- 'this.promiser_ppid' - the ID of the parent process running cf-agent
Deprecations:
- 'splitstring' - deprecated by 'string_split'
- 'track_value'
- 'skipverify'
Bug fixes: for a complete list of fixed bugs, see https://cfengine.com/dev
- various fixes in evaluation and variable resolution
- Improve performance of list iteration
- Removed limitation of input length to internal buffer sizes
- directories ending with "/" are not ignored
- lsdir() always return a list now, never a scalar
- 'abortclasses' fixed to work in common bundles and other cases
- namespaced 'edit_line' bundles now work
- lists are interpolated in correct order
- cf-serverd reloads policies properly when they change
- lots of leaks (memory and file descriptor) fixed
3.5.3
Changes:
- Improved security checks of symlink ownership. A symlink created by a user
pointing to resources owned by a different user will no longer be followed.
- Changed the way package versions are compared in package promises.
In previous versions the comparison was inconsistent. This has been fixed,
but may also lead to behavior changes in certain cases.
Bug fixes:
- fix cf-monitord crash due to incorrect array initialization
- fix cf-serverd stat()'ing the file tree every second
- correctly populate sys.hardware_addresses variable
- add support for Debian's GNU/kfreebsd to build system
- fix possible stack corruption in guest_environments promises
- work-around hostname trunctation in HP-UX's uname
- fix body copy purging of empty directories
- make discovery and loading of avahi libraries more robust
- compile and packaging fixes for HP-UX, AIX and Solaris
- fix fatal error in lsdir() when directory doesn't exist
- fix epoch calculation for stime inrange calculation
3.5.2
Bug fixes:
- fix delayed abortclasses checking
- fix maplist arguments bug
- fix segfaults in cf-pomises
- fix build on Solaris 10/SmartOS
- sanitize characters from /etc/issue in sys.flavor for Debian
- Fix segfault when dealing with files or data > 4K
- Don't truncate keys to 126 characters in getindices
- files created via log_* actions now have mode 600
- fix wrong log message when a promise is ignored due to 'ifvarclass'
not matching
- fix lifetime of persistent classes
- fix segfault when process_select body had no process_result attribute
Default to AND'ed expression of all specified attributes
- include system message in output when acl promises fail
- fix invocation of standard_services bundle and corresponding promise
compliance
3.5.1
Changes:
- file changes are logged with log level Notice, not Error
- the CFEngine Standard Library in masterfiles/libraries is now split into
promise-type specific policy files, and lives in a version-specific
directory. This should have no impact on current code, but allows more
granular include of needed stdlib elements
Bug fixes:
- fix recursive copying of files
- respect classes in templates
- fix timestamps on Windows
- fix non-root cf-agent flooding syslog
- fix email flood from cf-execd due to timestamps in agent output
- Preserve security context when editing or copying local files
- fix path for sys.crontab on redhat systems
- prevent incorrect "insert_lines promise uses the same select_line_matching
anchor" warning
- Fix regression of setting VIPADDRESS to 127.0.0.1
- Fix "changes" promise not receiving status when file is missing
- Fix symlinks being destroyed when editing them
- Fix missing "promise kept" status for the last line in a file
3.5.0
New features:
- classes promises now take an optional scope constraint.
- new built-in functions: every, none, some, nth, sublist, uniq, filter
- every
- none
- some
- nth
- sublist
- uniq
- filter
- classesmatching
- strftime
- filestat
- ifelse
- maparray
- format
- cf-promises flag --parse-tree is replaced by --policy-output-format=,
requiring the
user to specify the output format (none, cf, json)
- cf-promises allows partial check of policy (without body common control)
without integrity check; --full-check enforces integrity check
- agent binaries support JSON input format (.json file as generated by
cf-promises)
- cf-key: new options --trust-key/-t and --print-digest/-p
- Class "failsafe_fallback" is defined in failsafe.cf when main policy
contains errors and failsafe is run because of this
- add scope attribute for body classes
- Better diagnostics of parsing errors
- Error messages from parser now show the context of error
- new cf-agent option: --self-diagnostics
- new output format, and --legacy-output
- warnings for cf-promises.
- Enable zeroconf-discovery of policy hubs for automatic bootstrapping
if Avahi is present
- Support for sys.cpus on more platforms than Linux & HPUX
Changes:
- parser no longer allows ',' after promiser or promisee. must be either
';' or lval
- Make parser output in GCC compatible format the only supported format
(remove --gcc-brief-format flag)
- Silence license warnings in Enterprise Free25 installations
- action_policy => "warn" causes not_kept classes to be set on promise
needing repair.
- command line option version (-V) now prints a shorter parsable version
without graphic
- implicit execution of server and common bundles taking arguments is
skipped in cf-serverd.
- WARNING: option --policy-server removed, require option to --bootstrap
instead
- process promises don't log if processes are out of range unless you
run in verbose mode
- reports promises are now allowed in any context
- cf-report has been removed
- cf-execd: --once implies --no-fork
- Version info removed from mail subject in the emails sent by cf-execd.
The subject will only contain "[fqname/ipaddress]" instead of
"communnity/nova [fqname/ipaddress]"
Please change your email filters accordingly if necessary.
- "outputs" promise type is retired. Their semantics was not clear, and
the functionality
is better suited for control body setting, not a promise.
- Tokyo Cabinet databases are now automatically checked for
correctness during opening. It should prevent a number of issues
with corrupted TC databases causing binaries to hang.
- Improved ACL handling on Windows, which led to some syntax changes.
We now consistently use the term "default" to describe ACLs that can be
inherited by child objects. These keywords have received new names:
acl_directory_inherit -> acl_default
specify_inherit_aces -> specify_default_aces
The old keywords are deprecated, but still valid. In addition, a new keyword
"acl_inherit" controls inheritance behavior on Windows. This feature does
not exist on Unix platforms.
- Networking code is moved from libpromises to its own library,
libcfnet. Work has begun on making the API more sane and thread-safe.
Lots of legacy code was removed.
- Add getaddrinfo() replacement in libcompat (borrowed from PostgreSQL).
- Replace old deprecated and non thread-safe resolver calls with
getaddrinfo() and getnameinfo().
- Hostname2IPString(), IPString2Hostname() are now thread-safe, and are
returning error when resolution fails.
- Running cf-execd --once now implies --no-fork, and also does not wait
for splaytime to pass.
- execresult(), returnszero() and commands promises no longer requires
the first word word to be an absolute path when using the shell.
- commands promises useshell attribute now accepts "noshell" and "useshell"
values. Boolean values are accepted but deprecated.
- returnszero() now correctly sets the class name in this scenario:
"commandfailed" not => returnszero("/bin/nosuchcommand", "noshell");
Bugfixes:
- bundles are allowed to be empty
- Fixed '.' and '-' not being accepted by a commands module.
- Correct parsing of list variables by a command module.
- Fixed issue with package management and warn.
- Fixed JSON crash.
- Improved error checking when using fgets().
- Fixed error message when deleting nonexistent files.
- Honor warn-only when purging from local directory.
- Make sure "restart" and "reload" are recognized keywords in packages.
- Allocate memory dynamically to avoid out-of-buffer or out-of-hash
situations
- fix edit_xml update of existing attributes
- use failsafe policy from compile-time specified workdir
- ifvarclass checked from classes promises in common bundles
- do not wait for splaytime when executing only once
- disable xml editing functionality when libxml2 doesn't provide necessary APIs
- Out-of-tree builds should work again, fixed a bunch of related bugs.
- Fixed race condition in file editing.
- Fixed memory leak in cf-serverd and others
Bugfixes:
- make qualified arrays expand correcty (Redmine #1998, Mantis #1128)
- correct possible errors in tcdb files when opening
- avoid possible db corruption when mixing read/write and cursor operations
- Allow umask value of 002 (Redmine #2496)
3.4.4 (Bugfix and Stability release)
Bugfixes:
- prevent possible crash when archiving files (GitHub #316)
- don't create symlinks to cf-know in update policy
- don't enable xml support if libxml2 is too old (Redmine #1937)
3.4.3 (Bugfix and Stability release)
Bugfixes:
- Don't flood error messages when processes are out of defined range
- prevent segmentation fault in cf-monitord -x (Redmine #2021)
- when copying files, use same file mode as source file, rather than 0600
(Redmine #1804)
- include xpath in messages generated by edit_xml operations
(Redmine #2057)
Some urgent patches to 3.1.3.
Class validation parse bug fixed.
Global zone handling error for solaris fixed.
Package architectures handled correctly (bug #456).
Reading and writing of key name "root-.pub" eliminated (bug #442, #453).
cf-serverd crash because of race condition on SERVER_KEYSEEN fixed.
Lock purging to avoid remnant complexity explosion (bug #430).
Some copyright notices added that got lost.
3.1.3 (Stability release)
Major memory leaks in cf-monitord, cf-execd, cf-serverd fixed (bug #427).
The daemons now show no growth even with very complex policies.
cf-serverd crash due to race condition in DeleteScope() fixed (bug #406).
Added 30 second timeout on recv() on Linux.
package_noverify_returncode implemented (bug #256).
A flexible mechanism for setting classes based on return codes of
commands has been introduced. Allows for setting promise kept,
repaired or failed based on any return codes. This is currently
implemented for commands-promises, package-manager commands and
transformer in files. In classes body, see attributes
kept_returncodes, repaired_returncodes, failed_returncodes (bug
#248, #329).
New function ip2host - reverse DNS lookup (bug #146).
3.1.2 (Scalability/efficiency release)
Big efficiency improvements by caching output from
cf-promises. Can also be used for much more efficent policy
deployment (only pull if changed).
Caching state of ps command for greater efficiency. Reloaded for each bundle.
Index class lookup improves efficiency of class evaluation for huge configurations.
Fixed issue where certain promiser strings got corrupted.
Minor memory access issues fixed.
Iterator bug introduced in 3.1.0 fixed
3.1.1 (Bugfix release)
Memory leaks in server tracked down and fixed.
List expansion bug (one list items not executed) fixed.
Security issue introduced by change of runcommand shell policy fixed.
If users defined a runcommand for cf-runagent/cf-serverd communication,
possible to execute commands.
cf-key -s command for showing key hash/IP address identity pairs
3.1.0
Change in storage of public keys. Cfengine now hashes the public key and uses this
as the keyname. Keys will be converted automatically.
The old dynamic addresses lists are deprecated.
Caching of dns and key information for greater server speed.
Change in last-seen format reflects the public key usage.
New package policy addupdate - installs package if not there and
updates it otherwise.
Support for package_changes => "bulk" in file repository as well.
New special function readstringarrayidx, similar to readstringarray,
but uses integer indeces. Very useful if first row elements are
not good identifiers (e.g. contains spaces, non-unique, etc.).
Change two log formats to use time() instead of date()
- filechanges
- total compliance
Change from using md5 to sha256 as default digest for commercial version,
community retains md5 for compat.
Commands not returning 0 in commands-promises are flagged
as repair_failed.
Adjustable timeout on connect(). Defaults to 10 seconds, adjustable
with default_timeout in agent control.
Redesign of the knowledge map infrastructure.
Now possible to use variables to call methods, e.g
methods:
"name $(list)" usebundle => $(list)("abc");
See reference manual notes
Changes to normal ordering to optimize execution.
Increased stability by always initializing Attribute and Promise
structures.
When running cf-promises in dry-run mode (-n), the user does not need
to put binaries in WORKDIR/bin. For example, non-privileged users can verify root
policies.
Source control revision added in version string if run in verbose mode
(e.g. "cf-promises -vV"). This needs some refining, uses revision of a header now.
New semantics in return values of list functions. Null values are now allowed
and there is no iteration over empty lists. The value "cf_null" is reserved for
use as a null iterator.
3.0.5p1
Showing paths allowed/denied access to when cf-serverd is run in verbose mode.
Bug in server fixed for dynamic addresses.
File handle closure bugfix - too many open databases.
Seg fault in mount files fix.
Twin used in cf-execd without checking.
Check_root set wrong directory permissions at source not destination.
Error message degraded in body definition.
Undefined body not warned as error.
Various build enahncements.
Package_list_update called only once per manager, and fixed crash.
Version number bug in packages.
3.0.5
Encryption problems fixed - client key buffer was uninitialized.
Classes-promisers are now automatically canonified when class
strings are defined, to simplifying the use of variables in classes.
New scalars sys.cf_version and sys.nova_version that hold Cfengine version information.
Attribute package_delete_convention added, to allow customizable
package name in delete command during update.
package_list_update_ifelapsed limit added.
Private variable $(firstrepo) is available in package_name_convention
and package_delete_convention in order to expand the full path to
a package, which is required by some managers.
Some of the threading code is rewritten and made more robust. This includes
synchronizing access to the lastseen database from the server.
Bad initialization of BSD flags fixed
Multiple variable expansion issues in control fixed for server and agent
Allow ignore_missing_bundles to affect methods: bundles too
Run agent trust dialogue fixed
Bug in CPU monitoring, increasing time scale caused linear decay
of CPU measurement.
Bug in Setuid log storage, fix.
Hooks added for new Nova virtualization promises.
Multithreading mutex failed to collide during cfservd leading to dropped authentication under heavy load.
3.0.4
Class cancellation in promises to create better class feedback,
allows emulation of switch/case semantics etc
Value of SA measurement promises
Special function getenv() which returns the contents of an
environment variable (on all platforms).
New function translatepath for generic Windows
New function escape() to escape literals as regular expressions (like SQL)
New function host2ip for caching IP address lookup
New function regextract for setting variables with backreferences
New variables for the components $(sys.cf_agent), $(sys.cf_know) etc
pointing to the binaries.
More robust integrated database implementation; closing all
handles when receiving signals, self-healing on corruption.
Package installation on localhost without a manager like yum completed,
multiple repositories searched, and universal methods.
Numerous bugfixes
3.0.3
sha256 .. new hashes in openssl included in syntax tree.
End of line autocropping in readfile (hopefully intelligent)
hashmatch function incorrectly implemented -
old debugging code left behind. Fix.
sys.crontab variable
Unknown user is now interpretated as "same user",
so that we give cfengine a chance to fix
Unregistered addresses no longer report "(Non registered IP)",
but return as the address itself when doing reverse lookups.
(Sorry for the long commit message)
for building expert systems which administrate and configure large
computer networks. Cfengine uses the idea of classes and a primitive
form of intelligence to define and automate the configuration of
large systems in the most economical way possible. Cfengine is
designed to be a part of computer immune system.
Cfengine 3 is operationally backwards compatible with Cfengine 2, but the
language is not. Cfengine 3 is not a drop-in replacement for Cfengine 2.
wiz
jperkin
fhajny
agc
pettai
tron
asau
shattered
sekiya