2.17.1:
[BUGFIX] TSDB: Fix query performance regression that increased memory and CPU usage
2.17.0:
[FEATURE] TSDB: Support isolation
[ENHANCEMENT] PromQL: Allow more keywords as metric names
[ENHANCEMENT] React UI: Add normalization of localhost URLs in targets page
[ENHANCEMENT] Remote read: Read from remote storage concurrently
[ENHANCEMENT] Rules: Mark deleted rule series as stale after a reload
[ENHANCEMENT] Scrape: Log scrape append failures as debug rather than warn
[ENHANCEMENT] TSDB: Improve query performance for queries that partially hit the head
[ENHANCEMENT] Consul SD: Expose service health as meta label
[ENHANCEMENT] EC2 SD: Expose EC2 instance lifecycle as meta label
[ENHANCEMENT] Kubernetes SD: Expose service type as meta label for K8s service role
[ENHANCEMENT] Kubernetes SD: Expose label_selector and field_selector
[ENHANCEMENT] Openstack SD: Expose hypervisor id as meta label
[BUGFIX] PromQL: Do not escape HTML-like chars in query log
[BUGFIX] React UI: Fix data table matrix values
[BUGFIX] React UI: Fix new targets page not loading when using non-ASCII characters
[BUGFIX] Remote read: Fix duplication of metrics read from remote storage with external labels
[BUGFIX] Remote write: Register WAL watcher and live reader metrics for all remotes, not just the first one
[BUGFIX] Scrape: Prevent removal of metric names upon relabeling
[BUGFIX] Scrape: Fix 'superfluous response.WriteHeader call' errors when scrape fails under some circonstances
[BUGFIX] Scrape: Fix crash when reloads are separated by two scrape intervals
2.16.0:
[FEATURE] React UI: Support local timezone on /graph
[FEATURE] PromQL: add absent_over_time query function
[FEATURE] Adding optional logging of queries to their own file
[ENHANCEMENT] React UI: Add support for rules page and "Xs ago" duration displays
[ENHANCEMENT] React UI: alerts page, replace filtering togglers tabs with checkboxes
[ENHANCEMENT] TSDB: Export metric for WAL write errors
[ENHANCEMENT] TSDB: Improve query performance for queries that only touch the most recent 2h of data.
[ENHANCEMENT] PromQL: Refactoring in parser errors to improve error messages
[ENHANCEMENT] PromQL: Support trailing commas in grouping opts
[ENHANCEMENT] Scrape: Reduce memory usage on reloads by reusing scrape cache
[ENHANCEMENT] Scrape: Add metrics to track bytes and entries in the metadata cache
[ENHANCEMENT] promtool: Add support for line-column numbers for invalid rules output
[ENHANCEMENT] Avoid restarting rule groups when it is unnecessary
[BUGFIX] React UI: Send cookies on fetch() on older browsers
[BUGFIX] React UI: adopt grafana flot fix for stacked graphs
[BUFGIX] React UI: broken graph page browser history so that back button works as expected
[BUGFIX] TSDB: ensure compactionsSkipped metric is registered, and log proper error if one is returned from head.Init
[BUGFIX] TSDB: return an error on ingesting series with duplicate labels
[BUGFIX] PromQL: Fix unary operator precedence
[BUGFIX] PromQL: Respect query.timeout even when we reach query.max-concurrency
[BUGFIX] PromQL: Fix string and parentheses handling in engine, which affected React UI
[BUGFIX] PromQL: Remove output labels returned by absent() if they are produced by multiple identical label matchers
[BUGFIX] Scrape: Validate that OpenMetrics input ends with
[BUGFIX] Remote read: return the correct error if configs can't be marshal'd to JSON
[BUGFIX] Remote write: Make remote client Store use passed context, which can affect shutdown timing
[BUGFIX] Remote write: Improve sharding calculation in cases where we would always be consistently behind by tracking pendingSamples
[BUGFIX] Ensure prometheus_rule_group metrics are deleted when a rule group is removed
2.15.2:
[BUGFIX] TSDB: Fixed support for TSDB blocks built with Prometheus before 2.1.0.
[BUGFIX] TSDB: Fixed block compaction issues on Windows.
2.15.1:
[BUGFIX] Fixed race on concurrent queries against the same data.
2.15.0:
[CHANGE] Discovery: Removed prometheus_sd_kubernetes_cache_* metrics. Additionally prometheus_sd_kubernetes_workqueue_latency_seconds and prometheus_sd_kubernetes_workqueue_work_duration_seconds metrics now show correct values in seconds.
[CHANGE] Remote write: Changed query label on prometheus_remote_storage_* metrics to remote_name and url.
[FEATURE] API: Added new endpoint for exposing per metric metadata /metadata.
[ENHANCEMENT] TSDB: Significantly reduced memory footprint of loaded TSDB blocks.
[ENHANCEMENT] TSDB: Significantly optimized what we buffer during compaction which should result in lower memory footprint during compaction.
[ENHANCEMENT] TSDB: Improve replay latency.
[ENHANCEMENT] TSDB: WAL size is now used for size based retention calculation.
[ENHANCEMENT] Remote read: Added query grouping and range hints to the remote read request
[ENHANCEMENT] Remote write: Added prometheus_remote_storage_sent_bytes_total counter per queue.
[ENHANCEMENT] promql: Improved PromQL parser performance.
[ENHANCEMENT] React UI: Implemented missing pages like /targets
[ENHANCEMENT] promql: Prometheus now accepts spaces between time range and square bracket. e.g [ 5m]
[BUGFIX] Config: Fixed alertmanager configuration to not miss targets when configurations are similar.
[BUGFIX] Remote write: Value of prometheus_remote_storage_shards_desired gauge shows raw value of desired shards and it's updated correctly.
[BUGFIX] Rules: Prometheus now fails the evaluation of rules and alerts where metric results collide with labels specified in labels field.
[BUGFIX] API: Targets Metadata API /targets/metadata now accepts empty match_targets parameter as in the spec.
2.14.0
[SECURITY/BUGFIX] UI: Ensure warnings from the API are escaped.
[FEATURE] API: /api/v1/status/runtimeinfo and /api/v1/status/buildinfo endpoints added for use by the React UI.
[FEATURE] React UI: implement the new experimental React based UI.
Can be found by under /new.
Not all pages are implemented yet.
[FEATURE] Status: Cardinality statistics added to the Runtime & Build Information page.
[ENHANCEMENT/BUGFIX] Remote write: fix delays in remote write after a compaction.
[ENHANCEMENT] UI: Alerts can be filtered by state.
[BUGFIX] API: lifecycle endpoints return 403 when not enabled.
[BUGFIX] Build: Fix Solaris build.
[BUGFIX] Promtool: Remove false duplicate rule warnings when checking rule files with alerts.
[BUGFIX] Remote write: restore use of deduplicating logger in remote write.
[BUGFIX] Remote write: do not reshard when unable to send samples.
[BUGFIX] Service discovery: errors are no longer logged on context cancellation.
[BUGFIX] UI: handle null response from API properly.
2.13.1:
[BUGFIX] Fix panic in ARM builds of Prometheus.
[BUGFIX] promql: fix potential panic in the query logger.
[BUGFIX] Multiple errors of http: superfluous response.WriteHeader call in the logs.
2.13.0:
[SECURITY/BUGFIX] UI: Fix a Stored DOM XSS vulnerability with query history CVE-2019-10215.
[CHANGE] Metrics: renamed prometheus_sd_configs_failed_total to prometheus_sd_failed_configs and changed to Gauge
[ENHANCEMENT] Include the tsdb tool in builds.
[ENHANCEMENT] Service discovery: add new node address types for kubernetes.
[ENHANCEMENT] UI: show warnings if query have returned some warnings.
[ENHANCEMENT] Remote write: reduce memory usage of the series cache.
[ENHANCEMENT] Remote read: use remote read streaming to reduce memory usage.
[ENHANCEMENT] Metrics: added metrics for remote write max/min/desired shards to queue manager.
[ENHANCEMENT] Promtool: show the warnings during label query.
[ENHANCEMENT] Promtool: improve error messages when parsing bad rules.
[ENHANCEMENT] Promtool: more promlint rules.
[BUGFIX] Promtool: fix recording inconsistency due to duplicate labels.
[BUGFIX] UI: fixes service-discovery view when accessed from unhealthy targets.
[BUGFIX] Metrics format: OpenMetrics parser crashes on short input.
[BUGFIX] UI: avoid truncated Y-axis values.
2.12.0:
[FEATURE] Track currently active PromQL queries in a log file.
[FEATURE] Enable and provide binaries for mips64 / mips64le architectures.
[ENHANCEMENT] Improve responsiveness of targets web UI and API endpoint.
[ENHANCEMENT] Improve remote write desired shards calculation.
[ENHANCEMENT] Flush TSDB pages more precisely.
[ENHANCEMENT] Add prometheus_tsdb_retention_limit_bytes metric.
[ENHANCEMENT] Add logging during TSDB WAL replay on startup.
[ENHANCEMENT] Improve TSDB memory usage.
[BUGFIX] Check for duplicate label names in remote read.
[BUGFIX] Mark deleted rules' series as stale on next evaluation.
[BUGFIX] Fix JavaScript error when showing warning about out-of-sync server time.
[BUGFIX] Fix promtool test rules panic when providing empty exp_labels.
[BUGFIX] Only check last directory when discovering checkpoint number.
[BUGFIX] Fix error propagation in WAL watcher helper functions.
[BUGFIX] Correctly handle empty labels from alert templates.
2.11.0:
[CHANGE] Remove max_retries from queue_config (it has been unused since rewriting remote-write to utilize the write-ahead-log).
[CHANGE] The meta file BlockStats no longer holds size information. This is now dynamically calculated and kept in memory. It also includes the meta file size which was not included before.
[CHANGE] Renamed metric from prometheus_tsdb_wal_reader_corruption_errors to prometheus_tsdb_wal_reader_corruption_errors_total.
[FEATURE] Add option to use Alertmanager API v2.
[FEATURE] Added humanizePercentage function for templates.
[FEATURE] Include InitContainers in Kubernetes Service Discovery.
[FEATURE] Provide option to compress WAL records using Snappy.
[ENHANCEMENT] Create new clean segment when starting the WAL.
[ENHANCEMENT] Reduce allocations in PromQL aggregations.
[ENHANCEMENT] Add storage warnings to LabelValues and LabelNames API results.
[ENHANCEMENT] Add prometheus_http_requests_total metric.
[ENHANCEMENT] Enable openbsd/arm build.
[ENHANCEMENT] Remote-write allocation improvements.
[ENHANCEMENT] Query performance improvement: Efficient iteration and search in HashForLabels and HashWithoutLabels.
[ENHANCEMENT] Allow injection of arbitrary headers in promtool.
[ENHANCEMENT] Allow passing external_labels in alert unit tests groups.
[ENHANCEMENT] Allows globs for rules when unit testing.
[ENHANCEMENT] Improved postings intersection matching.
[ENHANCEMENT] Reduced disk usage for WAL for small setups.
[ENHANCEMENT] Optimize queries using regexp for set lookups.
[BUGFIX] resolve race condition in maxGauge.
[BUGFIX] Fix ZooKeeper connection leak.
[BUGFIX] Improved atomicity of .tmp block replacement during compaction for usual case.
[BUGFIX] Fix "unknown series references" after clean shutdown.
[BUGFIX] Re-calculate block size when calling block.Delete.
[BUGFIX] Fix unsafe snapshots with head block.
[BUGFIX] prometheus_tsdb_compactions_failed_total is now incremented on any compaction failure.
2.10.0:
[CHANGE/BUGFIX] API: Encode alert values as string to correctly represent Inf/NaN.
[FEATURE] Template expansion: Make external labels available as $externalLabels in alert and console template expansion.
[FEATURE] TSDB: Add prometheus_tsdb_wal_segment_current metric for the WAL segment index that TSDB is currently writing to.
[FEATURE] Scrape: Add scrape_series_added per-scrape metric.
[ENHANCEMENT] Discovery/kubernetes: Add labels __meta_kubernetes_endpoint_node_name and __meta_kubernetes_endpoint_hostname.
[ENHANCEMENT] Discovery/azure: Add label __meta_azure_machine_public_ip.
[ENHANCEMENT] TSDB: Simplify mergedPostings.Seek, resulting in better performance if there are many posting lists.
[ENHANCEMENT] Log filesystem type on startup.
[ENHANCEMENT] Cmd/promtool: Use POST requests for Query and QueryRange.
[ENHANCEMENT] Web: Sort alerts by group name.
[ENHANCEMENT] Console templates: Add convenience variables $rawParams, $params, $path.
[BUGFIX] TSDB: Don't panic when running out of disk space and recover nicely from the condition.
[BUGFIX] TSDB: Correctly handle empty labels.
[BUGFIX] TSDB: Don't crash on an unknown tombstone reference.
[BUGFIX] Storage/remote: Remove queue-manager specific metrics if queue no longer exists.
[BUGFIX] PromQL: Correctly display {__name__="a"}.
[BUGFIX] Discovery/kubernetes: Use service rather than ingress as the name for the service workqueue.
[BUGFIX] Discovery/azure: Don't panic on a VM with a public IP.
[BUGFIX] Discovery/triton: Always read HTTP body to completion.
[BUGFIX] Web: Fixed Content-Type for js and css instead of using /etc/mime.types.
2.9.2:
[BUGFIX] Make sure subquery range is taken into account for selection
[BUGFIX] Exhaust every request body before closing it
[BUGFIX] Cmd/promtool: return errors from rule evaluations
[BUGFIX] Remote Storage: string interner should not panic in release
[BUGFIX] Fix memory allocation regression in mergedPostings.
2.9.1:
[BUGFIX] Discovery/kubernetes: fix missing label sanitization
[BUGFIX] Remote_write: Prevent reshard concurrent with calling stop
2.9.0:
This releases uses Go 1.12, which includes a change in how memory is released
to Linux. This will cause RSS to be reported as higher, however this is harmless
and the memory is available to the kernel when it needs it.
[CHANGE/ENHANCEMENT] Update Consul to support catalog.ServiceMultipleTags.
[FEATURE] Add honor_timestamps scrape option.
[ENHANCEMENT] Discovery/kubernetes: add present labels for labels/annotations.
[ENHANCEMENT] OpenStack SD: Add ProjectID and UserID meta labels.
[ENHANCEMENT] Add GODEBUG and retention to the runtime page.
[ENHANCEMENT] Add support for POSTing to /series endpoint.
[ENHANCEMENT] Support PUT methods for Lifecycle and Admin APIs.
[ENHANCEMENT] Scrape: Add global jitter for HA server.
[ENHANCEMENT] Check for cancellation on every step of a range evaluation.
[ENHANCEMENT] String interning for labels & values in the remote_write path.
[ENHANCEMENT] Don't lose the scrape cache on a failed scrape.
[ENHANCEMENT] Reload cert files from disk automatically.
[ENHANCEMENT] Use fixed length millisecond timestamp format for logs.
[ENHANCEMENT] Performance improvements for postings.
[BUGFIX] Remote Write: fix checkpoint reading.
[BUGFIX] Check if label value is valid when unmarshaling external labels from YAML.
[BUGFIX] Promparse: sort all labels when parsing.
[BUGFIX] Reload rules: copy state on both name and labels.
[BUGFIX] Exponentation operator to drop metric name in result of operation.
[BUGFIX] Config: resolve more file paths.
[BUGFIX] Promtool: resolve relative paths in alert test files.
[BUGFIX] Set TLSHandshakeTimeout in HTTP transport.
[BUGFIX] Use fsync to be more resilient to machine crashes.
[BUGFIX] Keep series that are still in WAL in checkpoints.
[BUGFIX] Fix output sample values for scalar-to-vector comparison operations.
The Prometheus monitoring system and time series database.
Prometheus, a Cloud Native Computing Foundation project, is a systems and
service monitoring system. It collects metrics from configured targets at
given intervals, evaluates rule expressions, displays the results, and
can trigger alerts if some condition is observed to be true.
