librewolf
/
bunkerized-nginx
mirror of https://github.com/bunkerity/bunkerized-nginx
4
0
Fork 1
Code Issues Projects Releases Wiki Activity

use PCRE regex instead of LUA pattern and edit cors doc

This commit is contained in:
florian 2023-05-16 23:21:01 +02:00
parent 4378f18cc8
commit a9be973d5f
No known key found for this signature in database
GPG Key ID: 3D80806F12602A7C
20 changed files with 161 additions and 140 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
TODO
View File

@ -3,5 +3,4 @@
- Plugins
- sessions helpers in utils
- sessions security : check IP address, check UA, ...
- CORS : edit security tuning doc + edit example
- fix db warnings (Got an error reading communication packets)

147
docs/security-tuning.md
View File

@ -100,15 +100,24 @@ STREAM support :x:
[Cross-Origin Resource Sharing](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) lets you manage how your service can be contacted from different origins. Please note that you will have to allow the `OPTIONS` HTTP method using the `ALLOWED_METHODS` if you want to enable it (more info [here](#allowed-methods)). Here is the list of settings related to CORS :
| Setting | Default | Context |Multiple| Description |
|------------------------|------------------------------------------------------------------------------------|---------|--------|--------------------------------------------------|
|`USE_CORS` |`no` |multisite|no |Use CORS |
|`CORS_ALLOW_ORIGIN` |`*` |multisite|no |Value of the Access-Control-Allow-Origin header. |
|`CORS_EXPOSE_HEADERS` |`Content-Length,Content-Range` |multisite|no |Value of the Access-Control-Expose-Headers header.|
|`CORS_MAX_AGE` |`86400` |multisite|no |Value of the Access-Control-Max-Age header. |
|`CORS_ALLOW_CREDENTIALS`|`no` |multisite|no |Send the Access-Control-Allow-Credentials header. |
|`CORS_ALLOW_METHODS` |`GET, POST, OPTIONS` |multisite|no |Value of the Access-Control-Allow-Methods header. |
|`CORS_ALLOW_HEADERS` |`DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range`|multisite|no |Value of the Access-Control-Allow-Headers header. |
| Setting | Default | Context |Multiple| Description |
|------------------------|------------------------------------------------------------------------------------|---------|--------|-------------------------------------------------------------------|
|`USE_CORS` |`no` |multisite|no |Use CORS |
|`CORS_ALLOW_ORIGIN` |`*` |multisite|no |Allowed origins to make CORS requests : PCRE regex or *. |
|`CORS_EXPOSE_HEADERS` |`Content-Length,Content-Range` |multisite|no |Value of the Access-Control-Expose-Headers header. |
|`CORS_MAX_AGE` |`86400` |multisite|no |Value of the Access-Control-Max-Age header. |
|`CORS_ALLOW_CREDENTIALS`|`no` |multisite|no |Send the Access-Control-Allow-Credentials header. |
|`CORS_ALLOW_METHODS` |`GET, POST, OPTIONS` |multisite|no |Value of the Access-Control-Allow-Methods header. |
|`CORS_ALLOW_HEADERS` |`DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range`|multisite|no |Value of the Access-Control-Allow-Headers header. |
|`CORS_DENY_REQUEST` |`yes` |multisite|no |Deny request and don't send it to backend if Origin is not allowed.|
Here is some examples of possible values for `CORS_ALLOW_ORIGIN` setting :
- `*` will allow all origin
- `^https://www\.example\.com$` will allow `https://www.example.com`
- `^https://.+\.example.com$` will allow any origins when domain ends with `.example.com`
- `^https://(www\.example1\.com|www\.example2\.com)$` will allow both `https://www.example1.com` and `https://www.example2.com`
- `^https?://www\.example\.com$` will allow both `https://www.example.com` and `http://www.example.com`
## HTTPS / SSL/TLS
@ -265,30 +274,30 @@ STREAM support :warning:
You can use the following settings to set up blacklisting :
| Setting | Default | Description |
| :-------------------------: | :----------------------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------- |
|`USE_BLACKLIST` |`yes` |Activate blacklist feature. |
|`BLACKLIST_IP` | |List of IP/network, separated with spaces, to block. |
|`BLACKLIST_IP_URLS` |`https://www.dan.me.uk/torlist/?exit` |List of URLs, separated with spaces, containing bad IP/network to block. |
|`BLACKLIST_RDNS_GLOBAL` |`yes` |Only perform RDNS blacklist checks on global IP addresses. |
|`BLACKLIST_RDNS` |`.shodan.io .censys.io` |List of reverse DNS suffixes, separated with spaces, to block. |
|`BLACKLIST_RDNS_URLS` | |List of URLs, separated with spaces, containing reverse DNS suffixes to block. |
|`BLACKLIST_ASN` | |List of ASN numbers, separated with spaces, to block. |
|`BLACKLIST_ASN_URLS` | |List of URLs, separated with spaces, containing ASN to block. |
|`BLACKLIST_USER_AGENT` | |List of User-Agent, separated with spaces, to block. |
|`BLACKLIST_USER_AGENT_URLS` |`https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list`|List of URLs, separated with spaces, containing bad User-Agent to block. |
|`BLACKLIST_URI` | |List of URI, separated with spaces, to block. |
|`BLACKLIST_URI_URLS` | |List of URLs, separated with spaces, containing bad URI to block. |
|`BLACKLIST_IGNORE_IP` | |List of IP/network, separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_IP_URLS` | |List of URLs, separated with spaces, containing IP/network to ignore in the blacklist. |
|`BLACKLIST_IGNORE_RDNS` | |List of reverse DNS suffixes, separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_RDNS_URLS` | |List of URLs, separated with spaces, containing reverse DNS suffixes to ignore in the blacklist.|
|`BLACKLIST_IGNORE_ASN` | |List of ASN numbers, separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_ASN_URLS` | |List of URLs, separated with spaces, containing ASN to ignore in the blacklist. |
|`BLACKLIST_IGNORE_USER_AGENT` | |List of User-Agent, separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_USER_AGENT_URLS`| |List of URLs, separated with spaces, containing User-Agent to ignore in the blacklist. |
|`BLACKLIST_IGNORE_URI` | |List of URI, separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_URI_URLS` | |List of URLs, separated with spaces, containing URI to ignore in the blacklist. |
| Setting | Default | Context |Multiple| Description |
|----------------------------------|------------------------------------------------------------------------------------------------------------------------------|---------|--------|------------------------------------------------------------------------------------------------|
|`USE_BLACKLIST` |`yes` |multisite|no |Activate blacklist feature. |
|`BLACKLIST_IP` | |multisite|no |List of IP/network, separated with spaces, to block. |
|`BLACKLIST_IP_URLS` |`https://www.dan.me.uk/torlist/?exit` |global |no |List of URLs, separated with spaces, containing bad IP/network to block. |
|`BLACKLIST_RDNS_GLOBAL` |`yes` |multisite|no |Only perform RDNS blacklist checks on global IP addresses. |
|`BLACKLIST_RDNS` |`.shodan.io .censys.io` |multisite|no |List of reverse DNS suffixes, separated with spaces, to block. |
|`BLACKLIST_RDNS_URLS` | |global |no |List of URLs, separated with spaces, containing reverse DNS suffixes to block. |
|`BLACKLIST_ASN` | |multisite|no |List of ASN numbers, separated with spaces, to block. |
|`BLACKLIST_ASN_URLS` | |global |no |List of URLs, separated with spaces, containing ASN to block. |
|`BLACKLIST_USER_AGENT` | |multisite|no |List of User-Agent (PCRE regex), separated with spaces, to block. |
|`BLACKLIST_USER_AGENT_URLS` |`https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list`|global |no |List of URLs, separated with spaces, containing bad User-Agent to block. |
|`BLACKLIST_URI` | |multisite|no |List of URI (PCRE regex), separated with spaces, to block. |
|`BLACKLIST_URI_URLS` | |global |no |List of URLs, separated with spaces, containing bad URI to block. |
|`BLACKLIST_IGNORE_IP` | |multisite|no |List of IP/network, separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_IP_URLS` | |global |no |List of URLs, separated with spaces, containing IP/network to ignore in the blacklist. |
|`BLACKLIST_IGNORE_RDNS` | |multisite|no |List of reverse DNS suffixes, separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_RDNS_URLS` | |global |no |List of URLs, separated with spaces, containing reverse DNS suffixes to ignore in the blacklist.|
|`BLACKLIST_IGNORE_ASN` | |multisite|no |List of ASN numbers, separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_ASN_URLS` | |global |no |List of URLs, separated with spaces, containing ASN to ignore in the blacklist. |
|`BLACKLIST_IGNORE_USER_AGENT` | |multisite|no |List of User-Agent (PCRE regex), separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_USER_AGENT_URLS`| |global |no |List of URLs, separated with spaces, containing User-Agent to ignore in the blacklist. |
|`BLACKLIST_IGNORE_URI` | |multisite|no |List of URI (PCRE regex), separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_URI_URLS` | |global |no |List of URLs, separated with spaces, containing URI to ignore in the blacklist. |
When using stream mode, only IP, RDNS and ASN checks will be done.
@ -298,19 +307,20 @@ STREAM support :warning:
You can use the following settings to set up greylisting :
| Setting | Default | Description |
| :-------------------------: | :----------------------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------- |
| `USE_GREYLIST` | `no` | When set to `yes`, will enable greylisting based on various criteria. |
| `GREYLIST_IP` | | List of IPs and networks to greylist. |
| `GREYLIST_IP_URLS` | | List of URL containing IP and network to greylist. |
| `GREYLIST_RDNS` | | List of reverse DNS to greylist. |
| `GREYLIST_RDNS_URLS` | | List of URLs containing reverse DNS to greylist. |
| `GREYLIST_ASN` | | List of ASN to greylist. |
| `GREYLIST_ASN_URLS` | | List of URLs containing ASN to greylist. |
| `GREYLIST_USER_AGENT` | | List of User-Agents to greylist. |
| `GREYLIST_USER_AGENT_URLS` | | List of URLs containing User-Agent(s) to greylist. |
| `GREYLIST_URI` | | List of requests URI to greylist. |
| `GREYLIST_URI_URLS` | | List of URLs containing request URI to greylist. |
| Setting |Default| Context |Multiple| Description |
|--------------------------|-------|---------|--------|----------------------------------------------------------------------------------------------|
|`USE_GREYLIST` |`no` |multisite|no |Activate greylist feature. |
|`GREYLIST_IP` | |multisite|no |List of IP/network, separated with spaces, to put into the greylist. |
|`GREYLIST_IP_URLS` | |global |no |List of URLs, separated with spaces, containing good IP/network to put into the greylist. |
|`GREYLIST_RDNS_GLOBAL` |`yes` |multisite|no |Only perform RDNS greylist checks on global IP addresses. |
|`GREYLIST_RDNS` | |multisite|no |List of reverse DNS suffixes, separated with spaces, to put into the greylist. |
|`GREYLIST_RDNS_URLS` | |global |no |List of URLs, separated with spaces, containing reverse DNS suffixes to put into the greylist.|
|`GREYLIST_ASN` | |multisite|no |List of ASN numbers, separated with spaces, to put into the greylist. |
|`GREYLIST_ASN_URLS` | |global |no |List of URLs, separated with spaces, containing ASN to put into the greylist. |
|`GREYLIST_USER_AGENT` | |multisite|no |List of User-Agent (PCRE regex), separated with spaces, to put into the greylist. |
|`GREYLIST_USER_AGENT_URLS`| |global |no |List of URLs, separated with spaces, containing good User-Agent to put into the greylist. |
|`GREYLIST_URI` | |multisite|no |List of URI (PCRE regex), separated with spaces, to put into the greylist. |
|`GREYLIST_URI_URLS` | |global |no |List of URLs, separated with spaces, containing bad URI to put into the greylist. |
When using stream mode, only IP, RDNS and ASN checks will be done.
@ -320,19 +330,20 @@ STREAM support :warning:
You can use the following settings to set up whitelisting :
| Setting | Default | Description |
| :-------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------------- |
| `USE_WHITELIST` | `yes` | When set to `yes`, will enable whitelisting based on various criteria. |
| `WHITELIST_IP` | `20.191.45.212 40.88.21.235 40.76.173.151 40.76.163.7 20.185.79.47 52.142.26.175 20.185.79.15 52.142.24.149 40.76.162.208 40.76.163.23 40.76.162.191 40.76.162.247 54.208.102.37 107.21.1.8` | List of IP and network to whitelist. The default list contains IP from DuckDuckGo crawler. |
| `WHITELIST_IP_URLS` | `` | List of URLs containing IP and network to whitelist. |
| `WHITELIST_RDNS` | `.google.com .googlebot.com .yandex.ru .yandex.net .yandex.com .search.msn.com .baidu.com .baidu.jp .crawl.yahoo.net .fwd.linkedin.com .twitter.com .twttr.com .discord.com` | List of reverse DNS to whitelist. Default list contains various reverse DNS of search engines and social media crawlers. |
| `WHITELIST_RDNS_URLS` | | List of URLs containing reverse DNS to whitelist. |
| `WHITELIST_ASN` | `32934` | List of ASN to whitelist. The default list contains the ASN of Facebook. |
| `WHITELIST_ASN_URLS` | | List of URL containing ASN to whitelist. |
| `WHITELIST_USER_AGENT` | | List of User-Agent to whitelist. |
| `WHITELIST_USER_AGENT_URLS` | | List of URLs containing User-Agent to whitelist. |
| `WHITELIST_URI` | | List of requests URI to whitelist. |
| `WHITELIST_URI_URLS` | | List of URLs containing request(s) URI to whitelist. |
| Setting | Default | Context |Multiple| Description |
|---------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|--------|----------------------------------------------------------------------------------|
|`USE_WHITELIST` |`yes` |multisite|no |Activate whitelist feature. |
|`WHITELIST_IP` |`20.191.45.212 40.88.21.235 40.76.173.151 40.76.163.7 20.185.79.47 52.142.26.175 20.185.79.15 52.142.24.149 40.76.162.208 40.76.163.23 40.76.162.191 40.76.162.247 54.208.102.37 107.21.1.8`|multisite|no |List of IP/network, separated with spaces, to put into the whitelist. |
|`WHITELIST_IP_URLS` | |global |no |List of URLs, separated with spaces, containing good IP/network to whitelist. |
|`WHITELIST_RDNS_GLOBAL` |`yes` |multisite|no |Only perform RDNS whitelist checks on global IP addresses. |
|`WHITELIST_RDNS` |`.google.com .googlebot.com .yandex.ru .yandex.net .yandex.com .search.msn.com .baidu.com .baidu.jp .crawl.yahoo.net .fwd.linkedin.com .twitter.com .twttr.com .discord.com` |multisite|no |List of reverse DNS suffixes, separated with spaces, to whitelist. |
|`WHITELIST_RDNS_URLS` | |global |no |List of URLs, separated with spaces, containing reverse DNS suffixes to whitelist.|
|`WHITELIST_ASN` |`32934` |multisite|no |List of ASN numbers, separated with spaces, to whitelist. |
|`WHITELIST_ASN_URLS` | |global |no |List of URLs, separated with spaces, containing ASN to whitelist. |
|`WHITELIST_USER_AGENT` | |multisite|no |List of User-Agent (PCRE regex), separated with spaces, to whitelist. |
|`WHITELIST_USER_AGENT_URLS`| |global |no |List of URLs, separated with spaces, containing good User-Agent to whitelist. |
|`WHITELIST_URI` | |multisite|no |List of URI (PCRE regex), separated with spaces, to whitelist. |
|`WHITELIST_URI_URLS` | |global |no |List of URLs, separated with spaces, containing bad URI to whitelist. |
When using stream mode, only IP, RDNS and ASN checks will be done.
@ -410,15 +421,19 @@ STREAM support :x:
The following settings are related to the Limiting requests feature :
| Setting | Default | Description |
| :--------------: | :-----: | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `USE_LIMIT_REQ` | `yes` | When set to `yes`, will limit the number of requests for a given IP on each URL within a period of time. |
| `LIMIT_REQ_URL` | `/` | The URL that will be limited. The special URL `/` will define a default limit for all URLs. |
| `LIMIT_REQ_RATE` | `2r/s` | The limit to apply to the corresponding URL. Syntax is `Xr/Y` where **X** is the number of request(s) and **Y** the period of time (s for second, m for minute, h for hour and d for day). |
| Setting |Default| Context |Multiple| Description |
|-----------------------|-------|---------|--------|---------------------------------------------------------------------------------------------|
|`USE_LIMIT_REQ` |`yes` |multisite|no |Activate limit requests feature. |
|`LIMIT_REQ_URL` |`/` |multisite|yes |URL (PCRE regex) where the limit request will be applied or special value / for all requests.|
|`LIMIT_REQ_RATE` |`2r/s` |multisite|yes |Rate to apply to the URL (s for second, m for minute, h for hour and d for day). |
|`USE_LIMIT_CONN` |`yes` |multisite|no |Activate limit connections feature. |
|`LIMIT_CONN_MAX_HTTP1` |`10` |multisite|no |Maximum number of connections per IP when using HTTP/1.X protocol. |
|`LIMIT_CONN_MAX_HTTP2` |`100` |multisite|no |Maximum number of streams per IP when using HTTP/2 protocol. |
|`LIMIT_CONN_MAX_STREAM`|`10` |multisite|no |Maximum number of connections per IP when using stream. |
Please note that you can add different rates for different URLs by adding a number as a suffix to the settings for example : `LIMIT_REQ_URL_1=/url1`, `LIMIT_REQ_RATE_1=5r/d`, `LIMIT_REQ_URL_2=/url2`, `LIMIT_REQ_RATE_2=1r/m`, ...
Please note that you can add different rates for different URLs by adding a number as a suffix to the settings for example : `LIMIT_REQ_URL_1=^/url1$`, `LIMIT_REQ_RATE_1=5r/d`, `LIMIT_REQ_URL_2=^/url2/subdir/.*$`, `LIMIT_REQ_RATE_2=1r/m`, ...
Another important thing to note is that `LIMIT_REQ_URL` accepts LUA patterns.
Another important thing to note is that `LIMIT_REQ_URL` values are PCRE regex.
## Country

73
docs/settings.md
View File

@ -58,15 +58,17 @@ STREAM support :x:
Bot detection by using a challenge.
| Setting | Default | Context |Multiple| Description |
|---------------------------|------------|---------|--------|---------------------------------------------------------------------|
|`USE_ANTIBOT` |`no` |multisite|no |Activate antibot feature. |
|`ANTIBOT_URI` |`/challenge`|multisite|no |Unused URI that clients will be redirected to to solve the challenge.|
|`ANTIBOT_RECAPTCHA_SCORE` |`0.7` |multisite|no |Minimum score required for reCAPTCHA challenge. |
|`ANTIBOT_RECAPTCHA_SITEKEY`| |multisite|no |Sitekey for reCAPTCHA challenge. |
|`ANTIBOT_RECAPTCHA_SECRET` | |multisite|no |Secret for reCAPTCHA challenge. |
|`ANTIBOT_HCAPTCHA_SITEKEY` | |multisite|no |Sitekey for hCaptcha challenge. |
|`ANTIBOT_HCAPTCHA_SECRET` | |multisite|no |Secret for hCaptcha challenge. |
| Setting | Default | Context |Multiple| Description |
|---------------------------|------------|---------|--------|------------------------------------------------------------------------------------------------------------------------------|
|`USE_ANTIBOT` |`no` |multisite|no |Activate antibot feature. |
|`ANTIBOT_URI` |`/challenge`|multisite|no |Unused URI that clients will be redirected to to solve the challenge. |
|`ANTIBOT_RECAPTCHA_SCORE` |`0.7` |multisite|no |Minimum score required for reCAPTCHA challenge. |
|`ANTIBOT_RECAPTCHA_SITEKEY`| |multisite|no |Sitekey for reCAPTCHA challenge. |
|`ANTIBOT_RECAPTCHA_SECRET` | |multisite|no |Secret for reCAPTCHA challenge. |
|`ANTIBOT_HCAPTCHA_SITEKEY` | |multisite|no |Sitekey for hCaptcha challenge. |
|`ANTIBOT_HCAPTCHA_SECRET` | |multisite|no |Secret for hCaptcha challenge. |
|`ANTIBOT_TIME_RESOLVE` |`60` |multisite|no |Maximum time (in seconds) clients have to resolve the challenge. Once this time has passed, a new challenge will be generated.|
|`ANTIBOT_TIME_VALID` |`86400` |multisite|no |Maximum validity time of solved challenges. Once this time has passed, clients will need to resolve a new one. |
### Auth basic
@ -112,9 +114,9 @@ Deny access based on internal and external IP/network/rDNS/ASN blacklists.
|`BLACKLIST_RDNS_URLS` | |global |no |List of URLs, separated with spaces, containing reverse DNS suffixes to block. |
|`BLACKLIST_ASN` | |multisite|no |List of ASN numbers, separated with spaces, to block. |
|`BLACKLIST_ASN_URLS` | |global |no |List of URLs, separated with spaces, containing ASN to block. |
|`BLACKLIST_USER_AGENT` | |multisite|no |List of User-Agent, separated with spaces, to block. |
|`BLACKLIST_USER_AGENT` | |multisite|no |List of User-Agent (PCRE regex), separated with spaces, to block. |
|`BLACKLIST_USER_AGENT_URLS` |`https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list`|global |no |List of URLs, separated with spaces, containing bad User-Agent to block. |
|`BLACKLIST_URI` | |multisite|no |List of URI, separated with spaces, to block. |
|`BLACKLIST_URI` | |multisite|no |List of URI (PCRE regex), separated with spaces, to block. |
|`BLACKLIST_URI_URLS` | |global |no |List of URLs, separated with spaces, containing bad URI to block. |
|`BLACKLIST_IGNORE_IP` | |multisite|no |List of IP/network, separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_IP_URLS` | |global |no |List of URLs, separated with spaces, containing IP/network to ignore in the blacklist. |
@ -122,9 +124,9 @@ Deny access based on internal and external IP/network/rDNS/ASN blacklists.
|`BLACKLIST_IGNORE_RDNS_URLS` | |global |no |List of URLs, separated with spaces, containing reverse DNS suffixes to ignore in the blacklist.|
|`BLACKLIST_IGNORE_ASN` | |multisite|no |List of ASN numbers, separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_ASN_URLS` | |global |no |List of URLs, separated with spaces, containing ASN to ignore in the blacklist. |
|`BLACKLIST_IGNORE_USER_AGENT` | |multisite|no |List of User-Agent, separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_USER_AGENT` | |multisite|no |List of User-Agent (PCRE regex), separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_USER_AGENT_URLS`| |global |no |List of URLs, separated with spaces, containing User-Agent to ignore in the blacklist. |
|`BLACKLIST_IGNORE_URI` | |multisite|no |List of URI, separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_URI` | |multisite|no |List of URI (PCRE regex), separated with spaces, to ignore in the blacklist. |
|`BLACKLIST_IGNORE_URI_URLS` | |global |no |List of URLs, separated with spaces, containing URI to ignore in the blacklist. |
### Brotli
@ -157,15 +159,16 @@ STREAM support :x:
Cross-Origin Resource Sharing.
| Setting | Default | Context |Multiple| Description |
|------------------------|------------------------------------------------------------------------------------|---------|--------|--------------------------------------------------|
|`USE_CORS` |`no` |multisite|no |Use CORS |
|`CORS_ALLOW_ORIGIN` |`*` |multisite|no |Value of the Access-Control-Allow-Origin header. |
|`CORS_EXPOSE_HEADERS` |`Content-Length,Content-Range` |multisite|no |Value of the Access-Control-Expose-Headers header.|
|`CORS_MAX_AGE` |`86400` |multisite|no |Value of the Access-Control-Max-Age header. |
|`CORS_ALLOW_CREDENTIALS`|`no` |multisite|no |Send the Access-Control-Allow-Credentials header. |
|`CORS_ALLOW_METHODS` |`GET, POST, OPTIONS` |multisite|no |Value of the Access-Control-Allow-Methods header. |
|`CORS_ALLOW_HEADERS` |`DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range`|multisite|no |Value of the Access-Control-Allow-Headers header. |
| Setting | Default | Context |Multiple| Description |
|------------------------|------------------------------------------------------------------------------------|---------|--------|-------------------------------------------------------------------|
|`USE_CORS` |`no` |multisite|no |Use CORS |
|`CORS_ALLOW_ORIGIN` |`*` |multisite|no |Allowed origins to make CORS requests : PCRE regex or *. |
|`CORS_EXPOSE_HEADERS` |`Content-Length,Content-Range` |multisite|no |Value of the Access-Control-Expose-Headers header. |
|`CORS_MAX_AGE` |`86400` |multisite|no |Value of the Access-Control-Max-Age header. |
|`CORS_ALLOW_CREDENTIALS`|`no` |multisite|no |Send the Access-Control-Allow-Credentials header. |
|`CORS_ALLOW_METHODS` |`GET, POST, OPTIONS` |multisite|no |Value of the Access-Control-Allow-Methods header. |
|`CORS_ALLOW_HEADERS` |`DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range`|multisite|no |Value of the Access-Control-Allow-Headers header. |
|`CORS_DENY_REQUEST` |`yes` |multisite|no |Deny request and don't send it to backend if Origin is not allowed.|
### Client cache
@ -251,9 +254,9 @@ Allow access while keeping security features based on internal and external IP/n
|`GREYLIST_RDNS_URLS` | |global |no |List of URLs, separated with spaces, containing reverse DNS suffixes to put into the greylist.|
|`GREYLIST_ASN` | |multisite|no |List of ASN numbers, separated with spaces, to put into the greylist. |
|`GREYLIST_ASN_URLS` | |global |no |List of URLs, separated with spaces, containing ASN to put into the greylist. |
|`GREYLIST_USER_AGENT` | |multisite|no |List of User-Agent, separated with spaces, to put into the greylist. |
|`GREYLIST_USER_AGENT` | |multisite|no |List of User-Agent (PCRE regex), separated with spaces, to put into the greylist. |
|`GREYLIST_USER_AGENT_URLS`| |global |no |List of URLs, separated with spaces, containing good User-Agent to put into the greylist. |
|`GREYLIST_URI` | |multisite|no |List of URI, separated with spaces, to put into the greylist. |
|`GREYLIST_URI` | |multisite|no |List of URI (PCRE regex), separated with spaces, to put into the greylist. |
|`GREYLIST_URI_URLS` | |global |no |List of URLs, separated with spaces, containing bad URI to put into the greylist. |
### Gzip
@ -318,15 +321,15 @@ STREAM support :warning:
Limit maximum number of requests and connections.
| Setting |Default| Context |Multiple| Description |
|-----------------------|-------|---------|--------|--------------------------------------------------------------------------------|
|`USE_LIMIT_REQ` |`yes` |multisite|no |Activate limit requests feature. |
|`LIMIT_REQ_URL` |`/` |multisite|yes |URL where the limit request will be applied. |
|`LIMIT_REQ_RATE` |`2r/s` |multisite|yes |Rate to apply to the URL (s for second, m for minute, h for hour and d for day).|
|`USE_LIMIT_CONN` |`yes` |multisite|no |Activate limit connections feature. |
|`LIMIT_CONN_MAX_HTTP1` |`10` |multisite|no |Maximum number of connections per IP when using HTTP/1.X protocol. |
|`LIMIT_CONN_MAX_HTTP2` |`100` |multisite|no |Maximum number of streams per IP when using HTTP/2 protocol. |
|`LIMIT_CONN_MAX_STREAM`|`10` |multisite|no |Maximum number of connections per IP when using stream. |
| Setting |Default| Context |Multiple| Description |
|-----------------------|-------|---------|--------|---------------------------------------------------------------------------------------------|
|`USE_LIMIT_REQ` |`yes` |multisite|no |Activate limit requests feature. |
|`LIMIT_REQ_URL` |`/` |multisite|yes |URL (PCRE regex) where the limit request will be applied or special value / for all requests.|
|`LIMIT_REQ_RATE` |`2r/s` |multisite|yes |Rate to apply to the URL (s for second, m for minute, h for hour and d for day). |
|`USE_LIMIT_CONN` |`yes` |multisite|no |Activate limit connections feature. |
|`LIMIT_CONN_MAX_HTTP1` |`10` |multisite|no |Maximum number of connections per IP when using HTTP/1.X protocol. |
|`LIMIT_CONN_MAX_HTTP2` |`100` |multisite|no |Maximum number of streams per IP when using HTTP/2 protocol. |
|`LIMIT_CONN_MAX_STREAM`|`10` |multisite|no |Maximum number of connections per IP when using stream. |
### Miscellaneous
@ -522,8 +525,8 @@ Allow access based on internal and external IP/network/rDNS/ASN whitelists.
|`WHITELIST_RDNS_URLS` | |global |no |List of URLs, separated with spaces, containing reverse DNS suffixes to whitelist.|
|`WHITELIST_ASN` |`32934` |multisite|no |List of ASN numbers, separated with spaces, to whitelist. |
|`WHITELIST_ASN_URLS` | |global |no |List of URLs, separated with spaces, containing ASN to whitelist. |
|`WHITELIST_USER_AGENT` | |multisite|no |List of User-Agent, separated with spaces, to whitelist. |
|`WHITELIST_USER_AGENT` | |multisite|no |List of User-Agent (PCRE regex), separated with spaces, to whitelist. |
|`WHITELIST_USER_AGENT_URLS`| |global |no |List of URLs, separated with spaces, containing good User-Agent to whitelist. |
|`WHITELIST_URI` | |multisite|no |List of URI, separated with spaces, to whitelist. |
|`WHITELIST_URI` | |multisite|no |List of URI (PCRE regex), separated with spaces, to whitelist. |
|`WHITELIST_URI_URLS` | |global |no |List of URLs, separated with spaces, containing bad URI to whitelist. |

2
examples/cors/autoconf.yml
View File

@ -16,7 +16,7 @@ services:
labels:
- bunkerweb.SERVER_NAME=app1.example.com
- bunkerweb.USE_CORS=yes
- bunkerweb.CORS_ALLOW_ORIGIN=https://app2.example.com
- bunkerweb.CORS_ALLOW_ORIGIN=^https://app2\.example\.com$$
- bunkerweb.REMOTE_PHP=myapp1
- bunkerweb.REMOTE_PHP_PATH=/app

2
examples/cors/docker-compose.yml
View File

@ -23,7 +23,7 @@ services:
- USE_CLIENT_CACHE=yes
- USE_GZIP=yes
- app1.example.com_USE_CORS=yes
- app1.example.com_CORS_ALLOW_ORIGIN=https://app2.example.com
- app1.example.com_CORS_ALLOW_ORIGIN=^https://app2\.example\.com$$
- app1.example.com_ALLOWED_METHODS=GET|POST|HEAD|OPTIONS
- app1.example.com_REMOTE_PHP=myapp1
- app1.example.com_REMOTE_PHP_PATH=/app

2
examples/cors/variables.env
View File

@ -9,7 +9,7 @@ DISABLE_DEFAULT_SERVER=yes
USE_CLIENT_CACHE=yes
USE_GZIP=yes
app1.example.com_USE_CORS=yes
app1.example.com_CORS_ALLOW_ORIGIN=https://app2.example.com
app1.example.com_CORS_ALLOW_ORIGIN=^https://app2\.example\.com$
app1.example.com_ALLOWED_METHODS=GET|POST|HEAD|OPTIONS
app1.example.com_LOCAL_PHP=/run/php/php-fpm.sock
app1.example.com_LOCAL_PHP_PATH=/var/www/html/app1.example.com

13
src/bw/lua/bunkerweb/utils.lua
View File

@ -626,4 +626,17 @@ utils.new_cachestore = function()
return require "bunkerweb.cachestore":new(use_redis)
end
utils.regex_match = function(str, regex, options)
local all_options = "o"
if options then
all_options = all_options .. options
end
local match, err = ngx.re.match(str, regex, all_options)
if err then
logger:log(ngx.ERR, "error while matching regex " .. regex .. "with string " .. str)
return nil
end
return match
end
return utils

8
src/common/core/blacklist/blacklist.lua
View File

@ -294,7 +294,7 @@ function blacklist:is_blacklisted_uri()
-- Check if URI is in ignore list
local ignore = false
for i, ignore_uri in ipairs(self.lists["IGNORE_URI"]) do
if ngx.ctx.bw.uri:match(ignore_uri) then
if utils.regex_match(ngx.ctx.bw.uri, ignore_uri) then
ignore = true
break
end
@ -302,7 +302,7 @@ function blacklist:is_blacklisted_uri()
-- Check if URI is in blacklist
if not ignore then
for i, uri in ipairs(self.lists["URI"]) do
if ngx.ctx.bw.uri:match(uri) then
if utils.regex_match(ngx.ctx.bw.uri, uri) then
return true, "URI " .. uri
end
end
@ -315,7 +315,7 @@ function blacklist:is_blacklisted_ua()
-- Check if UA is in ignore list
local ignore = false
for i, ignore_ua in ipairs(self.lists["IGNORE_USER_AGENT"]) do
if ngx.ctx.bw.http_user_agent:match(ignore_ua) then
if utils.regex_match(ngx.ctx.bw.http_user_agent, ignore_ua) then
ignore = true
break
end
@ -323,7 +323,7 @@ function blacklist:is_blacklisted_ua()
-- Check if UA is in blacklist
if not ignore then
for i, ua in ipairs(self.lists["USER_AGENT"]) do
if ngx.ctx.bw.http_user_agent:match(ua) then
if utils.regex_match(ngx.ctx.bw.http_user_agent, ua) then
return true, "UA " .. ua
end
end

4
src/common/core/blacklist/jobs/blacklist-download.py
View File

@ -46,9 +46,7 @@ def check_line(kind: str, line: bytes) -> Tuple[bool, bytes]:
if asn_rx.match(real_line):
return True, real_line
elif kind in ("USER_AGENT", "IGNORE_USER_AGENT"):
return True, line.replace(b"\\ ", b" ").replace(b"\\.", b"%.").replace(
b"\\\\", b"\\"
).replace(b"-", b"%-")
return True, b"(?:\\b)" + line + b"(?:\\b)"
elif kind in ("URI", "IGNORE_URI"):
if uri_rx.match(line):
return True, line

8
src/common/core/blacklist/plugin.json
View File

@ -81,7 +81,7 @@
"BLACKLIST_USER_AGENT": {
"context": "multisite",
"default": "",
"help": "List of User-Agent, separated with spaces, to block.",
"help": "List of User-Agent (PCRE regex), separated with spaces, to block.",
"id": "blacklist-user-agent",
"label": "Blacklist User-Agent",
"regex": "^.*$",
@ -99,7 +99,7 @@
"BLACKLIST_URI": {
"context": "multisite",
"default": "",
"help": "List of URI, separated with spaces, to block.",
"help": "List of URI (PCRE regex), separated with spaces, to block.",
"id": "blacklist-uri",
"label": "Blacklist URI",
"regex": "^( *(/[\\w\\].~:/?#[@!$&'()*+,;=-]*)(?!.*\\2(?!.)) *)*$",
@ -171,7 +171,7 @@
"BLACKLIST_IGNORE_USER_AGENT": {
"context": "multisite",
"default": "",
"help": "List of User-Agent, separated with spaces, to ignore in the blacklist.",
"help": "List of User-Agent (PCRE regex), separated with spaces, to ignore in the blacklist.",
"id": "blacklist-ignore-user-agent",
"label": "Blacklist ignore User-Agent",
"regex": "^.*$",
@ -189,7 +189,7 @@
"BLACKLIST_IGNORE_URI": {
"context": "multisite",
"default": "",
"help": "List of URI, separated with spaces, to ignore in the blacklist.",
"help": "List of URI (PCRE regex), separated with spaces, to ignore in the blacklist.",
"id": "blacklist-ignore-uri",
"label": "Blacklist ignore URI",
"regex": "^( *(/[\\w\\].~:/?#[@!$&'()*+,;=-]*)(?!.*\\2(?!.)) *)*$",

9
src/common/core/cors/cors.lua
View File

@ -40,7 +40,7 @@ function cors:header()
ngx.header.Vary = "Origin"
end
-- Check if Origin is allowed
if self.variables["CORS_ALLOW_ORIGIN"] ~= "*" and not ngx.ctx.bw.http_origin:match(self.variables["CORS_ALLOW_ORIGIN"]) then
if ngx.ctx.bw.http_origin and self.variables["CORS_DENY_REQUEST"] == "yes" and self.variables["CORS_ALLOW_ORIGIN"] ~= "*" and not utils.regex_match(ngx.ctx.bw.http_origin, self.variables["CORS_ALLOW_ORIGIN"]) then
self.logger:log(ngx.WARN, "origin " .. ngx.ctx.bw.http_origin .. " is not allowed")
return self:ret(true, "origin " .. ngx.ctx.bw.http_origin .. " is not allowed")
end
@ -78,11 +78,8 @@ function cors:access()
return self:ret(true, "service doesn't use CORS")
end
-- Deny as soon as possible if needed
if self.variables["CORS_DENY_REQUEST"] == "yes" and ngx.ctx.bw.http_origin then
if self.variables["CORS_ALLOW_ORIGIN"] ~= "*" and not ngx.ctx.bw.http_origin:match(self.variables["CORS_ALLOW_ORIGIN"]) then
self.logger:log(ngx.WARN, "origin " .. ngx.ctx.bw.http_origin .. " is not allowed")
return self:ret(true, "origin " .. ngx.ctx.bw.http_origin .. " is not allowed, denying access", utils.get_deny_status())
end
if ngx.ctx.bw.http_origin and self.variables["CORS_DENY_REQUEST"] == "yes" and self.variables["CORS_ALLOW_ORIGIN"] ~= "*" and not utils.regex_match(ngx.ctx.bw.http_origin, self.variables["CORS_ALLOW_ORIGIN"]) then
return self:ret(true, "origin " .. ngx.ctx.bw.http_origin .. " is not allowed, denying access", utils.get_deny_status())
end
-- Send CORS policy with a 204 (no content) status
if ngx.ctx.bw.request_method == "OPTIONS" and ngx.ctx.bw.http_origin then

2
src/common/core/cors/plugin.json
View File

@ -18,7 +18,7 @@
"CORS_ALLOW_ORIGIN": {
"context": "multisite",
"default": "*",
"help": "Allowed origins to make CORS requests (LUA pattern) or *.",
"help": "Allowed origins to make CORS requests : PCRE regex or *.",
"id": "cors-allow-origin",
"label": "Allowed origins",
"regex": "^.*$",

4
src/common/core/greylist/greylist.lua
View File

@ -232,7 +232,7 @@ end
function greylist:is_greylisted_uri()
-- Check if URI is in greylist
for i, uri in ipairs(self.lists["URI"]) do
if ngx.ctx.bw.uri:match(uri) then
if utils.regex_match(ngx.ctx.bw.uri, uri) then
return true, "URI " .. uri
end
end
@ -243,7 +243,7 @@ end
function greylist:is_greylisted_ua()
-- Check if UA is in greylist
for i, ua in ipairs(self.lists["USER_AGENT"]) do
if ngx.ctx.bw.http_user_agent:match(ua) then
if utils.regex_match(ngx.ctx.bw.http_user_agent, ua) then
return true, "UA " .. ua
end
end

4
src/common/core/greylist/jobs/greylist-download.py
View File

@ -46,9 +46,7 @@ def check_line(kind: str, line: bytes) -> Tuple[bool, bytes]:
if asn_rx.match(real_line):
return True, real_line
elif kind == "USER_AGENT":
return True, line.replace(b"\\ ", b" ").replace(b"\\.", b"%.").replace(
b"\\\\", b"\\"
).replace(b"-", b"%-")
return True, b"(?:\\b)" + line + b"(?:\\b)"
elif kind == "URI":
if uri_rx.match(line):
return True, line

4
src/common/core/greylist/plugin.json
View File

@ -81,7 +81,7 @@
"GREYLIST_USER_AGENT": {
"context": "multisite",
"default": "",
"help": "List of User-Agent, separated with spaces, to put into the greylist.",
"help": "List of User-Agent (PCRE regex), separated with spaces, to put into the greylist.",
"id": "greylist-user-agent",
"label": "Greylist User-Agent",
"regex": "^.*$",
@ -99,7 +99,7 @@
"GREYLIST_URI": {
"context": "multisite",
"default": "",
"help": "List of URI, separated with spaces, to put into the greylist.",
"help": "List of URI (PCRE regex), separated with spaces, to put into the greylist.",
"id": "greylist-uri",
"label": "Greylist URI",
"regex": "^.*$",

4
src/common/core/limit/limit.lua
View File

@ -75,7 +75,7 @@ function limit:init()
local i = 0
for srv, vars in pairs(variables) do
for var, value in pairs(vars) do
if var:match("LIMIT_REQ_URL") then
if utils.regex_match(var, "LIMIT_REQ_URL") then
local url = value
local rate = vars[var:gsub("URL", "RATE")]
if data[srv] == nil then
@ -106,7 +106,7 @@ function limit:access()
local rate = nil
local uri = nil
for k, v in pairs(self.rules) do
if k ~= "/" and ngx.ctx.bw.uri:match(k) then
if k ~= "/" and utils.regex_match(ngx.ctx.bw.uri, k) then
rate = v
uri = k
break

2
src/common/core/limit/plugin.json
View File

@ -18,7 +18,7 @@
"LIMIT_REQ_URL": {
"context": "multisite",
"default": "/",
"help": "URL where the limit request will be applied.",
"help": "URL (PCRE regex) where the limit request will be applied or special value / for all requests.",
"id": "limit-req-url",
"label": "Limit request URL",
"regex": "^[\\w\\].~:/^%?#[@!$&'()*+,;=-]+$",

4
src/common/core/whitelist/jobs/whitelist-download.py
View File

@ -46,9 +46,7 @@ def check_line(kind: str, line: bytes) -> Tuple[bool, bytes]:
if asn_rx.match(real_line):
return True, real_line
elif kind == "USER_AGENT":
return True, line.replace(b"\\ ", b" ").replace(b"\\.", b"%.").replace(
b"\\\\", b"\\"
).replace(b"-", b"%-")
return True, b"(?:\\b)" + line + b"(?:\\b)"
elif kind == "URI":
if uri_rx.match(line):
return True, line

4
src/common/core/whitelist/plugin.json
View File

@ -81,7 +81,7 @@
"WHITELIST_USER_AGENT": {
"context": "multisite",
"default": "",
"help": "List of User-Agent, separated with spaces, to whitelist.",
"help": "List of User-Agent (PCRE regex), separated with spaces, to whitelist.",
"id": "whitelist-user-agent",
"label": "Whitelist User-Agent",
"regex": "^.*$",
@ -99,7 +99,7 @@
"WHITELIST_URI": {
"context": "multisite",
"default": "",
"help": "List of URI, separated with spaces, to whitelist.",
"help": "List of URI (PCRE regex), separated with spaces, to whitelist.",
"id": "whitelist-uri",
"label": "Whitelist URI",
"regex": "^( *(/[\\w\\].~:/?#[@!$&'()*+,;=-]*)(?!.*\\2(?!.)) *)*$",

4
src/common/core/whitelist/whitelist.lua
View File

@ -288,7 +288,7 @@ end
function whitelist:is_whitelisted_uri()
-- Check if URI is in whitelist
for i, uri in ipairs(self.lists["URI"]) do
if ngx.ctx.bw.uri:match(uri) then
if utils.regex_match(ngx.ctx.bw.uri, uri) then
return true, "URI " .. uri
end
end
@ -299,7 +299,7 @@ end
function whitelist:is_whitelisted_ua()
-- Check if UA is in whitelist
for i, ua in ipairs(self.lists["USER_AGENT"]) do
if ngx.ctx.bw.http_user_agent:match(ua) then
if utils.regex_match(ngx.ctx.bw.http_user_agent, ua) then
return true, "UA " .. ua
end
end