Merge branch 'dev' of github.com:bunkerity/bunkerweb into dev

This commit is contained in:
florian 2023-06-01 21:43:01 +02:00
commit b27958a19c
No known key found for this signature in database
GPG Key ID: 3D80806F12602A7C
201 changed files with 4853 additions and 3583 deletions

View File

@ -1,5 +1,40 @@
# Changelog
## v1.5.1 -
- [BUGFIX] New version checker in logs displays "404 not found"
- [BUGFIX] New version checker in UI
- [BUGFIX] Only get the right keys from plugin.json files when importing plugins
- [BUGFIX] Remove external resources for Google fonts in UI
- [BUGFIX] Support multiple plugin uploads in one zip when using the UI
- [BUGFIX] Variable being ignored instead of saved in the database when value is empty
- [BUGFIX] ALLOWED_METHODS regex working with LOCK/UNLOCK methods
- [BUGFIX] Custom certificate bug after the refactoring
- [BUGFIX] Fix wrong variables in header phase (fix CORS feature too)
- [PERFORMANCE] Reduce CPU usage of scheduler
- [FEATURE] Add Turnstile antibot mode
- [MISC] Add LOG_LEVEL=warning for docker socket proxy in docs, examples and boilerplates
- [MISC] Temp remove VMWare provider for Vagrant integration
## v1.5.0 - 2023/05/23
- Refactoring of almost all the components of the project
- Dedicated scheduler service to manage jobs and configuration
- Store configuration in a database backend
- Improved web UI and make it working with all integrations
- Improved internal LUA code
- Improved internal cache of BW
- Add Redis support when using clustered integrations
- Add RHEL integration
- Add Vagrant integration
- Init support of generic TCP/UDP (stream)
- Init support of IPv6
- Improved CI/CD : UI tests, core tests and release automation
- Reduce Docker images size
- Fix and improved core plugins : antibot, cors, dnsbl, ...
- Use PCRE regex instead of LUA patterns
- Connectivity tests at startup/reload with logging
## v1.5.0-beta - 2023/05/02
- Refactoring of almost all the components of the project

View File

@ -247,8 +247,7 @@ You will find more information in the [Ansible section](https://docs.bunkerweb.i
We maintain ready to use Vagrant boxes hosted on Vagrant cloud for the following providers :
- vmware_desktop
- virtualbox
- virtualbox
- libvirt
You will find more information in the [Vagrant section](https://docs.bunkerweb.io/1.5.0/integrations/#vagrant) of the documentation.
@ -304,13 +303,14 @@ BunkerWeb comes with a plugin system to make it possible to easily add new featu
Here is the list of "official" plugins that we maintain (see the [bunkerweb-plugins](https://github.com/bunkerity/bunkerweb-plugins) repository for more information) :
| Name | Version | Description | Link |
| :------------: | :-----: | :------------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------: |
| **ClamAV** | 0.1 | Automatically scans uploaded files with the ClamAV antivirus engine and denies the request when a file is detected as malicious. | [bunkerweb-plugins/clamav](https://github.com/bunkerity/bunkerweb-plugins/tree/main/clamav) |
| **CrowdSec** | 0.1 | CrowdSec bouncer for BunkerWeb. | [bunkerweb-plugins/crowdsec](https://github.com/bunkerity/bunkerweb-plugins/tree/main/crowdsec) |
| **Discord** | 0.1 | Send security notifications to a Discord channel using a Webhook. | [bunkerweb-plugins/discord](https://github.com/bunkerity/bunkerweb-plugins/tree/main/discord) |
| **Slack** | 0.1 | Send security notifications to a Slack channel using a Webhook. | [bunkerweb-plugins/slack](https://github.com/bunkerity/bunkerweb-plugins/tree/main/slack) |
| **VirusTotal** | 0.1 | Automatically scans uploaded files with the VirusTotal API and denies the request when a file is detected as malicious. | [bunkerweb-plugins/virustotal](https://github.com/bunkerity/bunkerweb-plugins/tree/main/virustotal) |
| Name | Version | Description | Link |
| :------------: | :-----: | :------------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------: |
| **ClamAV** | 1.0 | Automatically scans uploaded files with the ClamAV antivirus engine and denies the request when a file is detected as malicious. | [bunkerweb-plugins/clamav](https://github.com/bunkerity/bunkerweb-plugins/tree/main/clamav) |
| **CrowdSec** | 1.0 | CrowdSec bouncer for BunkerWeb. | [bunkerweb-plugins/crowdsec](https://github.com/bunkerity/bunkerweb-plugins/tree/main/crowdsec) |
| **Discord** | 1.0 | Send security notifications to a Discord channel using a Webhook. | [bunkerweb-plugins/discord](https://github.com/bunkerity/bunkerweb-plugins/tree/main/discord) |
| **Slack** | 1.0 | Send security notifications to a Slack channel using a Webhook. | [bunkerweb-plugins/slack](https://github.com/bunkerity/bunkerweb-plugins/tree/main/slack) |
| **VirusTotal** | 1.0 | Automatically scans uploaded files with the VirusTotal API and denies the request when a file is detected as malicious. | [bunkerweb-plugins/virustotal](https://github.com/bunkerity/bunkerweb-plugins/tree/main/virustotal) |
| **Coraza** | 0.1 | Inspect requests using a the Coraza WAF (alternative of ModSecurity). | [bunkerweb-plugins/coraza](https://github.com/bunkerity/bunkerweb-plugins/tree/main/coraza) |
You will find more information in the [plugins section](https://docs.bunkerweb.io/1.5.0/plugins) of the documentation.

View File

@ -1231,7 +1231,6 @@ Configuration of BunkerWeb is done by using specific role variables :
-->
List of supported providers :
- vmware_desktop
- virtualbox
- libvirt
@ -1243,10 +1242,10 @@ Similar to other BunkerWeb integrations, the Vagrant setup uses **NGINX version
By using the provided Vagrant box based on Ubuntu 22.04 "Jammy", you benefit from a well-configured and integrated setup, allowing you to focus on developing and securing your applications with BunkerWeb without worrying about the underlying infrastructure.
Here are the steps to install BunkerWeb using Vagrant on Ubuntu with the supported virtualization providers (VirtualBox, VMware, and libvirt):
Here are the steps to install BunkerWeb using Vagrant on Ubuntu with the supported virtualization providers (VirtualBox, and libvirt):
1. Make sure you have Vagrant and one of the supported virtualization providers (VirtualBox, VMware, or libvirt) installed on your system.
1. Make sure you have Vagrant and one of the supported virtualization providers (VirtualBox or libvirt) installed on your system.
2. There are two ways to install the Vagrant box with BunkerWeb: either by using a provided Vagrantfile to configure your virtual machine or by creating a new box based on the existing BunkerWeb Vagrant box, offering you flexibility in how you set up your development environment.
=== "Vagrantfile"
@ -1259,7 +1258,6 @@ Here are the steps to install BunkerWeb using Vagrant on Ubuntu with the support
Depending on the virtualization provider you choose, you may need to install additional plugins:
* For **VMware**, install the `vagrant-vmware-desktop` plugin. For more information, see the [Vagrant documentation](https://www.vagrantup.com/docs/providers).
* For **libvirt**, install the `vagrant-libvirt plugin`. For more information, see the [Vagrant documentation](https://www.vagrantup.com/docs/providers).
* For **VirtualBox**, install the `vagrant-vbguest` plugin. For more information, see the [Vagrant documentation](https://www.vagrantup.com/docs/providers).
@ -1271,14 +1269,13 @@ Here are the steps to install BunkerWeb using Vagrant on Ubuntu with the support
Depending on the virtualization provider you choose, you may need to install additional plugins:
* For **VMware**, install the `vagrant-vmware-desktop` plugin. For more information, see the [Vagrant documentation](https://www.vagrantup.com/docs/providers).
* For **libvirt**, install the `vagrant-libvirt plugin`. For more information, see the [Vagrant documentation](https://www.vagrantup.com/docs/providers).
* For **VirtualBox**, install the `vagrant-vbguest` plugin. For more information, see the [Vagrant documentation](https://www.vagrantup.com/docs/providers).
After installing the necessary plugins for your chosen virtualization provider, run the following command to start the virtual machine and install BunkerWeb:
```shell
vagrant up --provider=virtualbox # or --provider=vmware_desktop or --provider=libvirt
vagrant up --provider=virtualbox # or --provider=libvirt
```
Finally, to access the virtual machine using SSH, execute the following command:
@ -1298,9 +1295,6 @@ Vagrant.configure("2") do |config|
# Uncomment the desired virtualization provider
# For VirtualBox (default)
config.vm.provider "virtualbox"
# For VMware
# config.vm.provider "vmware_desktop" # Windows
# config.vm.provider "vmware_workstation" # Linux
# For libvirt
# config.vm.provider "libvirt"
end

View File

@ -13,6 +13,7 @@ Here is the list of "official" plugins that we maintain (see the [bunkerweb-plug
| **Discord** | 1.0 | Send security notifications to a Discord channel using a Webhook. | [bunkerweb-plugins/discord](https://github.com/bunkerity/bunkerweb-plugins/tree/main/discord) |
| **Slack** | 1.0 | Send security notifications to a Slack channel using a Webhook. | [bunkerweb-plugins/slack](https://github.com/bunkerity/bunkerweb-plugins/tree/main/slack) |
| **VirusTotal** | 1.0 | Automatically scans uploaded files with the VirusTotal API and denies the request when a file is detected as malicious. | [bunkerweb-plugins/virustotal](https://github.com/bunkerity/bunkerweb-plugins/tree/main/virustotal) |
| **Coraza** | 0.1 | Inspect requests using a Core Rule Set and deny malicious ones. | [bunkerweb-plugins/coraza](https://github.com/bunkerity/bunkerweb-plugins/tree/main/coraza) |
## How to use a plugin
@ -275,30 +276,29 @@ A file named **plugin.json** and written at the root of the plugin folder must c
```json
{
"id": "myplugin",
"order": 42,
"name": "My Plugin",
"description": "Just an example plugin.",
"version": "1.0",
"id": "myplugin",
"name": "My Plugin",
"description": "Just an example plugin.",
"version": "1.0",
"stream": "partial",
"settings": {
"DUMMY_SETTING": {
"context": "multisite",
"default": "1234",
"help": "Here is the help of the setting.",
"id": "dummy-id",
"label": "Dummy setting",
"regex": "^.*$",
"type": "text"
}
},
"jobs": [
{
"name": "my-job",
"file": "my-job.py",
"every": "hour"
}
]
"settings": {
"DUMMY_SETTING": {
"context": "multisite",
"default": "1234",
"help": "Here is the help of the setting.",
"id": "dummy-id",
"label": "Dummy setting",
"regex": "^.*$",
"type": "text"
}
},
"jobs": [
{
"name": "my-job",
"file": "my-job.py",
"every": "hour"
}
]
}
```
@ -307,7 +307,6 @@ Here are the details of the fields :
| Field | Mandatory | Type | Description |
| :-----------: | :-------: | :----: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id` | yes | string | Internal ID for the plugin : must be unique among other plugins (including "core" ones) and contain only lowercase chars. |
| `order` | yes | int | When the plugin should be executed during the access phase : `1` for whitelisting, `2` for blacklisting, `3` for "standard security feature" or `999` if your settings don't use the access phase. |
| `name` | yes | string | Name of your plugin. |
| `description` | yes | string | Description of your plugin. |
| `version` | yes | string | Version of your plugin. |

View File

@ -1,5 +1,5 @@
mkdocs==1.4.3
mkdocs-material==9.1.13
mkdocs-material==9.1.15
pytablewriter==0.64.2
mike==1.1.2
jinja2<3.1.0

View File

@ -61,6 +61,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -70,6 +70,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -41,6 +41,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -67,6 +67,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -51,6 +51,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -51,6 +51,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -51,6 +51,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -51,6 +51,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -51,6 +51,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -55,6 +55,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker
@ -101,4 +102,4 @@ networks:
bw-services:
volumes:
bw-data:
bw-data:

View File

@ -70,6 +70,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -35,7 +35,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=drupaldb
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password
volumes:
db-data:

View File

@ -45,6 +45,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker
@ -66,7 +67,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=drupaldb
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password
networks:
- bw-services

View File

@ -3,4 +3,4 @@ drupalPassword: "changeme42"
drupalEmail: "contact@example.com"
mariadb:
auth:
password: "changeme1337"
password: "changeme1337"

View File

@ -33,7 +33,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=drupaldb
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password
deploy:
placement:
constraints:

View File

@ -24,4 +24,4 @@ volumes:
networks:
bw-services:
external: true
name: bw-services
name: bw-services

View File

@ -41,6 +41,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -3,4 +3,4 @@ ghostPassword: "changeme42"
ghostHost: "www.example.com"
mysql:
auth:
password: "changeme1337"
password: "changeme1337"

View File

@ -45,6 +45,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -55,6 +55,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -36,7 +36,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=joomla_db
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match JOOMLA_DB_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match JOOMLA_DB_PASSWORD)
volumes:
joomla-data:

View File

@ -45,6 +45,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker
@ -68,7 +69,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=joomla_db
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match JOOMLA_DB_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match JOOMLA_DB_PASSWORD)
networks:
- bw-services
volumes:

View File

@ -3,4 +3,4 @@ joomlaPassword: "changeme42"
joomlaEmail: "contact@example.com"
mariadb:
auth:
password: "changeme1337"
password: "changeme1337"

View File

@ -36,13 +36,12 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=joomla_db
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match JOOMLA_DB_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match JOOMLA_DB_PASSWORD)
deploy:
placement:
constraints:
- "node.role==worker"
networks:
bw-services:
external: true

View File

@ -48,6 +48,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -55,7 +55,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=magentodb
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MAGENTO_DATABASE_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MAGENTO_DATABASE_PASSWORD)
volumes:
magento-data:

View File

@ -46,6 +46,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker
@ -85,7 +86,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=magentodb
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MAGENTO_DATABASE_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MAGENTO_DATABASE_PASSWORD)
networks:
- bw-services

View File

@ -7,4 +7,4 @@ magentoLastName: "Doe"
magentoAdminUri: "admin"
mariadb:
auth:
password: "changeme1337"
password: "changeme1337"

View File

@ -50,7 +50,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=magentodb
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MAGENTO_DATABASE_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MAGENTO_DATABASE_PASSWORD)
deploy:
placement:
constraints:

View File

@ -61,6 +61,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -44,6 +44,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -13,10 +13,10 @@ services:
- moodle-files:/bitnami/moodle
- moodle-data:/bitnami/moodledata
environment:
- MOODLE_USERNAME=admin # replace with your moodle admin username
- MOODLE_PASSWORD=password # replace with your moodle admin password
- MOODLE_EMAIL=moodle@example.com # replace with your moodle admin email
- MOODLE_SITE_NAME=My Moodle # replace with your moodle site name
- MOODLE_USERNAME=admin # replace with your moodle admin username
- MOODLE_PASSWORD=password # replace with your moodle admin password
- MOODLE_EMAIL=moodle@example.com # replace with your moodle admin email
- MOODLE_SITE_NAME=My Moodle # replace with your moodle site name
- MOODLE_DATABASE_HOST=mydb
- MOODLE_DATABASE_NAME=moodle
- MOODLE_DATABASE_USER=user
@ -36,10 +36,10 @@ services:
aliases:
- mydb
environment:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=moodle
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MOODLE_DATABASE_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MOODLE_DATABASE_PASSWORD)
- MARIADB_CHARACTER_SET=utf8mb4
- MARIADB_COLLATE=utf8mb4_unicode_ci

View File

@ -42,6 +42,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker
@ -53,10 +54,10 @@ services:
- moodle-files:/bitnami/moodle
- moodle-data:/bitnami/moodledata
environment:
- MOODLE_USERNAME=admin # replace with your moodle admin username
- MOODLE_PASSWORD=password # replace with your moodle admin password
- MOODLE_EMAIL=moodle@example.com # replace with your moodle admin email
- MOODLE_SITE_NAME=My Moodle # replace with your moodle site name
- MOODLE_USERNAME=admin # replace with your moodle admin username
- MOODLE_PASSWORD=password # replace with your moodle admin password
- MOODLE_EMAIL=moodle@example.com # replace with your moodle admin email
- MOODLE_SITE_NAME=My Moodle # replace with your moodle site name
- MOODLE_DATABASE_HOST=mydb
- MOODLE_DATABASE_NAME=moodle
- MOODLE_DATABASE_USER=user
@ -69,10 +70,10 @@ services:
volumes:
- db-data:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=moodle
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MOODLE_DATABASE_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MOODLE_DATABASE_PASSWORD)
- MARIADB_CHARACTER_SET=utf8mb4
- MARIADB_COLLATE=utf8mb4_unicode_ci
networks:

View File

@ -4,4 +4,4 @@ moodlePassword: "changeme42"
moodleEmail: "admin@example.com"
mariadb:
auth:
password: "changeme1337"
password: "changeme1337"

View File

@ -36,10 +36,10 @@ services:
networks:
- bw-services
environment:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=moodle
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MOODLE_DATABASE_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MOODLE_DATABASE_PASSWORD)
- MARIADB_CHARACTER_SET=utf8mb4
- MARIADB_COLLATE=utf8mb4_unicode_ci
deploy:

View File

@ -56,7 +56,6 @@ services:
bunkerweb.CUSTOM_CONF_MODSEC_nextcloud=
SecRule REQUEST_FILENAME "@rx ^/remote.php/dav/files/" "id:2000,ctl:ruleRemoveByTag=attack-protocol,ctl:ruleRemoveByTag=attack-generic,nolog"
mydb:
image: mariadb
command: --transaction-isolation=READ-COMMITTED --binlog-format=ROW
@ -70,7 +69,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=nc
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MYSQL_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MYSQL_PASSWORD)
volumes:
nc-files:

View File

@ -71,6 +71,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker
@ -110,7 +111,6 @@ volumes:
db-data:
nc-files:
networks:
bw-universe:
ipam:

View File

@ -48,7 +48,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=nc
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MYSQL_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match MYSQL_PASSWORD)
deploy:
placement:
constraints:

View File

@ -13,7 +13,7 @@ services:
environment:
- APP_FULL_BASE_URL=https://www.example.com # replace with your URL
- DATASOURCES_DEFAULT_HOST=mydb
- DATASOURCES_DEFAULT_PASSWORD=db-user-pwd # replace with a stronger password (must match MYSQL_PASSWORD)
- DATASOURCES_DEFAULT_PASSWORD=db-user-pwd # replace with a stronger password (must match MYSQL_PASSWORD)
- DATASOURCES_DEFAULT_USERNAME=user
- DATASOURCES_DEFAULT_DATABASE=passbolt
volumes:
@ -45,10 +45,10 @@ services:
aliases:
- mydb
environment:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=passbolt
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match DATASOURCES_DEFAULT_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match DATASOURCES_DEFAULT_PASSWORD)
volumes:
gpg-data:

View File

@ -7,7 +7,7 @@ services:
- 80:8080
- 443:8443
environment:
- SERVER_NAME=www.example.com # replace with your domain
- SERVER_NAME=www.example.com # replace with your domain
- API_WHITELIST_IP=127.0.0.0/8 10.20.30.0/24
- AUTO_LETS_ENCRYPT=yes
- DISABLE_DEFAULT_SERVER=yes
@ -43,6 +43,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker
@ -58,7 +59,7 @@ services:
environment:
- APP_FULL_BASE_URL=https://www.example.com # replace with your URL
- DATASOURCES_DEFAULT_HOST=mydb
- DATASOURCES_DEFAULT_PASSWORD=db-user-pwd # replace with a stronger password (must match MYSQL_PASSWORD)
- DATASOURCES_DEFAULT_PASSWORD=db-user-pwd # replace with a stronger password (must match MYSQL_PASSWORD)
- DATASOURCES_DEFAULT_USERNAME=user
- DATASOURCES_DEFAULT_DATABASE=passbolt
volumes:
@ -81,10 +82,10 @@ services:
volumes:
- db-data:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=passbolt
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match DATASOURCES_DEFAULT_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match DATASOURCES_DEFAULT_PASSWORD)
networks:
- bw-services

View File

@ -11,7 +11,7 @@ services:
environment:
- APP_FULL_BASE_URL=https://www.example.com # replace with your URL
- DATASOURCES_DEFAULT_HOST=mydb
- DATASOURCES_DEFAULT_PASSWORD=db-user-pwd # replace with a stronger password (must match MYSQL_PASSWORD)
- DATASOURCES_DEFAULT_PASSWORD=db-user-pwd # replace with a stronger password (must match MYSQL_PASSWORD)
- DATASOURCES_DEFAULT_USERNAME=user
- DATASOURCES_DEFAULT_DATABASE=passbolt
volumes:
@ -45,10 +45,10 @@ services:
networks:
- bw-services
environment:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=passbolt
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match DATASOURCES_DEFAULT_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match DATASOURCES_DEFAULT_PASSWORD)
deploy:
placement:
constraints:

View File

@ -48,6 +48,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker
@ -65,7 +66,6 @@ services:
volumes:
bw-data:
networks:
bw-universe:
ipam:

View File

@ -51,6 +51,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -47,6 +47,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -45,6 +45,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -10,4 +10,4 @@ service:
type: ClusterIP
mariadb:
auth:
password: "changeme1337"
password: "changeme1337"

View File

@ -44,6 +44,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -51,6 +51,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -33,7 +33,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=redminedb
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match REDMINE_DB_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match REDMINE_DB_PASSWORD)
volumes:
redmine-data:

View File

@ -41,6 +41,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker
@ -65,7 +66,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=redminedb
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match REDMINE_DB_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match REDMINE_DB_PASSWORD)
networks:
- bw-universe
- bw-services

View File

@ -6,4 +6,4 @@ service:
type: ClusterIP
mariadb:
auth:
password: "changeme1337"
password: "changeme1337"

View File

@ -33,7 +33,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=redminedb
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match REDMINE_DB_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match REDMINE_DB_PASSWORD)
deploy:
placement:
constraints:

View File

@ -44,6 +44,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -49,6 +49,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -42,6 +42,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -50,18 +50,19 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker
app1:
image: istio/tcp-echo-server:1.2
command: [ "9000", "app1" ]
command: ["9000", "app1"]
networks:
- bw-services
app2:
image: istio/tcp-echo-server:1.2
command: [ "9000", "app2" ]
command: ["9000", "app2"]
networks:
- bw-services

View File

@ -50,6 +50,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -40,6 +40,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -51,6 +51,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -20,7 +20,8 @@ services:
- www.example.com_REVERSE_PROXY_URL=/changeme/ # replace with another url
- www.example.com_REVERSE_PROXY_HOST=http://bw-ui:7000
- www.example.com_REVERSE_PROXY_HEADERS=X-Script-Name /changeme # replace with another url
- www.example.com_REVERSE_PROXY_INTERCEPT_ERRORS=no
- www.example.com_INTERCEPTED_ERROR_CODES=400 404 405 413 429 500 501 502 503 504
- www.example.com_SECURITY_POLICY=object-src 'none'; frame-ancestors 'self';
labels:
- "bunkerweb.INSTANCE"
networks:
@ -60,6 +61,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -43,7 +43,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=wp
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match WORDPRESS_DB_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match WORDPRESS_DB_PASSWORD)
volumes:
wp-data:

View File

@ -42,6 +42,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker
@ -66,7 +67,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=wp
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match WORDPRESS_DB_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match WORDPRESS_DB_PASSWORD)
networks:
- bw-services

View File

@ -34,7 +34,7 @@ services:
- MYSQL_ROOT_PASSWORD=db-root-pwd # replace with a stronger password
- MYSQL_DATABASE=wp
- MYSQL_USER=user
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match WORDPRESS_DB_PASSWORD)
- MYSQL_PASSWORD=db-user-pwd # replace with a stronger password (must match WORDPRESS_DB_PASSWORD)
deploy:
placement:
constraints:

View File

@ -8,4 +8,4 @@ wordpressTablePrefix: "changeme_"
wordpressScheme: "https"
mariadb:
auth:
password: "changeme1337"
password: "changeme1337"

View File

@ -50,6 +50,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -50,6 +50,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -50,6 +50,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -50,6 +50,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -50,6 +50,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -50,6 +50,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -51,6 +51,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -51,6 +51,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -44,6 +44,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -34,6 +34,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -44,6 +44,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -34,6 +34,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -45,6 +45,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -34,6 +34,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -41,6 +41,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -34,6 +34,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker

View File

@ -51,6 +51,7 @@ services:
- SERVICES=1
- SWARM=1
- TASKS=1
- LOG_LEVEL=warning
networks:
- bw-docker
deploy:

View File

@ -55,6 +55,7 @@ services:
- SERVICES=1
- SWARM=1
- TASKS=1
- LOG_LEVEL=warning
networks:
- bw-docker
deploy:

View File

@ -51,6 +51,7 @@ services:
- SERVICES=1
- SWARM=1
- TASKS=1
- LOG_LEVEL=warning
networks:
- bw-docker
deploy:

View File

@ -51,6 +51,7 @@ services:
- SERVICES=1
- SWARM=1
- TASKS=1
- LOG_LEVEL=warning
networks:
- bw-docker
deploy:

View File

@ -51,6 +51,7 @@ services:
- SERVICES=1
- SWARM=1
- TASKS=1
- LOG_LEVEL=warning
networks:
- bw-docker
deploy:

View File

@ -51,6 +51,7 @@ services:
- SERVICES=1
- SWARM=1
- TASKS=1
- LOG_LEVEL=warning
networks:
- bw-docker
deploy:

View File

@ -1,15 +1,18 @@
from os import getenv
from time import sleep
#!/usr/bin/python3
from ConfigCaller import ConfigCaller
from Database import Database
from logger import setup_logger
from os import getenv
from threading import Lock
from time import sleep
from typing import Optional
from ConfigCaller import ConfigCaller # type: ignore
from Database import Database # type: ignore
from logger import setup_logger # type: ignore
class Config(ConfigCaller):
def __init__(self, ctrl_type, lock=None):
def __init__(self, lock: Optional[Lock] = None):
super().__init__()
self.__ctrl_type = ctrl_type
self.__lock = lock
self.__logger = setup_logger("Config", getenv("LOG_LEVEL", "INFO"))
self.__instances = []
@ -77,6 +80,9 @@ class Config(ConfigCaller):
)
sleep(5)
if self.__lock:
self.__lock.acquire()
# update instances in database
err = self._db.update_instances(self.__instances)
if err:
@ -98,4 +104,7 @@ class Config(ConfigCaller):
f"Can't save autoconf custom configs in database: {err}, custom configs may not work as expected",
)
if self.__lock:
self.__lock.release()
return success

View File

@ -1,14 +1,23 @@
#!/usr/bin/python3
from abc import ABC, abstractmethod
from os import getenv
from threading import Lock
from time import sleep
from typing import Literal, Optional, Union
from Config import Config
from logger import setup_logger
from logger import setup_logger # type: ignore
class Controller(ABC):
def __init__(self, ctrl_type, lock=None):
class Controller(Config):
def __init__(
self,
ctrl_type: Union[Literal["docker"], Literal["swarm"], Literal["kubernetes"]],
lock: Optional[Lock] = None,
):
super().__init__(lock)
self._type = ctrl_type
self._instances = []
self._services = []
@ -24,15 +33,16 @@ class Controller(ABC):
self._configs = {
config_type: {} for config_type in self._supported_config_types
}
self._config = Config(ctrl_type, lock)
self.__logger = setup_logger("Controller", getenv("LOG_LEVEL", "INFO"))
self._logger = setup_logger(
f"{self._type}-controller", getenv("LOG_LEVEL", "INFO")
)
def wait(self, wait_time):
def wait(self, wait_time: int) -> list:
all_ready = False
while not all_ready:
self._instances = self.get_instances()
if not self._instances:
self.__logger.warning(
self._logger.warning(
f"No instance found, waiting {wait_time}s ...",
)
sleep(wait_time)
@ -40,7 +50,7 @@ class Controller(ABC):
all_ready = True
for instance in self._instances:
if not instance["health"]:
self.__logger.warning(
self._logger.warning(
f"Instance {instance['name']} is not ready, waiting {wait_time}s ...",
)
sleep(wait_time)
@ -59,8 +69,7 @@ class Controller(ABC):
def get_instances(self):
instances = []
for controller_instance in self._get_controller_instances():
for instance in self._to_instances(controller_instance):
instances.append(instance)
instances.extend(self._to_instances(controller_instance))
return instances
@abstractmethod
@ -76,20 +85,18 @@ class Controller(ABC):
pass
def _set_autoconf_load_db(self):
if not self._config._db.is_autoconf_loaded():
ret = self._config._db.set_autoconf_load(True)
if not self._db.is_autoconf_loaded():
ret = self._db.set_autoconf_load(True)
if ret:
self.__logger.warning(
self._logger.warning(
f"Can't set autoconf loaded metadata to true in database: {ret}",
)
def get_services(self):
services = []
for controller_service in self._get_controller_services():
for service in self._to_services(controller_service):
services.append(service)
for static_service in self._get_static_services():
services.append(static_service)
services.extend(self._to_services(controller_service))
services.extend(self._get_static_services())
return services
@abstractmethod
@ -106,8 +113,8 @@ class Controller(ABC):
def _is_service_present(self, server_name):
for service in self._services:
if not "SERVER_NAME" in service or service["SERVER_NAME"] == "":
if not "SERVER_NAME" in service or not service["SERVER_NAME"]:
continue
if server_name == service["SERVER_NAME"].split(" ")[0]:
if server_name == service["SERVER_NAME"].strip().split(" ")[0]:
return True
return False

View File

@ -1,30 +1,29 @@
from os import getenv
#!/usr/bin/python3
from typing import Any, Dict, List
from docker import DockerClient
from re import compile as re_compile
from traceback import format_exc
from docker.models.containers import Container
from Controller import Controller
from ConfigCaller import ConfigCaller
from logger import setup_logger
class DockerController(Controller, ConfigCaller):
class DockerController(Controller):
def __init__(self, docker_host):
Controller.__init__(self, "docker")
ConfigCaller.__init__(self)
super().__init__("docker")
self.__client = DockerClient(base_url=docker_host)
self.__logger = setup_logger("docker-controller", getenv("LOG_LEVEL", "INFO"))
self.__custom_confs_rx = re_compile(
r"^bunkerweb.CUSTOM_CONF_(SERVER_HTTP|MODSEC_CRS|MODSEC)_(.+)$"
)
def _get_controller_instances(self):
def _get_controller_instances(self) -> List[Container]:
return self.__client.containers.list(filters={"label": "bunkerweb.INSTANCE"})
def _get_controller_services(self):
def _get_controller_services(self) -> List[Container]:
return self.__client.containers.list(filters={"label": "bunkerweb.SERVER_NAME"})
def _to_instances(self, controller_instance):
def _to_instances(self, controller_instance) -> List[dict]:
instance = {}
instance["name"] = controller_instance.name
instance["hostname"] = controller_instance.name
@ -40,18 +39,18 @@ class DockerController(Controller, ConfigCaller):
instance["env"][variable] = value
return [instance]
def _to_services(self, controller_service):
def _to_services(self, controller_service) -> List[dict]:
service = {}
for variable, value in controller_service.labels.items():
if not variable.startswith("bunkerweb."):
continue
real_variable = variable.replace("bunkerweb.", "", 1)
if not self._is_multisite_setting(real_variable):
if not self._is_setting_context(real_variable, "multisite"):
continue
service[real_variable] = value
return [service]
def _get_static_services(self):
def _get_static_services(self) -> List[dict]:
services = []
variables = {}
for instance in self.__client.containers.list(
@ -71,14 +70,14 @@ class DockerController(Controller, ConfigCaller):
for variable, value in variables.items():
prefix = variable.split("_")[0]
real_variable = variable.replace(f"{prefix}_", "", 1)
if prefix == server_name and self._is_multisite_setting(
real_variable
if prefix == server_name and self._is_setting_context(
real_variable, "multisite"
):
service[real_variable] = value
services.append(service)
return services
def get_configs(self):
def get_configs(self) -> Dict[str, Dict[str, Any]]:
configs = {config_type: {} for config_type in self._supported_config_types}
# get site configs from labels
for container in self.__client.containers.list(
@ -106,10 +105,8 @@ class DockerController(Controller, ConfigCaller):
] = value
return configs
def apply_config(self):
return self._config.apply(
self._instances, self._services, configs=self._configs
)
def apply_config(self) -> bool:
return self.apply(self._instances, self._services, configs=self._configs)
def process_events(self):
self._set_autoconf_load_db()
@ -118,27 +115,22 @@ class DockerController(Controller, ConfigCaller):
self._instances = self.get_instances()
self._services = self.get_services()
self._configs = self.get_configs()
if not self._config.update_needed(
if not self.update_needed(
self._instances, self._services, configs=self._configs
):
continue
self.__logger.info(
self._logger.info(
"Caught Docker event, deploying new configuration ..."
)
if not self.apply_config():
self.__logger.error("Error while deploying new configuration")
self._logger.error("Error while deploying new configuration")
else:
self.__logger.info(
self._logger.info(
"Successfully deployed new configuration 🚀",
)
if not self._config._db.is_autoconf_loaded():
ret = self._config._db.set_autoconf_load(True)
if ret:
self.__logger.warning(
f"Can't set autoconf loaded metadata to true in database: {ret}",
)
self._set_autoconf_load_db()
except:
self.__logger.error(
self._logger.error(
f"Exception while processing events :\n{format_exc()}"
)

View File

@ -8,6 +8,9 @@ RUN mkdir -p /usr/share/bunkerweb/deps && \
cat /tmp/req/requirements.txt /tmp/req/requirements.txt.1 > /usr/share/bunkerweb/deps/requirements.txt && \
rm -rf /tmp/req
# Update apk
RUN apk update
# Install python dependencies
RUN apk add --no-cache --virtual .build-deps g++ gcc musl-dev jpeg-dev zlib-dev libffi-dev cairo-dev pango-dev gdk-pixbuf-dev openssl-dev cargo postgresql-dev
@ -60,7 +63,7 @@ RUN apk add --no-cache bash && \
chmod 750 /usr/share/bunkerweb/cli/main.py /usr/share/bunkerweb/helpers/*.sh /usr/bin/bwcli /usr/share/bunkerweb/autoconf/main.py /usr/share/bunkerweb/deps/python/bin/*
# Fix CVEs
RUN apk add "libcrypto3>=3.0.8-r4" "libssl3>=3.0.8-r4"
RUN apk add --no-cache "libcrypto3>=3.1.1-r0" "libssl3>=3.1.1-r0"
VOLUME /data /etc/nginx

View File

@ -1,26 +1,24 @@
from os import getenv
#!/usr/bin/python3
from time import sleep
from traceback import format_exc
from typing import List
from kubernetes import client, config, watch
from kubernetes.client.exceptions import ApiException
from threading import Thread, Lock
from Controller import Controller
from ConfigCaller import ConfigCaller
from logger import setup_logger
class IngressController(Controller, ConfigCaller):
class IngressController(Controller):
def __init__(self):
Controller.__init__(self, "kubernetes")
ConfigCaller.__init__(self)
self.__internal_lock = Lock()
super().__init__("kubernetes", self.__internal_lock)
config.load_incluster_config()
self.__corev1 = client.CoreV1Api()
self.__networkingv1 = client.NetworkingV1Api()
self.__internal_lock = Lock()
self.__logger = setup_logger("Ingress-controller", getenv("LOG_LEVEL", "INFO"))
def _get_controller_instances(self):
def _get_controller_instances(self) -> list:
return [
pod
for pod in self.__corev1.list_pod_for_all_namespaces(watch=False).items
@ -30,7 +28,7 @@ class IngressController(Controller, ConfigCaller):
)
]
def _to_instances(self, controller_instance):
def _to_instances(self, controller_instance) -> List[dict]:
instance = {}
instance["name"] = controller_instance.metadata.name
instance["hostname"] = controller_instance.status.pod_ip
@ -48,7 +46,9 @@ class IngressController(Controller, ConfigCaller):
pod = container
break
if not pod:
self.__logger.warning(f"Missing container bunkerweb in pod {controller_instance.metadata.name}")
self._logger.warning(
f"Missing container bunkerweb in pod {controller_instance.metadata.name}"
)
else:
for env in pod.env:
instance["env"][env.name] = env.value or ""
@ -65,10 +65,10 @@ class IngressController(Controller, ConfigCaller):
instance["env"][variable] = value
return [instance]
def _get_controller_services(self):
def _get_controller_services(self) -> list:
return self.__networkingv1.list_ingress_for_all_namespaces(watch=False).items
def _to_services(self, controller_service):
def _to_services(self, controller_service) -> List[dict]:
if not controller_service.spec or not controller_service.spec.rules:
return []
@ -76,7 +76,7 @@ class IngressController(Controller, ConfigCaller):
# parse rules
for rule in controller_service.spec.rules:
if not rule.host:
self.__logger.warning(
self._logger.warning(
"Ignoring unsupported ingress rule without host.",
)
continue
@ -88,22 +88,22 @@ class IngressController(Controller, ConfigCaller):
location = 1
for path in rule.http.paths:
if not path.path:
self.__logger.warning(
self._logger.warning(
"Ignoring unsupported ingress rule without path.",
)
continue
elif not path.backend.service:
self.__logger.warning(
self._logger.warning(
"Ignoring unsupported ingress rule without backend service.",
)
continue
elif not path.backend.service.port:
self.__logger.warning(
self._logger.warning(
"Ignoring unsupported ingress rule without backend service port.",
)
continue
elif not path.backend.service.port.number:
self.__logger.warning(
self._logger.warning(
"Ignoring unsupported ingress rule without backend service port number.",
)
continue
@ -114,7 +114,7 @@ class IngressController(Controller, ConfigCaller):
).items
if not service_list:
self.__logger.warning(
self._logger.warning(
f"Ignoring ingress rule with service {path.backend.service.name} : service not found.",
)
continue
@ -132,7 +132,7 @@ class IngressController(Controller, ConfigCaller):
# parse tls
if controller_service.spec.tls: # TODO: support tls
self.__logger.warning("Ignoring unsupported tls.")
self._logger.warning("Ignoring unsupported tls.")
# parse annotations
if controller_service.metadata.annotations:
@ -145,15 +145,15 @@ class IngressController(Controller, ConfigCaller):
continue
variable = annotation.replace("bunkerweb.io/", "", 1)
server_name = service["SERVER_NAME"].split(" ")[0]
server_name = service["SERVER_NAME"].strip().split(" ")[0]
if not variable.startswith(f"{server_name}_"):
continue
variable = variable.replace(f"{server_name}_", "", 1)
if self._is_multisite_setting(variable):
if self._is_setting_context(variable, "multisite"):
service[variable] = value
return services
def _get_static_services(self):
def _get_static_services(self) -> List[dict]:
services = []
variables = {}
for instance in self.__corev1.list_pod_for_all_namespaces(watch=False).items:
@ -168,12 +168,10 @@ class IngressController(Controller, ConfigCaller):
if container.name == "bunkerweb":
pod = container
break
if not pod :
if not pod:
continue
variables = {
env.name: env.value or "" for env in pod.env
}
variables = {env.name: env.value or "" for env in pod.env}
if "SERVER_NAME" in variables and variables["SERVER_NAME"].strip():
for server_name in variables["SERVER_NAME"].strip().split(" "):
@ -181,14 +179,14 @@ class IngressController(Controller, ConfigCaller):
for variable, value in variables.items():
prefix = variable.split("_")[0]
real_variable = variable.replace(f"{prefix}_", "", 1)
if prefix == server_name and self._is_multisite_setting(
real_variable
if prefix == server_name and self._is_setting_context(
real_variable, "multisite"
):
service[real_variable] = value
services.append(service)
return services
def get_configs(self):
def get_configs(self) -> dict:
configs = {config_type: {} for config_type in self._supported_config_types}
for configmap in self.__corev1.list_config_map_for_all_namespaces(
watch=False
@ -201,12 +199,12 @@ class IngressController(Controller, ConfigCaller):
config_type = configmap.metadata.annotations["bunkerweb.io/CONFIG_TYPE"]
if config_type not in self._supported_config_types:
self.__logger.warning(
self._logger.warning(
f"Ignoring unsupported CONFIG_TYPE {config_type} for ConfigMap {configmap.metadata.name}",
)
continue
elif not configmap.data:
self.__logger.warning(
self._logger.warning(
f"Ignoring blank ConfigMap {configmap.metadata.name}",
)
continue
@ -215,7 +213,7 @@ class IngressController(Controller, ConfigCaller):
if not self._is_service_present(
configmap.metadata.annotations["bunkerweb.io/CONFIG_SITE"]
):
self.__logger.warning(
self._logger.warning(
f"Ignoring config {configmap.metadata.name} because {configmap.metadata.annotations['bunkerweb.io/CONFIG_SITE']} doesn't exist",
)
continue
@ -250,46 +248,41 @@ class IngressController(Controller, ConfigCaller):
self._instances = self.get_instances()
self._services = self.get_services()
self._configs = self.get_configs()
if not self._config.update_needed(
if not self.update_needed(
self._instances, self._services, configs=self._configs
):
self.__internal_lock.release()
locked = False
continue
self.__logger.info(
self._logger.info(
f"Catched kubernetes event ({watch_type}), deploying new configuration ...",
)
try:
ret = self.apply_config()
if not ret:
self.__logger.error(
self._logger.error(
"Error while deploying new configuration ...",
)
else:
self.__logger.info(
self._logger.info(
"Successfully deployed new configuration 🚀",
)
if not self._config._db.is_autoconf_loaded():
ret = self._config._db.set_autoconf_load(True)
if ret:
self.__logger.warning(
f"Can't set autoconf loaded metadata to true in database: {ret}",
)
self._set_autoconf_load_db()
except:
self.__logger.error(
self._logger.error(
f"Exception while deploying new configuration :\n{format_exc()}",
)
self.__internal_lock.release()
locked = False
except ApiException as e:
if e.status != 410:
self.__logger.error(
self._logger.error(
f"API exception while reading k8s event (type = {watch_type}) :\n{format_exc()}",
)
error = True
except:
self.__logger.error(
self._logger.error(
f"Unknown exception while reading k8s event (type = {watch_type}) :\n{format_exc()}",
)
error = True
@ -299,13 +292,11 @@ class IngressController(Controller, ConfigCaller):
locked = False
if error is True:
self.__logger.warning("Got exception, retrying in 10 seconds ...")
self._logger.warning("Got exception, retrying in 10 seconds ...")
sleep(10)
def apply_config(self):
return self._config.apply(
self._instances, self._services, configs=self._configs
)
def apply_config(self) -> bool:
return self.apply(self._instances, self._services, configs=self._configs)
def process_events(self):
self._set_autoconf_load_db()

View File

@ -1,30 +1,29 @@
from os import getenv
#!/usr/bin/python3
from time import sleep
from traceback import format_exc
from threading import Thread, Lock
from typing import Any, Dict, List
from docker import DockerClient
from base64 import b64decode
from docker.models.services import Service
from Controller import Controller
from ConfigCaller import ConfigCaller
from logger import setup_logger
class SwarmController(Controller, ConfigCaller):
class SwarmController(Controller):
def __init__(self, docker_host):
Controller.__init__(self, "swarm")
ConfigCaller.__init__(self)
super().__init__("swarm")
self.__client = DockerClient(base_url=docker_host)
self.__internal_lock = Lock()
self.__logger = setup_logger("Swarm-controller", getenv("LOG_LEVEL", "INFO"))
def _get_controller_instances(self):
def _get_controller_instances(self) -> List[Service]:
return self.__client.services.list(filters={"label": "bunkerweb.INSTANCE"})
def _get_controller_services(self):
def _get_controller_services(self) -> List[Service]:
return self.__client.services.list(filters={"label": "bunkerweb.SERVER_NAME"})
def _to_instances(self, controller_instance):
def _to_instances(self, controller_instance) -> List[dict]:
instances = []
instance_env = {}
for env in controller_instance.attrs["Spec"]["TaskTemplate"]["ContainerSpec"][
@ -48,18 +47,18 @@ class SwarmController(Controller, ConfigCaller):
)
return instances
def _to_services(self, controller_service):
def _to_services(self, controller_service) -> List[dict]:
service = {}
for variable, value in controller_service.attrs["Spec"]["Labels"].items():
if not variable.startswith("bunkerweb."):
continue
real_variable = variable.replace("bunkerweb.", "", 1)
if not self._is_multisite_setting(real_variable):
if not self._is_setting_context(real_variable, "multisite"):
continue
service[real_variable] = value
return [service]
def _get_static_services(self):
def _get_static_services(self) -> List[dict]:
services = []
variables = {}
for instance in self.__client.services.list(
@ -81,14 +80,14 @@ class SwarmController(Controller, ConfigCaller):
for variable, value in variables.items():
prefix = variable.split("_")[0]
real_variable = variable.replace(f"{prefix}_", "", 1)
if prefix == server_name and self._is_multisite_setting(
real_variable
if prefix == server_name and self._is_setting_context(
real_variable, "multisite"
):
service[real_variable] = value
services.append(service)
return services
def get_configs(self):
def get_configs(self) -> Dict[str, Dict[str, Any]]:
configs = {}
for config_type in self._supported_config_types:
configs[config_type] = {}
@ -106,7 +105,7 @@ class SwarmController(Controller, ConfigCaller):
config_type = config.attrs["Spec"]["Labels"]["bunkerweb.CONFIG_TYPE"]
config_name = config.name
if config_type not in self._supported_config_types:
self.__logger.warning(
self._logger.warning(
f"Ignoring unsupported CONFIG_TYPE {config_type} for Config {config_name}",
)
continue
@ -115,7 +114,7 @@ class SwarmController(Controller, ConfigCaller):
if not self._is_service_present(
config.attrs["Spec"]["Labels"]["bunkerweb.CONFIG_SITE"]
):
self.__logger.warning(
self._logger.warning(
f"Ignoring config {config_name} because {config.attrs['Spec']['Labels']['bunkerweb.CONFIG_SITE']} doesn't exist",
)
continue
@ -127,10 +126,8 @@ class SwarmController(Controller, ConfigCaller):
)
return configs
def apply_config(self):
return self._config.apply(
self._instances, self._services, configs=self._configs
)
def apply_config(self) -> bool:
return self.apply(self._instances, self._services, configs=self._configs)
def __event(self, event_type):
while True:
@ -146,31 +143,31 @@ class SwarmController(Controller, ConfigCaller):
self._instances = self.get_instances()
self._services = self.get_services()
self._configs = self.get_configs()
if not self._config.update_needed(
if not self.update_needed(
self._instances, self._services, configs=self._configs
):
self.__internal_lock.release()
locked = False
continue
self.__logger.info(
self._logger.info(
f"Catched Swarm event ({event_type}), deploying new configuration ..."
)
if not self.apply_config():
self.__logger.error(
self._logger.error(
"Error while deploying new configuration"
)
else:
self.__logger.info(
self._logger.info(
"Successfully deployed new configuration 🚀",
)
except:
self.__logger.error(
self._logger.error(
f"Exception while processing Swarm event ({event_type}) :\n{format_exc()}"
)
self.__internal_lock.release()
locked = False
except:
self.__logger.error(
self._logger.error(
f"Exception while reading Swarm event ({event_type}) :\n{format_exc()}",
)
error = True
@ -179,7 +176,7 @@ class SwarmController(Controller, ConfigCaller):
self.__internal_lock.release()
locked = False
if error is True:
self.__logger.warning("Got exception, retrying in 10 seconds ...")
self._logger.warning("Got exception, retrying in 10 seconds ...")
sleep(10)
def process_events(self):

View File

@ -1,21 +1,20 @@
#!/usr/bin/python3
from os import _exit, getenv
from os import _exit, getenv, sep
from os.path import join
from signal import SIGINT, SIGTERM, signal
from sys import exit as sys_exit, path as sys_path
from traceback import format_exc
from pathlib import Path
sys_path.extend(
(
"/usr/share/bunkerweb/deps/python",
"/usr/share/bunkerweb/utils",
"/usr/share/bunkerweb/api",
"/usr/share/bunkerweb/db",
)
)
for deps_path in [
join(sep, "usr", "share", "bunkerweb", *paths)
for paths in (("deps", "python"), ("utils",), ("api",), ("db",))
]:
if deps_path not in sys_path:
sys_path.append(deps_path)
from logger import setup_logger
from logger import setup_logger # type: ignore
from SwarmController import SwarmController
from IngressController import IngressController
from DockerController import DockerController
@ -70,12 +69,11 @@ try:
_exit(1)
# Process events
Path("/var/tmp/bunkerweb/autoconf.healthy").write_text("ok")
Path(sep, "var", "tmp", "bunkerweb", "autoconf.healthy").write_text("ok")
logger.info("Processing events ...")
controller.process_events()
except:
logger.error(f"Exception while running autoconf :\n{format_exc()}")
sys_exit(1)
finally:
Path("/var/tmp/bunkerweb/autoconf.healthy").unlink(missing_ok=True)
Path(sep, "var", "tmp", "bunkerweb", "autoconf.healthy").unlink(missing_ok=True)

View File

@ -3,6 +3,9 @@ FROM nginx:1.24.0-alpine AS builder
# Copy dependencies sources folder
COPY src/deps /tmp/bunkerweb/deps
# Update apk
RUN apk update
# Compile and install dependencies
RUN apk add --no-cache --virtual .build-deps bash autoconf libtool automake geoip-dev g++ gcc curl-dev libxml2-dev pcre-dev make linux-headers musl-dev gd-dev gnupg brotli-dev openssl-dev patch readline-dev && \
mkdir -p /usr/share/bunkerweb/deps && \
@ -51,6 +54,7 @@ COPY --from=builder --chown=0:101 /usr/share/bunkerweb /usr/share/bunkerweb
RUN apk add --no-cache pcre bash python3 && \
cp /usr/share/bunkerweb/helpers/bwcli /usr/bin/ && \
mkdir -p /var/tmp/bunkerweb && \
mkdir -p /var/run/bunkerweb && \
mkdir -p /var/www/html && \
mkdir -p /etc/bunkerweb && \
mkdir -p /data/cache && ln -s /data/cache /var/cache/bunkerweb && \
@ -58,8 +62,8 @@ RUN apk add --no-cache pcre bash python3 && \
for dir in $(echo "configs/http configs/stream configs/server-http configs/server-stream configs/default-server-http configs/default-server-stream configs/modsec configs/modsec-crs") ; do mkdir "/data/${dir}" ; done && \
chown -R root:nginx /data && \
chmod -R 770 /data && \
chown -R root:nginx /var/cache/bunkerweb /etc/bunkerweb /var/tmp/bunkerweb /usr/bin/bwcli && \
chmod 770 /var/cache/bunkerweb /var/tmp/bunkerweb && \
chown -R root:nginx /var/cache/bunkerweb /etc/bunkerweb /var/tmp/bunkerweb /var/run/bunkerweb /usr/bin/bwcli && \
chmod 770 /var/cache/bunkerweb /var/tmp/bunkerweb /var/run/bunkerweb && \
chmod 750 /usr/share/bunkerweb/cli/main.py /usr/share/bunkerweb/gen/main.py /usr/share/bunkerweb/helpers/*.sh /usr/share/bunkerweb/entrypoint.sh /usr/bin/bwcli /usr/share/bunkerweb/deps/python/bin/* && \
chown -R root:nginx /etc/nginx && \
chmod -R 770 /etc/nginx && \
@ -69,7 +73,7 @@ RUN apk add --no-cache pcre bash python3 && \
ln -s /proc/1/fd/1 /var/log/nginx/access.log
# Fix CVEs
RUN apk add "libcrypto3>=3.0.8-r4" "libssl3>=3.0.8-r4" "curl>=8.1.0-r0" "libcurl>=8.1.0-r0"
RUN apk add "libcrypto3>=3.0.8-r4" "libssl3>=3.0.8-r4" "curl>=8.1.0-r0" "libcurl>=8.1.0-r0" "libwebp>=1.2.4-r2" "ncurses-libs>=6.3_p20221119-r1" "ncurses-terminfo-base>=6.3_p20221119-r1"
VOLUME /data /etc/nginx

View File

@ -21,7 +21,7 @@ trap "trap_exit" TERM INT QUIT
# trap SIGHUP
function trap_reload() {
log "ENTRYPOINT" "" "Catched reload operation"
if [ -f /var/tmp/bunkerweb/nginx.pid ] ; then
if [ -f /var/run/bunkerweb/nginx.pid ] ; then
log "ENTRYPOINT" "" "Reloading nginx ..."
nginx -s reload
if [ $? -eq 0 ] ; then
@ -50,7 +50,7 @@ pid="$!"
# wait while nginx is running
wait "$pid"
while [ -f "/var/tmp/bunkerweb/nginx.pid" ] ; do
while [ -f "/var/run/bunkerweb/nginx.pid" ] ; do
wait "$pid"
done

Some files were not shown because too many files have changed in this diff Show More