Add Plugin Pages feature

This commit is contained in:
TheophileDiot 2022-07-12 15:35:26 +02:00
parent 795dfc0778
commit 2c4efe9d0e
7 changed files with 477 additions and 113 deletions

View File

@ -369,3 +369,62 @@ end
### Jobs
BunkerWeb uses an internal job scheduler for periodic tasks like renewing certificates with certbot, downloading blacklists, downloading MMDB files, ... You can add tasks of your choice by putting them inside a subfolder named **jobs** and listing them in the **plugin.json** metadata file. Don't forget to add the execution permissions for everyone to avoid any problems when a user is cloning and installing your plugin.
### Plugin page
Plugin pages are used to display information about your plugin. You can create a page by creating a subfolder named **ui** next to the file **plugin.json** and putting a **template.html** file inside it. The template file will be used to display the page.
A plugin page can have a form that is used to submit data to the plugin. To get the values of the form, you need to put a **actions.py** file in the **ui** folder. Inside the file, **you must define a function that has the same name as the plugin**. This function will be called when the form is submitted. You can then use the **request** object (from the library flask) to get the values of the form. The form's action must finish with **/plugins/<*plugin_id*>**.
!!! info "Template variables"
Your template file can use template variables to display the content of your plugin. Like *Jinja2*, the template variables can be accessed by using the `{{` and `}}` delimiters. To use template variables, your custom function must return a dictionary with the template variables. The dictionary keys are the template variables names and the values are the values to display. Example :
```json
{
"foo": "bar"
}
```
```html
<html>
<body>
<p>{{ foo }}</p>
</body>
</html>
```
Will display : `bar`
If you want to submit your form through a POST request, you need to add the following line to your form :
```html
<input type="hidden" name="csrf_token" value="{{ csrf_token() }}" />
```
Otherwise, the form will not be submitted because of the CSRF token protection.
!!! tip "Plugins pages"
Plugins pages are displayed in the **Plugins** section of the Web UI.
For example, I have a plugin called **myplugin** and I want to create a custom page. I just have to create a subfolder called **ui** and put a **template.html** file inside it. I want my plugin to display a form that will submit the data to the plugin. I can then use the **request** object (from the library flask) to get the values of the form. For that I create a **actions.py** file in the same **ui** folder as my **template.html** file. I define a function called **myplugin** that returns a dictionary with the template variables I want to display.
```html
<html>
<body>
<p>{{ foo }}</p>
<form action="/plugins/myplugin" method="POST">
<input type="hidden" name="csrf_token" value="{{ csrf_token() }}" />
<input type="text" name="foo" />
<input type="submit" value="Submit" />
</form>
</body>
</html>
```
```python
from flask import request
def myplugin():
return {
"foo": request.form["foo"]
}
```

View File

@ -1,6 +1,11 @@
from email import message
import os
from shutil import rmtree, copytree, chown
from logging import getLogger, INFO, ERROR, StreamHandler, Formatter
from traceback import format_exc
from typing import Optional
from jinja2 import Template
from threading import Thread
from flask import (
Flask,
flash,
@ -12,15 +17,16 @@ from flask import (
url_for,
)
from flask_login import LoginManager, login_required, login_user, logout_user
from flask_wtf.csrf import CSRFProtect, CSRFError
from flask_wtf.csrf import CSRFProtect, CSRFError, generate_csrf
from json import JSONDecodeError, load as json_load
from bs4 import BeautifulSoup
from datetime import datetime, timezone
from dateutil.parser import parse as dateutil_parse
from requests import get
from requests.utils import default_headers
from sys import path as sys_path, exit as sys_exit
from sys import path as sys_path, exit as sys_exit, modules as sys_modules
from copy import deepcopy
from re import match as re_match
from docker import DockerClient
from docker.errors import (
NotFound as docker_NotFound,
@ -144,6 +150,9 @@ try:
WTF_CSRF_SSL_STRICT=False,
USER=user,
SEND_FILE_MAX_AGE_DEFAULT=86400,
PLUGIN_ARGS=None,
RELOADING=False,
TO_FLASH=[],
)
except FileNotFoundError as e:
logger.error(repr(e), e.filename)
@ -161,6 +170,50 @@ app.jinja_env.globals.update(gen_folders_tree_html=gen_folders_tree_html)
app.jinja_env.globals.update(check_settings=check_settings)
def manage_bunkerweb(method: str, operation: str = "reloads", *args):
# Do the operation
if method == "services":
if operation == "new":
operation, error = app.config["CONFIG"].new_service(args[0])
elif operation == "edit":
operation = app.config["CONFIG"].edit_service(args[1], args[0])
elif operation == "delete":
operation, error = app.config["CONFIG"].delete_service(args[2])
if error:
app.config["TO_FLASH"].append({"content": operation, "type": "error"})
else:
app.config["TO_FLASH"].append({"content": operation, "type": "success"})
if method == "global_config":
operation = app.config["CONFIG"].edit_global_conf(args[0])
app.config["TO_FLASH"].append({"content": operation, "type": "success"})
elif method == "plugins":
app.config["CONFIG"].reload_config()
if operation == "reload":
operation = app.config["INSTANCES"].reload_instance(args[0])
elif operation == "start":
operation = app.config["INSTANCES"].start_instance(args[0])
elif operation == "stop":
operation = app.config["INSTANCES"].stop_instance(args[0])
elif operation == "restart":
operation = app.config["INSTANCES"].restart_instance(args[0])
else:
operation = app.config["INSTANCES"].reload_instances()
if isinstance(operation, list):
for op in operation:
app.config["TO_FLASH"].append(
{"content": f"Reload failed for the instance {op}", "type": "error"}
)
elif operation.startswith("Can't"):
app.config["TO_FLASH"].append({"content": operation, "type": "error"})
else:
app.config["TO_FLASH"].append({"content": operation, "type": "success"})
app.config["RELOADING"] = False
@login_manager.user_loader
def load_user(user_id):
return User(user_id, vars["ADMIN_PASSWORD"])
@ -172,7 +225,7 @@ csrf.init_app(app)
@app.errorhandler(CSRFError)
def handle_csrf_error(e):
def handle_csrf_error(_):
"""
It takes a CSRFError exception as an argument, and returns a Flask response
@ -192,8 +245,13 @@ def index():
@app.route("/loading")
@login_required
def loading():
next_url = request.values.get("next")
return render_template("loading.html", next=next_url)
next_url: str = request.values.get("next", None) or url_for("home")
message: Optional[str] = request.values.get("message", None)
return render_template(
"loading.html",
message=message if message is not None else "Loading",
next=next_url,
)
@app.route("/home")
@ -291,30 +349,25 @@ def instances():
flash("Missing INSTANCE_ID parameter.", "error")
return redirect(url_for("loading", next=url_for("instances")))
# Do the operation
if request.form["operation"] == "reload":
operation = app.config["INSTANCES"].reload_instance(
request.form["INSTANCE_ID"]
)
elif request.form["operation"] == "start":
operation = app.config["INSTANCES"].start_instance(
request.form["INSTANCE_ID"]
)
elif request.form["operation"] == "stop":
operation = app.config["INSTANCES"].stop_instance(
request.form["INSTANCE_ID"]
)
elif request.form["operation"] == "restart":
operation = app.config["INSTANCES"].restart_instance(
request.form["INSTANCE_ID"]
)
app.config["RELOADING"] = True
Thread(
target=manage_bunkerweb,
name="Reloading instances",
args=("instances", request.form["operation"], request.form["INSTANCE_ID"]),
).start()
if operation.startswith("Can't"):
flash(operation, "error")
else:
flash(operation)
return redirect(url_for("loading", next=url_for("instances")))
return redirect(
url_for(
"loading",
next=url_for("instances"),
message=(
f"{request.form['operation'].title()}ing"
if request.form["operation"] is not "stop"
else "Stopping"
)
+ " instance",
)
)
# Display instances
instances = app.config["INSTANCES"].get_instances()
@ -394,33 +447,32 @@ def services():
error = 0
# Do the operation
# Reload instances
app.config["RELOADING"] = True
Thread(
target=manage_bunkerweb,
name="Reloading instances",
args=(
"services",
request.form["operation"],
variables,
request.form.get("OLD_SERVER_NAME", None),
request.form.get("SERVER_NAME", None),
),
).start()
message = ""
if request.form["operation"] == "new":
operation, error = app.config["CONFIG"].new_service(variables)
message = f"Creating service {variables['SERVER_NAME'].split(' ')[0]}"
elif request.form["operation"] == "edit":
operation = app.config["CONFIG"].edit_service(
request.form["OLD_SERVER_NAME"], variables
message = (
f"Saving configuration for service {request.form['OLD_SERVER_NAME']}"
)
elif request.form["operation"] == "delete":
operation, error = app.config["CONFIG"].delete_service(
request.form["SERVER_NAME"]
)
message = f"Deleting service {request.form['SERVER_NAME']}"
if error:
flash(operation, "error")
return redirect(url_for("loading", next=url_for("services")))
flash(operation)
# Reload instances
_reloads = app.config["INSTANCES"].reload_instances()
if not _reloads:
for _reload in _reloads:
flash(f"Reload failed for the instance {_reload}", "error")
else:
flash("Successfully reloaded instances")
return redirect(url_for("loading", next=url_for("services")))
return redirect(url_for("loading", next=url_for("services"), message=message))
# Display services
services = app.config["CONFIG"].get_services()
@ -461,26 +513,25 @@ def global_config():
if error:
return redirect(url_for("loading", next=url_for("global_config")))
error = 0
# Do the operation
operation = app.config["CONFIG"].edit_global_conf(variables)
if error:
flash(operation, "error")
return redirect(url_for("loading", next=url_for("global_config")))
flash(operation)
# Reload instances
_reloads = app.config["INSTANCES"].reload_instances()
if not _reloads:
for _reload in _reloads:
flash(f"Reload failed for the instance {_reload}", "error")
else:
flash("Successfully reloaded instances")
app.config["RELOADING"] = True
Thread(
target=manage_bunkerweb,
name="Reloading instances",
args=(
"global_config",
"reloads",
variables,
),
).start()
return redirect(url_for("loading", next=url_for("global_config")))
return redirect(
url_for(
"loading",
next=url_for("global_config"),
message="Saving global configuration",
)
)
# Display services
services = app.config["CONFIG"].get_services()
@ -558,12 +609,12 @@ def configs():
flash(operation)
# Reload instances
_reloads = app.config["INSTANCES"].reload_instances()
if not _reloads:
for _reload in _reloads:
flash(f"Reload failed for the instance {_reload}", "error")
else:
flash("Successfully reloaded instances")
app.config["RELOADING"] = True
Thread(
target=manage_bunkerweb,
name="Reloading instances",
args=("configs",),
).start()
return redirect(url_for("loading", next=url_for("configs")))
@ -868,24 +919,24 @@ def plugins():
error = 0
# Fix permissions for plugins folders
for root, dirs, files in os.walk("/opt/bunkerweb/plugins", topdown=False):
for name in files + dirs:
chown(os.path.join(root, name), "nginx", "nginx")
os.chmod(os.path.join(root, name), 0o770)
app.config["CONFIG"].reload_config()
if operation:
flash(operation)
# Reload instances
_reloads = app.config["INSTANCES"].reload_instances()
if not _reloads:
for _reload in _reloads:
flash(f"Reload failed for the instance {_reload}", "error")
else:
flash("Successfully reloaded instances")
app.config["RELOADING"] = True
Thread(
target=manage_bunkerweb,
name="Reloading instances",
args=("plugins",),
).start()
# Remove tmp folder
if os.path.exists("/opt/bunkerweb/tmp/ui"):
try:
rmtree("/opt/bunkerweb/tmp/ui")
@ -893,8 +944,11 @@ def plugins():
pass
app.config["CONFIG"].reload_plugins()
return redirect(url_for("loading", next=url_for("plugins")))
return redirect(
url_for("loading", next=url_for("plugins"), message="Reloading plugins")
)
# Initialize plugins tree
plugins = [
{
"name": "plugins",
@ -918,8 +972,47 @@ def plugins():
],
}
]
# Populate plugins tree
plugins_pages = app.config["CONFIG"].get_plugins_pages()
return render_template("plugins.html", folders=plugins)
pages = []
active = True
for page in plugins_pages:
with open(
f"/opt/bunkerweb/"
+ (
"plugins"
if os.path.exists(
f"/opt/bunkerweb/plugins/{page.lower()}/ui/template.html"
)
else "core"
)
+ f"/{page.lower()}/ui/template.html",
"r",
) as f:
# Convert the file content to a jinja2 template
template = Template(f.read())
pages.append(
{
"id": page.lower().replace(" ", "-"),
"name": page,
# Render the template with the plugin's data if it corresponds to the last submitted form else with the default data
"content": template.render(csrf_token=generate_csrf)
if app.config["PLUGIN_ARGS"] is None
or app.config["PLUGIN_ARGS"]["plugin"] != page.lower()
else template.render(
csrf_token=generate_csrf, **app.config["PLUGIN_ARGS"]["args"]
),
# Only the first plugin page is active
"active": active,
}
)
active = False
app.config["PLUGIN_ARGS"] = None
return render_template("plugins.html", folders=plugins, pages=pages)
@app.route("/plugins/upload", methods=["POST"])
@ -944,6 +1037,85 @@ def upload_plugin():
return {"status": "ok"}, 201
@app.route("/plugins/<plugin>", methods=["GET", "POST"])
@login_required
def custom_plugin(plugin):
if not re_match(r"^[a-zA-Z0-9_-]{1,64}$", plugin):
flash(
f"Invalid plugin id, <b>{plugin}</b> (must be between 1 and 64 characters, only letters, numbers, underscores and hyphens)",
"error",
)
return redirect(url_for("loading", next=url_for("plugins")))
if not os.path.exists(
f"/opt/bunkerweb/plugins/{plugin}/ui/actions.py"
) and not os.path.exists(f"/opt/bunkerweb/core/{plugin}/ui/actions.py"):
flash(
f"The <i>actions.py</i> file for the plugin <b>{plugin}</b> does not exist",
"error",
)
return redirect(url_for("loading", next=url_for("plugins")))
# Add the custom plugin to sys.path
sys_path.append(
f"/opt/bunkerweb/"
+ (
"plugins"
if os.path.exists(f"/opt/bunkerweb/plugins/{plugin}/ui/actions.py")
else "core"
)
+ f"/{plugin}/ui/"
)
try:
# Try to import the custom plugin
import actions
except:
flash(
f"An error occurred while importing the plugin <b>{plugin}</b>:<br/>{format_exc()}",
"error",
)
return redirect(url_for("loading", next=url_for("plugins")))
error = False
res = None
try:
# Try to get the custom plugin custom function and call it
method = getattr(actions, plugin)
res = method()
except AttributeError:
flash(
f"The plugin <b>{plugin}</b> does not have a <i>{plugin}</i> method",
"error",
)
error = True
return redirect(url_for("loading", next=url_for("plugins")))
except:
flash(
f"An error occurred while executing the plugin <b>{plugin}</b>:<br/>{format_exc()}",
"error",
)
error = True
finally:
# Remove the custom plugin from the shared library
sys_path.pop()
del sys_modules["actions"]
del actions
if (
request.method != "POST"
or error is True
or res is None
or isinstance(res, dict) is False
):
return redirect(url_for("loading", next=url_for("plugins")))
app.config["PLUGIN_ARGS"] = {"plugin": plugin, "args": res}
flash(f"Your action <b>{plugin}</b> has been executed")
return redirect(url_for("loading", next=url_for("plugins")))
@app.route("/cache", methods=["GET"])
@login_required
def cache():
@ -1212,6 +1384,21 @@ def login():
return render_template("login.html")
@app.route("/check_reloading")
@login_required
def check_reloading():
if app.config["RELOADING"] is False:
for f in app.config["TO_FLASH"]:
if f["type"] == "error":
flash(f["content"], "error")
else:
flash(f["content"])
app.config["TO_FLASH"].clear()
return jsonify({"reloading": app.config["RELOADING"]})
@app.route("/logout")
@login_required
def logout():

View File

@ -1,8 +1,8 @@
from copy import deepcopy
from os import listdir
from flask import flash
from operator import xor
from os.path import isfile
from typing import Tuple
from typing import List, Tuple
from json import load as json_load
from uuid import uuid4
from glob import iglob
@ -13,35 +13,48 @@ from subprocess import run, DEVNULL, STDOUT
class Config:
def __init__(self):
with open("/opt/bunkerweb/settings.json", "r") as f:
self.__settings = json_load(f)
self.__settings: dict = json_load(f)
self.__plugins = []
for filename in iglob("/opt/bunkerweb/core/**/plugin.json"):
with open(filename, "r") as f:
self.__plugins.append(json_load(f))
for filename in iglob("/opt/bunkerweb/plugins/**/plugin.json"):
with open(filename, "r") as f:
self.__plugins.append(json_load(f))
self.__plugins.sort(key=lambda plugin: plugin.get("name"))
self.__plugins_settings = {
**{k: v for x in self.__plugins for k, v in x["settings"].items()},
**self.__settings,
}
self.reload_plugins()
def reload_plugins(self) -> None:
self.__plugins.clear()
self.__plugins = []
self.__plugins_pages = []
for filename in iglob("/opt/bunkerweb/core/**/plugin.json"):
with open(filename, "r") as f:
self.__plugins.append(json_load(f))
for foldername in iglob("/opt/bunkerweb/plugins/*"):
content = listdir(foldername)
if "plugin.json" not in content:
continue
with open(f"{foldername}/plugin.json", "r") as f:
plugin = json_load(f)
self.__plugins.append(plugin)
if "ui" in content:
if "template.html" in listdir(f"{foldername}/ui"):
self.__plugins_pages.append(plugin["name"])
for foldername in iglob("/opt/bunkerweb/core/*"):
content = listdir(foldername)
if "plugin.json" not in content:
continue
with open(f"{foldername}/plugin.json", "r") as f:
plugin = json_load(f)
self.__plugins.append(plugin)
if "ui" in content:
if "template.html" in listdir(f"{foldername}/ui"):
self.__plugins_pages.append(plugin["name"])
for filename in iglob("/opt/bunkerweb/plugins/**/plugin.json"):
with open(filename, "r") as f:
self.__plugins.append(json_load(f))
self.__plugins.sort(key=lambda plugin: plugin.get("name"))
self.__plugins_pages.sort()
self.__plugins_settings = {
**{k: v for x in self.__plugins for k, v in x["settings"].items()},
**self.__settings,
@ -148,9 +161,12 @@ class Config:
def get_plugins_settings(self) -> dict:
return self.__plugins_settings
def get_plugins(self) -> dict:
def get_plugins(self) -> List[dict]:
return self.__plugins
def get_plugins_pages(self) -> List[str]:
return self.__plugins_pages
def get_settings(self) -> dict:
return self.__settings

View File

@ -1,5 +1,5 @@
import os
from typing import Any
from typing import Any, Union
from subprocess import run
from api.API import API
@ -126,17 +126,17 @@ class Instances:
return instances
def reload_instances(self) -> list[str]:
def reload_instances(self) -> Union[list[str], str]:
not_reloaded: list[str] = []
for instance in self.get_instances():
if instance.health is False:
not_reloaded.append(instance.name)
continue
if self.reload_instance(None, instance).startswith("Can't reload"):
if self.reload_instance(instance=instance).startswith("Can't reload"):
not_reloaded.append(instance.name)
return not_reloaded
return not_reloaded or "Successfully reloaded instances"
def reload_instance(self, id: int = None, instance: Instance = None) -> str:
if instance is None:

View File

@ -287,3 +287,41 @@
background: inherit;
color: inherit;
}
.plugins-pages .row .col-form-label,
.plugins-pages .row .col-8 {
background-color: transparent;
}
.nav-pills .nav-link.active,
.nav-pills .nav-link:hover {
background-color: rgba(64, 187, 107, 0.5) !important;
background-clip: border-box !important;
}
.plugins-pages .nav-link span {
color: #000 !important;
}
.plugins-pages aside {
background: none !important;
}
[data-theme="dark"] .plugins-pages,
[data-theme="dark"] .wrapper {
background-color: #222 !important;
}
[data-theme="dark"] .plugins-pages .bg-light {
background-color: rgba(var(--bs-dark-rgb), var(--bs-bg-opacity)) !important;
border-color: #111 !important;
border: 1px solid #343636 !important;
}
[data-theme="dark"] .plugins-pages .nav-link span {
color: #fff !important;
}
[data-theme="dark"] .nav-pills .nav-link.active {
background-color: #535353 !important;
}

View File

@ -3,13 +3,13 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Loading...</title>
<title>{{ message }}...</title>
<link rel="icon" href="data:image/svg+xml, %%3Csvg version='1.0' xmlns='http://www.w3.org/2000/svg' width='96.000000pt' height='96.000000pt' viewBox='0 0 96.000000 96.000000' preserveAspectRatio='xMidYMid meet'%%3E%%3Cg transform='translate(0.000000,96.000000) scale(0.100000,-0.100000)'%%0Afill='%%23085577' stroke='none'%%3E%%3Cpath d='M535 863 c-22 -2 -139 -17 -260 -34 -228 -31 -267 -43 -272 -85 -2%%0A-10 23 -181 55 -379 l57 -360 400 0 400 0 20 40 c16 31 20 59 19 125 -1 100%%0A-24 165 -73 199 -41 29 -46 57 -22 111 30 67 29 188 -3 256 -13 28 -37 60 -53%%0A72 -55 39 -169 62 -268 55z m-15 -348 c30 -16 60 -61 60 -90 0 -10 -8 -33 -17%%0A-52 -16 -34 -16 -41 0 -116 9 -44 15 -82 12 -85 -6 -7 -92 -21 -131 -21 l-31%%0A-1 -6 85 c-4 75 -8 89 -31 112 -20 20 -26 36 -26 70 0 38 5 50 34 79 39 39 86%%0A45 136 19z'/%%3E%%3C/g%%3E%%3C/svg%%3E" type="image/svg+xml"/>
<link rel="stylesheet" type="text/css" href="css/loading.css" />
</head>
<body>
<div class="cover">
<h1>Loading...</h1>
<h1>{{ message }}...</h1>
<p class="lead">
<div class="lds-roller"><div></div><div></div><div></div><div></div><div></div><div></div><div></div><div></div></div>
</p>
@ -23,6 +23,19 @@
</footer>
</body>
<script>
window.location.replace("{{ next }}");
check_reloading();
const reloading = setInterval(check_reloading, 1000);
async function check_reloading() {
const response = await fetch(`${location.href.replace("/loading", "/check_reloading")}`);
if (response.status === 200) {
res = await response.json();
if (res.reloading === false) {
clearInterval(reloading);
window.location.replace("{{ next }}");
}
}
}
</script>
</html>

View File

@ -38,6 +38,57 @@
</div>
</div>
</div>
{% if pages %}
<div class="container-fluid mt-3 mb-3 plugins-pages">
<div class="pb-3 flex-grow-1 d-flex flex-column flex-md-row">
<div class="row flex-grow-md-1 flex-grow-1">
<aside
class="col-md-3 flex-grow-md-1 flex-shrink-1 flex-grow-0 sticky-top pb-md-0 pb-3"
>
<div class="bg-light border rounded-3 p-1 h-100 sticky-top">
<ul
class="nav nav-pills flex-md-column flex-row mb-auto justify-content-start text-truncate"
id="pills-tab"
role="tablist"
>
{% for page in pages %}
<li class="nav-item">
<a
class="nav-link d-flex flex-row justify-content-between align-items-center px-2 text-truncate{{' active' if page['active']}}"
id="plugins-pages-{{ page['id'] }}-tab"
data-bs-toggle="pill"
href="#plugins-pages-{{ page['id'] }}"
role="tab"
aria-controls="plugins-pages-{{ page['id'] }}"
aria-selected=""
>
<span class="d-md-inline">{{ page["name"] }}</span>
</a>
</li>
{% endfor %}
</ul>
</div>
</aside>
<main class="col overflow-auto h-100">
<div class="bg-light border rounded-3 p-3">
<div class="tab-content" id="plugins-pages-content">
{% for page in pages %}
<div
class="tab-pane fade{{' show active' if page['active']}}"
id="plugins-pages-{{ page['id'] }}"
role="tabpanel"
aria-labelledby="plugins-pages-{{ page['id'] }}-tab"
>
{{ page["content"]|safe }}
</div>
{% endfor %}
</div>
</div>
</main>
</div>
</div>
</div>
{% endif %}
<div
class="modal fade"
id="modal-delete"