Added Docs

pylog2html/README.md

@@ -0,0 +1,59 @@
+
+# Visualizing NGINX Logs With Plotly
+The code has two portions:
+* A) NGINX outputting logs in a json format
+* B) The docker/standalone python script
+
+## Setting up - NGINX
+The only difference to NGINX is to have a custom logging format
+
+For the nginx config file
+```
+
+http {
+
+...
+
+
+    log_format logger-json escape=json '{"source": "nginx", "time": $msec, "resp_body_size": $body_bytes_sent, "host": "$http_host", "address": "$remote_addr", "request_length": $request_length, "method": "$request_method", "uri": "$request_uri", "status": $status,  "user_agent": "$http_user_agent", "resp_time": $request_time, "upstream_addr": "$upstream_addr"}';
+    access_log /config/log/nginx/access.log logger-json;
+
+
+...
+
+}
+```
+
+
+## Setting up the docker container
+
+```
+git clone <this repo>
+cd ./docker-containers/pylog2html
+docker-compose up --force-recreate --build --remove-orphans -d
+```
+
+
+Side note:
+> By default, it uses 'access.log' as the main file to build the map off of, hence you'll need to copy that file from the nginx server.
+
+
+### View the logs
+```
+docker logs <pylog2html name>
+```
+
+
+
+## Periodic creation
+### Add a cron job
+```
+crontab -e
+```
+
+Add the following:
+```
+*/29 * * * * path/to/updater.sh
+*/32 * * * * path/to/updater.sh "upload"
+```
+

pylog2html/updater.sh

@@ -0,0 +1,29 @@
+#!/bin/bash
+
+NGINX_SERVER_HOSTNAME=""
+NGINX_LOGS_LOCATION=""
+NGINX_SAVE_LOCATION=""
+PLOT_SAVE_LOCATION=""
+PLOT_UPLOAD_LOCATION=""
+
+
+## Example values
+#NGINX_SERVER_HOSTNAME="HS-VPS-01"
+#NGINX_LOGS_LOCATION="docker/000_Rev-Pro/Data/log/nginx/access.log"
+#NGINX_SAVE_LOCATION="/home/user/analytics/11_logs2html/Data/"
+#PLOT_SAVE_LOCATION="/home/user/analytics/11_logs2html/Data/plot.html"
+#PLOT_UPLOAD_LOCATION="/home/user/docker/gohugo/Data/static/post/"
+
+
+
+
+if [[ -z NGINX_SERVER_HOSTNAME ]] || [[ -z "$NGINX_LOGS_LOCATION" ]] || [[ -z "$NGINX_SAVE_LOCATION" ]] || [[ -z "$PLOT_SAVE_LOCATION" ]] || [[ -z "$PLOT_UPLOAD_LOCATION" ]] ; then
+	echo "ERROR! Change default values!"
+	exit
+fi
+
+if [[ "$1" != "upload" ]]; then
+	scp "$NGINX_SERVER_HOSTNAME":"$NGINX_LOGS_LOCATION" "$NGINX_SAVE_LOCATION"
+else
+	scp "$PLOT_SAVE_LOCATION" "$NGINX_SERVER_HOSTNAME":"$PLOT_UPLOAD_LOCATION"
+fi