When you run LogDoctor for the first time, please take a minute to set-up the things it needs.<br/>
Head to the **configurations** section and give a look at least at the [logs format](#logs-format) settings. You have to tell the doctor what he'll be dealing with!
<br/><br/>
## Logs data
Archived (**gzipped**) log files can be used as well as normal files.
<br/>
### Storage
Parsed data will be stored in an [SQLite](https://www.sqlite.org/about.html) database, which makes it easy to transport/view/edit it as you please.<br/>
If LogDoctor's funcionalities aren't enough for your needs, you can always use a *DB manager* or the SQLite *API* to make your own queries and retrieve the data you need.
<br/>
### Examined fields
Not all the available log fields (expecially for *Apache2* and *Nginx*) are taken into consideration.<br/>
The considered fields are:
- **Date** and **Time**
- Request stuff: **Protocol**, **Method**, **URI** and **Query**
- Server stuff: **Bytes received**, **Bytes sent** and **Time taken**
- Client stuff: **User-agent**, **IP address**, **Cookie** and **Referrer site**
Further informations can be found in the [wiki](https://git.disroot.org/elB4RTO/LogDoctor/wiki/Examined-fields) or while running LogDoctor.
<br/>
### Logs options
Various options can be configured about log files.
<br/>
#### Usage control
When you parse a file, it will be hashed using the **SHA256** algorithm and the hash will be stored in another database, to keep track of which files you've already used and help you not parsing them twice.<br/>
##### Note
If you don't know, *SHA256* produces an irreversible hash, which means that no information about the file can be retrieved from the hash.<br/>
You have full control on the hashes database (same as for the logs-data database): you can move, delete, view or edit it the way you want (but you must keep the original file-name).<br/>
LogDoctor will **never** grab and/or use any information about you or the usage you make of it.
<br/>
#### Logs path
A different logs path can be used for any of the three supported *Web Servers*.<br/>
It can be the default system folder or any folder you decide to use, just set it in the options.
<br/>
#### Logs format
Before to start parsing logs, you must set-up the *log format* that LogDoctor will have to use.<br/>
Head to the **configurations** section, tap `Logs`, select the **Web Server** you want to configure and tap `Format`.<br/>
Once inside the **Format** section, you can insert the *log format string* you're using. Don't forget to use the `Generete preview` button to generate a *log line sample* and **check the correctness** of the format!<br/>
For reliability reasons, LogDoctor **does not** support the usage of the **Carriage Return** inside the log format string.
<br/>
##### Apache2
The log format string must be specified. Any format is supported, if valid.<br/>
To retrieve your format string:
- open the configuration file `/etc/apache2/apache2.conf`
- *usually*, the line you're looking for is the one starting with `LogFormat` and ending with `combined`. It should be somewhere near to the end of the file.
- you must not paste the whole line, just the part holding the *format string*.<br/>
please notice that you have to remove the enclosing quotes/apostrophes as well<br/>
More informations can be found in the [wiki](https://git.disroot.org/elB4RTO/LogDoctor/wiki/Apache2) or while setting the format.
<br/>
##### Nginx
The log format string must be specified. Any format is supported, if valid.<br/>
To retrieve your format string:<br/>
- open the configuration file `/usr/local/etc/nginx/nginx.conf`
- *usually*, the line you're looking for is the one starting with `log_format main`. It should be somwehere in the middle of the file
- one **important** thing: don't paste the indentations and new lines! The default line is usualy declared in consecutive lines, and indented. You must reduce it to a one consecutive string (by also removing the *apostrophes* in the middle of it). The best way is to do this job inside the configuration file, then save and restart Nginx to see if any error is thrown.<br/>
Example:
- this is the whole line:<br/>
```
log_format main '$remote_addr - $remote_user [$time_local] '
please notice that you have to remove the enclosing apostrophes/quotes as well<br/>
More informations can be found in the [wiki](https://git.disroot.org/elB4RTO/LogDoctor/wiki/Nginx) or while setting the format.
<br/>
##### IIS
Supported log formats are: **W3C**, **NCSA** and **IIS**.<br/>
The *NCSA* and *IIS* modules doesn't allow any modification from the user, so nothing more have to be specified.
The *W3C* module instead allows the user to decide which fields to log, and thus you must declare the *log format string* you're using.
To retrieve your format string (for the *W3C* module only):
- open any of the log files which have been generated by this module
- the line you're looking for is the one starting with `#Fields:`, usually at the beginning of the file.<br/>
Example:<br/>
- this is the whole line:<br/>
```
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken
```
- this is the *format string*:<br/>
```
date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken
```
More informations can be found in the [wiki](https://git.disroot.org/elB4RTO/LogDoctor/wiki/IIS) or while setting the format.
<br/><br/>
#### Blacklist
You can add elements to the **blacklist** to avoid storing the lines containing those elements.
Each web server has its own list.
<br/>
#### Warnlist
As for the *blacklist*, you can add elements to the **warnlist**.<br/>
*Warnlists* will mark with a **warning** the lines triggering them. Warnings can be viewed and modified in the relative [statistics](#warnings) section.
Each web server has its own lists.
<br/>
## Statistics
Most of the *statistics sections* allows you to set filters to the log fields, to skim data by only including lines matching those parameters.<br/>
<br/>
### Warnings
In the *warning* section you can view the lines which triggered a warning, as well as remove any of the warnings and/or add your own.<br/>
LogDoctor can automatically do a backup of your **logs database** file, so you can recover your data in case something goes wrong.<br/>
Move inside LogDoctor's folder (if you don't know/remember the path, open the `Utilities`→`Infos`>`Paths` menu to view it) and open the folder named "**backups**'.<br/>
Here you will find the backups with an increasing index, where '.1' represents the newest.
A new backup is made every time you quit LogDoctor after doing a job which affected the database in any way.
#### Note
Only the *logs-data database* will be backed-up, the *hashes database***won't**.<br/>
This is because it is unlikely (supposedly impossible) that a hash equals another, therefore they're supposed to be useful for a short period of time (that is, since you or your web server delete the original log files).
<br/>
### Estimated working speed
1~15 MB/s
May be higher or lower depending on the complexity of the logs, the complexity of the blacklist/warnlists, your hardware and the workload of your system during the execution.
If you have suggestions about how to improve it, please open an ![issue](https://git.disroot.org/elB4RTO/LogDoctor/issues).
If you want to contribute to the code, please read the [Contribution Guidelines](https://git.disroot.org/elB4RTO/LogDoctor/src/branch/main/CONTRIBUTING.md).
If you want to contribute to the translation, please read the [Translation Guidelines](#translation-guidelines).
If you have a request for a missing language or you're willing to contribute, please refer to [this issue](https://git.disroot.org/elB4RTO/LogDoctor/issues/10).
Since the whole application is build upon Qt, translations are made throught `.ts` [translation files](https://git.disroot.org/elB4RTO/LogDoctor/src/branch/main/logdoctor/translations).
<br/>
The easiest way to go is to use **Qt Linguist**:
- Download or clone this repo<br/>
- Open the `.ts` translation file of your language using QtLinguist<br/>
- **Push only the `.ts` files, you don't have to release them**: pull request containing `.qm` binaries won't be accepted.<br/>
<br/><br/>
If you don't want to install QtLinguist, you can do it the hard way, by opening the files with a text editor and do it manually:
- Here is a sample of some text waiting for translation<br/>
You can see the original text enclosed in the `<source>` tags. Don't edit it.<br/>
```
<message>
<locationfilename="..."line="n"/>
<source>This is the original text</source>
</message>
```
- Everything you need to do is to add the missing line containing the translated text, without editing the other lines.<br/>
Here is a sample after having translated it<br/>
```
<message>
<locationfilename="..."line="n"/>
<source>Original text</source>
<translationtype="unfinished">Translated text goes here</translation>
</message>
```
<br/>
If you don't feel comfortable with any of the above solutions, please open an ![issue](https://git.disroot.org/elB4RTO/LogDoctor/issues) and write your translations/corrections there, in a clear way.
<br/>
#### Translation guidelines
*[hints from Qt](https://doc.qt.io/qt-6/linguist-overview.html)*
Just follow some simple guidelines to ensure a correct and clear contribution:
- Respect the structure of the phrase: if *(for example)* it starts or ends with a whitespace or something, please do the same.
- Please leave your translations marked as *unfinished*, they will be checked and un-marked after having been verified.
- If you see a translation marked as *finished*, please edit it only if you're sure that your definition suits better.