From bee43e234fb7e75aec0b8444decff0cd77251f96 Mon Sep 17 00:00:00 2001
From: Hector <dev@hsmith.org>
Date: Tue, 31 Aug 2021 06:03:29 +0000
Subject: [PATCH 1/4] first pass at updating readme

---
 README.md | 47 ++++++++++++++++++++++++++++++++---------------
 1 file changed, 32 insertions(+), 15 deletions(-)

diff --git a/README.md b/README.md
index 4873284..60398e7 100644
--- a/README.md
+++ b/README.md
@@ -12,25 +12,26 @@ Go tool to collect and export metrics on Fail2Ban
 4. Metrics
 
 ## 1. How to use
+The exporter can collect metrics from 2 locations: the fail2ban server socket, and the fail2ban server database.
 
-Run the exporter by providing it with a fail2ban database to read data from.
-Read access to the database is required.
 Once the exporter is running, metrics are available at `localhost:9191/metrics`.
-
 The default port is `9191`, but this can be modified with the `-port` flag.
 
-**Note:** By default fail2ban stores the database file at: `/var/lib/fail2ban/fail2ban.sqlite3`
+### 1.1. Socket
+The recommended way to run the exporter is to point it at the fail2ban server socket.
+This allows the exporter to communicate with the server in the same way `fail2ban-client` does and ensures the metrics it collects are exactly the same as the values reported by `fail2ban-client status <jail>`.
 
-**Fail2Ban Jails**
+The default path to the socket is: `/var/run/fail2ban/fail2ban.sock`
 
-fail2ban can be configured to process different log files and use different rules for each one.
-These separate configurations are referred to as *jails*.
+### 1.2. Deprecated: Database
+Reading metrics from the database has been deprecated in favour of using the socket.
+The reason being that database metrics do not always align with the output of `fail2ban-client status <jail>` and cause confusion.
+See [#11]() for more details.
 
-For example, fail2ban can be configured to watch the system logs for failed SSH connections and Nextcloud logs for failed logins.
-In this configuration, there will be two jails - one for IPs banned from the SSH logs, and one for IPs banned from the Nextcloud logs.
+*To run the exporter in this mode:*
 
-This tool exports several metrics *per jail*, meaning that it is possible to track how many IPs are being banned in each jail as well as the overall total.
-This can be useful to track what services are seeing more failed logins.
+Run the exporter with the path to the fail2ban database.
+The default path to the database is: `/var/lib/fail2ban/fail2ban.sqlite3`
 
 ## 2. Docker
 
@@ -46,10 +47,11 @@ See the [registry page](https://gitlab.com/hectorjsmith/fail2ban-prometheus-expo
 
 ### 2.1. Volumes
 
-The docker image is designed to run by mounting the fail2ban sqlite3 database.
-The database should be mounted at: `/app/fail2ban.sqlite3`
+The docker image is designed to run by mounting either the fail2ban sqlite3 database of the fail2ban run folder.
+- The database should be mounted at: `/app/fail2ban.sqlite3`
+- The run folder should be mounted at: `/var/run/fail2ban`
 
-The database can be mounted with read-only permissions.
+Both paths can be mounted with readong (`ro`) permissions.
 
 ### 2.2. Docker run
 
@@ -59,6 +61,7 @@ Use the following command to run the forwarder as a docker container.
 docker run -d \
     --name "fail2ban-exporter" \
     -v /var/lib/fail2ban/fail2ban.sqlite3:/app/fail2ban.sqlite3:ro \
+    -v /var/run/fail2ban:/var/run/fail2ban:ro \
     -p "9191:9191"
     registry.gitlab.com/hectorjsmith/fail2ban-prometheus-exporter:latest
 ```
@@ -74,6 +77,7 @@ services:
     image: registry.gitlab.com/hectorjsmith/fail2ban-prometheus-exporter:latest
     volumes:
     - /var/lib/fail2ban/fail2ban.sqlite3:/app/fail2ban.sqlite3:ro
+    - /var/run/fail2ban/:/var/run/fail2ban:ro
     ports:
     - "9191:9191"
 ```
@@ -95,7 +99,20 @@ $ fail2ban-prometheus-exporter -h
 
 Access exported metrics at `/metrics` (on the provided port).
 
-**Note:** All metric names include the `fail2ban_` prefix to make sure they are unique and easier to find.
+**Note: Fail2Ban Jails**
+
+fail2ban can be configured to process different log files and use different rules for each one.
+These separate configurations are referred to as *jails*.
+
+For example, fail2ban can be configured to watch the system logs for failed SSH connections and Nextcloud logs for failed logins.
+In this configuration, there will be two jails - one for IPs banned from the SSH logs, and one for IPs banned from the Nextcloud logs.
+
+This tool exports several metrics *per jail*, meaning that it is possible to track how many IPs are being banned in each jail as well as the overall total.
+This can be useful to track what services are seeing more failed logins.
+
+### 4.1. Socket Metrics
+
+### 4.2. Database Metrics
 
 Exposed metrics:
 * `up` - Returns 1 if the service is up

From 9dee6d2ae8fd8c6d98460a8c67d2a94170966183 Mon Sep 17 00:00:00 2001
From: Hector <dev@hsmith.org>
Date: Tue, 31 Aug 2021 10:04:52 +0000
Subject: [PATCH 2/4] second pass at updating readme

---
 README.md | 72 +++++++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 65 insertions(+), 7 deletions(-)

diff --git a/README.md b/README.md
index 60398e7..83d4521 100644
--- a/README.md
+++ b/README.md
@@ -5,9 +5,6 @@ Go tool to collect and export metrics on Fail2Ban
 ## Table of Contents
 1. How to use
 2. Docker
-    1. Volumes
-    2. Docker run
-    3. Docker compose
 3. CLI usage
 4. Metrics
 
@@ -51,7 +48,10 @@ The docker image is designed to run by mounting either the fail2ban sqlite3 data
 - The database should be mounted at: `/app/fail2ban.sqlite3`
 - The run folder should be mounted at: `/var/run/fail2ban`
 
-Both paths can be mounted with readong (`ro`) permissions.
+Both paths can be mounted with readonly (`ro`) permissions.
+
+**NOTE:** While it is possible to mount the `fail2ban.sock` file directly, I recommend mounting the parent folder instead.
+The `.sock` file is deleted by fail2ban on shutdown and then re-created on startup and this causes problems for the docker mount.
 
 ### 2.2. Docker run
 
@@ -88,9 +88,11 @@ services:
 $ fail2ban-prometheus-exporter -h
 
   -db string
-        path to the fail2ban sqlite database
+        path to the fail2ban sqlite database (deprecated)
   -port int
         port to use for the metrics server (default 9191)
+  -socket string
+        path to the fail2ban server socket
   -version
         show version info and exit
 ```
@@ -99,7 +101,7 @@ $ fail2ban-prometheus-exporter -h
 
 Access exported metrics at `/metrics` (on the provided port).
 
-**Note: Fail2Ban Jails**
+**Fail2Ban Jails**
 
 fail2ban can be configured to process different log files and use different rules for each one.
 These separate configurations are referred to as *jails*.
@@ -112,7 +114,63 @@ This can be useful to track what services are seeing more failed logins.
 
 ### 4.1. Socket Metrics
 
-### 4.2. Database Metrics
+Exposed metrics:
+* `up` - Returns 1 if the fail2ban server is up and connection succeeds
+* `errors` - Number of errors since startup
+    * `db` - Errors connecting to the database
+    * `socket_conn` - Errors connecting to the fail2ban socket (e.g. connection refused)
+    * `socket_req` - Errors sending requests to the fail2ban server (e.g. invalid responses)
+* `jail_count` - Number of jails configured in fail2ban
+* `jail_banned_current` (per jail) - Number of IPs currently banned
+* `jail_banned_total` (per jail) - Total number of banned IPs since fail2ban startup (includes expired bans)
+* `jail_failed_current` (per jail) - Number of current failures
+* `jail_failed_total` (per jail) - Total number of failures since fail2ban startup
+
+**Sample**
+
+```
+# HELP f2b_errors Number of errors found since startup
+# TYPE f2b_errors counter
+f2b_errors{type="db"} 0
+f2b_errors{type="socket_conn"} 0
+f2b_errors{type="socket_req"} 0
+# HELP f2b_jail_banned_current Number of IPs currently banned in this jail
+# TYPE f2b_jail_banned_current gauge
+f2b_jail_banned_current{jail="recidive"} 5
+f2b_jail_banned_current{jail="sshd"} 15
+# HELP f2b_jail_banned_total Total number of IPs banned by this jail (includes expired bans)
+# TYPE f2b_jail_banned_total gauge
+f2b_jail_banned_total{jail="recidive"} 6
+f2b_jail_banned_total{jail="sshd"} 31
+# HELP f2b_jail_count Number of defined jails
+# TYPE f2b_jail_count gauge
+f2b_jail_count 2
+# HELP f2b_jail_failed_current Number of current failures on this jail's filter
+# TYPE f2b_jail_failed_current gauge
+f2b_jail_failed_current{jail="recidive"} 5
+f2b_jail_failed_current{jail="sshd"} 6
+# HELP f2b_jail_failed_total Number of total failures on this jail's filter
+# TYPE f2b_jail_failed_total gauge
+f2b_jail_failed_total{jail="recidive"} 7
+f2b_jail_failed_total{jail="sshd"} 125
+# HELP f2b_up Check if the fail2ban server is up
+# TYPE f2b_up gauge
+f2b_up 1
+```
+
+The metrics above correspond to the matching fields in the `fail2ban-client status <jail>` command:
+```
+Status for the jail: sshd|- Filter
+|  |- Currently failed: 6
+|  |- Total failed:     125
+|  `- File list:        /var/log/auth.log
+`- Actions
+   |- Currently banned: 15
+   |- Total banned:     31
+   `- Banned IP list:   ...
+```
+
+### 4.2. Database Metrics (deprecated)
 
 Exposed metrics:
 * `up` - Returns 1 if the service is up

From 50393257144dd0c25aed6a7e13782fd7eebf99ba Mon Sep 17 00:00:00 2001
From: Hector <dev@hsmith.org>
Date: Tue, 31 Aug 2021 10:40:52 +0000
Subject: [PATCH 3/4] update readme content

---
 README.md | 135 ++++++++++++++++++++++++++++++++++--------------------
 1 file changed, 85 insertions(+), 50 deletions(-)

diff --git a/README.md b/README.md
index 83d4521..d360996 100644
--- a/README.md
+++ b/README.md
@@ -3,36 +3,75 @@
 Go tool to collect and export metrics on Fail2Ban
 
 ## Table of Contents
-1. How to use
-2. Docker
-3. CLI usage
+1. Introduction
+2. Running the Exporter
+3. Running in Docker
 4. Metrics
 
-## 1. How to use
-The exporter can collect metrics from 2 locations: the fail2ban server socket, and the fail2ban server database.
+## 1. Introduction
+The exporter can collect metrics from 2 locations: the fail2ban server socket and the fail2ban server database.
 
 Once the exporter is running, metrics are available at `localhost:9191/metrics`.
-The default port is `9191`, but this can be modified with the `-port` flag.
+
+(The default port is `9191` but can be modified with the `-port` flag)
 
 ### 1.1. Socket
 The recommended way to run the exporter is to point it at the fail2ban server socket.
-This allows the exporter to communicate with the server in the same way `fail2ban-client` does and ensures the metrics it collects are exactly the same as the values reported by `fail2ban-client status <jail>`.
+This allows the exporter to communicate with the server the same way `fail2ban-client` does and ensures the metrics it collects align with the values reported by `fail2ban-client status <jail>`.
 
 The default path to the socket is: `/var/run/fail2ban/fail2ban.sock`
 
 ### 1.2. Deprecated: Database
-Reading metrics from the database has been deprecated in favour of using the socket.
+The original way to collect metrics is to read them from the fail2ban database.
+This has now been deprecated in favour of using the socket.
 The reason being that database metrics do not always align with the output of `fail2ban-client status <jail>` and cause confusion.
-See [#11]() for more details.
+See [#11](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/-/issues/11) for more details.
 
-*To run the exporter in this mode:*
+If necessary, these metrics can still be exported by providing the database path to the exporter.
 
-Run the exporter with the path to the fail2ban database.
-The default path to the database is: `/var/lib/fail2ban/fail2ban.sqlite3`
+The default path to the fail2ban database is: `/var/lib/fail2ban/fail2ban.sqlite3`
 
-## 2. Docker
+## 2. Running the Exporter
 
-An official docker image is available on the Gitlab container registry.
+The exporter is compiled and released as a single binary.
+This makes it very easy to run in any environment.
+No additional runtime dependencies are required.
+
+Compiled binaries for various platforms are provided in each release.
+See the [releases page](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/-/releases) for more information.
+
+**Usage**
+```
+$ fail2ban-prometheus-exporter -h
+
+  -db string
+        path to the fail2ban sqlite database (deprecated)
+  -port int
+        port to use for the metrics server (default 9191)
+  -socket string
+        path to the fail2ban server socket
+  -version
+        show version info and exit
+```
+
+**Example**
+
+```
+fail2ban-prometheus-exporter -socket /var/run/fail2ban/fail2ban.sock -port 9191
+```
+
+Note that the exporter will need read access to the fail2ban socket or database.
+
+### 2.1. Compile from Source
+
+The code can be compiled from source by running `go build` inside the `src/` folder.
+Go version `1.15` or greater is required.
+
+Run `go mod download` to download all necessary dependencies before running the build.
+
+## 3. Running in Docker
+
+If use of docker is desired, an official docker image is available on the Gitlab container registry.
 Use it by pulling the following image:
 
 ```
@@ -42,7 +81,7 @@ registry.gitlab.com/hectorjsmith/fail2ban-prometheus-exporter:latest
 Use the `:latest` tag to get the most up to date code (less stable) or use one of the version tagged images to use a specific release.
 See the [registry page](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/container_registry) for all available tags.
 
-### 2.1. Volumes
+### 3.1. Volumes
 
 The docker image is designed to run by mounting either the fail2ban sqlite3 database of the fail2ban run folder.
 - The database should be mounted at: `/app/fail2ban.sqlite3`
@@ -52,21 +91,22 @@ Both paths can be mounted with readonly (`ro`) permissions.
 
 **NOTE:** While it is possible to mount the `fail2ban.sock` file directly, I recommend mounting the parent folder instead.
 The `.sock` file is deleted by fail2ban on shutdown and then re-created on startup and this causes problems for the docker mount.
+See [this reply](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/-/issues/11#note_665003499) for more details.
 
-### 2.2. Docker run
+### 3.2. Docker run
 
-Use the following command to run the forwarder as a docker container.
+Use the following command to run the exporter as a docker container.
 
 ```
 docker run -d \
     --name "fail2ban-exporter" \
     -v /var/lib/fail2ban/fail2ban.sqlite3:/app/fail2ban.sqlite3:ro \
     -v /var/run/fail2ban:/var/run/fail2ban:ro \
-    -p "9191:9191"
+    -p "9191:9191" \
     registry.gitlab.com/hectorjsmith/fail2ban-prometheus-exporter:latest
 ```
 
-### 2.3. Docker compose
+### 3.3. Docker compose
 
 The following is a simple docker-compose file to run the exporter.
 
@@ -82,26 +122,11 @@ services:
     - "9191:9191"
 ```
 
-## 3. CLI usage
-
-```
-$ fail2ban-prometheus-exporter -h
-
-  -db string
-        path to the fail2ban sqlite database (deprecated)
-  -port int
-        port to use for the metrics server (default 9191)
-  -socket string
-        path to the fail2ban server socket
-  -version
-        show version info and exit
-```
-
 ## 4. Metrics
 
-Access exported metrics at `/metrics` (on the provided port).
+Access exported metrics at the `/metrics` path on the configured port.
 
-**Fail2Ban Jails**
+**Note on Fail2Ban Jails**
 
 fail2ban can be configured to process different log files and use different rules for each one.
 These separate configurations are referred to as *jails*.
@@ -112,7 +137,10 @@ In this configuration, there will be two jails - one for IPs banned from the SSH
 This tool exports several metrics *per jail*, meaning that it is possible to track how many IPs are being banned in each jail as well as the overall total.
 This can be useful to track what services are seeing more failed logins.
 
-### 4.1. Socket Metrics
+### 4.1. Socket-based Metrics
+
+These are the metrics exported by reading data from the fail2ban server socket.
+All metrics are prefixed with `f2b_`.
 
 Exposed metrics:
 * `up` - Returns 1 if the fail2ban server is up and connection succeeds
@@ -172,6 +200,13 @@ Status for the jail: sshd|- Filter
 
 ### 4.2. Database Metrics (deprecated)
 
+These are the original metrics exported by the initial release of the exporter.
+They are all based on the data stored in the fail2ban sqlite3 database.
+
+*These metrics are deprecated and will be removed in a future release.*
+
+All metrics are prefixed with `fail2ban_`.
+
 Exposed metrics:
 * `up` - Returns 1 if the service is up
 * `errors` - Returns the number of errors found since startup
@@ -186,22 +221,22 @@ Exposed metrics:
 **Sample**
 
 ```
-# HELP fail2ban_bad_ips Number of bad IPs stored in the database (per jail).
+# HELP fail2ban_bad_ips (Deprecated) Number of bad IPs stored in the database (per jail).
 # TYPE fail2ban_bad_ips gauge
-fail2ban_bad_ips{jail="jail1"} 6
-fail2ban_bad_ips{jail="jail2"} 8
-# HELP fail2ban_banned_ips Number of banned IPs stored in the database (per jail).
+fail2ban_bad_ips{jail="recidive"} 0
+fail2ban_bad_ips{jail="sshd"} 0
+# HELP fail2ban_banned_ips (Deprecated) Number of banned IPs stored in the database (per jail).
 # TYPE fail2ban_banned_ips gauge
-fail2ban_banned_ips{jail="jail1"} 3
-fail2ban_banned_ips{jail="jail2"} 2
-# HELP fail2ban_up Was the last fail2ban query successful.
-# TYPE fail2ban_up gauge
-fail2ban_up 1
-# HELP fail2ban_enabled_jails Enabled jails.
+fail2ban_banned_ips{jail="recidive"} 0
+fail2ban_banned_ips{jail="sshd"} 0
+# HELP fail2ban_enabled_jails (Deprecated) Enabled jails.
 # TYPE fail2ban_enabled_jails gauge
-fail2ban_enabled_jails{jail="jail1"} 1
-fail2ban_enabled_jails{jail="jail2"} 1
-# HELP fail2ban_errors Number of errors found since startup.
+fail2ban_enabled_jails{jail="recidive"} 1
+fail2ban_enabled_jails{jail="sshd"} 1
+# HELP fail2ban_errors (Deprecated) Number of errors found since startup.
 # TYPE fail2ban_errors counter
 fail2ban_errors{type="db"} 0
+# HELP fail2ban_up (Deprecated) Was the last fail2ban query successful.
+# TYPE fail2ban_up gauge
+fail2ban_up 1
 ```

From 2fa9f5bcae5c4230ab5f158332963ff7b7c5aa94 Mon Sep 17 00:00:00 2001
From: Hector <dev@hsmith.org>
Date: Tue, 31 Aug 2021 12:38:50 +0000
Subject: [PATCH 4/4] tweak readme wording

---
 README.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/README.md b/README.md
index d360996..8c0c592 100644
--- a/README.md
+++ b/README.md
@@ -89,8 +89,8 @@ The docker image is designed to run by mounting either the fail2ban sqlite3 data
 
 Both paths can be mounted with readonly (`ro`) permissions.
 
-**NOTE:** While it is possible to mount the `fail2ban.sock` file directly, I recommend mounting the parent folder instead.
-The `.sock` file is deleted by fail2ban on shutdown and then re-created on startup and this causes problems for the docker mount.
+**NOTE:** While it is possible to mount the `fail2ban.sock` file directly, it is recommended to mount the parent folder instead.
+The `.sock` file is deleted by fail2ban on shutdown and re-created on startup and this causes problems for the docker mount.
 See [this reply](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/-/issues/11#note_665003499) for more details.
 
 ### 3.2. Docker run