From 2bffd3b9d44c7d0340791c79971afca5c321d991 Mon Sep 17 00:00:00 2001 From: Hector <hector@hjs.dev> Date: Sun, 20 Feb 2022 10:17:36 +0000 Subject: [PATCH 1/8] start rewrite of readme file & move grafana.json --- README.md | 153 ++++++++++-------- .../grafana/dashboard.json | 0 2 files changed, 90 insertions(+), 63 deletions(-) rename grafana.json => examples/grafana/dashboard.json (100%) diff --git a/README.md b/README.md index 2795f9b..88ab315 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,96 @@ # Fail2Ban Prometheus Exporter -Go tool to collect and export metrics on Fail2Ban +Collect metrics from a running fail2ban instance. ## Table of Contents -1. Introduction -2. Running the Exporter -3. Running in Docker -4. Metrics +1. Quick Start +2. Metrics + +3. Running the Exporter +4. Running in Docker +5. Metrics + +## 1. Quick Start + +The exporter can be run as a standalone binary or a docker container. + +### 1.1. Standalone + +The following command will start collecting metrics from the `/var/run/fail2ban/fail2ban.sock` file and expose them on port `9191`. + +``` +$ fail2ban_exporter --collector.f2b.socket=/var/run/fail2ban/fail2ban.sock --web.listen-address=":9191" + +2022/02/20 09:54:06 fail2ban exporter version 0.5.0 +2022/02/20 09:54:06 starting server at :9191 +2022/02/20 09:54:06 reading metrics from fail2ban socket: /var/run/fail2ban/fail2ban.sock +2022/02/20 09:54:06 metrics available at '/metrics' +2022/02/20 09:54:06 ready +``` + +### 1.2. Docker + +**Docker run** +``` +docker run -d \ + --name "fail2ban-exporter" \ + -v /var/run/fail2ban:/var/run/fail2ban:ro \ + -p "9191:9191" \ + registry.gitlab.com/hectorjsmith/fail2ban-prometheus-exporter:latest +``` + +**Docker compose** + +``` +version: "2" +services: + exporter: + image: registry.gitlab.com/hectorjsmith/fail2ban-prometheus-exporter:latest + volumes: + - /var/run/fail2ban/:/var/run/fail2ban:ro + ports: + - "9191:9191" +``` + +## 2. Metrics + +The exporter exposes the following metrics: + +*All metric names are prefixed with `f2b_`* + +| Metric | Description | Example | +|------------------------------|------------------------------------------------------------------------------------|-----------------------------------------------------| +| `up` | Returns 1 if the exporter is up and running | `f2b_up 1` | +| `errors` | Count the number of errors since startup by type | | +| `errors{type="socket_conn"}` | Errors connecting to the fail2ban socket (e.g. connection refused) | `f2b_errors{type="socket_conn"} 0` | +| `errors{type="socket_req"}` | Errors sending requests to the fail2ban server (e.g. invalid responses) | `f2b_errors{type="socket_req"} 0` | +| `jail_count` | Number of jails configured in fail2ban | `f2b_jail_count 2` | +| `jail_banned_current` | Number of IPs currently banned per jail | `f2b_jail_banned_current{jail="sshd"} 15` | +| `jail_banned_total` | Total number of banned IPs since fail2ban startup per jail (includes expired bans) | `f2b_jail_banned_total{jail="sshd"} 31` | +| `jail_failed_current` | Number of current failures per jail | `f2b_jail_failed_current{jail="sshd"} 6` | +| `jail_failed_total` | Total number of failures since fail2ban startup per jail | `f2b_jail_failed_total{jail="sshd"} 125` | +| `jail_config_ban_time` | How long an IP is banned for in this jail (in seconds) | `f2b_config_jail_ban_time{jail="sshd"} 600` | +| `jail_config_find_time` | How far back the filter will look for failures in this jail (in seconds) | `f2b_config_jail_find_time{jail="sshd"} 600` | +| `jail_config_max_retry` | The max number of failures allowed before banning an IP in this jail | `f2b_config_jail_max_retries{jail="sshd"} 5` | +| `version` | Version string of the exporter and fail2ban | `f2b_version{exporter="0.5.0",fail2ban="0.11.1"} 1` | + +### 2.1. Grafana + +The metrics exported by this tool are compatible with Prometheus and Grafana. +A sample grafana dashboard can be found in the [grafana.json](/examples/grafana/dashboard.json) file. +Just import the contents of this file into a new Grafana dashboard to get started. + +*(Sample dashboard is compatible with Grafana `8.3.3` and above)* + + + + +--- + + + + + ## 1. Introduction This exporter collects metrics from a running fail2ban instance. @@ -20,11 +104,6 @@ This allows the data collected by the exporter to always align with the output o The default location of the socket is: `/var/run/fail2ban/fail2ban.sock` -## 1.1. Grafana - -The metrics exported by this tool are compatible with Prometheus and Grafana. A sample grafana dashboard can be found in the `grafana.json` file. Just import the contents of this file into a new Grafana dashboard to get started. - -*(Sample dashboard is compatible with Grafana `8.3.3` and above)* ## 2. Running the Exporter @@ -37,7 +116,7 @@ See the [releases page](https://gitlab.com/hectorjsmith/fail2ban-prometheus-expo **CLI Usage** ``` -$ fail2ban-prometheus-exporter -h +$ fail2ban_exporter -h usage: fail2ban-prometheus-exporter [<flags>] Flags: @@ -119,18 +198,7 @@ docker run -d \ ### 3.3. Docker compose -The following is a simple docker-compose file to run the exporter. -``` -version: "2" -services: - exporter: - image: registry.gitlab.com/hectorjsmith/fail2ban-prometheus-exporter:latest - volumes: - - /var/run/fail2ban/:/var/run/fail2ban:ro - ports: - - "9191:9191" -``` ## 4. Metrics @@ -170,47 +238,6 @@ Exposed metrics: **Sample** ``` -# HELP f2b_errors Number of errors found since startup -# TYPE f2b_errors counter -f2b_errors{type="socket_conn"} 0 -f2b_errors{type="socket_req"} 0 -# HELP f2b_jail_banned_current Number of IPs currently banned in this jail -# TYPE f2b_jail_banned_current gauge -f2b_jail_banned_current{jail="recidive"} 5 -f2b_jail_banned_current{jail="sshd"} 15 -# HELP f2b_jail_banned_total Total number of IPs banned by this jail (includes expired bans) -# TYPE f2b_jail_banned_total gauge -f2b_jail_banned_total{jail="recidive"} 6 -f2b_jail_banned_total{jail="sshd"} 31 -# HELP f2b_jail_count Number of defined jails -# TYPE f2b_jail_count gauge -f2b_jail_count 2 -# HELP f2b_jail_failed_current Number of current failures on this jail's filter -# TYPE f2b_jail_failed_current gauge -f2b_jail_failed_current{jail="recidive"} 5 -f2b_jail_failed_current{jail="sshd"} 6 -# HELP f2b_jail_failed_total Number of total failures on this jail's filter -# TYPE f2b_jail_failed_total gauge -f2b_jail_failed_total{jail="recidive"} 7 -f2b_jail_failed_total{jail="sshd"} 125 -# HELP f2b_config_jail_ban_time How long an IP is banned for in this jail (in seconds) -# TYPE f2b_config_jail_ban_time gauge -f2b_config_jail_ban_time{jail="recidive"} 604800 -f2b_config_jail_ban_time{jail="sshd"} 600 -# HELP f2b_config_jail_find_time How far back will the filter look for failures in this jail (in seconds) -# TYPE f2b_config_jail_find_time gauge -f2b_config_jail_find_time{jail="recidive"} 86400 -f2b_config_jail_find_time{jail="sshd"} 600 -# HELP f2b_config_jail_max_retries The number of failures allowed until the IP is banned by this jail -# TYPE f2b_config_jail_max_retries gauge -f2b_config_jail_max_retries{jail="recidive"} 5 -f2b_config_jail_max_retries{jail="sshd"} 5 -# HELP f2b_up Check if the fail2ban server is up -# TYPE f2b_up gauge -f2b_up 1 -# HELP f2b_version Version of the exporter and fail2ban server -# TYPE f2b_version gauge -f2b_version{exporter="0.3.0",fail2ban="0.11.1"} 1 ``` The metrics above correspond to the matching fields in the `fail2ban-client status <jail>` command: diff --git a/grafana.json b/examples/grafana/dashboard.json similarity index 100% rename from grafana.json rename to examples/grafana/dashboard.json From 7899e4db2588c69c710f8f8e4f08760ce7e0da43 Mon Sep 17 00:00:00 2001 From: Hector <hector@hjs.dev> Date: Sun, 20 Feb 2022 12:46:50 +0000 Subject: [PATCH 2/8] finish rewriting readme file --- README.md | 181 +++++++++++------------------------------------------- 1 file changed, 35 insertions(+), 146 deletions(-) diff --git a/README.md b/README.md index 88ab315..c195e1f 100644 --- a/README.md +++ b/README.md @@ -5,10 +5,9 @@ Collect metrics from a running fail2ban instance. ## Table of Contents 1. Quick Start 2. Metrics - -3. Running the Exporter -4. Running in Docker -5. Metrics +3. Configuration +4. Building +5. Textfile metrics ## 1. Quick Start @@ -52,6 +51,10 @@ services: - "9191:9191" ``` +**NOTE:** While it is possible to mount the `fail2ban.sock` file directly, it is recommended to mount the parent folder instead. +The `.sock` file is deleted by fail2ban on shutdown and re-created on startup and this causes problems for the docker mount. +See [this reply](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/-/issues/11#note_665003499) for more details. + ## 2. Metrics The exporter exposes the following metrics: @@ -74,6 +77,18 @@ The exporter exposes the following metrics: | `jail_config_max_retry` | The max number of failures allowed before banning an IP in this jail | `f2b_config_jail_max_retries{jail="sshd"} 5` | | `version` | Version string of the exporter and fail2ban | `f2b_version{exporter="0.5.0",fail2ban="0.11.1"} 1` | +The metrics above correspond to the matching fields in the `fail2ban-client status <jail>` command: +``` +Status for the jail: sshd|- Filter +| |- Currently failed: 6 +| |- Total failed: 125 +| `- File list: /var/log/auth.log +`- Actions + |- Currently banned: 15 + |- Total banned: 31 + `- Banned IP list: ... +``` + ### 2.1. Grafana The metrics exported by this tool are compatible with Prometheus and Grafana. @@ -82,42 +97,14 @@ Just import the contents of this file into a new Grafana dashboard to get starte *(Sample dashboard is compatible with Grafana `8.3.3` and above)* +## 3. Configuration +The exporter is configured with CLI flags and environment variables. +There are no configuration files. - ---- - - - - - - -## 1. Introduction -This exporter collects metrics from a running fail2ban instance. - -Once the exporter is running, metrics are available at `localhost:9191/metrics`. - -(The default port is `9191` but can be modified with the `--web.listen-address` flag) - -The exporter communicates with the fail2ban server over its socket. -This allows the data collected by the exporter to always align with the output of the `fail2ban-client`. - -The default location of the socket is: `/var/run/fail2ban/fail2ban.sock` - - -## 2. Running the Exporter - -The exporter is compiled and released as a single binary. -This makes it very easy to run in any environment. -No additional runtime dependencies are required. - -Compiled binaries for various platforms are provided in each release. -See the [releases page](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/-/releases) for more information. - -**CLI Usage** +**CLI flags** ``` -$ fail2ban_exporter -h -usage: fail2ban-prometheus-exporter [<flags>] +usage: fail2ban_exporter [<flags>] Flags: -h, --help Show context-sensitive help (also try --help-long and --help-man). @@ -136,123 +123,25 @@ Flags: **Environment variables** -The tool can also be configured using environment variables. Each CLI parameter has a corresponding environment variable. +Each environment variable corresponds to a CLI flag. +If both are specified, the CLI flag takes precedence. -``` -F2B_COLLECTOR_SOCKET -F2B_COLLECTOR_TEXT_PATH -F2B_WEB_LISTEN_ADDRESS -F2B_WEB_BASICAUTH_USER -F2B_WEB_BASICAUTH_PASS -``` +| Environment variable | CLI flag | +|---------------------------|----------------------------------| +| `F2B_COLLECTOR_SOCKET` | `--collector.f2b.socket` | +| `F2B_COLLECTOR_TEXT_PATH` | `--collector.textfile.directory` | +| `F2B_WEB_LISTEN_ADDRESS` | `--web.listen-address` | +| `F2B_WEB_BASICAUTH_USER` | `--web.basic-auth.username` | +| `F2B_WEB_BASICAUTH_PASS` | `--web.basic-auth.password` | -**Example** - -``` -fail2ban-prometheus-exporter --collector.f2b.socket=/var/run/fail2ban/fail2ban.sock --web.listen-address=":9191" -``` - -Note that the exporter will need read access to the fail2ban socket. - -### 2.1. Compile from Source +## 4. Building The code can be compiled from source by running `go build` inside the `src/` folder. Go version `1.15` or greater is required. Run `go mod download` to download all necessary dependencies before running the build. -## 3. Running in Docker - -An official docker image is available on the Gitlab container registry. -Use it by pulling the following image: - -``` -registry.gitlab.com/hectorjsmith/fail2ban-prometheus-exporter:latest -``` - -Use the `:latest` tag to get the latest stable release. Or use the `:nightly` tag for the latest (unstable) version. -See the [registry page](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/container_registry) for all available tags. - -### 3.1. Volumes - -The docker image is designed to run by mounting the fail2ban run folder. -The run folder should be mounted in the container at: `/var/run/fail2ban`. - -The folder can be mounted with read-only (`ro`) permissions. - -**NOTE:** While it is possible to mount the `fail2ban.sock` file directly, it is recommended to mount the parent folder instead. -The `.sock` file is deleted by fail2ban on shutdown and re-created on startup and this causes problems for the docker mount. -See [this reply](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/-/issues/11#note_665003499) for more details. - -### 3.2. Docker run - -Use the following command to run the exporter as a docker container. - -``` -docker run -d \ - --name "fail2ban-exporter" \ - -v /var/run/fail2ban:/var/run/fail2ban:ro \ - -p "9191:9191" \ - registry.gitlab.com/hectorjsmith/fail2ban-prometheus-exporter:latest -``` - -### 3.3. Docker compose - - - -## 4. Metrics - -Access exported metrics at the `/metrics` path on the configured port. - -**Note on Fail2Ban Jails** - -fail2ban can be configured to process different log files and use different rules for each one. -These separate configurations are referred to as *jails*. - -For example, fail2ban can be configured to watch the system logs for failed SSH connections and Nextcloud logs for failed logins. -In this configuration, there will be two jails - one for IPs banned from the SSH logs, and one for IPs banned from the Nextcloud logs. - -This tool exports several metrics *per jail*, meaning that it is possible to track how many IPs are being banned in each jail as well as the overall total. -This can be useful to track what services are seeing more failed logins. - -### 4.1. Fail2Ban Metrics - -These are the metrics exported by reading data from the fail2ban server socket. -All metrics are prefixed with `f2b_`. - -Exposed metrics: -* `up` - Returns 1 if the fail2ban server is up and connection succeeds -* `errors` - Number of errors since startup - * `socket_conn` - Errors connecting to the fail2ban socket (e.g. connection refused) - * `socket_req` - Errors sending requests to the fail2ban server (e.g. invalid responses) -* `jail_count` - Number of jails configured in fail2ban -* `jail_banned_current` (per jail) - Number of IPs currently banned -* `jail_banned_total` (per jail) - Total number of banned IPs since fail2ban startup (includes expired bans) -* `jail_failed_current` (per jail) - Number of current failures -* `jail_failed_total` (per jail) - Total number of failures since fail2ban startup -* `jail_config_ban_time` (per jail) - How long an IP is banned for in this jail (in seconds) -* `jail_config_find_time` (per jail) - How far back the filter will look for failures in this jail (in seconds) -* `jail_config_max_retry` (per jail) - The max number of failures allowed before banning an IP in this jail -* `version` - Version string of the exporter and fail2ban - -**Sample** - -``` -``` - -The metrics above correspond to the matching fields in the `fail2ban-client status <jail>` command: -``` -Status for the jail: sshd|- Filter -| |- Currently failed: 6 -| |- Total failed: 125 -| `- File list: /var/log/auth.log -`- Actions - |- Currently banned: 15 - |- Total banned: 31 - `- Banned IP list: ... -``` - -### 4.2. Textfile Metrics +## 5. Textfile metrics For more flexibility the exporter also allows exporting metrics collected from a text file. From 8cb2b1fe5085ec16d7216f75221ce764e19f5c03 Mon Sep 17 00:00:00 2001 From: Hector <hector@hjs.dev> Date: Sun, 20 Feb 2022 16:18:02 +0000 Subject: [PATCH 3/8] tweak readme file --- README.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index c195e1f..89871d4 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ Collect metrics from a running fail2ban instance. 1. Quick Start 2. Metrics 3. Configuration -4. Building +4. Building from source 5. Textfile metrics ## 1. Quick Start @@ -126,7 +126,7 @@ Flags: Each environment variable corresponds to a CLI flag. If both are specified, the CLI flag takes precedence. -| Environment variable | CLI flag | +| Environment variable | Corresponding CLI flag | |---------------------------|----------------------------------| | `F2B_COLLECTOR_SOCKET` | `--collector.f2b.socket` | | `F2B_COLLECTOR_TEXT_PATH` | `--collector.textfile.directory` | @@ -134,12 +134,14 @@ If both are specified, the CLI flag takes precedence. | `F2B_WEB_BASICAUTH_USER` | `--web.basic-auth.username` | | `F2B_WEB_BASICAUTH_PASS` | `--web.basic-auth.password` | -## 4. Building +## 4. Building from source -The code can be compiled from source by running `go build` inside the `src/` folder. -Go version `1.15` or greater is required. +The simplest way to build the project is to run the `build/snapshot` make command. +This will use `goreleaser` to build out binaries and archives for the project. +Binaries are stored in the `dist/` folder. -Run `go mod download` to download all necessary dependencies before running the build. +Alternatively, `go mod download` and `go build` can be used from the `src/` folder to build out the project. +This will download dependencies and build the project. ## 5. Textfile metrics From a742c4d4549a5a6cadfc7dc14ef2aefa54bdbc45 Mon Sep 17 00:00:00 2001 From: Hector <hector@hjs.dev> Date: Sun, 20 Feb 2022 16:20:45 +0000 Subject: [PATCH 4/8] update changelog --- CHANGELOG.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7836b92..5513824 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,9 @@ The format is based on [Keep a Changelog], and this project adheres to [Semantic ## [Unreleased] ### Added +- (23e073f) feat: add example systemd service file +- (3911eca) feat: rename output binary and archives +- (f6e328a) feat: correctly handle shutdown signals - (6e575aa) feat: rewrite cli flags and environment variables - (0f0efe5) feat: remove startup script from docker image - (e2902b8) feat: improve logging on startup @@ -18,6 +21,7 @@ The format is based on [Keep a Changelog], and this project adheres to [Semantic - (93da909) fix: use correct flag in dockerfile ([#18](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/issues/18)) ### BREAKING CHANGE +- Release binary name has been changed to `fail2ban_exporter`. - Replace `--socket` flag with `--collector.f2b.socket`. - Merge `--port` flag and `--web.listen-address` into a single flag. - Remove `--collector.textfile` flag, its value is now derived from `--collector.textfile.directory`. From 9a752571baab5d8241ca3bf9bbdc1c105bbdabdf Mon Sep 17 00:00:00 2001 From: Hector <hector@hjs.dev> Date: Sun, 20 Feb 2022 16:24:31 +0000 Subject: [PATCH 5/8] fix minor error in readme file --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 89871d4..64102fc 100644 --- a/README.md +++ b/README.md @@ -79,7 +79,8 @@ The exporter exposes the following metrics: The metrics above correspond to the matching fields in the `fail2ban-client status <jail>` command: ``` -Status for the jail: sshd|- Filter +Status for the jail: sshd +|- Filter | |- Currently failed: 6 | |- Total failed: 125 | `- File list: /var/log/auth.log From 43465937bfdefcb4a12201fc6a07b3f48732629e Mon Sep 17 00:00:00 2001 From: Hector <hector@hjs.dev> Date: Sun, 20 Feb 2022 16:31:38 +0000 Subject: [PATCH 6/8] add note on available docker tags --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 64102fc..20b51e4 100644 --- a/README.md +++ b/README.md @@ -51,6 +51,9 @@ services: - "9191:9191" ``` +Use the `:latest` tag to get the latest stable release. Or use the `:nightly` tag for the latest (unstable) version. +See the [registry page](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/container_registry) for all available tags. + **NOTE:** While it is possible to mount the `fail2ban.sock` file directly, it is recommended to mount the parent folder instead. The `.sock` file is deleted by fail2ban on shutdown and re-created on startup and this causes problems for the docker mount. See [this reply](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/-/issues/11#note_665003499) for more details. From 3ed60135dd3b199bed1314e6b8b5ad93ef43874c Mon Sep 17 00:00:00 2001 From: Hector <hector@hjs.dev> Date: Sun, 20 Feb 2022 16:34:47 +0000 Subject: [PATCH 7/8] add link to releases page to download binaries --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 20b51e4..fda871b 100644 --- a/README.md +++ b/README.md @@ -27,6 +27,8 @@ $ fail2ban_exporter --collector.f2b.socket=/var/run/fail2ban/fail2ban.sock --web 2022/02/20 09:54:06 ready ``` +Binary files for each release can be found on the [releases](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/-/releases) page. + ### 1.2. Docker **Docker run** From 033e22b1844196640d9c39859e6466001fbda3ac Mon Sep 17 00:00:00 2001 From: Hector <hector@hjs.dev> Date: Sun, 20 Feb 2022 16:41:46 +0000 Subject: [PATCH 8/8] add pipeline badge to readme --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index fda871b..b55199d 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,7 @@ # Fail2Ban Prometheus Exporter +[](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter) + Collect metrics from a running fail2ban instance. ## Table of Contents