docs: update changelog for release

Update the CHANGELOG file for the v0.2.0 release.

CHANGELOG.md

@@ -4,7 +4,27 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog], and this project adheres to [Semantic Versioning].
 
-## [Unreleased]## [0.1.0] - 2021-03-28
+## [Unreleased]
+
+## [0.2.0] - 2021-08-31
+*Collect metrics through fail2ban socket - based on [#11](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/issues/11)*
+
+### Added
+- (39133d0) feat: collect new up metric from fail2ban socket
+- (4da46f3) feat: export metrics with socket errors
+- (bd841c3) feat: set up metric to 0 if errors found
+- (1964dde) feat: export metrics for failed/banned counts
+- (2ab1f7d) feat: support reading fail2ban socket in docker
+- (1282d63) feat: new metric for enabled jails ([#1](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/issues/1))
+
+### Fixed
+- (526b1c7) fix: update banned metrics to exclude expired bans ([#11](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/issues/11))
+
+### Deprecated
+- Use of the fail2ban database has been deprecated. The exporter now collects metrics through the fail2ban socket file. See [#11](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/-/issues/11) for more details.
+
+## [0.1.0] - 2021-03-28
+*Initial release*
 
 ### Added
 - (6355c9e) feat: fail on startup if database file does not exist ([#8](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/issues/8))
@@ -18,13 +38,12 @@ The format is based on [Keep a Changelog], and this project adheres to [Semantic
 ### Fixed
 - (0842419) fix: compile tool without cgo_enabled flag
 
+## 0.0.0 - 2021-02-05
 
 ---
 
-*This changelog is automatically generated by [git-chglog]*
-
 [Keep a Changelog]: https://keepachangelog.com/en/1.0.0/
 [Semantic Versioning]: https://semver.org/spec/v2.0.0.html
-[git-chglog]: https://github.com/git-chglog/git-chglog
 [Unreleased]: https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/compare/0.1.0...main
 [0.1.0]: https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/compare/0.0.0...0.1.0
+[0.2.0]: https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/compare/0.1.0...0.2.0

Makefile

@@ -15,7 +15,7 @@ format:
 	cd src/ && go vet $(go list ./... | grep -v /vendor/)
 
 generateChangelog:
-	./tools/git-chglog_linux_amd64 --config tools/chglog/config.yml 0.0.0.. > CHANGELOG.md
+	./tools/git-chglog_linux_amd64 --config tools/chglog/config.yml 0.0.0.. > CHANGELOG_gen.md
 
 build/snapshot:
 	./tools/goreleaser_linux_amd64 --snapshot --rm-dist --skip-publish

README.md

@@ -3,38 +3,75 @@
 Go tool to collect and export metrics on Fail2Ban
 
 ## Table of Contents
-1. How to use
-2. Docker
-    1. Volumes
-    2. Docker run
-    3. Docker compose
-3. CLI usage
+1. Introduction
+2. Running the Exporter
+3. Running in Docker
 4. Metrics
 
-## 1. How to use
+## 1. Introduction
+The exporter can collect metrics from 2 locations: the fail2ban server socket and the fail2ban server database.
 
-Run the exporter by providing it with a fail2ban database to read data from.
-Read access to the database is required.
 Once the exporter is running, metrics are available at `localhost:9191/metrics`.
 
-The default port is `9191`, but this can be modified with the `-port` flag.
+(The default port is `9191` but can be modified with the `-port` flag)
 
-**Note:** By default fail2ban stores the database file at: `/var/lib/fail2ban/fail2ban.sqlite3`
+### 1.1. Socket
+The recommended way to run the exporter is to point it at the fail2ban server socket.
+This allows the exporter to communicate with the server the same way `fail2ban-client` does and ensures the metrics it collects align with the values reported by `fail2ban-client status <jail>`.
 
-**Fail2Ban Jails**
+The default path to the socket is: `/var/run/fail2ban/fail2ban.sock`
 
-fail2ban can be configured to process different log files and use different rules for each one.
-These separate configurations are referred to as *jails*.
+### 1.2. Deprecated: Database
+The original way to collect metrics is to read them from the fail2ban database.
+This has now been deprecated in favour of using the socket.
+The reason being that database metrics do not always align with the output of `fail2ban-client status <jail>` and cause confusion.
+See [#11](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/-/issues/11) for more details.
 
-For example, fail2ban can be configured to watch the system logs for failed SSH connections and Nextcloud logs for failed logins.
-In this configuration, there will be two jails - one for IPs banned from the SSH logs, and one for IPs banned from the Nextcloud logs.
+If necessary, these metrics can still be exported by providing the database path to the exporter.
 
-This tool exports several metrics *per jail*, meaning that it is possible to track how many IPs are being banned in each jail as well as the overall total.
-This can be useful to track what services are seeing more failed logins.
+The default path to the fail2ban database is: `/var/lib/fail2ban/fail2ban.sqlite3`
 
-## 2. Docker
+## 2. Running the Exporter
 
-An official docker image is available on the Gitlab container registry.
+The exporter is compiled and released as a single binary.
+This makes it very easy to run in any environment.
+No additional runtime dependencies are required.
+
+Compiled binaries for various platforms are provided in each release.
+See the [releases page](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/-/releases) for more information.
+
+**Usage**
+```
+$ fail2ban-prometheus-exporter -h
+
+  -db string
+        path to the fail2ban sqlite database (deprecated)
+  -port int
+        port to use for the metrics server (default 9191)
+  -socket string
+        path to the fail2ban server socket
+  -version
+        show version info and exit
+```
+
+**Example**
+
+```
+fail2ban-prometheus-exporter -socket /var/run/fail2ban/fail2ban.sock -port 9191
+```
+
+Note that the exporter will need read access to the fail2ban socket or database.
+
+### 2.1. Compile from Source
+
+The code can be compiled from source by running `go build` inside the `src/` folder.
+Go version `1.15` or greater is required.
+
+Run `go mod download` to download all necessary dependencies before running the build.
+
+## 3. Running in Docker
+
+If use of docker is desired, an official docker image is available on the Gitlab container registry.
 Use it by pulling the following image:
 
 ```
@@ -44,26 +81,32 @@ registry.gitlab.com/hectorjsmith/fail2ban-prometheus-exporter:latest
 Use the `:latest` tag to get the most up to date code (less stable) or use one of the version tagged images to use a specific release.
 See the [registry page](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/container_registry) for all available tags.
 
-### 2.1. Volumes
+### 3.1. Volumes
 
-The docker image is designed to run by mounting the fail2ban sqlite3 database.
-The database should be mounted at: `/app/fail2ban.sqlite3`
+The docker image is designed to run by mounting either the fail2ban sqlite3 database of the fail2ban run folder.
+- The database should be mounted at: `/app/fail2ban.sqlite3`
+- The run folder should be mounted at: `/var/run/fail2ban`
 
-The database can be mounted with read-only permissions.
+Both paths can be mounted with readonly (`ro`) permissions.
 
-### 2.2. Docker run
+**NOTE:** While it is possible to mount the `fail2ban.sock` file directly, it is recommended to mount the parent folder instead.
+The `.sock` file is deleted by fail2ban on shutdown and re-created on startup and this causes problems for the docker mount.
+See [this reply](https://gitlab.com/hectorjsmith/fail2ban-prometheus-exporter/-/issues/11#note_665003499) for more details.
 
-Use the following command to run the forwarder as a docker container.
+### 3.2. Docker run
+
+Use the following command to run the exporter as a docker container.
 
 ```
 docker run -d \
     --name "fail2ban-exporter" \
     -v /var/lib/fail2ban/fail2ban.sqlite3:/app/fail2ban.sqlite3:ro \
-    -p "9191:9191"
+    -v /var/run/fail2ban:/var/run/fail2ban:ro \
+    -p "9191:9191" \
     registry.gitlab.com/hectorjsmith/fail2ban-prometheus-exporter:latest
 ```
 
-### 2.3. Docker compose
+### 3.3. Docker compose
 
 The following is a simple docker-compose file to run the exporter.
 
@@ -74,31 +117,100 @@ services:
     image: registry.gitlab.com/hectorjsmith/fail2ban-prometheus-exporter:latest
     volumes:
     - /var/lib/fail2ban/fail2ban.sqlite3:/app/fail2ban.sqlite3:ro
+    - /var/run/fail2ban/:/var/run/fail2ban:ro
     ports:
     - "9191:9191"
 ```
 
-## 3. CLI usage
-
-```
-$ fail2ban-prometheus-exporter -h
-
-  -db string
-        path to the fail2ban sqlite database
-  -port int
-        port to use for the metrics server (default 9191)
-  -version
-        show version info and exit
-```
-
 ## 4. Metrics
 
-Access exported metrics at `/metrics` (on the provided port).
+Access exported metrics at the `/metrics` path on the configured port.
 
-**Note:** All metric names include the `fail2ban_` prefix to make sure they are unique and easier to find.
+**Note on Fail2Ban Jails**
+
+fail2ban can be configured to process different log files and use different rules for each one.
+These separate configurations are referred to as *jails*.
+
+For example, fail2ban can be configured to watch the system logs for failed SSH connections and Nextcloud logs for failed logins.
+In this configuration, there will be two jails - one for IPs banned from the SSH logs, and one for IPs banned from the Nextcloud logs.
+
+This tool exports several metrics *per jail*, meaning that it is possible to track how many IPs are being banned in each jail as well as the overall total.
+This can be useful to track what services are seeing more failed logins.
+
+### 4.1. Socket-based Metrics
+
+These are the metrics exported by reading data from the fail2ban server socket.
+All metrics are prefixed with `f2b_`.
+
+Exposed metrics:
+* `up` - Returns 1 if the fail2ban server is up and connection succeeds
+* `errors` - Number of errors since startup
+    * `db` - Errors connecting to the database
+    * `socket_conn` - Errors connecting to the fail2ban socket (e.g. connection refused)
+    * `socket_req` - Errors sending requests to the fail2ban server (e.g. invalid responses)
+* `jail_count` - Number of jails configured in fail2ban
+* `jail_banned_current` (per jail) - Number of IPs currently banned
+* `jail_banned_total` (per jail) - Total number of banned IPs since fail2ban startup (includes expired bans)
+* `jail_failed_current` (per jail) - Number of current failures
+* `jail_failed_total` (per jail) - Total number of failures since fail2ban startup
+
+**Sample**
+
+```
+# HELP f2b_errors Number of errors found since startup
+# TYPE f2b_errors counter
+f2b_errors{type="db"} 0
+f2b_errors{type="socket_conn"} 0
+f2b_errors{type="socket_req"} 0
+# HELP f2b_jail_banned_current Number of IPs currently banned in this jail
+# TYPE f2b_jail_banned_current gauge
+f2b_jail_banned_current{jail="recidive"} 5
+f2b_jail_banned_current{jail="sshd"} 15
+# HELP f2b_jail_banned_total Total number of IPs banned by this jail (includes expired bans)
+# TYPE f2b_jail_banned_total gauge
+f2b_jail_banned_total{jail="recidive"} 6
+f2b_jail_banned_total{jail="sshd"} 31
+# HELP f2b_jail_count Number of defined jails
+# TYPE f2b_jail_count gauge
+f2b_jail_count 2
+# HELP f2b_jail_failed_current Number of current failures on this jail's filter
+# TYPE f2b_jail_failed_current gauge
+f2b_jail_failed_current{jail="recidive"} 5
+f2b_jail_failed_current{jail="sshd"} 6
+# HELP f2b_jail_failed_total Number of total failures on this jail's filter
+# TYPE f2b_jail_failed_total gauge
+f2b_jail_failed_total{jail="recidive"} 7
+f2b_jail_failed_total{jail="sshd"} 125
+# HELP f2b_up Check if the fail2ban server is up
+# TYPE f2b_up gauge
+f2b_up 1
+```
+
+The metrics above correspond to the matching fields in the `fail2ban-client status <jail>` command:
+```
+Status for the jail: sshd|- Filter
+|  |- Currently failed: 6
+|  |- Total failed:     125
+|  `- File list:        /var/log/auth.log
+`- Actions
+   |- Currently banned: 15
+   |- Total banned:     31
+   `- Banned IP list:   ...
+```
+
+### 4.2. Database Metrics (deprecated)
+
+These are the original metrics exported by the initial release of the exporter.
+They are all based on the data stored in the fail2ban sqlite3 database.
+
+*These metrics are deprecated and will be removed in a future release.*
+
+All metrics are prefixed with `fail2ban_`.
 
 Exposed metrics:
 * `up` - Returns 1 if the service is up
+* `errors` - Returns the number of errors found since startup
+* `enabled_jails` - Returns 1 for each jail that is enabled, 0 if disabled.
 * `bad_ips` (per jail)
     * A *bad IP* is defined as an IP that has been banned at least once in the past
     * Bad IPs are counted per jail
@@ -109,15 +221,22 @@ Exposed metrics:
 **Sample**
 
 ```
-# HELP fail2ban_bad_ips Number of bad IPs stored in the database (per jail).
+# HELP fail2ban_bad_ips (Deprecated) Number of bad IPs stored in the database (per jail).
 # TYPE fail2ban_bad_ips gauge
-fail2ban_bad_ips{jail="jail1"} 6
-fail2ban_bad_ips{jail="jail2"} 8
-# HELP fail2ban_banned_ips Number of banned IPs stored in the database (per jail).
+fail2ban_bad_ips{jail="recidive"} 0
+fail2ban_bad_ips{jail="sshd"} 0
+# HELP fail2ban_banned_ips (Deprecated) Number of banned IPs stored in the database (per jail).
 # TYPE fail2ban_banned_ips gauge
-fail2ban_banned_ips{jail="jail1"} 3
-fail2ban_banned_ips{jail="jail2"} 2
-# HELP fail2ban_up Was the last fail2ban query successful.
+fail2ban_banned_ips{jail="recidive"} 0
+fail2ban_banned_ips{jail="sshd"} 0
+# HELP fail2ban_enabled_jails (Deprecated) Enabled jails.
+# TYPE fail2ban_enabled_jails gauge
+fail2ban_enabled_jails{jail="recidive"} 1
+fail2ban_enabled_jails{jail="sshd"} 1
+# HELP fail2ban_errors (Deprecated) Number of errors found since startup.
+# TYPE fail2ban_errors counter
+fail2ban_errors{type="db"} 0
+# HELP fail2ban_up (Deprecated) Was the last fail2ban query successful.
 # TYPE fail2ban_up gauge
 fail2ban_up 1
 ```

docker/run.sh

@@ -3,7 +3,19 @@
 # Print version to logs for debugging purposes
 /app/fail2ban-prometheus-exporter -version
 
+db_path=/app/fail2ban.sqlite3
+socket_path=/var/run/fail2ban/fail2ban.sock
+
+# Blank out the file paths if they do not exist - a hacky way to only use these files if they were mounted into the container.
+if [ ! -f "$db_path" ]; then
+    db_path=""
+fi
+if [ ! -S "$socket_path" ]; then
+    socket_path=""
+fi
+
 # Start the exporter (use exec to support graceful shutdown)
 # Inspired by: https://akomljen.com/stopping-docker-containers-gracefully/
 exec /app/fail2ban-prometheus-exporter \
-    -db /app/fail2ban.sqlite3
+    -db "$db_path" \
+    -socket "$socket_path"

src/cfg/cfg.go

@@ -15,13 +15,15 @@ type AppSettings struct {
 	VersionMode        bool
 	MetricsPort        int
 	Fail2BanDbPath     string
+	Fail2BanSocketPath string
 }
 
 func Parse() *AppSettings {
 	appSettings := &AppSettings{}
 	flag.BoolVar(&appSettings.VersionMode, "version", false, "show version info and exit")
 	flag.IntVar(&appSettings.MetricsPort, "port", 9191, "port to use for the metrics server")
-	flag.StringVar(&appSettings.Fail2BanDbPath, "db", "", "path to the fail2ban sqlite database")
+	flag.StringVar(&appSettings.Fail2BanDbPath, "db", "", "path to the fail2ban sqlite database (deprecated)")
+	flag.StringVar(&appSettings.Fail2BanSocketPath, "socket", "", "path to the fail2ban server socket")
 
 	flag.Parse()
 	appSettings.validateFlags()
@@ -31,8 +33,8 @@ func Parse() *AppSettings {
 func (settings *AppSettings) validateFlags() {
 	var flagsValid = true
 	if !settings.VersionMode {
-		if settings.Fail2BanDbPath == "" {
-			fmt.Println("missing flag 'db'")
+		if settings.Fail2BanDbPath == "" && settings.Fail2BanSocketPath == "" {
+			fmt.Println("at least one of the following flags must be provided: 'db', 'socket'")
 			flagsValid = false
 		}
 		if settings.MetricsPort < minServerPort || settings.MetricsPort > maxServerPort {

src/db/db.go

@@ -7,7 +7,8 @@ import (
 )
 
 const queryBadIpsPerJail = "SELECT j.name, (SELECT COUNT(1) FROM bips b WHERE j.name = b.jail) FROM jails j"
-const queryBannedIpsPerJail = "SELECT j.name, (SELECT COUNT(1) FROM bans b WHERE j.name = b.jail) FROM jails j"
+const queryBannedIpsPerJail = "SELECT j.name, (SELECT COUNT(1) FROM bans b WHERE j.name = b.jail AND b.timeofban + b.bantime >= strftime('%s','now') + 0) FROM jails j"
+const queryJailNameToEnabled = "SELECT j.name, j.enabled FROM jails j"
 
 type Fail2BanDB struct {
 	DatabasePath string
@@ -36,6 +37,10 @@ func (db *Fail2BanDB) CountBadIpsPerJail() (map[string]int, error) {
 	return db.RunJailNameToCountQuery(queryBadIpsPerJail)
 }
 
+func (db *Fail2BanDB) JailNameToEnabledValue() (map[string]int, error) {
+	return db.RunJailNameToCountQuery(queryJailNameToEnabled)
+}
+
 func (db *Fail2BanDB) RunJailNameToCountQuery(query string) (map[string]int, error) {
 	stmt, err := db.sqliteDB.Prepare(query)
 	defer db.mustCloseStatement(stmt)

src/exporter.go

@@ -3,15 +3,20 @@ package main
 import (
 	"fail2ban-prometheus-exporter/cfg"
 	fail2banDb "fail2ban-prometheus-exporter/db"
+	"fail2ban-prometheus-exporter/socket"
 	"fmt"
+	"log"
+	"net/http"
+
 	_ "github.com/mattn/go-sqlite3"
 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/prometheus/client_golang/prometheus/promhttp"
-	"log"
-	"net/http"
 )
 
-const namespace = "fail2ban"
+const (
+	deprecatedNamespace = "fail2ban"
+	namespace           = "f2b"
+)
 
 var (
 	version = "dev"
@@ -19,67 +24,255 @@ var (
 	date    = "unknown"
 	builtBy = "unknown"
 
-	metricUp = prometheus.NewDesc(
-		prometheus.BuildFQName(namespace, "", "up"),
-		"Was the last fail2ban query successful.",
+	deprecatedMetricUp = prometheus.NewDesc(
+		prometheus.BuildFQName(deprecatedNamespace, "", "up"),
+		"(Deprecated) Was the last fail2ban query successful.",
 		nil, nil,
 	)
-	metricBannedIpsPerJail = prometheus.NewDesc(
-		prometheus.BuildFQName(namespace, "", "banned_ips"),
-		"Number of banned IPs stored in the database (per jail).",
+	deprecatedMetricBannedIpsPerJail = prometheus.NewDesc(
+		prometheus.BuildFQName(deprecatedNamespace, "", "banned_ips"),
+		"(Deprecated) Number of banned IPs stored in the database (per jail).",
 		[]string{"jail"}, nil,
 	)
-	metricBadIpsPerJail = prometheus.NewDesc(
-		prometheus.BuildFQName(namespace, "", "bad_ips"),
-		"Number of bad IPs stored in the database (per jail).",
+	deprecatedMetricBadIpsPerJail = prometheus.NewDesc(
+		prometheus.BuildFQName(deprecatedNamespace, "", "bad_ips"),
+		"(Deprecated) Number of bad IPs stored in the database (per jail).",
+		[]string{"jail"}, nil,
+	)
+	deprecatedMetricEnabledJails = prometheus.NewDesc(
+		prometheus.BuildFQName(deprecatedNamespace, "", "enabled_jails"),
+		"(Deprecated) Enabled jails.",
+		[]string{"jail"}, nil,
+	)
+	deprecatedMetricErrorCount = prometheus.NewDesc(
+		prometheus.BuildFQName(deprecatedNamespace, "", "errors"),
+		"(Deprecated) Number of errors found since startup.",
+		[]string{"type"}, nil,
+	)
+
+	metricErrorCount = prometheus.NewDesc(
+		prometheus.BuildFQName(namespace, "", "errors"),
+		"Number of errors found since startup",
+		[]string{"type"}, nil,
+	)
+	metricServerUp = prometheus.NewDesc(
+		prometheus.BuildFQName(namespace, "", "up"),
+		"Check if the fail2ban server is up",
+		nil, nil,
+	)
+	metricJailCount = prometheus.NewDesc(
+		prometheus.BuildFQName(namespace, "", "jail_count"),
+		"Number of defined jails",
+		nil, nil,
+	)
+	metricJailFailedCurrent = prometheus.NewDesc(
+		prometheus.BuildFQName(namespace, "", "jail_failed_current"),
+		"Number of current failures on this jail's filter",
+		[]string{"jail"}, nil,
+	)
+	metricJailFailedTotal = prometheus.NewDesc(
+		prometheus.BuildFQName(namespace, "", "jail_failed_total"),
+		"Number of total failures on this jail's filter",
+		[]string{"jail"}, nil,
+	)
+	metricJailBannedCurrent = prometheus.NewDesc(
+		prometheus.BuildFQName(namespace, "", "jail_banned_current"),
+		"Number of IPs currently banned in this jail",
+		[]string{"jail"}, nil,
+	)
+	metricJailBannedTotal = prometheus.NewDesc(
+		prometheus.BuildFQName(namespace, "", "jail_banned_total"),
+		"Total number of IPs banned by this jail (includes expired bans)",
 		[]string{"jail"}, nil,
 	)
 )
 
 type Exporter struct {
 	db                         *fail2banDb.Fail2BanDB
+	socketPath                 string
+	lastError                  error
+	dbErrorCount               int
+	socketConnectionErrorCount int
+	socketRequestErrorCount    int
 }
 
 func (e *Exporter) Describe(ch chan<- *prometheus.Desc) {
-	ch <- metricUp
-	ch <- metricBadIpsPerJail
-	ch <- metricBannedIpsPerJail
+	if e.db != nil {
+		ch <- deprecatedMetricUp
+		ch <- deprecatedMetricBadIpsPerJail
+		ch <- deprecatedMetricBannedIpsPerJail
+		ch <- deprecatedMetricEnabledJails
+		ch <- deprecatedMetricErrorCount
+	}
+	if e.socketPath != "" {
+		ch <- metricServerUp
+		ch <- metricJailCount
+		ch <- metricJailFailedCurrent
+		ch <- metricJailFailedTotal
+		ch <- metricJailBannedCurrent
+		ch <- metricJailBannedTotal
+	}
+	ch <- metricErrorCount
 }
 
 func (e *Exporter) Collect(ch chan<- prometheus.Metric) {
-	ch <- prometheus.MustNewConstMetric(
-		metricUp, prometheus.GaugeValue, 1,
-	)
-	e.collectBadIpsPerJailMetrics(ch)
-	e.collectBannedIpsPerJailMetrics(ch)
+	if e.db != nil {
+		e.collectDeprecatedBadIpsPerJailMetrics(ch)
+		e.collectDeprecatedBannedIpsPerJailMetrics(ch)
+		e.collectDeprecatedEnabledJailMetrics(ch)
+		e.collectDeprecatedUpMetric(ch)
+		e.collectDeprecatedErrorCountMetric(ch)
+	}
+	if e.socketPath != "" {
+		s, err := socket.ConnectToSocket(e.socketPath)
+		if err != nil {
+			log.Printf("error opening socket: %v", err)
+			e.socketConnectionErrorCount++
+		} else {
+			defer s.Close()
+		}
+		e.collectServerUpMetric(ch, s)
+		if err == nil && s != nil {
+			e.collectJailMetrics(ch, s)
+		}
+	}
+	e.collectErrorCountMetric(ch)
 }
 
-func (e *Exporter) collectBadIpsPerJailMetrics(ch chan<- prometheus.Metric) {
+func (e *Exporter) collectDeprecatedUpMetric(ch chan<- prometheus.Metric) {
+	var upMetricValue float64 = 1
+	if e.lastError != nil {
+		upMetricValue = 0
+	}
+	ch <- prometheus.MustNewConstMetric(
+		deprecatedMetricUp, prometheus.GaugeValue, upMetricValue,
+	)
+}
+
+func (e *Exporter) collectDeprecatedErrorCountMetric(ch chan<- prometheus.Metric) {
+	ch <- prometheus.MustNewConstMetric(
+		deprecatedMetricErrorCount, prometheus.CounterValue, float64(e.dbErrorCount), "db",
+	)
+}
+
+func (e *Exporter) collectDeprecatedBadIpsPerJailMetrics(ch chan<- prometheus.Metric) {
 	jailNameToCountMap, err := e.db.CountBadIpsPerJail()
+	e.lastError = err
+
 	if err != nil {
+		e.dbErrorCount++
 		log.Print(err)
 	}
 
 	for jailName, count := range jailNameToCountMap {
 		ch <- prometheus.MustNewConstMetric(
-			metricBadIpsPerJail, prometheus.GaugeValue, float64(count), jailName,
+			deprecatedMetricBadIpsPerJail, prometheus.GaugeValue, float64(count), jailName,
 		)
 	}
 }
 
-func (e *Exporter) collectBannedIpsPerJailMetrics(ch chan<- prometheus.Metric) {
+func (e *Exporter) collectDeprecatedBannedIpsPerJailMetrics(ch chan<- prometheus.Metric) {
 	jailNameToCountMap, err := e.db.CountBannedIpsPerJail()
+	e.lastError = err
+
 	if err != nil {
+		e.dbErrorCount++
 		log.Print(err)
 	}
 
 	for jailName, count := range jailNameToCountMap {
 		ch <- prometheus.MustNewConstMetric(
-			metricBannedIpsPerJail, prometheus.GaugeValue, float64(count), jailName,
+			deprecatedMetricBannedIpsPerJail, prometheus.GaugeValue, float64(count), jailName,
 		)
 	}
 }
 
+func (e *Exporter) collectDeprecatedEnabledJailMetrics(ch chan<- prometheus.Metric) {
+	jailNameToEnabledMap, err := e.db.JailNameToEnabledValue()
+	e.lastError = err
+
+	if err != nil {
+		e.dbErrorCount++
+		log.Print(err)
+	}
+
+	for jailName, count := range jailNameToEnabledMap {
+		ch <- prometheus.MustNewConstMetric(
+			deprecatedMetricEnabledJails, prometheus.GaugeValue, float64(count), jailName,
+		)
+	}
+}
+
+func (e *Exporter) collectErrorCountMetric(ch chan<- prometheus.Metric) {
+	ch <- prometheus.MustNewConstMetric(
+		metricErrorCount, prometheus.CounterValue, float64(e.dbErrorCount), "db",
+	)
+	ch <- prometheus.MustNewConstMetric(
+		metricErrorCount, prometheus.CounterValue, float64(e.socketConnectionErrorCount), "socket_conn",
+	)
+	ch <- prometheus.MustNewConstMetric(
+		metricErrorCount, prometheus.CounterValue, float64(e.socketRequestErrorCount), "socket_req",
+	)
+}
+
+func (e *Exporter) collectServerUpMetric(ch chan<- prometheus.Metric, s *socket.Fail2BanSocket) {
+	var serverUp float64 = 0
+	if s != nil {
+		pingSuccess, err := s.Ping()
+		if err != nil {
+			e.socketRequestErrorCount++
+			log.Print(err)
+		}
+		if err == nil && pingSuccess {
+			serverUp = 1
+		}
+	}
+	ch <- prometheus.MustNewConstMetric(
+		metricServerUp, prometheus.GaugeValue, serverUp,
+	)
+}
+
+func (e *Exporter) collectJailMetrics(ch chan<- prometheus.Metric, s *socket.Fail2BanSocket) {
+	jails, err := s.GetJails()
+	var count float64 = 0
+	if err != nil {
+		e.socketRequestErrorCount++
+		log.Print(err)
+	}
+	if err == nil {
+		count = float64(len(jails))
+	}
+	ch <- prometheus.MustNewConstMetric(
+		metricJailCount, prometheus.GaugeValue, count,
+	)
+
+	for i := range jails {
+		e.collectJailStatsMetric(ch, s, jails[i])
+	}
+}
+
+func (e *Exporter) collectJailStatsMetric(ch chan<- prometheus.Metric, s *socket.Fail2BanSocket, jail string) {
+	stats, err := s.GetJailStats(jail)
+	if err != nil {
+		e.socketRequestErrorCount++
+		log.Printf("failed to get stats for jail %s: %v", jail, err)
+		return
+	}
+
+	ch <- prometheus.MustNewConstMetric(
+		metricJailFailedCurrent, prometheus.GaugeValue, float64(stats.FailedCurrent), jail,
+	)
+	ch <- prometheus.MustNewConstMetric(
+		metricJailFailedTotal, prometheus.GaugeValue, float64(stats.FailedTotal), jail,
+	)
+	ch <- prometheus.MustNewConstMetric(
+		metricJailBannedCurrent, prometheus.GaugeValue, float64(stats.BannedCurrent), jail,
+	)
+	ch <- prometheus.MustNewConstMetric(
+		metricJailBannedTotal, prometheus.GaugeValue, float64(stats.BannedTotal), jail,
+	)
+}
+
 func printAppVersion() {
 	fmt.Println(version)
 	fmt.Printf("    build date:  %s\r\n    commit hash: %s\r\n    built by:    %s\r\n", date, commit, builtBy)
@@ -92,8 +285,13 @@ func main() {
 	} else {
 		log.Print("starting fail2ban exporter")
 
-		exporter := &Exporter{
-			db: fail2banDb.MustConnectToDb(appSettings.Fail2BanDbPath),
+		exporter := &Exporter{}
+		if appSettings.Fail2BanDbPath != "" {
+			log.Print("database-based metrics have been deprecated and will be removed in a future release")
+			exporter.db = fail2banDb.MustConnectToDb(appSettings.Fail2BanDbPath)
+		}
+		if appSettings.Fail2BanSocketPath != "" {
+			exporter.socketPath = appSettings.Fail2BanSocketPath
 		}
 		prometheus.MustRegister(exporter)
 

src/go.mod

@@ -3,6 +3,8 @@ module fail2ban-prometheus-exporter
 go 1.15
 
 require (
+	github.com/kisielk/og-rek v1.1.0
 	github.com/mattn/go-sqlite3 v1.14.6
+	github.com/nlpodyssey/gopickle v0.1.0
 	github.com/prometheus/client_golang v1.9.0
 )

src/go.sum

@@ -137,6 +137,8 @@ github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7V
 github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
 github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
 github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
+github.com/kisielk/og-rek v1.1.0 h1:u10TvQbPtrlY/6H4+BiFsBywwSVTGFsx0YOVtpx3IbI=
+github.com/kisielk/og-rek v1.1.0/go.mod h1:6ihsOSzSAxR/65S3Bn9zNihoEqRquhDQZ2c6I2+MG3c=
 github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
 github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
 github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
@@ -175,6 +177,8 @@ github.com/nats-io/nats.go v1.9.1/go.mod h1:ZjDU1L/7fJ09jvUSRVBR2e7+RnLiiIQyqyzE
 github.com/nats-io/nkeys v0.1.0/go.mod h1:xpnFELMwJABBLVhffcfd1MZx6VsNRFpEugbxziKVo7w=
 github.com/nats-io/nkeys v0.1.3/go.mod h1:xpnFELMwJABBLVhffcfd1MZx6VsNRFpEugbxziKVo7w=
 github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
+github.com/nlpodyssey/gopickle v0.1.0 h1:9wjwRqXsOSYWZl4c4ko472b6RW+VB1I441ZcfFg1r5g=
+github.com/nlpodyssey/gopickle v0.1.0/go.mod h1:YIUwjJ2O7+vnBsxUN+MHAAI3N+adqEGiw+nDpwW95bY=
 github.com/oklog/oklog v0.3.2/go.mod h1:FCV+B7mhrz4o+ueLpx+KqkyXRGMWOYEvfiXtdGtbWGs=
 github.com/oklog/run v1.0.0/go.mod h1:dlhp/R75TPv97u0XWUtDeV/lRKWPKSdTuV0TZvrmrQA=
 github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo=

src/socket/decoder.go

@@ -0,0 +1,8 @@
+package socket
+
+// Py_builtins_str is used by the pickle decoder to parse the server response into a format Go can understand
+type Py_builtins_str struct{}
+
+func (c Py_builtins_str) Call(args ...interface{}) (interface{}, error) {
+	return args[0], nil
+}

src/socket/fail2banSocket.go

@@ -0,0 +1,134 @@
+package socket
+
+import (
+	"fmt"
+	"github.com/kisielk/og-rek"
+	"github.com/nlpodyssey/gopickle/types"
+	"net"
+	"strings"
+)
+
+type Fail2BanSocket struct {
+	socket  net.Conn
+	encoder *ogórek.Encoder
+}
+
+type JailStats struct {
+	FailedCurrent int
+	FailedTotal   int
+	BannedCurrent int
+	BannedTotal   int
+}
+
+func ConnectToSocket(path string) (*Fail2BanSocket, error) {
+	c, err := net.Dial("unix", path)
+	if err != nil {
+		return nil, err
+	}
+	return &Fail2BanSocket{
+		socket:  c,
+		encoder: ogórek.NewEncoder(c),
+	}, nil
+}
+
+func (s *Fail2BanSocket) Close() error {
+	return s.socket.Close()
+}
+
+func (s *Fail2BanSocket) Ping() (bool, error) {
+	response, err := s.sendCommand([]string{pingCommand, "100"})
+	if err != nil {
+		return false, newConnectionError(pingCommand, err)
+	}
+
+	if t, ok := response.(*types.Tuple); ok {
+		if (*t)[1] == "pong" {
+			return true, nil
+		}
+		return false, fmt.Errorf("unexpected response data (expecting 'pong'): %s", (*t)[1])
+	}
+	return false, newBadFormatError(pingCommand, response)
+}
+
+func (s *Fail2BanSocket) GetJails() ([]string, error) {
+	response, err := s.sendCommand([]string{statusCommand})
+	if err != nil {
+		return nil, err
+	}
+
+	if lvl1, ok := response.(*types.Tuple); ok {
+		if lvl2, ok := lvl1.Get(1).(*types.List); ok {
+			if lvl3, ok := lvl2.Get(1).(*types.Tuple); ok {
+				if lvl4, ok := lvl3.Get(1).(string); ok {
+					splitJails := strings.Split(lvl4, ",")
+					return trimSpaceForAll(splitJails), nil
+				}
+			}
+		}
+	}
+	return nil, newBadFormatError(statusCommand, response)
+}
+
+func (s *Fail2BanSocket) GetJailStats(jail string) (JailStats, error) {
+	response, err := s.sendCommand([]string{statusCommand, jail})
+	if err != nil {
+		return JailStats{}, err
+	}
+
+	stats := JailStats{
+		FailedCurrent: -1,
+		FailedTotal:   -1,
+		BannedCurrent: -1,
+		BannedTotal:   -1,
+	}
+
+	if lvl1, ok := response.(*types.Tuple); ok {
+		if lvl2, ok := lvl1.Get(1).(*types.List); ok {
+			if filter, ok := lvl2.Get(0).(*types.Tuple); ok {
+				if filterLvl1, ok := filter.Get(1).(*types.List); ok {
+					if filterCurrentTuple, ok := filterLvl1.Get(0).(*types.Tuple); ok {
+						if filterCurrent, ok := filterCurrentTuple.Get(1).(int); ok {
+							stats.FailedCurrent = filterCurrent
+						}
+					}
+					if filterTotalTuple, ok := filterLvl1.Get(1).(*types.Tuple); ok {
+						if filterTotal, ok := filterTotalTuple.Get(1).(int); ok {
+							stats.FailedTotal = filterTotal
+						}
+					}
+				}
+			}
+			if actions, ok := lvl2.Get(1).(*types.Tuple); ok {
+				if actionsLvl1, ok := actions.Get(1).(*types.List); ok {
+					if actionsCurrentTuple, ok := actionsLvl1.Get(0).(*types.Tuple); ok {
+						if actionsCurrent, ok := actionsCurrentTuple.Get(1).(int); ok {
+							stats.BannedCurrent = actionsCurrent
+						}
+					}
+					if actionsTotalTuple, ok := actionsLvl1.Get(1).(*types.Tuple); ok {
+						if actionsTotal, ok := actionsTotalTuple.Get(1).(int); ok {
+							stats.BannedTotal = actionsTotal
+						}
+					}
+				}
+			}
+			return stats, nil
+		}
+	}
+	return stats, newBadFormatError(statusCommand, response)
+}
+
+func newBadFormatError(command string, data interface{}) error {
+	return fmt.Errorf("(%s) unexpected response format - cannot parse: %v", command, data)
+}
+
+func newConnectionError(command string, err error) error {
+	return fmt.Errorf("(%s) failed to send command through socket: %v", command, err)
+}
+
+func trimSpaceForAll(slice []string) []string {
+	for i := range slice {
+		slice[i] = strings.TrimSpace(slice[i])
+	}
+	return slice
+}

src/socket/protocol.go

@@ -0,0 +1,65 @@
+package socket
+
+import (
+	"bufio"
+	"bytes"
+	"fmt"
+	"github.com/nlpodyssey/gopickle/pickle"
+)
+
+const (
+	commandTerminator    = "<F2B_END_COMMAND>"
+	pingCommand          = "ping"
+	statusCommand        = "status"
+	socketReadBufferSize = 1024
+)
+
+func (s *Fail2BanSocket) sendCommand(command []string) (interface{}, error) {
+	err := s.write(command)
+	if err != nil {
+		return nil, err
+	}
+	return s.read()
+}
+
+func (s *Fail2BanSocket) write(command []string) error {
+	err := s.encoder.Encode(command)
+	if err != nil {
+		return err
+	}
+	_, err = s.socket.Write([]byte(commandTerminator))
+	if err != nil {
+		return err
+	}
+	return nil
+}
+
+func (s *Fail2BanSocket) read() (interface{}, error) {
+	reader := bufio.NewReader(s.socket)
+
+	data := []byte{}
+	for {
+		buf := make([]byte, socketReadBufferSize)
+		_, err := reader.Read(buf)
+		if err != nil {
+			return nil, err
+		}
+		data = append(data, buf...)
+		containsTerminator := bytes.Contains(data, []byte(commandTerminator))
+		if containsTerminator {
+			break
+		}
+	}
+
+	bufReader := bytes.NewReader(data)
+	unpickler := pickle.NewUnpickler(bufReader)
+
+	unpickler.FindClass = func(module, name string) (interface{}, error) {
+		if module == "builtins" && name == "str" {
+			return &Py_builtins_str{}, nil
+		}
+		return nil, fmt.Errorf("class not found: " + module + " : " + name)
+	}
+
+	return unpickler.Load()
+}