How to prevent Brute Force Attacks Against your 3D-Printer - Fail2Ban

So you've installed Octoprint to control your 3D-printer in your browser. Maybe you even went a step further and decided to connect your Octoprint to the internet. Now you want to make sure that your printer is safe from hacker attacks.

Hardening Octoprint against Brute Force attacks (this is a hacker utilizing a script to try different passwords until she finally finds the right one to get inside your Octoprint) is not the most important measure against hackers. However, it is one you shouldn't leave out as you gain security at the cost of only 20 minutes of work.

What we will do
We will use a simple yet effective method to prevent Brute Force attacks: A user is allowed to try only a certain number of attempts (e.g. 5) to enter the correct password to connect to your Octoprint within a certain time frame (e.g. 10 minutes). If she doesn't succeed to do so, the user is banned from further attempts for another period of time (e.g. 15 minutes). In this example, an attacker cannot try more than 4 passwords every 10 minutes if she knows about your security measures (or not more than 5 passwords every 15 minutes if she doesn't know those measures). Provided you have a reasonably good password, this would effectively prevent a Brute Force attack.

To achieve all of this, we will use

  • OctoPrint version 1.3.12 or higher
  • HAProxy as reverse proxy (preinstalled on OctoPi)
  • RSyslog as logging server (also preinstalled on OctoPi)
  • Fail2Ban to realize the blocking
  • If you use OctoPi: OctoPi version 0.16.0 or higher

Prerequisite
The following prerequisites are essential to follow the instructions:

  1. You made the most sure that your 3D-printer hardware is safe against self ignition. Be aware that these machines can burn. This is dangerous enough even if you are near to fight the fire. But it is absolutely tragic when there is no one near to do so. Take a moment to think of the harm such a fire can do to other people (maybe your loved ones, or maybe your neighbors) who are not involved and have no chance to prevent it. Even if your 3D printer is not a cheap ripoff that has never seen a quality control, think of automatic solutions like a fire safe enclosure or a fire extinguishing ball.
  2. Make sure you relly need your 3D printer on the internet. Your 3D printer is much safer, if it's just on your LAN. Is the extra risk really worth it? Even if you answer "Yes", ask yourself if you have the skills to keep security standards on a responsible level. Remember, there are other ways to remotely access your printer apart from putting it on the internet - see here: OctoPrint.org - A Guide To Safe Remote Access of OctoPrint.
  3. Okay, you are still with me, so you decided to put your instance of OctoPi to the Internet. The most important step in terms of security is to put HTTP-Authentication "in front of" Octopi. Also, without HTTP-Authentication, Fail2Ban would not work. @foosel explains how to setup HTTP-Authentication here: How to enable HTTP authentication on haproxy.

Tutorial
Ok, let's start. This tutorial has three parts: First, we will configure RSyslog in order to get logs from HAProxy. Second, we configure HAProxy to use RSyslog and record connection attempts. Finally, we instruct Fail2Ban to crawl the logs for failed logins and ban IP addresses that failed too often.

1. Rsyslog
Use SSH to log into your Octoprint system. Make sure you have RSyslog installed by issuing

$ sudo apt install rsyslog

Go to /etc/rsyslog.d/ and see what's there

$ cd /etc/rsyslog.d/
$ ls

You will probably see a file 49-haproxy.conf. Without touching this one, we create a new one with a lower number

$ sudo nano 40-octoprint.conf

Put the following content inside

# Collect log with UDP
$ModLoad imudp
$UDPServerAddress 127.0.0.1
$UDPServerRun 514

# Logfile for Octoprint-Requests
local2.info /var/log/octoprint-requests.log

Of course, you can use another filename and another port but keep it in mind as we will need that later.

Restart RSyslog with

$ sudo service rsyslog restart

and make sure it is running with

$ sudo service rsyslog status

2. HAProxy
HAProxy should already be installed and configured for HTTP-Authentication (see prerequisites). So, we only need to tell HAProxy how to log.

Locate the HAProxy configuration file for Octoprint - most likely at /etc/haproxy/haproxy.cfg. You might want to create a backup of this file with

$ sudo cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg_old

Next, open /etc/haproxy/haproxy.cfg and add some statements (if they are not there already).

global
    [...]
    log 127.0.0.1:514 local2 info

With that statement you instruct HAProxy to send a log with log level info to port 514 which is monitored by RSyslog.

defaults
    log global
    option httplog
    [...]

Instead to putting this statement in defaults you could also put it into the corresponding frontend where HTTP-Authentication is configured (frontend public in most cases), but usually defaults is fine.

Restart haproxy

$ sudo service haproxy restart

From now on, Octoprint should log connection requests. To verify, visit your Octoprint in a browser and then check the logs

$ cat /var/log/octoprint-requests.log

3. Fail2Ban
Finally we start the magic. Install fail2ban with

$ sudo apt install fail2ban

Per default it is configured to only check ssh connections. Keep this in mind from now on: if you mistype your username or password a couple of times when you connect to your Octoprint you will get banned for about 10 minutes. Although I wouldn't suggest to expose your SSH-port to the internet anyway, I still think this is a good feature for a device connected to the internet. But if you don't want it, you will have to change it in the config file(s).

Let's go on and configure Fail2Ban to also monitor HTTP-Authentication attempts.

Go to /etc/fail2ban

$ cd /etc/fail2ban/

Make sure, there is a filter file for HAProxy

$ ls filter.d/haproxy-http-auth.conf

If not, you will need to update to a more recent version of fail2ban (or even update the underlying operating system).

Now we will make a configuration file for HAProxy

sudo nano jail.d/haproxy-http-auth.conf

(notice we are now in folder jail.d not in filter.d like before).

In this file, enter the following

[haproxy-http-auth]
# HAProxy by default doesn't log to file you'll need to set it up to forward
# logs to a syslog server which would then write them to disk.
# See "haproxy-http-auth" filter for a brief cautionary note when setting
# maxretry and findtime.
enabled = true
port = 80,8080,443
filter = haproxy-http-auth
logpath  = /var/log/octoprint-requests.log
maxretry = 5
findtime = 600
bantime = 900

What does that do?

  • enabled = true enables the rule we just created
  • port = 80,8080,443 defines, which ports should be closed in case of a ban - change or append this, if other ports are exposed
  • filter = haproxy-http-auth defines the filter rule by which Fail2Ban decides if a connection attempt failed. As we use HAProxy HTTP-Authentication haproxy-http-auth is fine (remember we checked if this rule exists before?)
  • logpath = /var/log/octoprint-requests.log tells Fail2Ban which log file to monitor for failed login attempts
  • maxretry = 6 how many login attempts before an IP address gets banned. Take care: Because of the way, HAProxy manages HTTP-Authentication, every user FIRST hits a failed login attempt BEFORE the login window even pops up. In other words, even successful logins will have at least one failed attempt. Therefore, maxretry = 6 really means 5 attempts.
  • findtime = 600 Fail2Ban looks back 600 seconds (10 minutes) for failed logins
  • bantime = 900 When an IP address gets banned, unban it after 900 seconds (15 minutes)

Finally, restart fail2ban with

$ sudo service fail2ban restart

All done. Now, let's check if it works.

Open a browser and enter the URL of your OctoPrint. A HTTP-Authentication should appear. Enter a wrong username and/or password for 5 times. The dialog shouldn't appear a sixth time, instead you should see an error page.

Now you would either have to wait for 15 minutes or instead connect to OctoPrint again using SSH and enter:

$ iptables -n -L

The IP address of the device that just got banned should be listed.

State

$ fail2ban-client set haproxy-http-auth unbanip <IP Address>

to unban the device.

Congratulations, Fail2Ban seems to work as expected.

What to keep in mind
While Fail2Ban is a good way to prevent Brute Force attacks from one attacking computer, this method wouldn't be able to defend against a bot network. So if somebody has access to a big number of devices spread accross the internet (all using different public IP addresses), to access your printer, you would need to take further measures to prevent that.

Further Readings

4 Likes

Thanks for the guide. I followed all your instructions and tripled checked everything (also noticed you spelt proxy wrong in "Next, open /etc/haporoxy/haproxy.cfg and add some statements") but fail2ban does not ban brute force attempts. Looking through my logs in /var/log/octoprint-requests.log, I suspect that the regex formula isn't capturing the failed login?

Here are what my logs look like:

Jan  8 04:29:01 localhost haproxy[21422]: ::ffff:2.136.144.51:28322 [08/Jan/2022:04:29:01.592] public public/<NOSRV> -1/-1/-1/-1/0 401 253 - - PR-- 5/5/0/0/0 0/0 "GET / HTTP/1.1"
Jan  8 04:29:03 localhost haproxy[21422]: ::ffff:2.136.144.51:2329 [08/Jan/2022:04:29:03.683] public public/<NOSRV> -1/-1/-1/-1/0 401 253 - - PR-- 5/5/0/0/0 0/0 "GET / HTTP/1.1"
Jan  8 04:29:05 localhost haproxy[21422]: ::ffff:2.136.144.51:48742 [08/Jan/2022:04:29:05.339] public public/<NOSRV> -1/-1/-1/-1/0 401 253 - - PR-- 5/5/0/0/0 0/0 "GET / HTTP/1.1"
Jan  8 04:29:07 localhost haproxy[21422]: ::ffff:2.136.144.51:5556 [08/Jan/2022:04:29:07.205] public public/<NOSRV> -1/-1/-1/-1/0 401 253 - - PR-- 4/4/0/0/0 0/0 "GET / HTTP/1.1"
Jan  8 04:29:07 localhost haproxy[21422]: ::ffff:2.136.144.51:19411 [08/Jan/2022:04:29:07.169] public octoprint/octoprint1 0/0/37/8/46 200 515 - - ---- 3/3/1/1/0 0/0 "GET /sockjs/info?t=1641587346220 HTTP/1.1"
Jan  8 04:29:08 localhost haproxy[21422]: ::ffff:2.136.144.51:20427 [08/Jan/2022:04:29:08.859] public public/<NOSRV> -1/-1/-1/-1/0 401 253 - - PR-- 5/5/0/0/0 0/0 "GET / HTTP/1.1"
Jan  8 04:29:10 localhost haproxy[21422]: ::ffff:2.136.144.51:45422 [08/Jan/2022:04:29:10.718] public public/<NOSRV> -1/-1/-1/-1/0 401 253 - - PR-- 5/5/0/0/0 0/0 "GET / HTTP/1.1"
Jan  8 04:29:12 localhost haproxy[21422]: ::ffff:2.136.144.51:26237 [08/Jan/2022:04:29:12.569] public public/<NOSRV> -1/-1/-1/-1/0 401 253 - - PR-- 5/5/0/0/0 0/0 "GET / HTTP/1.1"
Jan  8 04:29:14 localhost haproxy[21422]: ::ffff:2.136.144.51:16052 [08/Jan/2022:04:29:14.199] public public/<NOSRV> -1/-1/-1/-1/0 401 253 - - PR-- 5/5/0/0/0 0/0 "GET / HTTP/1.1"
Jan  8 04:29:16 localhost haproxy[21422]: ::ffff:2.136.144.51:12742 [08/Jan/2022:04:29:16.186] public public/<NOSRV> -1/-1/-1/-1/0 401 253 - - PR-- 6/6/0/0/0 0/0 "GET / HTTP/1.1"
Jan  8 04:29:17 localhost haproxy[21422]: ::ffff:2.136.144.51:57367 [08/Jan/2022:04:29:17.696] public public/<NOSRV> -1/-1/-1/-1/0 401 253 - - PR-- 5/5/0/0/0 0/0 "GET / HTTP/1.1"
Jan  8 04:29:18 localhost haproxy[21422]: ::ffff:2.136.144.51:21223 [08/Jan/2022:04:29:18.219] public public/<NOSRV> -1/-1/-1/-1/0 401 253 - - PR-- 5/5/0/0/0 0/0 "GET /favicon.ico HTTP/1.1"

I would really love your help with this. Thanks!

Sorry for replying so late.

I currently don't have an installation of Octoprint at hand anymore so I cannot reproduce possible mistakes at the moment. Maybe it is related to your network running in IPv6 mode?

I don't know if you will find the help you need in this community, but you could try to ask at https://discourse.haproxy.org/.

Please keep us updated about your findings. Also I can try to help you later, when I am around an Octoprint installation again.