If you are managing a farm of clusters with a common mission, for example, a set VoIP cluster or a Web Hosting farm, One of the hardest things is the repetitive management work. In a 100-server cluster environment, when an attacker hits one node, eventually that attacker will get to another node and continue the attack. One of the biggest exposures here is that usually (not always), cluster's nodes share a common database. Hitting a winning vulnerability it is just a matter of time for the other peers.
With all this said, I will explain my approach that tries to fix this situation. At the end of this reading, you will understand how to have a proactive secured environment.
First, some security concepts.
In the security field, a honeypot is a security concept that homologates to the image of a bear eating honey. The bear is so busy eating the honey that he doesn't pay attention to anything else. With this said, an attacker will keep his activity on a server that is designed to be unprotected.
Some authors instead of comparing against a bear, use files. Flies got stuck because of the honey thickness and they won't turn to the next server. Anyway, any of those analogies matches what a honeypot is for.
Honeypots are also used to gather intelligence about new attack techniques, but in this case, we are just using it for its very first purpose: keeping attackers busy and disclosing them.
Fail2ban is a python project that phrases logs through regular expressions (filters) and applies a configured action when it matches. Usually, those actions are firewall related actions. Fail2ban configuration is based on jails. A jail is a set of parameters where you specify the filter, the source log, the action, the quarantine time and other parameters.
Fail2ban by itself operates at a reactive approach. The attacker needs to fail n times before getting blocked (usually 3). But if we build a fail2ban cluster, the reactiveness of fail2ban could be converted to proactiveness. And this is what we want to accomplish.
In a given scenario, you have several servers. They could be part of the same cluster (such as VoIP or Web) or each one may have its own mission. You optionally have a honeypot server. An attacker will eventually find that honeypot (which is easier since it is unprotected on purpose) and any other production server with an open port to the Internet.
Each production server has a fail2ban daemon and a rsyslog daemon running on it, but what it makes this interesting is that they also report their actions to a central fail2ban cluster server. The mission of this server is to store all the information sent by the production servers' fail2ban and inform the others about a confirmed offending IP. Proactively, the other servers will block that attacker.
In this example, we have two attackers. One is kept in the honeypot server, the other is trying to hack through an email server. The honeypot server won't block the attacker; the mail server will eventually block the attacker. In both cases, the two servers will inform the fail2ban cluster server about the attempts.
In a matter of seconds, that central server will inform the rest of the servers to block that offending IP. When the attacker moves on from the mail server and it arrives at the next one (PBX server), that IP will be already blocked and any attempt of connectivity will just be dropped.
This proactive approach will keep safe your servers.
These days VPS'es companies such as Digital Ocean or Vultr are very common to be the host of many companies' core servers. It is a win-win using them, they provide excellent service at a low price. However, have you realized that their autonomous system is well-known? Because of this, and the way the hacking scripts work if they start attacking IP 18.104.22.168 and your server might have 22.214.171.124 they will eventually get to your server. It is just a matter of time.
If you use one of these VPS providers to host the core of your business, for example, a VoIP cluster. Hacking one server usually means hacking them all. Cluster's usually share a common database. It is easier to try a winning username - password combination than a blind brute force.
So, let's just agree that having this is better than just reacting.
I can think about the following:
I am pretty sure there are more scenarios.
The easiest way is by hiring me as your consultant. I will deal with the configurations to make this happens. However, if you have enough IT skills and you just need a guide, you just can visit the Bitbucket address in this article. I will be updating it.
The software has two components. A Rsyslogd configuration that reports the fail2ban.log activity to the central node and a script that sends commands to the nodes. Later, when I have finished all, I will publish an RPM for those CentOS fans out there.
Good luck!blog comments powered by Disqus