Installation of PFELK on ubuntu. ELK for pfSense

As per my promise or I can say mention of pfSense installation I am presenting the installation guide. By the way, I just copied every step from the GitHub repository document if anyone is wondering. Steps given in the official documentation are perfect and straight forward. I honestly rewrote it because I was running out of ideas and I promised it in the previous post. 

But one thing I would say that if you are a beginner like me don’t use scripted install, do it manually. It is good to know what components are being installed to get it done.

The one important thing I would say about ELK (btw, I have told this many times before but I need content, JK ;)) is that it is a base for so many SIEM and cannot be neglected from learning it from the base. You can customize it so much to basically make it look like you :). Now enough of these fillers let’s get straight into the content.

Note: This is tested in ubuntu

• Let’s add Oracle Java Repository

sudo add-apt-repository ppa:linuxuprising/java

• Adding Maxmind repository

sudo add-apt-repository ppa:maxmind/ppa

• Download and install the public GPG signing key

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

• Install apt-transport-https package

sudo apt-get install apt-transport-https

• Add ELK repository

echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list

• Update the repository

sudo apt-get update

• Install Java 13

sudo apt-get install oracle-java13-installer

• Install Maxmind GeoIP database provider package

sudo apt install geoipupdate

• Configuring Maxmind
  • Create Maxmind account by going to following link https://www.maxmind.com/en/geolite2/signup
  • Login to your Max Mind Account; navigate to “My License Key” under “Services” and Generate new license key
    • If asked create a config file. Select one that will create one.
• Now edit the config file for max mind

sudo nano /etc/GeoIP.conf

• Modify lines 7 & 8 as follows (Replace 000 with your number)

AccountID 0000
LicenseKey 0000

• Modify line 13 as follows

EditionIDs GeoLite2-City GeoLite2-Country GeoLite2-ASN

• Download the GeoIP database

sudo geoipupdate -d /usr/share/GeoIP/

• Add cron job to update GeoIP database
  • Create a new cron job file

sudo nano /etc/cron.weekly/geoipupdate

• Add following line into the file

0 1 * * * geoipupdate -d /usr/share/GeoIP

• Let’s get to the installation of ElasticStack

sudo apt-get install elasticsearch && sudo apt-get install kibana && sudo apt-get install logstash

• Edit Kibana Configuration

sudo nano /etc/kibana/kibana.yml

• Modify the following items

server.port: 5601
server.host: "0.0.0.0"

• Change the directory add logstash configuration

cd /etc/logstash/conf.d

• Download configuration files for logstash parsing and filtering

sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/01-inputs.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/05-firewall.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/30-geoip.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/50-outputs.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/10-others.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/20-suricata.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/25-snort.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/35-rules-desc.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/45-cleanup.conf

• Make new patterns folder and navigate

sudo mkdir /etc/logstash/conf.d/patterns
cd /etc/logstash/conf.d/patterns/

• Download pattern config file

sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/patterns/pfelk.grok

• Get back to the previous folder and open 01-inputs.conf file in a text editor

cd ..
nano 01-inputs.conf

• Edit the following things in the file

Change line 9; the "if [host] =~ ..." should point to your pfSense/OPNsense IP address
Change line 12-16; (OPTIONAL) to point to your second PF IP address or ignore

If you are using pfSense uncommit line 28 and commit out line 25
or if using OPNsense uncommit line 25 and commit out line 28

• Disable Swap to increase performance

sudo swapoff -a

• Set current time zone

sudo timedatectl set-timezone EST

• Configure service to start on boot and restart the services

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service
sudo /bin/systemctl enable kibana.service
sudo /bin/systemctl enable logstash.service

systemctl start elasticsearch 
systemctl start kibana 
systemctl start logstash

This was all server-side configuration. Let’s configure the pfSense firewall

• In pfSense navigate to Status->System Logs, then click on Settings.
• In OPNsense navigate to System->Settings->Logging
• At the bottom check “Enable Remote Logging”
• (Optional) Select a specific interface to use for forwarding
• Enter the ELK local IP into the field “Remote log servers” with port 5140 (eg 192.168.100.50:5140)
• Under “Remote Syslog Contents” check “Everything”
• Click Save

Now we need to define index pattern to get the data to in the kibana. It is generally used for the segregation of logs.

• In your web browser go to the ELK local IP using port 5601 (ex: 192.168.0.1:5601) 
• Click the gear icon (management) in the lower left
• Click Kibana -> Index Patters
• Click Create New Index Pattern
• Type “pf-*” into the input box, then click Next Step
• Now select Timestamp

Help on index pattern defining. https://www.elastic.co/guide/en/kibana/current/index-patterns.html

You can make your own dashboard as per your needs but you use the following link to get json files to import the dashboard. https://github.com/3ilson/pfelk/tree/master/Dashboard/v5.5

• In your web browser go to the ELK local IP using port 5601 (ex: 192.168.0.1:5601)
• Click Management -> Saved Objects
• You can import the dashboards found in the Dashboard folder via the Import bottom in the top-right corner.

That it, for now, you are ready to use the Elastic stack for pfSense firewall.

All credit: https://github.com/a3ilson/pfelk