Installation of PFELK on ubuntu. ELK for pfSense

As per my promise or I can say mention of pfSense installation I am presenting the installation guide. By the way, I just copied every step from the GitHub repository document if anyone is wondering. Steps given in the official documentation are perfect and straight forward. I honestly rewrote it because I was running out of ideas and I promised it in the previous post. 

But one thing I would say that if you are a beginner like me don’t use scripted install, do it manually. It is good to know what components are being installed to get it done.

The one important thing I would say about ELK (btw, I have told this many times before but I need content, JK ;)) is that it is a base for so many SIEM and cannot be neglected from learning it from the base. You can customize it so much to basically make it look like you :). Now enough of these fillers let’s get straight into the content.

Note: This is tested in ubuntu

  • Let’s add Oracle Java Repository
sudo add-apt-repository ppa:linuxuprising/java
  • Adding Maxmind repository
sudo add-apt-repository ppa:maxmind/ppa
  • Download and install the public GPG signing key
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
  • Install apt-transport-https package
sudo apt-get install apt-transport-https
  • Add ELK repository
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
  • Update the repository
sudo apt-get update
  • Install Java 13
sudo apt-get install oracle-java13-installer
  • Install Maxmind GeoIP database provider package
sudo apt install geoipupdate
  • Configuring Maxmind
    • Create Maxmind account by going to following link https://www.maxmind.com/en/geolite2/signup
    • Login to your Max Mind Account; navigate to “My License Key” under “Services” and Generate new license key
      • If asked create a config file. Select one that will create one.
  • Now edit the config file for max mind
sudo nano /etc/GeoIP.conf
  • Modify lines 7 & 8 as follows (Replace 000 with your number)
AccountID 0000
LicenseKey 0000
  • Modify line 13 as follows
EditionIDs GeoLite2-City GeoLite2-Country GeoLite2-ASN
  • Download the GeoIP database
sudo geoipupdate -d /usr/share/GeoIP/
  • Add cron job to update GeoIP database
    • Create a new cron job file
sudo nano /etc/cron.weekly/geoipupdate
  • Add following line into the file
0 1 * * * geoipupdate -d /usr/share/GeoIP
  • Let’s get to the installation of ElasticStack
sudo apt-get install elasticsearch && sudo apt-get install kibana && sudo apt-get install logstash
  • Edit Kibana Configuration
sudo nano /etc/kibana/kibana.yml
  • Modify the following items
server.port: 5601
server.host: "0.0.0.0"
  • Change the directory add logstash configuration
cd /etc/logstash/conf.d
  • Download configuration files for logstash parsing and filtering
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/01-inputs.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/05-firewall.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/30-geoip.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/50-outputs.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/10-others.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/20-suricata.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/25-snort.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/35-rules-desc.conf
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/45-cleanup.conf
  • Make new patterns folder and navigate
sudo mkdir /etc/logstash/conf.d/patterns
cd /etc/logstash/conf.d/patterns/
  • Download pattern config file
sudo wget https://raw.githubusercontent.com/3ilson/pfelk/master/etc/logstash/conf.d/patterns/pfelk.grok
  • Get back to the previous folder and open 01-inputs.conf file in a text editor
cd ..
nano 01-inputs.conf
  • Edit the following things in the file
Change line 9; the "if [host] =~ ..." should point to your pfSense/OPNsense IP address
Change line 12-16; (OPTIONAL) to point to your second PF IP address or ignore

If you are using pfSense uncommit line 28 and commit out line 25
or if using OPNsense uncommit line 25 and commit out line 28
  • Disable Swap to increase performance
sudo swapoff -a
  • Set current time zone
sudo timedatectl set-timezone EST
  • Configure service to start on boot and restart the services
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service
sudo /bin/systemctl enable kibana.service
sudo /bin/systemctl enable logstash.service

systemctl start elasticsearch 
systemctl start kibana 
systemctl start logstash

This was all server-side configuration. Let’s configure the pfSense firewall

  • In pfSense navigate to Status->System Logs, then click on Settings.
  • In OPNsense navigate to System->Settings->Logging
  • At the bottom check “Enable Remote Logging”
  • (Optional) Select a specific interface to use for forwarding
  • Enter the ELK local IP into the field “Remote log servers” with port 5140 (eg 192.168.100.50:5140)
  • Under “Remote Syslog Contents” check “Everything”
  • Click Save

Now we need to define index pattern to get the data to in the kibana. It is generally used for the segregation of logs.

  • In your web browser go to the ELK local IP using port 5601 (ex: 192.168.0.1:5601) 
  • Click the gear icon (management) in the lower left
  • Click Kibana -> Index Patters
  • Click Create New Index Pattern
  • Type “pf-*” into the input box, then click Next Step
  • Now select Timestamp

Help on index pattern defining. https://www.elastic.co/guide/en/kibana/current/index-patterns.html

You can make your own dashboard as per your needs but you use the following link to get json files to import the dashboard. https://github.com/3ilson/pfelk/tree/master/Dashboard/v5.5

  • In your web browser go to the ELK local IP using port 5601 (ex: 192.168.0.1:5601)
  • Click Management -> Saved Objects
  • You can import the dashboards found in the Dashboard folder via the Import bottom in the top-right corner.

That it, for now, you are ready to use the Elastic stack for pfSense firewall.

All credit: https://github.com/a3ilson/pfelk

5 thoughts on “Installation of PFELK on ubuntu. ELK for pfSense”

  1. Thanks for the review!

    Although since posting, we made a number of changes and most of the referenced links no longer work. However, the referenced credit link contains the latest and greatest!

    -Andrew

    Reply
    • Thank you for giving me your feedback. I realized there are so many things that needed more description. Here are some explanations for feedback.

      1. To download please follow wget commands given in the post to download the configuration files for Logstash.
      2. These tweaks are optional so they are not included in the post. But you can refer my OSSEC and ELK to learn about elasticsearch tweaks and logstash working. https://snehpatel.com/index.php/getting-ossec-server-logs-into-elk-stack/
      3. My post hasn’t been updated and the link has become irrelevant. In that section, I am trying to explain things about importing the dashboard. you can try the following link to get the dashboard and import it. https://github.com/3ilson/pfelk/tree/master/Dashboard/v5.5

      I will update the post to make it more clear.
      But for now if you need help this this please feel free to email me on x786@protonmail.ch

      Reply

Leave a Comment