Elastic SIEM 7 Part I - Install and Configure Elasticsearch and Kibana on Debian Linux

Elastic SIEM Architecture

Elasticsearch is a real-time, distributed storage, search, and analytics engine.

Kibana is an open source analytics and visualization platform designed to work with Elasticsearch. Kibana will be used to search, view, and interact with data stored in Elasticsearch indices.

Notes

Versions

• OS : Debian 10
• Elasticsearch : 7

Links

• Download link : https://www.elastic.co/
• Official guide : https://www.elastic.co/guide/
• Elastic Stack Editions https://www.elastic.co/subscriptions

Components

Main

• Elasticsearch : Distributed, RESTful search and analytics.
• Kibana : Visualize your data. Navigate the Stack.
• Beats : Collect, parse, and ship in a lightweight fashion.

Others

• Logstash : Ingest, transform, enrich, and output.
• Filebeat : Real-time insight into log data.

Ports

• ElastiSearch default port : http://IP_ADDRESS:9200
• Kibana web access : http://IP_ADDRESS:5601
• Logstash default port : 9600

Installing Elasticsearch (Debian Server)

We can install it, on a Debian system from raw packages or via Package Managers. We will see both methods.

Installing From raw packages (dpkg)

Elasticsearch

• Download Elasticsearch package and the associate sha512 message digest to /tmp/ :

root@host:~# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.12.0-amd64.deb https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.12.0-amd64.deb.sha512 -P /tmp/

• Install Elasticsearch service :

root@host:~# dpkg -i /tmp/elasticsearch-7.12.0-amd64.deb

Kibana

• Download Kibana package and the associate sha512 message digest to /tmp/ :

root@host:~# wget https://artifacts.elastic.co/downloads/kibana/kibana-7.12.0-amd64.deb https://artifacts.elastic.co/downloads/kibana/kibana-7.12.0-amd64.deb.sha512 -P /tmp/

• Install Kibana service :

root@host:~# dpkg -i /tmp/kibana-7.12.0-amd64.deb

Installing From Package Managers (apt)

Prerequisites

• Import the Elasticsearch PGP key :

root@host:~# wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -

• Install apt-transport-https :

root@host:~# apt update && apt-get install apt-transport-https

• Save the repository definition :

root@host:~# echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | tee /etc/apt/sources.list.d/elastic-7.x.list

Elasticsearch

• Install Elasticsearch :

root@host:~# apt update && apt-get install elasticsearch

Kibana

• Install Kibana :

root@host:~# apt update && apt-get install kibana

Configuring

Elasticsearch

• Edit /etc/elasticsearch/elasticsearch.yml to :
  • Set the bind address to all : 0.0.0.0
  • Declare one node configuration : single-node
  • List of hosts : ["0.0.0.0"]

network.host: 0.0.0.0
discovery.type: single-node
discovery.seed_hosts: ["0.0.0.0"]

• Still in file configuration we can also set the name for the node :

node.name: std

• Start service :

root@host:~# systemctl start elasticsearch.service

• Check elastic service is running :

root@host:~# curl http://127.0.0.1:9200

{
  "name" : "std",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "StdgreaTBanDKphU4S0ceg",
  "version" : {
    "number" : "7.12.0",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "ff17057114c2199c9c1bbecc727003a907c0db7a",
    "build_date" : "2021-02-15T13:44:09.394032Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Kibana

• Edit /etc/kibana/kibana.yml to :
  • Set the bind address to all : 0.0.0.0

server.host: "0.0.0.0"

• Start service :

root@host:~# systemctl start kibana.service

• Check everything is fine :

root@host:~# curl -XGET 'http://localhost:9200/_cluster/health'

{"cluster_name":"elasticsearch","status":"yellow","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":18,"active_shards":18,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":4,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":81.81818181818183}

• Open Firefox and go to http://IP_SERVER:5601/, after few seconds, you should be able to see this web page :

AutoStart

To make our Elasticsearch and Kibana services start at boot, we need to do some modifications.

Elasticsearch

• Because it's slow to start, edit /usr/lib/systemd/system/elasticsearch.service file, and change the TimeoutStartSec value :

TimeoutStartSec=300

• Set Elasticsearch service start when the server boots :

root@host:~# systemctl enable elasticsearch.service

Kibana

• Set Kibana service start when the server boots :

root@host:~# systemctl enable kibana.service

Log Files

• Elasticsearch :

root@host:~# tail /var/log/elasticsearch/elasticsearch.log

• Kibana :

root@host:~# tail /var/log/kibana/kibana.log