Fundamentals of ELK Stack with Demo – Part 1

Reading Time: 3 minutes

The three opensource projects Elasticsearch, Logstash, and Kibana make up the ELK Stack. All of these are maintained by Elastic. Elasticsearch is an open source, full-text search and analysis engine. It stores the logs. Logstash is a log-aggregator, that ingests data from multiple sources. It ships, process and store logs. Kibana is a visualization layer that works on top of Elasticsearch. Elastic added another open source technology Beats. Moreover Elastic now refers to the stack as the Elastic Stack.

Use Cases of ELK Stack

• Netflix uses ELK, to monitor and analyse customer service operation’s security log.
• LinkedIn uses ELK to monitor performance and security.
• Tripwire uses ELK to support information packet log analysis.
• Medium use ELK to debug their production issues.

Steps to install ELK Stack

Step 1- Install Java

The ELK requires Java 8. However some components are compatible with Java 9, but not Logstash.

sudo apt-get install openjdk-8-jdk

To check the java version

java -version

Step 2- Install Nginx

Nginx is open source software for web serving, reverse proxying, caching, load balancing, media streaming etc. Therefore we need to have it installed.

sudo apt-get install nginx

Step 3- Add Elastic Repository

Elastic repositories enable access to all the open-source software in the ELK stack. Therefore to add repository we need to import the GPG key.

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

After that we will get an OK response.

Next, install the apt-transport-https package:

sudo apt-get install apt-transport-https

After that, add the Elastic repository to your system’s repository list.

echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee –a /etc/apt/sources.list.d/elastic-7.x.list

Step 4- Install Elasticsearch

Firstly, update the repositories before installing the ElasticSearch

sudo apt-get update

After that, install the ElasticSearch

sudo apt-get install elasticsearch

Configure Elastic Search for ELK Stack

Elasticsearch uses a configuration file to control how it behaves. Edit the file using nano-

sudo nano /etc/elasticsearch/elasticsearch.yml

Now scroll down to find the following lines and uncomment them. By removing the # from front of network.host and http.port-

#network.host: 192.168.0.1
#http.port: 9200

After that replace the 192.198.0.1 with ‘localhost’

Moreover below the Discovery section. Add one more line, as we are configuring a single node cluster:

discovery.type: single-node

Start the ElasticSearch

So start the Elasticsearch service by running a systemctl command-

sudo systemctl start elasticsearch.service

Enable Elasticsearch to start on boot-

sudo systemctl enable elasticsearch.service

So now to test ElasticSearch-

curl -X GET "localhost:9200"

The next part will cover the installation of kibana and logstash and monitoring the data.

References-

https://www.chaossearch.io/blog/elk-stack-pros-and-cons