The three opensource projects Elasticsearch, Logstash, and Kibana make up the ELK Stack. All of these are maintained by Elastic. Elasticsearch is an open source, full-text search and analysis engine. It stores the logs. Logstash is a log-aggregator, that ingests data from multiple sources. It ships, process and store logs. Kibana is a visualization layer that works on top of Elasticsearch. Elastic added another open source technology Beats. Moreover Elastic now refers to the stack as the Elastic Stack.
Use Cases of ELK Stack
- Netflix uses ELK, to monitor and analyse customer service operation’s security log.
- LinkedIn uses ELK to monitor performance and security.
- Tripwire uses ELK to support information packet log analysis.
- Medium use ELK to debug their production issues.
Steps to install ELK Stack
Step 1- Install Java
The ELK requires Java 8. However some components are compatible with Java 9, but not Logstash.
sudo apt-get install openjdk-8-jdk
To check the java version
Step 2- Install Nginx
Nginx is open source software for web serving, reverse proxying, caching, load balancing, media streaming etc. Therefore we need to have it installed.
sudo apt-get install nginx
Step 3- Add Elastic Repository
Elastic repositories enable access to all the open-source software in the ELK stack. Therefore to add repository we need to import the GPG key.
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
After that we will get an OK response.
Next, install the apt-transport-https package:
sudo apt-get install apt-transport-https
After that, add the Elastic repository to your system’s repository list.
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee –a /etc/apt/sources.list.d/elastic-7.x.list
Step 4- Install Elasticsearch
Firstly, update the repositories before installing the ElasticSearch
sudo apt-get update
After that, install the ElasticSearch
sudo apt-get install elasticsearch
Configure Elastic Search for ELK Stack
Elasticsearch uses a configuration file to control how it behaves. Edit the file using nano-
sudo nano /etc/elasticsearch/elasticsearch.yml
Now scroll down to find the following lines and uncomment them. By removing the # from front of network.host and http.port-
After that replace the 126.96.36.199 with ‘localhost’
Moreover below the Discovery section. Add one more line, as we are configuring a single node cluster:
Start the ElasticSearch
So start the Elasticsearch service by running a
sudo systemctl start elasticsearch.service
Enable Elasticsearch to start on boot-
sudo systemctl enable elasticsearch.service
So now to test ElasticSearch-
curl -X GET "localhost:9200"
The next part will cover the installation of kibana and logstash and monitoring the data.