Install ELK On Linux: A Complete Guide to Ubuntu, Debian, CentOS, RHEL, and so on.

Cloud computing and IT have come a long way, becoming the norm for most enterprises. One major reason behind their success is system monitoring. It prevents any IT system from going into overdrive and shutting down. It also provides much-needed data for future action and decision-making. Thus, monitoring systems have also become core to a business.

Even if you run your own system, you’d want a monitoring tool to handle potential problems. This is where ELK comes into play. ELK, or Elastic Stack, is a collection of monitoring tools and projects that can be very beneficial to an administrator. It is available on Windows, macOS, and Linux. Thus, here is a complete guide to ELK and how to install it on Linux.

Table Of Contents

What is The ELK Stack?

ELK, as many of you must already know, stands for a suite of monitoring tools: Elasticsearch, Logstash, and Kibana. Earlier, these used to be open source. However, with version 7.11, the company started offering premium licensing. Still, ELK is one of the best monitoring and data analysis tools available on the market. These three tools, together, are responsible for searching for data, sorting collected data, and in-depth analysis of said data.

ELK Stack

Elasticsearch

Elasticsearch is the search engine amongst the three that provides real-time search and analytics. It stores data in an index with quick search and retrieval functionality. Furthermore, it can aggregate data for trends and patterns. The tool is very fast and apt for projects that involve security analysis.

Logstash

Logistash collects data, transforms it, and sends it to Elasticsearch for analysis. It receives data and filters it. This data is collected from files, system logs, and log shippers called Beats. The tool receives these files, filters them, and formats them in a way that is possible for Elasticsearch to read.

Kibana

Finally, there is Kibana, which is responsible for visualizing the data received from Elasticsearch. Visualization is important as it helps the user grasp the data much faster. The tool comes equipped with Histograms, Line graphs, pie charts, sunbursts, and more.

Why is the ELK stack important?

But what is so great about ELK that so much emphasis is put on it? The thing is, it’s not only about ELK. Rather, today, competition has grown so much that any firm can’t afford downtime for slow servers, as it will harm them a lot. Thus, system monitoring tools have automatically become second nature for IT teams and projects. And ELK is simply a good tool that can help administrators with the monitoring of their systems.

It fulfills the need for a log management and analytical solution to monitor your system architecture. It is a monitoring solution for developers and DevOps engineers to gain insight into a system and take future actions based on concrete data.

Different Use Cases of ELK Stack

The following table clearly emphasizes how ELK is used:

Use Case	Description
Security	Investigates security threats by analyzing data from multiple sources.
Monitoring	Keeps a look at system metrics, app performance, and health in real time.
Compliance	Helps with compliance by providing data for audits.
Troubleshooting	Helps with troubleshooting as it keeps logs of the system.
DevOps	Provides collaboration between development and operations.
Business Intelligence	Provides insight into trends and usage patterns.

How To Install ELK on Linux?

For those new to the tool can find a comprehensive guide here that can help with installation. However, remember that these are separate tools and thus, need to be installed separately. For optimum results, install them in the order of E>L>K.

Install Elasticsearch

Installing Elasticsearch can be very simple, no matter the distro you are on. Here’s how you can do it:

Method 1. Using Native Linux Repositories

Elasticsearch is available as a .tar.gz archive. All you have to do is download it from the website and install it:

Step 1. Open the Terminal Window and download either the .tar.gz package.

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.13.1-linux-x86_64.tar.gz

Or,

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.13.1-linux-x86_64.tar.gz.sha512

Step 2. Install it:

shasum -a 512 -c elasticsearch-8.13.1-linux-x86_64.tar.gz.sha512 
tar -xzf elasticsearch-8.13.1-linux-x86_64.tar.gz
cd elasticsearch-8.13.1/

Step 3. Run Elasticsearch using the command:

./bin/elasticsearch

Method 2. Using a DEB Package (For both Debian and Ubuntu)

Similarly, you can use a DEB package for installing Elasticsearch. It can be used for installing on any Ubuntu or Debian distro.

Step 1. First, download and install the Elasticsearch PGP Key:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch 
sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

Solution 2A. Using the APT Repo

Step 2A. Then, install the APT transport HTTPS package:

sudo apt-get install apt-transport-https

Step 3A. Now, save the repository definition to /etc/apt/sources.list.d/elastic-8.x.list:

echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" 
sudo tee /etc/apt/sources.list.d/elastic-8.x.list

Step 4A. Now, install the Elasticsearch package

sudo apt-get update && sudo apt-get install elasticsearch

Note: If two entries exist for the same Elasticsearch repository, you will see an error like this during apt-get update:

Duplicate sources.list entry https://artifacts.elastic.co/packages/8.x/apt/ ...`
Examine /etc/apt/sources.list.d/elasticsearch-8.x.list for the duplicate entry or locate the duplicate entry amongst the files in /etc/apt/sources.list.d/ and the /etc/apt/sources.list file.

Solution 2B. Manual Installation

Step 2B. You can also manually download the package from the website:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.13.1-amd64.deb
Wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.13.1-amd64.deb.sha512
shasum -a 512 -c elasticsearch-8.13.1-amd64.deb.sha512 
sudo dpkg -i elasticsearch-8.13.1-amd64.deb

Method 3. Install ELK On Linux Using RPM Repositories

You can also use the RPM repo or download the RPM package from the official website to install Elasticsearch on systems like OpenSUSE, CentOS, Red Hat, and other RPM-based distros.

Note: RPM install is not supported on distributions with old versions of RPM, such as SLES 11 and CentOS 5.

Step 1. First, download and install the GPG key

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Solution 3A. Installing via RPM Repo

Step 2Aa. Install For Older Red Hat and Cent OS, using:

sudo yum install --enablerepo=elasticsearch elasticsearch

Step 2A.b. Install For Fedora and Newer Red Hat

sudo dnf install --enablerepo=elasticsearch elasticsearch

Step 2A.c. Install For OpenSUSE

sudo zypper modifyrepo --enable elasticsearch
sudo zypper install elasticsearch;
sudo zypper modifyrepo --disable elasticsearch

Solution 3B. Manual Installation

Step 2B. Download and install directly from the website:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.13.1-x86_64.rpm
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.13.1-x86_64.rpm.sha512
shasum -a 512 -c elasticsearch-8.13.1-x86_64.rpm.sha512 
sudo rpm --install elasticsearch-8.13.1-x86_64.rpm

Method 4. Install ELK On Linux Using Docker

You can also use Docker to install Elasticsearch, Docker images are available for Elasticsearch in the registry.

Step 1. First, set up Docker’s apt repository. Start by adding the official GPG Key:

sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

Step 2. Add the repo to APT

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

Step 3. Now, install the Docker packages.

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Step 4. After that, create a new docker network:

docker network create elastic

Step 5. Add the Elasticsearch Docker image.

docker pull docker.elastic.co/elasticsearch/elasticsearch:8.13.1

Step 6. Then, start an Elasticsearch container.

docker run --name es01 --net elastic -p 9200:9200 -it -m 1GB docker.elastic.co/elasticsearch/elasticsearch:8.13.1

Step 7. Now, copy the generated elastic password and enrollment token.

Step 8. All you need to do now is to copy the http_ca.crt SSL certificate from the container to your local machine, using the following code:

docker cp es01:/usr/share/elasticsearch/config/certs/http_ca.crt.

Step 9. Finally, make a REST API call to ensure the Elasticsearch container is running.

curl --cacert http_ca.crt -u elastic:$ELASTIC_PASSWORD https://localhost:9200

Install Logstash

Before proceeding, note that the website offers repositories for APT and YUM-based distributions only.

Method 1. Install ELK Using the APT repository

Step 1. Download and install the PSK:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elastic-keyring.gpg

Step 2A. Those using Debian need to install the apt-transport-https package

sudo apt-get install apt-transport-https

Step 3. Save the repository to /etc/apt/sources.list.d/elastic-8.x.list:

echo "deb [signed-by=/usr/share/keyrings/elastic-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-8.x.list

Step 4. Update the APT-GET repo:

sudo apt-get update

Step 5. Finally, install it:

sudo apt-get update && sudo apt-get install logstash

Method 2. Using the YUM repository

Step 1. Again, start by downloading the PSK:

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Step 2. Then, add it in your /etc/yum.repos.d/ directory as a .repo file containing the following information: (preferably by the name logstash.repo)

[logstash-8.x]

name=Elastic repository for 8.x packages

baseurl=https://artifacts.elastic.co/packages/8.x/yum

gpgcheck=1

gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch

enabled=1

autorefresh=1

type=rpm-md

Step 3. Save the file and run it using:

sudo yum install logstash

Installing Kibana

Lastly, you have to install Kibana on your system to complete the ELK Stack.

Method 1. Install ELK On Linux Using the Native Repositories

Just like Elasticsearch, Kibana is also available on the website as a .tar.gz package.

Step 1. So start by downloading and installing Kibana directly:

curl -O https://artifacts.elastic.co/downloads/kibana/kibana-8.13.1-linux-x86_64.tar.gz
curl https://artifacts.elastic.co/downloads/kibana/kibana-8.13.1-linux-x86_64.tar.gz.sha512 | shasum -a 512 -c - 
tar -xzf kibana-8.13.1-linux-x86_64.tar.gz
cd kibana-8.13.1/

Method 2. Building ELK From Source

Step 1. Download and install the PSK:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

Solution 2A. Using APT Repo

Step 2A. Now, install the APT transport HTTPS package

sudo apt-get install apt-transport-https

Step 3A. Save the repository to /etc/apt/sources.list.d/elastic-8.x.list:

echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" 
sudo tee /etc/apt/sources.list.d/elastic-8.x.list

Step 4A. Finally, install the Kibana package:

sudo apt-get update && sudo apt-get install kibana

Solution 2B. Manual Download and Installation

Step 2B. Or, you can directly download and install it:

wget https://artifacts.elastic.co/downloads/kibana/kibana-8.13.1-amd64.deb
shasum -a 512 kibana-8.13.1-amd64.deb 
sudo dpkg -i kibana-8.13.1-amd64.deb

Method 3. Install ELK Using The RPM Repository

Then, just like Elasticsearch, use RPM repo for Kibana on systems that run OpenSUSE, SLES, Red Hat, and Oracle

Step 1. Download and install the public signing key:

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Step 2. Now, create a file named kibana.repo in the /etc/yum.repos.d/ directory for Red Hat-based distributions and in the /etc/zypp/repos.d/ directory for openSUSE-based distributions

[kibana-8.x]

name=Kibana repository for 8.x packages