How to Install Graylog on Ubuntu 16.04

By Ghulam Qadir, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud’s incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

Graylog is a powerful open-source log management platform that aggregates and extracts important data from server logs, which are often sent using the Syslog protocol. It also allows you to search and visualize the logs in a web interface.

Graylog is compatible and works well with Alibaba Cloud Elastic Compute Service (ECS) instances. In this tutorial, we’ll install and configure Graylog on Ubuntu 16.04, and set up a simple input that receives system logs.

Prerequisites

  1. You must have Alibaba Cloud Elastic Compute Service (ECS) activated and verified your valid payment method. If you are a new user, you can get a free account in your Alibaba Cloud account. If you don’t know about how to setup your ECS instance, you can refer to this tutorial or quick-start guide.
  2. You should set up your server’s hostname.
  3. Access to VNC console in your Alibaba Cloud or SSH client installed in your PC.

Your ECS Ubuntu 16.04 server must have at least 2 GB of RAM, private networking enabled, and a non-root user set up.

After completing the prerequisites, log in as root user with your root username & password via SSH client (e.g. Putty) or VNC console available in your Alibaba Cloud account dashboard.

Before you installing Graylog, you’ll need:

  1. Oracle JDK 8 installed, which you can do by following the “Installing the Oracle JDK” section of this Java installation article.
  2. Elasticsearch 2.x, which you can install by following Steps 1 and 2 of the Elasticsearch installation tutorial. Certain versions of Graylog only work with certain versions of Elasticsearch. For example, Graylog 2.x does not work with Elasticsearch 5.x. Refer to this Graylog-Elasticsearch version comparison table for the exact version. This tutorial uses Elasticsearch 2.4.4 and Graylog 2.2.
  3. MongoDB, which can be installed by following the MongoDB tutorial.

Installing Default JRE/JDK

The easiest option for installing Java is using the version packaged with Ubuntu. Specifically, this will install OpenJDK 8, the latest and recommended version.

First, update the package index.

Next, install Java. Specifically, this command will install the Java Runtime Environment (JRE).

There is another default Java installation called the JDK (Java Development Kit). The JDK is usually only needed if you are going to compile Java programs or if the software that will use Java specifically requires it.

The JDK does contain the JRE, so there are no disadvantages if you install the JDK instead of the JRE, except for the larger file size.

You can install the JDK with the following command:

Installing Oracle JDK

If you want to install the Oracle JDK, which is the official version distributed by Oracle, you will need to follow a few more steps.

First, add Oracle’s PPA, then update your package repository.

Then, depending on the version you want to install, execute one of the following commands:

Oracle JDK 8

This is the latest stable version of Java at time of writing, and the recommended version to install. You can do so using the following command:

Oracle JDK 9

This is a developer preview and the general release is scheduled for March 2017. It’s not recommended that you use this version because there may still be security issues and bugs. There is more information about Java 9 on the official JDK 9 website.

To install JDK 9, use the following command:

Managing Java

There can be multiple Java installations on one server. You can configure which version is the default for use in the command line by using update-alternatives, which manages which symbolic links are used for different commands.

The output will look something like the following. In this case, this is what the output will look like with all Java versions mentioned above installed.

You can now choose the number to use as a default. This can also be done for other Java commands, such as the compiler (javac), the documentation generator (javadoc), the JAR signing tool (jarsigner), and more. You can use the following command, filling in the command you want to customize.

Setting the JAVA_HOME Environment Variable

Many programs, such as Java servers, use the JAVA_HOME environment variable to determine the Java installation location. To set this environment variable, we will first need to find out where Java is installed. You can do this by executing the same command as in the previous section:

Copy the path from your preferred installation and then open /etc/environment using nano or your favorite text editor.

At the end of this file, add the following line, making sure to replace the highlighted path with your own copied path.

Save and exit the file, and reload it.

You can now test whether the environment variable has been set by executing the following command:

This will return the path you just set.

Install Elasticsearch

Elasticsearch is one of the main component which requires Graylog to run, acts as a search server, offers a real-time distributed search and analytics with the RESTful web interface. Elasticsearch stores all the logs sent by the Graylog server and displays the messages whenever user request over the built-in web interface.

This guide covers configuration settings that are required for Graylog.

Let’s install the Elasticsearch. First download and install GPG signing key.

Configure Elasticsearch repository by running below command.

Update repository cache and install Elasticsearch.

Make Elasticsearch to start automatically on the system startup.

Configuring Elasticsearch

We need to modify the Elasticsearch configuration file so that the cluster name matches the one set in the Graylog configuration file. To keep things simple, we’ll set the Elasticsearch cluster name to the default Graylog name of graylog. You may set it to whatever you wish, but make sure you update the Graylog configuration file to reflect that change.

Open the Elasticsearch configuration file in your editor:

Find the following line:

Change the cluster.name value to graylog:

Save the file and exit your editor.

Since we modified the configuration file, we have to restart the service for the changes to take effect.

Now that you have configured Elasticsearch, let’s move on to installing Graylog on Elasticsearch.

Disable dynamic scripts to avoid remote execution, by adding the following lines to the server.conf.

Restart the Elasticsearch service to read the new configurations.

Wait at least a minute to let the Elasticsearch get fully restarted. Elastisearch should be now listening on 9200 for processing HTTP request, use a CURL to check the response.

Ensure that cluster name shows as “graylog”

Optional: Test the health of Elasticsearch cluster, make sure the output yields the cluster status as “green”

Install MongoDB 3.2

Download and install the latest MongoDB from the official website. Import public key on the terminal to begin.

Add mongodb repository by creating the /etc/apt/sources.list.d/mongodb-org.list file using following command.

Install MongoDB using the following command.

Start the MongoDB and enable it on the system start-up.

Installing Graylog

In this step, we we’ll install the Graylog server.

First, download the package file containing the Graylog repository configuration. Visit the Graylog download page to find the current version number. We’ll use version 2.2 for this tutorial.

Next, install the repository configuration from the .deb package file, again replacing 2.2 with the version you downloaded.

Now that the repository configuration has been updated, we have to fetch the new list of packages. Execute this command:

Next, install the graylog-server package:

Lastly, start Graylog automatically on system boot with this command:

Graylog is now successfully installed, but it’s not started yet. We have to configure it before it will start.

You must set a secret to secure the user passwords, use the pwgen command to the same.

If you get an error like “pwgen: command not found”, install pwgen using the following command.

Place the secret like below.

Next is to set a hash (sha256) password for the root user (not to be confused with the system user, root user of graylog is admin). You will need this password to login into the web interface, admin’s password can’t be changed using web interface; you must edit this variable to set.

Replace “yourpassword” with your own.

Place the hash password.

You can setup email address admin user.

Set time zone of root (admin) user.

Graylog server will try to find the Elasticsearch nodes automatically by using multicast mode. But when it comes to larger network, it is recommended to use unicast mode which is best suited one for production.

Add the following entry to graylog server.conf file, replace ipaddress with your ipaddress. You can add multiple hosts with comma separated.

Set only one master node by defining the below variable, the default setting is true.

If you add any second Graylog node, set it to false to make the node as a slave. Master node does some periodic tasks that slave nodes won’t perform.

Set the number of log messages to keep per index; it is recommended to have several smaller indices instead of larger ones.

The following parameter defines to have a total number of indices, if this number is reached old index will be deleted.

Shards setting rely on the number of nodes in the particular Elasticsearch cluster, if you have only one node, set it as 1.

This the number of replicas for your indices, if you have only one node in Elasticsearch cluster; set it as 0.

Install Graylog Web Interface

From the version 2.x, no more extra web interface component, the web interface is being served directly by Graylog server.

Configure Graylog web interface by editing the server.conf file.

Modify the below entries to let Graylog Web Interface to connect to the Graylog server.

Restart Graylog service.

Make Graylog server to start automatically on system startup.

You can check out the server startup logs; it will be useful for you to troubleshoot Graylog in case of any issue.

On the successful start of graylog-server, you should get the following message in the log file.

Accessing Graylog Web Interface

The web interface will now be listening on port 9000, point your browser to http://ip-add-ress:9000.

Login with username “admin” and the password you configured at root_password_sha2 on server.conf.

Once you logged in, you would see the getting started page.

Click on System/Overview to know the status of Graylog server.

Click on System/Overview to know the status of Graylog server.

Configure Graylog Inputs

Graylog inputs need to be configured to receive the logs from the external source, i.e., Syslog or any logging system.

Click System –> Inputs –> Select Syslog UDP and then click Launch new input. Fill with the values in the screen like below.

Once you have created the inputs, configure rsyslog or forward any system logs to your–ip-address:1514

Following screenshot shows the logs received by Graylog (Graylog console –> Search).

That’s all! You have successfully installed Graylog 2.0.3 on Ubuntu 16.04.

Conclusion

You now have a working Graylog server with an input source that can collect logs from other servers.

Next, you might want to look into setting up dashboards, alerts, and streams. Dashboards provide a quick overview of your logs. Streams categorize messages, which you can monitor with alerts. To learn more about configuring the more advanced features of Graylog, you can find instructions in the Graylog documentation.

Reference: https://www.alibabacloud.com/blog/how-to-install-graylog-on-ubuntu-16-04_594046?spm=a2c41.12106575.0.0 �K�x

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store