Efficiently Monitor Nginx Web Servers Using Alibaba Cloud Elasticsearch

Written by Liu Xiaoguo, an Evangelist of the Elasticsearch Community in China, and edited by Lettie and Dayu

Released by ELK Geek

Nginx is a popular, open-source web server used for millions of applications around the world and is second only to Apache. It also acts as a reverse proxy, HTTP cache, and load balancer. From the operation and security perspective, Nginx needs to be monitored in real-time because it plays a critical role in various application architectures.

Basics of Nginx Log Entries

Nginx provides a variety of log entry options, including log entry to files, conditional log entry, and syslog log entry. Nginx generates two log types that are used for operation monitoring and troubleshooting: error logs and access logs.

By default, both logs are usually located under /var/log/nginx, but this location may vary from system to system.

Nginx Error Logs

Error logs contain diagnostic information that is used to troubleshoot operational problems. Nginx error_log can be used to specify the path of log files and the severity level of logs, and it can be used in main, http, mail, stream, server, and location in the sequence.

A sample log is as follows:

Nginx Access Logs

Access logs contain information about all requests sent to and served by Nginx. Therefore, they are valuable resources for performance monitoring and security. The default format for Nginx access logs is a combined format but may vary between distributions. Use the access_log pseudo command to set the log file path and log format like the operations on error logs.

Sample Log:

Preparations

The simplest way to send Nginx logs to Alibaba Cloud Elasticsearch is via Filebeat. Install Node.js, Nginx, and Filebeat on an Elastic Compute Service (ECS) instance and activate Alibaba Cloud Elasticsearch.

Install Node.js

Node.js is a simple web server runtime environment based on the Express framework. Install Node.js using the following command on an Alibaba Cloud ECS instance.

Enable port 3000 in the ECS security group, and enter the ECS IP address in the address bar of the browser: http://121.41.xx.xx:3000/hello

If the following output appears in the console and browser, it indicates that the Node.js application is running successfully.

Install Nginx and Set It as a Reverse Proxy

Use the following command on the Alibaba Cloud ECS instance to install Nginx:

To stop Nginx, run the following command:

To start the Web server when Nginx stops, run the following command:

To stop and then start the service again, run the following command:

Download and Install Filebeat

In the Alibaba Cloud ECS environment, download the Filebeat installation package, and decompress it.

Install Filebeat

Configure Filebeat

Modify filebeat-6.7.0-linux-x86_64/filebeat.yml as follows:

Start the Nginx Module

Configure the Nginx Module

Run the following command to display the Nginx module on the Kibana Dashboard.

At this point, the installation has been completed.

Display of Kibana Dashboard

Log on to Kibana and click “Dashboard” to select the Nginx module that has been imported.

Click “[Filebeat Nginx] Overview”.

With this, the monitoring process of Nginx logs is completed. View all the information about Nginx in the Kibana Dashboard.

Statement

This article is adapted from the article “Beats: Use the Elastic Stack to Monitor Nginx Web Servers” and modified based on Alibaba Cloud service environment authorization.

Source: (Page in Chinese) https://me.csdn.net/UbuntuTouch

The Alibaba Cloud Elastic Stack is completely compatible with open-source Elasticsearch and has nine unique capabilities

Original Source:

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.