Efficiently Monitor Nginx Web Servers Using Alibaba Cloud Elasticsearch

Image for post
Image for post

Written by Liu Xiaoguo, an Evangelist of the Elasticsearch Community in China, and edited by Lettie and Dayu

Released by ELK Geek

Nginx is a popular, open-source web server used for millions of applications around the world and is second only to Apache. It also acts as a reverse proxy, HTTP cache, and load balancer. From the operation and security perspective, Nginx needs to be monitored in real-time because it plays a critical role in various application architectures.

Basics of Nginx Log Entries

By default, both logs are usually located under /var/log/nginx, but this location may vary from system to system.

# cd /var/log/nginx/
# ls
access.log error.log

Nginx Error Logs

A sample log is as follows:

2020/04/22 10:06:21 [error] 9289#0: *4128 connect() failed (111: Connection refused) while connecting to upstream, client:, server: notest004, request: "GET /admin/ HTTP/1.1", upstream: "", host: ""

Nginx Access Logs

Sample Log: - - [22/Apr/2020:06:26:39 +0800] "GET / HTTP/1.1" 502 559 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3494.0 Safari/537.36" "-"


Install Node.js

# yum install nodejs
# yum install -y npm
##### View node and npm version #######
# node -v
# npm -v
#### Deploy the web server of nodejs, download simple nodejs code from git
# git clone https://github.com/liu-xiao-guo/samplenodejs
#### Go to the root directory of samplenodejs and run the following commands ####
# cd samplenodejs/
# npm install
# npm start

Enable port 3000 in the ECS security group, and enter the ECS IP address in the address bar of the browser: http://121.41.xx.xx:3000/hello

Image for post
Image for post

If the following output appears in the console and browser, it indicates that the Node.js application is running successfully.

Image for post
Image for post

Install Nginx and Set It as a Reverse Proxy

# yum install nginx
#### Configure reverse proxy ####
# vim /etc/nginx/nginx.conf
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name notest004;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
index index.html index.htm;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
#### Start nginx ###
# systemctl start nginx

To stop Nginx, run the following command:

systemctl stop nginx

To start the Web server when Nginx stops, run the following command:

systemctl start nginx

To stop and then start the service again, run the following command:

systemctl restart nginx

Download and Install Filebeat

Install Filebeat

# wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.7.0-linux-x86_64.tar.gz
# tar -zxvf filebeat-6.7.0-linux-x86_64.tar.gz

Configure Filebeat

Modify filebeat-6.7.0-linux-x86_64/filebeat.yml as follows:

#-------------------------- Elasticsearch output ------------------------------
# Array of hosts to connect to.
hosts: ["es-cn-0pp19tb10000688c8p.elasticsearch.aliyuncs.com:9200"]
username: "elastic"
password: "Elastic123"

host: "https://es-cn-0pp19tb1000888c8p.kibana.elasticsearch.aliyuncs.com:5601"

Start the Nginx Module

# ./filebeat modules enable nginx

Configure the Nginx Module

# ./filebeat setup
Image for post
Image for post

At this point, the installation has been completed.

Display of Kibana Dashboard

Image for post
Image for post

Click “[Filebeat Nginx] Overview”.

Image for post
Image for post

With this, the monitoring process of Nginx logs is completed. View all the information about Nginx in the Kibana Dashboard.


Source: (Page in Chinese) https://me.csdn.net/UbuntuTouch

Image for post
Image for post

The Alibaba Cloud Elastic Stack is completely compatible with open-source Elasticsearch and has nine unique capabilities

Original Source:

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store