How Beats Import RabbitMQ Logs to the Alibaba Cloud Elastic Stack for Visual Analysis

Image for post
Image for post

By Liu Xiaoguo, an Elastic Community Evangelist in China and edited by Lettie and Dayu

Released by ELK Geek

Introduction to RabbitMQ

Image for post
Image for post

A message may contain any type of information. For example, it may have information about a process or task that should be started on another application, server, or a simple text message. The queue manager stores a message until an application connects to the queue manager and pulls the message from the queue. The application then processes the message.

Image for post
Image for post

A message queue simply contains ‘client applications’ that are defined as producers and applications that are called consumers. Producers create messages and transmit them to the proxy, that is, the message queue. Consumers connect to the queue to subscribe to messages that they need to process. Applications function as a message producer as well as a consumer or both. Messages are stored in the queue until consumers retrieve them.

RabbitMQ Logs

In RabbitMQ 3.8.2, specify the position in Rabbitmq.conf where the RabbitMQ log file is stored. I will display the file during the RabbitMQ installation.

Preparation

1) Prepare the Alibaba Cloud Elasticsearch 6.7 environment and use the created account and password to log on to Kibana.
2) Prepare the Alibaba Cloud Logstash 6.7 environment.
3) Prepare the RabbitMQ service.
4) Install Filebeat.

Install RabbitMQ

# cd /usr/lib/rabbitmq/bin
# rabbitmq-server start

Configure RabbitMQ

# vim /etc/rabbitmq/rabbitmq.config
{lager, [
%%
%% Log directory, taken from the RABBITMQ_LOG_BASE env variable by default.
%% {log_root, "/var/log/rabbitmq"},
%%
%% All log messages go to the default "sink" configured with
%% the `handlers` parameter. By default, it has a single
%% lager_file_backend handler writing messages to "$nodename.log"
%% (ie. the value of $RABBIT_LOGS).
{handlers, [
{lager_file_backend, [{file, "rabbit.log"},
{level, info},
{date, ""},
{size, 0}]}
]},
{extra_sinks, [

{rabbit_channel_lager_event, [{handlers, [
{lager_forwarder_backend,
[lager_event, info]}]}]},
{rabbit_conection_lager_event, [{handlers, [
{lager_forwarder_backend,
[lager_event, error]}]}]}

]}

After we set the configuration, the log file name is changed to “rabbit.log” and the log level is set to “info”. To apply the modified configuration file, run the following command to restart rabbitmq-server:

##### Switch to the bin directory to start RabbitMQ because RabbitMQ is installed 
by using the RPM package. ###
# cd /usr/lib/rabbitmq/bin
# rabbitmq-server start

View the output rabbit.log in the /var/log/rabbitmq directory.

Install the RabbitMQ Demo Application

# git clone https://github.com/rabbitmq/rabbitmq-jms-client-spring-boot-trader-demo
#### Switch to the root directory where the application is located. ######
# mvn clean package
# java -jar target/rabbit-jms-boot-demo-1.2.0-SNAPSHOT.jar
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v1.5.8.RELEASE)
2020-05-11 10:16:46.089 INFO 28119 --- [ main] com.rabbitmq.jms.sample.StockQuoter : Starting StockQuoter v1.2.0-SNAPSHOT on zl-test001 with PID 28119 (/root/rabbitmq-jms-client-spring-boot-trader-demo/target/rabbit-jms-boot-demo-1.2.0-SNAPSHOT.jar started by root in /root/rabbitmq-jms-client-spring-boot-trader-demo)
2020-05-11 10:16:46.092 INFO 28119 --- [ main] com.rabbitmq.jms.sample.StockQuoter : No active profile set, falling back to default profiles: default
2020-05-11 10:16:46.216 INFO 28119 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@1de0aca6: startup date [Mon May 11 10:16:46 CST 2020]; root of context hierarchy
2020-05-11 10:16:47.224 INFO 28119 --- [ main] com.rabbitmq.jms.sample.StockConsumer : connectionFactory => RMQConnectionFactory{user='guest', password=xxxxxxxx, host='localhost', port=5672, virtualHost='/', queueBrowserReadMax=0}
2020-05-11 10:16:48.054 INFO 28119 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup
2020-05-11 10:16:48.062 INFO 28119 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting beans in phase 0
......
### After the preceding configuration, enter the log directory to view RabbitMQ logs.
# pwd
/var/log/rabbitmq
erl_crash.dump rabbit.log rabbit@zl-test001.log rabbit@zl-test001_upgrade.log
log

rabbit.log is the file name that we configured previously.

The following sections describe how to use Alibaba Cloud Filebeat and Logstash to import these logs to Alibaba Cloud Elasticsearch.

Configure Alibaba Cloud Filebeat

2) Specify the collector name, installation version, output, and YAML configuration.

Image for post
Image for post

The following snapshot shows the Filebeat.input configuration.

filebeat.inputs:
# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.
- type: log
# Change to true to enable this input configuration.
enabled: true
fields:
log_type: rabbitmq-server
# Paths that should be crawled and fetched. Glob based paths.
paths:
- /var/log/rabbitmq/*log
fields_under_root: true
encoding: utf-8
ignore_older: 3h

3) Select an ECS instance in the same VPC as Logstash and start the ECS instance.

Image for post
Image for post

After the collector takes effect, start the Filebeat service.

Image for post
Image for post

Configure Alibaba Cloud Logstash

1) Go to the Pipeline Management page of the Logstash instance.

2) Click Create Pipeline and configure the pipeline.

The following snapshot shows the configuration file.

input {
beats {
port => 8100
}
}

filter {
grok {
match => { "message" => ["%{TIMESTAMP_ISO8601:timestamp} \[%{LOGLEVEL:log_level}\] \<%{DATA:field_misc}\> %{GREEDYDATA:message}"] }
}
}

output {
elasticsearch {
hosts => "es-cn-42333umei000u1zp5.elasticsearch.aliyuncs.com:9200"
user => "elastic"
password => "E222ic@123"
index => "rabbitmqlog-%{+YYYY.MM.dd}"
}
}

3) Define pipeline parameters and click Save and Deploy.

Image for post
Image for post

4) After the successful deployment, view the index data stored on Elasticsearch, which indicates that Elasticsearch has stored the data processed by Logstash.

Image for post
Image for post

Use Kibana to View Log Documents

Image for post
Image for post

Specify the time filter field and create the index pattern module.

Image for post
Image for post

Go to the Discover page, select the created index pattern module, and use the filter to filter out RabbitMQ logs.

Image for post
Image for post

The preceding figure shows the filtered out RabbitMQ logs.

Configure Metricbeat to Collect RabbitMQ Metrics

Click Add metric data.

Image for post
Image for post

Click RabbitMQ metrics.

Image for post
Image for post

Download and install Metricbeat.

Image for post
Image for post

Configure Metricbeat based on the preceding instructions.

Modify the /etc/metricbeat/metricbeat.yml file to configure the Elasticsearch cluster link information.

setup.kibana:
# Kibana Host
# Scheme and port can be left out and will be set to the default (http and 5601)
# In case you specify and additional path, the scheme is required: http://localhost:5601/path
# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
host: "https://es-cn-451233mei000u1zp5.kibana.elasticsearch.aliyuncs.com:5601"
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["es-cn-4591jumei000u1zp5.elasticsearch.aliyuncs.com:9200"]
# Enabled ilm (beta) to use index lifecycle management instead daily indices.
#ilm.enabled: false
# Optional protocol and basic auth credentials.
#protocol: "https"
username: "elastic"
password: "12233"

Start the RabbitMQ module and the Metricbeat service.

# sudo metricbeat modules enable rabbitmq
##### Set the dashboard. #######
# sudo metricbeat setup
# sudo service metricbeat start

Restart the Metricbeat service and click Check data on the following page. If the prompt “Data successfully received from this module” appears, data has been received from the module.

Image for post
Image for post

Click the RabbitMQ metrics dashboard to view the monitoring dashboard.

Image for post
Image for post

Statement: This article is an authorized revision of the article “Beats: Use the Elastic Stack to Monitor Redis” based on the Alibaba Cloud service environment.

Source: (Page in Chinese) https://elasticstack.blog.csdn.net/

Image for post
Image for post

The Alibaba Cloud Elastic Stack is completely compatible with open-source Elasticsearch and has nine unique capabilities.

Original Source:

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store