A Quick Guide to Analyzing Apache Logs on Alibaba Cloud Log Service

Installing Logstash Within the ECS

wget https://artifacts.elastic.co/downloads/logstash/logstash-5.5.3.tar.gz
tar -xzvf logstash-5.5.3.tar.gz

Establishing the Logstash Pipeline

input {   
}
# a note in this section indicates that this filter can be selected
filter {
}
output {
}
  • Set input to the data source
  • Set output to the target
  • A filter is optional, you can normally use it to set data filtering logic
input {
file {
path => "/usr/local/demoData/*.log"
start_position => beginning
}
}
output {
ElasticSearch {
hosts => ["http://*******************:9200"]
user => "*******"
password => "***********"
}
}
input {
http {
host => "**********"
port => "**********"
}
}

Analyzing Apache Logs Using Logstash Filter

filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}"}
}
}
66.249.73.135 - - [04/Jan/2015:05:30:06 +0000] "GET /blog/web/firefox-scrolling-fix.html HTTP/1.1" 200 8956 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
{
"clientip" : "66.249.73.135",
"ident" : ,
"auth" : ,
"timestamp" : "04/Jan/2015:05:30:06 +0000",
"verb" : "GET",
"request" : "/blog/web/firefox-scrolling-fix.html",
"httpversion" : "HTTP/1.1",
"response" : "200",
"bytes" : "8956",
"referrer" : "http://www.google.com/bot.html",
"agent" : "Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25"
}
filter {
geoip {
source => "clientip"
}
}
"geoip":{
"timezone":"America/Los_Angeles",
"ip":"66.249.73.135",
"latitude":37.419200000000004,
"continent_code":"NA",
"city_name":"Mountain View",
"country_name":"United States",
"country_code2":"US",
"dma_code":807,
"country_code3":"US",
"region_name":"California",
"location":{
"lon":-122.0574,
"lat":37.419200000000004
},
"postal_code":"94043",
"region_code":"CA",
"longitude":-122.0574
},

Conclusion

--

--

--

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

READ/DOWNLOAD@^ QBasic Fundamentals and Style with an Introduction to Microsoft Visual Basic…

Independent services

Hi jacket by @fashionnova Use my coupon code “ANAKXO”to get 15% #fashionnova #…

Fast growing architectures with serverless and .NET Core

Real-time data processing pipeline — Part 1 — Visual time series data generation

Helidon Extension for Visual Studio Code

Supporting Pre-Peak Scale-up and Post-Peak Scale-Down with DBFS

Backup your WordPress Website to AWS S3 using AWS CLI on regular intervals

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com

More from Medium

JMX Exporter for Kafka Metrics

Confluent Kafka Multi-Region Cluster in 2 minutes

Kafka Change Replication Factor to 3

ROSETTA ERROR in starting Kafka Zookeeper on MAC M1