A Quick Guide to Analyzing Apache Logs on Alibaba Cloud Log Service

Installing Logstash Within the ECS

wget https://artifacts.elastic.co/downloads/logstash/logstash-5.5.3.tar.gz
tar -xzvf logstash-5.5.3.tar.gz

Establishing the Logstash Pipeline

input {   
}
# a note in this section indicates that this filter can be selected
filter {
}
output {
}
  • Set input to the data source
  • Set output to the target
  • A filter is optional, you can normally use it to set data filtering logic
input {
file {
path => "/usr/local/demoData/*.log"
start_position => beginning
}
}
output {
ElasticSearch {
hosts => ["http://*******************:9200"]
user => "*******"
password => "***********"
}
}
input {
http {
host => "**********"
port => "**********"
}
}

Analyzing Apache Logs Using Logstash Filter

filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}"}
}
}
66.249.73.135 - - [04/Jan/2015:05:30:06 +0000] "GET /blog/web/firefox-scrolling-fix.html HTTP/1.1" 200 8956 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
{
"clientip" : "66.249.73.135",
"ident" : ,
"auth" : ,
"timestamp" : "04/Jan/2015:05:30:06 +0000",
"verb" : "GET",
"request" : "/blog/web/firefox-scrolling-fix.html",
"httpversion" : "HTTP/1.1",
"response" : "200",
"bytes" : "8956",
"referrer" : "http://www.google.com/bot.html",
"agent" : "Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25"
}
filter {
geoip {
source => "clientip"
}
}
"geoip":{
"timezone":"America/Los_Angeles",
"ip":"66.249.73.135",
"latitude":37.419200000000004,
"continent_code":"NA",
"city_name":"Mountain View",
"country_name":"United States",
"country_code2":"US",
"dma_code":807,
"country_code3":"US",
"region_name":"California",
"location":{
"lon":-122.0574,
"lat":37.419200000000004
},
"postal_code":"94043",
"region_code":"CA",
"longitude":-122.0574
},

Conclusion

--

--

--

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Learn to code. Learn Python.

GitHub Action Self Hosted Runners

Python Sanic vs GoLang Mux

Dask on Azure Databricks

Best Java EE Frameworks Tools in 2021

Fun with Skybox’s in Unity

Maximum in Subarrays of length K

A Roadmap to Software Engineering jobs (For College students)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com

More from Medium

Historize elastic APM server data

How to deploy microservice to Google Kubernetes (GKE) with sed in Github matrix

Airflow on Google Cloud Composer

Best dev tools in GCP: Cloud Trace, Debugger, Logging