System Monitoring using Prometheus and Grafana

Prerequisites

Update the System

sudo apt-get update

Installing Docker

wget -qO- https://get.docker.com/ | sh
sudo usermod -aG docker $(whoami)
sudo apt-get -y install python-pip
sudo pip install docker-compose

Installing Prometheus

nano ~/prometheus.yml
# A scrape configuration scraping a Node Exporter and the Prometheus server
# itself.
scrape_configs:
# Scrape Prometheus itself every 5 seconds.
- job_name: 'prometheus'
scrape_interval: 5s
target_groups:
- targets: ['localhost:9090']
# Scrape the Node Exporter every 5 seconds.
- job_name: 'node'
scrape_interval: 5s
target_groups:
- targets: ['YourServerIP:9100']
docker run -d -p 9090:9090 -v ~/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus -config.file=/etc/prometheus/prometheus.yml -storage.local.path=/prometheus -storage.local.memory-chunks=10000
docker ps
docker logs container_id
docker inspect container_id
...
"Mounts": [
{
"Source": "/home/sammy/prometheus.yml",
"Destination": "/etc/prometheus/prometheus.yml",
"Mode": "",
"RW": true
},
{
"Name": "111b0abc850a9c758ee35ed5cff69077423a629566082a605a01d8207d57cd6c",
"Source": "/var/lib/docker/volumes/111b0abc850a9c758ee35ed5cff69077423a629566082a605a01d8207d57cd6c/_data",
"Destination": "/prometheus",
"Driver": "local",
"Mode": "",
"RW": true
}
],
...

Installing Node Exporter

docker run -d -p 9100:9100 -v "/proc:/host/proc" -v "/sys:/host/sys" -v "/:/rootfs" --net="host" prom/node-exporter -collector.procfs /host/proc -collector.sysfs /host/proc -collector.filesystem.ignored-mount-points "^/(sys|proc|dev|host|etc)($|/)

Installing Grafana

docker run -d -p 3000:3000 -e "GF_SECURITY_ADMIN_PASSWORD=YourPassword" -v ~/grafana_db:/var/lib/grafana grafana/grafana

Adding Prometheus as a Grafana Data Source

Importing the Prometheus Stats Dashboard

Analyzing the Data

Alert Management

Conclusion

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store