System Monitoring using Prometheus and Grafana

Prerequisites

  1. You must have Alibaba Cloud Elastic Compute Service (ECS) activated and verified your valid payment method. If you are a new user, you can get a free account in your Alibaba Cloud account. If you don’t know about how to set up your ECS instance, you can refer to this tutorial or quick-start guide. Your ECS instance must have at least 1GB RAM and 1 Core processor.
  2. User with sudo access

Update the System

sudo apt-get update

Installing Docker

wget -qO- https://get.docker.com/ | sh
sudo usermod -aG docker $(whoami)
sudo apt-get -y install python-pip
sudo pip install docker-compose

Installing Prometheus

nano ~/prometheus.yml
# A scrape configuration scraping a Node Exporter and the Prometheus server
# itself.
scrape_configs:
# Scrape Prometheus itself every 5 seconds.
- job_name: 'prometheus'
scrape_interval: 5s
target_groups:
- targets: ['localhost:9090']
# Scrape the Node Exporter every 5 seconds.
- job_name: 'node'
scrape_interval: 5s
target_groups:
- targets: ['YourServerIP:9100']
docker run -d -p 9090:9090 -v ~/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus -config.file=/etc/prometheus/prometheus.yml -storage.local.path=/prometheus -storage.local.memory-chunks=10000
docker ps
docker logs container_id
docker inspect container_id
...
"Mounts": [
{
"Source": "/home/sammy/prometheus.yml",
"Destination": "/etc/prometheus/prometheus.yml",
"Mode": "",
"RW": true
},
{
"Name": "111b0abc850a9c758ee35ed5cff69077423a629566082a605a01d8207d57cd6c",
"Source": "/var/lib/docker/volumes/111b0abc850a9c758ee35ed5cff69077423a629566082a605a01d8207d57cd6c/_data",
"Destination": "/prometheus",
"Driver": "local",
"Mode": "",
"RW": true
}
],
...

Installing Node Exporter

docker run -d -p 9100:9100 -v "/proc:/host/proc" -v "/sys:/host/sys" -v "/:/rootfs" --net="host" prom/node-exporter -collector.procfs /host/proc -collector.sysfs /host/proc -collector.filesystem.ignored-mount-points "^/(sys|proc|dev|host|etc)($|/)

Installing Grafana

docker run -d -p 3000:3000 -e "GF_SECURITY_ADMIN_PASSWORD=YourPassword" -v ~/grafana_db:/var/lib/grafana grafana/grafana

Adding Prometheus as a Grafana Data Source

  1. Name: Prometheus
  2. Type: Prometheus
  3. URL: http://:9090, (default port is 9090)
  4. Access: proxy
  5. Basic Auth: According to Your Server

Importing the Prometheus Stats Dashboard

  1. Uptime: This single stat graph is the uptime or the time since the Prometheus server was brought online. If your server has recently restarted you will find it very useful.
  2. Local Storage Memory Series: Current number of series held in memory will be displayed here.
  3. Internal Storage Queue Length: Ideally, this queue length should be Empty (0) or a low number.
  4. Sample Ingested: This graph displays the count of samples ingested by the Prometheus server, as measured over the last 5 minutes, per time series in the range vector. When troubleshooting an issue on IRC or Github, this is often the first stat requested by the Prometheus team. This number should align with the number of metrics you believe you are ingesting.
  5. Scrapes: Prometheus scrapes metrics from instrumented jobs. There is two scrapes graph, the first one is, Target scrapes graph will show how frequently targets are scraped as measured over the last 5 minutes. Another one is, the Scrape Duration graph will show how long the scrapes are taking with percentiles available as series.

Analyzing the Data

Alert Management

Conclusion

--

--

--

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Novel Points: Exploit the Heap Overflow Bug *CTF 2019 oob-v8

Term Frequency Statistics in PostgreSQL Full-Text Searches

How to create a WLSDM Generic DevOps MBean by Govindan Padmanaban

How MaxCompute Helped YCLOSET Build Smart O&M Tools

Back Up MySQL with Percona Xtrabackup

The recipe of instability

Choose the most suitable inductor for your DC-DC converter (2)

Leveraging billions of data points to build the future of hospitality

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com

More from Medium

How to Schedule Pods on Nodes in Kubernetes

Setup Container Cassandra and run commands

Distributing an application as Helm chart on Artifact Hub

Scaling microser through sidecars