Kubernetes Ingress Controller Log Collection

We have previously discussed how to deploy an Ingress access layer that provides high reliability in a Kubernetes cluster in the article Best Practices for Deploying a High-Reliability Kubernetes Ingress Controller. Ingress manages external access to services in a Kubernetes cluster. The Ingress Controller records request logs that are very important for tracking the entire request link. The following content describes how to import the collected Kubernetes Ingress Controller logs to Log Service to facilitate retrieval and analysis of service requests.

Prepare the Environment

  1. Apply for a Kubernetes cluster on the Container Service console.
  2. Deploy the service and configure the Ingress Controller to provide services for external systems.

Import the Collected Logs to Log Service

Perform the following steps to import the collected Ingress Controller logs to Log Service:

  1. Create a log project and configure a machine group.
  2. Configure LogtailConfig (log collection configurations).

Step 1: Deploy Logtail DaemonSet.

Note: Skip this step if you have deployed Logtail DaemonSet in the cluster.

curl http://logtail-release.oss-cn-hangzhou.aliyuncs.com/docker/k8s/logtail-daemonset.yaml > logtail-daemonset.yaml

# Modify the following configuration items according to your environment.
vim logtail-daemonset.yaml
...
env:
- name: "ALIYUN_LOGTAIL_CONFIG"
value: "/etc/ilogtail/conf/${your_region_name}/ilogtail_config.json"
- name: "ALIYUN_LOGTAIL_USER_ID"
value: "${your_aliyun_user_id}"
- name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
value: "${your_machine_group_name}"
...
kubectl apply -f logtail-daemonset.yaml
daemonset "logtail" created
kubectl -n kube-system get pod | grep logtail
logtail-nbpl8 1/1 Running 0 52s
logtail-nr9hf 1/1 Running 0 52s
  1. ${your_aliyun_user_id}: Alibaba Cloud UID.
  2. ${your_machine_group_name}: custom ID for configuring a machine group.

Step 2: Create a log project and configure a machine group.

Note: Skip this step if you have created a project for collecting the Kubernetes cluster logs in the corresponding region and you plan to use this project.

Step 3: Configure LogtailConfig (log collection configurations).

Create a Logstore.

Import the Collected Logs to the Self-Built ES

This section describes how to collect Ingress Controller logs to the self-built Elasticsearch (ES). Container Service also provides an appropriate solution if you want to collect and import Kubernetes cluster pod logs to the self-built ES. For more information, see A solution to log collection problems of Kubernetes clusters by using log-pilot, Elasticsearch, and Kibana.

Step 1: Deploy log-pilot, ES, and Kibana.

Note: Skip this step if you have deployed them in the cluster.

Step 2: Perform log collection configurations.

To collect Ingress Controller logs, add the following log collection configurations:

vim nginx-ingress-controller-log-patch.yaml
spec:
template:
spec:
containers:
- name: nginx-ingress-controller
env:
# ingress-controller is the index name.
- name: aliyun_logs_ingress-controller
value: "stdout"

# If you deploy nginx-ingress-controller by using Deployment:
kubectl -n kube-system patch deployment nginx-ingress-controller --patch "$(cat nginx-ingress-controller-log-patch.yaml)"
# If you deploy nginx-ingress-controller by using DaemonSet:
kubectl -n kube-system patch ds nginx-ingress-controller --patch "$(cat nginx-ingress-controller-log-patch.yaml)"

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.