Using Sidecar Mode for Kubernetes Log Collection

Double 11 The Biggest Deals of the Year. 40% OFF on selected cloud servers with a free 100 GB data transfer! Click here to learn more.

As a key project of the Cloud Native Computing Foundation (CNCF), Kubernetes has been developing rapidly in the last two years backed by the powerful communities of Google and Redhat. Besides becoming a leader in the container orchestration field, Kubernetes is now developing towards the standard PaaS configuration.

Log Collection Modes

Logs are an indispensable part of a system. Official documents of Kubernetes introduce multiple log collections modes, which can be summarized into the native mode, DaemonSet mode, and SideCar mode.

  1. Native mode: Run the kubectl logs command to directly view the local logs or use the log driver of Docker Engine to redirect logs to files, syslog, or Fluentd.
Image for post
Image for post

Comparison of Collection Modes

Each collection mode has its advantages and disadvantages. The following table provides a simple comparison among these collection modes:

Image for post
Image for post

The preceding table indicates that:

  1. Functions provided by the native mode are weak. Therefore, the native mode is not recommended for the production system; otherwise, it is difficult to complete troubleshooting and data statistics.

Log Collection Modes for Kubernetes Supported by Log Service

Both the DaemonSet and Sidecar modes have their own advantages and disadvantages, but neither is applicable to all scenarios. Alibaba Cloud Log Service supports both the DaemonSet and SideCar modes and is improved to better adapt to the Kubernetes dynamic scenarios.

Both modes are implemented based on Logtail. Today, millions of log service clients (Logtails) have been deployed. They are now collecting PB-scale data generated by tens of thousands of applications and have been tested on years of Double 11 and Double 12 shopping festivals.

DaemonSet Collection Mode

In DaemonSet mode, Logtail makes a lot of adaptation efforts, including:

  1. Users can use only one command and one parameter to deploy Logtail and complete automatic initialization of resources.
Image for post
Image for post

SideCar Collection Mode

Configuration and use of the SideCar mode are similar to collection of data on the VMs or PMs. From the Logtail container point of view, Logtail works on a VM and needs to collect a certain log file or some log files on this VM.

Image for post
Image for post

However, the following problems must be solved in the container scenario:

  1. Configuration: The orchestration method must be used to configure the agent container.

Currently, the Logtail container allows you to configure related parameters using environmental variables and supports definition of a machine group using a user defined ID. This properly solves the preceding two problems.

SideCar Configuration Example

In SideCar mode, you can install and configure the log components as follows:

Step 1. Deploy the Logtail container

  1. When deploying a Pod, attach the log path onto the local machine and the corresponding volume onto the Logtail container.

Tips:

  1. We recommend that you configure health check for the Logtail container so that the Logtail container can recover automatically when the operating environment or any core is abnormal.
apiVersion: batch/v1
kind: Job
metadata:
name: nginx-log-sidecar-demo
namespace: kube-system
spec:
template:
metadata:
name: nginx-log-sidecar-demo
spec:
# Configuration of volumes
volumes:
- name: nginx-log
emptyDir: {}
containers:
# Configuration of the primary container
- name: nginx-log-demo
image: registry.cn-hangzhou.aliyuncs.com/log-service/docker-log-test:latest
command: ["/bin/mock_log"]
args: ["--log-type=nginx", "--stdout=false", "--stderr=true", "--path=/var/log/nginx/access.log", "--total-count=1000000000", "--logs-per-sec=100"]
volumeMounts:
- name: nginx-log
mountPath: /var/log/ngin
# Configuration of the Logtail SideCar container
- name: logtail
image: registry.cn-hangzhou.aliyuncs.com/log-service/logtail:latest
env:
# aliuid
- name: "ALIYUN_LOGTAIL_USER_ID"
value: "165421******3050"
# Configuration of the machine group using a user-defined ID
- name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
value: "nginx-log-sidecar"
# Startup configuration (used to select the region of Logtail)
- name: "ALIYUN_LOGTAIL_CONFIG"
value: "/etc/ilogtail/conf/cn-hangzhou/ilogtail_config.json"
# Sharing volumes with the primary container
volumeMounts:
- name: nginx-log
mountPath: /var/log/nginx
# Health check
livenessProbe:
exec:
command:
- /etc/init.d/ilogtaild
- status
initialDelaySeconds: 30
periodSeconds: 30

Step 2. Configure the machine group

As shown in the following figure, create a Logtail machine group on the Log Service console, and select a user-defined ID for the machine group so that Logtail can adapt to changes of the IP address of the Pod. The procedure is as follows:

  1. Activate Log Service and create a project and Logstore. For more information, see Preparations.
Image for post
Image for post

Step 3. Configure the collection mode

After creating the machine group, complete the collection configuration of the corresponding files. Currently, the following log formats are supported: single-line log, Nginx access log, delimiter log, JSON log, and regular expression log. For more information, see Text log configuration method. In this example, the configuration is as follows:

Image for post
Image for post

Step 4. Query logs

After you complete the log collection configuration and apply it to the machine group, the logs can be collected in one minute. You can go to the query page of Logstore to query the collected logs.

Image for post
Image for post

Advanced Tutorials of Log Service

Log Service provides a complete log solution, in which log collection is only the first step. For more information about functions of Log Service, see the following articles:

  1. Log context query: https://www.alibabacloud.com/help/doc-detail/48148.htm

Reference:https://www.alibabacloud.com/blog/using-sidecar-mode-for-kubernetes-log-collection_594173?spm=a2c41.12293703.0.0

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store