Using Sidecar Mode for Kubernetes Log Collection

Log Collection Modes

Logs are an indispensable part of a system. Official documents of Kubernetes introduce multiple log collections modes, which can be summarized into the native mode, DaemonSet mode, and SideCar mode.

  1. Native mode: Run the kubectl logs command to directly view the local logs or use the log driver of Docker Engine to redirect logs to files, syslog, or Fluentd.
  2. DaemonSet mode: A log agent is deployed on every Kubernetes node. The log agent collects the container logs and sends them to the server.
  3. SideCar mode: A Pod runs a SideCar log agent container to collect logs generated by the primary container in the Pod.

Comparison of Collection Modes

Each collection mode has its advantages and disadvantages. The following table provides a simple comparison among these collection modes:

  1. Functions provided by the native mode are weak. Therefore, the native mode is not recommended for the production system; otherwise, it is difficult to complete troubleshooting and data statistics.
  2. The DaemonSet mode allows only one log agent on each node. In this mode, less resources are consumed, but scalability and tenant isolation are limited. Therefore, the DaemonSet mode is applicable to clusters that have few functions or few services.
  3. The SideCar mode allows deployment of a log agent for every Pod. In this mode, more resources are consumed, but flexibility is improved and the multi-tenant isolation performance is good. Therefore, the SideCar mode is applicable to large-sized Kubernetes clusters or clusters that act as the PaaS platform serving multiple service parties.

Log Collection Modes for Kubernetes Supported by Log Service

Both the DaemonSet and Sidecar modes have their own advantages and disadvantages, but neither is applicable to all scenarios. Alibaba Cloud Log Service supports both the DaemonSet and SideCar modes and is improved to better adapt to the Kubernetes dynamic scenarios.

DaemonSet Collection Mode

In DaemonSet mode, Logtail makes a lot of adaptation efforts, including:

  1. Users can use only one command and one parameter to deploy Logtail and complete automatic initialization of resources.
  2. Logtail supports configuration in CRD mode, supports Kubernetes console, kubectl, and kube API, and is seamlessly integrated with Kubernetes release and deployment.
  3. Logtail supports Kubernetes RBAC authorization and STS authorization management.

SideCar Collection Mode

Configuration and use of the SideCar mode are similar to collection of data on the VMs or PMs. From the Logtail container point of view, Logtail works on a VM and needs to collect a certain log file or some log files on this VM.

  1. Configuration: The orchestration method must be used to configure the agent container.
  2. Dynamics: Logtail must adapt to changes of the IP address and hostname of the Pod.

SideCar Configuration Example

In SideCar mode, you can install and configure the log components as follows:

Step 1. Deploy the Logtail container

  1. When deploying a Pod, attach the log path onto the local machine and the corresponding volume onto the Logtail container.
  2. For the Logtail container, configure ALIYUN_LOGTAIL_USER_ID, ALIYUN_LOGTAIL_CONFIG, and ALIYUN_LOGTAIL_USER_DEFINED_ID. For more information about the meanings and values of parameters, see Standard Docker log collection.
  1. We recommend that you configure health check for the Logtail container so that the Logtail container can recover automatically when the operating environment or any core is abnormal.
  2. In the following example, the Logtail image is used to access the image repository of the Alibaba Cloud Hangzhou public network. You can replace the image with the one in your local region and uses the intranet for access.
apiVersion: batch/v1
kind: Job
name: nginx-log-sidecar-demo
namespace: kube-system
name: nginx-log-sidecar-demo
# Configuration of volumes
- name: nginx-log
emptyDir: {}
# Configuration of the primary container
- name: nginx-log-demo
command: ["/bin/mock_log"]
args: ["--log-type=nginx", "--stdout=false", "--stderr=true", "--path=/var/log/nginx/access.log", "--total-count=1000000000", "--logs-per-sec=100"]
- name: nginx-log
mountPath: /var/log/ngin
# Configuration of the Logtail SideCar container
- name: logtail
# aliuid
value: "165421******3050"
# Configuration of the machine group using a user-defined ID
value: "nginx-log-sidecar"
# Startup configuration (used to select the region of Logtail)
value: "/etc/ilogtail/conf/cn-hangzhou/ilogtail_config.json"
# Sharing volumes with the primary container
- name: nginx-log
mountPath: /var/log/nginx
# Health check
- /etc/init.d/ilogtaild
- status
initialDelaySeconds: 30
periodSeconds: 30

Step 2. Configure the machine group

As shown in the following figure, create a Logtail machine group on the Log Service console, and select a user-defined ID for the machine group so that Logtail can adapt to changes of the IP address of the Pod. The procedure is as follows:

  1. Activate Log Service and create a project and Logstore. For more information, see Preparations.
  2. On the Machine Groups page of the Log Service console, click Create Machine Group.
  3. Set Machine Group Identification to User-defined Identity. In the User-defined Identity text box, enter ALIYUN_LOGTAIL_USER_DEFINED_ID configured in step 1.

Step 3. Configure the collection mode

After creating the machine group, complete the collection configuration of the corresponding files. Currently, the following log formats are supported: single-line log, Nginx access log, delimiter log, JSON log, and regular expression log. For more information, see Text log configuration method. In this example, the configuration is as follows:

Step 4. Query logs

After you complete the log collection configuration and apply it to the machine group, the logs can be collected in one minute. You can go to the query page of Logstore to query the collected logs.

Advanced Tutorials of Log Service

Log Service provides a complete log solution, in which log collection is only the first step. For more information about functions of Log Service, see the following articles:

  1. Log context query:
  2. Fast query:
  3. Real-time analysis:
  4. Fast analysis:
  5. Log-based alarm configuration:
  6. Dashboard configuration:



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website: