OpenKruise: A Powerful Tool for Sidecar Container Management

Preface

Background

  1. A business pod contains multiple Sidecar containers, such as O&M, security, and proxy containers. The business team should not only configure the main containers, but also learn to configure these Sidecar containers. This increases the workloads of the business team and the risks in Sidecar container configuration.
  2. The Sidecar container needs to be restarted together with the main business container after the upgrade. The Sidecar container supports hundreds of online businesses, so it is extremely difficult to coordinate and promote the upgrades of a large number of online Sidecar containers.
  3. If there are no effective updates for Sidecar containers with different online configurations and versions, it will pose great potential risks to the management of Sidecar containers.

OpenKruise SidecarSet

  1. Separate configuration management: Each Sidecar container is configured with separate SidecarSet configuration to facilitate management.
  2. Automatic injection: Automatic Sidecar container injection is implemented in scenarios of Pod creation, Pod scale-out and Pod reconstruction.
  3. In-place upgrade: Sidecar containers can be upgraded in-place without the reconstruction of any pods, so that the main business container is not affected. In addition, a wide range of gray release policies are included.

Sidecar Logging Architectures

Automatic Injection

# sidecarset.yaml
apiVersion : apps.kruise.io/v1alpha1
Kind : SidecarSet
metadata:
name: test-sidecarset
spec:
# Select Pods through the selector
selector:
matchLabels:
app: web-server
# Specify a namespace to take effect
namespace: ns-1
# container definition
containers:
- name: logtail
image: logtail:1.0.0
# Share the specified volume
volumeMounts:
- name: web-log
mountPath: /var/log/web
# Share all volumes
shareVolumePolicy: disabled
# Share environment variables
transferEnv:
- sourceContainerName: web-server
# TZ indicates the time zone, for example, the environment variable TZ = Asia/Shanghai in the web-server container
envName: TZ
volumes:
- name: web-log
emptyDir: {}

Pod selector

  • The selector is supported to select the pods to be injected. In the above example, the pod of labels[app] = web-server is selected to inject the logtail container. Alternatively, labels[inject/logtail] = true can be added in all pods to inject a global Sidecar.
  • namespace: SidecarSet is globally valid by default. This parameter can be configured to make it valid to a specific namespace.

Data volume sharing

  • Share the specified volume: Use volumeMounts and volumes to share a specified volume with the main container. In the above example, a web-log volume is shared to achieve log collection.
  • Share all volumes: Use shareVolumePolicy = enabled | disabled to specify whether to mount all volumes in the Pod’s main container, which is often used for Sidecar containers such as log collection. If the configuration is enabled, all mount points in the application container are injected into the same Sidecar path, unless there are data volumes and mount points declared by Sidecar.

Share environment variables

In-place Upgrade

Gray Release

apiVersion: apps.kruise.io/v1alpha1
kind: SidecarSet
metadata:
name: sidecarset
spec:
# ...
updateStrategy:
type: RollingUpdate
partition: 980
maxUnavailable: 10%
apiVersion: apps.kruise.io/v1alpha1
kind: SidecarSet
metadata:
name: sidecarset
spec:
# ...
updateStrategy:
type: RollingUpdate
maxUnavailable: 10%
  • If it is a number, the controller will update the pods with (replicas — partition) to the latest version.
  • If it is a percentage, the controller will update the pods with (replicas * (100% — partition)) to the latest version.
  • Under {matched pod} = 100, partition = 50, and maxUnavailable = 10, the controller will release 50 pods to the new version, but the release is limited to 10. That is, only 10 pods are released at the same time. A pod is released one after another until 50 pods are all released.
  • Under {matched pod} = 100, partition = 80, and maxUnavailable = 30, the controller will release 20 Pods to the new version. The controller releases all 20 pods at the same time because the number of maxUnavailable is met.

Canary Release

apiVersion: apps.kruise.io/v1alpha1
kind: SidecarSet
metadata:
name: sidecarset
spec:
# ...
updateStrategy:
type: RollingUpdate
selector:
matchLabels:
- canary.release: true
maxUnavailable: 10%

Scatter Release

  • For the pod set upgrade, multiple upgrades with the same order are guaranteed.
  • The selection priority is (the smaller, the more prioritized): unscheduled < scheduled, pending < unknown < running, not-ready < ready, newer pods < older pods.
apiVersion: apps.kruise.io/v1alpha1
kind: SidecarSet
metadata:
name: sidecarset
spec:
# ...
updateStrategy:
type: RollingUpdate
# Configure Pod labels and assume all pods contain the labels[app_name]
scatterStrategy:
- key: app_name
value: nginx
- key: app_name
value: web-server
- key: app_name
value: api-gateway
maxUnavailable: 10%

Practices

# sidecarset.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: SidecarSet
metadata:
name: logtail-sidecarset
spec:
selector:
matchLabels:
app: nginx
updateStrategy:
type: RollingUpdate
maxUnavailable: 10%
containers:
- name: logtail
image: log-service/logtail:0.16.16
# when recevie sigterm, logtail will delay 10 seconds and then stop
command:
- sh
- -c
- /usr/local/ilogtail/run_logtail.sh 10
livenessProbe:
exec:
command:
- /etc/init.d/ilogtaild
- status
resources:
limits:
memory: 512Mi
requests:
cpu: 10m
memory: 30Mi
##### share this volume
volumeMounts:
- name: nginx-log
mountPath: /var/log/nginx
transferEnv:
- sourceContainerName: nginx
envName: TZ
volumes:
- name: nginx-log
emptyDir: {}
apiVersion: v1
kind: Pod
metadata:
labels:
# matches the SidecarSet's selector
app: nginx
name: test-pod
spec:
containers:
- name: nginx
image: log-service/docker-log-test:latest
command: ["/bin/mock_log"]
args: ["--log-type=nginx", "--stdout=false", "--stderr=true", "--path=/var/log/nginx/access.log", "--total-count=1000000000", "--logs-per-sec=100"]
volumeMounts:
- name: nginx-log
mountPath: /var/log/nginx
envs:
- name: TZ
value: Asia/Shanghai
volumes:
- name: nginx-log
emptyDir: {}
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
test-pod 2/2 Running 0 118s
$ kubectl get pods test-pod -o yaml |grep 'logtail:0.16.16'
image: log-service/logtail:0.16.16
$ kubectl get sidecarset logtail-sidecarset -o yaml | grep -A4 status
status:
matchedPods: 1
observedGeneration: 1
readyPods: 1
updatedPods: 1
$ kubectl edit sidecarsets logtail-sidecarset# sidecarset.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: SidecarSet
metadata:
name: logtail-sidecarset
spec:
containers:
- name: logtail
image: log-service/logtail:0.16.18
$ kubectl get pods |grep test-pod
test-pod 2/2 Running 1 7m34s
$ kubectl get pods test-pod -o yaml |grep 'image: logtail:0.16.18'
image: log-service/logtail:0.16.18

$ kubectl describe pods test-pod
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Killing 5m47s kubelet Container logtail definition changed, will be restarted
Normal Pulling 5m17s kubelet Pulling image "log-service/logtail:0.16.18"
Normal Created 5m5s (x2 over 12m) kubelet Created container logtail
Normal Started 5m5s (x2 over 12m) kubelet Started container logtail
Normal Pulled 5m5s kubelet Successfully pulled image "log-service/logtail:0.16.18"

Summary

Original Source:

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

4.97K Followers

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com