Kubernetes Extended Resource and Device Plugin Modules

ecent reports on Kubernetes GPU scheduling and operation mechanisms suggest that the company will deprecate the traditional alpha.kubernetes.io/nvidia-gpu main code in version 1.11, and will completely remove the GPU-related scheduling and deployment code from the main code.

Instead, the two Kubernetes built-in modules, Extended Resource and Device Plugin, together with the device plugins developed by device providers, will implement scheduling from a device cluster to working nodes, and then bind devices with containers.

Let me briefly introduce the two modules of Kubernetes:

  • Extended Resource:
    This is a custom resource expansion method. Developers need to report the names and the total number of resources to the API server. The scheduler increases or decreases the number of available resources based on the creation or deletion in the resource pod and determines nodes that satisfy resource requirements during scheduling. The increment and decrement of Extended Resource must be integers. For example, you can allocate 1 GPU but cannot allocate 0.5 GPUs. This function is stable in version 1.8 because it only replaces opaque integer resources and changes some names. If one removes the keyword integer, is it possible to allocate 0.5 GPUs in the future?
  • Device Plugin:
    it provides a general device plugin mechanism and standard device API interface. Equipment vendors can expand devices such as the GPU, FPGA, high-performance NIC, and InfiniBand by implementing APIs, without modifying the Kubelet main code. This feature is in Alpha versions of Kubernetes 1.8 and 1.9 and will be in the Beta version of Kubernetes 1.10. This feature is still new and needs to be enabled by setting --feature-gates=DevicePlugins=true.

Device Plugin Design

API Design

service DevicePlugin {
// returns a stream of []Device
rpc ListAndWatch(Empty) returns (stream ListAndWatchResponse) {}
rpc Allocate(AllocateRequest) returns (AllocateResponse) {}

Of them,

  • ListAndWatch: Kubelet calls this API to discover devices and update device status (for example, a device becomes unhealthy).
  • Allocate: When Kubelet creates a container to use a device, Kubelet calls this API to perform operations on the device and obtain the device, volume, and environment variable configurations required for container initialization.

Plugin Lifecycle Management

After the plugin starts, Kubelet establishes a persistent listAndWatch connection to the plugin. When detecting an unhealthy device, the plugin automatically notifies the Kubelet. If the device is idle, Kubelet moves it out of the allocatable list; if the device is used by a pod, Kubelet kills the pod.

The plugin monitors the Kubelet status by using the Kubelet socket. If Kubelet restarts, the plugin also restarts and registers with Kubelet again.

Image for post
Image for post

Deployment Methods

Implementation Example

apiVersion: apps/v1
kind: Deployment
name: tf-notebook
app: tf-notebook
spec: template: # define the pods specifications
app: tf-notebook
- name: tf-notebook
image: tensorflow/tensorflow:1.4.1-gpu-py3
nvidia.com/gpu: 1


Read similar articles and learn more about Alibaba Cloud products and solutions at www.alibabacloud.com.

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store