Configure a New Capability for Kubernetes PaaS in 20 Minutes
by @wonderflow
In November 2020, KubeVela was officially launched. As a simple, easy-to-use, and highly extensible application management platform and core engine, Alibaba Cloud engineers can build a cloud-native PaaS. An instance will be used in this article to explain how to get a new KubeVela-based PaaS capability online within 20 minutes.
Before the tutorial in this article, please install KubeVela and its dependent Kubernetes environments on your local computer.
Basic Structure of KubeVela Extensions
The basic architecture of KubeVela is shown in the following figure:
KubeVela adds Workload Type and Trait to extend capabilities for users. The service provider of the platform registers and extends through the Definition file and shows the extended functionality up through the Appfile. The official documents have the basic writing procedures, including extension examples of Workload and Trait:
- Workload Type extension with OpenFaaS as an example
- Workload Type extension with Relational Database Service (RDS) as an example
- Trait extension with KubeWatch as an example
The following section takes a built-in WorkloadDefinition as an example to introduce the basic structure of the Definition file:
apiVersion: core.oam.dev/v1alpha2
kind: WorkloadDefinition
metadata:
name: webservice
annotations:
definition.oam.dev/description: "`Webservice` is a workload type to describe long-running, scalable, containerized services that have a stable network endpoint to receive external network traffic from customers.
If workload type is skipped for any service defined in Appfile, it will be defaulted to `Web Service` type."
spec:
definitionRef:
name: deployments.apps
extension:
template: |
output: {
apiVersion: "apps/v1"
kind: "Deployment"
spec: {
selector: matchLabels: {
"app.oam.dev/component": context.name
}
template: {
metadata: labels: {
"app.oam.dev/component": context.name
}
spec: {
containers: [{
name: context.name
image: parameter.image
if parameter["cmd"] != _|_ {
command: parameter.cmd
}
if parameter["env"] != _|_ {
env: parameter.env
}
if context["config"] != _|_ {
env: context.config
}
ports: [{
containerPort: parameter.port
}]
if parameter["cpu"] != _|_ {
resources: {
limits:
cpu: parameter.cpu
requests:
cpu: parameter.cpu
}}
}]
}}}
}
parameter: {
// +usage=Which image would you like to use for your service
// +short=i
image: string // +usage=Commands to run in the container
cmd?: [...string] // +usage=Which port do you want customer traffic sent to
// +short=p
port: *80 | int
// +usage=Define arguments by using environment variables
env?: [...{
// +usage=Environment variable name
name: string
// +usage=The value of the environment variable
value?: string
// +usage=Specifies a source the value of this var should come from
valueFrom?: {
// +usage=Selects a key of a secret in the pod's namespace
secretKeyRef: {
// +usage=The name of the secret in the pod's namespace to select from
name: string
// +usage=The key of the secret to select from. Must be a valid secret key
key: string
}
}
}]
// +usage=Number of CPU units for the service, like `0.5` (0.5 CPU core), `1` (1 CPU core)
cpu?: string
}
It seems long and complicated, but don’t worry, it is divided into two parts:
- Definition Registration Without Extended Fields
- CUE Template for Appfile
This article will break down these sections and make detailed explanations. It is very simple to learn.
Definition Registration Without Extended Fields
apiVersion: core.oam.dev/v1alpha2
kind: WorkloadDefinition
metadata:
name: webservice
annotations:
definition.oam.dev/description: "`Webservice` is a workload type to describe long-running, scalable, containerized services that have a stable network endpoint to receive external network traffic from customers.
If workload type is skipped for any service defined in Appfile, it will be defaulted to `Web Service` type."
spec:
definitionRef:
name: deployments.appsThis part is no more than 11 lines. In total, 3 lines introduce functions of webservice and 5 lines are fixed format. Only 2 lines have specific information.
definitionRef:
name: deployments.appsThese two lines represent the CRD name used in this Definition and the format is <resources>.<api-group>. People familiar with Kubernetes know the resources are located through api-group, version, and kind. kind corresponds to resources in the K8s restful API. Take Deployment and Ingress as an example, the relationship is as listed below:
Why introduce the concept of resources when a kind exists? A CRD, in addition to kind, has fields like status and replica. These fields need to be decoupled from the spec and updated separately in the RestfulAPI. Therefore, there are additional resources besides the one that corresponds to kind. For example, the status of the Deployment is deployments/status.
The simplest way to write Definition without extension is to splice them according to the combination of Kubernetes resources and write the name, resources, and api-group correspondingly:
apiVersion: core.oam.dev/v1alpha2
kind: WorkloadDefinition
metadata:
name: <Write name>
spec:
definitionRef:
name: <Write resources>.<Write api-group>The steps for O&M feature registration (TraitDefinition) are the same:
apiVersion: core.oam.dev/v1alpha2
kind: TraitDefinition
metadata:
name: <Write name>
spec:
definitionRef:
name: <Write resources>.<Write api-group>Then, Ingress, as an extension of KubeVela, is written:
apiVersion: core.oam.dev/v1alpha2
kind: TraitDefinition
metadata:
name: ingress
spec:
definitionRef:
name: ingresses.networking.k8s.ioIn addition, some other model layer functions have been added to TraitDefinition:
appliesToWorkloads: It indicates the Workload types that apply to the trait.conflictWith: It indicates the other types of traits that this trait conflicts with.workloadRefPath: It indicates the Workload field contained in a trait. This field is automatically filled when the trait object is generated in KubeVela.
All of these capabilities are optional. This article doesn’t cover its usage, but it will be elaborated on in subsequent articles.
Now that we have discussed a basic extension mode without extensions, the remainder of this article is about abstract templates based on CUE.
CUE Template for Appfile
For more details about CUE, please see Basic Introduction to CUE. This article will not elaborate on CUE.
KubeVela’s Appfile is simple to write, but the Kubernetes objects are relatively complex YAML files. KubeVela provides an easier way to keep the Appfile simple and extensible. This is what a CUE Template does in the Definition.
CUE Format Template
Let’s take a look at a YAML file of a Deployment, as shown below. Many of them are fixed frameworks in the template part. Users only need to fill in a small number of fields in the parameter part:
apiVersion: apps/v1
kind: Deployment
meadata:
name: mytest
spec:
template:
spec:
containers:
- name: mytest
env:
- name: a
value: b
image: nginx:v1
metadata:
labels:
app.oam.dev/component: mytest
selector:
matchLabels:
app.oam.dev/component: mytestIn KubeVela, the fixed format of a Definition file is divided into output and parameter. Output is the template section, and parameter is the parameter section.
Modify the Deployment YAML file to the template format of the Definition:
output: {
apiVersion: "apps/v1"
kind: "Deployment"
metadata: name: "mytest"
spec: {
selector: matchLabels: {
"app.oam.dev/component": "mytest"
}
template: {
metadata: labels: {
"app.oam.dev/component": "mytest"
}
spec: {
containers: [{
name: "mytest"
image: "nginx:v1"
env: [{name:"a",value:"b"}]
}]
}}}
}This format is very similar to JSON. This is the format of CUE, which is a superset of JSON. In other words, the CUE format adds some simple rules based on JSON rules, making it easier to read and use:
- It has an annotation style of the C language.
- Double quotation marks representing field names can be defaulted without special symbols.
- The commas at the end of the field values can be defaulted, and it is compatible even with commas at the end of the fields.
- The outermost braces can be omitted.
Template Parameter in CUE Format — Variable Reference
After the template section, it’s a parameter section. The parameter is a variable reference.
parameter: {
name: string
image: string
}
output: {
apiVersion: "apps/v1"
kind: "Deployment"
spec: {
selector: matchLabels: {
"app.oam.dev/component": parameter.name
}
template: {
metadata: labels: {
"app.oam.dev/component": parameter.name
}
spec: {
containers: [{
name: parameter.name
image: parameter.image
}]
}}}
}As shown in the preceding example, the template parameter in KubeVela is created through parameter, and parameter is used as a reference to replace some fields in output.
Complete Definition and Usage in Appfile
Through the combination of the two parts above, we can already write a complete Definition file.
apiVersion: core.oam.dev/v1alpha2
kind: WorkloadDefinition
metadata:
name: mydeploy
spec:
definitionRef:
name: deployments.apps
extension:
template: |
parameter: {
name: string
image: string
}
output: {
apiVersion: "apps/v1"
kind: "Deployment"
spec: {
selector: matchLabels: {
"app.oam.dev/component": parameter.name
}
template: {
metadata: labels: {
"app.oam.dev/component": parameter.name
}
spec: {
containers: [{
name: parameter.name
image: parameter.image
}]
}}}
}For debugging, it can be split into two files. One part is in the YAML file, named def.yaml:
apiVersion: core.oam.dev/v1alpha2
kind: WorkloadDefinition
metadata:
name: mydeploy
spec:
definitionRef:
name: deployments.apps
extension:
template: |The other is in the CUE file named def.cue:
parameter: {
name: string
image: string
}
output: {
apiVersion: "apps/v1"
kind: "Deployment"
spec: {
selector: matchLabels: {
"app.oam.dev/component": parameter.name
}
template: {
metadata: labels: {
"app.oam.dev/component": parameter.name
}
spec: {
containers: [{
name: parameter.name
image: parameter.image
}]
}}}
}First, format def.cue. During this process, the CUE tool does some verification. It can also perform deeper debugging by running the cue command.
cue fmt def.cueAfter debugging, the YAML file can be assembled using the script:
./hack/vela-templates/mergedef.sh def.yaml def.cue > mydeploy.yamlApply the YAML file to the Kubernetes cluster:
$ kubectl apply -f mydeploy.yaml
workloaddefinition.core.oam.dev/mydeploy createdOnce the new ability kubectl is applied to Kubernetes, there is no need to restart or update. Users of KubeVela can see the new capability and use it immediately.
$ vela workloads
Automatically discover capabilities successfully ✅ Add(1) Update(0) Delete(0)TYPE CATEGORY DESCRIPTION
+mydeploy workload description not definedNAME DESCRIPTION
mydeploy description not defined
The usage method in Appfile is listed below:
name: my-extend-app
services:
mysvc:
type: mydeploy
image: crccheck/hello-world
name: mysvcExecute vela up to run the application:
$ vela up -f docs/examples/blog-extension/my-extend-app.yaml
Parsing vela appfile ...
Loading templates ...Rendering configs for service (mysvc)...
Writing deploy config to (.vela/deploy.yaml)Applying deploy configs ...
Checking if app has been deployed...
App has not been deployed, creating a new deployment...
✅ App has been deployed 🚀🚀🚀
Port forward: vela port-forward my-extend-app
SSH: vela exec my-extend-app
Logging: vela logs my-extend-app
App status: vela status my-extend-app
Service status: vela status my-extend-app --svc mysvc
Let’s check the status of the application. Through HEALTHY Ready: 1/1, we know that the application is running normally.
$ vela status my-extend-app
About: Name: my-extend-app
Namespace: env-application
Created at: 2020-12-15 16:32:25.08233 +0800 CST
Updated at: 2020-12-15 16:32:25.08233 +0800 CSTServices: - Name: mysvc
Type: mydeploy
HEALTHY Ready: 1/1
Advanced Usage of Definition Templates
Above, we have already experienced the whole process of extending KubeVela through template replacement. Some complex requirements, such as conditional judgments, loops, complex types, require some advanced usage.
Struct Parameters
If there are some complex parameters in the template that contain structs and nested structs, the struct definition can be used.
1. Define a struct type that contains 1 string, 1 integer, and 1 struct:
#Config: {
name: string
value: int
other: {
key: string
value: string
}
}2. Use this struct type as an array in a variable:
parameter: {
name: string
image: string
config: [...#Config]
}3. The variable reference is used in the same target:
output: {
...
spec: {
containers: [{
name: parameter.name
image: parameter.image
env: parameter.config
}]
}
...
}4. The Appfile is written according to the structure defined by the parameter:
name: my-extend-app
services:
mysvc:
type: mydeploy
image: crccheck/hello-world
name: mysvc
config:
- name: a
value: 1
other:
key: mykey
value: myvalueCondition Judgment
Some parameters are added baed on certain conditions:
parameter: {
name: string
image: string
useENV: bool
}
output: {
...
spec: {
containers: [{
name: parameter.name
image: parameter.image
if parameter.useENV == true {
env: [{name: "my-env", value: "my-value"}]
}
}]
}
...
}Write a value in Appfile:
name: my-extend-app
services:
mysvc:
type: mydeploy
image: crccheck/hello-world
name: mysvc
useENV: trueDefault Parameters
In some cases, a parameter may or may not exist, which means a parameter is optional. Thus, it should match the conditions. When a field does not exist, the judgment condition is _variable ! = _|_ .
parameter: {
name: string
image: string
config?: [...#Config]
}
output: {
...
spec: {
containers: [{
name: parameter.name
image: parameter.image
if parameter.config != _|_ {
config: parameter.config
}
}]
}
...
}In this case, the config of the Appfile does not need to be filled in. If you fill it in, you should render it, and vice versa.
Default Value
The following can be used to set a default value for some parameters:
parameter: {
name: string
image: *"nginx:v1" | string
}
output: {
...
spec: {
containers: [{
name: parameter.name
image: parameter.image
}]
}
...
}In this case, the image parameter can be omitted in Appfile. The nginx:v1 can be used by default:
name: my-extend-app
services:
mysvc:
type: mydeploy
name: mysvcLoop
Map-Type Loop
parameter: {
name: string
image: string
env: [string]: string
}
output: {
spec: {
containers: [{
name: parameter.name
image: parameter.image
env: [
for k, v in parameter.env {
name: k
value: v
},
]
}]
}
}Write the following in the Appfile:
name: my-extend-app
services:
mysvc:
type: mydeploy
name: "mysvc"
image: "nginx"
env:
env1: value1
env2: value2Array-Type Loop
parameter: {
name: string
image: string
env: [...{name:string,value:string}]
}
output: {
...
spec: {
containers: [{
name: parameter.name
image: parameter.image
env: [
for _, v in parameter.env {
name: v.name
value: v.value
},
]
}]
}
}Write the following in the Appfile:
name: my-extend-app
services:
mysvc:
type: mydeploy
name: "mysvc"
image: "nginx"
env:
- name: env1
value: value1
- name: env2
value: value2Built-In Context Variable in KubeVela
You may notice that the names defined in the parameter are written twice in the Appfile. One is written under the services and each service is distinguished by its name. The other is written in specific name parameters. The repetition should not be written by users. Therefore, a built-in context is defined in KubeVela, which stores some common environment context information, such as the application names and keys. A name parameter doesn't have to be added while using the context in the template. The name parameter is automatically added when KubeVela runs the rendering template.
parameter: {
image: string
}
output: {
...
spec: {
containers: [{
name: context.name
image: parameter.image
}]
}
...
}Annotation Enhancements in KubeVela
KubeVela also provides some extensions to the annotations of cuelang to automatically generate documents for the CLI.
parameter: {
// +usage=Which image would you like to use for your service
// +short=i
image: string // +usage=Commands to run in the container
cmd?: [...string]
...
}
Annotations with +usgae at the beginning will become the descriptions of parameters, and the annotations with +short at the beginning are abbreviations used in CLI.
Summary
This article introduces the process and principle of adding a new capability in KubeVela and the writing method of a capability template through real-world cases and detailed descriptions.
After the addition of a new capability by a platform administrator, how can users of the platform learn how to use it? KubeVela can add new capabilities and automatically generate usage documents in Markdown format. You can check the KubeVela official website; all of the usage documents in References/Capabilities are automatically generated according to the template of each capability. Finally, you welcome to write some interesting extensions and submit them to KubeVela's Community Warehouse.
