First Knative Attempt: A Quick Guide to Continuous Integration and Continuous Delivery

By Dongdao

After a prolonged discussion in the Knative community on how to replace the Build module with Tekton, the official Knative Build team has declared that Knative Build is no longer recommended.

Understanding Tekton is easy if you are familiar with the Knative Build. While Knative Build projects itself as a Kubernetes-native Build resource, Tekton describes itself as a Kubernetes-native pipeline resource. Though their positioning is very close, Tekton is designed with richer and more complete features. Hence, the community finally chose Tekton.

Let’s take a look at the core concepts of Tekton in the following section.

Quick Start to Tekton

Tekton consists of five core concepts and each of them provides services in the form of Custom Resource Definitions (CRDs). Let’s take a quick look at the five concepts in the following sections.

1) Task

It’s a task running template. It is referred to as a template because a task definition contains a variable, and for running the task, a particular value must be specified for the variable. A Tekton task is similar to a function in terms of definition. A task defines input parameters by using inputs.params, and a default value may be specified for each input parameter. The steps field of a task indicates the substeps of the current task. In detail, each step runs one image. Use the template syntax used by the input parameters of the task to set the startup parameters of the images.

kind: Task
metadata:
  name: task-with-parameters
spec:
  inputs:
    params:
      - name: flags
        type: array
      - name: someURL
        type: string
  steps:
    - name: build
      image: registry.cn-hangzhou.aliyuncs.com/knative-sample/alpine:3.9
      command: ["sh", "-c"]
      args: [ "echo ${inputs.params.flags} ; echo ${someURL}"]

TaskRun

A task never runs directly after definition. It is similar to a function that must be called before running. Therefore, define a TaskRun to ultimately run the task. TaskRun sets the parameters required by the task and refers to the to-be-run task by using the taskRef field.

apiVersion: tekton.dev/v1alpha1
kind: TaskRun
metadata:
  name: run-with-parameters
spec:
  taskRef:
    name: task-with-parameters
  inputs:
    params:
      - name: flags
        value: "--set"
      - name: someURL
        value: "https://github.com/knative-sample"

Pipeline

One TaskRun is able to run only one task. There is a need for a pipeline for orchestrating multiple tasks. A pipeline is a template for orchestrating tasks. The params field of the pipeline declares the input parameters that are required during the run process. The spec.tasks field of the pipeline defines the tasks to be orchestrated. The value of the Tasks field is an array. Tasks in the array are not run in the order of declaration of the array, instead, they are run in the order declared by runAfter. When parsing a CRD, the Tekton controller parses the order of the tasks and then sequentially runs the tasks according to the specified order. When orchestrating tasks, the pipeline needs to pass a required parameter into each task. The values of these parameters may come from the params of the pipeline.

apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: pipeline-with-parameters
spec:
  params:
    - name: context
      type: string
      description: Path to context
      default: /some/where/or/other
  tasks:
    - name: task-1
      taskRef:
        name: build
      params:
        - name: pathToDockerFile
          value: Dockerfile
        - name: pathToContext
          value: "${params.context}"
    - name: task-2
      taskRef:
        name: build-push
      runAfter:
        - source-to-image
      params:
        - name: pathToDockerFile
          value: Dockerfile
        - name: pathToContext
          value: "${params.context}"

PipelineRun

Similar to a task, a pipeline also doesn’t run directly after the definition. To run a pipeline, PipelineRun is required. PipelineRun helps to set the required input parameters for a pipeline and run the pipeline.

apiVersion: tekton.dev/v1alpha1
kind: PipelineRun
metadata:
  name: pipelinerun-with-parameters
spec:
  pipelineRef:
    name: pipeline-with-parameters
  params:
    - name: "context"
      value: "/workspace/examples/microservices/leeroy-web"

PipelineResource

After defining the four core concepts of Tekton in the preceding sections, now it’s clear how to define a task, run a task, and orchestrate tasks. However, another critical component, PipelineResource helps to share resources between tasks. For example, storing information about a Git repository in PipelineResource helps to share the data across all pipelines.

piVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: wizzbang-git
  namespace: default
spec:
  type: git
  params:
    - name: url
      value: https://github.com/wizzbangcorp/wizzbang.git
    - name: revision
      value: master

Authorization Information

It is imperative to authenticate the Git repositories and image repositories before use. Hence, such scenarios require a mechanism for setting authentication information. Tekton is a native orchestration system of Kubernetes and thus, allows to directly use the ServiceAccount mechanism of Kubernetes to implement authentication.

Let’s consider an example to understand this better.

• Define a Secret that saves the authentication information of the image repository.

apiVersion: v1
kind: Secret
metadata:
  name: ack-cr-push-secret
  annotations:
    tekton.dev/docker-0: https://registry.cn-hangzhou.aliyuncs.com
type: kubernetes.io/basic-auth
stringData:
  username: <cleartext non-encoded>
  password: <cleartext non-encoded>

• Define ServiceAccount with the preceding Secret.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: pipeline-account
secrets:
- name: ack-cr-push-secret

• Execute the following commands to reference ServiceAccount in PipelineRun.

apiVersion: tekton.dev/v1alpha1
kind: PipelineRun
metadata:
  generateName: tekton-kn-sample-
spec:
  pipelineRef:
    name: build-and-deploy-pipeline
... ...
  serviceAccount: pipeline-account

Hello World

Refer to the sample which is the Hello World for a complete Tekton. Let’s experience this Hello World together.

Kubernetes cluster, Knative, and Tekton are the prerequisites to go ahead with the demonstration. To complete the installation of Tekton, directly submit the release-v0.5.2.yaml file in tekton-knative to the Kubernetes cluster. Now, let's start to experience the automated process from source coding to building and eventually deployment based on Tekton.

Clone Code

Clone the code locally using the git clone https://github.com/knative-sample/tekton-knative command.

Create PipelineResource

For the main content, see resources/picalc-git.yaml.

The following figure shows how to save https://github.com/knative-sample/tekton-knative in the resource folder so that it is usable for other resources as well.

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: tekton-knative-git
spec:
  type: git
  params:
    - name: revision
      value: master
    - name: url
      value: https://github.com/knative-sample/tekton-knative

Create a Task

Next, create a task. For this example, let’s create the following two tasks:

• 1) source-to-image

This task is to compile the source code into an image. Refer to the tasks/source-to-image.yaml file for the main content.

Use Kaniko compile Docker images in containers. The parameters of this task are used to set some compilation context information, such as Dockerfile, ContextPath, and target image tags.

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: source-to-image
spec:
  inputs:
    resources:
      - name: git-source
        type: git
    params:
      - name: pathToContext
        description: The path to the build context, used by Kaniko - within the workspace
        default: .
      - name: pathToDockerFile
        description: The path to the dockerfile to build (relative to the context)
        default: Dockerfile
      - name: imageUrl
        description: Url of image repository
      - name: imageTag
        description: Tag to apply to the built image
        default: "latest"
  steps:
    - name: build-and-push
      image: registry.cn-hangzhou.aliyuncs.com/knative-sample/kaniko-project-executor:v0.10.0
      command:
        - /kaniko/executor
      args:
        - --dockerfile=${inputs.params.pathToDockerFile}
        - --destination=${inputs.params.imageUrl}:${inputs.params.imageTag}
        - --context=/workspace/git-source/${inputs.params.pathToContext}
      env:
      - name: DOCKER_CONFIG
        value: /builder/home/.docker

• 2) deploy-using-kubectl

Refer to the tasks/deploy-using-kubectl.yaml file for the main content.

As shown in the following code, this task mainly obtains information about the target image by using parameters and then runs a sed command to replace _IMAGE_ in the Knative Service yaml file with the target image. Then, use kubectl to publish the image to Kubernetes.

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: deploy-using-kubectl
spec:
  inputs:
    resources:
      - name: git-source
        type: git
    params:
      - name: pathToYamlFile
        description: The path to the yaml file to deploy within the git source
      - name: imageUrl
        description: Url of image repository
      - name: imageTag
        description: Tag of the images to be used.
        default: "latest"
  steps:
    - name: update-yaml
      image: alpine
      command: ["sed"]
      args:
        - "-i"
        - "-e"
        - "s;__IMAGE__;${inputs.params.imageUrl}:${inputs.params.imageTag};g"
        - "/workspace/git-source/${inputs.params.pathToYamlFile}"
    - name: run-kubectl
      image: registry.cn-hangzhou.aliyuncs.com/knative-sample/kubectl:v0.5.0
      command: ["kubectl"]
      args:
        - "apply"
        - "-f"
        - "/workspace/git-source/${inputs.params.pathToYamlFile}"

Define a Pipeline

Now, let’s orchestrate the two tasks created in the preceding section using a pipeline.

apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: build-and-deploy-pipeline
spec:
  resources:
    - name: git-source
      type: git
  params:
    - name: pathToContext
      description: The path to the build context, used by Kaniko - within the workspace
      default: src
    - name: pathToYamlFile
      description: The path to the yaml file to deploy within the git source
    - name: imageUrl
      description: Url of image repository
    - name: imageTag
      description: Tag to apply to the built image
  tasks:
  - name: source-to-image
    taskRef:
      name: source-to-image
    params:
      - name: pathToContext
        value: "${params.pathToContext}"
      - name: imageUrl
        value: "${params.imageUrl}"
      - name: imageTag
        value: "${params.imageTag}"
    resources:
      inputs:
        - name: git-source
          resource: git-source
  - name: deploy-to-cluster
    taskRef:
      name: deploy-using-kubectl
    runAfter:
      - source-to-image
    params:
      - name: pathToYamlFile
        value:  "${params.pathToYamlFile}"
      - name: imageUrl
        value: "${params.imageUrl}"
      - name: imageTag
        value: "${params.imageTag}"
    resources:
      inputs:
        - name: git-source
          resource: git-source

Authentication Information

As shown in the following code, define a Secret and ServiceAccount, and bind the ServiceAccount with the permission to run the Knative Service.

First, create a secret to save the authentication information of the image repository.

• Replace tekton.dev/docker-0 with the address of the image repository that you want to push.
• Replace username with the authentication username of the image repository.
• Replace the password with the authentication password of the image repository.

apiVersion: v1
kind: Secret
metadata:
  name: ack-cr-push-secret
  annotations:
    tekton.dev/docker-0: https://registry.cn-hangzhou.aliyuncs.com
type: kubernetes.io/basic-auth
stringData:
  username: <cleartext non-encoded>
  password: <cleartext non-encoded>

Save the information to the file, and then submit the file to Kubernetes by running the kubectl apply-f command.

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: pipeline-account
secrets:
- name: ack-cr-push-secret
---
apiVersion: v1
kind: Secret
metadata:
  name: kube-api-secret
  annotations:
    kubernetes.io/service-account.name: pipeline-account
type: kubernetes.io/service-account-token
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: pipeline-role
rules:
- apiGroups: ["serving.knative.dev"]
  resources: ["services"]
  verbs: ["get", "create", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: pipeline-role-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: pipeline-role
subjects:
- kind: ServiceAccount
  name: pipeline-account

Define a PipelineRun

The authentication information corresponding to ServiceAccount is bound to PipelineRun for running.

For more details, refer to run/picalc-pipeline-run.yaml file.

apiVersion: tekton.dev/v1alpha1
kind: PipelineRun
metadata:
  generateName: tekton-kn-sample-
spec:
  pipelineRef:
    name: build-and-deploy-pipeline
  resources:
    - name: git-source
      resourceRef:
        name: tekton-knative-git
  params:
    - name: pathToContext
      value: "src"
    - name: pathToYamlFile
      value: "knative/helloworld-go.yaml"
    - name: imageUrl
      value: "registry.cn-hangzhou.aliyuncs.com/knative-sample/tekton-knative-helloworld"
    - name: imageTag
      value: "1.0"
  trigger:
    type: manual
  serviceAccount: pipeline-account

Run Hello World

Prepare pipeline resources as shown below.

kubectl apply -f tasks/source-to-image.yaml -f tasks/deploy-using-kubectl.yaml  -f resources/picalc-git.yaml -f image-secret.yaml -f pipeline-account.yaml -f pipeline/build-and-deploy-pipeline.yaml

Run create to submit PipelineRun to the Kubernetes cluster. Use the create command instead of the apply command since a PipelineRun is created each time, and a PipelineRun resource is created based on generateName in the create command of kubectl.

kubectl create -f run/picalc-pipeline-run.yaml

Finally, the pod information displays as shown in the following snippet.

└─# kubectl get pod
NAME                                                        READY   STATUS      RESTARTS   AGE
tekton-kn-sample-45d84-deploy-to-cluster-wfrzx-pod-f093ef   0/3     Completed   0          8h
tekton-kn-sample-45d84-source-to-image-7zpqn-pod-c2d20c     0/2     Completed   0

Original Source:

First Knative Attempt: A Quick Guide to Continuous Integration and Continuous Delivery