Measuring Integration Test Coverage Rate in PouchContainer e

PouchContainer uses unit testing and integration testing to ensure code quality. When submitting PR, developers are required to provide corresponding code of unit testing and integration testing. This requirement ensures regression quality, reduces code review costs, and improves cooperation efficiency. PouchContainer uses go testto measure the coverage rates of unit testing and integration testing, and works with TravisCI and Codecov to run tests for every PR submission and display data about test coverage rates clearly. It is complex to measure the coverage rate of integration testing. This document describes how such measurement is done in PouchContainer.

Go Test Coverage Rate

Before introducing integration test coverage rate measurement, we need to understand the working of coverage rate measurement in Golang. In Golang, coverage rates are measured by overriding the package source code before compilation to add statistics and then compiling and executing the code. The following describes the specific execution process with reference to the preceding document.

package size
func Size(a int) string {
switch {
case a < 0:
return "negative"
case a == 0:
return "zero"
case a < 10:
return "small"
}
return "enormous"
}
$ cat size_test.go
package size
import (
"testing"
"fmt"
)
type Test struct {
in int
out string
}
var tests = []Test{
{-1, "negative"},
{5, "small"},
}
func TestSize(t *testing.T) {
fmt.Println("a")
for i, test := range tests {
size := Size(test.in)
if size != test.out {
t.Errorf("#%d: Size(%d)=%s; want %s", i, test.in, size, test.out)
}
}
}
$ go test -x -cover -coverprofile=./size.out
WORK=/var/folders/d2/0gxc6wf16hb6t8ng0w00czpm0000gn/T/go-build982568783
mkdir -p $WORK/test/_test/
mkdir -p $WORK/test/_test/_obj_test/
cd $WORK/test/_test/_obj_test/
/usr/local/go/pkg/tool/darwin_amd64/cover -mode set -var GoCover_0 -o .size.go /Users/letty/work/code/go/src/test/size.go
cd /Users/letty/work/code/go/src/test
/usr/local/go/pkg/tool/darwin_amd64/compile -o $WORK/test/_test/test.a -trimpath $WORK -p test -complete -buildid 6033df309978241f19d83a0e6bad252ee3ba376e -D _/Users/letty/work/code/go/src/test -I $WORK -pack $WORK/test/_test/_obj_test/size.go ./size_test.go
cd $WORK/test/_test
/usr/local/go/pkg/tool/darwin_amd64/compile -o ./main.a -trimpath $WORK -p main -complete -D "" -I . -I $WORK -pack ./_testmain.go
cd .
/usr/local/go/pkg/tool/darwin_amd64/link -o $WORK/test/_test/test.test -L $WORK/test/_test -L $WORK -w -extld=clang -buildmode=exe $WORK/test/_test/main.a
$WORK/test/_test/test.test -test.coverprofile=./size.out -test.outputdir /Users/letty/work/code/go/src/test
a
PASS
coverage: 60.0% of statements
ok test 0.006s
$ cat .size.go
package size
func Size(a int) string {
GoCover_0.Count[0] = 1
switch {
case a < 0:
GoCover_0.Count[2] = 1
return "negative"
case a == 0:
GoCover_0.Count[3] = 1
return "zero"
case a < 10:
GoCover_0.Count[4] = 1
return "small"
}
GoCover_0.Count[1] = 1
return "enormous"
}
var GoCover_0 = struct {
Count [5]uint32
Pos [3 * 5]uint32
NumStmt [5]uint16
} {
Pos: [3 * 5]uint32{
3, 4, 0x9001a, // [0]
12, 12, 0x130002, // [1]
5, 6, 0x14000d, // [2]
7, 8, 0x10000e, // [3]
9, 10, 0x11000e, // [4]
},
NumStmt: [5]uint16{
1, // 0
1, // 1
1, // 2
1, // 3
1, // 4
},
}
$ cat size.out
mode: set
test/size.go:3.26,4.9 1 1
test/size.go:12.2,12.19 1 0
test/size.go:5.13,6.20 1 1
test/size.go:7.14,8.16 1 0
test/size.go:9.14,10.17 1 1

PouchContainer Test Coverage Rate

PouchContainer integrates the CodeCov tool to upload a statistics file on test coverage rates to the CodeCov website each time when TravisCI is executed for visual display and continuous tracking of test coverage rates. TravisCI and CodeCov are easy to integrate. A statistics file on test coverage rates can be uploaded after a file named coverage.txt is generated in the test path and the CodeCov script is called in the .travis.yml file. For related commands, see pouch. You can also view the CodeCov script for the implementation details. The following describes how to measure the coverage rates of unit testing and integration testing in PouchContainer.

Measure the Unit Test Coverage Rate

In PouchContainer, you only need to run the go test cover command to measure the coverage rate of a unit test. To get the code used to measure the unit test coverage rate, go to pouch/hack/build and find the unit-test() function, which runs the unit test of each package and appends the measured test coverage rate to the coverage.txt file. Note that unrelated packages such as the vendor and types directories must be excluded during test coverage rate measurement; otherwise, the measurement may be inaccurate.

Measuring the Integration Test Coverage Rate

In PouchContainer integration testing, daemon APIs and command lines are tested by starting pouch daemon and running the pouch command or sending API requests. Normally, pouch daemon is compiled by go build. The test coverage rate cannot be measured because no counters are inserted into the source code.

  1. Add the testserver function to the hack/build script to compile the main package and generate an executable test file.
  2. In the hack/make.sh script, execute the test file generated in Step 2 in the background, and run API and command line tests.
  3. Send a signal to the test process and collect the test coverage rate after the test.
package mainimport (
"os"
"os/signal"
"strings"
"syscall"
"testing"
)
func TestMain(t *testing.T) {
var (
args []string
)
for _, arg := range os.Args {
switch {
case strings.HasPrefix(arg, "DEVEL"):
case strings.HasPrefix(arg, "-test"):
default:
args = append(args, arg)
}
}
waitCh := make(chan int, 1) os.Args = args
go func() {
main()
close(waitCh)
}()
signalCh := make(chan os.Signal, 1)
signal.Notify(signalCh, syscall.SIGINT, syscall.SIGQUIT, syscall.SIGTERM, syscall.SIGHUP)
select {
case <-signalCh:
return
case <-waitCh:
return
}
}
# go test -c -race -cover -covermode=atomic -o pouchd-test -coverpkg $pkgs
# pouchd-test -test.coverprofile=$DIR/integrationcover.out DEVEL --debug

Conclusion

To measure the integration test coverage rate, you need to use the tools provided by Golang with flexibility, and adapt test files based on the characteristics of your project code. After measurement of integration test coverage rate is added, the coverage rate of PouchContainer increases from 18% (only for measurement of unit test coverage rate) to 40% to reflect the current test status more accurately.

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store