Develop Function Compute by Installing Third-Party Dependencies

Image for post
Image for post

By Yi Xian

This article shows the new feature of the Fun tool and fun install, which can help you easily install apt and pip software packages. Also, through this, you can install these third-party dependencies to the system directory of Function Compute, developing Function Compute. But before we get to all the nitty gritty stuff of this article, let’s first take a look at several important concepts mentioned in this article.

Background Information

Next, there’s Fun, which is a deployment tool for serverless applications. It helps you manage resources, such as Function Compute, API Gateway, and Log Service. You can use Fun to develop, build, and deploy resources by describing specified resources in the template.yml file.

Last, there’s also fun install, which is a sub-command of the Fun tool for installing pip and apt dependencies. It provides two forms, command line interface and fun.yml description file.

Note: The tips described in this article require the Fun version 2.9.3 and later.

For Function Compute, installing third-party dependencies is a big pain point. However, Fun install solidifies the best practices into tools based on previous experiences and achievements, which is convenient for users to install dependencies.

Procedure

Use the Fun Install Init

Run the fun install init command in the root directory of the Function Compute project, and choose a runtime.

$ fun install init
? Select runtime (Use arrow keys)
python2.7
python3
nodejs6
nodejs8
java8
php7.2

A fun.yml file is generated in the current directory. The content is as follows:

runtime: python2.7
tasks: []

Install pip package dependency

$ fun install --runtime python2.7 --package-type pip tensorflow
skip pulling image aliyunfc/runtime-python2.7:build-1.2.0...
Task => [UNNAMED]
=> PYTHONUSERBASE=/code/.fun/python pip install --user tensorflow

Below is a description of the parameters in the above code script:

  • --runtime specifies the runtime. If the fun.yml file has been initialized, this option can be omitted because the runtime is declared in the fun.yml.
  • --package-type specifies the type of dependency to install. pip and apt are the current two optional values.
  • tensorflow is a pip package name.

This command is run in the container provided by fc-docker. The commands run inside the container are printed line by line. For example, the PYTHONUSERBASE=/code/.fun/python pip install --user tensorflow command is actually run, too.

After the installation has complete, a .fun directory is generated, the runable files are placed in the .fun/python/bin directory, and the library files are placed in the .fun/python/lib/python2.7/site-packages.

.fun
└── python
├── bin
│ ├── freeze_graph
│ ├── markdown_py
│ ├── pbr
│ ├── saved_model_cli
│ ├── tensorboard
│ ├── tflite_convert
│ ├── toco
│ └── toco_from_protos
└── lib
└── python2.7
└── site-packages
├── tensorboard
├── tensorboard-1.12.2.dist-info
├── tensorflow
├── tensorflow-1.12.0.dist-info
├── termcolor-1.1.0.dist-info
...

Compared with the previous pip install -t . <package-name> method, the storage location of installed files for the "fun install" command is more organized, and the dependency files and code files are separated, which is convenient to clean up, and initialize the dependency files with OSS or NAS after splitting. However, this also brings a new issue, which is that it requires the user to customize the environment variable library file before it can be found by the program. For the convenience of users, the fun install env is provided to print the necessary environment variables.

$ fun install env
LD_LIBRARY_PATH=/code/.fun/root/usr/lib/x86_64-linux-gnu:/code:/code/lib:/usr/local/lib
PATH=/code/.fun/root/usr/local/bin:/code/.fun/root/usr/local/sbin:/code/.fun/root/usr/bin:/code/.fun/root/usr/sbin:/code/.fun/root/sbin:/code/.fun/root/bin:/code/.fun/python/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin:/bin
PYTHONUSERBASE=/code/.fun/python

If you use fun local and fun deploy for debugging and deployment, you do not need to worry about environment variables, which have been set up for you.

Use — save for Persistence

$ fun install --runtime python2.7 --package-type pip --save tensorflow
skip pulling image aliyunfc/runtime-python2.7:build-1.2.0...
Task => [UNNAMED]
=> PYTHONUSERBASE=/code/.fun/python pip install --user tensorflow

After the --save parameter is added to the above command, check the contents of fun.yml:

runtime: python2.7
tasks:
- pip: tensorflow
local: true

Then, directly run fun install without parameters to run the tasks in sequence.

$ fun install
skip pulling image aliyunfc/runtime-python2.7:build-1.2.0...
Task => [UNNAMED]
=> PYTHONUSERBASE=/code/.fun/python pip install --user tensorflow

Use -v to Display Detailed Logs

$ fun install -v
skip pulling image aliyunfc/runtime-python3.6:build-1.2.0...
Task => [UNNAMED]
=> apt-get update (if need)
Ign http://mirrors.aliyun.com stretch InRelease
Get:1 http://mirrors.aliyun.com stretch-updates InRelease [91.0 kB]
Get:2 http://mirrors.aliyun.com stretch-backports InRelease [91.8 kB]
Get:3 http://mirrors.aliyun.com stretch/updates InRelease [94.3 kB]
Hit http://mirrors.aliyun.com stretch Release.gpg
Hit http://mirrors.aliyun.com stretch Release
Get:4 http://mirrors.aliyun.com stretch-updates/main Sources [3911 B]
....

Install apt package dependency

$ fun install --runtime python3 --package-type apt libzbar0
skip pulling image aliyunfc/runtime-python3.6:build-1.2.0...
Task => [UNNAMED]
=> apt-get update (if need)
=> apt-get install -y -d -o=dir::cache=/code/.fun/tmp libzbar0
=> bash -c 'for f in $(ls /code/.fun/tmp/archives/*.deb); do dpkg -x $f /code/.fun/root; done;'
=> bash -c 'rm -rf /code/.fun/tmp/archives'

The usage method and its parameters of this command are similar to that of the pip package dependency. You only need to set --package-type to apt, and the package name to the deb package name that can be installed by apt-get.

Use fun.yml

The file format of fun.yml is as follows:

runtime: python3
tasks:
- name: install libzbar0
apt: libzbar0
local: true
- name install Pillow by pip
pip: Pillow
local: true
- name: just test shell task
shell: echo '111' > 1.txt

The runtime field is required. Currently, three types of tasks are available: apt, pip, and shell. The fun.yml file is stored in the directory to which the function codeUri in the template.yml file pointed. If multiple functions are declared in the template.yml and stored in different codeUri directories, multiple fun.yml files need to be created.

The name field of all tasks is optional. If the name field is not entered, the output is as follows:

Task => [UNNAMED]

apt/pip tasks

name: install libzbar0
apt: libzbar0
local: true

The preceding task description is equivalent to the following command:

fun install --package-type apt libzbar0

During installation using fun install, use the --save parameter to generate the description structure of the above task in the fun.yml file of the current directory.

The local field defaults to true, indicating that the dependency will be installed in the .fun subdirectory of the current directory, and will be packed together in the zip package. Set it to false to install the dependency into the system directory. This is generally used to compile the dependency. For example, if an execution file or library is required at compile or build time but not at run time, then local: false can be set, so that the file will be ignored during packaging, and the file size of the final zip package will not be affected.

shell tasks

name: install from source
shell: ./autogen.sh --disable-report-builder --disable-lpsolve --disable-coinmp

Examples

runtime: python3
tasks:
- apt: libzbar0
local: true
- pip: Pillow
local: true
- pip: pyzbar
local: true

Use fun install to install dependencies

$ fun install
skip pulling image aliyunfc/runtime-python3.6:build-1.2.0...
Task => [UNNAMED]
=> apt-get update (if need)
=> apt-get install -y -d -o=dir::cache=/code/.fun/tmp libzbar0
=> bash -c 'for f in $(ls /code/.fun/tmp/archives/*.deb); do dpkg -x $f /code/.fun/root; done;'
=> bash -c 'rm -rf /code/.fun/tmp/archives'
Task => [UNNAMED]
=> PYTHONUSERBASE=/code/.fun/python pip install --user Pillow
Task => [UNNAMED]
=> PYTHONUSERBASE=/code/.fun/python pip install --user pyzbar

The content of the template.yml file is as follows:

ROSTemplateFormatVersion: '2015-09-01'
Transform: 'Aliyun::Serverless-2018-04-03'
Resources:
pyzbar-srv:
Type: 'Aliyun::Serverless::Service'
pyzbar-fun:
Type: 'Aliyun::Serverless::Function'
Properties:
Handler: index.handler
Runtime: python3
Timeout: 60
MemorySize: 128
CodeUri: .

The content of the index.py file is as follows:

from pyzbar.pyzbar import decode
from pyzbar.pyzbar import ZBarSymbol
from PIL import Image
def handler(event, context):
img = Image.open('./qrcode.png')
return decode(img, symbols=[ZBarSymbol.QRCODE])[0].data

Use fun local to execute locally

fun local invoke pyzbar-fun
skip pulling image aliyunfc/runtime-python3.6:1.2.0...
Thalassiodracon
RequestId: 964980d1-1f1b-4f91-bfd8-eadd26a307b3 Billed Duration: 630 ms Memory Size: 1998 MB Max Memory Used: 32 MB

Thalassiodracon is the output result after identification.

Summary

Original Source

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store