Cloud Monitor Log Generation from Sample Application

By Amit Maity, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud’s incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.


Alibaba Cloud Monitor service allows monitoring of cloud resource usage and health of the application/resources by collecting various monitoring metrics. In this tutorial, we will see how Cloud Monitor can help us notified in case of an exception in an application. Sample application, which is used here to demonstrate the cloud monitor use case, is a simple web application for uploading file into OSS bucket.

This application will write error logs into Alibaba Log Service and logs are being pulled into Cloud Monitor to monitor sample application exceptions.

There are few more use cases possible using the same idea.

  • Measure throughput or performance of the application from log entry timings.


In this below design, I have implemented a scenario where system administrator is notified by email if there is any failure while using file upload web application. The failure could be loss of connection between OSS and application or any other OSS technical/permission issue.

File upload web application is designed to write the error log message into “Log Service”. Cloud Monitor monitors the log message written into Log Service. If Log Monitor identified number of error log exceeds the threshold count, then it will automatically send email alert to the system administrator stating that web application encountered an error.


This tutorial can easily be understood and followed if you are familiar with the below technologies and concepts,

  • Alibaba Cloud Environment
    If you do not have access, get 1 year free access with $10 credit using the below link
    Alibaba Cloud Free Account

Server Configurations

To develop and run the web application you will need a host server (Alibaba Cloud ECS or any other windows/unix Server) with python 2.7 installed.

Download and install following Python packages and Alibaba Cloud Python SDKs in host server

Purchase and Activate following services in your cloud account,

  • Object Storage Service (OSS)

Configuring OSS, Log Service and Cloud Monitor

Please choose a single region for configuring the all the required Alibaba cloud services before you proceed. This will help you to avoid any technical challenges for integrating multi region services.

  1. Create a new bucket in your OSS console to store the files, which you are going to upload using this sample application.

Keep a note of bucket name and OSS end point. Later you’ll have to update the configuration file of the application with bucket name and OSS end point.

2. Create Log Service project in Log Service console. Choose region name same as OSS region.

3. Under Log Service project, create logstore with below screeshot.

4. Click on the Endpoints button as shown below to get the log service end points. This end point url needs to be updated in config.cfg file.

5. Open Cloud Service Monitor console for Log Service. You should be able to see log service project and associated LogStore, which are setup in previous steps.

6. Click on Alarm Rules link to create Alarm Rule to trigger email alert for error log entry in Log Service. This alarm is threshold-based alarm, which is executed if the monitoring criteria specified in rule description matches with the monitoring data.

Configure “Alarm Rules” to trigger the alert if there is at least 1 error log line is written into log service per minute.

Configure “Notification Method” to specify the email address of the alert email recipient. Create a contact group with the email address of the recipient if there is no contact group available.

Fill up the Email Subject and Email Remark fields with the email subject and email body content as per your requirement.

Sample Application Walkthrough

Here is the folder and file structure,

sampleWebApplication││  config.cfg|│  LICENSE│││└─── app││└─── templatesbase.htmlupload.html

a. config.cfg

This file keeps all required configuration values for running the application. Update the configuration parameters in this file according to your cloud account/service details, before running the application

AccessKeyId = <Cloud Account Access Key>AccessKeySecret = < Cloud Account Access Secret Key >LogEndpoint = <Log Service End points url>i.e. ap-southeast-1.log.aliyuncs.comLogProject = <Log Service Project Name>i.e. sampleapplicationLogstore = <Log Service Project Logstore name>i.e. sampleapplicationlogstoreOSSEndpoint = <OSS End Point url>i.e. oss-cn-shenzhen.aliyuncs.comOSSBucketName = <OSS Bucket Name>i.e. samplewebapp


This is to hold configuration options for Flask application sessions.


This script is for parsing the required configuration parameter values from config.cfg and return the values to main python scripts for using in invoking cloud APIs.

# Read config.cfg and parse the configuration parametersparser = ConfigParser.ConfigParser() Assign each of the configuration parameters values to separate varaiablesaccessKeyId = parser.get("Base", "AccessKeyId")accessKeySecret = parser.get("Base", "AccessKeySecret")logendpoint = parser.get("Base", "LogEndpoint")...return accessKeyId,accessKeySecret,logendpoint,logproject,logstore . . .


This script defines the reusable function write() to write log messages into logstore of the log service.

Import the common class from to retrieve the access key id and secrect key and other required parameters for calling log service API,

from common import Common

Import log service modules

from aliyun.log.logitem import LogItemfrom aliyun.log.putlogsrequest import PutLogsRequest

Defining write static method with below parameters to invoke log service APIs

@staticmethoddef write(client, project, logstore, topic, source, contents):

client: Mandatory, LogClient object

project: Mandatory, name of the Log Service project

logstore: Mandatory, name of the Logstore in Log Service Project

topic: Optional, to classify the log messages by topic

source: Optional, to update the source of the log messages

contents: Mandatory, actual content of the log message

Build an array of log item object with the log message content as Log service API expects collection of log item object.

logItem = LogItem()logItem.set_time(int(time.time()))logItem.set_contents(contents)logitemList.append(logItem)

Get Log Request Object with all necessary request parameter values and pass the request object to put_logs method of Log Service API to write the log message into Log Service logstore.

req = PutLogsRequest(project, logstore, topic, source, logitemList)res = client.put_logs(req)

Below scripts are written to develop web application using Flask web framework. To understand the basic of Flask web framework, please refer Flask Documentation.


This script is for referring and pointing to flask application package developed for this demonstration.

Use below commands to run this flask application,

set FLASK_APP=samplewebapp.pyflask run

f. app/__init.py__

Import Flask package and instantiate Flask application

from flask import Flaskapp = Flask(__name__)

g. app/templates/base.html

This is HTML template so that this can be rendered as web page by Flask engine. This template file contains basic HTML web page structure with url links in the navigation bar.

h. app/templates/upload.html

This is a template to display the file upload form and extension to base.html web page.

{% block content %}<form action = "/upload" method = "POST"enctype = "multipart/form-data"><input type = "file" name = "file" /><input type = "submit"/></form>{% endblock %}

Using below syntax in base.html, upload.html form is appended to the main base.html web page.

{% block content %}{% endblock %}

i. app/

Web application forms view and controller logic is included in this script. This renders the html templates to display web form in the browser. This script is heart of this sample application.

upload_file() view function mapped to the /upload URL that creates a file upload web form, and passes it to the html templates for rendering

@app.route('/upload',methods = ['GET', 'POST'])def upload_file():if request.method == 'POST':. . . . .bucket.put_object(f.filename,'CustomUpload')flash('File uploaded successfully!'). . . . .return render_template('upload.html')

Once file is uploaded by the browser, upload_file() function uses OSS2 API method put_object to upload the file into OSS bucket.

Additional logic for writing into Log Service is included in this script,

Import SDKs, common and log modules

from aliyun.log.logclient import LogClientsys.path.insert(0, os.path.dirname(os.path.abspath(__file__)) + "/..")from common import Commonfrom log import Log

Retrieve the configuration parameter values from config.cfg

accessKeyId,accessKeySecret,logEndPoint,logProject,logStore,ossEndPoint,ossBucketName,securityToken = Common.LoadConfig()

Instantiate LogClient class with authentication details before using Log Service API

client = LogClient(logProject+'.'+logEndPoint, accessKeyId, accessKeySecret)

Use exception handling for OSS bucket write API. In case of API failure, update contents variable with error message and then use Log.write function to upload the error message into Log Service log store.

try:bucket.put_object(f.filename,'CustomUpload'). . . . .except Exception, e:contents = [('Msg', str(e) )]Log.write(client, logProject, logStore, '', '', contents). . . . .

Run and Test

Now let’s run the application.

In windows command prompt, change directory to sampleWebApplication folder and then import the flask application, by setting FLASK_APP environment variable as follows,

In Unix environment, use export command instead of set.

Run the flask application using “flask run” command,

Open the web application in any browser using the url

Upload any file and click submit button.

Once the file is successfully uploaded, you’ll see the uploaded file in OSS bucket

Now, to simulate the upload failure let’s delete this bucket and again try uploading the same file.

This time application is not be able to upload and displaying message — “File upload failure” in web page.

Reason or log details of upload failure is available in sampleapplogstore console in Raw Logs tab as shown in below screenshot.

Above log entry will cause Cloud Monitor alarm threshold to exceed. Due to this, “Send Email Alert” alarm in Cloud Monitor is triggered which is visible in Cloud Monitor console.

As “Send Email Alert” alarm is triggered and you should receive alert email from cloud monitoring service. Below is the sample email, which is generated by Cloud Monitoring Alarm.


Follow me to keep abreast with the latest technology news, industry insights, and developer trends.