Setting Up Log Service to Analyze Windows Event Logs

By Steve Chen, Solutions Architect

Solution Design and Implementation

There are many enterprises who use Windows systems as the majority of the on-premises or cloud infrastructure. However, with the growing number of the hosts and configurations, it becomes very difficult for the IT administrator to manage such a large scale of asset as a part of their daily tasks.

Alibaba Cloud Log Service is a cost-effective and easy-to-manage product to collect local logs from all different log sources, centralize the log storage on the cloud, and analyze the logs in a various ways.

Following sections generally introduce a solution of how to collect logs from a Windows Active Directory (AD) server, store them in Alibaba Cloud Log Service, and define different metrics in the dashboard to analyze AD activities.

The solution architecture is presented in the following graph. Logtail is a log collection agent provided by Alibaba Cloud Log Service. You can use Logtail to collect logs from servers such as Alibaba Cloud Elastic Compute Service (ECS) instances in real time, however it can work directly with Windows Event Log. So we introduce an open source agent which is called Winlogbeat to be connected with the AD Event Log and output all the event logs to a local file in JSON format, and then Logtail will timely monitor any change in the JSON file and update the incremental log to the Log Service. Once Windows logs arrive the log service on the cloud, customer can either analyze the logs in the Log Service by using SQL scripts and dashboard metrics, or ship the logs to Object Storage Service (OSS) for achieve, or move to MaxCompute or EMR which are Alibaba Cloud’s big data warehouse products to process the data in a complex way which might involve machine learnings and AI.

Installing the Winlogbeat and Logtail Agent Locally on the AD Server

Download and install Winlogbeat, check the service is up and running as following,

Make the configuration in the Winlogbeat to be able to connect to windows event log and save logs to a local JSON file. Following is just an example of the configuration.

Download and install Alibaba Cloud Logtail agent locally on the AD server as well, and the detail of the installation can be found in the following link. In the configuration file, make sure Logtail monitor the Winlogbeat JSON file.

Configuring Logtail and Logstore

Set up logstore and Logtail in Alibaba Cloud console in order to receive the log from Logtail, which is installed on the AD. The details of the configurations can be found in the following page:

After the proper configuration, you should be able to receive and see following Windows event logs from the AD server in the logstore console.

Log Analysis Cases by Using SQL Scripts

Based on the logs collected from the AD, you can implement metrics based on your business concern by using SQL scripts. Following are the examples in the demo.

Logon Failure Statistics

Logon Attempt against Disabled Account

User Added to Admin Group

Consecutive Logon Failures

Create Custom Tables and Use Custom OSS Backed Tables in Dashboards

Consecutive Logon Failure against High Value Assets

Data Presentation on Dashboard

The following are the dashboards defined in the Log Service console for the same SQL analytics cases, which will presents the statistics in different charts, table and graphs.


Follow me to keep abreast with the latest technology news, industry insights, and developer trends.