Locate Online Node.js Memory Leaks

Background

Currently, two popular trends in cloud computing are containerization and microservices. In a microservices architecture, enterprises may need to solve a problem in their actual development scenarios. This issue is in particular the conflict between achieving stable service-side APIs while also being flexible with diverse user requirements.

Heap Snapshots

Obtain Heap Snapshots

To analyze and locate memory leak problems, first we need to obtain the objects and the reference relationship between them in the heap when a Node.js process experiences a memory leak. A heap snapshot is the file where the objects in the heap and the reference relationship are saved. The V8 engine provides an interface that allows you to easily obtain heap snapshots in real time.

Install heapdump

Run the following command to install the heapdump module:

npm install heapdump
const heapdump = require('heapdump');
'use strict';
const heapdump = require('heapdump');
const path = require('path');
setInterval(function() {
let filename = Date.now() + '.heapsnapshot';
heapdump.writeSnapshot(path.join(__dirname, filename));
}, 30 * 1000);
kill -USR2 <the PID of the process for which you want to obtain a heap snapshot>

Install v8-profiler

Run the following command to install the v8-profiler module:

npm install v8-profiler
'use strict';
const v8Profiler = require('v8-profiler-node8');
const snapshot = v8Profiler.takeSnapshot();
// Obtain transform stream of heap snapshots
const transform = snapshot.export();
// Process heap snapshot stream
transform.on('data', data => console.log(data));
// Delete data after data is processed
transform.on('finish', snapshot.delete.bind(snapshot));

Node.js Performance Platform

The above methods require you to install the npm module and embed corresponding hot operation logic into the code. In Node.js Performance Platform, the task of obtaining heap snapshots has been integrated into the runtime. After an application is connected to the platform, heap snapshots of processes can be obtained online for analysis, without having to modify business code:

Describe Heap Snapshots

Field Description

After opening the heap snapshot obtained in the previous section in any document reader, you can see that the snapshot is a large JSON file:

{
snapshot: {},
nodes: [],
edges: [],
strings: []
}
  • meta.node_types: an array. Elements in the array represent the meaning of each element of a node. In this example, the first of the six elements represents the node type, which also belongs to a limited array.
  • meta.edge_fields: an array. The length of the array is the number of elements that represent an edge. In this example, every three elements in the “edges” array represent one edge.
  • meta.node_types: an array. Elements in the array represent the meaning of each element of an edge. In this example, the first of the three elements represents the edge type, which also belongs to a limited array.

Nodes and Edges

The preceding information describes each node and each edge in the memory relation graph. However, the relationship between nodes and edges is not shown in the graph.

Locate Memory Leaks

Based on the information described in the previous section, we can obtain a memory relation graph like this:

References:

Original Source

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store