Protecting Go Language Applications with the Graphene Library OS on Intel® SGX®-Secured Alibaba Cloud

Intel: Alexander M. Merritt, Isaku Yamahata, Gordon King, Kai Huang, Chen Zhang, Zhaorong Hou, Lei Li

Alibaba Cloud: Yanhui Zhao, Xiaoning Li, Li Shen

At the annual RSA USA Conference this year in San Francisco, Alibaba announced a collaboration with Intel for enabling protection of unmodified[1] Go language application binaries inside Graphene for Intel® SGX® enclaves on Alibaba Cloud ECS Bare Metal instance with Intel® SGX®.

Golang™ is a modern programming language developed at Google® Inc. with the goal to empower developers to be more productive. The language includes concurrency mechanisms, a new type system, and dynamic memory management to improve the safety and ease of implementation of applications. Today, Go powers many cloud-native systems and services in the open-source and enterprise communities, such as for databases, secure key management, cryptocurrency, etc. Ensuring a secure environment is paramount to users, developers, and providers of these services. Enabling SGX® protection for Go applications opens the door for customers to take advantage of the services of Alibaba Cloud, knowing their Go applications can use Intel® SGX® capabilities to help secure their application code, sensitive data, and behaviors.

The Graphene Library OS is an open-source framework that provides Linux®-compatible operating system services inside an Intel® SGX® enclave, within a library linked into the application. One objective of Graphene is to lower the entry point for launching unmodified applications in an SGX®-secured environment, and to provide seamless coupling for multi-process applications to work together via remote procedure call (RPC) coordination, while isolating arbitrary, unrelated groups of applications. By providing Graphene on Alibaba Cloud ECS Bare Metal instances with Intel SGX®, customers can more readily take advantage of the security and isolation Intel® SGX® provides, with less effort. Currently, these servers are available in several regions in China. When selecting the servers, look for Intel® Xeon® E3–1240 v6 (Skylake®) processors. Select Ubuntu® 16.04 as the OS installed on these servers.

Challenges with Go and SGX®

To help maintain the security of application code, an Intel® SGX®-enabled CPU carefully controls the execution context when either entering or exiting an enclave. For example:

• A user execution stack is not visible outside the enclave, either in the same process or when executing in the operating system kernel;
• Register values are loaded with synthetic data upon exit (and correctly restored upon re-entry);
• Certain instructions, such as SYSCALL, are forbidden from use at runtime within an enclave, forcing a protected exit of the enclave for handling; and,
• A limited, specific set of pages of physical memory are designated for use only within an enclave, a level of protection added to the traditional virtual memory translation layers.

A typical workflow for enabling applications in SGX® is to carve out the memory within an application that needs protection, and help ensure that enclave exiting happens as infrequently as possible, to limit the impact on performance. By enabling Graphene for Go applications, we can make this process easier, without the need by a user to manually partition their applications in enclave-internal and enclave-external components.

The Go runtime over-allocates ample amounts of virtual address space at application launch, then manages the physical backing of virtual pages at runtime, a common performance technique for heap memory management[2]. On SGX®-enabled CPUs, there is a limit to the number of physical memory pages which can be added to the set of all enclaves executing on a CPU package, currently at 64 GiB[3]. A process on Linux® can easily allocate terabytes of unbacked virtual memory space, as the Go runtime does, but in order to compose an enclave for an application using Graphene, we must indiscriminately add all such potential use of virtual memory space. This presents a critical barrier for launching even simple “Hello World” example applications.

In the short-term, we have identified how to patch unmodified Go application binaries to help limit the total size of virtual memory used for the Go garbage-collected heap space. In parallel, we are pursuing longer-term solutions to upstream in the Go language sources that help enable runtime adjustment of this virtual memory allocation size. Intel processors implementing SGX2® support larger enclave address spaces and are able to dynamically add to and remove from the enclave encrypted memory pages at runtime, i.e., after the enclave has been created and launched.

As SGX® disables certain instructions within an enclave, we are presented with another challenge for executing unmodified Go applications. Typically, applications are compiled to be dynamically linked to facilitate modularity in development: at launch, a dynamic linker pieces together a process image composed of the application executable and all dependent libraries, discovered at runtime from storage. Symbols referenced in the application are resolved by the linker, to point to instances found in dynamically linked-in libraries. One advantage of dynamic linking is to allow libraries to be modified without recompiling an application, as the linker lazily resolves symbols at application launch. To support existing C-based applications, Graphene provides a customized GNU C Library implementation that does not issue such instructions, enabling the use of operating system services in Graphene within an enclave, without recompilation.

On the other hand, Go applications are compiled to static binaries, meaning all dependent libraries they use are linked into the primary executable binary, and no linker is executed at application launch. Graphene cannot provide alternative implementations of Go runtime routines for a Go application to use, after an application was compiled statically. The Go runtime, like the GNU C Library, contains code that will directly interface with the operating system kernel, such as the execution of SYSCALL instructions.

We are collaborating on a solution comprised of minimal offline and online binary patching to unmodified Go application binaries that intercept instructions like SYSCALL without exiting an enclave, redirecting execution to the Linux®-compatible Graphene Library OS. Preserving correctness of the original Go implementation and the x86–64 and Linux® kernel-user space application binary interfaces (ABI) are a priority.

Challenges with Go and Graphene

As the Go runtime sits directly over the operating system kernel interface, its implementation can make assumptions about the behavior of this interface — the Linux® x86–64 Application Binary Interface (ABI). By sliding in the Graphene Library OS between the Go runtime and the kernel, we encounter challenges which touch on these assumptions, discussed below.

The Linux® kernel-user space ABI specifies system calls will not modify the user stack, i.e., writing values, or growing or shrinking the stack. When user code initially calls into the Library OS to emulate Linux® functionality, we are still using the application stack. The challenge encountered here relates to Go’s management of its stacks: they are bounded but resizable. Growing a Go user stack is triggered in Go routines when the remaining stack size is below a threshold (e.g., 128 bytes). Sometimes, when a Go application’s execution is redirected to the Graphene Library OS, we may have an amount of stack space that is insufficient for execution within the Library OS, potentially risking stack overflow. For Go routines invoking each other this is not a problem, because checks are in place to manage the stack. Within Graphene we have to be aware of this, as the Go runtime does not expect any other software layer to exist between it and the operating system interface, such as the Library OS layer.

Instead of interfacing with Go to participate in requesting the user stack be resized, we modify Graphene to instead dedicate a per-thread stack that is swapped in upon entry into Graphene, and swapped out when returned back to the Go application. Doing so, we can preserve the Go stack and limit dependencies between layers, improving application stability.

Other challenges include specially handling corner cases around this interface, such as how thread and signal management deviate from the traditional GNU C Library implementation. Supporting and protecting Go applications presents additional challenges that we will not have space to dive into in this blog entry.

As a team, we are actively participating in the Graphene community so that these features will be included in the next release. We are also planning to host this feature on our own repository for the purpose of public evaluation and validation.

Learn More about Go, Graphene, and Intel® SGX®

We hope this blog highlighted some interesting challenges we — Intel and Alibaba — are addressing, in our effort to improve security and protection of Go language applications by making features of Intel SGX more easily available to use.

To learn more about Intel® SGX® please see here. For working with Intel® SGX® on Alibaba Cloud ECS Bare Metal instances with Intel® SGX®, please see here.

More about the Go programming language can be found on their website.

The Graphene Library OS is an open-source project. To read more, please visit the homepage here. Its development is active on GitHub, found here.

About Alibaba Cloud

Established in 2009, Alibaba Cloud, the cloud computing arm of Alibaba Group, is among the world’s top three IaaS providers according to Gartner, and the largest provider of public cloud services in China according to IDC. Alibaba Cloud provides a comprehensive suite of cloud computing services to businesses worldwide, including merchants doing business on Alibaba Group marketplaces, start-ups, corporations and government organizations. Alibaba Cloud is the official Cloud Services Partner of the International Olympic Committee.

About Intel®

We are a world leader in the design and manufacturing of essential technologies that power the cloud and an increasingly smart, connected world. We offer computing, networking, data storage, and communications solutions to a broad set of customers spanning multiple industries. In 1968, Intel® was incorporated in California (reincorporated in Delaware in 1989), in what became known as Silicon Valley, and our technology has been at the heart of computing breakthroughs ever since.

We’re now in the midst of a corporate transformation as we grow beyond our traditional PC and server businesses into data-rich markets addressing the explosive demands to process, analyze, store, and transfer data. The transformation is well underway, with our data-centric businesses representing an increasing share of our overall revenue.

Our vision is to build a smart and connected world that runs on Intel® solutions. This vision is supported by our commitment to corporate responsibility, our relentless pursuit of Moore’s Law, and the talent of our amazing employees.

References

[1] “Unmodified” here means no source-code-level changes, nor recompilation of original sources.

[2] See also jemalloc. Allocation of virtual memory via mmap(2) is not thread-scalable on Linux®, but faulting in of physical memory can occur concurrently among threads. See https://doi.org/10.1145/2150976.2150998

[3] cpuid -1 | grep -i maxenclavesize

Intel Legal Notices

No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.

Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No product or component can be absolutely secure. Check with your system manufacturer or retailer or learn more at intel.com.

All product and company names are trademarks™ or registered® trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.

Intel, the Intel logo, SGX, Xeon, Skylake are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.

Google and the Google logo are registered trademarks of Google LLC.

Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.

Ubuntu is a registered trademark of Canonical Ltd.

*Other names and brands may be claimed as the property of others.

© Intel Corporation

Original Source

https://www.alibabacloud.com/blog/594889?spm=a2c41.13092806.0.0