TeaDSL: A Multi-language SDK Solution for All API Gateways

Image for post
Image for post

By Puling

In the field of cloud computing, OpenAPIs are unambiguously acknowledged as the most vital tools for developers. OpenAPI allows programmers to include instructions in their programming, like sending text messages, organizing resources, and managing permissions. It simply allows performing another OpenAPI call if one is not enough. As more companies are opting for open platforms for flexibility, OpenAPIs are increasingly favored as an indispensable part of the integration infrastructure. However, a question remains as to whether the OpenAPIs are really consumer-friendly. This article explores the crucial challenges related to OpenAPIs and introduces tailored solutions.


Problems with OpenAPIs

Image for post
Image for post
Figure 1. 1st-generation OpenAPIs are composed of interfaces and have ill-defined documentation.

Well, here are the problems with this approach. First of all, the vaguely defined parameters are barely tested and there is no guarantee that they will work. Secondly, in order to ensure the security of the OpenAPIs, we must implement authentication/authorization measures, for example, with digital signatures. The providers must ensure the uniqueness of each customer’s signature, making it an error-prone process. Thirdly, providers must repackage the OpenAPIs for customers that write in other programming languages.

But that’s not all. Sometimes after you’ve successfully tested your OpenAPIs, you may run into errors, like a connection timeout or wrong return value. As OpenAPIs are never going to be completely stable, you must prepare for the inevitable challenges during the versioning, like changing data structures or making incompatible changes.

Not only are the consumers displeased with the hard-to-use OpenAPIs, but sometimes the maintainers also get weary of following up. After you publish an OpenAPI, you will find that writing brief documentation will no longer suffice, since you must also provide detailed descriptions of each parameter and even of the signing algorithm to facilitate programming in other languages. But afterward, you may notice that the signing algorithm is only good for some consumers, and the majority still rely on you to create an SDK. So, you build and provide an SDK in the language that you are most familiar with.

Image for post
Image for post
Figure 2. 2nd-generation OpenAPIs can be implemented by SDKs, but this implementation is only supported by a few languages.

As your OpenAPIs are being consumed in more and more languages, you must cater to the need. In my case as a Java developer, not only do I need to write the SDK for Java, I must also provide SDKs in all the other languages. Otherwise, the customers will not be able to implement the intelligent code completion on IDEs.

In short, the application of OpenAPIs could be very complicated:

  • Good API documentation is the most basic requirement.
  • An SDK is required for better developer experience, and features like the detail encapsulation and IDE auto-completion must be implemented.
  • Documentation must come with code samples to demonstrate real-world use-cases.
  • CLI can also be provided to make writing Bash scripts easier.
  • Test cases are a must for practicing continuous integration and performance assurance.

To think that the providers must not only satisfy all these needs but also do so in multiple languages, it’s critical to understand why the design process encompasses so much detail and requires hours of redundant labor work. In addition, it’s imperative to maintain consistency throughout, since even a nuanced deviancy could lead to changes further downstream, and introduce problems with the customers.

Image for post
Image for post
Figure 3. As the number of consumers increases, OpenAPI providers need to develop a more comprehensive set of tools and offer support for more programming languages.

To create a simplified experience, an API gateway might be used to solve these problems and provide solutions for signature validation, throttling, and the generation of SDKs and documentation.

But as there were too many API gateways around, let alone the different types, signature algorithms, and serialization formats, the workload for completing these tasks increases exponentially.

Image for post
Image for post
Figure 4. Multiple kinds of OpenAPIs and API gateways are used as the platform of a company grows.

If problems like OpenAPI/SDK inconsistencies, documentation errors, or demo data errors occur, it is because the work is simply too trivial, and not too difficult. Just as we learned from our own experience, many repetitions of even the simplest task will become challenging in the end.

How TeaDSL Works

  • It is used as an intermediate language that supports different kinds of API gateways. It supports the description of different types of OpenAPIs in an API gateway in a uniform fashion.
  • It is translated to generate code in other languages. In other words, it allows the generation of SDKs in different languages with unified intermediate representation.
  • We can consider a collection of OpenAPIs as a library on account of the intermediate representation, and then write code samples on this basis to achieve the unified generation of code samples for multiple languages.

Therefore, the core capability of TeaDSL is to describe OpenAPIs with an intermediate syntax, which is much similar to a programming language, and realize the integration of SDK creation, OpenAPI description, and code sample generation in a variety of programming languages.

Before TeaDSL, we had to develop independent workflows to create SDKs for OpenAPIs in different API gateways, which meant that we had to undergo repetitive process redesigns for different API gateways.

Image for post
Image for post
Figure 5. The workload for providing SDKs in N programming languages for M gateways is M * N.

With TeaDSL acting as an intermediate layer, workloads are converged and made efficient as SDK providers can now focus on the transformations from OpenAPIs in the gateways to TeaDSL and the final generation.

Image for post
Image for post
Figure 6. The workload becomes M + N with the aid of TeaDSL.

As shown in the preceding diagrams, TeaDSL reduced the workload from M * N to M + N, meaning that it could raise the efficiency to a greater degree for a higher volume workload.

TeaDSL can be used to create an OpenAPI architecture from scratch with support for the OpenAPI code sample and test case generation.

The following section describes how TeaDSL handles OpenAPIs and API gateways of different styles and different programming languages.

TeaDSL and API Gateways

However, not all existing OpenAPIs are RESTful, which means that if an OpenAPI is not RESTful, you cannot use Swagger tools to document or generate an SDK for the OpenAPI.

In order to rectify this situation, we have designed a two-step operation:

  • Establish a new set of standards applicable to all OpenAPI styles.
  • Develop a tooling ecosystem based on these standards to expand the functionality.

With these two steps, we would be able to utilize powerful tools for each and every OpenAPI, whether RESTful or not, with no need for migration.

Design of New Standards

protocol: string, // http or https
port: number, // tcp port
host: string, // domain
request: {
method: string, // http method
pathname: string, // path name
query: map[string]string, // query string
headers: map[string]string, // request headers
body: readable // request body
response: {
statusCode: number, // http method
statusMessage: string, // path name
headers: map[string]string, // response headers
body: readable // response body

One can compare OpenAPIs to buildings. Their styles may vary, yet all OpenAPIs have a nearly identical composition. The only things that differentiate one from another are the handlings and permutations of compositional elements. The essential differences between OpenAPI styles are distinct serialization processes. We separated the serialization process from the data model and extracted the data structure.

For example, the following data model is more easily visualized:

model User {
username: string,
age: number

The data to be transmitted to different gateways can be JSON-formatted or XML-formatted, but either way, they are both readable streams of byte data.

toJSON(user: User): string
toXML(user: User): string

The result is as follows:

__request.body = toJSON(user);
__request.body = toXML(user);

Then, we wrapped an OpenAPI request/response pair to a method:

api getUser(username: string): User {
__request.method = 'GET';
__request.pathname = `/users/${username}`;
__request.headers = {
host = 'hostname',
} returns {
var body = readAsJSON(__response.body);
return body;

Although the preceding code cannot be executed, it demonstrates that different gateways and styles will be supported, which is implemented as follows:

  • The request/response protocol, or HTTP, is used as the core model.
  • Several methods, such as toJSON, toXML, and readAsJSON, are introduced to separate the data structure from the serialization process.
  • The process is wrapped into a method.

These methods have different implementations in different programming languages, but we ensure consistency with a uniform signature:

function toXML(data: $Model): string;
function toJSON(data: $Model): string;

The above explanation shows how TeaDSL supports different gateways and OpenAPI styles. In this abstract process, different gateways are handled similarly with methods, only the data structure is presented.

TeaDSL and Programming Languages

  • Easy as it seems, this method results in inflexible and hard-to-maintain code.
  • The generated code is prone to naming conflicts and syntax errors.

As mentioned earlier, we’ve revamped this approach into using a DSL that has lexical, syntax, and semantics rules and runs a series of validations before generating code in the target language. These functions of DSL are not available in templates.

In general, DSLs may not enjoy wide applicability, but some DSLs have been steadily gaining popularity among front-end engineers, such as CoffeeScript, Babel, JSX, and TypeScript. For this reason, we have studied the design of different programming languages and devised our own custom syntax. We also draw on the translation mechanism of compilers and implemented multi-language support with a single model.

The following diagram illustrates the phases of TeaDSL:

Image for post
Image for post

The following are the three major application scenarios of TeaDSL:

  • Create SDKs in multiple languages.
  • Create OpenAPI-related code samples in multiple languages.
  • Create OpenAPI-related test cases in multiple languages.

Use TeaDSL to generate output to different target scenarios through strong validation of the intermediate language to deliver full language support. This saves time and effort for maintainers as they no longer need to spend hours writing code samples for different languages. TeaDSL will be automatically updated with the latest language support features, with no need for additional programming.


Check out the Alibaba Cloud SDKs using TeaDSL. On Alibaba Cloud Open Platform, we are extending TeaDSL through integration with more accessible tools to build a more compatible ecosystem than Swagger.

Original Source:

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store