How Can Teams Perform R&D Online and Remotely?
In order to win this inevitable battle and fight against COVID-19, we must work together and share our experiences around the world. Join us in the fight against the outbreak through the Global MediXchange for Combating COVID-19 (GMCC) program. Apply now at https://covid-19.alibabacloud.com/
By Zhang Liaoyuan (Yunyue)
In response to the national call against the novel coronavirus, various major corporations have closed their doors and told their staff to work from home. This means online collaboration has become a hot topic. In contrast to major corporations, which have sufficient financial resources and capabilities to build remote office systems, small- and medium-sized enterprises (SMEs) lack both the budget and experience to build their own systems. Alibaba Cloud R&D Collaboration (RDC) sees itself as an R&D efficiency engine for digital enterprises. During the current epidemic, we hope to share our experience and tools with SMEs, allowing them to securely and efficiently develop software from home. For this reason, we invited Zhang Liaoyuan, a senior technical expert at Alibaba, to discuss the key aspects of collaborative online development.
High Visibility and Rapid Connectivity Are the Foundation of Online R&D Collaboration
In order to cope with the complexity and uncertainty of Internet businesses, modern software development has gradually adopted a customer-oriented approach. In this approach, small teams or individuals have powerful capabilities while production factors are rapidly connected, enabling the continuous, fast, and high-quality delivery of valuable products. With an ever more specific division of labor, the overall software production process becomes a process of breaking down and solving specific problems and then continuously integrating and releasing solutions.
In this approach to software development, agility and coordination are the keys to success.
Internet technology has made interconnection extremely simple
The development of communication technology has greatly promoted the use of Internet applications. Instant messaging tools have also gone beyond chat functions. DingTalk now supports project groups, video calls, video conferencing, and a variety of office automatic (OA) applications on its platform, helping people quickly connect with each other.
Connectivity is a necessary condition of collaboration. The connectivity between people and things can help companies implement online organization, communication, and collaboration.
The development of production tools has significantly improved the visibility of the software production process
Online R&D processes have taken software engineering visibility to new heights. The advance of production tools has made the production process completely transparent, with implicit work gradually becoming explicit. Collaboration relies on information sharing and the visibility of the production process. In the well-known Scrum development framework, transparency is listed as the first of its three pillars.
Note: Scrum’s three pillars are transparency, inspection, and adaptation.
The evolution of software architectures and deployment methods has been conducive to the division of labor.
According to Conway’s law, system architectures mirror the communication structures of the organizations that design them. Then, in turn, the system architecture counteracts attempts to integrate the communication structure or software. In the context of cloud development, middleware, and midend strategies, the business layer primarily focuses on business innovation, and the division of labor becomes increasingly specific.
The evolution of deployment architectures has allowed some parts of systems to be independently deployed. As a result, the value of a small team or individual is increasing, and the links between teams and people are gradually evolving from the traditional tree structure to a mesh-like structure. Collaboration occurs in just such a mesh environment. In this environment, we can clearly identify production requirements to be incorporated and then quickly collaborate online.
For knowledge-based economic activities, high visibility and fast connectivity equal flexible collaboration, and online capabilities are the underlying foundation. Online work allows people to come together for collaboration anytime, anywhere. As a result, telecommuting is now open to small offices and home offices (SOHOs).
The ability to work online is the foundation of digital collaboration and the prerequisite to the further evolution of intelligent solutions. This is the trend that defines modern software R&D methods.
Now, let’s discuss how to achieve online collaboration across regions and time zones by looking at three scenarios: R&D collaboration, code collaboration, and release collaboration. In this way, companies can quickly connect their staff to truly enter the era of digital R&D.
Online Project Collaboration
Requirement collaboration starts with the mapping and visualization of end-to-end value flows.
Visibility is the foundation of cooperation. On the electronic dashboard, Alibaba Cloud turns requirements into flow elements to visualize the end-to-end value stream. We view flow efficiency as the core around which requirement delivery is organized. Any visualization of the end-to-end value flow must meet the following requirements. 1) Value-driven: Each flow element reflects a business requirement, which signifies real value for users. 2) Front-to-back: The goal of visualization is the “end-to-end” value flow. It starts from the user’s problem and ends with the resolution of this problem.
By using the following criteria, we can test visualization performance:
- Does it reflect the end-to-end delivery process?
- Can it immediately reflect bottlenecks and problems that affect the value flow?
- Can you collaborate and make decisions based on this visual information?
At the same time, the entire process from project collaboration to software release can be interconnected. Code submission and release information can also be instantly reflected on the requirement card, and work statuses can be promptly synchronized. By reducing unnecessary communication, project management can focus on value delivery and problem-solving.
Manage the Value Flow and Build a Value Feedback Loop for Better Delivery Control
The key to software delivery is the flow of customer value, while organizational barriers, communication delays, and collaborative congestion are the main obstacles. The end-to-end visual value stream dashboard can be used to direct product demand scheduling and the daily task assignment to teams in order to organize routine project collaboration. From right to left, the review process of requirement delivery statuses runs from test engineers and development engineers to product managers and cross-functional collaboration. At the same time, any risks and problems relevant to the requirements are highlighted and displayed on the dashboard. This way, they are quickly fed back to the responsible people through DingTalk or other instant communication tools. With this instant discovery and response mechanism, problems are quickly gathered together.
This creates a complete value feedback loop from demand planning, demand scheduling, and daily stand-up meetings to demand review. The arrangement of monthly planning, weekly scheduling, and daily stand-up meetings is determined by the overall pace of delivery. All of this can be done through the online electronic dashboard.
Limit Online Products and Accelerate the Delivery of Business Solutions
The key influences on requirement (value) flow efficiency are batch and parallel processing. We can accelerate requirement delivery by limiting the number of ongoing products. At the same time, digital collaboration is conducive to the accumulation of performance data during the R&D process. By setting baselines and goals for efficiency improvement, with the customer response cycle and quality improvement used as specific metrics, we can drive the discovery and resolution of problems and establish a foundation for continuous improvement.
Online Code Collaboration
Code Collaboration Means Collective Code Ownership
In this context, team members are jointly responsible for the code. Basic code hosting and management capabilities are implemented by using the distributed version control system, Git. In the current approach to code collaboration, branches are the vehicles of collaboration. Developers around the world can set up different branches based on product requirements for simultaneous development. In an appropriate branch mode, scattered work can be quickly integrated and traced by version. The readme file for each code library describes the code design in detail. It establishes basic code quality management standards, such as unit testing and automated static scanning of incremental code, to ensure that existing functions are not affected during continuous code incrementation. This makes collaboration possible.
Enhance Code Visibility to Facilitate Code Collaboration
Tools such as the Alibaba Coding Guidelines have greatly improved the visibility of code, allowing programmers to extract key information from large amounts of code. With these tools, programmers can check the complexity and analyze repetitions and dependencies of their code, or identify the domain language and sensitive information.
In addition, Cloud IDE and distributed code hosting tools in the cloud promote social programming, whether it is pair programming or code review. This makes the code itself and the coding process explicit and provides the following benefits:
- It promotes the sharing of knowledge within the team, improves the overall capabilities of the team, ensures that the team is working in a unified and standardized manner, and prevents unreadable code.
- At the same time, the early use of tools increases the discovery rate of potential defects by 30%.
- Transparent code and group discussions provide encouragement and foster active thinking and the pursuit of excellence.
In online code collaboration, code sharing and the cultivation of the reuse culture depend on the complex permission control system of a code platform. This is a double-edged sword. On the one hand, the more open the platform, the greater the risk of code leakage. On the other hand, the more confined the platform, the lower the collaboration efficiency. To ensure that developers fully benefit from code collaboration, note the following issues:
Sensitive information in the code: For example, the code may contain database passwords. If these passwords are intentionally or unintentionally disclosed, the company’s business can be adversely affected.
Access control: This often includes access control based on IP addresses, revoking the permissions of former employees, and setting the visible range of your code library.
Identification of unexpected behavior: This involves post-hoc audits and ex-ante warnings, such as the detection and warning of suspicious behavior when a user downloads a large amount of code.
Code collaboration encourages the pursuit of technical excellence and cultivates a spirit of software craftsmanship. With the help of advanced code collaboration tools and technical means, it promotes the visibility of the code and the coding process. At the same time, it allows every programmer to submit every line of code with confidence.
Online Release Collaboration
The delivery of modern enterprise-level software often involves collaboration between different roles and interaction across multiple systems. To ensure reliable delivery, the delivery process is often complex in design and is difficult to fully standardize. For Alibaba, we have benefited from a complete set of DevOps methodologies and products that have been developed over many years. As a result, we can ensure an online, efficient, and reliable software delivery process. This is how Alibaba quickly develops high-quality software to keep up with the changing market.
Enabling Online Release Collaboration through Process Standards and Procedural Improvements
Online release collaboration must be based on a single delivery pipeline. First, we need to map out a complete software integration and delivery process. To do this, we must build a complete system that covers the procedure from making changes to delivery, and use tools to connect the integration and delivery processes. We must also specify the input and output standards of each stage in the resulting process, with downstream activities based on upstream output quality. Second, we need to establish a quality assurance system that is suited to the process and automate each procedure. Third, we must establish a feedback mechanism to facilitate the precise location of problems, instant response, and rapid correction. By establishing a corresponding measurement feedback mechanism, we can continuously optimize the process and procedures.
True online release collaboration must meet the following requirements: instrumentalized process, automated deployment, automated testing, and digital feedback.
Feature Branches Drive Online Collaboration among People and Roles
To allow multiple people to seamlessly collaborate in R&D and code functions to be freely and reliably combined and delivered, Alibaba has created a set of AoneFlow code branch management methods based on its experience. These methods transform feature changes to units and use the CI/CD pipeline to cover the entire continuous delivery process from construction and testing to deployment. In the process, messages, emails, and DingTalk chatbots are used to notify and trace information about each step and task. This allows roles such as R&D personnel, tester, O&M personnel, and configuration administrator to collaborate online and work asynchronously.
Break the Boundary between R&D and O&M with Cloud Native Technology
Many cloud native technologies, such as Kubernetes, serverless, service mesh, and cloud IDEs, have achieved impressive success in the past year. This set of out-of-the-box open-source software allows SMEs to quickly acquire refined software delivery capabilities, such as complex traffic governance, phased release, A/B testing, multi-environment management, one-click infrastructure startup, and rapid scaling. However, enterprises face many challenges when adopting new technologies, such as the complex construction of open-source software, black-screen interaction design, lack of R&D management methods, and lack of enterprise permission management capabilities. Alibaba is also actively integrating CI/CD tools, test environment management methods, application O&M concepts, DevOps collaboration methodologies, and other technologies into cloud native technologies to provide developers with new technology solutions ready to go out-of-the-box.
RDC Empowers One-stop Online R&D Collaboration
To do a good job, you must first find the right tools. RDC provides end-to-end online collaboration services and development tools that cover the entire lifecycle from requirement collection to product development, testing, release, O&M, and operations. This provides an easy and complete solution for R&D collaboration.
With R&D Collaboration, a typical day at the office might go like this:
- During the morning meeting, the team aligns requirements and tasks and assigns tasks based on the lean dashboard.
- Developers write code based on features and create change branches.
- Developers complete programming in an offline or cloud development environment and then submit the code.
- Submitting the code automatically triggers a code scan, and the scanned code is sent to the designated code reviewer for review.
- The code that has been reviewed automatically triggers the integration and release pipeline, which automatically completes building and generates Docker images that are deployed in the development environment, integration environment, and staging environment respectively.
- After the corresponding verification work is done, the new version enters the pending release status, which triggers the online review process. After the O&M personnel completes the review, the version is released.
- DingTalk is used to report any problems during the process, following the no news is good news principle. Problems are promptly reported to the appropriate person to allow for accurate feedback, immediate response, and quick correction.
- The system works to avoid unnecessary SMS communication as excessive noise will reduce the efficiency of collaboration.
Through RDC, we want to share the practical R&D experience and advanced tools accumulated over the years so that SMEs can quickly implement software R&D while their personnel work from home.
About R&D Collaboration: RDC, an enterprise-level one-stop DevOps solution, derived from Alibaba’s advanced management concepts and engineering practices. This product seeks to become the R&D efficiency engine for digital enterprises. RDC provides end-to-end online collaboration services and R&D tools covering the entire lifecycle from requirement collection to product development, testing, release, O&M, and operations. Through the application of artificial intelligence and cloud native technologies, developers can improve their R&D efficiency and rapidly and continuously deliver valuable products to customers.
While continuing to wage war against the worldwide outbreak, Alibaba Cloud will play its part and will do all it can to help others in their battles with the coronavirus. Learn how we can support your business continuity at https://www.alibabacloud.com/campaign/fight-coronavirus-covid-19