Video AI: Next-Generation Intelligent Video Production

Intelligent Production and Editing

  1. Intelligent capture: On a capture device, if the media asset platform requires user-generated content (UGC) and professionally-generated content (PGC), Alibaba Cloud’s short video software development kit (SDK) can provide product-level features for video recording and non-linear editing, including eye enlarging, face slimming and other advanced beautify effects, as well as animated stickers and special effects based on facial recognition and face image tracking techniques. This helps you make more diversified and creative videos.
  2. Intelligent editing: The video resources that are captured can be edited in the cloud or on a device. On a device, Alibaba Cloud’s short video SDK provides custom materials and supports multiple types of montage, filters, transitions, and music special effects. On the cloud, cloud-based editing and multimodal content interpretation technologies are used to automatically re-produce quality content. Compared to traditional video production, intelligent production reduces the processing duration to seconds and reduces the investment into devices significantly.
  3. Real-time subtitles: Different from traditional manual conversion and translation, intelligent production can perform automatic voice-to-text conversion through automatic speech recognition (ASR), store the text at the corresponding timeline, and then automatically translate the text from the original language to the language required. The amount of human intervention is greatly reduced. This technology is applicable to not only offline videos, but also live conference videos for real-time subtitles production.

Intelligent Media Asset Management

  1. Content review: Illegal content may result in serious threats for enterprise operation. Intelligent review can be triggered through workflows or called APIs. It can help identify pornographic, reactionary, violent, terrorist, and politically sensitive content to control and handle risks. A media blacklist is set up based on the illegal content identified to further improve review efficiency.
  2. Intelligent thumbnail extraction: This feature automatically selects the best keyframe or clip to be the video thumbnail to better illustrate the core content and increase views.
  3. Intelligent catalog: Traditional intensive catalog takes about 2 to 4 hours for a 1-hour video. In the age of Internet and content explosion, intelligent catalog can apply technologies, such as video auto-classification, flagging, character recognition, and speech recognition, to generate video information, add videos to the media asset database, and make intelligent recommendations based on scenarios of natural language processing (NLP) and part-of-speech filtering. The whole process is driven by algorithms with no need of human labor.
  4. Intelligent cloud-based broadcasting: The cloud caster integrates the multimodal content interpretation technology to automatically add character information to the video and produce video highlights in a real-time and precise manner. This solution helps lower the threshold and costs of professional broadcasting instruction equipment, advanced editing software, and professional personnel. With no need for human labor, this solution can reduce the cost by hundreds of times and realize collaboration across multiple locations.

Intelligent Distribution and Consumption

In terms of distribution and consumption, the intelligent media search engine developed based on video DNA (one for each medium) precisely delivers the highest quality content to users. The fingerprint index to video DNA features stability regardless of any change to file format, editing, compression, or rotation. It also helps to effectively identify user-generated videos and replicated videos to avoid redundancy in content searching and improve user experience. In addition, video DNA is also applicable to the protection of video copyright.

Use Case 1: World Cup Highlights

During this year’s World Cup which was live-streamed by Youku, Alibaba Cloud’s intelligent video production solution was applied to instantly generate match highlights within a minimum of ten seconds, improving the production efficiency by nearly 10 times. 20% of the World Cup short videos were produced by AI.

Use Case 2: Intelligent Sports Meeting

An intelligent sports meeting was held at this year’s Computing Conference. The intelligent video production solution was also used to generate intelligent highlights of the surfing and 3x3 basketball games.

Building a Global Integrated Media Asset Production and Management Platform

More and more enterprises are engaged in global businesses, when the Internet sees lowering limit of time and space. Based on an infrastructure with global coverage, Alibaba Cloud’s next-generation media asset service supports multi-region high-speed data synchronization, and enables collaborative production, management, and control of multiple centers in and outside China anytime, anywhere. In addition, Alibaba Cloud CDN has over 1,500 nodes across the globe, allowing customers to fast distribute media content to more than 70 countries in six continents.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website: