A Deep Dive Into DinamicX: Enabling Visually Impaired People to Shop Online
At present, there are more than 17 million visually impaired people in China. Despite the high penetration of the Internet today, they are unable to enjoy the convenience brought by the Internet like most people. For example, they are unable to shop online using a mobile phone. Technology can improve this situation and make their lives just as convenient as someone that is not visually impaired. This article shares technical solutions developed by the Taobao Mobile Team using DinamicX to support accessibility and illustrates the solutions with examples. We hope that this article can spark some inspiration among mobile-side developers.
What is Accessibility?
Accessibility covers a broad scope. It generally means that no obstacle exists in the development process and that activities can go on smoothly. In daily life, there are many accessibility designs. For example, in some public places, such as railway stations, airports, and shopping malls, barrier-free elevators and barrier-free toilets are set up for people with leg disabilities, or hearing aids are provided for people with hearing impairments.
These facilities are intended to provide disabled people with the same opportunities as non-disabled people. Disability (as discussed here) can vary according to the form and severity and can be divided into four categories: impairments of cognition, vision, hearing, and mobility.
Disability can also be divided into two categories: permanent disability and situational disability.
Permanent disability includes visual impairment, hearing impairment, and mobility impairments, such as wheelchair dependence.
- Trying to Use a Mobile Phone While Driving: Car shaking causes situational visual impairment, physical impairment, and attention deficit.
- Receiving an Incoming Voice Message on a Chat App During a Meeting: The sound of playing the voice message will disturb the surrounding colleagues. The voice-to-text feature provided by chat apps is one of the accessibility services.
- Encountering Language Barriers While Traveling Abroad: This is a situational verbal communication barrier.
- Carrying Many Bags While Shopping: This is a situational physical limitation.
In terms of IT services, we give accessibility a new definition that assures every user’s desire to be understood.
To some extent, information accessibility is a feature that is oriented to special people in the interactive design of intelligent products. This feature allows people to enjoy various features of products in hardware and software more equally.
DinamicX’s Accessibility Support
In the rest of the article, DinamicX’s support for accessibility refers to the support for barrier-free access to information, which is aimed at people with visual impairments. Through technical means, visually impaired people can smoothly obtain and use information when they use apps.
Core Sections of Taobao Mobile
The Homepage, Details, Shopping Cart, Buy, Order, Order List, and My Taobao pages are the core of Taobao Mobile. Currently, the user interfaces (UIs) of these pages are drawn with DinamicX as the rendering engine.
As an important part of the support for the core sections of Taobao Mobile, the DinamicX SDK is responsible for accessibility.
What is DinamicX?
DinamicX is positioned to be a client-side dynamic solution that features unified capabilities for Android, iOS, and H52. It provides basic guarantees for high performance and high availability on the Taobao Mobile app. Through community-based operations, we hope to continuously enrich DinamicX’s capabilities and content and improve its rendering performance and stability, turning DinamicX into a standard solution for client-side dynamic systems within Alibaba.
The core technology of the dynamic template solution is an engine that includes downloading, loading, parsing, and rendering, which helps you generate views dynamically.
DinamicX’s support for accessibility is divided into two parts:
- DinamicX SDK’s cross-platform support for accessibility
- Bayonet-based verification on the template development platform
To develop a cross-platform unified and dynamic solution, we must smooth the differences between Android and iOS and reduce the business side’s (template developers’) cognitive cost to support accessibility. After a lot of discussions, all of the members of our team, including those engaged in Android development, iOS development, and testing, outlined unified accessibility behaviors while ensuring unification on Android and iOS.
System Native Accessibility
iOS Native Accessibility
There are several types of iOS native logic:
isAccessibilityElementis set to YES for a view, no matter whether
accesibilityLabelis set, all its child views cannot take focus.
- The value of the
isAccessibilityElementproperty of UILabel defaults to NO. If this value is ever actively set by code or set to NO, the UILabel text out cannot be automatically read out in the parent container.
- If you want the parent container to automatically read all of the UILabel text after this container takes focus, you must set
isAccessibilityElementto NO and
accessibilityElementsHiddento NO. The
isAccessibilityElementproperty of labels must retain the original default value and cannot be set to any other values.
- If a parent view is nested and the
accessibilityElementproperty of all parent views is set to off, the
accessibilityLabelproperty values of TextViews of all of the child views under this parent view will be automatically read out in sequence. All of the automatically read text will eventually be read out on the root view.
The following table includes the system API operations:
Android Native Accessibility
Android’s view accessibility states include three types:
- Without Accessibility Information: For example, by default, ImageView and View have no accessibility information.
- With Accessibility Information: For example, ImageView is set with
setContentDescription, or the accessibility information of TextView is its own text.
- With the Interactive View That Has Accessibility Information: For example, ImageView is set with both
setOnClickListener, TextView is set with
setFocusable (true)or EditText, and Checkbox is by default the interactive views that have accessibility information.
The relations among these three accessibility states in its parent layout are described below:
The following table includes the system API operations:
DinamicX SDK-Defined Accessibility Properties
To smooth the differences between Android and iOS and simplify the accessibility logic, DinamicX provides two accessibility properties for accessibility features.
As shown in the following example, the view is selected when it is touched, and the “Jump to Details” text is read out:
accessibilityText="Jump to Details"
Unified Accessibility Behaviors on Android and iOS
The following figure shows the current unified behaviors on Android and iOS and describes the status of a layout and a child view based on different accessibility property values.
Processing on Android and iOS
To achieve the unified behaviors on Android and iOS shown in the preceding figure, the respective operations have been performed.
The following figure shows the mapping from the SDK to system API operations based on template properties:
The following figure shows the mapping from the SDK to system API operations based on template properties. In Android, layout and non-layout views are treated differently.
Accessibility handling by layout views:
Accessibility handling by non-layout views:
<!--When auto represents click, the text information under the layout can be read out-->
text="This is a textView"
text="This is a textView with focus"
accessibilityText="This is a textView with focus"
text="This is a FastTextView"
text="This is a textView that does not need to be read out"
accessibilityText="This is an ImageView click"
Demonstration of Sample Template on a Mobile Phone
Look at the following video. The layout accessibility is set as auto, so this layout will be selected, and the text that contains accessibility information will be read out, but the second and fourth TextViews will not be read out. onTap and accessibility are set to on for the second TextView, so it is an interactive view in this case and must be selected separately. For the fourth TextView, accessibility is set to off. As such, it is disabled with the accessibility feature, and it is not selectable and cannot be read out. Moreover, it will not be selected and read out by the layout.
Accessibility Verification Bayonet
Accessibility support is one thing, whereas how to guide developers to write code is another thing.
Many dynamic solutions, including native itself, support accessibility features, but this kind of support is one-way. If only the product supports accessibility, but developers do not support it, the product still lacks accessibility features.
Why don’t developers support accessibility features?
- First, the publicity for accessibility is insufficient, and the priority of accessibility is not high in development. Developers lack such awareness, which results in insufficient accessibility test cases.
- Second, supporting accessibility features incurs certain costs. Moreover, a set of standards and specifications that can tell developers under what circumstances and how to support accessibility are still absent.
- Third, supervision and control over the process are absent, so developers may forget to include accessibility features.
To better enable visually impaired users to use Taobao Mobile, help the business side locate errors in accessibility features, and reduce the workload of accessibility regression tests, we have launched the accessibility verification bayonet. It intelligently detects accessibility problems and determines whether a template is qualified by calling the accessibility service. As such, it ensures that the publishing of each template supports accessibility.
Adding the accessibility verification bayonet is the most critical part of accessibility work. Currently, the core sections of Taobao Mobile are based on DinamicX, and DinamicX templates are all developed on the component platform. Therefore, when we add this bayonet, developers will have to support accessibility. Otherwise, they cannot publish dynamic templates.
There is a flow chart of DinamicX development pattern with the accessibility verification bayonet below:
The current rules for verification using this bayonet are listed below. Some rules are added to avoid differences between Android and iOS:
- For non-interactive views, such as ImageView, FrameLayout, and LinearLayout, if the onTap property is set, the bayonet checks whether a view contains accessibility properties. If it does not contain these properties, the verification fails, and suggestions are given: Set accessibility to on, enable accessibility focus, and set
- For non-interactive views, such as ImageView, FrameLayout, and LinearLayout, if accessibility is set to on,
accessibilityTextmust be set to xx at the same time.
- When the onTap property is set for a child view, accessibility must not be set to on for its parent layout. Otherwise, this child view cannot take focus.
- If auto is set for a layout, you must set onTap and also set accessibility to on for the TextView. Otherwise, the TextView cannot take focus.
- The accessibility property cannot be set to a dynamic expression.
Currently, all templates developed on the core pages of Taobao Mobile, including the Homepage, Details, Shopping Cart, My Taobao, Order Details, and Order List, must pass the bayonet-based verification.
We will continue to do our best to make Taobao Mobile a product that is inclusive for all our customers, especially visually impaired people. We hope that Taobao Mobile can engage with them and become more than a shopping app.
When developers develop templates, I hope they will no longer need the verification bayonet pop-up to remind them of accessibility features. I hope they will think about visually impaired people from the moment they start writing a template. I hope Taobao Mobile will be the best shopping tool for the visually impaired.
The readers of this article will see the results of their efforts to work with accessibility in mind. One day in the future, you may speak with a visually impaired user of Taobao Mobile and be touched to learn about the impact you have had.
I hope that the readers of this article will think about accessibility design as public welfare. Completing an accessibility feature should inspire you. Technology can be a tool for creating value and a better world.
Caring about accessibility is caring about public welfare. The change starts with you!
Visually impaired people can shop with their hearing using voice-over narration features on mobile phones.
An engineer from the Taobao Accessibility Laboratory is conducting an accessibility test with his eyes closed.
The views expressed herein are for reference only and don’t necessarily represent the official views of Alibaba Cloud.