A Deep Dive Into DinamicX: Enabling Visually Impaired People to Shop Online

By Dabao

At present, there are more than 17 million visually impaired people in China. Despite the high penetration of the Internet today, they are unable to enjoy the convenience brought by the Internet like most people. For example, they are unable to shop online using a mobile phone. Technology can improve this situation and make their lives just as convenient as someone that is not visually impaired. This article shares technical solutions developed by the Taobao Mobile Team using DinamicX to support accessibility and illustrates the solutions with examples. We hope that this article can spark some inspiration among mobile-side developers.

What is Accessibility?

These facilities are intended to provide disabled people with the same opportunities as non-disabled people. Disability (as discussed here) can vary according to the form and severity and can be divided into four categories: impairments of cognition, vision, hearing, and mobility.

Disability can also be divided into two categories: permanent disability and situational disability.

Permanent Disability

Situational Disability

  • Receiving an Incoming Voice Message on a Chat App During a Meeting: The sound of playing the voice message will disturb the surrounding colleagues. The voice-to-text feature provided by chat apps is one of the accessibility services.
  • Encountering Language Barriers While Traveling Abroad: This is a situational verbal communication barrier.
  • Carrying Many Bags While Shopping: This is a situational physical limitation.

In terms of IT services, we give accessibility a new definition that assures every user’s desire to be understood.

To some extent, information accessibility is a feature that is oriented to special people in the interactive design of intelligent products. This feature allows people to enjoy various features of products in hardware and software more equally.

DinamicX’s Accessibility Support

Core Sections of Taobao Mobile

As an important part of the support for the core sections of Taobao Mobile, the DinamicX SDK is responsible for accessibility.

What is DinamicX?

The core technology of the dynamic template solution is an engine that includes downloading, loading, parsing, and rendering, which helps you generate views dynamically.

DinamicX’s support for accessibility is divided into two parts:

  • DinamicX SDK’s cross-platform support for accessibility
  • Bayonet-based verification on the template development platform

To develop a cross-platform unified and dynamic solution, we must smooth the differences between Android and iOS and reduce the business side’s (template developers’) cognitive cost to support accessibility. After a lot of discussions, all of the members of our team, including those engaged in Android development, iOS development, and testing, outlined unified accessibility behaviors while ensuring unification on Android and iOS.

Technical Solutions

System Native Accessibility

iOS Native Accessibility

  • When isAccessibilityElement is set to YES for a view, no matter whether accesibilityLabel is set, all its child views cannot take focus.
  • The value of the isAccessibilityElement property of UILabel defaults to NO. If this value is ever actively set by code or set to NO, the UILabel text out cannot be automatically read out in the parent container.
  • If you want the parent container to automatically read all of the UILabel text after this container takes focus, you must set isAccessibilityElement to NO and accessibilityElementsHidden to NO. The isAccessibilityElement property of labels must retain the original default value and cannot be set to any other values.
  • If a parent view is nested and the accessibilityElement property of all parent views is set to off, the accessibilityLabel property values of TextViews of all of the child views under this parent view will be automatically read out in sequence. All of the automatically read text will eventually be read out on the root view.

The following table includes the system API operations:

Android Native Accessibility

  • Without Accessibility Information: For example, by default, ImageView and View have no accessibility information.
  • With Accessibility Information: For example, ImageView is set with setContentDescription, or the accessibility information of TextView is its own text.
  • With the Interactive View That Has Accessibility Information: For example, ImageView is set with both setContentDescription and setOnClickListener, TextView is set with setFocusable (true) or EditText, and Checkbox is by default the interactive views that have accessibility information.

The relations among these three accessibility states in its parent layout are described below:

The following table includes the system API operations:

DinamicX SDK-Defined Accessibility Properties

XML Example

<ImageView
width="100"
height="100"
accessibility="on"
accessibilityText="Jump to Details"
onTap="@openUrl{'detail'}"
imageUrl="https://img.alicdn.com/tfs/TB1FuMQQFXXXXXLXXXXXXXXXXXX-420-420.jpg"
/>

Unified Accessibility Behaviors on Android and iOS

Processing on Android and iOS

iOS

Android

Accessibility handling by layout views:

Accessibility handling by non-layout views:

Example

Sample Template

<LinearLayout
backgroundColor="#eeeeee"
height="match_content"
width="375"
orientation="vertical"
disableFlatten="true"
>
<LinearLayout
marginLeft="@triple{@data{cellType},20,50}"
backgroundColor="#f2f2f2"
height="match_content"
width="match_parent"
orientation="vertical"
disableFlatten="true"
accessibility="auto"
>
<!--When auto represents click, the text information under the layout can be read out-->
<TextView
width="match_content"
height="match_content"
textColor="#ff051b28"
textSize="12"
marginTop="20"
marginBottom="20"
text="This is a textView"
/>
<TextView
width="match_content"
height="match_content"
textColor="#ff051b28"
textSize="12"
marginTop="20"
marginBottom="20"
text="This is a textView with focus"
accessibility="on"
onTap="@rTap{}"
accessibilityText="This is a textView with focus"
/>
<FastTextView
width="match_content"
height="match_content"
textColor="#ff051b28"
textSize="12"
marginTop="20"
marginBottom="20"
text="This is a FastTextView"
/>
<TextView
width="match_content"
height="match_content"
textColor="#ff051b28"
textSize="12"
marginTop="20"
marginBottom="20"
accessibility="off"
text="This is a textView that does not need to be read out"
/>
<ImageView
width="100"
height="100"
marginLeft="20"
marginTop="12"
borderWidth="3ap"
borderColor="#FF0000"
accessibility="on"
accessibilityText="This is an ImageView click"
onTap="@rTap{'Test'}"
imageUrl="https://img.alicdn.com/tfs/TB1FuMQQFXXXXXLXXXXXXXXXXXX-420-420.jpg"
/>
</LinearLayout>
</LinearLayout>

Demonstration of Sample Template on a Mobile Phone

Accessibility Verification Bayonet

Many dynamic solutions, including native itself, support accessibility features, but this kind of support is one-way. If only the product supports accessibility, but developers do not support it, the product still lacks accessibility features.

Why don’t developers support accessibility features?

  • First, the publicity for accessibility is insufficient, and the priority of accessibility is not high in development. Developers lack such awareness, which results in insufficient accessibility test cases.
  • Second, supporting accessibility features incurs certain costs. Moreover, a set of standards and specifications that can tell developers under what circumstances and how to support accessibility are still absent.
  • Third, supervision and control over the process are absent, so developers may forget to include accessibility features.

To better enable visually impaired users to use Taobao Mobile, help the business side locate errors in accessibility features, and reduce the workload of accessibility regression tests, we have launched the accessibility verification bayonet. It intelligently detects accessibility problems and determines whether a template is qualified by calling the accessibility service. As such, it ensures that the publishing of each template supports accessibility.

Adding the accessibility verification bayonet is the most critical part of accessibility work. Currently, the core sections of Taobao Mobile are based on DinamicX, and DinamicX templates are all developed on the component platform. Therefore, when we add this bayonet, developers will have to support accessibility. Otherwise, they cannot publish dynamic templates.

There is a flow chart of DinamicX development pattern with the accessibility verification bayonet below:

The current rules for verification using this bayonet are listed below. Some rules are added to avoid differences between Android and iOS:

  • For non-interactive views, such as ImageView, FrameLayout, and LinearLayout, if the onTap property is set, the bayonet checks whether a view contains accessibility properties. If it does not contain these properties, the verification fails, and suggestions are given: Set accessibility to on, enable accessibility focus, and set accessibilityText to xx.
  • For non-interactive views, such as ImageView, FrameLayout, and LinearLayout, if accessibility is set to on, accessibilityText must be set to xx at the same time.
  • When the onTap property is set for a child view, accessibility must not be set to on for its parent layout. Otherwise, this child view cannot take focus.
  • If auto is set for a layout, you must set onTap and also set accessibility to on for the TextView. Otherwise, the TextView cannot take focus.
  • The accessibility property cannot be set to a dynamic expression.

Currently, all templates developed on the core pages of Taobao Mobile, including the Homepage, Details, Shopping Cart, My Taobao, Order Details, and Order List, must pass the bayonet-based verification.

Vision

When developers develop templates, I hope they will no longer need the verification bayonet pop-up to remind them of accessibility features. I hope they will think about visually impaired people from the moment they start writing a template. I hope Taobao Mobile will be the best shopping tool for the visually impaired.

The readers of this article will see the results of their efforts to work with accessibility in mind. One day in the future, you may speak with a visually impaired user of Taobao Mobile and be touched to learn about the impact you have had.

I hope that the readers of this article will think about accessibility design as public welfare. Completing an accessibility feature should inspire you. Technology can be a tool for creating value and a better world.

Caring about accessibility is caring about public welfare. The change starts with you!

Visually impaired people can shop with their hearing using voice-over narration features on mobile phones.

An engineer from the Taobao Accessibility Laboratory is conducting an accessibility test with his eyes closed.

The views expressed herein are for reference only and don’t necessarily represent the official views of Alibaba Cloud.

Original Source:

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store