Robust roles on Android

I’ve heard some opinions over the years that accessibility on Android is difficult to work with and sometimes “hacky” to do certain things, but I’ve come to enjoy how the accessibility framework is put together, and I hope to be able to share some of that with you and demystify some of the inner workings. There is a lot that goes into the accessibility framework in Android, so for now, we will just focus on 4.1.2 Name, Role, Value, but primarily just “Role”.

TalkBack is not a standard

TalkBack is a powerful tool for testing accessibility on Android devices, but keep in mind that it is not the actual standard. For example, if you are using TalkBack and you hear it announce “Submit button double tap to activate” — that may sound correct, but the role of button might not be applied correctly — the role may be applied as part of the accessible name.

Let’s assume that the button has a name of “Submit button” with no actual role applied. Now that might be mostly a non-issue for TalkBack users because “users know what they know/want” (this is a common misunderstanding in user experience design). TalkBack users hear “Submit button” so they know it is a button, but what about non-speech output users like Braille display users for example?

Braille display users would also get “Submit button”, but that’s not actually what Braille users are expecting. They are expecting “Submit btn” with “button” abbreviated. This because space is a premium on Braille displays. Braille displays are limited to how many characters they can display at once before the user has to “pan” to read more content.

The larger the Braille display, the more characters can be displayed at once, but a larger Braille display is significantly more expensive than a smaller one. The abbreviated “btn” for Braille versus the full “button” text is a 50% space savings for Braille users. This is one reason why it is important to apply roles as they are intended. Also keep in mind, that depending on the user’s settings, Braille displays will also have a status cell followed by an empty cell before the actual Braille output begins. So in reality, most users are limited to two less Braille cells for their actual output.

The Braille output below compares a control called “Submit button” with the role added as part of the name, versus a control called “Submit” with the role added in the way that is expected by the accessibility service, which allows the Braille service to abbreviate the role as “btn”. The Braille output is using the Liblouis, US 8 dot in English Braille table which is one of many different types of Braille output.

It is up to the developer to apply roles in the way that that the accessibility framework is expecting it so that any accessibility service can do what it needs to do with the content and produce the expected results for the user. This is the “robust” portion of “Perceivable, Operable, Understandable, and Robust” (POUR). If the role is not applied in a manner that can be programmatically determined, then it is a failure of 4.1.2 Name, Role, Value.

Double tap to activate

There are some opinions that TalkBack announcing “Double tap to activate” is enough information for the user to know that this is a button or something actionable. This is something that TalkBack does, which can be helpful. But TalkBack isn’t the standard; it is a tool we can use to help determine if accessibility is properly implemented. This announcement is considered a usage hint and can be easily disabled in the TalkBack settings. If a user has this disabled because they are familiar with how to interact with TalkBack, they will not know if a control can be activated unless it has a role like “button”.

What causes TalkBack to announce “Double tap to activate”? This is just a hint for a view that has the OnClickListener attached to it. This is the same scenario as a screen reader like NVDA announcing “clickable” when navigating a web page. Something can be clickable but doesn’t actually do anything for the user. For example, a View may have the OnClickListener attached to it just to collect some analytics on where a user taps on the screen. It doesn’t have to actually do anything for the user in order to be announced with “Double tap to activate”.

Consider the code below where I am creating a generic View. This View has an OnClickListener set on it, but I decide to not do anything with it, just leave the listener empty.

TextView emptyClickView = findViewById(R.id.emptyClick);
emptyClickView.setContentDescription("I am not a button");

emptyClickView.setOnClickListener(new View.OnClickListener() {

    @Override
    public void onClick(View v) {
        // do nothing
    }

});

As the image above shows, the onClick doesn’t do anything, but TalkBack will announce this as “I am not a button. Double tap to activate” because there is an OnClickListener provided. Users should not have to rely on the “Double tap to activate” announcement to provide any concrete information on the role.

You will hear something similar for a view when the OnLongClickListener is attached to it in the form of “Double tap and hold to long press”. This is just letting the user know that they can long press to perform an action.

As I mentioned before, this is something that TalkBack does to help users. This is not an accessibility standard. Anyone can make screen reader for Android, and what the screen reader (or any accessibility service) should be looking for is a role like “button” to tell the user what to expect when they click this.

Methods of providing roles

Native controls

Native controls automatically have roles. If you use a Button or any View that extends from android.widget.Button, then it will automatically have the role of “button”. This is similar to HTML in that if you keep it simple and use the built-in widgets, then it just works. But sometimes that isn’t an option, or it’s more work to customize a Button widget to look how you need it to, or you need the Button widget to contain a child view (which it isn’t designed to handle). This is where we can get creative.

View.getAccessibilityClassName()

The first thing that you can do is to create a custom class and extend from android.view.View. This is the base class of all of the standard widgets. Within that class, there’s a method that is meant to be overridden called getAccessibilityClassName. This is where the operating system first looks for a role when building the accessibility tree (you can override this, which we’ll go over later). We’ll extend from LinearLayout since I want this to be a vertical stack container.

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:orientation="vertical">
    <TextView
        android:id="@+id/textView2"
        android:layout_width="match_parent"
        android:layout_height="22dp"
        android:text="This is a custom button" />

    <TextView
        android:id="@+id/textView"
        android:layout_width="match_parent"
        android:layout_height="22dp"
        android:text="With multiple children" />
</LinearLayout>

    <com.tpgi.myapplication.CustomButton
        android:layout_width="match_parent"
        android:layout_height="50dp"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/notButton"
        android:importantForAccessibility="yes"
        android:screenReaderFocusable="true"/>

public class CustomButton extends LinearLayout {

    public CustomButton(Context context) {
        super(context);
        LayoutInflater inflater = (LayoutInflater) context
                .getSystemService(Context.LAYOUT_INFLATER_SERVICE);
        inflater.inflate(R.layout.custom_button, this, true);
    }

    public CustomButton(Context context, AttributeSet attributeSet) {
        super(context, attributeSet);
        LayoutInflater inflater = (LayoutInflater) context
                .getSystemService(Context.LAYOUT_INFLATER_SERVICE);
        inflater.inflate(R.layout.custom_button, this, true);
    }

    @Override
    public CharSequence getAccessibilityClassName() {
        // return a button class name for this
        // You have to use the getName() of native Android widget classes
        return Button.class.getName();
    }
}

This is broken up into 3 utterances by TalkBack:

1. “Button”;
2. “This is a custom button”;
3. “With multiple children”.

If you inspect the accessibility tree of this view, you will find that it is a Button with two TextView children. This is why TalkBack is announcing it in this order.

You could make this a single utterance for the actual text by concatenating all of the text from the child elements into the contentDescription for the parent view, but that would be an ease of use enhancement (one that I do actually recommend for speed of navigation). TalkBack will always announce the role in an utterance separate from the content of the view.

This is just a single line of code assuming you’re using only one constructor. Theoretically, you can override getContentDescription for the custom view, but Google doesn’t recommend doing that and will show a warning in Android Studio.

public CustomButton(Context context, AttributeSet attributeSet) {
    super(context, attributeSet);
    LayoutInflater inflater = (LayoutInflater) context
            .getSystemService(Context.LAYOUT_INFLATER_SERVICE);
    inflater.inflate(R.layout.custom_button, this, true);
    this.setContentDescription("This is a custom button \nWith multiple children");
}

More information about the getAccessibilityClassName() method can be found in the developer documentation for the View class.

AccessibilityDelegate

The AccessibilityDelegate is my favorite way of applying roles in Android. It offers complete control of the accessibility information for the view. We’re only going to be looking at setting a role, but with the AccessibilityDelegate, you can do pretty much anything (within the realms of the API) including reacting to accessibility events, applying custom accessibility actions, and even making custom modes of navigation.

Now let’s get back to the AccessibilityDelegate and roles. The AccessibilityDelegate allows us to modify the AccessibilityNodeInfo for the view. This is the information that an accessibility service like TalkBack reads to determine all characteristics of a View. The code below is creating a class called ButtonDelegate that extends from View.AccessibilityDelegate. Inside of that class, we are overriding the onInitializeAccessibilityNodeInfo and calling setClassName on the AccessibilityNodeInfo object to override the role presented to accessibility services.

public class ButtonDelegate extends View.AccessibilityDelegate {
    @Override
    public void onInitializeAccessibilityNodeInfo(@NonNull View view, @NonNull AccessibilityNodeInfo info) {
        super.onInitializeAccessibilityNodeInfo(view, info);
        // set the class name to a Button
        info.setClassName(Button.class.getName());
    }
}

Now that we have a custom AccessibilityDelegate we can reuse this class to override the role of any View and make it a button. In the example below, I have created a simple TextView. Normally, these would be announced without a role, but since we have the ButtonDelegate, we can easily add a role. On the TextView, call setAccessibilityDelegate and set it to the new delegate we created. And that’s it!

We can create a simple TextView in XML.

<TextView
    android:id="@+id/delegateView"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:text="Custom Delegate" />

And finally set the AccessibilityDelegate to our custom one.

TextView delegateView = findViewById(R.id.delegateView);
ButtonDelegate buttonDelegate = new ButtonDelegate();

delegateView.setAccessibilityDelegate(buttonDelegate);

This causes the TextView to now be identified as a Button, and we can verify this with an accessibility inspector. The view is just a single view — with no children and a text property of “Custom delegate” — which is what we created in the XML, aside from the role that we just overrode in the Java file.

Using a role description

There’s an additional way to apply a role to a view using the AccessibilityNodeInfo — or the AccessibilityNodeInfoCompat class to be exact. The AccessibilityNodeInfoCompat class is just a helper class that Google has created to extend the functionality of the base AccessibilityNodeInfo and make it a little easier to handle. The setup for the compat class is the same as creating a custom AccessibilityDelegate except you use AccessibilityNodeInfoCompat to wrap the AccessibilityNodeInfo object (it’ll make more sense with the code example below). Within AccessibilityNodeInfoCompat, there is a method called setRoleDescription. This method allows us to set any text as the role for the View. This method does not take localization into account, so if you’re using this method, you’ll need to do that on your own.

It is also important to take into account that other accessibility services may not know what to do with a roleDescription since it is not part of the base AccessibilityNodeInfo class. Braille displays will not abbreviate this information (based on a quick search of the TalkBack source) such as “btn” for the “Button” role since the role is not being applied as it is expected for a button. In other words, it’s not robust.

The only practical use case for this method is when you’re creating a tab control, since tabs do not exist in the base Android widgets. Google has one as part of the Material components but android.widget (the base Android widget class) does not contain a Tab class. In the code below, I am applying the “Tab” role to a TextView that we created in the XML.

<TextView
    android:id="@+id/roleDescriptionView"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:text="Role description"/>

Create a custom TabDelegate that extends from AccessibilityDelegate.

public class TabDelegate extends View.AccessibilityDelegate{
    @Override
    public void onInitializeAccessibilityNodeInfo(@NonNull View view, @NonNull AccessibilityNodeInfo info){
        super.onInitializeAccessibilityNodeInfo(view, info);
        // create an AccessibilityNodeInfoCompat object
        AccessibilityNodeInfoCompat infoCompat = AccessibilityNodeInfoCompat.wrap(info);
        // set the role description to "Tab"
        infoCompat.setRoleDescription("Tab");
    }
}

Apply the TabDelegate to the TextView that we created.

TextView roleDescriptionView = findViewById(R.id.roleDescriptionView);
TabDelegate tabDelegate = new TabDelegate();

roleDescriptionView.setAccessibilityDelegate(tabDelegate);

This causes the TextView to be announced as a “Tab”, and with an accessibility inspector we can confirm that the role description is set to “Tab”. The tab control that Google has created (like in the Google Play Store) would be exposed in the accessibility inspector in exactly the same way.

Clickable ImageView

Any ImageView that is clickable will automatically be announced as a button by TalkBack. This is something that I do not agree with, because TalkBack is inferring a role based on two properties, instead of relying on just a single property (like the classname or role description). It is helpful for users, but for third-party accessibility services, it means that they will need to follow this convention as well to have parity with TalkBack. And a third-party developer wouldn’t know about this inference without knowing the TalkBack source code.

In the TalkBack source code, it checks to see if the node is an ImageView and if it is clickable. If both of those are true, then it’s treated as a button.

Sometimes you just can’t apply roles

I will admit that sometimes, it isn’t possible or practical to apply roles for some situations. Google seems to be correcting this with their Jetpack Compose framework, but for View-based projects, if you can’t access the View object for a certain view, then you can’t apply an AccessibilityDelegate to that view. This seems to be limited to a few circumstances:

1. Menus — specifically ones that are announced as “popup menu”. These are often found as part of the top action bar as part of the overflow menu (kebab or 3 vertical dots icon). For these, you can set the text content of the menu, but you don’t have practical access to the actual View object and therefore can’t apply any customizations to the accessibility information.
2. Settings Fragment — or anything to do with a SettingsFragment or SettingsActivity. These are generally portions of the app that look like the built-in settings app. These classes use XML files to build user preferences layouts. It makes it very easy to build out user options and preferences, but these classes completely hide the View objects from developers. So you don’t have access to apply an AccessibilityDelegate.

Misconceptions on ListView and RecyclerView

I’ve run into opinions that claim it’s not possible to set roles on a ListView and a RecyclerView. It is important to keep in mind that a ListView and a RecyclerView are nothing more than fancy scroll views that contain stacked children views. As I mentioned before, if you have access to the View object, you can set an AccessibilityDelegate; and if you can set an AccessibilityDelegate, you can provide a role.

With a ListView and a RecyclerView, there are a few different ways to build them, but most of the time, they are built with an XML layout containing the view that you want to act as the template of the repeating view of the stack. This is one where you can create a custom view and override the role using one of the methods I mentioned earlier.

Below is an example using a ViewHolder (which is specific to a RecyclerView, but the concepts would still apply to a ListView). Note that this is not the full class of a custom RecyclerView — only the relevant part for overriding the role.

public static class ViewHolder extends RecyclerView.ViewHolder {
        private final TextView textView;

        public ViewHolder(View v) {
            super(v);
            v.setOnClickListener(new View.OnClickListener() {
                @Override
                public void onClick(View v) {
                    //do something
                }
            });
            View.AccessibilityDelegate accessibilityDelegate = new View.AccessibilityDelegate() {
              @Override
              public void onInitializeAccessibilityNodeInfo(View view, AccessibilityNodeInfo info) {
                  super.onInitializeAccessibilityNodeInfo(view, info);
                  // set the role to a button
                  info.setClassName(Button.class.getName());
              }
            };
            textView = v.findViewById(R.id.textView);
            textView.setAccessibilityDelegate(accessibilityDelegate);
        }

        public TextView getTextView() {
            return textView;
        }
    }

The output of this is similar to the previous examples except the RecyclerView adds list semantics. So on top of announcing as a “Button”, it also announces the index of the button in the list.

Conclusion

While it may seem convenient to rely on “Double tab to activate” as an indication of a role, in the end, it is just a usage hint and can be disabled by the user. There are several methods of applying roles in native Android apps, and I will admit, that some methods aren’t intuitive (or documented very well). But once you get the hang of manipulating the accessibility nodes, it’s fairly easy and doesn’t take very many lines of code.

If you are already making a custom class that extends from View, then you can override a single method to set the role of the view. This is two lines of code (excluding the closing curly brace and the @Override annotation). One rule of thumb is that if you’re going to be making a custom clickable view and it doesn’t have any checked states (meaning its a button and not a checkbox or radio button), go ahead and override the getAccessibilityClassName() method to return Button.class.getName().

If custom views ain’t your bag, or you need a little more control over the accessibility node, then the AccessibilityDelegate is the way to go. And keep in mind that if you have access to the View object, then you also have access to do what you need to do with the AccessibilityDelegate.

Coding roles (or any of the other accessibility properties) in the way that is expected is how you create a robust solution for any assistive services (not just TalkBack) to use. And if new accessibility features are released for existing accessibility services, and your app has implemented accessibility in a robust way, you will likely need to do nothing for your users to enjoy the new accessibility features. Because the expected properties are already there and ready to be taken advantage of.

Resources

• Android Accessibility Inspector
• View.AccessibilityDelegate
• AccessibilityNodeInfo
• View.getAccessibilityClassName()
• RecyclerView

Image credit: Merve Sehirli Nasir.