The Nuance of a Preflight Accessibility Audit for Gutenberg

I’ve explained my concerns and ideas for this a few times over Slack and Twitter, and now I’m taking to my blog to jot down some of my release plan concerns for Gutenberg as we approach the mid-November 2018 launch of WordPress 5.0.

First, I’m pleased that there is an intent by the elected accessibility launch lead to conduct an independent Gutenberg accessibility audit. I have two primary concerns about the process of this audit: the scope of the audit and how issues will be addressed or an action plan created for addressing them. Let’s start with the audit scope.

Accessibility Audit Scope

There are two general approaches to auditing Gutenberg. First, a block-by-block audit at the component level to ensure that individual controls are accessible. For example, the Cover Image has color controls that need to be tested and verified, both programmatically and for their experience. Second, there is an overall user experience audit that needs to be conducted that documents issues relating to the entire content creation and publishing process end to end.

The block-by-block approach is a chase-down audit to fix regressions or other accessibility issues, and I have no doubt that there will be some bugs of varying severity. My larger concern is with a more user experience-driven test, which will need relative data or context in order to be meaningful. Conducting the test from the lens of is this new experience accessible is significantly different than is this new experience creating better outcomes for everyone using it. How these tests are carried out will go a long way to the legitimacy of these tests as user research.

Personally, I think any UX accessibility test should proceed using three suites: Gutenberg, classic editor, and a third-party competitor or other content creation platform. This would give a wide range of information and data that would tell a better story of how Gutenberg performs–not just at the accessibility level, but on the overall experience for all users as well.

Action Plan for Discovered Issues

Finding issues won’t be a challenge, and it would be naive to think that Gutenberg (or any software) is going to be perfect from launch. For the sake of transparency, it’s important to detail how issues will be documented and resolved. There should be established criteria that explicitly states:

  • What the issue is
  • The severity/impact of the issue
  • Development priority of this issue
  • When it will be resolved
  • Who will be responsible for ensuring that it is resolved
  • What an accessible alternative is in the meantime

This is incredibly important for our technical leaders that have accessibility obligations under federal law (or similar) who cannot deploy software that violates accessibility standards. There should also be an amount of predictability to how these issues are handled so decision making isn’t made on an ad-hoc basis.

This process will be one deliverable adjacent to an action plan following the results of the audit. If this isn’t done, it is my fear that stakeholders (and the community) will continue to lose trust and faith in the project leadership to anticipate and mitigate these issues, particularly those that are vested in ensuring an accessible WordPress experience.

Leave a comment

Your email address will not be published. Required fields are marked *