Why Manual Accessibility Testing is Necessary
Accessibility Checker is an excellent tool for identifying some of the most common accessibility problems on websites and those which can be identified via automated code review or AI. It is important to note, though, that not all accessibility problems can be identified automatically.
By some estimates, only 30% of accessibility problems can be identified (or corrected) automatically, which is why you’ll frequently hear Equalize Digital and other accessibility experts talk about why accessibility overlay tools and plugins don’t work. It’s also why our plugin uses the phrase “100% passed tests” and not “100% accessible” when auditing your WordPress posts and pages.
Web Content Accessibility Guidelines (WCAG) on Manual Testing
WCAG 2.0 AA is the standard that most government-funded websites need to follow under Section 508. This is generally considered to be the bare minimum standard for meeting business/private website accessibility requirements under the Americans with Disabilities Act (ADA) as well. Though no clear legal guidance exists for how businesses can meet ADA requirements on their website, from a “risk of lawsuit perspective,” having a website that complies with WCAG AA is considered best practice.
The most recent standard is WCAG 2.1, which was released in June of 2018. WCAG 2.2 was originally targeted for release in November 2020 but was delayed due to COVID and will be released sometime in 2021.
WCAG 2.0 AA contains 38 “success criteria” or items that need to achieved to ensure accessibility. 34 of the success criteria require manual review by a human. WCAG 2.1 AA added 12 new success criteria that also require manual review, and WCAG 2.2 is expected to add more success criteria that can only be reviewed by a human and not automatically.
If you want to ensure true WCAG, Section 508, or ADA compliance, in addition to auditing your website with Accessibility Checker, it is critical that you also check your website for accessibility manually.
What is Manual Accessibility Testing?
Manual accessibility testing is the review process a human goes through to determine if any accessibility problems exist on a website. Examples of manual accessibility testing include:
- Navigating through the website with only a keyboard to ensure that all content can be accessed, used, and engaged with by a user who cannot use a mouse.
- Listening to the website with a screen reader to make sure text content and controls can be accessed by a screen reader and is understandable when read aloud.
- Watching embedded media to check for the presence and accuracy of closed captions, and that no rapid flashing is present.
- Using the website on multiple device sizes and at multiple resolutions to ensure that it is just as easy to use on a phone as on a desktop computer and can be used if significantly zoomed in.
Manual accessibility testing vs. user testing
Manual accessibility testing is different from user testing. When you perform user testing on your website, the users are doing manual tests, however, the users participating in user testing sessions are not typically looking at every web page exhaustively. Rather, they are attempting to accomplish certain goals that you have laid out for them in the testing brief.
As part of our accessibility audit and remediation process, we recommend completing automated and manual accessibility testing in tandem – then resolving all issues (in multiple rounds, as needed) – before performing user testing. We like to use user testing sessions to prove accessibility and only weed out a few small remaining issues. In our process, we conduct user testing once we believe the website is fully accessible, and after all of our manual testing is complete.
How to Test Website Accessibility Manually
Create a plan
If your website is already live and you’re trying to identify accessibility problems that need to be fixed, your approach to manual accessibility testing will likely be different than if you’re testing a newly developed but not yet launched website.
If your website is not yet live, you can move through the content starting with the home page and then checking interior pages in whatever order makes sense to you.
If your website is currently live, then we recommend testing your pages in this order:
- Test the page on your website that receives the most traffic (check Google Analytics or whichever tacking software you’re using).
- Test pages where a conversion is expected to happen or that have an impact on your bottom line. This includes product pages, cart and checkout pages, pages with forms, ad landing pages, or pages where people need to go to find your phone number.
- Test your home page.
- Test one representative page from each post type on your website (i.e., one staff bio page, one event calendar page, one blog post) and the archive page for any of these post types.
- Continue testing through the rest of your content in order of the most traffic received to least traffic received until you have tested every public URL on your website.
Start by testing keyboard navigation
The first place we like to start when performing manual accessibility tests is with keyboard navigation.
Not all people who visit your website will be able to use a mouse and, as a result, these people will be using a keyboard only to navigate the web. This includes blind and visually impaired users who cannot see and will be using a keyboard in combination with a screen reader and sighted people who can see but are unable to use a mouse because of motor impairments or injuries. This latter group could include people with quadriplegia, people who have ALS (amyotrophic lateral sclerosis) like Stephen Hawking, people, like veterans, who have lost their arms, and the elderly. In certain situations, there are also non-disabled users who prefer to use their keyboard only to move through websites or complete certain tasks, such as filling in forms.
Your first round of keyboard testing will benefit the group of keyboard users who are sighted.
To keyboard test your website, load the page you want to test in any browser and then use only the keyboard (typically the tab key and arrows) to move through every page and interact with every feature. You want to make sure that everything you can do on your website with a mouse can also be done with a keyboard only.
Things to ask yourself when keyboard testing your website:
- Can you always see the focus and where you currently are on the page or do you easily lose track of where your focus is? Every link or button should have an obvious outline or color change when you tab to it.
- Do the colors for focus states meet contrast ratios?
- Can you move to different pages using the navigation menu?
- Can you quickly jump to content without having to tab through the navigation menu? (I.e., are there skip links present on your website?)
- If a popup opens can you close it? Is your focus locked into the popup until you engage with it or close it, or can you tab out of the popup without closing it?
- Can you interact with any special features like maps, sliders, and video or audio players?
- If you interact with a feature does your focus shift unexpectedly?
- Can you fill out and submit forms?
- Can you complete a purchase or do any other actions you want your visitors to take?
Test your website with screen readers
As noted above, people who are blind or visually impaired will navigate your website and interact with it using a screen reader. After performing keyboard tests to check your website for a visible focus state and to ensure everything can be interacted with as expected, the next step is to test your website with at least one screen reader.
There are three common screen readers that you can choose from for testing: VoiceOver, NVDA, and JAWS. If you have a Mac, VoiceOver is built into your computer and is a great place to start. If you don’t have a Mac, NVDA is a free open-source screen reader that you can download and use on your PC. NVDA can also be used on Macs. JAWS is a paid screen reader, so may not be where you want to start, but is very popular with blind and visually impaired users, so would also be good to fit into your testing plan as your budget allows.
Because screen readers do not all say the same thing for each element they encounter, we recommend testing your website with at least two different screen readers. This will ensure you have a full picture of what users of different technology will encounter on your website.
To test a web page with a screen reader, install or turn on your selected screen reader and load the page you want to test. Move through your website from top to bottom just as you did during keyboard navigation. Listen to what the screen reader says as you go down the page and test all the features.
Listen for anything that sounds confusing or incorrect, for example:
- Links that don’t describe their purpose: you might hear “Clickable graphic, clickable graphic” or “link, link, link” over and over again and have no idea what it’s supposed to do or how they’re different from one another. This also applies to ambiguous links (like “learn more”) that have no meaning when heard out of context – “Learn more about what?”
- Elements with confusing interaction: we see this a lot with tabbed content or accordions. If the tab is clicked or the accordion is opened but the focus does not move to the content inside the tab or accordion, and if the content is not immediately available with a tab or arrow action, screen reader users will not know how to find the content.
- Hidden interruptions: if you have a slider or media automatically playing elsewhere on the page, that can interrupting what the screen reader is currently reading lower down on the page and be disruptive to the user.
Check embedded media and linked content for accessibility
There are a number of items related to media and other embedded content that need to be tested manually. Whether you have a YouTube video, a slider, an embedded PDF or Doc, or a third-party widget or iframe on your website, you need to make sure that these embeds are all accessible too.
There are a number of considerations you need to take regarding different media types whether they are linked to or embedded on the page. We have specific information about those items in other help articles:
If you have embedded content from social media (like an Instagram post) or an iframe from a third-party site, these need to be able to be fully interacted with just as the rest of your website can be. Make sure that you test them with both keyboard navigation and a screen reader.
Get Help With Manual Testing
Manually testing a website for accessibility doesn’t have to be overwhelming. With the right tools and a thoughtful approach, you can move page-by-page through your website, making it accessible one page at a time.
However, if manual testing feels too complicated or if you would like an expert review of your content, Equalize Digital is happy to help. You can purchase a remediation plan that includes both manual accessibility analysis and fixes, or contact us if you would like a full accessibility audit.
