Are you overwhelmed at the prospect of conducting accessibility audits for large websites that have numerous pages? In this talk, Natalie covered strategies for efficiently planning and conducting audits for large-scale sites, including WordPress sites with hundreds or thousands of posts, pages, WooCommerce products, and other types of content.
Natalie explored how to stay organized and track progress throughout the project, including techniques to streamline your testing efforts, making informed trade-offs, and striking a balance between automated, manual, and functional testing approaches.
The talk covered how to track and record accessibility issues effectively and how to organize issues to help remediation efforts go as smoothly as possible.
Thanks to Our Sponsors
Paid Memberships Pro is the most complete WordPress membership plugin—and it puts you in control. Build your dream membership site exactly how you want it. PMPro has enabled thousands of creators from all over the world to grow robust membership businesses around their content. Plus, PMPro is 100% free to use. Restrict content, accept payments, and manage subscriptions right from your WordPress admin.
Watch the Recording
If you missed the meetup or would like a recap, watch the video below or read the transcript. If you have questions about what was covered in this meetup please tweet us @EqualizeDigital on Twitter or join our Facebook group for WordPress Accessibility.
Read the Transcript
>> AMBER HINDS: I am going to kick us off because it is officially five past the hour, and I want to make sure we have enough time for our speaker this morning, or evening, depending on where you are in the world, so I’m going to do a few announcements and then I will hand over. Feel free to continue introducing yourself in the chat if you want to, if you missed that opportunity.
If you haven’t been before, we have a Facebook group which is a great way to connect with people in between meetups. You can find it if you just search “WordPress Accessibility” on Facebook, or you can go to “Facebook.com/groups/WordPress.accessibility.” Ask questions, share things you’re working on, answer other people’s questions. We’re just trying to build a community around the meetup to help people in between events. It’s a great place to go. Go join it if you’re on Facebook.
People always ask, “Is this being recorded?” I can guarantee you it’s going to get asked in the chat later on, but the answer to that is, yes, this is being recorded. It takes us about two weeks to get the video edited, and then we get corrected captions and a full transcript, and then we will post the recording. You can find recordings for past meetups if you go to “EqualizeDigital.com/meetup,” and there’s a recordings heading, and under that, you can find all of the past recordings.
If you want to get notified when the recording is available, please join our email list. We send out news and event announcements, and you can find that by going to “EqualizeDigital.com/focus-state.”
The other way that you can listen is via audio. We had a few people request that, so we are putting just the audio from the meetup recordings on our podcast, “Accessibility Craft.” That goes out every other week, and then every other week, we have sort of a conversation about craft beverages and web accessibility, and you can find that if you go to “AccessibilityCraft.com.”
We are seeking additional sponsors for the meetup. The WordPress Foundation, unfortunately, does not have the ability to cover the cost of live captions or post-event transcription, and so they have told us that we need to find our own sponsors or pay for it ourselves, so if you have a company who is interested in helping to support the meetup so that we can continue to make it as accessible as possible to all of our attendees, we would very much appreciate your help and would be happy to tell you more about sponsorship. You can contact myself and Paola, if you email “Meetup@EqualizeDigital.com,” and that’s a great way to learn more about sponsorship.
The other thing is, we always love to hear from people who are interested in talking, or people who say, “I don’t want to talk, but I would really like to learn X.” Because that gives us some ideas, and we can go out to the community and see if we can find speakers for those topics, so if you have any feedback for us or any questions about the meetup, you can send us an email at that email address, or you can tag us in the Facebook group.
So who am I? If you haven’t been before, my name is Amber Hinds. I’m the CEO of Equalize Digital. We’re a certified B corporation that specializes in WordPress accessibility, and we are the maker of the Accessibility Checker plugin for WordPress, which is an auditing tool that can be used in tandem with manual auditing techniques to help speed up testing of accessibility or finding accessibility problems on websites.
I’ve said our website a bunch of times, so I’m not going to say it. We’re on Twitter. I’m still keeping the bird [laughs]. We’re at “Equalize Digital” if you want to find us on social media.
We do have a sponsor this evening that I want to thank for helping cover the cost of live captions and that is Paid Memberships Pro. Paid Memberships Pro is a WordPress membership plugin that allows you to build your dream membership site. It is 100% free to use and open source. Even all of their paid add-ons are open source, but of course, it’s nice to purchase from them because they give really great support, and they are a huge advocate of the WordPress community, and do a lot to give back in the WordPress community. You can learn more about them if you go to “PaidMembershipsPro.com.”
We always like to encourage people to thank our sponsors on social media and literally tweet at them or send them a LinkedIn message and tag them and say, “Thanks for sponsoring WordPress Accessibility Meetup,” because it helps to encourage them to want to do it again, so if you’re on Twitter, they are “@PmProPlugin,” and you can also find them on LinkedIn and some of the other social media, so if you’re inclined this evening to thank them for their sponsorship, we would very much appreciate that.
There are a couple of upcoming events that you might want to mark your calendars for. On Thursday, November 2nd at 10 am Central Time, McKenna Letterman [phonetic] is going to be talking about, “Doing More With Less Aria.” So ways that you can use less Aria, but still ensure that you are building accessible websites.
Then on Monday, November 20th at 7 pm Central, the same time slot, we’re going to have Reed Pirnach [phonetic] talking about “Component Pattern Libraries for Accessible WordPress Experiences.”
Next, on Thursday, December 7th at 10 am Central, Andrew Malis [phonetic] and Mike McAfrey [phonetic] are going to talk about, “Beyond WCAG Compliance: Steps In Your Websites Commitment To Digital Inclusion.”
Maybe I’m going to get my slide to change. Or maybe not. The wrong direction. Here we go. Arrow keys. Better than a mouse. [laughs]
I’m going to add a spotlight here so everyone can see our speaker, Natalie MacLees. I’m very excited to have you here, Natalie, so welcome.
Natalie is a digital professional with over 25 years of experience building on the internet, and has had the privilege of working at Penn State University, where she gave a deep understanding of the importance of building websites to be accessible to everyone, and we’re really excited to have you here tonight speaking about how you handle accessibility on very large websites, because as I can imagine, you probably have like 500 websites at Penn State University, [chuckles] with many, many pages, so we’re very excited to hear more from you, and welcome.
>> NATALIE MACLEES: Thank you so much. Thanks for having me.
>> AMBER: Yes. I’m going to stop sharing so you can start sharing. While she is doing that, I do just want to note that I will be watching the Q&A module. I highly recommend, if you can, putting chats in the Q&A. I try to watch the actual webinar chat as well, but sometimes things get a little buried, so if you have questions, please put them in the Q &A.
>> NATALIE: All right, ready?
>> AMBER: Yes, I think so.
>> NATALIE: All right, so we’re going to talk about managing audits for big websites.
First, nice to meet all of you. Thank you so much for having me. A little overview of what we’re going to cover tonight. First up, “How to select what to test?” Because if we have a website with 10,000 or 50,000 pages, we’re not going to test all of those. [chuckles] That’s just not practical, so we’ll talk about how we figure out what we’re going to test.
We’ll talk about how to break up a big project and manage it. We’ll talk about how to optimize your testing, and take advantage of the tools available, and we’re going to talk about how to track remediation, and retesting, because if a developer says, “I’ve fixed this,” you can’t trust them. You need to retest it again and make sure, and how do we keep track of all of that?
I wanted to get started by just a quick reminder that audits don’t fix anything, so all and all it does is find the accessibility issues on the site that needs some attention, but it doesn’t actually change anything about the site at all, so when we’re conducting accessibility audits, we want to make sure that they are deliverable to our client or to the remediation team, is clear and actionable, even if it’s an enormous site with tens of thousands of pages.
So let’s go ahead and dive in and get started. First, we’ll talk about how we select what to test, and I like to get started with the site map. For most WordPress websites, this should be readily available. You used to need a plugin to have a site map on a WordPress site, but I think since WordPress 5.5, site maps have been just automatically generated in a WordPress Core. If you happen to come across site that needs an audit that doesn’t have a site map for some reason, you can crawl and generate one.
Sometimes I don’t always have access to the site at this stage of the project, so I will use a third-party tool. They’re usually SEO tools, and they are meant to keep an eye on the website over time, but they do a fantastic job of just generating a site map and giving you a list of URLs, and because they’re meant for SEO, they’re usually a monthly subscription fee, so I’ll just sign up, pay it for one month, and then cancel it, just to get the site map, so that I have something that I can get started from.
We need to identify not just URLs on pages of the site, but also PDF files that might need some attention, documents, spreadsheets, presentations, audio, video, et cetera. All of those things might need to be tested, so we need to make sure that we’re tracking all of them.
Once we have the site map, we can get started with some automated testing. Since automated testing doesn’t actually take a lot of manual effort, we can be a lot less restrictive about what we select for automated testing. You still probably don’t want to run tests on every page of the site, just because that generates a lot of noise.
If you have a WooCommerce site, for example, with a thousand products in it and we run automated testing on all 1000 product pages, there’s just going to be the same issues that are in the template over and over and over again, and it’s not actually that helpful, so we want to just make sure that the URLs we select for automated testing represent a good sample of the different types of templates and different types of pages on the site.
Plus, as a rough guideline, you’d want to pick two to three times what you’d pick out for manual testing, and then you can go ahead and run the automated testing right away. There’s nothing stopping you from doing that.
From there, you can plan out the manual testing. Now, since manual testing is time intensive, we have to plan carefully, and we have to figure out a way that we can prioritize the content that’s on a site, so let’s take a look at some ways that we can use to prioritize content on a really big site.
First up, traffic. If we have access to analytics, we can figure out which are the most popular pages of the site, and it makes sense to prioritize accessibility fixes on the most popular pages, because those are going to impact the most number of people.
We might also look at pages that had a large number of issues found in the automated testing, so if the automated testing finds, for some reason, a thousand issues on a page and most of the other pages only have about a dozen issues, that’s a good hint that maybe that page needs some attention, because it seems like there’s something going on there, so we can take a look at our automated reports and figure out which pages might look like they might be a little problematic and might need some manual testing to figure out what’s going on, and some attention from developers to get those issues fixed.
Then we want to look for key templates. If you understand how a WordPress theme works, figuring out what the templates are and which content uses which templates is relatively straightforward, and we want to identify content that uses each of the different templates that are in the theme.
Just a reminder that those templates, they can include conditional components or page sections, so we want to make sure that when we’re selecting a sample, that we’re getting all of those different conditional components.
As an example, if I have a WooCommerce site and I sell stickers and T-shirts and books, pages with the T-shirt products on them will probably have some additional selectors for T-shirt size, T-shirt color, maybe even T-shirt style, you know, if I want a crew neck versus a V-neck, and obviously, those selectors aren’t going to be on my stickers and books, so I want to make sure that I’m selecting products that have that section, products that don’t have that section.
The pages that present books for sale might have information on the format of the book. Is it an eBook? Is it a paperback? Is it a hardback? It might have the number of pages. It might have the publisher’s name.
So there might be a section with metadata about books that obviously wouldn’t show up on the T-shirts and the stickers, for example, so I’d want to make sure when I’m selecting specific products, specific URLs to do manual testing, that I’m getting a good sample, and I’d want to have a couple of pages with the T-shirt selectors, a couple of pages with the book selectors, and then a couple of pages of the stickers that had neither of those, and maybe had a different section altogether, so just to ensure that when you pick the individual pages, you’re getting a good sample of those conditional sections or components.
Finally, the importance of the content, and what do I mean by important content? We’ll dig into that a little bit. First up is the homepage. I can’t really think of an example where you’d run an audit on a site and not want to include the homepage, unless you were doing rounds of audits and you’d already done the homepage in an earlier round, but the homepage is a really important page. I know on websites I’ve managed over the years, it’s 40% to 50% of people who come to the site that land on the homepage, so it’s a pretty important page of a site.
Next, we want to think about navigation, which includes not just navigation that might be in our header and our footer or sidebar, but also different parts or pages of the site that relate to navigation.
As an example, our site map, we want to make sure that that’s accessible. If we have “search” functionality, we want to make sure the search form itself is accessible, and that the search result listings are also accessible and correctly marked up.
Our accessibility statement needs to be accessible, because everyone should be able to access that information, and we don’t want to have any barriers on our accessibility-statement page.
Any kind of contact information, so if we are sharing our address, phone number, email address, other links for other pages that the company might own, or if we have a contact form, we need to make sure that all of those things are accessible. Those are all very important content.
If you have any sections of the site that tell users how to use the site or are guides or help articles on how to do different things on the site, you want to make sure that those are accessible, because everybody should get access to those instructions.
Any legal pages you have, so if you have terms of service, privacy policy, pages like that, you’ll want to make sure that those are accessible. They’re high priority.
Finally, if you have any pages with settings and preferences. By default, out of the box, the subscriber role in WordPress does get access to the WordPress Admin to their own profile page, where they can change their password and maybe some other fields, depending on what you have going on on the site, and the WordPress development team does a pretty good job with the accessibility team to make sure that is accessible, but if you have WooCommerce or a membership plugin or some other kinds of plugins, that settings page gets moved to the front-end of the site, and you’ll just want to make sure that once it has moved there, that it is accessible, and everybody can access it to change their password, pick if they want dark mode or light mode on the site, whatever preferences that you have that people can set. Those are a high priority for accessibility.
All right, and then we want to get ready to do functional testing. Just like manual testing, functional testing can be really time intensive, so we want to plan carefully, and what kinds of things are we testing with functional testing? This is any kind of interactive tools or transactions on the website, so the kinds of things we want to take a look at are, if there’s a cart or checkout functionality. If you can book appointments. If there are modals, we want to make sure that those are managing focus correctly and make the “close” with the “escape” key, et cetera.
If there are carousels, those are a little bit challenging, for screen reader users in particular, so we want to pay careful attention to any carousels on the site. If there’s tabs, accordion, or other types of interactive elements, we want to make sure that we test those.
Any kind of forms, so we already talked about the contact form, but if you have forms for people to sign up for things or to request features or anything like that, you want to test all of your forms to make sure those are all accessible.
Of course, login and log out. Everybody needs to be able to do that, and commenting should be accessible.
I will note that there are plugins that you can get for your site that allow you to favorite comments, right? That’s not available out-of-the-box. Or favorite posts, and very few of those are accessible, so be careful if you add that functionality. Carefully select your plugin for that.
Then any kind of embedded content, so if you’ve embedded a social-media feed, if you’ve embedded videos from YouTube or Vimeo, for example, if you’ve embedded forms from a third party that aren’t built in your WordPress Admin. All of those kinds of things, you would want to make sure that you test to be sure that they’re accessible.
All right. Let’s talk about how we break up big projects, because I’ll tell you, what you don’t want to do is spend weeks or months doing a huge accessibility audit on 100 pages of a site, and turn it over, you know, turn over this list of 10,000 accessibility issues you found and find out that a good chunk of those are no longer valid because content has changed on the site, and to find out that the company has only assigned one developer to the remediation project, so we want to make sure that when we’re turning over the results of our audit, that they’re relevant, that they’re actionable.
So we want to think in sprints. The agile methodology is really helpful here, and we always think a sprint is going to go like this: that we do the audit, and then the developers remediate the issues that we found, and then everything is fixed, but the reality of how that usually goes is, you finds some issues, the developers remediate, test again, and find new issues. [chuckles] Or the fixes weren’t complete, or the fixes weren’t the correct fix for the problem, so it does tend to be a little bit of a cycle, so it’s best to be prepared for that and plan for that ahead of time.
Some questions to ask yourself as you’re thinking about breaking a big project into sprints. First, “How big is the remediation team?” So it’s good to know that up front, because if you only have one part-time developer who’s going to be working on remediation, you don’t want to overwhelm them and you don’t want to send them three years worth of work, because it’s probably not going to get done. It’s just going to be overwhelming, so have a good idea of who is going to be doing the remediation work and how many people there will be involved.
Then you want to know how much accessibility experience the remediation team has, because if it’s a team that’s relatively new to accessibility, they’re going to need a lot of coaching and a lot of hand-holding in order to be able to resolve the issues and fully understand them, and that’s a lot of drain on your own time, so you don’t want to overwhelm yourself and have no time left to do additional testing.
You want to find out how often the content on the website changes, so if there’s an SEO team that’s constantly playing around with different landing pages and funnels and things like that, it’s important to know that up front when you’re planning which URLs will be involved in the testing. You don’t want to waste time testing, like, a landing page that’s just up for a couple of days as a test, because even if you find issues, there’s no time to remediate it before it gets taken back offline again.
Finally, you want to look for things that you can shift left, and what this means is things that you can move earlier in the process so that they’re not hanging out there.
In the example I gave earlier where we have an SEO team who’s constantly testing different landing pages and funnels, are they consistently producing inaccessible pages with lots of issues on them? And if they are, what could we do to fix that situation? Could we provide a different tool for that team to use to build those pages? Could we provide some training for that team so that they would understand how to build things more accessibly? What can we move so that we’re not just doing a bunch of accessibility testing kind of at the very end of the process once new pages are live on the website?
For big websites, it’s important that we prevent new issues from happening. It’s just as important as making sure that we take care of the issues that are there.
>> AMBER: Can I interrupt for one second? Because there was a timely question. Could you explain what “shift left” means?
>> NATALIE: Oh, sure. Yes. I guess it’s very relevant if your native language is a left-to-right language. [chuckles] So you’d normally start on the left and go to the right if you’re listing out your process, right? So you’re going to do design, and then you might test the design, and then you might do development, and do some testing on the development.
What we mean when we say “shift left” is that, often, accessibility testing gets left for the very last step, all the way on the right. The very last thing over there. “Oh, yes, we should test this for accessibility,” and what we mean by shifting accessibility left is putting it earlier in the process. Can we think about accessibility in the design phase? Can the designer be thinking about color contrast, and usability, and different ways that we can accommodate people with cognitive disabilities in their design? Can we incorporate some kind of automated accessibility testing for developers so that every time they commit code, there’s some kind of automated accessibility testing running on that code? Can we run accessibility tests on a staging site or a development site, manual tests there to make sure that things are accessible before they are released publicly? That’s what we mean when we say “to shift left” on accessibility.
All right. Let’s take a look at some examples of sprints. I find that I’m often working with a remediation team who doesn’t have a lot of experience in accessibility, as clients will often might use their internal developers for, you know, when they have a large website, they have, usually, an internal development team and they just want that team to take care of remediation, but that team doesn’t necessarily have a lot of experience with accessibility.
So we might, as a first sprint, just do the homepage, especially if it’s a long or complicated page. That might be one sprint just on its own. We could go through any kind of contact, so if there’s contact information in the footer, a separate contact page, maybe there’s multiple contact pages. For a big company, there might be a contact page for each of their department, so we might have a sprint for accessibility where we just test and remediate issues specifically related to contact information and forms.
We might check the cart and checkout flow. Very important for an E-commerce site to make sure that’s all accessible. We might have a sprint where we just focus on navigation, just product pages on an E-commerce site. We might do just landing pages, so like we talked about earlier, an SEO team creating landing pages. We might do one where we just focus on blog posts, and we might do one where we just focus on the legal pages.
So we have lots of different ways that we could break a big project up into really small, manageable pieces that can be done in as little as a week or two.
Next, we want to figure out how we can optimize our testing and take advantage of all the tools that are available to us to make the process go as quickly and smoothly as possible, so we’ve got three different types of testing that we do as part of an audit. We have automated testing, manual testing, functional testing.
We all know automated testing is limited, right? It can only find 25% to 30% of the types of accessibility issues that can be found on a site, but automated testing can also offer some really helpful guidance for our manual and functional testing, and that can help us to save a little bit of time in those types of testing.
So an automated scan can identify images that are missing alt text, but of course, what still needs manual checking is, is that alt text relevant and accurate?
Automated testers can check if our form fields have labels, but they can’t check, “Are those labels helpful and descriptive?” And they can’t check, “Did we use some other method of applying a label to a form field?” For example, did we use the Aria label attribute on the element?
So we would want to make sure anything that gets flagged with automated testing.., sometimes those are just things that we need to check manually, and not actually issues.
All right, so we run the automated test first and we use those to help us identify problematic areas of the site or problematic components. Then we want to be sure that we’re recording all issues in one spot, so we want to make sure that our deliverable to the client, to the remediation team at the end of the audit is something that is easy to understand and actionable, and one way we do that is by having all of the issues together in one spot. We also want to identify the issues by severity and priority, so severity, I think we’re probably all familiar with. Critical issues are ones that are blocking one or more groups from accessing some page or part of the site, and then we have serious, moderate, lower-priority issues.
Now, you would think that the more severe issues are always going to be the highest priority and that is the case most of the time, but it’s not the case all of the time.
I’ll give you an example. If we have a serious issue, but it only occurs once on the site, and maybe it occurs on a page of the site that doesn’t have very much traffic, but then we have another issue that’s moderate, but it occurs 100 times on the site, and it occurs on pages that get a lot of traffic, we might actually decide that the moderate issue is a higher-priority fix than the serious issue, so when we know how often an issue happens, when we know how many pages of the site are affected, that helps us make some decisions to prioritize fixes.
All right, and then we need to make sure that we have a way to track remediation and retesting, because developers are going to work on the issues that have been found in the audit, and sometimes, the first try, they’ll get it right and the issue will be fixed; but other times, that’s not the case, so we want to have ways that we can track through the process, and make sure that we have a way to track when we have retested something and it is not meeting the accessibility guidelines and we need to send it back to the developer for more work.
Know that we often record accessibility issues in a spreadsheet, but a spreadsheet can be kind of hard to track. We could have a column for who’s assigned to it, we could have a column for the status, but it’s really hard to have a conversation about the issue. It’s hard to go back and forth with the developer a few times in a way that’s tracked in the spreadsheet to figure out what should be done to correct a certain issue or to help the developer understand the issue, but we do want to make sure that we get everything in one place, because the results of our audit, they have to be manageable and not overwhelming, if anything’s going to get done, so if we turn over something that’s confusing or overwhelming, the chances of the remediation getting completed are pretty low.
So we want to find a way to take all of our automated testing, our manual testing, and our functional testing and get that all into one place where the issues and statuses can be tracked and the assignments can be tracked, so manual and functional issues could be recorded directly in an issue tracker, or they could be imported from a spreadsheet. Automated tests are usually going to be a CSV. They might be like a JSON or something like that, but hopefully, whatever tool you’re using will allow you to import those kinds of issues and get them all into one place where they can be easily tracked. Manual issues could go directly into software, could go into a spreadsheet, and then imported later. Automated issues could be imported from whatever tool was used to run them.
If you’ll indulge me for just like two minutes, I’d like to share a little bit about a tool that my team and I have been working on, AAArdvark. We’ll be launching soon, and it is a tool for helping you to manage accessibility audits. It does include automated, manual, and functional testing all in one spot. All of the issues reported in one area. You can select an element in the UI of the website and report an issue with it right there.
Once you pick what the issue is, it will automatically figure out which WCAG guideline that applies to or success criterion that applies to and present you with some recommended fixes and additional information about it, so you don’t have to type that information in by hand.
You can also see directly on the website where both automated and manual issues are, and it will allow you to assign issues, track their status, have a common thread on each issue, and report on your progress over time.
We are launching soon. It’s AAAardvark Accessibility with three A’s, if you’d like to check it out. “AAArdvarkAccessibility.com.”
All right, thank you for indulging me just for a second on that. I just wanted to share about it. OK, so quick review of what we’ve talked about. We have four big challenges when we’re doing audits for big websites.
First, we need to select what we’re going to test, and that could depend on traffic. It could depend on the number of automatic issues that were found on the different pages. We want to make sure that we get key templates and include any conditional components or sections of those templates that might be visible or not on different pages, and then we want to think about the importance of different pages and ensure that we’re prioritizing the important content on the site.
Second, we want to break a big audit into sprints. We don’t want to deliver this enormous report at the end of a… You know, weeks or months-long audit to the client to try to remediate. We want to break it up into work that can be done in a week or two at a time so that we’re not overwhelming anybody and we’re making good progress on the website, and also, we don’t have issues that are months old that have been sitting there becoming obsolete.
We want to take advantage of automated testing and let that guide us along, and really integrate our automated, manual, and functional testing all together so that each type of testing supports the other two.
Then we want to make sure that we are delivering a report to the client and the remediation team at the end of the audit that is actionable, that’s easy to understand, that is easy to track and assign the issues, and get all of those in one place so that they’re really easy to understand.
OK, are there any other questions?
>> AMBER: Sorry. It always takes me a second to get myself back up. [chuckles]
>> NATALIE: No worries.
>> AMBER: Someone said they love the name, they’re excited about that, so we’ll all have to go check out AAArdvark Accessibility.
There were some questions that were asking specifically around what other tools you use for automated testing, so do you want to talk about some other tools that you recommend, or that you find helpful?
>> NATALIE: Sure. I really like the Axe tool from Deque. I find that one really helpful. I like that it’s integrated right into the browser.
I also like WAVE from WebAim. That tool is super helpful. I think it’s really nice also just as a site owner that you don’t have to install anything, that you can just go to their website and run the test and see the results right on their site, and not have to install a browser extension, you don’t have to do anything else.
I do like ANDI, that’s from the Social Security Administration that gets used in Section 508 testing. It’s a bookmarklet. It’ll help you along in a lot of manual tests, but it does some automated testing as well that I actually really like. The interface isn’t super pretty because, you know, it’s made by the government, but it is actually really helpful.
>> AMBER: Great. Thank you. Beth was asking: “Will AAArdvark have API endpoints?”
>> NATALIE: Yes, it will. Yes. [crosstalk ]
>> AMBER: Are those going to be there when it launches? Can you give us a preview?
[laughter]
>> NATALIE: They will be there when it launches. Can I give you a preview? I might be able to give you a preview. Let me see how our beta site is looking. [laughs]
>> AMBER: Everyone loves live demos. [laughs]
>> NATALIE: I know.
[crosstalk ]
[laughter]
Let me see. Trying to find a good browser window here. All right. Let me just take a peek at our beta site, make sure it’s not down or anything. I want to make sure that I can log into it.
I can answer another question while I’m doing this.
>> AMBER: OK. Simon asked: “Is it also useful to record effort to fix in addition to severity and priority?” Do you have any opinions on that?
>> NATALIE: Yes, you definitely could. You might decide that that’s one of the things that goes into determining your priority. You might make that part of your priority calculation, or you might track it separately. It’s up to you.
Some issues are super quick, like, a five-minute fix, and others are quite complicated, and might be a full day or even multiple days, so you might want to track that just to make it helpful, and maybe tackle all the quick issues first for some quick wins, and then come back later and plan out, you know, maybe a sprint just to fix a bigger accessibility issue that might have some issues.
>> AMBER: Yes. In my experience, sometimes it’s hard with that, you know, the effort level, because we often do it for entities that are not ourselves, right? If it’s your own team, it’s easy to know what the effort level is, but sometimes you think it’s low effort, and that person thinks it’s really high effort.
An example that came up on an audit for a community college, and we were doing audit and remediation, and we’re doing the hard stuff, like the code stuff, and we flagged that there were quite a few posts where they weren’t using the WordPress list block, and they were just typing it out with, like, a hyphen, or just typing the number one, and then hit “return.” So they were all in paragraphs and they weren’t actually in real lists, and we were just like, “Oh, this is a quick fix,” and then it came up that the content person literally didn’t know how to put things in a list and felt really overwhelmed when we were, like, “We’ll just put it on the list.” [chuckles] Right?
>> NATALIE: Right.
>> AMBER: And so I feel like that’s the one thing that is challenging about that effort, which is everyone’s effort is very different.
>> NATALIE: Yes, and if they’ve done that once, it’s just a few minutes to go fix it, but if they’ve written a hundred posts like that, [chuckles] it’s going to be harder to go back and fix.
>> AMBER: Yes. Do you want another question? Or are you…
>> NATALIE: Yes. Our beta is not in super great shape right now. I don’t have the right plugin installed on my test sites, and I don’t think I have anything super impressive to show. [laughs]
>> AMBER: That’s OK. That’s OK.
[laughter]
I know I put you on the spot.
[laughter]
Maureen was wondering if you could define what a functional issue is?
>> NATALIE: Oh, sure. The functional testing is testing through, like, a user flow, or a user story. Like, a user is going to put an item in their cart, and then they’re going to go to the “view cart” screen and then they’re going to go to the checkout screen and enter their payment information, and then they get a confirmation screen at the end, so that would be like a functional user flow, and we would want to test that from end to end. Can you, without using the mouse, get through that flow? Can you put an item in your cart? Can you get to the cart page? Can you enter your credit card and all of your payment and shipping information and get through to the confirmation page? Can you do that using a screen reader?
So a functionally issue would be that there’s some kind of problem in that flow, that there’s something that’s blocked or something that you’re not able to do with some kind of assistive technology or on certain different devices.
>> AMBER: Yes. Thank you.
>> NATALIE: Sure.
>> AMBER: Simon asked: “What do you do when sprints overlap with each other due to time constraints?”
>> NATALIE: Well, that is the most fun of all.
[laughter]
So in that case, you have to be mindful of any individual on the team. Are they involved in both sprints? Because if they are, you have to think about how you’re planning out work for them. They can’t get a full 40-hour work week on both sprints. [laughs] So you just have to kind of be mindful of who’s involved, and how much of their time is already taken up by the first sprint that’s wrapping up, and how much time will they realistically have left to work on the second sprint, and it is a balancing act to kind of try to figure out, like, should they spend half a week on each sprint? Should they just focus on that first one and get it out the door? And it’s going to depend on what the priorities are, on what the overall deadline is, you know, which way it goes, and sometimes when people have vacation schedule, [laughs] it’s going to come into it as well.
>> AMBER: Yes.
>> NATALIE: Yes, it’s a good balancing act and it’s a good exercise in figuring out how to pay attention to multiple priorities at the same time, and as with most things to do with web development and accessibility, the only real answer is always, “It depends.” “It depends on the situation.”
>> AMBER: Yes. I think we’ve found the same thing. If you only have one developer, it’s really hard, I think, to run multiple sprints at the same time, and so that’s really where I would say, if there’s only one person focused on fixing it, even if it’s not a developer, only one content person, you really do just have to prioritize at which point is, like, “Let’s just do this, and then when we’re done, do that,” and if this takes longer, it makes more sense to just push this down on the calendar instead of trying to start something new when you haven’t even finished the first thing.
>> NATALIE: Yes, yes. It can be really hard to do that, and some people get really overwhelmed by that, and don’t work well in that situation either. [chuckles]
>> AMBER: Yes. Lucy Candice [phonetic] said: “I’m struggling with how to quote a site audit without a ton of work.”
I’m guessing she might mean, like, spending a ton of time digging into how many problems already exist? Do you have any thoughts on that? Like, how you approach quoting them, or scoping, and even figuring out what the job is?
>> NATALIE: Yes. I know that can be really hard, because you feel like you have to do half the audit to figure out what the scope of the work is for the audit, and of course, you don’t want to have to work for the audit before you’ve even been paid.
I do find if you can talk to the client and get them to agree to maybe a smaller sprint or a series of smaller sprints that you could quote more accurately with a lot less work, that’s super helpful. If they won’t agree to that and they just want to know how much the whole thing is, sometimes you just have to take the best sample you can kind of grab without a lot of research and thought into it, run the automated testing on it to get a feel for, “Wow, how is this looking?” Is this above-average number of issues? Below-average number of issues?
Often, automated testing is an indication of how much more you’re going to find in a manual, and then you can go from there, but I think I’ve always had to quote, like, a range rather than a single price for what I think an audit is going to cost. Like, “It’s going to be probably at least this much, but no more than this much,” and it is really challenging to do without tons of work, and it still does take at least a couple of hours to come up with a reasonably accurate quote.
>> AMBER: Yes. We actually gave a talk earlier in the year. You can find it if you go to the recordings for anyone’s… Or maybe Paola can stick the link in there … About how we’ve been approaching. We do, like, audit, remediation, retainer-based projects. Mostly because we want to get away from what you were saying, which is an audit doesn’t fix anything. [chuckles] And so we’re, like, “Let’s find a few problems and fix them, and find a few problems and fix them.” Like that kind of thing.
I think we have the same problem, especially when you’re getting into remediation, where people are, like, “We’ll, how long will it take?” And it’s, like, “Well, I haven’t even audited your whole website, so telling you how long it’ll take to fix the whole thing is a little hard.”
Although, I do think what we have figured out is that we can give them a minimum idea, even just like by what you’re saying, like, we’ll typically use WAVE, and we have a modified version of our Accessibility Checker that allows us to scan a website that it’s not installed on, so we don’t even have to get access to their site to scan with our tool, and then we will do, like, a keyboard, like, we’ll just tab through the homepage and the nav menu, because that right there, like, if the whole navigation has to be rebuilt on a WordPress website, depending on how many hours they can budget in a month, that could be all their time just doing the nav menu.
>> NATALIE: Yes. Yes.
>> AMBER: So in that talk, we ended up being, like, “OK, we’re going to put minimum month commitments on these, because we know it’s not going to be less than this number of hours.”
I feel like if you’re just open with your clients about, you know, “Until I get in, I’m not really going to know, but the goal of the first month is that we’re going to have a better picture of what the subsequent months will look like”?
>> NATALIE: Of where we’re going, yes.
>> AMBER: Yes, and it’s like paid discovery for a website design project, right? Like, your clients have to pay you to do discovery before you can really fully finalize the quote for the whole build. I think it’s kind of the same thing.
>> NATALIE: Yes, and I think it’s helpful, too, to set the expectation with the client that accessibility is not, “We’re just going to do this one time and then we’re done forever.” [laughs]
>> AMBER: Yes.
>> NATALIE: This is, like, as long as you’re working on the website and making new content, you need to keep thinking about this.
>> AMBER: Yes. That was part of what motivated us to set these up, too, is like, there are subscription. People literally subscribed and it auto bills them every month. Now, some of the bigger institutions, they don’t pay with a credit card, right? Like, it’s still a PO, but it hits the PO and it’s the same price, but it’s the idea of trying to build this mindset that accessibility, it’s not, like, “Make it accessible and you’re all done.” But it’s something you should invest in, the same way you invest in SEO or web design or something like that.
>> NATALIE: Yes.
>> AMBER: Hopefully, that’s a helpful answer, Candice. If not, let me know. I did see she said in the chat she’s glad she’s not the only struggling with quoting on this. I think quoting can be very challenging on many fronts. [chuckles]
>> NATALIE: Yes.
>> AMBER: You know, [crosstalk ]
Yes, so we had an anonymous question that I don’t know if you know the answer to this one. It’s specific to the Contact Form 7 plugin. Do you know much about that plugin?
>> NATALIE: I know more about it than I wish I knew. [laughs]
>> AMBER: OK, awesome, so this person says, “Is there a way to apply Aria labels on Contact Form 7?
>> NATALIE: Oh, that’s a good question. I actually don’t know the answer to that off the top of my head, so I [crosstalk ] feel like I maybe have, but I’m not 100% sure. I’m sorry. Go ahead.
>> AMBER: Well, what I was going to say was, I haven’t used Contact Form 7 for many years, but when I last looked at it, I thought that it gave you a field where you select things and then it spits out HTML?
>> NATALIE: Yes.
>> AMBER: And so if that is the way it still works, I think the answer is yes, because you can just type in the HTML.
I would advise you to pick a different form plugin.
>> NATALIE: I would too, but if you must use Contact Form 7, there is a plugin called Accessible Defaults that you have to install before you make any forms in Contact Form 7. [laughs] It’s like accessible [crosstalk ] defaults for Contact Form 7.
Yes. It’s super nice, but if you’ve already got your form set up, it’s not that helpful. You have to install it before you make any forms, but it is helpful for getting your form to be accessible by default, because you do have to do a good amount of coding work on a Contact Form 7 form to make it accessible.
>> AMBER: I was trying to see if I could find it real quick. I found the “WordPress.com” page, but I wanted to find the WordPress… Oh, wait, if I click on that now, I should be able to get it too.
Nope, somewhere there’s the… There we go. I can put that in the chat for anyone who is interested.
>> NATALIE: Yes.
>> AMBER: There we go. OK.
So Eagle [phonetic] asked about links to different tools. Do you want to put your AAArdvark link in the chat for people? We’ll make sure that we get those in the notes as well.
There have been a few that Paola has been putting in. I know that she did put in a link to Axe and ANDI, and WAVE is just “Wave.webaim.org,” but we can also put that in the chat as well.
Gerson, I noticed, just commented that Accessible Defaults has not been updated for two years. I think it still works.
>> NATALIE: Yes. In my experience, it still works.
>> AMBER: Sometimes they don’t actually need to be updated. [chuckles] And this one’s from Joe Dolson, who is on the WordPress Core Accessibility Team, so I would assume that it is good. He wouldn’t let it languish if it wasn’t.
>> NATALIE: Yes.
>> AMBER: Simon had a follow-up on the effort to “fix” question: “Do you have a formula to calculate priority? Or is there some subjectivity to assigning priority?”
>> NATALIE: My answer to that is both, actually. I’ve been working out the formula that we’re going to use in AAArdvark, and figuring out how to take the severity and the number of pages the issue occurs on, and then in the future, an integration with Google Analytics that’ll let us get traffic volume for different pages, and to kind of take all of those things into account and have a formula that will calculate a priority, but of course, there’s always exceptions to what you can calculate automatically, and there might be an upcoming promotion or something that’s running that’s going to bump a certain page or section of the site up in priority, where you would have to manually go in and change it.
So you can do a formula to calculate it based on whatever things are important to you, time to fix, severity, number of occurrences, et cetera, but then you’re going to want some wiggle room in that to adjust that as needed for different situations that the formula obviously doesn’t know about.
>> AMBER: Yes. I know for us, like on the severity range, we… I’ve seen some companies where they do have, like, five different severities, and I’m, like, it’s too many.
Critical, like, really high is, it’s breaking, like, there’s no ability for someone with disabilities, and we have medium, which is, it’s not breaking. There’s maybe obvious workarounds or it’s a really minor piece, but it’s still technically a WCAG failure and therefore might file Section 508, or AODA in Ontario, or something like that.
Then we have low, which is best practice, but not clearly a failure of anything, and that’s all we do. It’s either it breaks something or it doesn’t, [chuckles] you know?
>> NATALIE: Yes. [laughs] That’s true. [laughs]
>> AMBER: Yes, but I like what you’re saying about trying to mathematically calculate in with, like, analytics page views to save the site owner from even having to think about that.
>> NATALIE: Yes, because you automatically know now which are the pages with the most traffic; and that, of course, changes over time, so you want to keep up with it.
>> AMBER: Yes. There were some comments and questions about the countdown timer on the AAArdvark website says 15 days until launch.
>> NATALIE: Yes.
>> AMBER: You have a captive audience right now. Everyone’s really excited, [laughter] which is fun, because I didn’t even know, so I’m, like, “This is neat too.” Is that like a beta? What’s coming in 15 days?
>> NATALIE: Yes. Yes, like a beta. I guess we could call it our minimum viable product. It’s what we’ll be releasing in 15 days, and we are planning to run a promotion at launch, where you could get a free account for one website, if you want to sign up and take advantage of that, but yes, we are looking at launching by November 7th or so. Yes.
>> AMBER: Very cool and exciting, someone else said: “I struggle to find accessibility instructions for a specific components, like how voiceover should read a carousel, or how to define Aria label for specific components. Do you have any good documentation sites or anything that tackles best practices for UI components?”
>> NATALIE: Yes, I do, actually. Haydon Pickering has a website and eBook called “Inclusive Components,” that has some really nice skeleton code of all different types of components and how they should be set up, and which attributes should be there, and how they should be read out by screen reader, and it kind of talks you through and lists all those things out, so that’s one of my favorite ones.
>> AMBER: And that’s “InclusiveComponents.design,” right?
>> NATALIE: Oh, yes. Dot design, yes. I knew it had a weird TLD. [laughs] Thank you, so that’s been a really helpful resource.
Then the W3C actually has a lot of pretty decent examples, actually, of different types of components and how they should be marked up, and they have, like, this magic link that will open it in a CodePen for you, so you can get a working example and kind of play with it and try changing things and then have a screen reader walk through it, so I’ve actually found that to be super helpful, and it’s just part of the same website that the WCAG guidelines are on.
>> AMBER: Yes. The other one that we look at a lot is the USWDS. Do you use that? [crosstalk ]
>> NATALIE: No, I haven’t used that one before.
>> AMBER: Let me see if I can pull it up real quick and put it in. It’s, “DesignSystem.digital.gov.” So this is the design system for federal websites in the United States. It doesn’t have everything, like, they don’t have carousels, but that’s because they don’t allow carousels to be used on their websites, but it has accordions, and it has, like, cards, and other kind of components, so that’s one that we’ve referenced quite a bit, and if you are building a website that’s federally funded, then you have to use that.
>> NATALIE: Yes. [laughs]
>> AMBER: I think I saw that there was one in the UK also, but I’m forgetting what the web address is for that.
>> NATALIE: Yes, you’re right. I have seen it and I don’t remember what it is either. It’s the same thing, the design library for government websites.
>> AMBER: In the UK, yes.
Gary [phonetic] said: “Do you have any advice in motivating open-source tools, tool teams in implementing accessibility recommendations?”
>> NATALIE: Oh, goodness. [laughs] So yes.
>> AMBER: Call them out on social media. I don’t know.
[laughter]
No, no, no. You probably have a much better answer than that.
[laughter]
>> NATALIE: So I have been involved with the WordPress Accessibility Team, although for the last year, I’ve been working really hard on AAArdvark and haven’t had a lot of time, but I have been involved with the accessibility team for WordPress for quite some time, and it can be really challenging to have your voice heard, and different communities are more and less open to it, so it kind of depends on which open-source product you’re talking about as well.
I have found that giving talks at conferences has been really helpful to get some attention and to help people understand what the issue is. I found, a lot of times with developers, a lot of developers make an assumption that people who can’t access the websites that they’ve built, for whatever reason, just need better tools, like, they just need better assistive technology, and that there’s nothing for the developer to actually do, so to help them understand how the choices they’re making in their development process are impacting other people and how they could make different choices that don’t actually take that much extra time and have a different outcome I think has been really helpful and to start to win people over that way.
Then just being active in the communities and asking for it over and over again, and I know it feels like it’s not working and you’re not getting anywhere, but it does actually work over time, and I think WordPress is more accessible at this point. I know it’s far from perfect, but I think it’s more accessible at this point than it would have been if the accessibility team wasn’t there, advocating and making presentations and getting attention for the cause.
>> AMBER: Yes. I think one of the things that has really helped with some of the plugins in WordPress… It’s really easy for those of us that know accessibility to try out a plugin and be, like, “Oh, it’s not accessible,” and just abandon it and never communicate that to anyone, because we’re just moving fast. We’re looking for something, we’ll be like, “Oh, this doesn’t work. I’m not going to use it.” Because it does take time to report things, but honestly, this year, that was one of my goals, which was that I was going to try and provide more feedback to plugin developers, because a lot of times, they just don’t know.
>> NATALIE: Yes, they don’t know. They have no idea.
>> AMBER: I have found that I’m getting positive response, but I think if you want to see that positive response and that motivation, it’s really helpful that you write really good… Like, if it’s on the support forums on “WordPress.org” or on a public GitHub repo, you have to really write a detailed explanation of what the problem is and how to fix it.
So some of these component libraries that we talked about, you know, if I am trying to plugin and it has an accordion that’s not accessible, I’ll be, like, “Your thing is missing this kind of Aria,” and I’ll literally list out, you know, like, “It doesn’t have Aria-expanded on it.” Or, “It’s not using buttons,” and I’ll list out what all the problems are, and then I’ll say, “Here’s where you can go see an example,” and I’ll link them over to the USWDS.
I found that putting that extra time in, if I can… Like, I try to do it, at least one or two a week, just to try and get it out there, but if you give a really good feedback, I think most developers are open to it and will respond positively.
I’ve seen somewhere, where we flagged something and the developer fixed it, like, within 24 hours, so it’s just amazing to me when it’s a free plugin on “WordPress.org” that they’re not getting paid for.
>> NATALIE: I’ve had that experience too, where they fixed it, like, right away, and it was super nice.
I think on the flip side of that, too, is to leave positive reviews for the people who are accessible. [laughs] And so I try to do that, too, to just say, “Hey, thank you so much for building an accessible product. I really appreciate it.” [laughs]
>> AMBER: Yes. I do that. I’ll leave them a five star review. Or the other thing is, for some of the plugins we use all the time, if I monitor their change logs or I’m on their email list, so I get notified when they’re updating, like Advanced Custom Fields, for example, because I’m like, “I want to know what’s happening in that, because we use it on all of our websites..,” and if I notice, in their change log, that they did an accessibility thing, I’ll frequently go on Twitter and I’ll tweet and tag them and be, like, “Hey, shout out to Advanced Custom Fields for making their settings page function with a keyboard.” That was a big thing that they did earlier this year.
I think positive reinforcement works, right? We talk about it with our children too. If all we do is shout at the people who are doing a bad job, I think it gives accessibility this negative kind of persona, and so it’s good to have a positive on it too, which is, like, “Hey, you’re doing a good job.” “Look at this other company.” You know, and then maybe that plugin’s competitors will be, like, “Well, crap, I’m not getting awesome shout-outs.” [laughter] So I need to do this too, right?
>> NATALIE: Yes. Fingers crossed.
>> AMBER: That’s the hope.
Miranda [phonetic] said: “Is there a good sample summary report I can reference?”
Miranda has logged accessibility bugs, but has never summarized it or written a report.
>> NATALIE: Oh, interesting. I can’t think of one off the top of my head besides the horrible ACR/VPAT, which is not helpful to anyone. [inaudible].
>> AMBER: Well, I don’t know if those are horrible.
[laughter]
For anyone who’s not familiar, can you describe what an ACR, Accessibility Conformance Report, and a VPAT is? And then maybe we can throw a link?
>> NATALIE: Yes. VPAT is Voluntary Product Accessibility Template. It’s the template you use to produce a report called an ACR, an Accessibility Conformance Report? Is the “C” conformance or compliance? [laughs]
>> AMBER: I think it’s “conformance.”
>> NATALIE: Yes, so it’s a standard format for reporting the state of accessibility in a product, which could be a website, but it could also be a desktop software, like Microsoft Word, and it just has you step through kind of one WCAG guideline or success criterion at a time and says if the product conform, not conform, or conform with exceptions. [laughs] And then you have to list out what the exceptions are.
>> AMBER: Yes. Paola put two links on the Section 508 website that explains to it. I’ll put a link to the VPAT for Accessibility Checker, so it’s a software; it’s not for a website, but if anyone wants to look at that format, you can.
The other thing that stood out to me, and I’m trying to see if I can find it real quick, as a good example of a written report that’s not an ACR was the Tenons [phonetic], which Tenon doesn’t exist anymore because they got purchased and kind of went away, but Tenon and Carl Groves, they were commissioned to do an accessibility audit of Gutenberg, and that report, everything in it has pretty much been fixed, but that report was more of a narrative style, and I think I can find it. Just giving me one second. [crosstalk ] another example.
>> NATALIE: Yes, that’s a good one.
>> AMBER: I’m on the blog post. Oh, here it is. OK, so this blog post on the WPCampus website? So WPCampus, if people aren’t familiar, is a nonprofit organization that is a WordPress in Higher Ed organization. It’s super awesome, and there’s a lot of accessibility conversations there. They were actually the ones who commissioned this.
So if you go down under the results of the accessibility audit heading, there is a 34-page executive summary which is a PDF, and if you click that, you can open that, and then there’s other documents here that talk about like their approach to the audit and things? So I think if you’re not familiar with writing audit reports, this actually might be a really interesting one to reference.
The narrative style, this is Tenon’s format. It’s not like it’s an official format. [crosstalk ].
>> NATALIE: Yes, it’s not a standard format.
>> AMBER: Yes. I mean, anyone can do what they want. There’s things in here that are similar to what you said, right? Like severity or that sort of stuff? So I think every company and every auditor has to decide what they want their format to be, but I think looking at this might be an interesting look, if you’ve never seen what other companies are doing, or if you’re trying to think about what your own format should be.
>> NATALIE: Yes.
>> AMBER: I don’t know of any other public ones like this. Do you?
>> NATALIE: Yes, I know. That’s why I was, like, “I’m having a hard time thinking of one.” [crosstalk ] if Drupal doesn’t have something, but I don’t know of it specifically.
>> AMBER: Yes. The other thought that I had, which I don’t have a link to one off the top of my head, but I feel like there are some colleges that have gone under a mandated remediation with the Office of Civil Rights as part of the Department of Education, and they usually have to publish their audits because they’re public record.
>> NATALIE: Nice.
>> AMBER: So I feel like if you’re a good Googler, you could probably find one or two of those for, like, a college website, and then that might give you some other formats, maybe.
Gerson asked: “Is the way of calculating severity and priority…” I think this is referencing what you had previously talked about with maybe using Google Analytics and things like that. “…Is that going to be available in the AAArdvark tool?”
>> NATALIE: It will be. Not at launch, but shortly after launch. [chuckles]
>> AMBER: I totally understand about that.
[laughter]
>> NATALIE: We had to do a 1.0 and draw a line in the sand. “This is in 1.0. This is after.”
>> AMBER: Yes, and then the last question that I think I have… Unless anyone else wants to add any.., and this is more of a technical question, but I’m going to answer it verbally so everyone has it.
Doug had asked about turning on the ability to save chat in Zoom, and this is a limitation of Zoom webinars. In Meetings, participants can save, but unfortunately, in Zoom webinars, we can’t, and I’ve spent a bunch of time digging into all the Zoom settings, and it can’t be done, but all of the links that we’ve put here will be in the recap.
Paola very awesomely goes through the chat log from the recording and pulls them out and puts them in a bulleted list, so once the recap goes out, you will have all of these links, if you don’t want to have to try and frantically click them all right now.
[laughter]
>> AMBER: I think that’s it for questions.
Natalie, can you tell people how they can get a hold of you if they want to follow up?
>> NATALIE: Oh, sure. Yes. Can I share my screen again real quick?
>> AMBER: Yes. Let me know if you can’t.
>> NATALIE: [chuckles] OK. I do have a last slide here with my email address, which is kind of a mouthful: “Natalie.maclees.nsquared.io.”
I don’t really use social media much anymore. [laughs] And then, of course, the website, “AaardvarkAccessibility.com.”
Once it’s launched, we’ll have contact information on it. I know right now is just the countdown.
>> AMBER: Awesome. Thank you so much. This has been really fun, and I know that you probably have at least 50 new people who have all gone to join and get notified when your tool becomes available. [laughs]
>> NATALIE: Oh good. I’m glad. I’m excited.
[laughter]
All right.
>> AMBER: Thank you everyone for attending. The recording will be available in about two weeks.
We’re going to do the slightly awkward thing where we say goodbye, but we sit for a minute, and I watch to make sure all of our captions go through, and then I will end the webinar. [chuckles]
[laughter]
So thanks so much, Natalie.
>> NATALIE: Thank you.
Links Mentioned
- Aaardvark Accessibility
- Axe Accessibility Testing Tools
- ANDI Accessibility Testing Tool
- WAVE Web Accessibility Evaluation Tool
- Grow Revenue with Accessibility Monitoring & Remediation Plans: Amber Hinds
- AI Alt Text Generator
- Every Alt
- EveryAlt Test Results vs AltText.ai
- Contact Form 7: Accessible Defaults
- Inclusive Components
- Patterns on W3
- U.S. Web Design System
- U.K. Design System
- Voluntary Product Accessibility Template (VPAT®)
- How to Create an Accessibility Conformance Report Using A VPAT®
- Is there an Accessibility Conformance Report (ACR) or VPAT available for Accessibility Checker?
- WPCampus releases results of the Gutenberg accessibility audit
About the Meetup
Learn more about WordPress Accessibility Meetup.
Article continued below.
Stay on top of web accessibility news and best practices.
Join our email list to get notified of changes to website accessibility laws, WordPress accessibility resources, and accessibility webinar invitations in your inbox.
Summarized Session Information
The session on conducting accessibility audits for large websites delves into a comprehensive methodology designed to tackle the inherent complexities of evaluating extensive digital landscapes for accessibility compliance. It underscores the necessity of a strategic, layered approach to auditing, highlighting the importance of selecting test subjects based on traffic, automated issue identification, key templates, and the significance of various content types. The session elucidates the critical role of breaking down the audit into manageable sprints, leveraging agile methodologies to ensure progress without overwhelming the team or letting issues become obsolete.
A significant focus is placed on optimizing the testing process by integrating automated, manual, and functional testing. Automated testing serves as a foundational scan, identifying a portion of the potential issues, which are then explored in depth through manual and functional testing to ensure a thorough audit. The methodology advocates for a centralized tracking system for issue documentation, prioritization by severity and impact, and a structured remediation and re-testing workflow to manage and address the identified issues efficiently.
The culmination of the audit process is the delivery of an actionable, comprehensible report to the client and remediation team, consolidating findings and guiding the remediation process. This ensures that the audit’s results are not only understandable but also actionable, facilitating effective resolution of accessibility barriers. The session wraps up by reiterating the importance of a structured, strategic approach to accessibility audits, emphasizing continuous improvement and a commitment to creating an inclusive and accessible web environment for all users.
Session Outline
- How to select what to test
- How to break up big projects
- How to optimize testing
- How to track remediation and re-testing
- Wrapping it all up
How to select what to test
When embarking on accessibility audits for large websites, it’s crucial to understand that the audit itself doesn’t directly fix the issues; it merely identifies areas where accessibility improvements are needed. The goal is to produce a clear and actionable report for clients or remediation teams, regardless of the website’s size.
Starting with the site map
The process begins with the site map, which is essential for WordPress websites and generally available by default from WordPress 5.5 onwards. If a site map is not available, it can be generated through crawling the site, often using third-party SEO tools. These tools, although primarily for SEO, serve well in creating a comprehensive list of URLs, including pages, PDFs, documents, spreadsheets, presentations, and multimedia content that need accessibility testing.
Automated testing
Automated testing is the next step and is less restrictive due to its minimal manual effort requirement. It’s not practical to test every page as it may result in redundant findings, especially in sites like WooCommerce, with numerous products sharing the same template.
The strategy involves selecting a representative sample of various templates and page types for automated testing, ideally two to three times the number selected for manual testing.
Manual testing and prioritization
Manual testing, being time-intensive, requires careful planning and prioritization. The prioritization criteria include:
- Traffic analysis to focus on the most visited pages.
- Automated test results to identify pages with a significant number of issues.
- Key templates and conditional components to ensure a variety of content types and functionalities are tested.
- Important content encompasses the homepage, navigation elements, accessibility statements, contact information, help guides, legal pages, and user settings and preferences.
Functional testing
Functional testing targets interactive tools and transactions on the website, including cart and checkout processes, appointment booking, modal dialogs, carousels, tabs, accordions, forms, login/logout mechanisms, commenting systems, and embedded content like social media feeds and videos. This step ensures that all interactive elements are accessible to users, paying special attention to the management of focus, keyboard navigability, and screen reader compatibility.
Homepage
The homepage is often the first point of contact for users with a website. It’s a central hub for navigation and information dissemination, making its accessibility paramount. Given its high traffic, ensuring the homepage is free from accessibility barriers is crucial for providing a positive initial experience for all users. The audit process should ensure that the homepage is navigable, content is easily readable, and all interactive elements are accessible.
Navigation
Navigation is the backbone of user experience on any website. This includes not only the main navigation menu but also footer links, sidebars, and any other means by which users move through the site. Ensuring these elements are accessible is vital for allowing users to explore the website fully. This means testing for keyboard navigability, screen reader compatibility, and clear labeling of links and buttons.
Accessibility statement
The accessibility statement is a critical document that outlines a website’s commitment to accessibility and provides information on accessibility features, limitations, and how users with disabilities can request accommodations or report issues.
Contact information
Access to contact information is essential for all users, especially for those who might encounter barriers elsewhere on the site and need to reach out for support or information. This includes testing the accessibility of email addresses, phone numbers, contact forms, and any other communication methods provided on the site. Users should be able to access and interact with these elements easily, ensuring they have the means to communicate with the organization.
Instructional and legal content
Guides, help articles, terms of service, privacy policies, and other legal or instructional content are foundational for user understanding and legal compliance. Ensuring these pages are accessible is important not only for user engagement but also for transparency and trust. Users rely on this information to understand how to use the site effectively and to be informed about their rights and the site’s policies regarding their data and privacy.
User settings and preferences
Many websites offer customization options for users, such as changing passwords, selecting preferences for site appearance (e.g., dark mode), and managing account settings. Ensuring these functionalities are accessible is crucial for allowing all users, regardless of their abilities, to personalize their experience on the site. This includes making sure that forms, buttons, and other interactive elements in these areas are fully accessible.
Prepare for functional testing
Given the time-intensive nature of functional testing, careful planning is essential to prioritize and effectively test these components.
Interactive tools and transactions form the core of functional testing. This includes a wide range of elements such as:
- Cart and checkout functionality: ensuring the entire purchasing process, from selection to payment, is accessible. This includes navigating the cart, modifying order quantities, applying discounts, and completing payment information.
- Appointment booking: verifying users can browse available times, select a preferred slot, fill in necessary details, and confirm bookings without encountering accessibility barriers.
- Modals: testing modals for proper focus management, ensuring that focus is trapped within the modal while it’s open and that users can close it using the “escape” key, enhancing the experience for keyboard and screen reader users.
- Carousels: given their complexity, especially for screen reader users, carousels require detailed testing to ensure they are accessible. This includes checking for the ability to pause auto-scrolling, navigate between items using keyboard controls, and ensure that content within carousels is fully accessible.
- Tabs and accordions: these elements must be tested to ensure they operate correctly with keyboard controls and are properly announced by screen readers, allowing users to access all content without barriers.
- Forms: all forms, including contact forms, sign-up sheets, feature request forms, etc., need thorough testing. This ensures that fields are labeled correctly, error messages are accessible, and users can complete and submit forms without accessibility issues.
- Login/logout processes: essential actions like logging in and out must be accessible to all users, ensuring they can easily access their accounts and sign out.
- Commenting systems: if the site includes commenting functionality, it’s vital to test these systems for accessibility. This includes not only the ability to post comments but also to navigate through existing comments and interact with them, such as liking or replying.
- Embedded content: testing is required for any embedded content, such as social media feeds, YouTube or Vimeo videos, and third-party forms. This ensures that these integrations do not introduce accessibility barriers, allowing all users to engage with the content.
- Plugins for enhanced interactivity: caution is advised when integrating plugins that add functionality like favoriting comments or posts, as many may not be accessible out of the box. Selecting plugins that adhere to accessibility standards is crucial to maintaining an inclusive user experience.
Functional testing is about verifying the accessibility of every interactive element on a website to ensure that users with disabilities can engage with content, complete transactions, and navigate the site as seamlessly as non-disabled users.
How to break up big projects
Addressing accessibility in large-scale projects necessitates a strategic approach to ensure efficiency and effectiveness. The complexity of auditing and remediating hundreds or even thousands of pages can be daunting, especially when faced with dynamic content and limited development resources.
Embracing agile methodology
The key to handling vast accessibility audits is to think in terms of sprints, a core concept of the agile methodology. This approach breaks down the monumental task of auditing and remediating an entire website into manageable phases. Each sprint focuses on a specific subset of pages or issues, allowing for rapid assessment, remediation, and retesting. This method not only streamlines the process but also ensures that the audit findings remain relevant and actionable throughout the project lifecycle.
The idealistic view of a sprint involves conducting an audit, having developers remediate the identified issues, and then moving on to the next batch of pages with the assumption that everything is fixed. However, the reality often involves a cyclical process where initial fixes might uncover new issues, or the implemented solutions might not fully address the original problems.
Acknowledging the iterative nature of accessibility work from the outset allows for better planning and resource allocation. It’s important to set realistic expectations with all stakeholders, including the timeline for audits, the scope of each sprint, and the potential need for multiple iterations to address complex or persistent issues. This approach also helps manage the workload for development teams, particularly when resources are limited, by prioritizing fixes that will have the most significant impact on accessibility.
Questions to ponder
How big is the remediation team? Understanding the capacity and bandwidth of the team assigned to remediation tasks is crucial. A small or part-time team will necessitate a more phased approach to avoid overwhelming team members with an unmanageable workload. This awareness helps in setting realistic goals and timelines for the project.
How much accessibility experience? The level of familiarity and expertise the team has with accessibility principles will significantly influence the project’s flow. Teams new to accessibility may require additional support, training, and guidance, impacting the time and resources needed for remediation. Knowing this can help in allocating the right amount of support and potentially scheduling training sessions to build the team’s capabilities.
How often does content change? The dynamics of the website’s content—how often it changes and the nature of these changes—play a vital role in planning the audit. If the website undergoes frequent updates, especially with temporary content like landing pages for short-term campaigns, it’s important to prioritize permanent or core pages to ensure the audit’s findings remain relevant and actionable.
What can we shift left? Identifying opportunities to integrate accessibility earlier in the content creation and development process can significantly reduce the occurrence of new issues. This might involve providing tools, resources, or training for teams regularly producing new content, such as an SEO team working on landing pages. By addressing accessibility proactively, you can reduce the need for extensive testing and remediation at later stages, making the entire process more efficient.
Reflecting on these questions allows project managers and auditors to structure their approach to big accessibility projects more effectively. It emphasizes the importance of understanding the project’s scope, the team’s capabilities, and the website’s dynamics.
Examples of sprints
Breaking down a large accessibility project into sprints allows for focused, manageable tasks that can significantly enhance the effectiveness of the remediation process. When dealing with teams that may not have extensive experience in accessibility, choosing strategic starting points and specific areas of focus can help in gradually building the team’s skills and confidence in handling accessibility issues.
Here are some illustrative examples of how sprints can be structured:
- Homepage sprint: given its importance as the first point of contact for many users, the homepage can constitute a sprint, especially if it’s lengthy or complex. This sprint would involve auditing and remediating accessibility issues on the homepage, setting a precedent for the quality and accessibility standards for the rest of the site.
- Contact information sprint: accessibility issues related to contact information and forms are critical for ensuring all users can communicate with the organization. This sprint could focus on the footer’s contact details, standalone contact pages, or department-specific contact pages, addressing form accessibility, ease of navigation, and clear communication channels.
- E-commerce flow sprint: for e-commerce sites, ensuring the accessibility of the cart and checkout process is paramount. This sprint would focus on making sure that users can navigate through product selections, understand their cart contents, and complete purchases without accessibility barriers.
- Navigation sprint: this sprint would focus on the site’s navigational elements, such as menus, breadcrumbs, and search functionality. Ensuring these are accessible is crucial for allowing users to find their way around the site efficiently.
- Product pages sprint: specifically for e-commerce sites, a sprint could be dedicated to auditing and remediating product pages. This would ensure that product descriptions, images, and purchasing options are accessible to all users.
- Landing pages sprint: tailored towards marketing efforts, this sprint would address accessibility issues on landing pages created by SEO teams or for specific campaigns. The focus would be ensuring these pages are inclusive, especially given their role in converting visitors.
- Blog posts sprint: a sprint focusing on the accessibility of blog content ensures that informational resources are accessible. This includes testing for text readability, image alt text, and navigation within and between posts.
- Legal pages sprint: ensuring that terms of service, privacy policies, and other legal documents are accessible is the focus of this sprint. These pages are crucial for trust and compliance, making their accessibility non-negotiable.
How to optimize testing
Utilizing a combination of automated testing, manual testing, and functional testing can enhance the effectiveness of audits. Each testing type plays a unique role in the comprehensive evaluation of a site’s accessibility, providing a layered approach to uncovering and rectifying issues.
Leveraging automated testing
Automated testing serves as a preliminary scan, capable of detecting 25% to 30% of potential accessibility issues. While its scope is limited, it provides invaluable guidance for subsequent manual and functional testing phases. Automated tools can swiftly identify common issues like missing alt text on images or unlabeled form fields. However, they fall short in assessing the relevance of alt text or the helpfulness of form labels, necessitating further manual evaluation.
Automated tests excel at highlighting areas or components that may present problems, streamlining the focus for deeper manual and functional analyses.
Integrating manual and functional testing
Following automated testing, manual and functional testing delves into the nuances that automated tools cannot discern. This includes evaluating the appropriateness of alt text, the descriptiveness of form labels, and the application of alternative labeling methods, such as the use of aria labels.
These testing stages are critical for a thorough audit, ensuring not just the presence but the quality and effectiveness of accessibility features.
Centralizing issue documentation
Efficiency in testing also relies on systematic documentation. Consolidating all identified issues into a single, accessible document facilitates a clear, actionable path for remediation teams. This approach aids in the organization and prioritization of fixes, making the audit’s findings both understandable and actionable
Prioritizing issues by severity and impact
Identifying issues is only part of the optimization process; equally important is the prioritization of these issues based on severity and impact. Critical issues that block access to content for specific user groups generally take precedence. However, prioritization is not solely determined by severity.
The frequency of an issue and its impact on site traffic also play crucial roles. For instance, a moderate issue appearing frequently on high-traffic pages may be prioritized over a serious issue that is isolated to a less visited area of the site. Understanding the scope and effect of each issue enables more strategic decision-making in the allocation of remediation efforts.
How to track remediation and re-testing
Efficiently managing the remediation and re-testing process is crucial for ensuring that accessibility issues identified during audits are resolved effectively. This involves establishing a system that allows for the tracking of each issue’s status, from initial identification through to successful remediation and verification. Here are key strategies for setting up a system.
Centralizing issue tracking
The foundation of an effective remediation process is a centralized system where all identified issues can be logged, tracked, and updated. While spreadsheets are a common tool for recording issues due to their simplicity and accessibility, they often fall short in supporting dynamic communication and detailed tracking of the remediation progress. Instead, employing a dedicated issue tracker or project management software can offer a more robust solution.
Structuring the tracking system
A well-organized issue tracker should allow for the categorization of issues by severity, type (automated, manual, functional), and status (open, in progress, resolved, needs re-testing). It should also facilitate assignment tracking, where each issue is assigned to a responsible developer or team for remediation. Essential features include:
- Assignee Column: Indicates who is responsible for addressing the issue.
- Status Tracking: Reflects the current state of the issue, including whether it has been resolved, is currently being worked on, or needs further attention.
- Commenting Capability: Enables back-and-forth communication between testers and developers, allowing for clarification of issues, discussion of potential solutions, and updates on progress.
Facilitating communication and collaboration
Effective remediation relies on clear and ongoing communication between all stakeholders involved in the accessibility improvement process. An issue tracker should support this by allowing users to comment on issues, attach files (such as screenshots or code snippets), and update statuses in real-time.
Importing data from testing tools
To streamline the integration of issues identified through automated, manual, and functional testing, the chosen tracking system should support the importation of data from various formats, such as CSV or JSON files. This capability allows for a seamless transition of issues from testing tools into the tracker, ensuring that all issues are accounted for and can be acted upon promptly.
Managing remediation workflow
Once issues are logged in the tracking system, the workflow typically follows a pattern of assignment, remediation, re-testing, and closure. Each issue should be periodically reviewed to ensure progress is being made.
Re-testing is a critical step after remediation efforts, as it verifies whether the fixes have successfully addressed the identified accessibility barriers. If an issue is not resolved in the first attempt, it should be reassigned for further work, with detailed feedback provided to guide the next round of remediation.
Ensuring manageability and clarity
The ultimate goal of the tracking system is to make the results of the audit manageable and actionable. By maintaining organization and clarity, the system helps prevent the remediation process from becoming overwhelming for developers and ensures that efforts are focused and effective.
Wrapping it all up
In summarizing the comprehensive approach to conducting accessibility audits for large websites, we’ve navigated through a series of strategic steps designed to tackle the inherent challenges effectively. These steps ensure that audits are thorough and manageable and lead to meaningful improvements in website accessibility. Here’s a quick recap of the key points discussed:
Selecting what to test
The initial challenge involves deciding which parts of the website to audit. This decision is influenced by several factors:
- Traffic analysis: prioritizing pages with higher traffic ensures that the most visited parts of the site are accessible.
- Automated issue identification: using automated tools to pinpoint areas with significant issues can help focus manual and functional testing efforts.
- Key templates and conditional components: identifying and testing the main templates used across the site, including any conditional content that may appear under specific circumstances, ensure comprehensive coverage.
- Importance of pages: prioritizing essential content, such as the homepage, contact information, and legal pages, ensures that critical information is accessible to all users.
Breaking the audit into sprints
To prevent the audit from becoming overwhelming and to avoid outdated issues, the audit process is divided into manageable sprints. Each sprint focuses on a specific area or aspect of the site, allowing for targeted testing and remediation within a concise timeframe. This approach facilitates continuous progress and prevents the accumulation of obsolete issues.
Integrating different testing methods
The audit leverages a combination of automated, manual, and functional testing to create a holistic view of the site’s accessibility. Automated testing provides a broad overview of potential issues, while manual and functional testing delve deeper into the nuances of user experience, ensuring that all types of accessibility barriers are identified and addressed.
Delivering an actionable report
The culmination of the audit is the delivery of a comprehensive yet understandable report to the client and remediation team. This report consolidates findings from all testing phases into a single document, categorizing issues by severity and priority and providing clear guidance for remediation. The goal is to present a clear, actionable plan that facilitates efficient issue resolution and enhances the site’s overall accessibility.
By adhering to these strategic steps, accessibility audits can be conducted efficiently and effectively, ensuring that large websites not only comply with accessibility standards but also offer an inclusive and equitable experience to all users. This structured approach to auditing, testing, and reporting facilitates ongoing improvements, demonstrating a commitment to accessibility and user-centric design.