Accessibility Checker version 1.21.0 corrected some of the underlying calculations displayed in Accessibility Checker Pro’s full-site summary report and the Audit History table. This post outlines the changes that were made, the reasoning behind those changes, and what you can expect to see in Accessibility Checker reports after upgrading to version 1.21.0.
Why Calculations Were Changed
As you might expect, we use Accessibility Checker in our remediation work. Accessibility Checker started as a way to “scratch our own itch” when we couldn’t find an automated accessibility testing tool that worked well in a WordPress environment and was affordable for organizations with tight marketing and website budgets.
Because we use the software in our daily work, we can test it on real-world websites and in remediation scenarios where we closely monitor changes over time. This results in a better view of necessary improvements to the software than testing solely on fake or static websites.
Recently, during several of our remediation projects, we realized that the numbers reported on the Accessibility Checker full-site report were inaccurate or misleading as the website improved. Because these numbers are used to report status to website stakeholders, we know how important it is that they are accurate and present a complete picture of the website.
The 1.21.0 version of Accessibility Checker corrects these numbers to present the most accurate picture of site accessibility status.
What Changed
Accessibility Checker version 1.21.0 includes the following changes:
URLs Scanned
This number was incorrectly counting URLs in the table where we store accessibility issues. This means that the number actually reflected the total number of posts with accessibility issues on them, not the number of posts that Accessibility Checker has scanned.
As posts were remediated and all issues were resolved, this number went down, falsely making it appear that Accessibility Checker was not scanning all content in the post types set to be scanned.
This number was corrected to count URLs with a stored meta value indicating that Accessibility Checker had scanned them. This is now an accurate picture of the number of URLs that Accessibility Checker has scanned.
Average Issues Per Page
Average Issues Per Page was previously calculated as the average number of issues found in each post when issues were present. The number did not include any posts with 100% Passed Tests (or zero issues), which artificially raised the value for average issues per page.
After discussions with customers and our internal team, we determined that excluding pages with zero issues does not make sense in a report summarizing the entire site’s status. The prior approach to this calculation did not present an actual average and made the site look worse than it was.
This average was changed to include “perfect score” pages with zero issues.
Average Issue Density
Average Issue Density was, like Average Issues Per Page, calculating an issue density only for posts with issues and not across the entire website. It also, on some websites, incorrectly queried issue density more than once per post.
Average Issue Density was changed to calculate a correct average of issue densities, including on posts where the issue density is zero because there are no issues on the page.
URLs with 100% Score
The number of URLs with 100% scores was not looking at the correct database table, and on many websites, this meant that the reported number of URLs with a 100% score was zero, even if many pages had a 100% score.
This calculation was changed to ensure it consistently returns an accurate number of URLs with 100% scores.
What You Can Expect To See
After updating Accessibility Checker, you should expect to see changes on any site where you are actively fixing problems, and there are posts with 100% scores.
Sites with issues on every post will not see changes.
Expected changes:
- URLs Scanned will go up.
This should closely match the total number of all posts you have set to be scanned in Accessibility Checker settings, less any new or unscanned posts. - Average Issues Per Page will go down.
This number may go down as low as to < 1 if most posts on your website are without problems. - Average Issue Density will go down.
This number may go down dramatically (to less than 1%) if a website has very few issues on it. - URLs with 100% score will go up.
If you have pages or posts with a 100% score this number will increase to a correct count of those posts.
These changes will be visible in the Most Recent Test Summary on the Accessibility Checker welcome page, in the dashboard widget, and in the Audit History table.
Example From a Real Website
To help visualize this, here are example “before” and “after” screenshots of the Accessibility Checker most recent test summary. The only changes on this site were updating the Accessibility Checker plugin.
Before:
Before the update, these were the numbers in the most recent test summary:
- 95.35% passed tests
- 0 unique errors
- 0 unique color contrast errors
- 53 unique warnings
- 66 ignored items
- Average issues per page: 3
- Average issue density: 1.29%
- 18 URLs scanned
- 4 of 5 post types checked
- 0 URLs with 100% score
After:
After the update, these are the numbers in the most recent test summary:
- 95.35% passed tests
- 0 unique errors
- 0 unique color contrast errors
- 53 unique warnings
- 66 ignored items
- Average issues per page: <1 (changed)
- Average issue density: 0.03% (changed)
- 316 URLs scanned (changed)
- 4 of 5 post types checked
- 301 URLs with 100% score (changed)
Questions?
Do you have questions about this update or the numbers in Accessibility Checker? Open a support ticket, and our support team will gladly help.
