Testing based on a checklist
All websites have a common denominator – the web components and features – making it possible to create universal group tests (also known as a “checklist”) used for testing the general quality of each website project.
A checklist is actually a set of “questions” used to investigate the quality of the website project, on different levels. For each question there must be a clear, positive or negative response; regardless of whether it will be formulated in text (OK/KO, yes /no, fulfilled/not fulfilled) or graphic format.
Checks are general to a maximum extent and are unrelated to functionality specific to a project. The checklist can include the following:
- programme code (validity, module size);
- accessibility (conforming with WCAG 2.0),
- on-site SEO elements (name and description of pages, key words, site maps...);
- links (link functionality);
- form (mandatory fields , validation, and sending e-mails...);
- appearance ( in different browsers, and resolution...);
- content (texts, images);
- other similar features
You can find extensive advice on the Internet about which controls a checklist should include. For inspiration, we outline one of the more comprehensive checklists here, but ideally you should put together the list yourself. You can adjust it to the specific requirements of the testing web application. The list can also cover the requirements of your clients, and also reflect internal company standards and guidelines.
We adopted such an approach at Lundegaard. Based on the application of internal procedures in developing a programme code, the quality requirements of our customers, and generally applicable standards, we have created a tailor-made checklist. The tests were divided into several logical groups and we arranged them by priority. The test result is ultimately indicated by one of four status categories:
- OK: test was successful
- KO: defects are identified
- It makes no sense to test, e.g. inspection forms, if the web application does not contain a form
- Untested: a check could not be carried out
The objective of testing based on a checklist is so that we can check the quality level of the website before it is fully launched, because the failures in the tests must be rectified and the tests carried out again.
The Checklist used at Lundegaard is available for download [PDF 120 KB].
Depending on the type and content of the check, the test can be manual or automatic. However, if repeated checks are involved, it is preferable to automate them as much as possible. Again, help for your programmes and services is freely available on the Internet (W3C validator, Pagespeed Insights, Webpagetest, SEO spider and others).
The disadvantage of certain online services is one URL is analysed, while their website project can have hundreds or even thousands of them. Therefore, in some checks our web crawler tools proved their worth – simple, custom-created small programmes, which allow us to quickly examine and evaluate the tested qualities in bulk, on multiple pages of a website project.
Measurability and comparability of the results
An undoubted additional advantage of testing based on a checklist is the simple measurability of its results. Since all website projects are subjected to the same set of tests, their quality may be compared. In the long term, there are opportunities to identify strengths and weaknesses in the “manufacturing” process.
At Lundegaard at the end of 2013, after some testing of website projects based on a checklist, we carried out a detailed analysis of the results. Over time it was interesting to see in which checks failed immediately in the first round and which “we had under control”. As a result, we identified the weakest points, analysed the cause of recurrent failures and adjusted the internal processes so that we can avoid these errors. Almost immediately we saw an increase in the proportion of successful tests.
The pesticide paradox
Prolonged use the same checklist can lead to a “pesticide paradox [PDF 1MB]”. This refers to one of the seven basic principles of testing based on the ISTQB (International Software Testing Qualification Board), which states that if the same tests are repeated constantly, over time, the same set of test cases do not identify any new errors. In testing based on a checklist this means that the tested web applications achieve a certain level of quality, but at the same time, scope is lost for further improvement. To overcome the pesticide paradox, the necessary checks listed on the existing checklist must be expanded or modified, in order to find new weaknesses or bottlenecks, and therefore raise quality levels.
Testing based on a checklist is an excellent way to verify the quality of website project, but only generally. In addition a website should therefore be tested from the perspective of functionality of a particular project, or the security or performance, to the extent requested by the customer.