In Switzerland there are no mandatory requirements that a laboratory has to fulfil in order to be allowed to offer asbestos analyses. The "official" list of laboratories conducting analyses of material samples is based on a self-declaration and is published on the website of the Swiss Asbestos Forum (FACH). According to that list, 30 of the 34 laboratories have quality accreditation and therefore regularly take part in so-called "inter-laboratory proficiency tests".

What does "proficiency test" mean in this case? In the test, specially prepared material mixtures are sent to the participating laboratories. They analyse the samples, and the results are subsequently compared. This enables to assess the internal quality of a laboratory. However: The participating laboratories are always aware that it is a proficiency test. Therefore, the respective samples may be analysed with greater effort than the routine material samples. Such proficiency tests, thus, do not automatically show how good the laboratory really is during the routine operation.

In order to assess the real quality of laboratory, the samples would have to come from a "normal" source (client). Only if the laboratory is not aware of being involved in a inter-laboratory test, the samples will be analysed in the same way as all other material samples. Furthermore, a "real" building material should be used, not the mixtures prepared for the proficiency tests.

Inter-laboratory test of FACH

This is exactly what FACH did in summer 2019: They collected a series of material samples from real construction sites and sent them under a cover name to all 34 laboratories from their list. However, the materials were analysed several times before by using different methods. This was done in cooperation with Suva's laboratories and the renowned reference laboratories IFA and ÖSBS in Germany and Austria. In order not to arouse any suspicion, the samples were sent as two deliveries from different fictitious clients.

The goal of this test was to determine the status quo of the asbestos analysis market. During the trial, the laboratories were instructed to report whether the samples contain asbestos or not. Determining the concentration of asbestos content or the asbestos type were not part of the mission. The results have been communicated to the laboratories, and some of them have released their results to the public. The main findings are as follows.

Average error rate

The average error rate was approx. 10%. Hence, on average one in ten samples has been analysed incorrectly (false positive or false negative). However, the range of errors is large:

• 0 errors: 14 of the 34 laboratories (41.1%) seem to have effective analysis protocols in place and achieved correct results for all of the samples.
1 error: Another 7 laboratories made 1 error.
• 2 to 3 errors were made in 6, or resp. 5 laboratories. This corresponds to an error rate of 15, or resp. 23%.
• 4 errors: Two laboratories had 4 wrong results within 13 samples of the test (error rate of 30%).

Error rate per material

When considering only the samples containing asbestos, the presence of asbestos was not recognized in 13% of those samples (false negatives). And vice versa, asbestos was identified in 4% of the asbestos-free materials (false positives).

The following table is showing how many errors were made by which materials (sample description is not always very precise).

Methods of analysis

There is no connection between the error rate and the analytical instruments used (PLM, SEM or TEM). Whether and how the samples were prepared plays a crucial role. However, the report of FACH does not provide any information on the preparation of the samples examined by laboratories. The results of FACH do not indicate which standard was applied.

On the other hand, it seems that domestic laboratories are making fewer errors. Out of 34 laboratories, twelve are based abroad and 22 in Switzerland. Out of the laboratories that operated error-free, 13 are located in Switzerland and one abroad.

Interpreting the results

How can we interpret the results? Is there any reason to fear that the high error rate of certain laboratories might have put people's lives at risk? Or in other words: How many expensive asbestos removals have been caused by false positive results?

The reality is somewhere in between: On average, the exposure to asbestos is nowadays already quite low. If the analytics were better, it would be probably a bit lower. However, there will definitely not be "dozens" of additional asbestos victims due to these errors.

Particular mention should be made of the high error rate for the acoustic panel in which eight of the 34 laboratories have not been able to identify the presence of asbestos (that is in total 23.5% of all laboratories). These panels usually contain the dangerous amosite asbestos. However, under normal conditions, several samples of these panels are taken so that the number of panels that is removed while wrongly assuming it is free of asbestos is certainly lower.

Which threshold should be aimed for?

There is no doubt that the error rate is far too high. It is also undisputed that a zero-error rate is desirable but not achievable in practice. However, what can be expected from a laboratory?

In the context of the article we published in 2017, we asked three Swiss laboratories to estimate how high they think the error rate on the market is. The interviewed laboratories estimated then the error rate on the market at 0.1 up to 1% (i.e. 1 error per 100 up to 1000 samples).

However, is the rate from 0.1 to 1% realistic? In particular due to the difficulty of performing certain analyses (heterogeneous materials, asbestos in the slightest traces, unclear assessment of naturally occurring asbestos, insufficient amount of sample material), this estimation seems rather unrealistic to us.

Consequences…

The FACH test of 2019 was the very first assessment of the current situation. It remained without any consequences for the laboratories. Based on this experience, FACH plans further "covert" inter-laboratory testing in the future. From 2020/21 onwards, only laboratories with a maximum of 1 error per 20 samples will be tolerated. The laboratories demonstrating a higher error rate will be removed from the FACH list.

The experience should have other consequences:

• Improvement of quality management in laboratories: Based on the results from 2019, laboratories that have participated in the trial with bad results have already taken measures to improve their quality. Thus, it can be assumed that the quality has improved already today.
• Inter-laboratory testing: It became apparent that the current quality assurance in laboratories only by "open" inter-laboratory analysis (i.e. the laboratory knows that they are analysing the samples for an inter-laboratory test) as a quality assurance measure is insufficient. Referring to that, the approach of covert tests adopted in Switzerland is recommended on international level.
• Standards and quality controls: Laboratories generally work in accordance with standards. The poor result of the inter-laboratory test can also indicate that the normative requirements for laboratories and/or the criteria for quality certification are insufficient. This applies in particular to the challenge of the growing number of "difficult" samples with very small concentrations of asbestos, such as wall plasters.
• Sampling strategy: For people conducting asbestos survey, it is important to take note of results of this inter-laboratory test as the number of samples needed to be taken, depends also on the reliability of the analytics. It may be appropriate to take several samples not only for plasters, but also for materials that are usually homogeneous. This factor should also be taken into account in standards for survey that provide indications on the number of samples to be taken.

Closing remarks

Quality assurance is an important topic already in the introductory course on asbestos surveys that we provide. We ask our participants for their opinion on how often errors occur in surveys, such as swapped, incorrectly labelled or contaminated samples, or laboratory results incorrectly transferred into the report. The discussion usually shows that the risk of errors made by diagnostics is probably as high, if not higher, as the one in laboratories.

There is no use in pointing a finger at laboratories and claiming their insufficient quality. Each and every member or organisation of the whole asbestos prevention value chain should take a moment and ask: Is the quality of my own work sufficient?

Authered by: Simon Schneebeli and Corin Gemperle