In Switzerland there are no mandatory requirements that a laboratory has to fulfil in order to be allowed to offer asbestos analyses. The "official" list of laboratories conducting analyses of material samples is based on a self-declaration and is published on the website of the Swiss Asbestos Forum (FACH). According to that list, 30 of the 34 laboratories have quality accreditation and therefore regularly take part in so-called "inter-laboratory proficiency tests".

What does "proficiency test" mean in this case? In the test, specially prepared material mixtures are sent to the participating laboratories. They analyse the samples, and the results are subsequently compared. This enables to assess the internal quality of a laboratory. However: The participating laboratories are always aware that it is a proficiency test. Therefore, the respective samples may be analysed with greater effort than the routine material samples. Such proficiency tests, thus, do not automatically show how good the laboratory really is during the routine operation.

In order to assess the real quality of laboratory, the samples would have to come from a "normal" source (client). Only if the laboratory is not aware of being involved in a inter-laboratory test, the samples will be analysed in the same way as all other material samples. Furthermore, a "real" building material should be used, not the mixtures prepared for the proficiency tests.

Inter-laboratory test of FACH

This is exactly what FACH did in summer 2019: They collected a series of material samples from real construction sites and sent them under a cover name to all 34 laboratories from their list. However, the materials were analysed several times before by using different methods. This was done in cooperation with Suva's laboratories and the renowned reference laboratories IFA and ÖSBS in Germany and Austria. In order not to arouse any suspicion, the samples were sent as two deliveries from different fictitious clients.

The goal of this test was to determine the status quo of the asbestos analysis market. During the trial, the laboratories were instructed to report whether the samples contain asbestos or not. Determining the concentration of asbestos content or the asbestos type were not part of the mission. The results have been communicated to the laboratories, and some of them have released their results to the public. The main findings are as follows.

Average error rate

The average error rate was approx. 10%. Hence, on average one in ten samples has been analysed incorrectly (false positive or false negative). However, the range of errors is large:

• 0 errors: 14 of the 34 laboratories (41.1%) seem to have effective analysis protocols in place and achieved correct results for all of the samples.
1 error: Another 7 laboratories made 1 error.
• 2 to 3 errors were made in 6, or resp. 5 laboratories. This corresponds to an error rate of 15, or resp. 23%.
• 4 errors: Two laboratories had 4 wrong results within 13 samples of the test (error rate of 30%).

Error rate per material

When considering only the samples containing asbestos, the presence of asbestos was not recognized in 13% of those samples (false negatives). And vice versa, asbestos was identified in 4% of the asbestos-free materials (false positives).

The following table is showing how many errors were made by which materials (sample description is not always very precise).

Methods of analysis

There is no connection between the error rate and the analytical instruments used (PLM, SEM or TEM). Whether and how the samples were prepared plays a crucial role. However, the report of FACH does not provide any information on the preparation of the samples examined by laboratories. The results of FACH do not indicate which standard was applied.

On the other hand, it seems that domestic laboratories are making fewer errors. Out of 34 laboratories, twelve are based abroad and 22 in Switzerland. Out of the laboratories that operated error-free, 13 are located in Switzerland and one abroad.

Interpreting the results

How can we interpret the results? Is there any reason to fear that the high error rate of certain laboratories might have put people's lives at risk? Or in other words: How many expensive asbestos removals have been caused by false positive results?

The reality is somewhere in between: On average, the exposure to asbestos is nowadays already quite low. If the analytics were better, it would be probably a bit lower. However, there will definitely not be "dozens" of additional asbestos victims due to these errors.

Particular mention should be made of the high error rate for the acoustic panel in which eight of the 34 laboratories have not been able to identify the presence of asbestos (that is in total 23.5% of all laboratories). These panels usually contain the dangerous amosite asbestos. However, under normal conditions, several samples of these panels are taken so that the number of panels that is removed while wrongly assuming it is free of asbestos is certainly lower.

Which threshold should be aimed for?

There is no doubt that the error rate is far too high. It is also undisputed that a zero-error rate is desirable but not achievable in practice. However, what can be expected from a laboratory?

In the context of the article we published in 2017, we asked three Swiss laboratories to estimate how high they think the error rate on the market is. The interviewed laboratories estimated then the error rate on the market at 0.1 up to 1% (i.e. 1 error per 100 up to 1000 samples).

However, is the rate from 0.1 to 1% realistic? In particular due to the difficulty of performing certain analyses (heterogeneous materials, asbestos in the slightest traces, unclear assessment of naturally occurring asbestos, insufficient amount of sample material), this estimation seems rather unrealistic to us.

Consequences…

The FACH test of 2019 was the very first assessment of the current situation. It remained without any consequences for the laboratories. Based on this experience, FACH plans further "covert" inter-laboratory testing in the future. From 2020/21 onwards, only laboratories with a maximum of 1 error per 20 samples will be tolerated. The laboratories demonstrating a higher error rate will be removed from the FACH list.

The experience should have other consequences:

• Improvement of quality management in laboratories: Based on the results from 2019, laboratories that have participated in the trial with bad results have already taken measures to improve their quality. Thus, it can be assumed that the quality has improved already today.
• Inter-laboratory testing: It became apparent that the current quality assurance in laboratories only by "open" inter-laboratory analysis (i.e. the laboratory knows that they are analysing the samples for an inter-laboratory test) as a quality assurance measure is insufficient. Referring to that, the approach of covert tests adopted in Switzerland is recommended on international level.
• Standards and quality controls: Laboratories generally work in accordance with standards. The poor result of the inter-laboratory test can also indicate that the normative requirements for laboratories and/or the criteria for quality certification are insufficient. This applies in particular to the challenge of the growing number of "difficult" samples with very small concentrations of asbestos, such as wall plasters.
• Sampling strategy: For people conducting asbestos survey, it is important to take note of results of this inter-laboratory test as the number of samples needed to be taken, depends also on the reliability of the analytics. It may be appropriate to take several samples not only for plasters, but also for materials that are usually homogeneous. This factor should also be taken into account in standards for survey that provide indications on the number of samples to be taken.

Closing remarks

Quality assurance is an important topic already in the introductory course on asbestos surveys that we provide. We ask our participants for their opinion on how often errors occur in surveys, such as swapped, incorrectly labelled or contaminated samples, or laboratory results incorrectly transferred into the report. The discussion usually shows that the risk of errors made by diagnostics is probably as high, if not higher, as the one in laboratories.

There is no use in pointing a finger at laboratories and claiming their insufficient quality. Each and every member or organisation of the whole asbestos prevention value chain should take a moment and ask: Is the quality of my own work sufficient?

Authered by: Simon Schneebeli and Corin Gemperle

The price, but also the quality of an asbestos survey depends to a large extend on the number of samples taken. Various organisations have tempted to give recommendations on the number of samples to take. If we compare these figures, we realise however that these recommendations diverge considerably. It appears that the experts are far from agreeing on this point. But how can we get to a better approach?

Note: The author of this article is not native English speaking. So please be kind with your critique on form and grammar.

Last year I was involved in a project where we had to survey buildings with a total surface of more than 200’000 m² (2,000,000 ft²) . The building owner initially estimated that several hundred samples needed to be taken. In the end, there were however more than 1000 samples that needed to be analysed. Justifiably the building owner asked whether so many samples are really necessary. We then compared different standards and recommendations. The results are summarized in the following table.
 

Switzerland: ASCA

The Swiss Association of Asbestos Consultants ASCA uses a statistical model: The calculations are based on the “homogeneous unit”. In the case of ceiling tiles for example, each tile is considered such a unit. The goal is to take enough samples so that you get a 75% probability that you find asbestos even if only 5% of the tiles contain it.

This method is interesting as it is based on calculations and not just on “good feeling”. But what is a “homogeneous unit”? And what about this hypothesis that 5% of the tiles contain asbestos? This approach still leaves a lot of autonomy to the person conducting the survey and generally leads to a very high number of samples.

Geneva: STIPI

The recommendations for building surveys of the ASCA are themselves based on recommendations from the authorities of Geneva (formerly STIPI, today called SABRA). This document is not in use anymore. It is mentioned here because the figures given there are a quite good representation of the common practice in the German part of Switzerland (in the French part, it's much more the ASCA recommendations that are followed).

 USA: EPA 

In the USA, the Environmental Protection Agency EPA published recommendations regarding asbestos surveys in 1985 (EPA 585 / 5-85 030 a, also called “pink book”). This document does mainly concern friable asbestos and requires at least 9 samples, except in small rooms:

• 3 samples for rooms of less than 1000 ft2 (92m2)
• 5 samples for rooms between 1000 ft2 (92m2) and 5000 ft2 (464m2)
• 7 samples for surfaces that are bigger than 5000 ft2 (464m2)

For samples of non-friable materials the document only speaks of “samples” in plural. It is thus assumed that one has to take at least 2 samples.

 Fr ance: Afnor 

In France, the AFNOR standard NF X 46-020 describes how to conduct an asbestos survey. This standard says that the person conducting the survey has to decide her or himself how many samples are required, but that normally it should follow the recommendations given in this standard itself.

For the floor tiles as well as for acoustic ceiling tiles, this standard says that 1 sample has to be taken for every type of product. This can be interpreted as if even for 10 rooms with acoustic ceiling tiles only 1 sample needs to be taken.

England: HSG 264

The English standard HSG 264 gives indications about the number of samples for a small number of materials. For homogeneous materials, the standard requires 1 sample, but for example for tube insulations, the standard explicitly requires one sample every 3 to 6 meters.

Germany : VDI

There is no German standard about asbestos building surveys. A working group of the Association of German Engineers VDI has published a paper in summer 2016 that contains data for certain materials. A new standard on this topic should be published in the future.

Interpretation

The diverging data shows mostly one thing: We are far from agreeing how many samples we should take.

Now we could do two things: We could either go towards more details, but with the risk that we create something that is so complex that it becomes unusable for the practitioner. Or we can say that in the end every specialist has to decide him or herself how many samples to take.

As a person that trains people in doing building surveys, it is for me not an option to simply say: “take as many samples as you think is necessary”. We need a more detailed approach.

But how to get there?

Criteria to determine the number of samples

Risk based approach: The first point to determine is: What is the reliability that we want to achieve or: How many accidental asbestos exposures (of workers or of people living in a building) do we consider as acceptable? Do we aim for a zero risk? Then we’d have to take a very high number of samples. Or do we say that a reduction by a certain factor is enough? If I say for example that a 99.9% reduction compared to exposure levels of the seventies is enough, the number of samples is much lower.

Homogeneity: The various standards agree on this: The number of samples to take depends heavily on the homogeneity of a material: The more heterogeneous it is, the more samples are necessary.

Reliability of the laboratories: The requirement of the EPA is presumably based on the hypothesis that there are errors in the laboratory. As those working in laboratories are just humans, it would indeed be wrong to assume a 100% reliability. But how high is this reliability? Is it still justified that we always have to take at least 2 samples or did laboratories get better over the last three decades which would allow fewer samples?

Asbestos content: This is linked to the reliability of the laboratory: For materials with a high concentration of asbestos, the number of errors is probably much lower than for materials that contain only a few tenths of a percent of asbestos like window putty.  

Friable/non-friable: A tar-based material that contains asbestos is much less dangerous than for example gypsum based materials. It is thus less of a problem if I such a material is accidentally declared as asbestos free.

Others: The goal of this article is rather to launch a discussion on this topic than to give the ultimate solution. Thus any complement is welcome.

Statistical approach

Through all these considerations it becomes clear: There is no simple approach for all materials. If we want to give recommendations for the sampling strategy, we have to differentiate (while still trying to stay simple enough that things are useful for the practitioner). We have to collect all kinds of data and knowledge from experts, for example about the risk homogeneity or the asbestos content.

Such an approach based on data and experience can be combined with a statistical approach: What is the risk level that we accept? Do we aim at zero risk? And how do we reach this goal?

Here is an example: The ASCA requires a reliability of 70% percent even if only 5% of the homogeneous units contain asbestos. If we consider one single flat, this value may seem rather low. But if we consider a whole lot of similar buildings, the actual exposure of the workers can already be massively reduced even with a small number of samples:

Conclusion

The goal of this article was mainly to show that we are far from agreeing how many samples are actually necessary and that – if we want to become more concrete here, we’d have to consider different factors and different approaches, including things like the homogeneity of a sample, the reliability of laboratories etc. An answer on how all this factors can be integrated can not be given here. I hope that this article can contribute to a brother debate on this topic.

One final remark: In the case mentioned in the introduction, it was decided to strictly follow the ASCA approach, which required way more than 1000 samples. The interesting point in this: It allowed to declare certain elements asbestos free that in a first step (with fewer samples) had been considered contaminated. The high number of samples may thus actually reduce the cost of asbestos removal. Higher costs for analysis may thus overall not necessarily be more expensive.