FluorideAlert - CALGARY FLUORIDE STUDY FATALLY FLAWED; KEY DATA OMITTED

A short appraisal of recent studies on fluoridation cessation in Alberta, Canada Two recent papers by McLaren et al1, 2 report on a set of linked studies conducted in Alberta, to assess the impact of the cessation of water fluoridation in Calgary on both the levels of caries in children and its distribution by socio-economic group. There have been a few cessation studies reported in the literature and they are of poor quality. A recent systematic review of the evaluation of water fluoridation published last year by the Cochrane Collaboration found that “There is insufficient information to determine the effect of stopping water fluoridation programmes on caries levels.” Cessation studies can provide a useful way of evaluating the impact of interventions, particularly if there is little long term carry-over effect of the intervention. Longer term follow up of these children would give a more reliable estimate as any carryover effects are reduced. However, as with studies measuring the introduction of water fluoridation, they need to be well designed and conducted in order to yield valid results. There are many risks of bias in introduction/withdrawal studies. The most robust study design would be a randomised controlled trial where areas are randomly allocated to start/cease water fluoridation. This ensures that comparator areas are more likley to be comparable.
This study only compares two big cities (Calgary – cessation; Edmonton – continue fluoridation). Though they are in the same Canadian province, they are unlikely to be comparable either at baseline or at follow up and are likely to have changed significantly and in different ways over the 9 year period studied. This is a major risk of bias. The researchers try to deal with this by comparing the level of caries in young children before the period of water cessation with the levels a few years after cessation. In other words, they compare the change in caries within each city over time. They then compare the changes between the two cities. They find that the rate of caries has increased in both cities, but more so in Calgary.
They then cautiously attribute that difference to the cessation of fluoridation. This attribution is not justified for several reasons: 1) The baseline levels of caries used are from a survey conducted in 2004/5. That is 6-7 years before the water fluoridation scheme was withdrawn in Calgary (2011). This assumes that there is no change in caries levels in each of the two 2 cities (more importantly no differential change) in that 6-7 year period and that the levels in 2004 (used as the baseline data in the study) are good proxies for the levels in 2011 when fluoridation ceased in Calgary.

This is a heroic assumption which is unlikely to be justified as over the period from 2004 to 2013 there has been a significant increase in caries in both groups. A key question is the extent to which the reported increase in caries in Calgary over and above that in Edmonton is due to cessation. If this is the case then one would expect the increase in caries to accelerate in the latter period of the study (after 2011). The paper only reports the change over the whole 9-year period. However, in their second paper (Table 1)2 the authors present data from a 2009 survey in Calgary which is closer to the time of cessation. Frustratingly they present analysis at the level of the tooth (deft) rather than tooth surfaces (defs) which is used in the first paper.1 This prevents direct comparison but one can assess whether the data they present (Table 1 of the second paper) gives a clue as to trends in caries in the last 4 years of the study period compared to the whole 9 years and whether these trends indicate an acceleration in the increase in caries in the later period as one would expect. They show that between 2009/10 and 2013/14 the mean number of decayed, extracted or filled primary teeth (deft) increased from 2.22 to 2.69 (a 21% relative increase). This represents a 5.3% average annual increase assuming the increase was linear). In paper 1, they report an increase in defs from 2004/5 to 2013/14 from 2.6 to 6.4 defs (a 146% relative increase); this represents a 16% average annual increase. This analysis suggests that the average annual increases in caries were significantly higher over the whole 9 years than in the final 4 years. This is contrary to what one would expect if fluoridation cessation was the primary driver of increases in caries over the period. In order to get an idea of the rate of increase in caries at tooth level between the 2004 and 2009 surveys one can use data reported by Alberta Health Services (www.albertahealthservices.ca/assets/programs/ps-1042857-coh-gensurvey.pdf).

The actual estimates of caries rates are not comparable to those reported in the study due to statistical weighting used and other factors, but one can look at the changes over time in this data set. The estimate of deft in 3 2004/5 for children in grade 2 (Table 2.1 page 5) was 1.53 and in 2009/10 was 2.06 (35% relative increase); an average annual increase of 7% over the five years prior to cessation. This analysis shows a higher average annual rate of increase in deft in the period before cessation (7%) than in the period which includes years after cessation in Calgary (5%). If the conclusions of the McLaren study are valid one would instead expect an acceleration in the increased rate of caries over the later period compared to the earlier one, not the reverse. This just demonstrates how difficult it is to infer anything causal from these sorts of study designs where one compares just 2 time points especially when the background prevalence of what one is measuring is changing over time for a variety of reasons. 2) Even if the levels of caries at the time of cessation were the same in 2011 as in 2004, this study would still be at risk of bias as it would assume that changes in the population composition of Edmonton and Calagary (especially those with young children) had remained constant or had changed in the same way over the period. These are dynamic cities that have seen considerable population change in that period and so this assumption is unlikely to be valid. We know from Table 2 that the levels of caries of those who were lifeling residents at the time of the 2013/14 survey was lower than in the general population of children of the same age (a difference 1.2 and 1.1 defs of all tooth surfaces and 1.1 and 0.8 defs for smooth surfaces for Calgary and Edmonton respectively). This just hints at the effect of population shifts, which are bound to have been even greater since 2004/5. 3)
The Cochrane review of evaluations of the effectiveness of water fluoridation3 reported that they: “found all studies to be at high risk of bias for confounding. We considered confounding factors for this outcome to be sugar consumption/dietary habits, SES, ethnicity and the use of other fluoride sources. We would have judged studies to be at low risk of confounding bias only if they had successfully controlled for all factors.” This study also fails to adjust for these confounding factors. 4) The sampling of children in the 2004 study was different between Calgary and Edmonton and this was different from that used in 2013/14 raising 4 questions about this biasing the results (though it is not clear in what way this bias would operate). 5) The study did not blind outcome assessors. Those measuring caries in the 2013/14 survey knew which children they were examining and this can introduce bias. The raters were well trained but unconscious measurement bias is well reported in medical research and can only be combated by blinded assessment. This is difficult to do but a few other studies have tried to combat it by using radiographs in order to blind assessors. This or other approaches could have been done in a sample of children to check for bias. 6) Measurement of exposure to fluoride would indicate whether there are differences between the cities. Such measurements (i.e., fingernail clippings) were taken but not reported in the paper. Unfortunately only a very small sample was used for this but it should still be reported. 7) The authors did not report on any changes in the prevalence of fluorosis in the population. 3 In a presentation to Public Health Ontario in 2014, the lead author stated that fluorosis data were being collected.4 8) The second paper tried to assess the effect of cessation on the socio-economic distribution of caries. a. The authors incorrectly argue that previous papers have shown that water fluoridation reduces disparities. The Cochrane Systematic review3 “found insufficient information to determine whether fluoridation reduces differences in tooth decay levels between children from poorer and more affluent backgrounds.” b. The study suffers from similar potential biases as the other study comparing levels with Edmonton. Here though they do not even have comparative data in other cities which have continued fluoridation. Thus this study is even weaker than the one referred to above. One cannot attribute cause and effect in a before/after study of this kind. c. It is hampered by the much lower sample size in the 2009 survey (557) than the 2013/14 survey (3230) it has less statistical power to show differences between the socioeconomic group as statistically significant. 5 The authors then use the fact that there are statistically significant differences between social groups in the later period but not the earlier one to support the view that cessation has increased inequalities. But actually this is partially due to the much larger sample size. The test of the interactions by year (the main test of change) is not statistically significant (Table 3 last column) but this is downplayed in the paper. d.

This paper is further undermined by the much lower response rate (57% in the 2013/14 survey than the original 2009 survey (81%), which may have introduced changes in the type of children included so biasing the results (though not clear in which direction). In summary Fluoridation of water generates controversy and there is uncertainty about potential benefits and harms, therefore we need robust evaluations.5 Whilst it is important to look at the impact of withdrawal of water fluoridation, the study design used by McLaren et al, means that the results are not able to cast much light on the effects of cessation of fluoridation, particularly because of the increasing background rates of caries in Alberta, the effect of population changes and the long time period between the baseline and 2014 survey. In the absence of a randomised controlled trial one should look at trends in caries over time and use methods such as interrupted time series analysis using rates of caries at several time points before and after the cessation point (though this depends on data availability). In addition one should study more than two comparator populations, incorporating data on several populations where cessation occurred and several where it was continued, selected randomly to prevent biased choices (they report that over 30 communities have opted to discontinue fluoridation so this may be possible).
The validity of the measurement of caries should have been checked by using radiographs and blind assessment to ensure that there was not observer bias. Data on exposure to fluoride should have been presented to see what change in levels was observed over the time period or at least between the two cities at the time of the follow-up survey, although this would be limited due to the very small sample of children in whom exposure was measured. 6 The authors themselves discuss several of the weaknesses of the study but their conclusion, that the findings “were consistent with an adverse effect of fluoridation cessation” is too strong. The danger is that people will take this at face value, ignore the very real weaknesses and over-interpret these studies, something of a feature in the water fluoridation debate from all sides of the argument.5 In conclusion I do not think these studies provide a valid assessment of the effect of fluoridation cessation on the levels or distribution of caries in these populations. More rigorous well-funded studies are still required to reduce the uncertainty. One of the most interesting and perhaps unexpected findings from the study though, is that caries levels are higher in both cities than recorded several years ago despite water fluoridation. This implies that even if water fluoridation is effective in reducing caries, it is not sufficient to combat poor oral health in this province

Trevor A Sheldon DSc FMedSci
Dean of the Hull York Medical School
Professor of Health Services Research and Policy
University of York, Heslington, York, UK YO10 5DD Tel: (01904) 321749 E-mail: trevor.sheldon@hyms.ac.uk