The cellular protein O⁶-Methylguanine-DNA methyltransferase (MGMT) is a DNA-repair protein that removes mutagenic and cytotoxic adducts from O⁶-guanine in DNA. Alkylating agents are among the most widely used chemotherapeutic drugs in human cancer. Alkylation induced by these compounds can produce either lethal double-strand cross-links, as is the case with bifunctional nitrosoureas (BCNU), or induce mismatch abortive repair and DNA fragmentation, as is the case with temozolomide 1 2 3 4 . The toxicity of alkylating agents is reduced in the presence of MGMT. Thus, MGMT confers resistance to alkylating agents in a wide spectrum of human tumours by reversing DNA toxicity. In brain neoplasms, hypermethylation of CpG islands in the MGMT gene promoter region, rather than mutation or deletion, is the major mechanism for the loss of MGMT function 2 5 6 7 . As a consequence, tumours with epigenetic silencing of MGMT gene become more sensitive to the killing effects of alkylating agents. Moreover, several studies have demonstrated that epigenetic silencing of MGMT is a relevant prognostic factor in patients with glioblastoma, anaplastic glioma and low grade glioma 8 9 10 11 12 13 14 . In fact, MGMT status has recently been recommended as a stratifying factor for patients in glioma trials 15 16 .

Many methods and protocols have been applied for MGMT analysis in gliomas, but to date there is no consensus on which strategy should be primarily employed 17 . Methylation-specific polymerase chain reaction (MSP) is the most commonly used test 9 . Indeed, in glioblastoma clinical trials, a strong correlation of the methylation status of MGMT with temozolomide response and patient outcome was shown. However, there are some methodological problems that limit the usefulness of this method in a routine diagnostic setting: it is complex, time-consuming, and highly dependent on tissue quality 18 19 . MGMT status can also be assessed by analyzing protein expression by immunohistochemistry (IHC). IHC is a reliable, commonly used method in diagnostic histopathology that is available in most laboratories. In addition, IHC is easier to use, less expensive and faster than MSP 20 21 22 23 24 25 26 27 28 29 , and consequently it has become the most frequently used method for the detection of MGMT protein expression in the past decade 30 . In this line, some retrospective clinical reports have also shown a prognostic association between MGMT protein expression and/or activity and outcome.

However, studies aimed at evaluating the correlation between aberrant promoter methylation and loss of protein expression have yielded contradictory results, not only in brain tumours but also in other neoplasms. While we and other authors have shown that the relationship between MGMT promoter methylation status and MGMT protein expression is not absolute 31 , other studies have found a strong correlation between homogeneous immunoreactivity and unmethylated promoter 32 . At present, there is a lack of data on which to base recommendations for a specific method or protocol for MGMT testing. Accordingly, there is a strong need for systematic comparisons and validation of intra- and interlaboratory reproducibility of different methods for MGMT assessment in order to identify the best method for clinical MGMT testing 33 .

The aim of this study was to perform a systematic review and a meta-analysis of the correlation between MGMT IHC and MSP in a large array of human brain and non-brain systemic tumours. Our primary objective was to assess the diagnostic accuracy of IHC at different cut-off values for test positivity. Because test accuracy is not a fixed property of a test 34 , we have also studied several possible sources of heterogeneity such as subgroups of patients, differing interpretations of results, and study design features.

This systematic review and meta-analysis was performed following previously published guidelines 34 35 36 37 .

The relevance of MGMT status as a potential prognostic or predictive factor in malignant glioma patients is supported by a number of independent studies. At present, detection of MGMT promoter methylation by MSP is the most commonly used method and for this reason it is considered the reference test in the present review. However, concerning day-to-day clinical practice, MSP is not yet part of the routine diagnostic work-up while MGMT assessment at RNA or protein-level are used 22 33 . The exact incidence of promoter methylation, protein or RNA expression varies according to the assessment test and among different studies 87 . An optimal method for diagnostic purposes should be widely available, easy to establish, cost-effective, reproducible both within a given laboratory and among different laboratories, and capable of yielding results that show consistent association with patient outcome 19 33 . In this regard MSP is a highly sensitive qualitative technique, but IHC has several advantages over it 88 .

Although strong agreement between MSP and IHC has been previously reported, there is growing evidence that MGMT promoter methylation assessment through MSP does not correlate well with MGMT protein expression as detected by IHC in brain tumours 25 26 31 68 89 . In addition, some studies have shown that MGMT promoter methylation and MGMT protein expression cannot be used interchangeably to predict patient survival or glioma chemosensitivity 68 90 . Results from the present meta-analysis support this evidence and suggest that cases selected by IHC may not always correspond to those selected by MSP. In fact, diagnostic accuracy estimates for MGMT protein expression by IHC were significantly lower for brain tumours than for other non-brain tumours (sensitivity, 53-64% vs. 60-81% respectively; specificity, 60-84% vs. 80-93% respectively). Similarly, positive and negative likelihood ratios did not provide convincing diagnostic accuracy for IHC in brain tumours (Additional file 9). Accordingly, the type of tumour (primary brain vs. non-brain systemic tumour) turned out to be an independent covariate of accuracy estimates in the meta-regression analysis beyond other methodological covariates such as cut-off value and type of antibody.

The reasons for these findings are not clear and different putative causes must be taken into consideration. First, there is a lack of a consistently defined cut-off value for the semiquantitative immunohistochemical scoring. Capper et al. proposed a cut-off of 15% immunolabeled cells for GBM and 35% for low grade gliomas 22 , Nakasu et al. proposed a cut-off value of 10-20% 88 , and Preusser et al. found the best agreement between MSP and IHC results when using a cut-off of 50% 20 . It is important to note that the cut-off value was not an independent covariate of accuracy in the present meta-regression analysis, whereas the type of tumour (primary brain vs non-brain) was independently associated with greater accuracy (Additional files 7 and 8). In addition, interobserver variability in discriminating positive and negative cells, specific immunostaining and background is another technical aspect of the IHC procedure 20 . Even when studies use the same explicit threshold, their implicit threshold may differ, especially if interpretation of the test requires pathology judgement 35 . Importantly, histological analysis of the tissue used for DNA extraction is not always performed (Additional file 3 and Additional file 4), and when the area of tumour used for MSP analysis is different from the one studied with IHC, necrosis and/or an overlarge sample of normal tissue might hamper the MSP results. Third, due to the fact that MSP relies on the different susceptibility of methylated versus unmethylated cytosines to sodium bisulfite modification and subsequent selective primers amplification, it is highly dependent on tissue quality and quantity, primer design, bisulfite treatment adequacy and PCR conditions 19 . Finally, MSP is so highly sensitive that a methylation band may be obtained even if cells that carry MGMT promoter methylation represent a small proportion among the majority of cells with unmethylated promoter 1 . Conversely, IHC may not be able to detect small clusters of cells that have lost protein expression 91 .

Apart from these technical issues, there are other confounding factors that may lead to false positive methylation results. Although it has been stated that the presence of a methylated MGMT allele can only be attributed to neoplastic cells 8 10 92 , some authors have demonstrated that MGMT promoter methylation may occur in non-neoplastic central nervous system tissue 3 or in normal-appearing mucosa several centimetres away from digestive tumours 56 93 . Moreover, Candiloro et al. 94 have shown low levels of methylation in peripheral blood of healthy individuals with the T allele of the rs16906252 polymorphism.

Moreover, regulation of MGMT expression in brain tumours seems to be a complex phenomenon in which abnormal methylation of the promoter region may not be the only determining factor 1 47 95 96 97 . Similar to genetic and chromosomal events, epigenetic changes may also be tissue- and tumour-specific 98 99 . In fact, the inconsistency between promoter methylation and protein expression assessed by IHC in gliomas is not limited to the MGMT gene, but has also been observed for other genes such as PTEN 100 . Gliomas are heterogeneous tumours and intratumoural heterogeneity of MGMT staining and methylation is a well-known event. Over time, variations in the methylation status of MGMT promoter within the same tumour have also been described, although the relevance of these events is unclear 31 89 101 . Interestingly, some factors, such as glucocorticoids, ionizing radiation and chemotherapy, can induce MGMT expression 26 102 . Thus, a further question to be addressed is whether tumour recurrences exhibit the MGMT status as the pre-treatment tumour or a different one. Unfortunately, data on this topic are limited and contradictory 103 . While some studies have demonstrated an increase in MGMT immunostaining 84 or a lower frequency of MGMT promoter methylation 87 104 105 in recurrent gliomas after chemotherapy, other authors have not observed any change 84 103 106 . Finally, both an increase and a decrease in MGMT expression have also been described for recurrent tumours 22 76 87 107 108 109 . A higher protein expression might indicate that the MGMT gene has been up-regulated by the treatment, although other possible explanations, such as selection of chemoresistant cells with high MGMT protein levels or intratumoral regional variations, can not be excluded 26 84 109 .

Finally, methylation is not biallelic in some tumours, leaving one allele actively expressing the protein while MGMT promoter methylation may be also observed 110 . In fact, MGMT gene is located on chromosome 10q, a region lost in the vast majority of GBM, implying that even in those GBM without promoter methylation, MGMT haploinsufficiency is likely 101 . Moreover, MGMT promoter CpG islands may present a differential pattern of methylation along the region, with some CpGs being more important than others with regard to gene transcription. In this sense, it has been suggested that the region commonly investigated by MSP might not to be among those that best correlate with protein expression 90 .

In an attempt to avoid some of the above mentioned problems, quantitative or semiquantitative methods such as MethylLight^® quantitative MPS, pyrosequencing, COBRA, etc. 66 67 70 83 87 89 111 have been reported by different groups in recent years. Whether these methods are more appropriate than MSP remains to be demonstrated in large cohorts of patients. Quantitative methods seem to provide better discrimination than classical gel-based MSP. However, as Karayan-Tapon et al. 46 note, before these methods can be used as clinical biomarkers, validation of them is required. Whichever gene is used for normalization, no quantitative-MSP assay can give a real, absolute measurement, and this might be a restriction. Moreover, completely quantitative or semiquantitative assays that normalize to a control gene or the copy number of the unmethylated MGMT promoter sequence might underestimate MGMT methylation, because contaminating nontumoral tissue will contribute to the signal of the normalizing gene 112 .

Both MGMT status at protein level and promoter methylation have been correlated with prognosis and chemosensitivity in glioma patients. As is shown in Additional file 3 and Additional file 4, the prognostic and predictive value of protein expression has been evaluated in some studies with contradictory results. Several authors have reported a significant association of MGMT expression assessed by immunohistochemistry with patients' overall or progression-free survival 22 23 31 88 113 114 115 116 117 . Some of them have even shown MGMT protein expression to be an independent predictor in the multivariate analysis 31 84 85 115 116 118 119 , whilst others have demonstrated a lack of correlation 29 46 58 74 . However, most published data were obtained from heterogeneous groups of patients with different grades and histologies, as well as distinct treatment protocols 31 . Although differences in study design could explain, at least in part, these contradictory results, other possibilities should be considered. In this sense, while those neoplastic cells that do not express MGMT may not be able to correct DNA damage induced by chemotherapy, loss of MGMT expression can also be a negative prognostic factor because of an increased susceptibility to acquiring other mutations 120 121 122 . Furthermore, due to variable interobserver agreement, insufficient correlation with MGMT promoter methylation status and the lack of a firm association with patient outcome 20 29 103 , MGMT IHC has not proved to be a clinically usable biomarker for routine diagnostic purposes and clinical decision-making.

Our review has several limitations. First, we excluded 17 studies because they did not provide data allowing construction of two-by-two tables, potentially resulting in less precise estimates of pooled diagnostic accuracy. Second, the statistical power of this meta-analysis was limited by the relatively small sample size of most included studies. Third, the QUADAS tool revealed that in approximately two-thirds of the studies partial verification bias was not clearly avoided, as not all cases evaluated with the index test went on to receive verification using the reference test. Another important aspect of study quality is the blinding of results of experimental and reference tests 123 . Unfortunately, in 84% of the studies, assessment of the reference test blinded for the IHC results was not explicitly stated by the authors, and in 73% of them no details were reported about blinding of the index test. Finally, publication bias was found in the present meta-analysis. Exclusion of non-English-language studies could contribute to explaining this fact, although a preference for publishing studies reporting positive results is a more plausible explanation 44 .

The present systematic review and meta-analysis has shown that assessment of MGMT protein expression by IHC is not in good concordance with results obtained with the MSP test. Discordance between the two tests seems to be higher for brain tumours even when comparing subgroups with identical cut-off value. Therefore, it seems that MGMT promoter methylation does not always reflect gene expression and, accordingly, the two methods cannot be used interchangeably. We conclude that protein expression assessed by IHC alone fails to reflect the promoter methylation status of MGMT, and thus in clinical diagnostic attempts the two methods will not select the same group of patients. This fact can be of crucial importance when stratifying patients in clinical studies according to their MGMT status.

Despite all the above mentioned aspects, MSP currently remains the most established method and the best approach to assessing MGMT status. It is also the technique for which the most convincing clinical correlations have been reported and, thus, it should be considered the reference test. Unfortunately, it is a relatively complex and time-consuming method not apt for routine clinical implementation in many centres 19 .

However, the analytical and clinical performance of MGMT immunoassaying seems to be inappropriate for routine diagnostic purposes. This fact, along with the lack of a robust association with MGMT promoter methylation as demonstrated in the present meta-analysis, precludes its use as a valuable biomarker for clinical decision making. It remains to be determined whether novel anti-MGMT antibodies directed against other epitopes would improve its performance 20 .

Accordingly, some authors have suggested the feasibility of using MSP combined with IHC for prognostic and predictive purposes 104 116 . Immunohistochemistry may represent a useful preliminary test to identify methylated cases while MSP should be performed in non-immunoreactive cases to identify truly methylated tumours 70 . Again, this issue deserves further investigation.

The authors declare that they have no competing interests.

All authors have participated sufficiently in the work to take public responsibility for appropriate portions of the content. MB and JI have made the design, review of the literature, and acquisition and analysis of data. They have also contributed to manuscript drafting and have approved its final version. AT has been involved in the interpretation of data, manuscript writing and critical revision, and has also approved the final version.