Introduction

Advances in genome-based microarray and sequencing technologies hold tremendous promise for improved understanding and management of disease and disease-related risk.1 Although the evidence to support the use of whole exome/genome sequencing (WES/WGS) technologies is still emerging,2 chromosome microarray (CMA) provides improved diagnostic yield over karyotyping,3, 4 is more cost effective from a diagnostic perspective,5, 6 and has been endorsed by professional societies as the first-tier diagnostic tool for children with unexplained developmental delay, multiple congenital anomalies, and/or neuropsychiatric disorders.5, 7, 8 Although commonplace in clinical care,9, 10 variation in technical approach combined with the interpretive complexity of copy number variation (CNV) continue to present diagnostic and medical management challenges.11, 12 These elements will remain challenging as we move towards WES/WGS. Although outcome-oriented data (ie, clinical and cost effectiveness) have not traditionally driven funding and regulatory decisions related to genetic/genomic tests in Canada,13, 14 the United States,15, 16 or the European Union,17 continued inattention to the downstream effects of their use may threaten quality, reimbursement for, and equitable access to these tests.13, 14, 15, 16, 18, 19, 20

Traditional measures of clinical effectiveness reflect final outcomes that can be measured in natural units (eg, life years gained) or preference-based measures (eg, quality-adjusted life years).21 In the context of genetic test evaluation, definitions of effectiveness acknowledge an intermediate outcome of utility that reflects a test result’s ‘usefulness and added value to patient management decision-making’.22, 23 Given the methodological challenges associated with ascertaining traditional clinical effectiveness data in the context of genetic diagnosis,16, 24 a much-needed evidence base, reflecting on utility in the form of clinical action or altered medical recommendations is beginning to emerge in the pediatric context. Saam et al25 surveyed 14 physicians of 48 pediatric patients who had variants detected by CMA and found that 71% had management changes after receiving the result. Coulter et al26 conducted a retrospective chart review of 1792 children who received CMA and found that abnormal variants generated a higher rate of recommendation for clinical action (54%) compared with variants of possible significance (34%; P=0.01). Ellison et al27 described physician response to 81 postnatal pathogenic CMA results and found that appropriate clinical action was taken in 94% of cases. Riggs et al20 described 146 conditions diagnosable by CMA for which management recommendations exist in the literature. In their own CMA database, 7% of cases carried a diagnosis associated with one of these 146 actionable conditions. Most recently, and similar to Coulter et al26, Henderson et al28 conducted a retrospective analysis of 187 children who had abnormal CMA results and found that 54.5% received specific medical management recommendations. Although suggestive of utility, existing studies are limited by their reliance on data that reflect hypothetical scenarios,20, 27 by lack of comparison groups,26, 28 by limited consideration of variants of uncertain significance,28 or by insufficient consideration of the factors beyond the CMA result itself that contribute to medical management in the context of pediatric genetics.12, 25, 26, 27, 29

In addition to carefully characterizing outcome metrics in this context, attention is owed to understanding the actual demands on laboratory, specialist, and surveillance services triggered by the use of genomic technologies. Understanding health service needs and utilization patterns is key to shaping and resourcing necessary services.18 Limited evidence is available to guide workforce decisions related to the appropriate number of genetics professionals in a given region,30 and health services research is beginning to emerge in the contexts of newborn screening31, 32 and cancer care.18 However, further attention to service level impact of genomic technologies is warranted. We sought to understand the nature of the recommended clinical actions that followed the use of CMA in a pediatric tertiary care context in order to gauge management and system-related impacts of current genome-based technologies and to be forward thinking about emerging and increasingly complex technologies.

Methods

Sample

The cases reviewed included all those for which CMAs were ordered from the The Hospital for Sick Children (SickKids) clinical cytogenetics laboratory in 2009–2011. Indications for CMA testing included developmental delay/intellectual disability and congenital anomalies; only cases for which clinical follow-up occurred at SickKids were included. CMA ordered for parental or unaffected family member studies were excluded. Cases for which post-CMA consults occurred off-site were excluded as records were unavailable. If the index patients passed away before CMA results were received/reported, the cases were excluded.

Microarray platform

In 2009–2011, an Agilent oligonucleotide custom design genomic array (Agilent Technologies, Santa Clara, CA, USA) platform was used.6 This platform contains approximately 44 000 oligos and is designed to detect gains and losses across the genome. The custom design contains high-density coverage (~50 kb resolution) in genomic regions associated with known genetic conditions. Genome-wide coverage contains an average probe spacing of ~75 kb and the array is designed to detect genomic gains/losses of ~225 kb across the genome.

Data collection

We conducted a retrospective medical record review using the hospital’s electronic and paper charts. Study data were collected and managed using Research Electronic Data Capture (REDCap, hosted at SickKids).33 REDCap is a secure, web-based application designed to support data capture for research studies. It provides: (1) an intuitive interface for validated data entry; (2) audit trails for tracking data manipulation and export procedures; (3) automated export procedures for seamless data downloads to common statistical packages; and (4) procedures for importing data from external sources. From each patient record, age, sex, ethnicity, baseline phenotypic status, family history, CMA result, and medical investigations prior to (ie, time 1 investigations) and subsequent to the receipt of CMA results (ie, time 2 investigations) were recorded. Data reflecting time 2 investigations were drawn from up to three post-CMA consult notes. Ethnicity was captured by recording the country of origin and applying the UN Statistics Division’s classification system.34 Baseline phenotypic status was ascertained through physical examination and medical history noted in the consult letter prior to CMA testing. Primary clinical features were recorded by organ system and severity (major or minor anomaly) and guided by the human phenotype ontology (HPO) standardization35 and clinical expertise (RW). Family history was recorded as positive, negative, or unknown. A positive history indicated that a family member (up to second-degree relative) had the same anomaly or syndrome as the proband. The classification of CMA results was based on laboratory interpretation as noted in the report. If result classification was not explicit in the report, the research genetic counselor (NH) assigned a guideline-based category.36 Once assigned by the research genetic counselor, all guideline-based classifications were verified by clinical/ cytogenetic expertise (SC/DS). CNV classifications were: (i) clinically significant, (ii) uncertain clinical significance, likely significant, (iii) uncertain clinical significance, (iv) uncertain clinical significance, likely benign, and (v) benign.36

Relevant medical recommendations included: laboratory testing, specialist consultations, diagnostic imaging, surveillance protocols, and investigations for other family members. We were limited to recording only medical recommendations documented in the consult notes or electronic charts. All time 2 investigations were new investigations that had not been recorded in the chart prior to the time 2 consult. One of the three indications was assigned to each of the recorded time 2 investigation: (i) prompted by CMA (ie, new investigation prompted either by evidence of an association between CNV-related genes and phenotype or for further clarification of the result), (ii) diagnostic quest (ie, new investigation prompted by continued efforts to establish a diagnosis), and (iii) ongoing care (ie, new investigation initiated because of a presenting clinical finding unrelated to the array result).

Analysis

With the exception of age, ethnicity, phenotype, and microarray result type, all independent variables were dichotomized. The HPO classification system enabled detailed phenotypic coding; four high-level categories, based upon the number of organ systems involved for major anomalies, were constructed for the purpose of this analysis. Eleven organ systems were represented by the phenotypic features presenting among this cohort: cardiovascular, central nervous system (including developmental delay/intellectual disability, autism spectrum disorders), ocular, auditory, gastro-intestinal, genito-urinary, head and neck, integumentary, musculoskeletal, respiratory, and growth. The levels of the phenotype variable included: (i) <2 organ systems involved, (ii) 2 systems involved, (iii) 3 systems involved, and (iv) >3 systems involved. Prior to characterizing this variable by organ system, we characterized this variable according to the presence/absence of congenital anomalies, developmental delay, and neuropsychiatric disorders (eg, autism spectrum disorder), as others have reported.26, 28 Finding no effect of phenotype (data not shown), we opted to characterize this variable by the number of organ systems affected, as a plausible proxy for severity. CMA results were categorized in three different ways. For CMA type, we first constructed a five-level variable characterizing result types as: (i) clinically significant, (ii) uncertain clinical significance, likely significant, (iii) uncertain clinical significance, (iv) uncertain clinical significance, likely benign, and (v) benign. Interested in the effect of characterizing uncertainty in different ways, we then constructed a broad characterization of uncertainty: (i) clinically significant, (ii) uncertain clinical significance, likely significant+uncertain clinical significance+uncertain clinical significance, likely benign, and (iii) benign. In addition, we used a narrow characterization of uncertainty: (i) clinically significant+uncertain clinical significance, likely significant, (ii) uncertain clinical significance, and (iii) uncertain clinical significance, likely benign+benign. Frequency data are presented to describe patient characteristics as well as the type of medical investigations recommended post-CMA. Binomial log link regression analyses were used to investigate the independent effects of each characteristic (ie, age, sex, ethnicity, family history, phenotype, CMA result type) on the presence/absence of a post-CMA medical recommendation. The final main effects model is reported with relative rates (RR) and 95% confidence intervals (CI). All statistical analyses were completed using relative risk37 and other R software packages.38

Results

Sample characteristics

Of the 4347 cases for which CMA was ordered at SickKids from 2009 to 2011, 1931/4347 (44.4%) met inclusion criteria. At the time that this cohort was established, SickKids was one of two provincial centres that offered CMA. For this reason, 2136 cases were excluded because the child was followed by a non-SickKids’ physician and we did not have access to necessary follow-up data. Other exclusions were: adult patients (n=7), deceased patients (n=28), parents enrolled in family studies (n=240), and cases used for validation studies (n=5). Of those included in the cohort, 457/1931 samples had a reportable microarray result (23.7%) (ie, clinically significant, uncertain clinical significance, likely significant, uncertain clinical significance, uncertain clinical significance, likely benign), and 1474/1931 results (76.3%) were considered benign. We constructed a sample of 752 cases, including all variant cases (n=457) and a sample of truly benign cases (n=295), representing a 2.5:1 ratio with clinically significant cases to improve our power. Benign cases were randomly selected per calendar month.

Of the 752 children included, 22.2% were aged <12 months, 38.3% were 1–5 years, 16.2% were 6–10 years, and 23.3% were >10 years. The sex distribution was relatively even with 55.9% male and 44.1% female. On clinical assessment, 27.9% presented with features involving <2 organ systems, 55.5% with 2–3 organ systems, and 16.6% with >3 organ systems. With respect to family history, 21.1% reported a positive family history related to the proband’s presentation and 78.9% did not. The sample consisted of children of European (31.4%), Asian (25.8%), and other ancestries including North/Latin American, Caribbean, African, Oceanic, and Euro-Asian (42.8%). Overall, 24.8% of the CMA results were clinically significant/likely clinically significant, 25.7% were of uncertain clinical significance, and 49.6% were uncertain clinical significance, likely benign/benign. The majority of ordering physicians were genetics specialists (96.8%) compared with non-genetics specialists, and given the requirement regarding access to follow-up medical records, the majority of ordering physicians were SickKids based (92.8%), in contrast to those ordered by community hospital or private practice-based physicians (Table 1).

Table 1 Sample characteristics
Full size table

Medical recommendations following CMA

Following the provision of CMA results to families, up to and including three clinical consults, 79.6% of children with reportable results (364/457) and 62.4% of children with benign results (184/295) received medical recommendations (n=548; Table 3). A total of 2508 new investigations were recommended for this group of 548 children (72.9% of the total cohort, ). Of the total number of recommendations made, the greatest proportion was for specialist consultations (40.8%) (Table 2, Figure 1).

Table 3 Factors associated with medical recommendations post-CMA: regression analyses with one predictor variable
Full size table
Table 2 Number and type of new recommendations following CMA results
Full size table
Figure 1
figure 1

Medical recommendations following CMA analysis.

Full size image

Although all new investigations followed chronologically from CMA result reporting, only 23.4% of all recommendations were prompted by the CMA result itself. Of those prompted by CMA, the greatest proportion was prompted by clinically significant CNVs (45.7%; ), followed by likely clinically significant CNVs (25.6%; ), CNVs of uncertain clinical significance (24.1%; ), and likely benign CNVs (4.6%; ) (Table 2). Similarly, of those prompted by the CNV itself, the recommendation that occurred most frequently was specialist consultation (37.7%; Table 2, Figure 1). For both the entire cohort and for the CMA-prompted sub-group, specialists included but were not limited to cardiologists, nephrologists, neurologists, developmental pediatricians, oncologists, ophthalmologists, dermatologists, endocrinologists, dentists, speech language pathologists, and nutritionists. Compared with all investigations that followed chronologically from the CMA, those that were prompted by the CNV included a higher proportion of investigations specific to surveillance protocols (15.0 versus 4.5%) and family investigations (14.0 versus 4.9%) (Table 2, Figure 1).

Notably, 31.3% of the total new investigations were recommended for children with benign CNV results, and on average, children in this category received as many as 2.7 new recommendations following the receipt of their benign result. Recommendations in this group were prompted by continued diagnostic quest (22.9%) and ongoing care warranted by presenting clinical concerns (77.1%) and included specialist consults (40.3%), medical imaging (34.4%), lab tests (20.4%), surveillance protocols (2.0%), and family member recommendations (2.9%) (Table 2).

Factors associated with medical recommendations following CMA

We sought to understand how specific patient characteristics were related to post-CMA medical recommendations. Table 3 describes if/how each patient characteristic is independently associated with the presence/absence of post-CMA medical recommendations. Three factors reached significance: age, CMA result type, and ordering physician. First, age ≥12 months was associated with at least 25% reduced rate of medical recommendations compared with age <12 months (RR1–5 years=0.75; 95% CI (0.69–0.82)). Second, CMA results that were clinically significant, likely clinically significant, and of uncertain clinical significance were associated with a 23–40% higher rate of medical recommendations when compared with benign CMA results (RRpath=1.36; 95% CI (1.21–1.53); RRlikely path=1.40; 95% CI (1.24–1.59); RRuncertain=1.23; 95% CI (1.09–1.38)). We also compared recommendation rates for clinically significant and benign variants to both broad and narrow characterizations of uncertain results. When those with clinically significant results were compared with those included in a broad classification of uncertain results (ie, uncertain+uncertain, likely significant+uncertain, likely benign), there was no significant difference in recommendation rates (RRbroad=1.09; 95% CI (0.99–1.2)). When those with clinically significant results were compared with those included in a narrow classification of uncertain results (ie, only ‘clear’ uncertain results in this classification), we found a marginal increase in the recommendation rate (RRnarrow=1.12; 95% CI (1.02–1.24)). Benign results were associated with a lower rate of recommendations, compared with uncertain results, regardless of how uncertainty was classified (RRbroad=0.80; 95% CI (0.72–0.90); RRnarrow=0.84; 95% CI (0.75–0.94)).

And finally, CMA ordered by physicians outside of SickKids and by non-genetics professionals was associated with a 20–24% increased rate of medical recommendations compared with CMA ordered by SickKids’ physicians (RR=1.24; 95% CI (1.12–1.38)) and genetics professionals (RR=1.20; 95% CI (1.10–1.32)). We did not identify an independent effect of phenotype (categorized according to the number of organ systems involved) on post-CMA recommendations (RR2=1.07; 95% CI (0.95–1.20); RR3=1.07; 95% CI (0.95–1.22); RR>3=1.10; 95% CI (0.96–1.26)) and did not detect an association between phenotype and age (P=1.0).

Discussion

Enhanced diagnostic yield compared with that attained by traditional karyotyping has prompted routine use of CMA in the pediatric genetics context. The nature of this technology is such, however, that with enhanced diagnostic yield comes increased diagnostic complexity. To the extent that CMA-related diagnoses are clear and directive of clinical action, little is known about the volume and nature of this indicated action. To the extent that CMA-related diagnoses remain uncertain, ensuing clinical action is speculative, subject to variation in clinical judgment, and has received even less empirical attention. In times of fiscal constraint, continued access to these and emerging genome-based technologies demands an evidence base that extends beyond their diagnostic utility.39

To this end, we report findings from a study that sought to understand the volume and nature of medical recommendations, following the receipt of reportable versus benign CMA results, for a cohort of children with developmental delay and/or congenital anomalies. A first finding is that, on average, five recommended investigations can be anticipated to follow the receipt of a CMA result. This is not specific to those prompted by the CNV itself but includes those initiated for continuation of care and further diagnostic query. In part, these investigations were likely driven by the common practice of initiating cardiology and nephrology referrals given the significant rate of related findings in individuals with chromosome abnormalities.40, 41 However, calls for consultations extended well beyond these two specialist groups. Therefore, multiple specialist groups, each sufficient in number, need not only be available to accept referrals for children who pursue CMA but must also be equipped to navigate the relevance of CMA findings for their patient’s care. In addition to specialist capacity building, primary care capacity building will likely be warranted as CMA testing has shifted from being available only in tertiary care settings to community-based settings.42 Our comparison of medical recommendations for those with reportable CMA results to those with benign CMA results makes a novel contribution to this evidence base. Although our modelling indicates that reportable CNVs are a more significant driver of recommendations than benign CNVs, the finding that children with benign results receive an average of 2.7 recommendations following CMA and consumed 31% of the recommendations made overall is highly relevant to the debate about if/how system-level efficiencies are enabled by genome-based technologies.43

Second, from the regression analysis we confirm the uncontested notion that medical recommendations are a function of multiple patient characteristics. Although we anchored this study around the pursuit of CMA, it is noteworthy that age, CMA result type, and type of ordering physician made independent contributions to medical recommendations. Specifically, younger children are more likely to receive post-CMA recommendations than older children; they are perhaps more significantly affected if they come to a specialist’s attention at a young age and may be candidates for a more extensive battery of assessments than children who present at older ages. As well, the older children in our cohort may have received more extensive investigations at a younger age (ie, not counted in our analysis). We did not identify an independent effect of phenotype. Although phenotypic variation intuitively drives clinical action, the fact that our cohort of children has a set of phenotypic features sufficient to make a referral to medical geneticists in a tertiary care centre perhaps dilutes any effect that phenotypic variation within this cohort might drive. Although an association between age and phenotype was not apparent and our severity-based characterization of phenotype was imperfect, further consideration of the relationship among phenotype, age, and severity is warranted. Finally, despite the small sample size of non-SickKids and non-geneticist ordering physicians, we speculate that patients for whom CMA was ordered by non-tertiary, non-specialized clinicians may not have had as extensive a diagnostic work up prior to pursuing CMA as those seen initially in a specialized tertiary care centre.

Using a more sophisticated design and analytical approach, we add rigor to the descriptive findings of others25, 27, 28 that suggest that clinically significant CMA results drive medical investigations. Compared with children with benign CMA results, children with clinically significant and uncertain results are more likely to receive medical recommendations. Although significant on balance, the strength of the contribution that variants of uncertain significance make to recommendation rate appears to depend on how uncertainty is characterized and to what it is compared. When uncertainty is characterized as narrow, for example, clinically significant results drive more recommendations than uncertain results and uncertain results drive only marginally more recommendations than benign results. Variants of uncertain significance are typically reported to clinicians in order to give clinicians the opportunity to consider the uncertain variant in the context of the patient’s complete clinical picture. The contribution that this clinical vantage point may make towards diagnostic thinking44 appears to be reflected in subsequent clinical decisions reported herein. From the perspective of gauging clinical utility of CMA, if we uphold the intermediate outcome of altered medical management as a relevant metric, reporting variants of uncertain significance appears to confer utility but to varying degrees. These findings pose—but do not answer—a question related to CMA reporting algorithms: would a simplified, two-level reporting scheme (ie, clinically significant variants versus not clinically significant variants), a shift away from the five-level reporting scheme, simplify or compromise diagnostic thinking and clinical actions at a time when the evidence base from which to understand these variants is still evolving? Although these findings suggest that the current granularity enables fine-tuned clinical decisions, future studies will need to address these nuances as interpretive categories and utility-related outcomes become better defined.

Unlike findings reported to date,20, 25, 26, 27, 28 we place the clinical utility of variant CMA results into context by comparing medical recommendations that ensue following benign/negative CMA results, we deepen our understanding of the clinical utility of variants of uncertain significance, and we analyse the independent effects of several factors that influence medical recommendations in addition to CMA results. However, we acknowledge several study limitations. First, our measure of clinical utility captures an intermediate outcome: that of recommended clinical action. It does not reflect upon more traditional definitions of clinical utility that are characterized by improvements in medical management or health outcomes.21 Second, as SickKids is a tertiary care academic centre, we acknowledge an ascertainment bias as included cases may be more significantly clinically affected than children for whom CMA is initiated in primary or community care settings. In addition, while we captured medical recommendations and some services used at our centre, we did not capture recommendations or services used at other centres. Extending data capture to these end points was complicated by incomplete on-site data and limited access to off-site data. Furthermore, while we captured services recommended, we did not capture services or investigations that might have been avoided as a result of CMA analysis, itself an important measure of utility.

Finally, while beyond the scope of these analyses, depicting the relationship between patient characteristics and medical recommendations could be used to predict the probability that medical recommendations will ensue for an individual patient with a given set of characteristics. Boone et al45 suggest that, ‘physicians must consider the clinical utility of a genomic test before it is ordered.’ To the extent that health-care providers may value guidance on the relevance of various genomic technologies now at their disposal, a predictive model of this sort may warrant further pursuit.

Conclusion

Our findings highlight the complexity of ascertaining unequivocal outcome data for a technology that generates information intended to serve both diagnostic and management purposes and is of uncertain and evolving significance. Although we have captured medical recommendations as a characterization of clinical utility, unconsidered in this analysis per se are the critical notions of diagnostic and personal utility; how value is represented in the complex realm of genomic medicine and whether it can or should be restricted to health-related value remains unresolved.24 Building an evidence base that begins to tackle these issues, however, provides a step forward in assisting their resolution.