Abstract
Objective
Current staging and risk-stratification systems for predicting survival or recurrence of patients with differentiated thyroid carcinoma may be ineffective at predicting outcomes in individual patients. In recent years, nomograms have been proposed as an alternative to conventional systems for predicting personalized clinical outcomes. We conducted a systematic review to evaluate the predictive performance of available nomograms for thyroid cancer patients.
Design and methods
PROSPERO registration (CRD42022327028). A systematic search was conducted without time and language restrictions. PICOT questions: population, patients with papillary thyroid cancer; comparator prognostic factor, single-arm studies; outcomes, overall survival, disease-free survival, cancer-specific survival, recurrence, central lymph node metastases, or lateral lymph node metastases; timing, all periods; setting, hospital setting. Risk of bias was assessed through PROBAST tool.
Results
Eighteen studies with a total of 20 prognostic models were included in the systematic review (90,969 papillary thyroid carcinoma patients). Fourteen models were at high risk of bias and four were at unclear risk of bias. The greatest concerns arose in the analysis domain. The accuracy of nomograms for overall survival was assessed in only one study and appeared limited (0.77, 95% CI: 0.75–0.79). The accuracy of nomograms for disease-free survival ranged from 0.65 (95% CI: 0.55–0.75) to 0.92 (95% CI: 0.91–0.95). The C-index for predicting lateral lymph node metastasis ranged from 0.72 to 0.92 (95% CI: 0.86–0.97). For central lymph node metastasis, the C-index of externally validated studies ranged from 0.706 (95% CI: 0.685–0.727) to 0.923 (95% CI: 0.893–0.946).
Conclusions
Our work highlights the extremely high heterogeneity among nomograms and the critical lack of external validation studies that limit the applicability of nomograms in clinical practice. Further studies ideally using commonly adopted risk factors as the backbone to develop nomograms are required.
Significance statement
Nomograms may be appropriate tools to plan treatments and predict personalized clinical outcomes in patients with papillary thyroid cancer. However, the nomograms developed to date are very heterogeneous, and their results seem to be closely related to the specific samples studied to generate the same nomograms. The lack of rigorous external validation procedures and the use of risk factors that sometimes appear to be far from those commonly used in clinical practice, as well as the great heterogeneity of the risk factors considered, limit the ability of nomograms to predict patient outcomes and thus their current introduction in clinical practice.
Introduction
Differentiated thyroid cancer (DTC) accounts for 90% of all thyroid cancers and is increasing threefold every decade (1) so that by 2040 it will be the fourth most common malignancy in the group aged 20–49 years (2). Papillary thyroid carcinoma (PTC) occurs in 80–85% of patients with DTC (3, 4, 5, 6, 7, 8, 9) and is three times more common in women than in men (10). In most cases, it is an indolent tumor with indolent cancer, slow disease progression, excellent prognosis, and a 10-year survival rate of more than 90% (11, 12, 13). Nevertheless, the risk of recurrence ranges from 5 to 21% (14, 15), with a prevalence of lymph node metastases of approximately 30–90%, occurring mainly in the central neck compartment (18–80%) (16, 17, 18).
Current staging systems for predicting the survival or recurrence of patients with PTC appear to be relatively ineffective at predicting outcomes in individual patients (19, 20). The AJCC staging system, which focuses primarily on patient survival, has been criticized for not accurately predicting the prognosis of patients with PTC, thus exposing them to overdiagnosis and overtreatment, especially in low-risk patients who account for the majority of PTCs and who present a very low mortality rate (21, 22). On the other hand, the American Thyroid Association (ATA) stratification system focused on the risk of persistent/recurrent disease does not seem to assign the proper relevance to specific factors, such as age, family history, histological subtypes, multifocality, the extent of vascular invasion, or extent of metastatic lymph node involvement, which are on the contrary associated with a higher recurrence risk (19). Other prognostic scoring systems, such as AGES (age, grade, extent of disease, size), AMES (age, metastasis, extent of disease, size), MACIS (metastasis, age at presentation, completeness of surgical resection, invasion, size), and EACCD (ensemble algorithm of clustering of cancer data), focus on identifying high-risk tumors and do not precisely aim to reduce the risk of overdiagnosis (23, 24).
In recent years, nomograms have been proposed as an alternative to conventional staging systems or even as new standards for many cancers (25, 26, 27, 28) because of their ability to generate a numerical probability of a clinical event tailored to the individual patient (29, 30), systematically and unbiasedly capturing complexity while balancing statistical modeling and risk stratification (31). Nomograms are pictorial representations of a mathematical model that incorporates multiple factors to predict a specific endpoint based on statistical methods. By including significant factors, usually determined by logistic or Cox regression, nomograms can provide an estimated probability of an event, such as death or recurrence, tailored to the profile of the individual patient (32). In PTC studies, such nomograms have been presented as accurate tools for predicting personalized clinical outcomes (disease-free survival or recurrence) to prevent potential overtreatment before or after surgery or improve the predictability of treatments before surgery, especially in cases when surgery can be avoided (33, 34), as they can account for risk factors not considered in stratified scoring systems. Moreover, nomograms have also been developed to detect central lymph node metastasis (CLNM) and lateral lymph node metastasis (LLNM) in preoperative stages. High-resolution neck ultrasound (US) and US-guided fine-needle aspiration (FNA) biopsy can significantly improve the diagnosis of PTC, but, because of the limitations of imaging technology, the detection rate of LLNM and especially LLNM before surgery is relatively low (diagnostic sensitivity: CLNM 51 to 58.3%: LLNM 27.4 to 84%) with a false-negative rate of 44.6% in both cases (35, 36, 37). Although preoperative computed tomography (CT) has been proposed to overcome the low sensitivity of US, its use remains controversial because of its high cost, radiation burden, and the stunning effect of contrast agents on iodine uptake (38). Nomograms have also been suggested to reduce the risk of persistent or recurrent CLNM and LLNM during follow-up: ATA guidelines recommend prophylactic central neck dissection in clinically node-negative PTC patients, especially for tumors larger than 4 cm or with extrathyroidal extension (ETE). However, because these guidelines are based on only two imaging features, they may be insufficient to predict a substantial number of subclinical CLMN.
Lateral lymph node dissections are not recommended unless suspicious LLNM is confirmed by preoperative imaging and FNA biopsy. However, given a prevalence of 55% of occult LLNM (39), i.e. not detectable preoperatively by imaging or clinically, many patients undergoing thyroidectomy may still have LLNM after surgery, and recurrence may occur rapidly (40). Nevertheless, lateral lymph node dissection is associated with complications such as hypoparathyroidism, neck pain, and chyle leak and has a higher complication rate (41).
Under these circumstances, appropriate and non-invasive tools such as nomograms that could quantify the risk of certain events (such as death, recurrence, CLNM, or LLNM) or prevent overdiagnosis could be helpful for the optimal management of PTC patients, but their accuracy is not well-defined yet. Therefore, this systematic review aims to evaluate the accuracy of nomograms developed to predict overall survival (OS), disease-free survival (DFS), recurrence, LLNM and CLNM in PTC patients. In particular, the following two questions are to be addressed:
-
How accurate are the nomograms in determining the risk of OS, DFS, and recurrence?
-
How accurate are the nomograms in determining the risk of CLNM and LLNM?
Materials and methods
This work was conducted according to the guidelines for systematic reviews of prognostic studies (42, 43, 44, 45). The methodological approach was registered in the PROSPERO database (CRD42022327028).
The components of the PICOT questions were: population, patients with papillary thyroid cancer; comparator prognostic factor, single-arm studies; outcomes, OS, DFS, cancer-specific survival, recurrence, CLNM, or LLNM; timing, all periods; setting, hospital setting.
Eligibility criteria
Only peer-reviewed research articles were considered. Eligible studies were selected according to the following criteria: (a) randomized controlled trials, prospective or retrospective studies; (b) studies that included patients with PTC; (c) studies that reported OS, DFS, cancer-specific survival, recurrence, CLNM, LLNM; (d) studies that presented a nomogram after independent factors were identified by logistic regression or Cox regression; (e) a nomogram was graphically reported. Repeat publications or with overlapping patients and studies that included patients with other types of thyroid malignancies, papillary thyroid microcarcinoma, previous thyroid surgery, and radiomic studies were excluded.
Databases searched
A systematic search strategy was performed in PubMed, Web of Science, and Scopus from April to June 2022 without time or language restrictions using the following keywords: (thyroid cancer OR differentiated thyroid cancer OR papillary thyroid carcinoma OR follicular thyroid cancer) AND (nomogram OR nomograms) AND (survival OR progression OR outcome OR event OR CLNM OR LLNM OR central lymph node metastasis OR lateral lymph node metastasis). Screening of titles/abstracts and removal of duplicates was performed by two independent reviewers (AC and DDA). The full texts of the remaining potentially relevant articles that met the inclusion and exclusion criteria were retrieved and reviewed by two additional independent reviewers (AV and PPO). Any disagreement was discussed until a consensus decision was reached. The final eligibility of each study was reviewed, and the reasons for exclusion were recorded. Two authors (MLG and AC) made the final selection of articles. In case of disagreement, a third experienced reviewer (LG) was consulted to reach a consensus.
Data extraction process
Two reviewers independently extracted data from the included studies according to the CHARMS-PF checklist (45) and entered them into a data sheet. All discrepancies were resolved by consensus. Study authors were not contacted to obtain unpublished data.
Data collected included (i) study characteristics: first author, year, country, observation period; (ii) sample size of the training group; (iii) validation method (internal or external validation); (iv) sample size of the validation group; (v) follow-up; (vi) number of risk factors analyzed; (vii) statistical model applied for the identified independent factors; (viii) number of independent factors included in the nomogram; (ix) outcome; (x) number of observed events related to the specific outcome; (xi) C-index or AUC (area under the curve) for the training/derivation and validation groups; (xii) O:E ratio, calibration slope or plot; and (xiii) standard error related to discrimination (AUC or c-index) and calibration (slope or O:E ratio). No data were extracted from graphs or figures.
Risk of bias assessment
The quality of predictive studies was assessed through PROBAST (Prediction model Risk Of Bias ASsessment Tool) (46, 47). This tool consists of 4 domains (participants, predictors, outcome, and analysis) containing 20 signaling questions for risk of bias assessment. The overall assessment of the risk of bias and concerns for applicability was carried out according to Moons and colleagues (46) rating. Potential disagreements were resolved through discussion and consensus among all authors.
Statistical analysis
Extracted data were entered into an electronic database by one review author (MLG) and reviewed by a second review author (PPO). In the PROSPERO registration of the present study, it was planned that if the clinical and methodological characteristics of the individual studies were sufficiently homogeneous, statistical measures of model performance (e.g. discrimination and calibration statistics) would be pooled meta-analytically across studies and a weighted pooled AUC, including the associated 95% confidence interval, would be calculated. In addition, forest plots were planned. Multivariable models could only be pooled if the same or at least very similar prognostic factors were used to fit the model. Random-effects models were to be used for the meta-analyses.
However, after data extraction, we found that the data on prognostic factors and models were too heterogeneous and poorly reported to perform a meta-analysis, and following Damen and colleagues (42), we decided not to adopt a meta-analytic approach.
Results
Search results
One thousand five hundred and sixty-nine studies were found by implementing the search strategy (Figure 1). After duplicates were excluded, 1423 articles were screened by title and abstract. Thirty-four articles met the inclusion criteria and were screened in full text. Fifteen articles were excluded because they were repeated publication/overlapped samples (n = 4), no surgical approach (n = 3), outcomes different to those considered in the study (n = 2), the nomograms were not reported (n = 2), statistical model used to determine risk factors differed from logistic regression or Cox regression (n = 2), nomograms used to evaluate the diagnostic accuracy of diffusion-weighted imaging (n = 1), and a study related to micro PTC (n = 1). Finally, 18 studies, for a total of 20 models, were included (10, 19, 20, 29, 35, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61). The studies included a total of 90,969 patients (Table 1).
Studies characteristics (No. of studies = 18, No. of models = 20).
| Author (year) | Country | Period | Type of study prediction | Validation method | Total sample enrolled | Sample size (training) | Sample size (validation) | Follow-up | Statistical model for independent factors | Outcome | No. of events |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Cao et al. (2021) (48) | China | 2000–2005 | DEV | Apparent validation | 660 | 660 | Same training set | 113.5 months | Cox regression | DFS a |
29b |
| Ding et al. (2019) (49) | China | 1997–2011 | DEV | Split-sample validation (3:1) | 1621 | 1215 | 406 | 67 months | Cox regression | DFS | 108c |
| Dou et al.
(2020) (50) |
China | 2016–2017 | DEV | Split-sample validation (3:1) | 653 | 460 | 193 | NR | Logistic regression | LLNM level II and levels III–IV |
LLNM level II: DEV: 72/VAL: 32 LLNM levels III–IV: DEV: 186/VAL: 71 |
| Feng et al. (2021) (35) | China | 2019–2020 | DEV | Split-sample validation | 886 | 617 | 269 | NR | Logistic regression | CLNM | Training set: 307 - validation set: 131 |
| Gao et al. (2022) (51) | China | 2019–2020 | DEV | Apparent validation | 296 | 296 | Same training set: INT VAL 1: 100 pz; INT VAL 2: 95 pz |
NR | Logistic regression | CLNM | 112/296 |
| Ge et al.
(2017) (19) |
China |
Training cohort: 2000–2009; validation cohort: Jan 2010–Dec 2010 |
DEV & VAL | External validation | 1369 | 1034 | 335 | Training cohort: 145.83 months (5–224 months); validation cohort: 71 months (6–78 months) |
Cox regression | RRFS | Training set: 120d; validation set: 36e |
| DRFS | Training set: 46f; validation set: 20g |
||||||||||
| Heng et al. (2020) (52) | China | 2017–2019 | DEV | Bootstrap validation | 434 | 434 | Same training set | NR | Logistic regression | LLNM | 142 |
| Hu et al. (2022) (10) | China | 2016–2019 | DEV | Apparent validation | 418 | 418 | Same training set | NR | Logistic regression | CLNM | 144h |
| Jianyong et al. (2018) (53) | China | 2013–2015 | DEV & VAL | External validation | 1788 | 896 | 896 + 306 | NR | Logistic regression | RECURRENCE | NR |
| Kim et al. (2016) (54) | South Korea | 1997–2016 | DEV & VAL | Split-sample validation & external validation | 13,277 | 7535 | INT VAL: 3228; EXT VAL: 2514 |
NR | Logistic regression | CLNM | Training set: 4217; internal validation set: 1806; external validation set: 987 |
| Lin et al. (2021) (55) | China | 2016–2021 | DEV | Bootstrap validation | 423 | 423 | Same training set | NR | Logistic regression | CLNM | 57 |
| Liu et al. (2019) (20) | China | SEER (2004–2013) | DEV | Split-sample validation (1:1) | 63,219 | 31,610 | 31,609 | 68 months (1–143 months) | Logistic regression | OS | 2015 |
| CSS | 545 | ||||||||||
| Liu et al. (2021) (56) | China | 2016–2018 | DEV | Apparent validation | 1198 | 1198 | Same training set | NR | Logistic regression | LLNM | 177 |
| Qi et al. (2021) (57) | China | 2017–2021 | DEV | Split-sample validation | 485 | 388 | 97 | NR | Logistic regression | DLNM | 98 |
| Sun et al. (2021) (58) | China | Training and INT VAL cohort: 2017–2019; EXT VAL cohort: 2017–2019 |
DEV & VAL | Cross-validation & external validation | 1991 | 1585 | 406 | NR | Logistic regression | CLNM | Training set: 483/1094; internal validation: 219/491; external validation: 222/406 |
| Tan et al. (2021) (29) | China | 2016–2019 | DEV | Apparent validation | 250 | 250 | Same training set | NR | Logistic regression | CCNLNM | 75/157 |
| Yang et al. (2020) (60) | China | 2016–2019 | DEV & VAL | Bootstrap validation + external validation | 1438 | 1252 | 186 | NR | Logistic regression | CLNM | Training set + internal validation set: 618; external validation set: 53% overall rate of nodal metastases |
| Zhuo et al. (2021) (61) | China | 2017–2019 | DEV | Split-sample validation | 253 | 138 | 115 | NR | Logistic regression | LLNM | 47 |
aLocal recurrence, distant metastasis, or death.
bEight local recurrence, 15 distant metastases, and 6 deaths.
cSixty-six lymph node recurrence and 42 non-lymph node recurrence.
dTwenty-eight residual thyroid recurrence, 59 lymph node recurrence, 26 residual thyroid and lymph node recurrence, and 7 other less common sites of recurrence.
eNine residual thyroid recurrence, 20 lymph node recurrence, 6 both residual thyroid and lymph node recurrence, and 1 other less common sites recurrence.
fThirty-seven lung metastasis recurrence, two bone metastasis recurrence, one brain metastasis recurrence, four both lung and bone metastasis recurrence, and two both lung and brain metastasis recurrence.
gSeven lung metastasis recurrence, one bone metastasis recurrence, one brain metastasis recurrence, two both lung and bone metastasis recurrence, and one both lung and brain metastasis recurrence.
hNinety-nine central lymph node metastasis, 28 central and lateral lymph node metastasis, and 17 skip metastasis of LLNM.
CCLNM, contralateral central lymph node metastases; CLNM, central lymph node metastasis, CSS, cancer-specific survival; DEP & VAL, development and validation; DEV, development; DFS, disease-free survival; DLNM, Delphian lymph node metastasis; DRFS, distant recurrence-free survival; EXT VAL, external validation; INT VAL, internal validation; LLNM, lateral lymph node metastasis; NR, not reported; OS, overall survival; RRFS, regional recurrence-free survival; VAL, validation.
Studies description
More than 50,000 patients were enrolled in the training set, 44,138 were used for validation of whom 3412 were in the external validation. Seventeen of the 19 studies were conducted in China (10, 19, 20, 29, 35, 48, 49, 50, 51, 52, 53, 55, 56, 57, 58, 60, 61), except for Kim's study, which was conducted in South Korea (54). Two studies began in 1997 and were completed in 2011 (49) and 2016 (54). The other studies spanned a period between 2000 and 2020. Thirteen studies applied an internal validation procedure (10, 20, 29, 35, 48, 49, 50, 51, 52, 55, 56, 57, 61). Specifically, five studies used an apparent validation (10, 29, 48, 51, 56), six studies used split-sample validation with different ratios (20, 35, 49, 50, 61), and the remaining two studies used bootstrap validation (52, 55). Five studies with a total of six models applied external validation (19, 53, 54, 58, 60). Only four studies (19, 20, 48, 49) reported follow-up. A total of 220 factors were considered to identify independent factors to be included in the nomograms. Age, sex, tumor size, ETE, nodal status, bilaterality, and multifocality were analyzed in almost all included studies. The complete list of risk factors for each study is shown in Table 2. Fifteen studies used multivariate logistic regression to identify factors after univariate logistic regression was previously performed (10, 19, 20, 29, 35, 50, 51, 52, 53, 54, 55, 56, 57, 58, 60, 61), whereas only three studies used multivariate Cox regression analysis (19, 48, 49). Overall, 132 identified factors using a P-value below a predetermined value (0.1 or 0.05) were reported in nomograms, with a mean of 6 factors for each nomogram. A full description of the nomogram risk factors is provided in Table 3. Calibration analysis was performed using calibration plots in 15 studies. Slope and/or intercept of the calibration were not reported in any of the included studies.
Description of risk factors included in univariate models to select independent factors for nomogram.
| Author | Risk factors analyzed in univariate and multivariate models (Cox regression or logistic regression) | No. of risk factors |
|---|---|---|
| Cao et al. (2021) (48) | Gender, age (≤45 vs >45, or ≤55 vs > 55), family history, cN staging (cN0 vs cN1), tumor invasion, multifocal, bilateral, T staging (T1 vs T2 vs T3 vs T4), total thyroidectomy, surgical residue, lateral cervical lymph node dissection, extra-lymph node invasion, maximum diameter of metastatic lymph nodes (no lymph node metastasis vs <0.2 cm vs 0.2–3 cm vs >3 cm vs unknown), number of lymph node metastasis (0 vs 0–4 vs ≥5 vs unknown), number of LNM in central region (0 vs 0–4 vs ≥5 vs unknown), number of LNM in lateral region (0 vs 0–4 vs ≥5 vs unknown) | 16 |
| Ding et al. (2019) (49) | Age (<30 vs ≥30), sex, bilaterality, tumor size (11–20 mm vs ≤10 mm, or 20 mm vs ≤ 10 mm), ETE, nodal status (N1a vs N0/Nx, or N1b vs N0/Nx) | 6 |
| Dou et al. (2020) (50) | Age (<55 vs ≥55), size (<10, 10–20, >20 cm), gender, location (upper vs middle vs lower vs whole), left vs right, bilaterality, HT, multifocality, ETE, prelaryngeal (0 vs 1–2 vs ≥3), pretracheal (0 vs 1–2 vs ≥3), paratracheal (0 vs 1–2 vs ≥3) | 12 |
| Feng et al. (2021) (35) | Sex, age (≥55 vs <55), BMI (normal vs overweight), diabetes, BRAF V600E mutation, CLT, maximum tumor size (≤1 cm vs >1 to ≤2 cm vs >2 to ≤4 cm vs >4 cm), number of foci (1 vs 2 vs ≥3), multifocality (solitary vs unilateral vs bilateral), location (upper vs middle/lower), nodular composition (cystic or spongiform vs mixed cystic and solid vs solid), echogenity (anechoic vs hyperechoic or isoechoic vs very hypoechoic), A/T (≤1 vs >1), margin (smooth vs lobulated or irregular vs ETE), echogenic foci (none or large comet-tail artifacts vs macrocalcifications vs peripheral calcifications vs punctate echogenic foci), no. of removed LNs in CC (≥6 vs <6) | 16 |
| Gao et al. (2022) (51) | Age (≤40 vs >40), sex, tumor size (≤1.0 vs >1.0 cm), aspect ratio (<1 vs ≥1), tumor location (left vs right vs isthmic), capsule contact, microcalcification, boundary (unclear vs clear), morphology (irregular vs regular), low echo, BRAFV600E (mutation vs wild), blood flow signal (poor vs rich), TSH (negative vs high), Tg (low vs normal vs high), TgAb (positive vs negative), TPOAb (positive vs negative) | 16 |
| Ge et al. (2017) (19) | Age, sex, family history, histological variants (mixed vs follicular vs classic), T staging (T4 vs T3 vs T2 vs T1), N staging (N1b vs N1a vs N0/Nx), tumor number (multiple vs solitary), maximal tumor diameter, capsule invasion, perineuronal invasion, intrathyroidal dissemination, vascular invasion, Hashimoto thyroiditis | 13 |
| Heng et al. (2020) (52) | Bilateral PTC, carcinoembryonic antigen (≤1.5 vs <1.5), Cr serum creatinine (≥70 vs <70), TSH (≥2.35 vs <2.35), red blood cell count (≥5 vs <5), monocyte (≥0.35 vs <0.35), neutrophil (≥3.5 vs 3.5), white blood cell count (≥6 vs <6), maximum tumor diameter (≥1 cm vs <1 cm), thyroid capsular invasion, ipsilateral nodular goiter, ipsilateral HT, BMI (>23 vs ≤23), multifocality, age (>40 vs ≤40) Note: thyroid function tests feature: TSH (reference: 0.35–5.00 mIU/mL), TgAb (reference: 0.00–115.00 IU/mL), TPOAb (reference: 0.00–34.00 IU/mL), Tg (reference: 1.40–78.00 mg/L). |
15 |
| Hu et al. (2022) (10) | Sex, age, TPOAb, TSH, T3, T4, FT3, FT4, multifocality, galectin, CK19, CK34, bilaterality, position, Hashimoto's thyroiditis, US suggested CLNM, tumor size, component, echogenicity, taller than wide, margin, boundary, microcalcification, extracapsular invasion | 24 |
| Jianyong et al. (2018) (53) | Age (≤45 vs >45, ≤55 vs >55), gender, prime location (middle + low vs upper), BTAF V600E mutation (positive vs negative), tumor number (single vs multiple), tumor size (≤1 cm vs >1 cm, or <2 cm vs ≥2 cm, or <4 cm vs ≥4 cm), T classification (T1 + T2 vs T3.+ T4), pN classification (N0 vs N1a+N1b, or N1a vs N1b), LN capsular invasion (absence vs presence), LN organ invasion (absence vs presence), metastatic LN number (≤5 excluding pN0 cases vs >5, ≤5 including pN0 cases vs >5), Largest metastatic LN diameter (≤3 excluding pN0 cases vs >3, ≤3 including pN0 cases vs >3) | 12 |
| Kim et al. (2016) (54) | Age (20–30 vs 30–40 vs 40–50 vs 50–60 vs 60–70 vs 70–80), sex, tumor size ( ≤0.5 cm, 0.5–1.0 cm, 1.0–2.0 cm, 2.0–4.0 cm, >4.0 cm), multiplicity, bilaterality, ETE, CLT | 7 |
| Lin et al. (2021) (55) | Age (≤35 vs >35), sex, BRAFV600E mutated, HT, multiple suspicious malignant foci (no focus with TIRADS of 4c/5 vs single focus with TIRADS of 4c/5 vs multiple foci with TIRADS of 4c/5), TIRADS (4a vs 4b vs 4c vs 5), diameter, solid composition, A/T > 1, echogenicity (isoechogenicity vs hyperechogenicity vs hypoechogenicity vs marked hypoechogenicity), margin (well-circumscribed, irregular, microlobulated), calcification (none vs microcalcification vs macrocalcification vs rim calcification), vascular pattern (absence vs perinodular vs intramodular vs perinodualr and intramodular), SMF at inferior part of thyroid lobe, SMF at superior part of thyroid lobe | 15 |
| Liu et al. (2019) (20) | Age, sex, race, marital status, tumor size, ETE, multifocality, surgery, radioactive iodine, T stage, N stage, M stage | 12 |
| Liu et al. (2021) (56) | Gender, age (≤55 vs 55), tumor size (<10 vs ≥10), multifocality, capsule invasion, HT, BRAFV600E protein status (negative vs positive), LNN (<3 vs ≥3), LNR (<0.565 vs ≥0.565) | 9 |
| Qi et al. (2021) (57) | Gender, age (<45 vs ≥45), location (upper vs middle vs lower vs isthmus vs full), multifocality, bilaterality, tumor size (<1.3 vs ≥1.3), shape (regular vs irregular), microcalcification, US-ETE, P-ETE, nodular goiter, US-CLNM, CLNM | 13 |
| Sun et al. (2021) (27) | Gender, age, diameter, location, position (S-I vs I-E vs V-D), composition, boundary, margin, echogenity, echo uniformity, shape, calcification, A/P, vascularization | 14 |
| Tan et al. (2021) (29) | Gender, HT, tumor size (<8.55 vs ≥8.55 mm), hypoechoic on neck US, microcalcification on neck US, capsular invasion, ETE, number of ipsi-CLNM (<1.5 vs ≥1.5), ratio of ipsi-CLNM (<0.16 vs ≥0.16), present ipsi-LLNM | 10 |
| Yang et al. (2020) (60) | BIL, CEA, Cr, CA2+, RBC, Mon, Neu, WBC, multifocality, MTD, TCI, with iNG, with iHT, BMI, age | 15 |
| Zhou et al. (2021) (61) | Ultrasound LN location (negative, left vs right, bilateral), LN smallest diameter (<2.6 vs ≥2.6 mm), LN maximum diameter (<7.5 vs ≥7.5 mm), cystic changes, LN calcification (negative, small vs big), LN vascularity (little/no vs rich), vascularity type (negative, peripheral vs central vs both) Computer tomography LN diameter (≤10.5 vs >10.5 mm), LN shape (regular vs irregular), LN boundary (clear vs unclear), cystic changes, LN calcification (negative, small vs big), LN fusion, LN enhancement, uneven enhancement |
13 |
A/P, capsular abutment-to-lesion perimeter ratio; A/T, the anteroposterior dimension divided by its transverse dimension; BMI, body mass index, CLNM, central cervical lymph node metastasis; CLT, chronic lymphocytic thyroiditis; ETE, extrathyroidal extension; HT, Hashimoto thyroiditis; I-E, interior-middle-exterior; LLNM, lateral lymph node metastasis; LN, lymph node; LNM, lymph node metastasis; S-I, superior-middle-inferior; V-D, ventral-middle-dorsal.
Description of risk factors included in nomograms (n = 20).
| Author | Risk factors included in nomogram | No. of factors included | Outcome |
|---|---|---|---|
| Cao et al. (2021) (48) | Age, cN, T stage, LLND, LN size, Ln number, LLN, CLN | 8 | DSF |
| Ding et al. (2019) (49) | Age, bilaterality, tumor size, nodal status | 4 | DSF |
| Dou et al. (2020) (50) | Size, location, prelaryngeal, paratracheal | 4 | LLNM level II |
| Size, location, prelaryngeal, paratracheal, pretracheal | 5 | LLNM levels-III/IV | |
| Feng et al. (2021) (35) | Sex, CLT, size, No. of foci, margin, location | 6 | CLNM |
| Gao et al. (2022) (51) | Tumor size, microcalcification, TgAb | 3 | CLNM |
| Ge et al. (2017) (19) | Family history, N staging, capsule invasion | 4 | RRFS |
| Family history, histological variants, capsule invasion, perineuronal invasion, vascular invasion | 5 | DRFS | |
| Heng et al. (2020) (52) | Age, thyroid capsular invasion, maximum tumor diameter, ipsilateral NG | 4 | LLNM |
| Hu et al. (2022) (10) | Age, sex, multifocality, US suggested CLNM, tumor size, extracapsular invasion | 7 | CLNM |
| Jianyong et al. (2018) (53) | PTC nodule number, largest PTC nodule diameter, lymph node metastatic number, largest lymph node diameter, lymph node invasion grade | 5 | Recurrence |
| Kim et al. (2016) (54) | Age, sex, tumor size, multiplicity, bilaterality, extrathyroidal extension, chronic lymphocytic thyroiditis | 7 | CLNM |
| Lin et al. (2021) (55) | BRAF with V600E mutated, age, calcification, diameter | 4 | CLNM |
| Liu et al. (2019) (20) | Age, sex, race, marital status, T stage, M stage, radioactive iodine, tumor size, extrathyroidal extension | 9 | OS |
| Age, sex, marital status, T stage, N stage, M stage, tumor size, extrathyroidal extension | 8 | CSS | |
| Liu et al. (2021) (56) | Size, BRAFV600E protein status, metastatic central lymph node, ratio of metastatic central lymph node | 4 | LLNM |
| Qi et al. (2021) (57) | Location, CLNM, bilaterality, PETE, NG, gender, age | 7 | DLNM |
| Location, USCLNM, bilaterality, shape, USETE, NG, gender, age | 8 | DLNM | |
| Location, CLNM, bilaterality, shape, USETE, NG, gender, age | 8 | DLNM | |
| Sun et al. (2021) (27) | Diameter, shape, calcification, A/P | 4 | CLNM |
| Tan et al. (2021) (29) | Ratio of ipsi-CLNM, sex, capsule invasion, HT, ipsi-LLNM | 5 | CCLNM |
| Yang et al. (2020) (60) | Thyroid capsular invasion, multifocality, creatinine, age, diameter, BMI, CEA | 7 | CLNM |
| Zhou et al. (2021) (61) | Sex, tumor size, tumor shape, nodules, lymph lode location (left, right, bilateral), lymph node vascularity | 6 | LLNM |
CCLNM, contralateral central neck lymph node metastasis; CLNM, central lymph node metastasis; DLNM, Delphian lymph node metastasis; NG, nodular goiter; PETE, pathology-based extrathyroidal extension; USCLNM, ultrasonic-based central lymph node metastasis; USETE, ultrasonic-based extrathyroidal extension.
Risk of bias assessment
According to PROBAST, 15 models had a high risk of bias (10, 19, 20, 29, 48, 49, 50, 51, 52, 53, 55, 57, 61), 4 had an unclear risk of bias (60), and only 1 had a low risk of bias, implying that the predictive performance of many studies could probably be lower if used in clinical practice (Table 4 and Figure 2). The highest level of ROB was observed in the analysis domain. The lack of adequate numbers of participants, inappropriate handling of categorical predictors (conversion of continuous to categorical variables without a prespecified method), and missing values were the most common problems. Thirteen models had an unclear risk of overfitting because no internal validation was performed, or they consisted of only a single random split-sample of participant data. Unclear information was also provided about the predictors and regression coefficients in the final models: it was difficult to determine whether the predictors and regression coefficients in the final model matched the reported results of the multivariable analysis.
PROBAST risk of bias for all included models.
Citation: Endocrine Connections 12, 4; 10.1530/EC-22-0457
PROBAST.
| Study | ROB | Applicability | Overall | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Participants | Predictors | Outcome | Analysis | Participants | Predictors | Outcome | ROB | Applicability | |
| Cao et al. (2021) (48) | ? | + | − | − | + | + | ? | − | ? |
| Ding et al. (2019) (49) | + | + | − | ? | + | + | ? | − | ? |
| Dou et al. (2020) (50) | + | + | + | − | + | + | + | − | + |
| Feng et al. (2021) (35) | + | + | + | + | + | + | + | + | + |
| Gao et al. (2022) (51) | + | + | + | − | + | + | + | − | + |
| Ge et al. (2017 A)a (19) | ? | + | − | − | + | + | − | − | − |
| Ge et al. (2017 B)a (19) | ? | + | − | − | + | + | − | − | − |
| Heng et al. (2020) (52) | + | + | + | − | + | + | + | − | + |
| Hu et al. (2022) (10) | + | + | + | − | + | + | + | − | + |
| Jianyong et al. (2018)a (53) | + | + | − | − | + | + | − | − | − |
| Kim et al. (2017)a (71) | + | + | + | ? | + | + | + | ? | + |
| Lin et al. (2021) (55) | + | + | + | − | + | + | + | − | + |
| Liu et al. (2019 A) (20) | + | + | + | − | + | + | + | - | + |
| Liu et al. (2019 B) (20) | + | + | − | − | + | + | ? | − | ? |
| Liu et al. (2021) (56) | + | + | + | ? | + | + | + | ? | + |
| Qi et al. (2021) (57) | ? | + | − | − | + | + | − | − | − |
| Sun et al. (2021)a (27) | + | + | + | ? | + | + | + | ? | + |
| Tan et al. (2021) (29) | + | + | + | − | + | + | + | − | + |
| Yang et al. (2020)a (60) | + | + | + | ? | + | + | + | ? | + |
| Zhuo et al. (2021) (61) | + | + | + | − | + | + | + | − | + |
+ indicates low ROB/low concern regarding applicability; − indicates high ROB/high concern regarding applicability; ? indicates unclear ROB/unclear concern regarding applicability – rating criteria by Moons and colleagues (19) – Table 11.
aStudy which was externally validated using an independent dataset.
PROBAST, prediction model risk of bias assessment tool; ROB, risk of bias.
External validation
Five studies (six models) were developed and externally validated using an independent dataset (19, 53, 54, 58, 60): three studies were rated at high RoB and the remaining three models were rated at unclear RoB. A common concern of all models was that the outcome was determined when the predictor was known; according to Moons and colleagues (46), models in which knowledge of the predictor did not affect the outcome were considered low-risk (54, 58, 60, 62), whereas those in which the outcome was recurrence were considered high-risk (19, 53). One study had fewer than 100 events (19) and one study probably did not have sufficient number of events (53). For all models, no information was provided on how to handle missing data and how to appropriately account for complexities.
Disease-free survival and recurrence
Five studies with a total of 68,967 patients developed nomograms for predicting DFS, cancer-free survival, or recurrence. The intended use of these models was clear. The C-index for the internal validation sets ranged from 0.70 (95% CI: 0.64–0.76) (49) to 0.92 (95% CI: 0.91–0.94) (20), while that for the validation set ranged from 0.65 (95% CI: 0.55–0.75) (49) to 0.92 (95% CI: 0.91–0.95) (20). The number of factors included in the final models ranged from four factors (49) to eight factors (20, 48). Logistic regression was used in two models, whereas three models were developed using Cox regression. In one study, a specific analysis was developed to distinguish between regional and distant recurrence-free survival. A higher C-index was found for predicting distant recurrence-free survival (0.83, 95% CI: 0.79–0.87). The study was externally validated in a sample of 335 patients and yielded a C-index of 0.89 (95% CI: 0.82–0.97) (19). Jianyong and colleagues (53) also validated using an external dataset and achieved a C-index of 0.78 (95% CI: 0.72–0.85) for predicting recurrence (53). Details of the risk factors included in each nomogram are shown in Table 5.
Development and validation results in studies predicting disease-free survival and recurrence (n = 5).
| Author (year) | Validation method | Total sample enrolled | Sample size (training) | Sample size (validation) | No. of risk factors analyzed | Statistical model for independent factors | No. of factors included in nomogram | Outcome | No. of events | C-index | Calibration Plot | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TRS | INT VAL | EXT VAL | |||||||||||
| Cao et al. (2021) (48) | Apparent validation | 660 | 660 | Validation using the same training set | 18 | Cox regression | 8 | DFS | 29 | - | 0.71 (95% CI: 0.57–0.84) |
NA | No |
| Ding et al. (2019) (49) | Split-sample validation (3:1) | 1621 | 1215 | 406 | 6 | Cox regression | 4 | DFS | 108 | 0.70 (95% CI: 0.64–076) |
0.65 (95% CI: 0.55–0.75) |
NA | Yes |
| Ge et al. (2017) (19) | External validation | 1369 | 1034 | 335 | 13 | Cox regression multivariate analysis | 4 | RRFS | Training set: 120; validation set: 36 |
- | 0.72 (95% CI: 0.70–0.75) |
0.72 (95% CI: 0.64–0.81) |
Yes |
| 5 | DRFS | Training set: 46; validation set: 20 |
- | 0.83 (95% CI: 0.79–0.87) |
0.89 (95% CI: 0.82–0.97) |
Yes | |||||||
| Jianyong et al. (2018) (53) | External validation | 1788 | 896 | Internal validation: 896; external validation: 306 |
13 | Logistic regression | 5 | Recurrence | NR | NA | 0.81 (95% CI: 0.71–0.89) | 0.78 (95% CI: 0.72–0.85) |
No |
| Liu et al. (2019) (20) | Split-sample validation (1:1) | 63,219 | 31,610 | 31,609 | 12 | Logistic regression | 9 | OS | 2015 | 0.776 (95% CI: 0.770–0.792) |
0.77 (95% CI: 0.753–0.787) |
NA | Yes |
| 8 | CSS | 545 | 0.924 (95% CI: 0.907–0.941) |
0.925 (95% CI: 0.905–0.945) |
NA | Yes | |||||||
CSS, cancer-specific survival; DFS, disease-free survival; DRFS, distant recurrence-free survival; NA, not applicable; NR, not reported; RRFS, regional recurrence-free survival; VAL, validation.
Overall survival
Only one study developed a model to predict OS in PTC patients. The study was performed on a sample of 63,219 patients (median follow-up: 68 months, range: 1–143 months). The data set was divided into two samples (1:1 ratio). Logistic regression was used to identify predictive factors, nine of which were included in the final model. A total of 2015 patients died during the follow-up period. The C-index was 0.77 (95% CI: 0.75–0.79). Full details of the studies are shown in Table 6.
Development and validation results in the study predicting overall survival and recurrence (n = 1).
| Author (year) | Validation method | Total sample enrolled | Sample size (training) | Sample size (validation) | No. of risk factors analyzed | Statistical model for independent factors | No. of factors included in nomogram | Outcome | No. of events | C-index | Calibration plot | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TRS | INT VAL | EXT VAL | |||||||||||
| Liu et al. (2019) (20) | Split-sample validation (1:1) | 63,219 | 31,610 | 31,609 | 12 | Logistic regression | 9 | OS | 2015 | 0.776 (95% CI: 0.770–0.792) |
0.77 (95% CI: 0.753–0.787) |
NA | Yes |
NA, not applicable; NR, not reported.
Lateral lymph node metastasis
Four studies with a total of 2538 patients developed a model for predicting LLNM. The intended use of these models was clear. None of the studies used an external dataset for validation. Two studies split the initial sample (3:1 ratio). The number of predictive factors, analyzed by univariate logistic regression, ranged from 8 (56) to 21 factors (61). The C-index for the internal validation sets ranged from 0.72 to 0.92 (95% CI: 0.86–0.97). One study analyzed two different outcomes (LLNM level II, and two for LLNM levels III–IV) for a total of four models (including and excluding the subregions of CLNM). The C-index was higher for for predicting LLNM level II (C-index 0.835, 95% CI: 0.79–0.86) and LLNM levels III–IV (C-index: 0.855, 95% CI: 0.82–0.87) in the models that excluded CLNM among prognostic factors (50). The complete results are shown in Table 7.
Development and validation results in studies predicting lateral lymph node metastasis (n = 4).
| Author (year) | Validation method | Total sample enrolled | Sample size (training) | Sample size (validation) | No. of risk factors analyzed | Statistical model for independent factors | No. of factors included in nomogram | Outcome | No. of events | C-index | Calibration plot | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TRS | INT VAL | EXT VAL | |||||||||||
| Dou et al. (2020) (50) | Split-sample validation (3:1) | 653 | 460 | 193 | 11 | Logistic regression | 4 | LLNM level II lymph node metastasis |
Dev. model: 72; valid. model: 32 |
- | 0.795 (95% CI: 0.78–0.83) |
NA | No |
| 4 | Dev. model: 72; valid. model: 32 |
- | 0.835 (95% CI: 0.79–0.86) |
NA | No | ||||||||
| 5 | LLNM levels III/IV lymph node metastasis |
Dev. model: 186; valid. model: 71 |
- | 0.835 (95% CI: 0.78–0.85) |
NA | No | |||||||
| 5 | Dev. model: 186; valid. model: 71 |
- | 0.855 (95% CI: 0.82–0.87) |
NA | No | ||||||||
| Heng et al. (2020) (52) | Bootstrap validation | 434 | 434 | 434 | 15 | Logistic regression | 4 | LLNM | 142 | 0.761 (95% CI: 0.707–0.815) |
0.759 (95% CI: 0.745–0.773) | NA | Yes |
| Liu et al. (2021) (55) | Apparent validation | 1198 | 1198 | 1198 | 8 | Logistic regression | 4 | LLNM | 177 | - | 0.714 (95% CI NR) | NA | Yes |
| Zhuo et al. (2021) (61) | Split-sample validation | 253 | 138 | 115 | 21 | Logistic regression | 6 | LLNM | 47 | - | 0.915 (95% CI: 0.862–0.967) |
NA | No |
NA, not applicable; NR, not reported.
Central lymph node metastasis
Nine studies (11 models) with a total of 19,464 patients were identified to predict CLNM risk. All studies used a logistic regression approach to identify potential prognostic factors, the number of which ranged from 7 (48, 54) to 24 factors (10). The final models included a mean of seven factors (s.d. ± 1.8). Three studies were externally validated using a total of 3106 patients with PTC. The C-index of these externally validated studies ranged from 0.706 (95% CI: 0.685–0.727) (54) to 0.923 (95% CI: 0.893–0.946) (58). For studies that were not externally validated, the C-index ranged from 0.715 (confidence interval not reported in the study) to 0.94 (95% CI: 0.888–0.991). Feng and colleagues (35), who validated their model using internal validation after splitting the sample 3:1, found a C-index of 0.558 (95% CI: 0.542–0.570) for the nomogram applied to the validation dataset of 269 PTC patients. Full details of the models predicting CLNM are shown in Table 8.
Development and validation results in studies predicting central lymph node metastasis (n = 9).
| Author (year) | Validation method | Total sample enrolled | Sample size (training) | Sample size (validation) | No. of risk factors analyzed | Statistical model for independent factors | No. of factors included in nomogram | Outcome | No. of events | C-index | Calibration plot | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TRS | INT VAL | EXT VAL | |||||||||||
| Feng et al. (2021) (35) | Split-sample validation | 886 | 617 | 269 | 16 | Logistic regression | 6 | CLNM | Training set: 307; validation set: 131 | 0.806 (95% CI: 0.771–0.825) |
0.558 (95% CI: 0.542–0.570) |
NA | Yes |
| Gao et al. (2022) (51) | Apparent validation | 296 | 296 | Validation 1: 100; validation 2: 95 | 7 | Logistic regression | 3 | CLNM | 112/296 | 0.715 (95% CI NR) | Validation 1: 0.718; validation 2: 0.738 | NA | Yes |
| Hu et al. (2022) (10) | Apparent validation | 418 | 418 | Validation using the same training set | 24 | Logistic regression | 7 | CLNM | 144 | - | 0.94 (95% CI: 0.888–0.991) |
NA | Yes |
| Kim et al. (2016) (54) | Split-sample validation & external validation | 13277 | Training dataset: 7535 | INT VAL: 3228; EXT VAL: 2514 |
7 | Logistic regression | 7 | CLNM | Training set: 4217; internal validation set: 1806; external validation set: 987 | 0.721 (95% CI: 0.709–0.732) |
0.706 (95% CI: 0.688–0.724) |
0.706 (95% CI: 0.685–0.727) |
Yes |
| Lin et al. (2021) (55) | Bootstrap validation | 423 | 423 | Same training set | 15 | Logistic regression | 4 | CLNM | 57 | - | 0.821 (95% CI: 0.768–0.875) |
NA | Yes |
| Qi et al. (2021) (57) | Split-sample validation (3:1) | 485 | 388 | 97 | 9 | Logistic regression | 7 | DLNM (Delphian lymph node metastasis) |
98 | 0.849 (95% CI: 0.804–0.893) |
0.786 (95% CI: 0.667–0.905) |
NA | Yes |
| 11 | 8 | 0.897 (95% CI: 0.860–0.934) |
0.868 (95% CI: 0.793–0.943) |
NA | Yes | ||||||||
| 13 | 8 | 0.896 (95% CI: 0.860–0.932) |
0.877 (95% CI: 0.786–0.967) |
NA | Yes | ||||||||
| Sun et al. (2021) (27) | Cross-validation & external validation | 1991 | 1585 | 406 | 14 | Logistic regression | 4 | CLNM | Training set: 483/1094; INT VAL: 219/491; EXT VAL: 222/406 |
0.919 (95% CI: 0.902–0.935) | 0.921 (95% CI: 0.893–0.943) | 0.923 (95% CI: 0.893–0.946) |
Yes |
| Tan et al. (2021) (29) | Apparent validation | 250 | 250 | 250 | 10 | Logistic regression | 5 | CCNLNM contralateral central neck lymph node metastases in uni-PTC patients |
75/157 | - | 0.881 (95% CI: 0.840–0.923) |
NA | Yes |
| Yang et al. (2020) (60) | Bootstrap validation + external validation | 1438 | 1252 | 186 | 15 | Logistic regression | 7 | CLNM | INT VAL: 618; EXT VAL: 53% overall rate of nodal metastases | 0.857 (95% CI: 0.821–0.894) |
0.854 (95% CI: 0.843–0.867) |
0.825 (95% CI: 0.793–0.857) |
Yes |
CSS, cancer-specific survival; DFS, disease-free survival; DRFS, distant recurrence-free survival; EXT VAL: external validation; INT VAL: internal validation; NA, not applicable; NR, not reported; RRFS, regional recurrence-free survival; TRS, training set; VAL, validation.
Discussion
PTC is the most common form of DTC. It tends to metastasize to the cervical lymph nodes, so the central compartment lymph nodes are the first to be involved in PTC recurrence (50). This systematic review aimed to evaluate the accuracy of prognostic nomograms in predicting OS, DFS, recurrence, LLNM, and CLNM in PTC patients.
For OS, only one model was identified: the accuracy of the model was set at 0.77 and it was not externally validated.
Moderate discriminatory accuracy was found for nomograms predicting recurrence, LLNM, and CLNM. The discrepancy between nomogram development and external validation was large. Of 20 nomograms, only 6 had external validation and none of the models was validated externally for LLNM. The lack of external validation represents a common problem in many studies using predictive models (63, 64) and is a significant drawback for generalizing their results and discourages using nomograms in clinical practice.
The criteria adopted for patient selection were heterogeneous, ranging from low-risk patients to those at very high risk (19, 35, 48, 49, 52, 53, 57, 61) without prior stratification. Only one study stratified the risk of LLNM according to the presence or absence of CCLM (56). In addition, although based on a large number of possible risk factors (6 to 24), nomograms were constructed with a maximum of nine factors, of which only three were nearly common independent factors in all nomograms: tumor size (8/10), age (5/10), and sex (4/10). Other predictors such as multifocality, ETE, or nodal status were mentioned as possible factors in almost all studies but were not always consistently listed in the included nomograms because they were not statistically significant in multivariable models. Some models considered biochemical analysis (i.e. thyroglobulin and thyroglobulin antibodies measurement), but their use appeared very limited to very few studies.
Focusing on two of the three most reported factors, some heterogeneity was found in the categorization methods. Tumor size, one of the pillars for TNM staging, was already demonstrated to be correlated with aggressive behavior: it has been reported that the larger the tumor, the more metastases occur (65, 66). Although the relationship between tumor size and recurrence and tumor size and CLNM or LLNM seems to be well established, the optimal threshold continues to be controversial also in the considered predictive nomograms (67, 68, 69, 70, 71). In addition, some studies, such as that of Chen and colleagues (26), paradoxically found a higher CLNM risk in tumor size ≤1 cm (26).
In the included studies, tumor size was reported with different classifications, from the simplest comparison between ≤1 and >1 cm to more analytical (≤0.5, 0.5–1.0, 1.0–2.0, 2.0–4.0, >4.0 cm), thus leading to higher heterogeneity.
The age range, although defined by the 7th (≤45 years vs >45 years) and the most recent 8th (≤55 years vs >55 years) AJCC TNM staging criteria (72), was adjusted in 4 of 10 studies with different cut-off values (≤30 years vs >30 years, ≤35 years vs >35 years, <40 years vs ≥40 years). Traditionally, patients older than 45 years have a slightly worse prognosis (72), whereas CLNM and LLNM incidence, although controversial, appears to be higher in younger patients (10, 62, 65, 73, 74, 75).
No heterogeneity was instead found regarding the role of sex: while it is well-established the higher incidence of PTC in women than in men, such that the female-to-male ratio is approximately 3–3.7:1 (35), men are at higher risk of CLNM (2.8 times) and LLNM (4.6 times) than women, respectively, likely because they are more frequently engaged in unhealthy behaviobrs such as smoking and alcohol consumption (10), and have a basal metabolism that could potentially accelerate tumor spread (76, 77, 78).
The methodology of the nomograms suffered from several limitations. First, all studies were retrospective cohorts with the consequent risk of selection bias. Second, none of the nomograms accounted for competing risk, which may lead to overestimating the true event rates (79). Third, continuous variables, such as age or tumor size, were converted into categorical ones, increasing the risk of bias. Fourth, when defined, the optimal cut-off value of the nomogram was set at 123 points (54): the predictive probability of such nomogram appeared low (59.8%), although it exceeded the sensitivity of ultrasonography or CT (54). Fifth, no meaningful data were reported on calibration: although the calibration is essential to assess the utility for clinical practice (80) and calibration plots were reported in almost all studies, we could not estimate the overall calibration of the nomograms because observed/expected rates or slope and intercept of calibration curve were missing or could not be calculated (these values were determined only for three studies).
The main limitation of this work is due to the restricted geographical distribution of the included nomograms and namely that all studies were conducted on Asian populations. Although the original search strategy identified more than 1500 articles, all non-Asian studies did not fulfil the inclusion criteria and were excluded. This situation seems to be due to lower percentages of studies conducted in western countries that aim to develop nomograms, the result of an extensive application of ATA and AJCC guidelines. In those few studies retrieved, the lack of a final nomogram that established a risk score (81) or the inclusion of different types of DTC (PTC, FTC, or MTC) to nomogram development (25, 82) prevented us from comparing nomograms validity among different populations.
This work highlights the extremely high heterogeneity among nomograms and the critical lack of external validation studies that limit the applicability of nomograms. In addition, the clinical utility of these tools appears scarce; whether and how patients or clinicians benefit from using nomograms is unclear.
In summary, although the nomograms developed for PTC patients are efficient and easy straightforward to read, they have limited clinical utility. Therefore, we conclude that their development should be not only evaluated on the basis of the statistical significance of independent factors (as shown in multivariate models) but using clinically relevant variables as a common denominator for all nomograms, given the well-established literature on risk factors for OS, DFS, recurrence, CLNM or LLNM risk in PTC patients. In this way, it will be possible to summarize the most reliable nomograms for clinical practice, identify those suitable for further research, and make suggestions for individual patient situations.
Declaration of interest
Authors declare that there is no conflict of interest that could be perceived as prejudicing the impartiality of the research reported.
Funding
This study did not receive any specific grant from any funding agency in the public, commercial or not-for-profit sector.
References
- 1↑
Roman BR, Morris LG, Davies L. The thyroid cancer epidemic, 2017 perspective. Current Opinion in Endocrinology, Diabetes, and Obesity 2017 24 332–336. (https://doi.org/10.1097/MED.0000000000000359)
- 2↑
Rahib L, Wehner MR, Matrisian LM, Nead KT. Estimated projection of US cancer incidence and death to 2040. JAMA Network Open 2021 4 e214708. (https://doi.org/10.1001/jamanetworkopen.2021.4708)
- 3↑
Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: A Cancer Journal for Clinicians 2018 68 394–424. (https://doi.org/10.3322/caac.21492)
- 4↑
Davies L, Welch HG. Current thyroid cancer trends in the United States. JAMA Otolaryngology– Head and Neck Surgery 2014 140 317–322. (https://doi.org/10.1001/jamaoto.2014.1)
- 5↑
Fagin JA, Wells SA Jr. Biologic and clinical perspectives on thyroid cancer. New England Journal of Medicine 2016 375 1054–1067. (https://doi.org/10.1056/NEJMra1501993)
- 6↑
Hundahl SA, Cady B, Cunningham MP, Mazzaferri E, McKee RF, Rosai J, Shah JP, Fremgen AM, Stewart AK, Holzer S. Initial results from a prospective cohort study of 5583 cases of thyroid carcinoma treated in the united states during 1996. U.S. and German thyroid cancer study group. An American College of Surgeons Commission on Cancer Patient Care Evaluation study. Cancer 2000 89 202–217. (https://doi.org/10.1002/1097-0142(20000701)89:1<202::aid-cncr27>3.0.co;2-a)
- 7↑
La Vecchia C, Malvezzi M, Bosetti C, Garavello W, Bertuccio P, Levi F, Negri E. Thyroid cancer mortality and incidence: a global overview. International Journal of Cancer 2015 136 2187–2195. (https://doi.org/10.1002/ijc.29251)
- 8↑
Siegel RL, Miller KD, Jemal A. Cancer statistics, 2019. CA: a Cancer Journal for Clinicians 2019 69 7–34. (https://doi.org/10.3322/caac.21551)
- 9↑
Wang Z, Mo C, Chen L, Kong L, Wu K, Zhu Y, Chen X. Application of competing risk model in the prognostic prediction study of patients with follicular thyroid carcinoma. Updates in Surgery 2022 74 735–746. (https://doi.org/10.1007/s13304-021-01103-6)
- 10↑
Hu Q, Zhang WJ, Liang L, Li LL, Yin W, Su QL, & Lin FF. Establishing a predictive nomogram for cervical lymph node metastasis in patients with papillary thyroid carcinoma. Frontiers in Oncology 2022 11 766650. (https://doi.org/10.3389/fonc.2021.766650)
- 11↑
Hay ID, Hutchinson ME, Gonzalez-Losada T, McIver B, Reinalda ME, Grant CS, Thompson GB, Sebo TJ, Goellner JR. Papillary thyroid microcarcinoma: a study of 900 cases observed in a 60-year period. Surgery 2008 144 980–987. (https://doi.org/10.1016/j.surg.2008.08.035)
- 12↑
Wang TS, Goffredo P, Sosa JA, Roman SA. Papillary thyroid microcarcinoma: an over-treated malignancy? World Journal of Surgery 2014 38 2297–2303. (https://doi.org/10.1007/s00268-014-2602-3)
- 13↑
Malterling RR, Andersson RE, Falkmer S, Falkmer U, Nilehn E, Jarhult J. Differentiated thyroid cancer in a Swedish county--long-term results and quality of life. Acta Oncologica 2010 49 454–459. (https://doi.org/10.3109/02841860903544600)
- 14↑
Liu FH, Kuo SF, Hsueh C, Chao TC, Lin JD. Postoperative recurrence of papillary thyroid carcinoma with lymph node metastasis. Journal of Surgical Oncology 2015 112 149–154. (https://doi.org/10.1002/jso.23967)
- 15↑
Grant CS Recurrence of papillary thyroid cancer after optimized surgery. Gland Surgery 2015 4 52–62. (https://doi.org/10.3978/j.issn.2227-684X.2014.12.06)
- 16↑
Chow SM, Law SC, Chan JK, Au SK, Yau S, Lau WH. Papillary microcarcinoma of the thyroid-prognostic significance of lymph node metastasis and multifocality. Cancer 2003 98 31–40. (https://doi.org/10.1002/cncr.11442)
- 17↑
Mehanna H, Al-Maqbili T, Carter B, Martin E, Campain N, Watkinson J, McCabe C, Boelaert K, Franklyn JA. Differences in the recurrence and mortality outcomes rates of incidental and nonincidental papillary thyroid microcarcinoma: a systematic review and meta-analysis of 21 329 person-years of follow-up. Journal of Clinical Endocrinology and Metabolism 2014 99 2834–2843. (https://doi.org/10.1210/jc.2013-2118)
- 18↑
Yu XM, Wan Y, Sippel RS, Chen H. Should all papillary thyroid microcarcinomas be aggressively treated? An analysis of 18,445 cases. Annals of Surgery 2011 254 653–660. (https://doi.org/10.1097/SLA.0b013e318230036d)
- 19↑
Ge MH, Cao J, Wang JY, Huang YQ, Lan XB, Yu B, Wen QL, Cai XJ. Nomograms predicting disease-specific regional recurrence and distant recurrence of papillary thyroid carcinoma following partial or total thyroidectomy. Medicine (Baltimore) 2017 96 e7575. (https://doi.org/10.1097/MD.0000000000007575)
- 20↑
Liu G, Liu Q, Sun SR. Nomograms for estimating survival in patients with papillary thyroid cancer after surgery. Cancer Management and Research 2019 11 3535–3544. (https://doi.org/10.2147/CMAR.S194366)
- 21↑
Shi LY, Liu J, Yu LJ, Lei YM, Leng SX, Zhang HY. Clinic-pathologic features and prognostic analysis of thyroid cancer in the older adult: a SEER based study. Journal of Cancer 2018 9 2744–2750. (https://doi.org/10.7150/jca.24625)
- 22↑
Tang J, Kong D, Cui Q, Wang K, Zhang D, Liao X, Gong Y, & Wu G. Racial disparities of differentiated thyroid carcinoma: clinical behavior, treatments, and long-term outcomes. World Journal of Surgical Oncology 2018 16 45. (https://doi.org/10.1186/s12957-018-1340-7)
- 23↑
Wang C, Dai L, Wu X, Wang Z. A nomogram for predicting overall-specific survival in thyroid cancer patients with total thyroidectomy: a SEER database analysis. Gland Surgery 2021 10 2546–2556. (https://doi.org/10.21037/gs-21-520)
- 24↑
Konturek A, Barczynski M, Nowak W, Richter P. Prognostic factors in differentiated thyroid cancer--a 20-year surgical outcome study. Langenbeck’s Archives of Surgery 2012 397 809–815. (https://doi.org/10.1007/s00423-011-0899-z)
- 25↑
Pathak KA, Mazurat A, Lambert P, Klonisch T, Nason RW. Prognostic nomograms to predict oncological outcome of thyroid cancers. Journal of Clinical Endocrinology and Metabolism 2013 98 4768–4775. (https://doi.org/10.1210/jc.2013-2318)
- 26↑
Chen LJ, Chung KP, Chang YJ, Chang YJ. Ratio and log odds of positive lymph nodes in breast cancer patients with mastectomy. Surgical Oncology 2015 24 239–247. (https://doi.org/10.1016/j.suronc.2015.05.001)
- 27↑
Sun Z, Xu Y, Li de de M, Wang ZN, Zhu GL, Huang BJ, Li K, Xu HM. Log odds of positive lymph nodes: a novel prognostic indicator superior to the number-based and the ratio-based N category for gastric cancer patients with R0 resection. Cancer 2010 116 2571–2580. (https://doi.org/10.1002/cncr.24989)
- 28↑
Schumacher P, Dineen S, Barnett C Jr, Fleming J, Anthony T. The metastatic lymph node ratio predicts survival in colon cancer. American Journal of Surgery 2007 194 827–831. (https://doi.org/10.1016/j.amjsurg.2007.08.030)
- 29↑
Tan HL, Huang BQ, Li GY, Wei B, Chen P, Hu HY, Liu M, Ou-Yang DJ, Yang Q & Qin ZE et al.A prediction model for contralateral central neck lymph node metastases in unilateral papillary thyroid cancer. International Journal of Endocrinology 2021 2021 6621067. (https://doi.org/10.1155/2021/6621067)
- 30↑
Wen Q, Yu Y, Yang J, Wang X, Wen J, Wen Y, Wang Y, Lyu J. Development and validation of a nomogram for predicting survival in patients with thyroid cancer. Medical Science Monitor 2019 25 5561–5571. (https://doi.org/10.12659/MSM.915620)
- 31↑
Lang BH, Wong CK. Validation and comparison of nomograms in predicting disease-specific survival for papillary thyroid carcinoma. World Journal of Surgery 2015 39 1951–1958. (https://doi.org/10.1007/s00268-015-3044-2)
- 32↑
Pan H, Shi X, Xiao D, He J, Zhang Y, Liang W, Zhao Z, Guo Z, Zou X & Zhang J et al.Nomogram prediction for the survival of the patients with small cell lung cancer. Journal of Thoracic Disease 2017 9 507–518. (https://doi.org/10.21037/jtd.2017.03.121)
- 33↑
Hei H, Song Y, Qin J. Individual prediction of lateral neck metastasis risk in patients with unifocal papillary thyroid carcinoma. European Journal of Surgical Oncology 2019 45 1039–1045. (https://doi.org/10.1016/j.ejso.2019.02.016)
- 34↑
Shu X, Tang L, Hu D, Wang Y, Yu P, Yang Z, Deng C, Wang D, Su X. Prediction model of pathologic central lymph node negativity in cN0 papillary thyroid carcinoma. Frontiers in Oncology 2021 11 727984. (https://doi.org/10.3389/fonc.2021.727984)
- 35↑
Feng JW, Hong LZ, Wang F, Wu WX, Hu J, Liu SY, Jiang Y, Ye J. A nomogram based on clinical and ultrasound characteristics to predict central lymph node metastasis of papillary thyroid carcinoma. Frontiers in Endocrinology (Lausanne) 2021 12 666315. (https://doi.org/10.3389/fendo.2021.666315)
- 36↑
Kim E, Park JS, Son KR, Kim JH, Jeon SJ, Na DG. Preoperative diagnosis of cervical metastatic lymph nodes in papillary thyroid carcinoma: comparison of ultrasound, computed tomography, and combined ultrasound with computed tomography. Thyroid 2008 18 411–418. (https://doi.org/10.1089/thy.2007.0269)
- 37↑
Ito Y, Tomoda C, Uruno T, Takamura Y, Miya A, Kobayashi K, Matsuzuka F, Kuma K, Miyauchi A. Ultrasonographically and anatomopathologically detectable node metastases in the lateral compartment as indicators of worse relapse-free survival in patients with papillary thyroid carcinoma. World Journal of Surgery 2005 29 917–920. (https://doi.org/10.1007/s00268-005-7789-x)
- 38↑
Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, Pacini F, Randolph GW, Sawka AM & Schlumberger M et al.2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid 2016 26 1–133. (https://doi.org/10.1089/thy.2015.0020)
- 39↑
Lim YS, Lee JC, Lee YS, Lee BJ, Wang SG, Son SM, Kim IJ. Lateral cervical lymph node metastases from papillary thyroid carcinoma: predictive factors of nodal metastasis. Surgery 2011 150 116–121. (https://doi.org/10.1016/j.surg.2011.02.003)
- 40↑
Zhan S, Luo D, Ge W, Zhang B, Wang T. Clinicopathological predictors of occult lateral neck lymph node metastasis in papillary thyroid cancer: a meta-analysis. Head and Neck 2019 41 2441–2449. (https://doi.org/10.1002/hed.25762)
- 41↑
Roh JL, Park JY, Park CI. Total thyroidectomy plus neck dissection in differentiated papillary thyroid carcinoma patients: pattern of nodal metastasis, morbidity, recurrence, and postoperative levels of serum parathyroid hormone. Annals of Surgery 2007 245 604–610. (https://doi.org/10.1097/01.sla.0000250451.59685.67)
- 42↑
Damen JAA, Moons KGM, van Smeden M, Hooft L. How to conduct a systematic review and meta-analysis of prognostic model studies. Clinical Microbiology and Infection 2022 [epub]. (https://doi.org/10.1016/j.cmi.2022.07.019)
- 43↑
Debray TP, Damen JA, Snell KI, Ensor J, Hooft L, Reitsma JB, Riley RD, Moons KG. A guide to systematic review and meta-analysis of prediction model performance. BMJ 2017 356 i6460. (https://doi.org/10.1136/bmj.i6460)
- 44↑
Riley RD, Moons KGM, Snell KIE, Ensor J, Hooft L, Altman DG, Hayden J, Collins GS, Debray TPA. A guide to systematic review and meta-analysis of prognostic factor studies. BMJ 2019 364 k4597. (https://doi.org/10.1136/bmj.k4597)
- 45↑
Riley RD, Moons KGM, Altman DG, Collins GS, Debray TPA. Systematic reviews of prognostic factor studies. Systematic Reviews in Health Research 2022 324–346. (https://doi.org/10.1002/9781119099369.ch17)
- 46↑
Moons KGM, Wolff RF, Riley RD, Whiting PF, Westwood M, Collins GS, Reitsma JB, Kleijnen J, Mallett S. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Annals of Internal Medicine 2019 170 W1–W33. (https://doi.org/10.7326/M18-1377)
- 47↑
Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, Reitsma JB, Kleijnen J, Mallett S & PROBAST Group†. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Annals of Internal Medicine 2019 170 51–58. (https://doi.org/10.7326/M18-1376)
- 48↑
Cao YM, Zhang TT, Li BY, Qu N, Zhu YX. Prognostic evaluation model for papillary thyroid cancer: a retrospective study of 660 cases. Gland Surgery 2021 10 2170–2179. (https://doi.org/10.21037/gs-21-100)
- 49↑
Ding Y, Mao Z, Ruan J, Su X, Li L, Fahey TJ 3rd, Wang W, Teng L. Nomogram-based new recurrence predicting system in early-stage papillary thyroid cancer. International Journal of Endocrinology 2019 2019 1029092. (https://doi.org/10.1155/2019/1029092)
- 50↑
Dou Y, Chen Y, Hu D, Xiong W, Xiao Q, Su X. Development and validation of web-based nomograms for predicting lateral lymph node metastasis in patients with papillary thyroid carcinoma. Gland Surgery 2020 9 172–182. (https://doi.org/10.21037/gs.2020.01.11)
- 51↑
Gao X, Luo W, He L, Cheng J, & Yang L. Predictors and a prediction model for central cervical lymph node metastasis in papillary thyroid carcinoma (cN0). Frontiers in Endocrinology (Lausanne) 2022 12 789310. (https://doi.org/10.3389/fendo.2021.789310)
- 52↑
Heng Y, Yang Z, Zhou L, Lin J, Cai W, Tao L. Risk stratification for lateral involvement in papillary thyroid carcinoma patients with central lymph node metastasis. Endocrine 2020 68 320–328. (https://doi.org/10.1007/s12020-020-02194-8)
- 53↑
Jianyong L, Jinjing Z, Zhihui L, Tao W, Rixiang G, Jingqiang Z. A nomogram based on the characteristics of metastatic lymph nodes to predict papillary thyroid carcinoma recurrence. Thyroid 2018 28 301–310. (https://doi.org/10.1089/thy.2017.0422)
- 54↑
Kim SK, Chai YJ, Park I, Woo JW, Lee JH, Lee KE, Choe JH, Kim JH, & Kim JS. Nomogram for predicting central node metastasis in papillary thyroid carcinoma. Journal of Surgical Oncology 2016 115 266–272. (https://doi.org/10.1002/jso.24512)
- 55↑
Lin P, Liang F, Ruan J, Han P, Liao J, Chen R, Luo B, Ouyang N, Huang X. A preoperative nomogram for the prediction of high-volume central lymph node metastasis in papillary thyroid carcinoma. Frontiers in Endocrinology (Lausanne) 2021 12 753678. (https://doi.org/10.3389/fendo.2021.753678)
- 56↑
Liu S, Liu C, Zhao L, Wang K, Li S, Tian Y, Jiao B, Gui Z, Yu T, Zhang L. A prediction model incorporating the BRAFV600E protein status for determining the risk of cervical lateral lymph node metastasis in papillary thyroid cancer patients with central lymph node metastasis. European Journal of Surgical Oncology 2021 600E 2774–2780. (https://doi.org/10.1016/j.ejso.2021.08.033)
- 57↑
Qi Q, Xu P, Zhang C, Guo S, Huang X, Chen S, Li Y, Zhou A. Nomograms combining ultrasonic features with clinical and pathological features for estimation of Delphian lymph node metastasis risk in papillary thyroid carcinoma. Frontiers in Oncology 2021 11 792347. (https://doi.org/10.3389/fonc.2021.792347)
- 58↑
Sun F, Zou Y, Huang L, Shi Y, Liu J, Cui G, Zhang X, Xia S. Nomogram to assess the risk of central cervical lymph node metastasis in patients with clinical N0 papillary thyroid carcinoma. Endocrine Practice 2021 27 1175–1182. (https://doi.org/10.1016/j.eprac.2021.06.010)
- 59↑
Wang K, Xu J, Li S, Liu S, Zhang L. Population-based study evaluating and predicting the probability of death resulting from thyroid cancer among patients with papillary thyroid microcarcinoma. Cancer Medicine 2019 8 6977–6985. (https://doi.org/10.1002/cam4.2597)
- 60↑
Yang Z, Heng Y, Lin J, Lu C, Yu D, Tao L, Cai W. Nomogram for predicting central lymph node metastasis in papillary thyroid cancer: a retrospective cohort study of two clinical centers. Cancer Research and Treatment 2020 52 1010–1018. (https://doi.org/10.4143/crt.2020.254)
- 61↑
Zhuo X, Yu J, Chen Z, Lin Z, Huang X, Chen Q, Zhu H, & Wan Y. Dynamic nomogram for predicting lateral cervical lymph node metastasis in papillary thyroid carcinoma. Otolaryngology–Head and Neck Surgery 2021 166 444–453. (https://doi.org/10.1177/01945998211009858)
- 62↑
Adam MA, Pura J, Goffredo P, Dinan MA, Reed SD, Scheri RP, Hyslop T, Roman SA, Sosa JA. Presence and number of lymph node metastases are associated with compromised survival for patients younger than age 45 years with papillary thyroid cancer. Journal of Clinical Oncology 2015 33 2370–2375. (https://doi.org/10.1200/JCO.2014.59.8391)
- 63↑
Dormosh N, Heymans MW, van der Velde N, Hugtenburg J, Maarsingh O, Slottje P, Abu-Hanna A, Schut MC. External validation of a prediction model for falls in older people based on electronic health records in primary care. Journal of the American Medical Directors Association 2022 23 1691–1697.e3. (https://doi.org/10.1016/j.jamda.2022.07.002)
- 64↑
Ramspek CL, Jager KJ, Dekker FW, Zoccali C, van Diepen M. External validation of prognostic models: what, why, how, when and where? Clinical Kidney Journal 2021 14 49–58. (https://doi.org/10.1093/ckj/sfaa188)
- 65↑
Gui CY, Qiu SL, Peng ZH, Wang M. Clinical and pathologic predictors of central lymph node metastasis in papillary thyroid microcarcinoma: a retrospective cohort study. Journal of Endocrinological Investigation 2018 41 403–409. (https://doi.org/10.1007/s40618-017-0759-y)
- 66↑
Huang J, Song M, Shi H, Huang Z, Wang S, Yin Y, Huang Y, Du J, Wang S & Liu Y et al.Predictive factor of large-volume central lymph node metastasis in clinical N0 papillary thyroid carcinoma patients underwent total thyroidectomy. Frontiers in Oncology 2021 11 574774. (https://doi.org/10.3389/fonc.2021.574774)
- 67↑
Ahn JE, Lee JH, Yi JS, Shong YK, Hong SJ, Lee DH, Choi CG, Kim SJ. Diagnostic accuracy of CT and ultrasonography for evaluating metastatic cervical lymph nodes in patients with thyroid cancer. World Journal of Surgery 2008 32 1552–1558. (https://doi.org/10.1007/s00268-008-9588-7)
- 68↑
Jiang LH, Yin KX, Wen QL, Chen C, Ge MH, Tan Z. Predictive risk-scoring model for central lymph node metastasis and predictors of recurrence in papillary thyroid carcinoma. Scientific Reports 2020 10 710. (https://doi.org/10.1038/s41598-019-55991-1)
- 69↑
Yan H, Zhou X, Jin H, Li X, Zheng M, Ming X, Wang R, Liu J. A study on central lymph node metastasis in 543 cN0 papillary thyroid carcinoma patients. International Journal of Endocrinology 2016 2016 1878194. (https://doi.org/10.1155/2016/1878194)
- 70↑
Hu D, Zhou J, He W, Peng J, Cao Y, Ren H, Mao Y, Dou Y, Xiong W & Xiao Q et al.Risk factors of lateral lymph node metastasis in cN0 papillary thyroid carcinoma. World Journal of Surgical Oncology 2018 16 30. (https://doi.org/10.1186/s12957-018-1336-3)
- 71↑
Kim SK, Park I, Hur N, Rayzah M, Lee JH, Choe JH, Kim JH, Kim JS. Patterns, predictive factors, and prognostic impact of contralateral lateral lymph node metastasis in N1b papillary thyroid carcinoma. Annals of Surgical Oncology 2017 24 1943–1950. (https://doi.org/10.1245/s10434-016-5761-7)
- 72↑
Kaliszewski K, Diakowska D, Nowak Ł, Wojtczak B, Rudnicki J. The age threshold of the 8th edition AJCC classification is useful for indicating patients with aggressive papillary thyroid cancer in clinical practice. BMC Cancer 2020 20 1166. (https://doi.org/10.1186/s12885-020-07636-0)
- 73↑
Farrag T, Lin F, Brownlee N, Kim M, Sheth S, Tufano RP. Is routine dissection of level II-B and V-A necessary in patients with papillary thyroid cancer undergoing lateral neck dissection for FNA-confirmed metastases in other levels. World Journal of Surgery 2009 33 1680–1683. (https://doi.org/10.1007/s00268-009-0071-x)
- 74↑
Merdad M, Eskander A, Kroeker T, Freeman JL. Predictors of level II and Vb neck disease in metastatic papillary thyroid cancer. Archives of Otolaryngology–Head and Neck Surgery 2012 138 1030–1033. (https://doi.org/10.1001/2013.jamaoto.393)
- 75↑
Nie X, Tan Z, Ge M, Jiang L, Wang J, Zheng C. Risk factors analyses for lateral lymph node metastases in papillary thyroid carcinomas: a retrospective study of 356 patients. Archives of Endocrinology and Metabolism 2016 60 492–499. (https://doi.org/10.1590/2359-3997000000218)
- 76↑
Zhang L, Wei WJ, Ji QH, Zhu YX, Wang ZY, Wang Y, Huang CP, Shen Q, Li DS, Wu Y. Risk factors for neck nodal metastasis in papillary thyroid microcarcinoma: a study of 1066 patients. Journal of Clinical Endocrinology and Metabolism 2012 97 1250–1257. (https://doi.org/10.1210/jc.2011-1546)
- 77↑
Rahbari R, Zhang L, Kebebew E. Thyroid cancer gender disparity. Future Oncology 2010 6 1771–1779. (https://doi.org/10.2217/fon.10.127)
- 78↑
Zahedi A, Bondaz L, Rajaraman M, Leslie WD, Jefford C, Young JE, Pathak KA, Bureau Y, Rachinsky I & Badreddine M et al.Risk for thyroid cancer recurrence is higher in men than in women independent of disease stage at presentation. Thyroid 2020 30 871–877. (https://doi.org/10.1089/thy.2018.0775)
- 79↑
Hebert AE, Kreaden US, Yankovsky A, Guo D, Li Y, Lee SH, Liu Y, Soito AB, Massachi S, Slee AE. Methodology to standardize heterogeneous statistical data presentations for combining time-to-event oncologic outcomes. PLoS One 2022 17 e0263661. (https://doi.org/10.1371/journal.pone.0263661)
- 80↑
Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, Vickers AJ, Ransohoff DF, Collins GS. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Annals of Internal Medicine 2015 162 W1–W73. (https://doi.org/10.7326/M14-0698)
- 81↑
Thompson AM, Turner RM, Hayen A, Aniss A, Jalaty S, Learoyd DL, Sidhu S, Delbridge L, Yeh MW & Clifton-Bligh R et al.A preoperative nomogram for the prediction of ipsilateral central compartment lymph node metastases in papillary thyroid cancer. Thyroid 2014 24 675–682. (https://doi.org/10.1089/thy.2013.0224)
- 82↑
Pathak KA, Lambert P, Nason RW, Klonisch T. Comparing a thyroid prognostic nomogram to the existing staging systems for prediction risk of death from thyroid cancers. European Journal of Surgical Oncology 2016 42 1491–1496. (https://doi.org/10.1016/j.ejso.2016.05.016)
