Determining Variables:
1. Variables were selected based on an extensive literature review of relevant metrics that correlate with competitiveness.1–4,7–12
2. The variables chosen included:
- Match rate
- Matched applicants
- Total applicants
- Residency positions are available each year
- Number of programs ranked per applicant
- United States Medical Licensing Exam (USMLE) Step 1 score
- USMLE Step 2 score
- Mean applicant research experiences
- Mean applicant research output
3. Variables were collected for general surgery residency programs over the course of 20 years. Descriptive data was conducted prior to equation formation to understand trends.
Equation Development:
1. Variables that significantly changed over the study period were used in equation development.
2. The number of programs ranked per applicant were chosen based on the theory that applicants increasing the number of ranked programs is a consequence of increasing competitiveness.
3. Positions were chosen in relationship with match rate due to the fact a higher applicant to position ratio results in higher competitiveness.
4. Match rate was used in each equation to contribute matched applicants and total applicants. While we theorized match rate is not a reliable competitiveness metric alone, the ratio of matched applicants to total applicants is necessary to include. We sought to determine the relationship of match rate with other variables for a consistent competitiveness metric.
Normalizing Equations:
1. Due to variability over time, the equations required normalization. Additionally, normalization to a value of 1 allows the user to interpret the data more easily.
- However, it is important to note that the intended application is used without normalization with a set numerical definition.
2. To create a normalized competitiveness index (NCI):
- Each CI (Equations 1-4) was averaged over time.
- The yearly CI was divided by the averaged CI.
- Values over one indicate higher competitiveness.
Testing the NCI against Match Rate:
1. The gold standard for defining competitiveness is currently match rate.
2. The relationship between NCI equations (Equations 1-4) and match rate were analyzed.
NCI vs Match Rate: Descriptive Analysis
1. The change in applicants for each year were calculated (Δ Applicants).
2. The change in positions for each year were calculated (Δ Positions).
3. The Δ Applicants was subtracted by the Δ Positions to estimate the annual change in position deficit.
- The median position deficit was 15 (IQR -53 to 103)
- Years that had a change in position deficit greater than 103 (>75th percentile) or less than -53 (<25th percentile) were marked to examine for potential change in competitiveness.
- For example, years with 100 new applicants without a corresponding change in positions or 100 fewer applicants (i.e., 100 available spots).
4. The marked years were qualitatively assessed for each NCI. Theoretically, a change greater than +103 would result in increased competitiveness seen by a higher NCI, and a change less than -53 would lead to a lower NCI for that year.
5. While this method was not used in final decision-making, it allowed for a general overview of the equations with trends in time.
- Figure 2 demonstrates examples of marked years with increased or decreased Δ Applicants - Δ Positions.
NCI vs Match Rate: Regression Analysis
1. The normalized general surgery match rate (Figure 1A) and normalized applicants per position were plotted over 20 years (Figure 1B) and a linear regression was performed.
2. Each NCI equation was plotted over 20 years (Figure 2) and a linear regression was performed.
- The slopes of the change in match rate were compared to the slope of the NCI using each equation.
- The slopes of the NCI equations were all significantly different than the match rate.
- Normalized match rate slope = 0.002
- Equation 1 slope = 0.005, p<0.001
- Equation 2 slope = 0.02, p<0.001
- Equation 3 slope = 0.05, p<0.001
- Equation 4 slope = 0.03, p<0.001
- The change in NCI each year was greater than the match rate.
NCI vs Match Rate: Correlation Analysis
1. The normalized general surgery match rate and each NCI equation were analyzed using a Pearson correlation over time.
2. There were no significant correlations to match rate when each equation was individually assessed.
- All Pearson r values were between -0.05-0.50 (Figure 3).
Testing the NCI against Applicant Metrics:
1. USMLE Step 1, Step 2, research experiences, research output, work experiences, and volunteer experiences were analyzed across a 20-year period for general surgery and a 10-year period for sub-specialty surgery.
2. USMLE Step 2 and research output were variables that had the most consistent growth over time.
NCI vs USMLE Step 2: Correlation Analysis
1. Each general surgery NCI equation (Equations 1-4) was correlated with Step 2 (Figure 4).
2. All four equations positively correlated with Step 2; however, Equation 3 had the strongest correlation (Pearson r=0.89, p<0.001) (Figure 4C).
3. As a reference, the Pearson r for the normalized match rate versus Step 2 was 0.34 (p=0.41).
NCI vs Research Output: Correlation Analysis
1. Research output was defined as the number of publications, presentations, and abstracts.
2. Each general surgery NCI equation (Equations 1-4) were correlated with research output (Figure 5).
3. All four equations were positively correlated with research output. Equation 3 had the strongest correlation (Pearson r=0.97, p<0.001). (Figure 5C).
4. As a reference, the Pearson r for the normalized match rate versus research output was 0.56 (p=0.12).
Sub-Analysis of Other Surgical Residencies:
1. Sub-specialty surgical residencies were selected based on their relationship to general surgery.
- Currently, there are three specialties that can be completed as either an integrated residency program or fellowship following general surgery residency: Plastic Surgery, Thoracic Surgery, and Vascular Surgery.
- Orthopedic Surgery, Neurosurgery, Otolaryngology, and Obstetrics and Gynecology were not selected based on fellowship options following general surgery residency.
2. All four equations were analyzed in relationship to match rate, applicants per position, and applicant metrics for each specialty.
Plastic Surgery Analysis:
1. Overall, match rate remained constant over time at 64% ± 7 (R2=0.10, p=0.37).
2. All equations were significantly different across time compared to match rate.
- Normalized match rate slope = 0.01
- Equation 1 slope = 0.02, p<0.001
- Equation 2 slope = 0.5, p<0.001
- Equation 3 slope = 0.08, p<0.001
- Equation 4 slope = 0.07, p<0.00
3. Match rate was not correlated with any of the equations (Pearson r range -0.1 to 0.50)
4. Step 2 scores were not correlated with any of the equations (range -0.01 to -0.31)
5. Research output was correlated with equation 1 (r=0.74), equation 2 (r=0.72), and equation 3 (0.87).
6. Data are displayed in Figure 6.
Vascular Surgery Analysis:
1. Results were similar with no significant change in match rate over time (51% ± 6, R2=0.001, p=0.99).
2. Match rates were not correlated with any of the equations (r=-0.6 to 0.10).
3. Step 2 scores were not correlated with any of the equations (range -0.05 to 0.27)
4. Research output was correlated with equation 1 (r=0.69), equation 2 (r=0.84), and equation 3 (0.94).
5. Data are displayed in Figure 7.
Thoracic Surgery Analysis:
1. Results were similar with no significant change in match rate over time (31% ± 4, R2=0.08, p=0.41).
2. Match rates were not correlated with any of the equations (r=-0.13 to 0.20).
3. The sample size for applicant metrics was not sufficient for analysis as data were only available from 2019-2021.