Evaluating Enchantment Lottery Performance: Awarded-to-Skipped Ratio

Introduction

In the third installment of examining the advice and exploring the data for the Enchantments Lottery, I’m going to introduce a ratio I developed to help rank the performance of application entries.

Recap

The previous two parts in this series uncovered characteristics of the 2021 lottery that seem to fly in the face of conventional wisdom around how to increase your odds of winning. I’ll summarize those points here.

First, applicants should NOT avoid applying for the Core Zone to increase their odds of winning. In 2021, 70% of awarded applicants applied for a Core Zone permit in at least one of their options receiving either a Core Zone permit (673) or a permit for another zone (1,039).

You can read more about this analysis in Part 1, Avoiding Bad Advice on Winning the Enchantments Lottery.

In the second analysis, I looked at how month of entry, day of the week, and group size affected an applicant’s chance of winning. That analysis produced a few interesting findings:

• Mondays are more popular than people think. The three worst days to apply are Friday, Monday, and Thursday.
• Saturdays were a dark horse. It received fewer application entries than on Monday or Tuesday. It nearly received less than Wednesday. Also, it was awarded the second most permits by day.
• Sunday is the best day to apply. Due to the closure of the walk-up lottery on Sundays in 2021, there was more inventory released in the early access lottery for that day. That meant there was 33% more capacity in the Core Enchantment Zone on Sundays in 2021. Coincidentally, Sunday received the least amount of application entries giving it the best awarded-to-applied ratio.
• Smaller group sizes generally do better at securing permits in the Core Enchantment Zone with a group size of 1 punching above its weight. However, due to the increased permit availability on Sundays, group sizes of 8 secured the most permits on Sundays in August for the Core Zone.

You can read more about this analysis in Part 2, Avoiding Bad Advice on Winning the Enchantments Lottery (Part II).

During the second part of the analysis, I started looking at the “skipped” application entries and developed a ratio that provides a different performance ranking for application entry characteristics. In this next part, I’ll explain the ratio and its findings.

Awarded-to-Skipped

In the first part of the analysis I noted that, although we can see which applications were awarded, we can’t see which applications had a chance at being awarded but failed. During the second part of the analysis, I realized that’s not entirely true!

Each application row has data for:

• awarded status
• awarded preference
• preferred entry data

Every awarded application shows the awarded preference number. The possible awarded preference values are 1, 2, or 3 because the applicant is allowed to submit three options (I use option and preference interchangeably).

Therefore, if an awarded row shows an awarded preference of 3, I assume that options 1 and 2 failed to secure a permit for lack of availability.

Using this logic, I separated each application into distinct entries, removed n/a entries, and filtered for all the values where the awarded preference was greater than the preference number. This gave me a collection of application entries that had a chance to win but didn’t.

Next, it was easy to use established methods for determining if an application entry was awarded.

With the skipped entries and awarded entries in hand, I calculated ratios for the zone, month, day of the week, and group size (Core Zone).

The only caveat is the data is still imprecise because some rows could have had all the entries fail, and we simply don’t know which rows those might be. Or, if they exist at all.

Zone

Looking at the Awarded-to-Skipped (AtS) broken down by zone it's easy to see what many people could have predicted. The Core Zone (0.58) had the worst AtS ratio which was close to 1:2. A little surprising to me was that Colchuck (0.93) had almost a 1:1 ratio.

All the other zones were well above a 1:1 with Eightmile/Caroline Zone having a whopping 7:1 ratio, meaning that for every 1 entry that was skipped, seven were awarded.

Month

The gap between "snowy" and "summer" months is pronounced by looking at the AtS for monthly data. September (0.98), August (1.00), and July (1.10) all have numbers hovering around a 1:1 ratio (in the Core Zone the numbers for these months were closer to 0.5 meaning for every one awarded, two were skipped).

On the flip side, June (2.03) and October (1.92) were awarded twice as often as they were skipped compared to the summer months and May (2.92) had the best ratio nearing 3:1.

Day of Week

The AtS by day of the week across the zones highlights Sunday's dominance. It almost was awarded at twice the rate it was skipped (1.92) which was way above the rest of the days, the closest being Wednesday (1.42).

Again, to highlight the surprising performance of Saturday (1.34), it outperformed Tuesday (1.33) and Monday (1.17).

Awarded-to-Skipped by Group Size in the Core Enchantment Zone

The Awarded-to-Skipped ratio for the group size in the Core Zone highlights the problems with choosing high odd numbers with 7 (0.31) and 5 (0.42) sporting the worst ratios.

The best group size for the Core Enchantment Zone was, generally small. 1 (1.31), 3 (0.72), and 2 (0.70) performed the best with a group size of one being by far the best selection to avoid having your entry skipped. Surprisingly, a group size of 6 (0.60) outperformed a group size of 4 (0.55).

There's a pretty big difference between 3 and 4. Although 3 is an odd number its AtS is close to 3:4 while 4 is closer to 1:2. Group size plays a huge role in the Core Enchantment Zone, so I think these ratios shouldn't be taken lightly.

Takeaways

The Awarded-to-Skipped ratio gives us a better look at performance for applications that had a chance at winning. As an applicant, we are at the mercy of luck with whether or not our application even has a chance. However, understanding the different AtS ratios may give us an advantage in ensuring the best shot at being awarded a permit given our application is selected.

We must remember these probabilities are not hard rules. They are theory. As we saw in part 2 of the analysis, group sizes of 8 outperformed every other group size on Sundays in August for the Core Enchantments. Conversely, looking at the AtS for group size you might be led to believe that 8 is a bad group size. There are exceptions.

What I hope to impart to the reader is your selection does matter outside of "just luck". Consequently, take care when choosing your zone, day of the week, or group size (if applying to the Core Enchantments).

You can view the repository for the analysis on Github.

Next Time

I'm tempted to lay out what I believe is a good set of choices given the findings in the analysis and my knowledge of the area. Or, I may see if I can clean up the analysis part and narrow down the stats to what I believe are the numbers worth looking at.

Then, apply those calculations to the 2022 lottery day. I did reach out to the USFS to see if they could provide me with the 2023 statistics in XML or CSV. Unfortunately, I have not heard back.