Challenge League Post-mortems

Since we introduced Challenge Leagues in mid-2013, we've run a total of 12, as six pairs of two. This manifesto post is an analysis of each league and what we learned for the design of future challenge leagues.

Initially, challenge leagues had a few primary goals:

• A fresh start to the economy for players who enjoy wealth accumulation and character development in a new environment.
• Harder gameplay than the regular permanent leagues.
• A way of testing out game mechanics or features that could later be used in the main game.
• A system of challenges to reward players for completing advanced content.

Anarchy (Standard, June 2013):
The Anarchy challenge league introduced rogue exiles. These were a game feature that we wanted to roll into the core Path of Exile experience, but only after testing. By making them the focus of this league, we anticipated getting a lot of feedback on their implementation.

Initially, a rogue exile would spawn in 5% of areas. This rate was intentionally set low so that they were an impactful encounter. They dropped a lot of items with a big rarity boost.

In the formative first few hours of this league, players didn't encounter enough rogue exiles to really feel the difference from the regular game. We increased the rate so that they spawned in 8% of areas, which was as high as we were willing to push it while they still dropped really well.

The challenge to kill all the rogue exiles was popular but had a few problems. Firstly, we failed to clearly communicate that the highest rogue exile only spawned in level 60+ areas. Secondly, players resorted to just doing quick repetitive runs of small areas in an attempt to see more exiles per hour. Some players expressed a dislike for "RNG-gating" challenges in this way. Other players noted that they enjoyed having specifically named goals in a challenge like this.

The economic benefit of farming rogue exiles had a moderate impact on the economy (compared to later challenge leagues). We noted that care needed to be taken with similar mechanics in the future.

From the Anarchy league, we learnt that RNG-gating is a useful tool but can be frustrating if balanced incorrectly. We also learnt that the league feature must be apparent in early areas, otherwise players don't really feel the theme of the league.

Onslaught (Hardcore, June 2013):
In the Onslaught league, monsters had a 20% bonus to their movement, attack and cast speeds. This resulted in the game being harder, without any additional reward or special features. While it was popular at the time with hardcore players, it's not really a league design we could use nowadays. It'd certainly feel underwhelming compared to contemporary leagues.

We found that having substantially different game balance in the two challenge leagues made it much harder to work on and discuss balance because feedback could come from two very different environments.

Once the league finished, feedback from players about the fast monsters caused us to increase the speeds of several monsters in the core game.

Domination (Standard, October 2013):
The Domination league introduced shrines to Path of Exile. We wanted the shrines to always affect a big pack of monsters around them, so the effects were designed to be bidirectional (e.g. there was no experience shrine).

Learning from the Anarchy league, we made sure that shrines were common and were in early areas. Players certainly encountered the shrines soon after starting the league, which wasn't necessarily the best experience for new players who struggled with the challenge of killing all of the monsters.

This league had a slight economic benefit because of the additional monsters. Compared to the rogue exiles, players said it was less rewarding (even though the rogue exiles were still in the game after Anarchy ended).

Nemesis (Hardcore, October 2013):
Nemesis was the first challenge league designed using the method that we currently use for contemporary challenge leagues. It was made to be a fun challenge league, rather than as a feature we wanted to test out.

In this league, rare monsters were given an additional Nemesis Mod, which improved their abilities and made the fight harder. Because this applied to all rare monsters, it was a common mechanic. The ways it improved the fights were interesting and created many YouTube-friendly moments (for example, encounters with the feared Storm Herald mod).

There was no economic benefit in this league other than the Inner Treasure mod, which was really popular. We began to suspect that high-variance economic benefit is more appropriate for challenge leagues than constant economic benefit.

Ambush (Standard, March 2014):
Back in the original Path of Exile design document (written in 2007-2009), I described the ability for chests to have mods. This is a feature I really wanted to get into the game, but it wasn't a development priority until after Path of Exile's release. By this point, it was clear that it'd make an awesome challenge league.

The idea was improved by the team, making two key changes - that special types of chests would get mods (rather than all of them) and that monster fights were mandatory. Strongboxes with mods were introduced as the Ambush challenge league. It was many players' favourite challenge league.

Ambush highlighted risk vs. reward in a strong way. You could create very difficult encounters, and would get rewarded with a lot of items. This created long term repercussions that affect Path of Exile to this day.

Invasion (Hardcore, March 2014):
The Invasion league was designed to test out the guest monsters feature (and internally, an improved way of handling monster packs). During development, we realised that in order to make this into a fun challenge league, we needed to add another element. Based on the Invasion theme, we decided to add unique Invasion bosses to every world area. We felt that if the extra monster mods in Nemesis were popular, then having custom-designed unique bosses would be even better. We intended them to be hard and memorable.

The Invasion bosses were arguably too difficult at first. While some of our players love a challenge, the league underperformed numerically (especially compared to the insanely popular Ambush league running at the same time). In addition, the drop rates of Invasion monsters were not anywhere near as high as that of the rogue exiles in the Anarchy league.

We learnt that adding mandatory dangerous roaming bosses to each area is something to be careful with. After some balance tweaking, Invasion's popularity improved and it is now recalled fondly by players.

Guest monsters added the variety that we were looking for to areas. Based on player feedback, we heavily toned them down several times after adding them back into the core game.

Rampage (Standard, August 2014):
We felt that the Ambush and Invasion challenge leagues overshadowed some of the Sacrifice of the Vaal content. When designing the Standard challenge league that launched alongside Forsaken Masters, we wanted to try a lighter approach.

The Rampage league introduced the idea of kill streaks. It was not a particularly popular feature and was eventually not added to the core game except in certain niche circumstances.

We learnt that good challenge leagues involve the environment/monsters changing, rather than characters' abilities. We also noted that Rampage was the only league that made the core game easier without any additional challenge.

Beyond (Hardcore, August 2014):
This league started with a living dead theme where monsters rise from areas where too many corpses are stacked. Because corpses are a resource that some classes consume, we moved in the direction of tracking the location of monster deaths rather than the actual corpses themselves. We wanted the league to actually work for characters that shatter monsters! Once we established the theme of demons from another realm, the mechanics came together pretty quickly. We liked the fact that (unlike Invasion), the bosses were spawned as a result of player actions and could be avoided (or induced) by careful play.

Like Invasion, many players found Beyond too difficult (even after some balance adjustments). An important difference was that Beyond was actually pretty rewarding, but in a subtle way. Players found substantially more items due to the demons, but didn't really notice. We learnt that item rewards need to be more spikey and noticeable, otherwise they can damage the economy silently.

Many players commented that they enjoyed the small amount of lore that Beyond contained. I expect us to return to that story in the future.

Torment (Standard, December 2014):
In December last year we released the 1.3.0 update, which had a large PvP focus and two new challenge leagues. We made sure to avoid PvP content in those leagues so that they'd appeal to players who weren't interested in all the new PvP events. Because the 1.3.0 release was smaller than Sacrifice of the Vaal or Forsaken Masters, we felt it was important to double down on production values for our challenge leagues.

We created many different types of Tormented Spirits that have different effects and behaviours, but found that it took some time for players to understand what all of them do. Their core behaviour (touching vs. possessing) was not immediately obvious to some players. There were problems with their AI around walls, which was something we worked hard on avoiding.

Thankfully, Tormented Spirits created great YouTube moments and are a popular feature in the live game to this day. They're very economically beneficial, but in a way that is more similar to Ambush and less like Beyond.

Torment was (at the time), the most expensive challenge league that we had created.

Bloodlines (Hardcore, December 2014):
The Bloodlines league was internally described as "Nemesis for magic monsters" and explores the design space of interactions between multiple members of a monster pack.

There were several hard Bloodlines mods that players disliked (Phylacteral Link, Order of the Frozen Sky, etc). We were probably too slow in removing or nerfing these. Part of this stems from the fact that it's a hardcore league so we want it to be challenging and at times, really difficult.

The Keepers of the Trove mod was a very successful way of adding economic benefit to the league. It's really high variance and very noticeable. Occasionally you get a shower of bad rare items and occasionally really good ones.

Warbands (Standard, July 2015):
When designing Warbands, we were acutely aware that it was going to launch alongside The Awakening, a large expansion that we spent millions of dollars developing. The Awakening included so much new content that it wasn't necessary to include a Standard challenge league that was extremely in the player's face.

Our initial idea for Warbands was "themed teams of rogue exiles". It was an excuse to try out our new advanced AI and to create more complex fights to augment the existing rogue exile encounters in the game. The area-matters theme was added late in development of Warbands, after seeing how fun that was in our internal Tempest testing. This added a meta-level of engagement that allowed players to find Warbands more frequently than if they were entirely random and rare.

Special items were created for the Warbands to drop. We erred a bit on the low side with these drops, but have enough data now that we can be more aggressive with similar mechanics in the future.

The Warbands league met our design goals. It didn't overshadow The Awakening, and it added meaningful group combat experiences. Its lore content was also well received. Feedback from players was that they wanted to encounter more Warband Leaders. These values have been increased as a trial for the one-month events.

While the Warbands themselves were encountered early in the game, their good rewards certainly were not. Other than some incidental item drops, you didn't get anything especially valuable until Merciless or later. This was a mistake and has strongly influenced the design of future challenge leagues. Impactful rewards need to be available at low levels also.

Tempest (Hardcore, July 2015):
This league was initially called Eclipse and was designed around world area mods that changed over time (as the moon became eclipsed by various influences). The area-matters theme was strong, but the overall Eclipse theme was hard to explain to people.

Playtesting showed that the storm mods were by far the most fun, as they made location matter on an entirely different scale. Skilled players could position themselves and monsters to gain benefit from or avoid certain storm blasts. Eventually, the entire league was rethemed around the storms and was renamed to Tempest. This made it a lot clearer and easier to explain. The best old eclipse properties were added as Tempest suffixes.

We were reasonably satisfied with the rate that players shared information about which Tempests affected which areas. There was more trolling/griefing than we anticipated, but the community worked around that with a system of upvoting and downvoting reports of mods.

Some players felt that Tempests occasionally blocked their progress, and they had to wait them out to get through that area safely. This is especially relevant because Tempest was a hardcore league. This type of risk-taking decision that changes over time is what we had in mind with the league.

Tempest is a slightly fatiguing league to play in. There's a lot going on, and it's pretty relentless. We will try to make future challenging leagues less fatiguing.

The economic impact of Tempest was incredibly swingy. Some of the map-only Tempests were extremely rewarding, but like Warbands, these rewards were not present early in the game.

Both of these challenge leagues rewarded players who engaged with them correctly in the late game, but generally just posed additional challenge for everyone else. We would like to offer more interesting rewards earlier in future leagues.

Surpassing Torment, Warbands and Tempest were the two most expensive challenge leagues we have created in terms of development resources.

Challenges and Prizes
Challenge leagues have traditionally contained a set of eight challenges that require participation in both simultaneous leagues to complete. These challenges were designed to be very difficult to complete. While millions of players would play the leagues and hundreds of thousands of players would engage with the challenges, only a few hundred players would actually complete all of them.

In terms of prizes for the challenges, we've traditionally given out t-shirts to the first 50 players to complete all of the challenges and occasionally some microtransactions to people who completed certain thresholds. Late last year we introduced a hideout totem pole that grows with each successive challenge completed.

We feel that we can do better than these prizes. We've been experimenting with alternate prize structures in our one-month events, and plan to switch the prize structure for the challenge leagues over to something similar.

We plan to stop giving shirts out to the first 50 players to complete all the challenges. Printing a low number of special shirts and shipping them internationally from New Zealand is very expensive. We feel it would be better to spend that money on the design of in-game prizes that can benefit everyone.

The next challenge leagues will have a set of 40 challenges, which range in difficulty. Completion of specific milestones (every 10 challenges, for example) will reward you with a microtransaction. Regular casual players who don't make it to maps can still receive some prizes for their participation in the challenge leagues.

Overall Challenge League Design
Challenge leagues are an important part of Path of Exile. Releasing them frequently (especially alongside expansions) is really good for new, existing and returning players alike.

We have learnt that the theme needs to be present early on for the league to have character. We have also learnt that rewards should be present throughout the league, rather than just available for top players in end-game content. We have also learnt that many players love the lore content of the leagues. While it's still too early to reveal the next leagues, it's safe to draw the conclusion that they have earlier rewards and more lore.

Over the last year, challenge leagues have started to take more and more of our development resources. They're basically a mini-expansion unto themselves. Expectations for content in challenge leagues have risen. If we released a league that just contained a numeric balance change like Onslaught, it would compare extremely poorly to leagues that had a lot of new content like Ambush or Torment.

Players don't switch between the two concurrent challenge leagues as much as we had hoped. Most players just pick one and stick to it, missing out on the experience of the other. Combined with the intense development cycles of these leagues and the fact that we want to continue to push the amount of content they contain, we have decided to change the structure of the next leagues.

When the next challenge league starts, there will be both a Standard and Hardcore version of it, with overlapping but slightly different content. The new league represents more effort than we put into any previous league, which is possible because we're not developing two completely different leagues simultaneously. It's too early to talk about specifics of this league, but we'll probably be announcing it alongside other new content later this year.

"

Shagsbeard wrote:

I find it interesting that you place a cost on each part of the game... I'd figure you just had payroll and monthly costs for servers and whatnot. I didn't know you assigned costs to specific results. How do you break down your costs into what effect that cost should be assigned to?

For challenge leagues, it's pretty easy. The unique items, art assets and effects that are created just for that league can be readily measured. Design time is easy to estimate (there were three guys who worked on it half-time for 9 weeks, etc, etc). It's a generous amount of estimating, but it easily lets us see whether a challenge league cost us $25k or $100k.

"

Cudez wrote:

Next leagues will be announced "later this year" ?!?

Is there going to be a long'ish pause in temp leagues or what? not that I mind since I play both temp and perm leagues but some might get upset.

We have a date in mind that is later this year but don't want to announce it until we know we can hit it. By saying "later this year" it lets us shuffle it around a bit while still being accurate. Btw, I am pretty sure I used the word "probably" in that sentence which has already been stripped out when it was quoted ;)