How the Open Beta launch went for us

The last few days have been amazing for us. After working for over six years on Path of Exile, we are proud to say that its Open Beta launch was a complete success. We've already addressed several early stability problems and are very pleased with how the game is currently performing. We hit a peak of 56,700 simultaneous player earlier today and we're looking forward to try to smash that record over the weekend. This is far more interest than we expected, though thankfully very close to what we purchased server capacity for! This news post describes what the launch was like from our point of view.

Leaving the team in Erik's capable hands, Jonathan and I flew to California last week to show Path of Exile to an array of journalists. We got back a few days before Open Beta was due to begin and soon realised the effect that our press tour had had on the exposure of the game. Website traffic was rising by the day and dozens of positive articles were appearing online. We quickly ordered more servers. This turned out to be a good idea.

I will always remember the day before Open Beta as a day of controlled chaos. With less than 24 hours to go we were putting finishing touches on most systems and performing final quality assurance. That day saw a lot of improvements to the final boss fight of Act Three and the general stability of the 0.10.0 patch. We managed to get everything done that we wanted, but it took the team all the time we had available. By midnight we were down to a core group of developers mostly working on creating videos and descriptions for the microtransaction effects. Several of us worked through the night, but by 7am we had a final build prepared for deployment. By this time, our sales figures were going absolutely crazy as everyone rushed to buy their last minute Closed Beta kiwi packs. We were receiving new Diamond supporters at the rate of several per hour. It was with great hesitation that I pressed the button to end the Closed Beta and deploy the 0.10.0 patch.

Our "twenty minutes of expected downtime" was over before I even noticed. It turns out that our server hardware is indeed as fast as Jonathan was telling me. With this deployment complete, 0.10.0 was live on the realm but players were not yet able to log in. We quickly logged in to check that everything was working, which it thankfully was. We had around 90 minutes before the countdown hit zero, so we let players download the patch and grabbed some breakfast.

Like all companies that want to transfer a lot of data, we were using a CDN (Content Delivery Network) to deliver the patches to our users. CDNs have a lot of servers all around the world so that people can download files quickly from the closest ones to them. During the Closed Beta, we frequently received emails commending us on how quickly people could patch from us.

This did not scale as well as we hoped when approximately fifty thousand people tried to download the 1.5gb patch at the same time. We had warned the CDN in advance, and they had bolstered our capacity, but more was needed. As they added servers to their network for us, more and more users started to patch. By the time the Open Beta countdown hit zero and users started to join the game, it was apparent that some users near very crowded CDN servers were going to take many hours to patch. We are very sorry about this and are doing what we can to keep it fast in the future.

Each hour, approximately 9000 users finished patching. We watched the number of online players rise steadily while the servers held up fine. Just as we began celebrating, a cake arrived, sent by a longtime fan and Diamond supporter tpapp157. We assembled the team together and enjoyed the cake. I made some vague semi-speech about how well sales, traffic and server stability were going, but I had been awake for 30 hours by that point so it probably didn't make much sense to the team.

Our peak hours for concurrent users are usually in the morning, NZ time (which is in the evening for Europe and in the afternoon for America - the only timezone asleep then is Asia, where we have done no marketing). This meant that by early afternoon we had hit a local maximum user count that the servers could comfortably handle. We prepared the 0.10.0b patch (patch notes here) and I went home to sleep. The patch was deployed successfully, though our CDN bandwidth issues meant that it took a while for some users to download the 8mb that the patch needed.

By about 5am, the realm had crashed several times, booting everyone off each time. When I arrived at the office, Jonathan was preparing a fix which was deployed, resolving that issue. There are now no known or unknown realm crashes. Since the last crash, the server has been very stable.

Today was spent answering emails. Our support email and address and PM box have been receiving literally thousands of requests, ranging from the simple account issues to complex tech support questions. Despite having full-time support staff, we moved many developers over to help with answering questions in an attempt to catch up with the queue. There is still a giant backlog while we look into hiring more support workers (only apply for that job if you're in Auckland, NZ). If you've written to us, please be patient because it may take us a few days to get to your message.

We also changed our policy on the deletion of microtransaction items today, and offered some extra stash tabs to Closed Beta supporters who bought a lot of tabs before the Open Beta started. We are constantly trying to improve our microtransaction shop so that it's as fair for everyone as possible.

We hit a peak online player count of 56,700 players online today. I expect we'll beat this total in coming days. Jonathan has a lot of interesting stats he plans to share with you about the Open Beta, so we'll try to post those over the weekend. Expect the 0.10.0c patch sometime within a few days also.

Overall, the day we entered Open Beta we had as many visits to the website as the entire first year of traffic after Path of Exile was announced in 2010. We sold approximately $200,000USD of Kiwi packs in the last two days of the Closed Beta. This has been earmarked for server and bandwidth costs, which are pretty high right now. I believe in the Closed Beta we crowdfunded a total of around US$2.5m over 35 weeks, though of course much of that has been spent on development costs.

Sales have of course been much more sane in Open Beta now that we're not selling Kiwi packs or Closed Beta keys and players have a lot of microtransaction credit to spend before they need to top up. Every purchase helps the development of the premier Action RPG for hardcore gamers. We are deeply thankful for the ongoing support, and look forward to sharing many adventures in Wraeclast with you!
Lead Developer. Follow us on: Twitter | YouTube | Facebook | Contact Support if you need help!
rings2
Good read, love that you let us know how things have been over the last day.
Personally very surprised how smooth the launch was and that everything went as well as it did.
Must say that ACT III looks so amazingly good, WAY over my expectations. it really puts ACT I and II in the dust in every aspect.
Cant even imagine what the future hold for us, blown away with ACT III...
Im so so happy that I found this game and supported it.

HURRAY HURRAY HURRAY FOR GGG!
IGN: Flashy | League:Nemesis | Time Zone: GMT 0 (UK) | Play Time: Somewhere Between 10:00 - 02:00
Last edited by Jillys on January 25, 2013 11:25 AM
You guys are awesome.
Still here, professional forum lurker at the moment.

Contact support@grindinggear.com to report issues relating to the game or forum. Thanks!
The Launch was very smooth! I dont care about those little crashes.

Great Game, Great Fun ! Thanks!!!!
I've searched for an upvote button but couldn't find :/
Timezone: GMT+2
IGN: Basubadelmevt
-----
view-thread/876906 - Ambush/Nemesis Challenge Rankings
Interesting read. Thanks for this fun game.


Did you think about new player models or different genders available in the shop ?
Wand's don't kill people, i kill people, with Wand's
game is great,i didnt had much time to play it so im just lvl 9 but everything works good for me and community is just great!at least till blizz fans dont come to troll here...I hope to see some expansion for act 3 or maybe act 4 in the feature or some sort of other interesting features.I must say that before this game come in OB and wile i played CB my brother and some other people didnt like much this game when i was showing to them,mostly cause of graphic but now they quit games they been playing and played this alot more then i did,and they love this game.Good job GGG.
Keep up the good work GGG and keep the good times rollin'.

Regards from Wraeclast.
http://www.pathofexile.com/forum/view-thread/294112 - Lowbie farming method
http://www.pathofexile.com/forum/view-thread/324883 - Simple griefing avoidance guide
I have been a king, I have been a slave,
I have been warrior, madman, fool, and knave.
Nontheless the "launch" was one of the smoothest i've had in a long time.
the second the count down hit 0 i could log in and play. you guys are officially amaezing getting this with this small of a team done what btw blizzard with a way bigger team did not get done.
love the game
keep it up
Congratulations again on the launch. Enjoying the new content so far, hope to see the game get even bigger and better in the future.
Would anybody like some... POUNDCAKE?

Report Forum Post

Report Account:

Report Type

Additional Info