Incident Report for April 1 Catastrophe

Path of Exile service was disrupted today by a severe bug introduced in a patch this morning. We would like to sincerely apologise for this incident and assure you that we will be taking steps to prevent similar problems occurring in the future. This news post is the incident report describing what went wrong and our expected timeline for resolving it.

During the week of March 23rd, one of our senior gameplay programmers was performing a routine refactor on our core skill code. In preparation for our upcoming expansion, changes were made to the functionality of Trigger Gems. The goal of the refactor was to improve the reliability of the gems that trigger spells on various conditions - Cast on Death, Cast When Damage Taken, etc. During this process, the names of the gems were entered by hand. Unfortunately, due to a combination of tiredness and the programmer's previously undiagnosed Toxoplasmosis infection, the word "Cast" was entered as "Cats" for these gems.

Unbeknownst to the programmer, the consequences of this small typo were severe.

This change was deployed to our production realm at 10:05am on Wednesday, April 1 (NZ time) as part of the 1.3.1g point release. Players reported that the Cats Trigger Gems now spawned black cats when their conditions were met. Cats When Damage Taken suddenly started creating cats in addition to endurance charges. A feline army mourned the passing of Hardcore Exiles who used the popular Cats on Death build. Needless to say, these new gems became a powerful combo with Chance to Flea.

The first report of this issue occurred at 10:09am. Our quality assurance department was able to reproduce the problem by 10:33am. At this time, an investigation into the origin of the problem began.

We are currently preparing a patch to resolve this issue and restore the Cats trigger gems to their previous functionality. We don't have an exact timeframe for the patch (our current attempted fix has issues with Leeches spawning for some reason), but we're expecting to have it resolved by the end of April 1, Pacific Time. Once this fix has been deployed, the trade value of claw weapons should begin to normalise.

In light of the fact that typos can summon animals from nowhere, we will be dramatically changing our development methodologies. Not only must programmers now copy/paste existing game terms (to avoid typographical errors), but we will be improving our code testing framework to detect the accidental introduction of animal names into the game (not only as gems but also other areas such as the Goats Reaver keystone).

On behalf of our development team, I would like to deeply apologise for this issue affecting the live realm and your play experience. This incident does not reflect the level of service you should expect from our company, and I will personally ensure that it does not happen again.
Lead Developer. Follow us on: Twitter | YouTube | Facebook | Contact Support if you need help!
maps3
It's ok Chris. Cats make mistakes too.
http://tldrexile.com
Last edited by Ocylix on Mar 31, 2015 6:39:01 PM
Amazing.
cats on damage taken <3
Almost got me, not april 1st here yet.
ign UntamedBride

My shop, thread 743136
Make an offer here or In game.
Usually online everyday, even if it's not for long
haha
Don't forget to drink your milk 👌
what the fuck did I just read
IGN: @Sleepless
Twitch.tv/iamsleeplesss
Woo! First page!


Also, Chance to Flea was pretty clever, but Goats Reaver was just brilliant.
Last edited by Varthorne on Mar 31, 2015 6:42:05 PM
nice April Fools!
trolololol

Edit: "Chance to Flea" damn can't stop laughing!!! xD
CI >> Life
Last edited by sonafab on Mar 31, 2015 6:41:35 PM

Report Forum Post

Report Account:

Report Type

Additional Info