Odd behavior with Anti Aliasing

Upon seeing I was only getting ~30fps on max settings I went into the options to ease the graphics a bit. Surprisingly, all of the options besides AA had virtually no effect on frame rate. Anti Aliasing though, had massive and seemingly odd effects. I don't usually pay much attention to this stuff so it may be normal, but the results I got certainly weren't what I was expecting. I tested each setting and these were the frame rates I got (all other settings remained maxed):

Off: 200FPS
1: 143FPS
2: 143FPS
3: 77FPS
4: 143FPS
5: 71FPS
6: 48FPS
7: 63FPS
8: 32FPS

Obviously setting #3 and #6 stood out, as you would expect them to get at least as much FPS as the subsequent settings. Also, the FPS being cut in half from 4 --> 5 and 7 --> 8 seemed a bit extreme. Beyond the weird behavior, it was unexpected that I would get such poor performance in a game with relatively simple graphics.

Hardware for the record:

i5-3570k @ 4.6Ghz
HD7870 2GB