Stash Tab API is extremely inefficient.

"

Maswasnos wrote:

I thought it meant Application Program Interface? The thing that external developers can use for their applications to interface with the program?

That is the most common use of the term API yes.

"

Mannoth wrote:

"

Maswasnos wrote:

I thought it meant Application Program Interface? The thing that external developers can use for their applications to interface with the program?

That is the most common use of the term API yes.

Application Programing Interface*

A couple points:

1. The size is actually much smaller because everyone should be using compression if they're reading the Stash Tab API. The compressed data is about 6x smaller.

2. The entire stash tab is sent because the API was designed to be stateless, i.e. each time you see a stash update you delete all of those items in your index and re-add them. I don't do this at ExileTools because I track historical data, so there is a bit more overhead.

3. The changes are batched to some degree on the backend because they only push out on backend save. In my testing, this was rarely more than every 30-60s. So one person doing this maliciously is barely going to register on the radar - it would take hundreds of people doing it to impact consumption. Same thing if you force it by zoning, one person would never be noticeable.

On the other hand, we're getting our own very exciting "doomsday" scenario right now thanks to GGG - every single stash tab from Talisman/HC is being sent out in a stream of updates for Standard, it's kinda exciting. It may have crashed poe.trade briefly. I'm seeing about 200 stashes / 6000 items give or take each update and it's taking me ~4-5s just to download those, let alone processing time.

"

pwx wrote:

*snip*

On the other hand, we're getting our own very exciting "doomsday" scenario right now thanks to GGG - every single stash tab from Talisman/HC is being sent out in a stream of updates for Standard, it's kinda exciting. It may have crashed poe.trade briefly. I'm seeing about 200 stashes / 6000 items give or take each update and it's taking me ~4-5s just to download those, let alone processing time.

I am waiting for the data to come... it is probably gonna crash me hard

I do not understand the problem. I think that there is lower then 1000 people that will update 20 premium stash tabs at the same time. So it will be 3Gb of data at one update. The data is super easy to parse (it can be easily made in linear time with very small constant) in less then 15 seconds. Then indexers need to update there data structures. Information about items is highly structured data, so it can be easily made for the same 15 seconds.

What is the problem? Items in POE is NOT ad Big Data, it is even not close in all meanings.

"

PGLoLo wrote:

So it will be 3Gb of data at one update.

That is exactly the problem.

Do you have an internet connection that can download 3GB of data every minute?
Does anyone have such internet connection?

Not to mention POE servers will have to send this amount of data evey minute to hundreads of people using public stash api.

its still better than the old system lol

or lack of a system....

So people will be buying 20 premium stashes and spam them in the off chance that network might get congested.

I am sure everyone would be doing this for the unfair financial gains especially after purchasing 20 premium stash tabs.

And then GGG can easily (and i mean fucking easily) impose a forced time out... and the guy just ripped 20 premium stashes because their life goal was to make the game run a little slower while gaining absolutely nothing.

And furthermore, even if GGG doesn't do shit about it, its still better than the old/none trading system.

"

mark1030 wrote:

Oh sure, just tell everybody about my plan for world domination. Thanks a lot.


Seriously though, good point. I'll have to take your word for the technical part of it because I don't know how the API works. Or even what API stands for.

 API is just "Application Programming Interface", a generic term used to describe a software method for interaction between 2 code processes (or humans and software). Just how slow the new PoE API is at allowing 3rd party selling sites to get our "wares for sale" is unclear, but if it's as bad as trying to show a build's stash from here in the PoE site then RNGesus help us all.

"

pwx wrote:

2. The entire stash tab is sent because the API was designed to be stateless, i.e. each time you see a stash update you delete all of those items in your index and re-add them. I don't do this at ExileTools because I track historical data, so there is a bit more overhead.

Couldn't GGG offer both? A stateless river, and a lower-bandwidth river which has specific changes only? I imagine the difficulty would be getting the correct initial state while listening to the lower-bandwidth river, but perhaps there is some form of account-specific query to fetch the correct initial state (off-river) to get this information. And the stateless river would still be an available information source for those who decide to go that route.