I will preface this by saying that there are previous similar topics which pertain to acquiring the position of an element in an ordereddatastore, its been previously mentioned that this is a difficult task.
Specifically i am attempting to determine if a value is within the top 2500 of entries in the ordereddatastore with hopes that it can be scaled to seeing if a value is in the top 5000.
The algorithm ideas makes use of hourly caching the value that lies at the 2500, which is acquired through actually manipulating the pages and storing the value. When this value is acquired once or twice it becomes easier to search for the value in the future through the use of upper and lower boundaries on the query parameters. This component of the algorithm is where i find myself struggling.
Finding a more efficient and reliable way of being able to get this data in order to cache and maintain it is what I would like some advice on primarily.