GreedyDataService || The simplest datastore solution that is session-locked and automated

Resources Community Resources
#5

Can you elaborate on this further?

Seems like a questionable design choice to me.

6 Likes
#6

This is made possible with boatbomber’s HeatUp module:

You are still using data stores, but memory service is responsible for changing your data saving keys, bypassing the data store limits.

5 Likes
#7

I’ve added a YouTube video to help you all out!

3 Likes
#8

Roblox imposes a 6 second cooldown per key (across all your servers) in Datastores

I’m pretty sure that’s false.

Datastores operate on a queue system - if you exceed 60 (+ numPlayers * 10) requests per minute in a server, subsequent calls will throttle.
Short of the inherent ping delay that comes with a HTTP call, there is no delay between calls to speak of, provided they don’t exceed the queue limit.
Feel free to test this though, maybe there’s something I don’t know of.

You realistically shouldn’t be hitting datastore limits either if you implement a server cache, do all your data operations on said cache, and only save the cache to datastores when you are irrevocably done with it.
That’s how ProfileService does it anyways, and it works.

It’s not entirely your fault here, I just don’t see the point of HeatUp at all.

7 Likes
#9

I think this was increased, but there is a cooldown per key. HeatUp removes the stress about this and replaces it with the much more flexible memory store service limits.

This also simplifies things since the code doesn’t have to do a bunch of caching and stuff, which in turn makes it more readable.

4 Likes
#10

I also don’t see why a cache would make things unreadable here.
It literally just means that instead of operating on a datastore (which comes with its own problems), you operate on a server cache (eg: a module script).

5 Likes
#11

Thanks for letting me know about that announcement.

I looked at the current limits and this is what I found:

image
image948×667 78.6 KB

Based on this I think it’s still much better to use MemoryStoreService:

image
image930×138 33.4 KB

Is it overkill? Probably. Nonetheless, it doesn’t harm anything to use it.

4 Likes
#12

Whatever floats your boat here, there’s just two things to keep in mind here:

  • If you do proceed with MemoryStores, you are now subject to twice the amount of HTTP calls you will need to make.
    That’s twice as likely your data operations may drop and mess something up.

  • If you decide to drop this feature entirely, you will basically be making a carbon copy of ProfileService.

8 Likes
#13

I should say not quite, since the setup of this is easier. Having to create and resolve sessions in profile service requires a lot of boilerplate code, while this just requires one single function call. I made this out of frustration coming from how unnessesary it is to write the boilerplate for profile service, and I wanted something simpler to speed up development.

I may drop heatup tho.

4 Likes
#14

That’s fair.

I wouldn’t use this myself still because I value reliability over convenience when it comes to data, but I think it’s fine for simple projects.

5 Likes
#15

I’ll definitely keep stress testing this as I do want it to be scalable for large projects, but as of now, definitely just test it in smaller scopes!

6 Likes
#16

Could be useful for leaderstats and those kinds of stuff.
Nice work, i may be seeing myself using it when i am testing stuff.

6 Likes
#17

That’s not a negative, that’s a huge seller for ProfileService. Much like BridgeNet2, although its API can be used raw, you get the most out of these resources when you write wrappers or libraries around them. As for the “complexity of resolving sessions” being spoken of here, it can be handed off to ProfileService with the ForceLoad string - you don’t need to concern yourself with any of that unless you want tailored session management logic. The setup is not as complex as you think.

The “unrivaled simplicity” of this module wrestles away control from developers about how to reason about data loading and session conflicts. Boilerplate can surely be annoying, though ProfileService is highly designed for that use case - it can be standalone, or integrate with your own code, which I found in every case (production-level experiences to be specific) that writing wrappers for ProfileService makes it shine the best, even better if it’s tailored.

A lot of developers are creating new resources to create competition against ProfileService which is respectable, but simplicity is not a high selling point especially if most of the loading logic is black boxed into the module itself and it doesn’t offer feature parity or superiority for more complex use cases given that this is suggested for dedicated projects as well.

My two cents: if developers are having trouble with ProfileService, it would be better to just write universally applicable wrappers for everyone to take advantage of so they can have the full benefits of ProfileService and more of the complex things handled already by said wrapper. It would be nice to see developers creating wrappers rather than adding to the already saturated market of less-feature alternatives over some small gripes with existing battle-tested designs.

I like ProfileService and Lapis on the account that they allow a great amount of extensibility. On the other hand though, this resource doesn’t sell me much beyond the point of “simplicity” which is a very common footgun for resource developers to sell on. Simple doesn’t necessarily mean better, it could even be harmful to an experience’s data architecture.

The section that tries to market itself as an alternative (“why should this be used over others”) is not really that convincing and tries to tack on real benefits as demerits against using them. I could write a wrapper for ProfileService that handles exactly all the same features offered by this resource without requiring developers to need to do “heavy setup”. Now, this doesn’t mean objectively that this resource shouldn’t be used - you have the freedom to pick whichever resources you feel best suit your needs and level of expertise as well as how willing you are to learn its application. It just doesn’t feel like a strong seller if you want to enter the saturated DataStore resources market.

10 Likes
#18

Agreed. And I also believe there are a couple of wrappers for ProfileService anyways.

2 Likes
#19

You should make it so that the client can access their own data through the same module!

This would make getting public/private data easier, and may also make saving incompatiable data types (Colors, Vector2, etc) easier. It would use the same methods, except Setter methods wouldn’t work, and they can only get the localplayer’s data.

3 Likes
#22

Fair points, but something I would like to mention is that simplicity over functionality isn’t always bad, that is, if that functionality isn’t being used. In my experience, my wrappers for it don’t do anything extra than what’s needed for just basic saving and loading, which is why, at this point, why not use this instead?

Part of me is frustrated that there are so many data store libraries on the DevForum since it does just feel like a “here we go again” type feeling.

“Simplicity” is a bit cliche on the forum here, but in this case I think it’s a fair sell. Lots of dirty work is automated, including session locking/unlocking, playerstat leaderboards, value/valueset declaration, reconciliation, and of course the base functionality.

I see myself using this over others now since I don’t use the extra functionality of external libraries and prefer a straightforward approach to set up a game with ease.

Which features would you like to see specifically? Admittedly it is a bit unhelpful to not know, albeit your feedback is nonetheless valuable.

I would like to point out that I am working on gdpr compliance.

Last note, I also am trying to gather feedback from other experienced resource creators, including loleris himself, to help improve this. Consider this an early baby!

3 Likes
#23

I can agree with this part, I am planning to recreate some modules but with less features and is easier to set up

3 Likes
#24

It strongly depends on where the “simplicity” lies. In this case, it seems as though “simplicity” refers to a non-extensive developer-facing API, which doesn’t really say much. ProfileService is an equally simple library in my opinion, it’s just loaded with a lot of developer-facing API for you to take advantage of. Certainly, not in all cases, but in most cases, this word can be a huge noob trap and lose out on benefits offered by other resources in more robust fashion. That’s part of where I say that personally, the selling points aren’t convincing.

I generally understand what you want to achieve with this library though - automate most of the things that you would otherwise be expected to write yourself with ProfileService (which already automates, so that’s to say it offers a higher level of abstraction) and leave developers to spend less time writing and more time deploying their data tools. The main thing I wanted to convey is that not having some of these things abstracted gives back control to developers and lets them reason about how most of the processes of their data systems are conducted, which is a valuable skill to have.

For pure example: ProfileService lets me decide how to handle the case of a locked session or profile that failed to load so I could either retry in a bit (assuming it’s not an outage) or send them away temporarily to a place that doesn’t require their save data. GreedyDataService immediately assumes that the player should get kicked. You can’t sidestep this without

  1. Setting loadingTimeout to math.huge which will cause an infinite yield.
  2. Modifying the source code, which another library would better serve by allowing you to hook into the loading process or handle state.

It’s reasonable to consider that not many features may be used and may be particularly niche therefore excluding them, but serving a wide variety of use cases - especially of part of your target audience is production-level deployment - really helps future proof the library for those power users. The fact that they’re also battle-tested also helps give me confidence in using them over a novel new library.

As a strong advocate for and user of ProfileService, I use most of its features. There’s a lot of things I like that it offers. While you and your consumers may not use most of the features there and that’s why they aren’t here, the wealth of features helps empower data tooling workflows that are cornerstone to providing developers and players with an appropriate save data experience.

  • Consistency. Player data is not the only thing I use data stores for: I also use them for global configurations, specifically world events. GreedyDataStore only supports player data. While that is not a negative, its inflexibility can lead to inconsistency with data management.

    • GreedyDataService only allows data storage under the GreedyDataStorage namespace. It does not offer support for names, scopes or key structures.

    • This is similar to the limitation that puts me away from DataStore2. A data library purely focused on players is not bad at all, but if another library allows me to use the same library and have the same guarantees for players and non-player entities, I will always preference the latter.

  • Reconciliation is at my discretion. I often reconcile immediately but for trickier requirements like data structure migration, I want to wait until I’ve gone through all migrations first before reconciling since reconcile assumes attaching the latest keys to the current data.

  • Versioning. Player data is incredibly tricky to get down well and it’s incredibly easy to make mistakes. Having support for version rollbacks helps developers gain the confidence they need to rectify data corruptions and other issues. The raw DataStoreService API for versioning and rollbacks requires boilerplate and is a particularly ugly process compared to being given an API that’s human readable and easy to use (e.g. Roblox enforces a UNIX timestamp; ProfileService allows nil, a DateTime object or a UNIX timestamp).

  • Mocking for testing data stores in Studio or test environments without data saving. My two cents is that every DataStore library should include the ability to mock so you can stage code that relies on save data. ProfileService offers this as the Mock member of a ProfileStore. Lapis offers this as a configuration that accepts either DataStoreService for live data or a mock library (e.g. buildthomas’ MockDataStoreService library) for local testing.

  • Clear design for GDPR compliance. Why is this coming soon? This should launch with it. As long as there’s even just a temporary solution before it’s officially supported by the library’s API, that will work well. As you are using an external library for data management, the structure of DataStores is not entirely clear and requires a deeper dive to figure this out.

  • No early-release APIs. GreedyDataService is greedy about how sessions are handled and the contract here is that sessions are active between join and leave, no questions asked. You as the developer cannot defer opening a session or close one early and determine what happens to a player when a session ends (e.g. should they teleport, be kicked, open a different session, etc).

  • Can’t force data to save. I’m cautious: I force saves at critical moments such as Robux purchases. ProfileService will take a force-saved profile out of the current cycle’s auto save queue for me and reset its timer.

    • There are times when you just don’t want to wait for an auto save cycle, especially with high cloud services pressure now. Peak times for your experience, larger experiences updating and crashing cloud services and holidays are all key points when force saving starts becoming a palatable idea so progress isn’t rolled back.

    • Shortening auto save intervals beyond 30 is a bold move depending on your data sizes given the existing throughput limits.

Fair sell? Sure, I speak for myself when I say it’s not convincing. A lot of developers, whether to their own benefit or not, are attracted to the idea of “simple”. It’s like a magic word that sets off all the happiness sensors. You don’t have the company of the APIs you’re not going to be using… though that’s irrelevant, you could just ignore them. Feature-rich libraries aren’t always used in full by their consumers, and you just need to pick out what you need. The more robust a library, the more cases it covers. Primarily why I’m an advocate for wrappers and tutorials moreso than “alternatives”.

5 Likes
#25

Thanks for the feedback, I’ll make adjustments accordingly. While I will keep the simple approach, I’ll add flexibility and functionality for what you’re looking for (so that developers such as yourself can override the default and use niche stuff). Appreciate it, keep an eye peeled for these updates soon :+1:

6 Likes
#26

Hey guys! I’m currently working on a bunch of changes that will be rolled out incrementally in the very-near future.

Check out my todo list here: https://github.com/MiaGobble/GreedyDataService/blob/main/todo.md

Now is the time to give feedback for these upcoming changes!

1 Like