A new release is here again, get in on the github releases page.
This one is relatively small, it just adds a new Icon by @Loominatrx and also adds support for adding a new entry to a datastore.
1 Like
Awesome tool you have created here!
Although i have some suggestions for the application itself.
-
Can you disable the CMD window when the application is running
It isn’t a huge issue or anything but disabling will get rid of whatever this is called. (IIRC you can disable it very easily in Visual Studio)
-
An option for darkmode maybe? who doesn’t love dark mode
Oh yeah I can definitely get rid of the console, will do that for the next release.
Dark mode is a little harder, the only dark theme I could find right now still only claims to be compatible with Qt5 and I would really want both 5 and 6. I’ll look into adding that and see how much work it will take to make it work with Qt6.
In the mean time I did just add styling support so you can have a little more variety in appearance. Two new styles in the next version will be the ‘windows vista’ style and the platform independent Fusion style that is available on both Mac and Windows, though neither is particularly dark.
1 Like
Once again, a new minor release is ready. This one cleans up a few things recently reported in the thread but has no real change to functionality.
The dark theme was slapped together very quickly so let me know if there is anything that doesn’t look right.
Win-Installis an installer containing both Qt5 and Qt6 versions of Open Cloud Tools.
Win-Qt6is the build users of Windows 10 and newer should use.
Win-Qt5is the build users of Windows 7 and 8 should use.
Macsupports recent versions of MacOS but I’m not actually sure which. To use the Mac version you must runxattr -cr /Applications/OpenCloudTools.appin the terminal after installing to make it runnable.Changes:
- Added selectable themes
- The console window no longer appears on Windows
- Added ‘Desktop Shortcut’ option to installer
Can the IP be changed to my localhost to give me more security in case my API key is leaked and cannot be used on a computer other than my own?
P.S: I love this, but I would modernise the graphical interface.
This is absolutely fantastic and super easy to use. Thank you for making this!!!
1 Like
You can absolutely use your own IP if you have one that doesn’t change a lot. The only real issue there is that some ISPs will reassign ip addresses on some schedule and then the key would stop working. You just have to deal with keeping the authorized ip up to date.
1 Like
First I’d like to say that this application is simply amazing, you should setup a ko-fi so I could donate a coffee or something! I’m in the process of attempting to dump a large datastore, 266m+ entries I’ve been letting it run for about 4 days as its enumerating entries however I had an internet outage and it said received HTTP aborting, I’d love to have a feature that would remember where you left so you could pause/continue.
2 Likes
Thanks for the suggestion, having that many entries in a datastore is a use-case I hadn’t really considered. I think I can do a couple things about it.
The easy one is to make the downloader more resilient to errors like that. Ideally it should probably never hard stop due to a server error once the download process has started. I can definitely fix the case of it quitting permanently when your internet connection goes out.
I can also add a separate mode for bulk download that progressively stores more info, including the list of available keys, in the db as the information becomes available. That way it would be easier to quit and resume. This one is a bunch more work though, so I probably won’t be able to do it particularly soon.
That said, with a datastore of that size it may just be impractical to fully download through Open Cloud like this. Each enumeration request returns the name of about 50 keys. If you are spending 4 days on the enumerating step then it will take at least 200 days to actually download the contents of those keys (400 if you checked resave). That is due to the rate limiting on Roblox’s side and there isn’t anything I would be able to do within the program to get around it.
2 Likes
So I’ve started having a go at this and it will take a little more effort than I first thought. The big issue is that I noticed a couple more places where I am hanging on to data per-key longer than needed.
Ordinarily this isn’t a problem, but with over 200m entries I need to save every byte possible to keep this feasible. Right now with just the retry fix in you still probably wouldn’t be able to start downloading, with the amount of data stored in memory per-key you would probably need 50+ gigs of ram just to keep it all in memory at once.
I’m working on restructuring some code to optimize this, but it will take a bit longer than I had thought at first.
1 Like
Man, I really appreciate you doing all of this! Thank you so much!
I got the key from the creator portal on roblox. Any ideas?
Solved:
Add 0.0.0.0/0 to whitelisted IPs.
That most likely happen because your ISP changed your IP and the Roblox API did not recognize the new one. Just update it everytime it needs to be updated.
After a ~2 month break, another release is finally here.
This still only includes some of the fixes that can be done for big datastores, but it is hopefully enough to do something with.
It still can’t resume across multiple runs of the program, but as long as the download progress dialog stays up all errors should be retryable now. The more siginificant errors will require pressing the retry button.
This also includes the number of enumerated entries in the progress message now, so you’ll be able to know how far along it is. After a few million probably check the memory usage to make sure it looks like there will be enough for all ~220m. Memory usage won’t continue growing after you finish the enumeration step so if you get that far you should be good.
Win-Installis an installer containing both Qt5 and Qt6 versions of Open Cloud Tools.
Win-Qt6is the build users of Windows 10 and newer should use.
Win-Qt5is the build users of Windows 7 and 8 should use.
Macsupports recent versions of MacOS but I’m not actually sure which. To use the Mac version you must runxattr -cr /Applications/OpenCloudTools.appin the terminal after installing to make it runnable.This release focuses on making OpenCloudTools better at dealing with large datastores and recovering from errors. Right now these improvements apply mainly to the ‘bulk download’ feature.
Changes:
- Allow retrying any non-bulk datastore operation
- Allow retrying all bulk operations during enumeration phase
- Allow retrying a bulk download at any point
- The log will display more information for non-http network errors
- The status message during bulk enumeration will now include the running total of entries found
- Limit text log to 10k lines
- Limit http log to 1000 entries
- Improve Dark Theme
Third-party libraries:
- Mac build now uses Qt 6.3.2
- Windows Qt6 build now uses Qt 6.3.2
- Windows Qt5 build now uses Qt 5.15.6
2 Likes
I didn’t need this software, but everything for some reason attracted me, good job mate.
Minor update today. This fixes a minor annoyance i’ve run into a few times where pasted text retains its formatting. It also includes the latest available version of Qt. This notably means that the “Fusion” theme will now automatically match your os choice for light mode or dark mode.
Win-Installis an installer containing both Qt5 and Qt6 versions of Open Cloud Tools.
Win-Qt6is the build users of Windows 10 and newer should use.
Win-Qt5is the build users of Windows 7 and 8 should use.
Macsupports recent versions of MacOS but I’m not actually sure which. To use the Mac version you must runxattr -cr /Applications/OpenCloudTools.appin the terminal after installing to make it runnable.This is a very minor bugfix release which also includes updates to the latest versions of Qt 5 and 6.
Changes:
- All multi-line text entry boxes now only accept plain text
Distribution changes:
- Mac build now uses Qt 6.4.1
- Windows Qt6 build now uses Qt 6.4.1
- Windows Qt5 build now uses Qt 5.15.7
1 Like
@Skrateboard This is awesome! Feel like I need to pay some Robux for this to be honest. let me know if there’s a way I can send over some support and thank you for putting this together.
1 Like
Don’t worry about it, I’m really only making this for myself because I wanted an easier way to manage a game I’m working on. Just sharing it here in for anyone else who finds it useful, I’d be doing this work all the same even if no one but me was using it.
1 Like
{"error":"INSUFFICIENT_SCOPE","message":"The api key does not have sufficient scope to perform this operation.","errorDetails":[{"errorDetailType":"DatastoreErrorInfo","datastoreErrorCode":"Forbidden"}]}
This happens after downloading a Datastore and trying to upload it. The API key has ALL datastore permissions.