C sided API for data structures

Feature Requests Engine Features
#1

As a Roblox developer, it is currently too hard to efficiently use data structures. While Lua can be used to implement data structures such as link lists, hash maps, heaps, trees, ect trivially, having a C sided API would speed up this process dramatically. In addition, it would save developers’ time from having to implement these data structures on their own. It would also allow newer developers to use data structures without intricate knowledge of how they actually work.

If Roblox were to implement this feature it would allow me to make my game more efficient and would smooth out the workflow thereby increasing productivity.

Pretty much every engine already has this implemented, so why can’t we?
I personally would find this very useful as there have been times where I needed something like this but didn’t have the skill or time to implement my own data structure system.

7 Likes
#2

I do support this very much, but I am wondering (very curiously) when have you needed to use a more specialized datastructure in your game?

In my scripting on Roblox (I like to think that I do more than just make games), I only use a red-black binary search tree to mimic the stable sort of UIGridLayouts and a binary heap to make my custom bytecode interpreter faster

I am also wondering what is missing in the Lua table implementation that you need a separate hash map?

Also, I think it would be nice if you extended this data structure feature request to allow for common algorithms because I think compression is especially slow and impractical right now on Roblox (https://devforum.roblox.com/t/im-so-excited-about-my-compression/75042/19)

#3

Using your red-black tree example, rebalancing your tree would be a lot faster C sided and you wouldn’t have had to program the insertion for your red-black tree which is pretty intensive to write considering all of the edge cases.

#4

I am not contesting the ease of use or speed of Roblox C implemented data structures; I fully agree with you that it is much easier to use something than to make it and that C++ is much faster than Lua

My post asks about your use cases and why you want a new hash map over the existing Lua table

#5

Everything can be implemented in a Lua table, but it’s a matter of doing things faster and easier.
It just makes the workflow smoother so you’re not spending time implementing red-black trees or hash maps. Oh, and did I mention that they’ll be faster than anything you can write in Lua because they’ll be faster than anything you can write in Lua.

Since the differences between hash maps and Lua tables are so few and far between, there’s basically no use case for them at least in the context of Lua. (Although my original post listed hash maps, I was merely giving that as an example of a data structure type.)

As for that example of a use case for a specialized data structure, I’m working on a pathfinder right now and it’d be really nice if there was a built-in graph system so I didn’t have to write it myself.

1 Like
#6

As much as I wanted to see this implemented (because speeeeeed), I doubt it will ever happen.

Disclaimer: Anyone who happens to be better at this matter than me, please correct any inaccuracies you might find - Albeit I know some stuff about C/C++, I can only speculate about how it is used by Roblox to power it’s engine and/or API.

As you can verify, C++ (That’s how engineers make the very Roblox core and API’s) is in a whole different league of Lua, in the sense that:

  • It is compiled, not interpreted (basically where it’s speed derives from);
  • It is rather at a lower level than most programming languages like Java (A string in C/C++ is an absurdly vague concept, that’s a char array) - hence why I personally call it a mid-level language;
  • Not dynamically typed, and other nuances™;
  • In the end, it’s not that straightforward enough for younger children (13 years old) to begin with, and that’s why Python is so popular, for example. Not being able to say that my previously number variable is now a string? No way!

So if Roblox was to open their C++ context, a whole new set of questions would arise (which I don’t think that have an easy answer):

  • Why having the developers learn two languages for the same task when in the end only one is needed?
  • How do we sandbox the C++ context so that malicious developers don’t interact with parts of the API we don’t want them to tamper with? Or can we guarantee they don’t find a way out of the sandbox and run malware on the client’s PC?
  • How long would it take to compile the code? Which compiler? What options?
  • What would be the style standard to be used?

So basically having this C sided API would require a massive weightlifting from their part, which I guess it’s not worth the time. One could eventually think about employing LuaJIT (basically Lua compiled IIRC), but it’s likely that other issues of their own would arise.

1 Like
#7

Yes, the way you have said that would not be implemented and would be very very stupid.
Asking for an API to be added does not equate to asking for a way for C to be executed in Lua.

I’m asking for a C sided API for data structures. I’m asking for a service to be implemented that would allow you to use data structures easily from Lua through the use of an API. This is the equivalent of every other API on roblox (i.e. raycasting, vectors, creating a part through instance.new. Literally everything on roblox is inside a C sided API.) I’m not asking for you to be able to run C code in Lua.

I’m asking for a way for newer developers and seasoned developers alike to not have to sit there and write out their own data structure system every single time they need it.