Hello 
I thought as this is a pretty common topic and I can’t remember any general resources for this sort of thing, so I thought I’d write up a resource on it, and hopefully help make some of my own experience on optimization for the time I have been programming on and off Roblox accessible on the devforum.
Optimization
Optimization is more general than programming and pretty much is just the process of adjusting something (maybe a math formula, or a program) to minimize or maximize something, for example, the result of the math, or the time it takes for said program to run.
When it comes to programming though, it mainly only ever focuses on reducing the amount of time something takes to compute, or for a program to run, by simplifying it, or taking shortcuts. Another time optimization applies to programming is in memory usage.
How exactly do you optimize code?
In short, there’s not really any one way to optimize code. Optimizing is a lot like programming itself and ties closely with a lot of the skills you learn while learning to program. There’s a lot of ways you might optimize certain things, and, there’s many approaches you can take.
Code simplification (cost complexity)
The most effective (and the most important) kind of optimization is mainly just simplification. Typically the less complex an algorithm is, the faster it will be. Complexity in this case doesn’t really mean how big, or how advanced a program is, but, more so how much of something it requires to do some amount of work. A useful analogy might be energy.
Something requires a lot of energy to move a certain distance. Weight could kind of be like memory complexity, and distance could be like time complexity, and the two end up giving you a rough idea of how expensive your algorithm is. Sometimes you might want to move something that doesn’t weigh much a far distance, or sometimes you might want to move something that ways a lot a short distance, or maybe you want something in between. That all comes down to what you’re trying to do, where, and why, and similarly in programming how you balance time complexity and memory complexity depend on the situation.
Often times, time complexity is more of a concern for many devs, because it has the biggest impact on player’s experiences. Memory usage won’t effect your fun as a player as much as lag might. Memory complexity is also important because it strictly limits what devices your code or game can run on.
This is the basic idea behind what you’ll often here when optimization gets brought up: “big O” and “little o” notation, which is just a way to say how “complex” an algorithm is. This is a useful metric for saying how fast something will be when you use it many times (E.g. how much time it costs, or how much memory it uses). You can read more about these notations in my post discussing them.
Your code is the sum of its parts
One really important detail that is often missed when it comes to complexity is that complexity only measures how things scale. An algorithm that has O(n^2) time complexity will get slower and slower the more n is (whatever n happens to mean) compared to an algorithm which is O(n), and that means that the O(n) algorithm is better the bigger n gets. You can have two algorithms, both O(n) time complexity, but one might still be significantly faster than the other.
The time it will take for your code to run, and the memory it’ll use really is just the time or memory it takes for each component added together as it executes.
If you want to optimize for speed, and you’re feeling stuck when it comes to reducing the complexity, you can often save big by making certain pieces faster. This often comes up as micro-optimization, where you are trying to save tiny amounts of time in various places to increase speed as much as you possibly can. This isn’t always worth it though!
Micro-optimization
Here’s a great analogy for micro-optimization, and also the above method of optimization, reducing the different values in that sum. Let’s say you’re looking to buy something, maybe a new mouse. Maybe you have two stores to pick from. At Store B you might be able to get the exact same mouse for 10 cents cheaper than at Store A. Store B might be a few more minutes away than Store A.
Well now maybe instead you’re a retailer buying that model of mouse. Ignoring the realism, as a retailer you might actually benefit a lot by buying from Store B if you want to buy a lot of mice. If you want to buy 100 mice, you save those 10 cents 100 times compared to Store A, which ends up being a whole 10 dollars.
Micro-optimization works the same way. Usually you’ve gotta go a little out of your way for a micro-optimization, and sometimes they will make your programming experience a little harder, just like how it might take a bit longer to get to Store B. But, the bits of time you save can add up if you are running code with that micro-optimization thousands, or millions of times (just like buying in bulk with those small savings can result in big savings in the end)
Deciding when certain micro-optimizations are worth it mainly comes down to weighing the effort it’ll take, and how it’ll effect your experience (and other people’s experience) working with your code, and how many times your code will benefit from that micro-optimization, such as in 3D voxel terrain, or in flow grids, cases where you’ll be doing many many many iterations at once.
You might save 10 cents on something that costs $2, or you might save 50 cents on something that costs $1.50. Likewise, maybe you save 100 microseconds on something that costs 2 milliseconds, or maybe you save 500 microseconds on something that costs 1.5 milliseconds. Compounding those savings a lot of times ends up potentially saving you a lot in the end.
Saving a few nanoseconds once is often not an effective time to apply micro-optimizations.
Summary
Optimization can be complicated, or simple, and, the ways you optimize can be very different depending on the situation. Weighing when you want to use certain optimizations can be hard, but when it comes to micro-optimizations you’re (usually) better off applying them when you will save those small amounts of time a lot for the effort.