First of all you need a way to check if someone is on the ground. The Humanoid.FloorMaterial property is excellent for this. Next you want to think about what “flying” means.
The characters velocity will either have magnitude 0 or they will be moving. If they aren’t on the ground and their velocity’s Y is remaining constant or 0 for at least a few seconds (I’d say 1.5 is a good value however you might do 2 to be safe) they are most definitely flying. If their velocity is negative and becomes positive they are definitely flying.
Next you need to know when to check. You should use Heartbeat. This is fired after physics take place. Stepped is the alternative and its fired before physics and because of this it isn’t useful for this situation.
Finally you need to know how to stop flying. You shouldn’t punish the player for movement exploits unless your script is confident they are exploiting (it’s up to you to decide) but rather prevent them. To prevent flight you can teleport the player to the ground with a RayCast.
Similar ideas can be applied to speed/teleporting. You can check if the player moves faster than their velocity times the heartbeat step give or take a few SPS. Velocity.Magnitude is their total speed (for walking it’s almost exactly equal to their WalkSpeed), (position-lastPosition).Magnitude is how far they moved. You can finally check if their velocity is too fast. Their max positive Y speed is their JumpPower and their max x and z speed is their WalkSpeed. Their max negative Y speed is infinity so you shouldn’t use negative Y. When you detect this you can set this to their last position and last velocity optionally added to step times their velocity to get their predicted position.
You should read my article for more information about anti exploits and some tips: How you should secure your game - A beginner guide for secure networking and developing anticheats - #43 by Hexcede
Sorry for the bump post. The way that I do it is that, in addition to what people have already mentioned, you can get the height of your lowest point in the map and the highest point in your map and check if the player’s humanoidrootpart (if it exists) in constrained between those numbers. If not, the player is outside the map. This is an older method that I use but, I have found it to work and be simple. However, if the player doesn’t go beyond your min and max constraints, that player could be flying, which is where this method has its pitfall. Most of the time though, they tend to go beyond the constraints.
I made an Anti-Fly script which checks player to make enoguh mistakes that we can count that as exploiting, take a look at this thread:
how do you check if the event in the script doesn’t respond when the server fires to the player.