You can try using a base part within the horse as reference for your velocity. Something similar to a HumanoidRootPart would work pretty well for this, although any sort of fixed invisible origin part inside the horse which isn’t naturally rotated and faces to the front of the horse works as well.
Example
BodyVelocity.Velocity = Horse.HumanoidRootPart.CFrame.lookVector * Speed
You can do this backwards as well by making the speed a negative integer.
BodyVelocity.Velocity = Horse.HumanoidRootPart.CFrame.lookVector * -Speed