Help with Objects Physics(Jittering)

Intro
Hello, I have been really fascinated with the new craze of mesh deformation oceans and I have been working on a system that simulates a simple physics setup where parts will float on the surface of the water. The current setup I have is to have 4 attachments on each corner of the model/part(PhysicsBox) and then if any of the attachments are under the water then change the velocity(Using a Linear Velocity) to move the part to the surface of the water.

Problem
The system almost works as intended with a few bugs, but the main one I notice is jittering. The part is completely smooth when coming up the wave, but as soon as it starts to go down it begins jittering.
Here is an Example
In the video, the ocean visualization is run on a client script while the physics of the part is completely handled on the server.

More Info
Here is the code currently running on the server script:

local floatables = game.Workspace.Floatables

for _, floatable in pairs(floatables:GetChildren()) do
	local physicsBox = floatable.PhysicsBox
	
	for _, at in pairs(physicsBox:GetChildren()) do
		if at:IsA("Attachment") then
			local linearVelocity = Instance.new("LinearVelocity", at)
			linearVelocity.Attachment0 = at
			linearVelocity.VectorVelocity = Vector3.zero
			--linearVelocity.Name = "LinearVelocity"
			linearVelocity.RelativeTo = Enum.ActuatorRelativeTo.World
			--linearVelocity.ForceLimitsEnabled = false
			linearVelocity.VelocityConstraintMode = Enum.VelocityConstraintMode.Plane
		end
	end
end

local oceanFunction = require(game.ReplicatedStorage.Ocean)

local depthBeforeSubmerged = 1
local displacementAmount = 3

game:GetService("RunService").Heartbeat:Connect(function()
	for _, floatable in pairs(floatables:GetChildren()) do
		for _, at in pairs(floatable.PhysicsBox:GetChildren()) do
			if at:IsA("Attachment") then
				local displacement = oceanFunction.calculateDisplacement(at.WorldPosition)
				local realGravity = workspace.Gravity*.28
				local realMass = floatable.PhysicsBox:GetMass()*21.952

				if at.WorldPosition.Y < displacement.Y then
					at.LinearVelocity.ForceLimitsEnabled = true
					local velocity = math.clamp((displacement.Y - at.WorldPosition.Y) / depthBeforeSubmerged, 0, 1) * displacementAmount
					at.LinearVelocity.PlaneVelocity = Vector2.new(0, velocity*realGravity/4)
					at.LinearVelocity.MaxForce = 1000000
				else
					at.LinearVelocity.PlaneVelocity = Vector2.new(0,-realGravity/4)
					at.LinearVelocity.ForceLimitsEnabled = true
					--at.LinearVelocity.MaxForce = 0
				end
			end
		end
	end
end)

Hopefully, I can find an answer to this issue or even figure out why this is happening. If it is something simple, then I apologize, but I am not very keen on physics. If you have any questions I will try to answer them as soon as possible. Thanks!