Skateboard bouncing against wall issue

I am currently creating a skateboard system where skateboards have the ability to travel up angled slopes/ramps smoothly. To achieve this I perform a downwards raycast in front of my character, retrieve the normal of the raycast, and then use that normal to build a new CFrame for the player character by which the normal of the raycast will be the upvector in the new CFrame. This all seems pretty standard and it seems to work for the most part, however one problem I continue to get when using this method is that sometimes the play will collide against the ramp instead of smoothly going up it. I have tried so many different things to fix this problem at this point ( changing the math, changing how I do the raycasting, orienting the skateboard instead of the player, etc etc, ) however none of these have gotten me any closer to fixing this annoying bug. Any ideas on how I might be able to fix this will be much appreciated :-).

Here is a video of the problem: https://gyazo.com/c152430cdf917a08c19d5a975136c38c
Here is the code snippet:

local Result = workspace:Spherecast(
(self.rootPart.CFrame * CFrame.new(0, 0, self.speed*DeltaTime*60)).Position + Vector3.new(0,5,0),2,
					-Vector3.yAxis*12, 
					Parameters
				)			
				
				
local speedMultipler = 50-(math.abs(self.speed)/config.TopSpeed)*50
if (Result and typeof(Result.Instance) == "Instance") then			
	if Result.Normal:Dot(Vector3.yAxis) < .2 then
	    print(Result)
	end
	local CurrentRightVector = self.rootPart.CFrame.RightVector
	local UpVector = Result.Normal
				
					
	NewRightVector = Vector3.yAxis:Cross(Result.Normal)
	NewFacialVector = NewRightVector:Cross(Result.Normal)			
					
	--NewFacialVector = CurrentRightVector:Cross(UpVector).Unit
	--NewRightVector = UpVector:Cross(NewFacialVector).Unit

        local alpha = math.clamp(DeltaTime*60/7+speedMultipler*DeltaTime*60/30,0,1)
	self.XZGyro.MaxTorque = Vector3.new(math.huge, 0, math.huge)
	self.XZGyro.CFrame = self.XZGyro.CFrame:Lerp(
				CFrame.fromMatrix(self.rootPart.Position, NewRightVector, UpVector,NewFacialVector),alpha)                                    
	...