Most efficient way to tween (or otherwise move) hundreds (or thousands) of models on the client?

Hello everyone! Currently, im working on a strategy game, and i want its scope to be quite big, so i want the npc movement system to be as optimized as possible
after a few months, i think i optimized it as much as i could (the server calculates the positions to which every moving npc should jump to every 0.1 seconds, which is later sent to the client via remote events, which creates and maintains models for these npcs until they are dead or otherwise deloaded)
the problem is that when more than a couple hundred npcs are moving, especially evident if there is more than a thousand, the client starts to lag quite a lot

the current setup i use is that every npc has a client side model assigned to it
-the model consists of an r6 rig, humanoid, humanoid colors, and a few accessories
-the model’s scale is 0.2 (not sure if that’s relevant but still mentioning it)

the model is later oriented according to the rotation that the npc is assigned, and a vector3 coord is drawn infront of the npc, that coord is used to represent where the client thinks the npc will be in 0.1 seconds

the npc modem is then tweened from
Its current position to this extrapolated point in 0.1 seconds (if the client is wrong, then the npc position will still get updated to the correct one on the next remote event in 0.1 seconds so its not that big of a deal)

what is concerning to me is that i also want the npcs to play basic animations depending on their state, and the fact that just moving them lags the game is a little disheartening

Previously, i’ve used PivotTo instead of tweens because the server sided code ran 50 times a second, but i was forced to optimize for the sake of network optimisation

one solution i thought of was derendering npcs that are off screen of the client, but
a. The game’s map is already split into areas where anything not in that area disappears
b. I am afraid of it being a false economy due to having to check every nearby npc’s position every heartbeat

Any thoughts appreciated, can provide code if needed once i’m on pc!

Edit: i also have a pretty low end pc from 2016 (3.4 GHz i5 and radeon pro 570 graphics card so i also hope that whatever solution works for me also works for mid-high tier mobile devices)

2 Likes

clarification:

-i am sure that it is specifically the moving part of the script that causes lag and not just hundreds of models existing in the game because:
a. the lag only happens when several hundred npcs are moving at once, and if the majority of them are standing still, the game runs just fine
b. i believe i optimized the npc models a lot as they all have cantouch, cancollide, and canquery turned off, fluid forces disabled, cast shadow set to false, and massless set to true (i didn’t anchor the parts because i believe that parts need to be unanchored in order to . If there are any other optimizations i can make then let me know please

looking over the code the main problem seems to be the fact that the tween loops every task.wait(), but setting it to anything slower than that makes the npc movement visually jittery as the tween often has to wait until the next loop iteration to check if an update from the server has been received, which results in the npc visually stopping and continuing movement. having said that, setting it to task.wait(0.05) does improve the lag noticably, although cpu usage is still within the yellow to orange with 400 npcs moving at the same time (which is the baseline i use for testing)

here is the full code, you can ignore everything before workspace.terrainfinishedgenerating as that is just setup functions

local workersrepfolder = game.ReplicatedStorage.NPCs
local FolderTable = {}

local OldPositions = {}
local Models = {}
local NpcHealthValues = {}

local GetnpcPos = require(game.ReplicatedStorage.GetNPCPositionClient)
local GetnpcRotation = require(game.ReplicatedStorage.GetNPCRotationClient)

local plr = game.Players.LocalPlayer
local tweenSS = game:GetService('TweenService')

workersrepfolder.ChildAdded:Connect(function(folder)
	table.insert(FolderTable, folder)
	folder.ChildRemoved:Connect(function(NPC)
		if NPC:FindFirstChild('Ignore') then
			if NPC.Ignore.Value == true then
				if Models[NPC.Name] then
					Models[NPC.Name]:Destroy()
					Models[NPC.Name] = nil
				end
				wait(.5)
				if not folder:FindFirstChild(NPC.Name) then
					OldPositions[NPC.Name] = nil
					GetnpcPos.WritePosition(NPC.Name, nil)
				end
			end
		else
			if Models[NPC.Name] then
				Models[NPC.Name]:Destroy()
				Models[NPC.Name] = nil
			end
		end
	end)
end)

local function FindGridFolder(GridValue)
	for i, folder in ipairs(workspace.NPCs:GetChildren()) do
		if folder.ParentGrid.Value == GridValue then
			return folder
		end
	end
end

local function findNPCmodel(NPC)
	local folder = FindGridFolder(NPC.Parent.ParentGrid.Value)
	
end

local connectedNPCs = {}

local function connectNPC(NPC)
	local Attributes = NPC:WaitForChild("ProgressBarAttributes")
	local connection
	task.spawn(function()
		connection = Attributes.AttributeChanged:Connect(function(attribute)
			if Models[NPC.Name] then
				if attribute == "active" and Attributes:GetAttribute("active") == true then
					local ui = game.ReplicatedStorage.TrainingProgressUI:Clone()
					ui.Adornee = Models[NPC.Name]
					ui.Container.Text.Text = Attributes:GetAttribute("text")
					ui.ParentGrid.Value = NPC.Parent.ParentGrid.Value
					ui.Parent = workspace.ShortLivespanUIs
					local time = Attributes:GetAttribute("time")
					local tweeninfo = TweenInfo.new(time, Enum.EasingStyle.Linear, Enum.EasingDirection.In)
					local tween = tweenSS:Create(ui.Container.Bar, tweeninfo, {Size = UDim2.new(1, 0, 1, 0)})
					tween:Play()
					local connection
					connection = tween.Completed:Connect(function()
						ui:Destroy()
						connection:Disconnect()
					end)
				end
			else
				warn('returned the connectNPC function because the NPC did not have a model')
				return
			end
		end)
	end)
	NPC.Health.Changed:Connect(function()
		if NPC.Health.Value == 0 then
			connection:Disconnect()
			connectedNPCs[NPC.Name] = nil
			return
		end
	end)
end

game.ReplicatedStorage.NPCs.ChildAdded:Connect(function(Folder)
	for i, NPC in ipairs(Folder:GetChildren()) do
		if not NPC:IsA('ObjectValue') and not connectedNPCs[NPC] then
			connectedNPCs[NPC.Name] = true
			connectNPC(NPC)
		end
	end
	Folder.ChildAdded:Connect(function(NPC)
		if not NPC:IsA('ObjectValue') and not connectedNPCs[NPC] then
			connectedNPCs[NPC.Name] = true
			connectNPC(NPC)
		end
	end)
end)

repeat wait() until workspace.TerrainFinishedGenerating.Value == true

local universalNpcValues = {}

for i, v in ipairs(game.ReplicatedStorage.UniversalNPCValues:GetChildren()) do
	universalNpcValues[v.Name] = v
end

local OldPositions = {} --[npcName] = Vector3
local OldRotations = {} --[npcName] = number (degrees)
local ActiveTweens = {} --[npcName] = Tween

local MOVE_TWEEN_INFO = TweenInfo.new(0.1, Enum.EasingStyle.Linear, Enum.EasingDirection.InOut)

@native function positionsClose(a, b, tolerance)
	local dx = a.X - b.X
	local dy = a.Y - b.Y
	local dz = a.Z - b.Z
	return (dx*dx + dy*dy + dz*dz) <= (tolerance)^2 --avoids math.sqrt apparently
end

--angle closeness without heavy modulo use
@native function anglesClose(a, b, tolerance)
	local diff = a - b
	if diff < -180 then
		diff = diff + 360
	elseif diff > 180 then
		diff = diff - 360
	end
	return math.abs(diff) <= (tolerance)
end

local universalNpcValues = {}
for i, v in ipairs(game.ReplicatedStorage.UniversalNPCValues:GetChildren()) do
	universalNpcValues[v.Name] = v
end

spawn(function()
	while task.wait() do
		for npcName, model in pairs(Models) do
			local newPos = GetnpcPos.GetPosition(npcName)
			local newRotDeg = GetnpcRotation.GetRotation(npcName) or 0
			local robloxYDeg = newRotDeg

			local oldPos = OldPositions[npcName]
			local oldRot = OldRotations[npcName]
			local activeTween = ActiveTweens[npcName]

			local speedValue = universalNpcValues[model.Name].Speed.Value
			if not speedValue or speedValue <= 0 then speedValue = 10 end -- fallback speed

			if not newPos then
				if activeTween then
					activeTween:Cancel()
					ActiveTweens[npcName] = nil
				end
				continue
			end

			local moved = not oldPos or not positionsClose(oldPos, newPos, 0.05)
			local rotated = not oldRot or not anglesClose(oldRot, robloxYDeg, 1)

			if moved or rotated then
				local targetCFrame = CFrame.new(newPos) * CFrame.Angles(0, math.rad(robloxYDeg), 0)

				if oldPos then
					if activeTween then
						activeTween:Cancel()
					end

					local distance = (newPos - oldPos).Magnitude
					local duration = distance / speedValue
					if duration < 0.05 then duration = 0.05 end -- clamp minimum tween time for smoothness

					local tweenInfo = TweenInfo.new(duration, Enum.EasingStyle.Linear, Enum.EasingDirection.InOut)
					local tw = tweenSS:Create(model.PrimaryPart, tweenInfo, { CFrame = targetCFrame })
					tw:Play()
					ActiveTweens[npcName] = tw
				else
					model:PivotTo(targetCFrame)
				end

				OldPositions[npcName] = newPos
				OldRotations[npcName] = robloxYDeg
			end
		end
	end
end)

while task.wait(.1) do
	--local time = os.clock()
	local SelectedGrid = plr.SelectedGrid.Value
	if SelectedGrid == nil then continue end
	for _, Folder in ipairs(FolderTable) do
		--if Folder.ParentGrid.Value ~= SelectedGrid then continue end --IMPORTANT: commented out for the sake of testing heavy load
		for i, NPC in ipairs(Folder:GetChildren()) do
			if NPC:IsA('ObjectValue') then continue end
			local npcName = NPC.Name
			local npcParentGridValue = NPC.Parent.ParentGrid.Value
			if npcParentGridValue ~= SelectedGrid then
				local model = Models[npcName]
				if model then
					model:Destroy()
					Models[npcName] = nil
				end
				continue
			end
			local currentModel = Models[npcName]
			local npcModelValue = NPC.Model.Value
			local healthVal = NPC.Health.Value
			local npcTypeObj = NPC:FindFirstChild("NpcType")
			if not npcTypeObj then
				--this happened ONCE during testing but now im paranoid
				continue
			end
			local NPCStats = game.ReplicatedStorage.UniversalNPCValues[npcTypeObj.Value] 
			local MaxHealthVal = NPCStats.MaxHealth.Value
			if not currentModel and npcModelValue then
				currentModel = npcModelValue:Clone()
				currentModel.ID.Value = npcName
				Models[npcName] = currentModel
				currentModel.Parent = FindGridFolder(npcParentGridValue)
				if OldPositions[npcName] then
					currentModel:PivotTo(CFrame.new(OldPositions[npcName]))
				end
			elseif npcModelValue == nil then
				continue
			elseif currentModel and npcParentGridValue == SelectedGrid then
				--this seems a little inefficient
				local folder = FindGridFolder(npcParentGridValue)
				if currentModel.Parent ~= folder then
					currentModel.Parent = folder
				end
			end
			
			if not NpcHealthValues[NPC] then
				NpcHealthValues[NPC] = healthVal
			elseif NpcHealthValues[NPC] ~= healthVal then
				NpcHealthValues[NPC] = healthVal
				local billboard =  currentModel.HealthBar
				if not billboard.Enabled then
					billboard.Enabled = true
				end
				billboard.Container.Number.Text = healthVal.."/"..MaxHealthVal
				billboard.Container.Bar.Size = UDim2.new(healthVal / MaxHealthVal, 0, 1, 0)
			end
		end
	end
	--print(os.clock() - time)
end```

Create custom tween with spatial hashing that handles updates depending on distance reduce amount of updates are done

Create custom tween with spatial hashing that handles updates depending on distance

can you please elaborate on what that would do? don’t tween the models as accurately the further away they are? and as i said i am scared of any distance based solutions being a false economy due to having to check distances of hundreds of npcs every heartbeat (or however often the code runs) ontop of all of the code that already exists

reduce amount of updates are done

while i could send out updates to the clients only once ever 0.3 seconds for example, through testing i found that any frequency below 10 times a second starts feeling quite unresponsive and annoying to manage
additionally, i don’t think that the loop that handles keeping the npcs up to date (the second one) is the source of the lag, because regardless of whether or not the npcs are moving, it completes the calculations in around 6 milliseconds for 400 npcs, which i am pretty happy with

Hi there!

There are a few big factors in play here:

  • Humanoids are incredibly unperformant. Get rid of them if possible.
  • Physics are compute-heavy. Anchor as many of the parts as you can.
  • Use Motor6Ds to move the parts. Motor6Ds are incredibly optimized for moving parts.

If you have any further questions regarding this, let me know!

1 Like

thanks for the reply! i have a couple of questions based on what you said;

  1. Are humanoids essential to playing animations? i am honestly not very experienced in this type of things so i honestly have no idea what can and can’t be animated via loading animation objs into them
  2. Can motor6ds still affect anchored parts? i forgot to mention that in the main post, but the reason why i don’t anchor the nps is because i think that anchored parts cannot be animated
  3. If conventional animation isn’t possible, would a system for motor6d translation work? I’ve heard of similar systems being used for animating characters inside of viewportframes (which i didn’t have time to implement yet for my UIs), but i am also not sure how scalable those systems are

Nope! While humanoids do enable animations, there is a better alternative: AnimationController.

Yep!

Well it should be!

1 Like

nice, thanks for the info!
one more thing though, how would you move the whole model via motor6ds as opposed to one of the joints?
(again ive never used motor6ds before so im sorry if this question is a little silly)

I believe it should move everything attached to it by default. Let me know how it turns out!

And if my information was helpful, I’d appreciate it if you marked one of my replies as “Solution.” Thanks!

1 Like

alright thanks, i’ll test this out and update on how it goes

2 Likes

update: achieved a way to smoothly move npcs with very little lag (for 400 npcs, the cpu usage didn’t get into yellow throughout the various movements i made the npcs do)
after trying out a few different configurations with motor6ds, like @AlexanderLindholt suggested, i found a few problems;
-motor6ds dont work with anchored parts after all, which would mean that I would have to keep the HumanoidRootPart unanchored, which made individual models lag more due to the physics calculations
-motor6ds need a distinct part0 and part1 property assigned to them in order to apply the offset properly, which meant that a new part would have to be created in order to base the npcs off of, which is honestly not too bad of a tradeoff, but i still felt like it muddled my system unnecessarily
-moto6ds “transform” properties don’t take world coordinates, which means that every time a position of an npc is updated, CFrame:ToObjectSpace() would have to be called, which also increased the cost of computing the npc

having said that, there is always a possibility that i didn’t study motor6ds well enough and that there are workarounds to these problems, but personally i didn’t want to spend additional time researching this. And despite these issues, i did decide to use motor6ds for animating the characters instead, coupled with @AlexanderLindholt’s suggestion to use animation controllers (which auto optimize), loading animations into each npc also had little impact on performance.

in the end, i decided to use a very similar system to what i had, but with lerp instead of tweens (since i only need a simple constant-speed animation to move the npcs forwards) and a few other modifications. There’s only two things i am not sure of:

  1. I have heard that PivotTo is not a very performant way to move npcs, but i don’t know any alternatives to it, as SetPrimaryPartCFrame is deprecated.
  2. I am not sure how well i handled the management of the ServerTarget table, so there is possibility of a memory leak occuring
    Regardless, i am very happy with the performance of the script, thanks for the help everyone!
    (here is the organized and documented script i currently use)
local ServerTarget = {} --[npcName] = { pos = Vector3, yaw = number }
local teleportVal = 2 --in studs
local teleportSQ = teleportVal * teleportVal
local snapDistance = 0.05 --in studs
local snapSQ = snapDistance * snapDistance

spawn(function()
	local last = os.clock()
	while task.wait() do
		local now = os.clock()
		local deltaTime = math.max(0.001, now - last)
		last = now

		for npcName, model in pairs(Models) do
			local serverPos = GetnpcPos.GetPosition(npcName)
			local serverYaw = GetnpcRotation.GetRotation(npcName) or 0
			if not serverPos or not model then
				ServerTarget[npcName] = nil
				continue
			end

			--basically checks if the npc was just created
			if not OldPositions[npcName] then
				local baseOrientation = baseRotations[model.Name] or blankCF
				local targetCF = CFrame.new(serverPos) * CFrame.Angles(0, math.rad(serverYaw), 0) * baseOrientation
				model:PivotTo(targetCF)
				OldPositions[npcName] = serverPos
				OldRotations[npcName] = serverYaw
				continue
			end

			--distance calculation (without using (pos - serverpos).Magnitude because from what i know it creates a 
			--temporary vector 3 internally that later gets garbage collected and i think thats slightly worse for performance)
			local currentPos = model.PrimaryPart.Position
			local dx = serverPos.X - currentPos.X
			local dy = serverPos.Y - currentPos.Y
			local dz = serverPos.Z - currentPos.Z
			local distSq = dx*dx + dy*dy + dz*dz

			--teleports npc if the distance is too far away from the current position
			if distSq >= teleportSQ then
				local baseOrientation = baseRotations[model.Name] or blankCF
				local targetCF = CFrame.new(serverPos) * CFrame.Angles(0, math.rad(serverYaw), 0) * baseOrientation
				model:PivotTo(targetCF)
				OldPositions[npcName] = serverPos
				OldRotations[npcName] = serverYaw
				ServerTarget[npcName] = nil
				continue
			end

			--same code as on the server, snaps the npc if its close to the destination
			if distSq <= snapSQ then
				local baseOrientation = baseRotations[model.Name] or blankCF
				local targetCF = CFrame.new(serverPos) * CFrame.Angles(0, math.rad(serverYaw), 0) * baseOrientation
				model:PivotTo(targetCF)
				OldPositions[npcName] = serverPos
				OldRotations[npcName] = serverYaw
				ServerTarget[npcName] = nil
				continue
			end

			--if none of the other checks trigger, the npcs then get moved via lerp
			local dist = math.sqrt(distSq)
			local speedValue = universalNpcValues[model.Name].Speed.Value or 10
			local maxMove = speedValue * deltaTime
			local alpha = math.min(1, maxMove / dist)

			local baseOrientation = baseRotations[model.Name] or blankCF
			local targetCF = CFrame.new(serverPos) * CFrame.Angles(0, math.rad(serverYaw), 0) * baseOrientation
			local nextCF = model.PrimaryPart.CFrame:Lerp(targetCF, alpha)
			model:PivotTo(nextCF)

			OldPositions[npcName] = model.PrimaryPart.Position
		end
	end
end)```
2 Likes

You can just use a part at 0, 0, 0. That’ll fix your problem.

The best solution is to use workspace:BulkMoveTo() to move the primary parts of every NPC.

Fortunately there’s a neat trick you can do to make the Transform property work in world coordinates. Simply set Part0 to workspace.Terrain and Part1 to the part you want to move. No need to create a new part either.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.