You could set each character model’s StreamingMode to PersistentPerPlayer, and then use AddPersistentPlayer and RemovePersistentPlayer on the server to stream the character only when the player is spectating.
Example code:
local characterImSpectating = workspace.CoolName
local myPlayer = Players.CoolName2
-- Stream in player (spectate has started)
characterImSpectating:AddPersistentPlayer(myPlayer)
-- Stream out player (spectate has ended)
characterImSpectating:RemovePersistentPlayer(myPlayer)
Sorry for replying so late, haven’t seen this post until now.