I've never heard of "cell" used to describe something like this. The closest I can think of is Bethesda games which separate the game into cells, but multiple of those can be loaded at once (e.g. exterior cells), so I'm not sure if it's a great usage of terminology. Unity uses the term "scene" which is more appropriate IMO.
As for replication, I would assume the correct behavior would be only the objects in the current scene would replicate to the client. Wouldn't make sense for me to be receiving updates for something I couldn't see. Scenes would be server-side (not client-side).