thats a smart idea i think but its a little risky, lets see if we can preserve it first
if we can’t preserve it, what i would do is internally store your own version of jobids which you can guarantee to be unique to your game (you will have to guarantee this of course), in this case you could store the job id as 5 characters because i assume your game will not be popular enough for (36)^5 servers
is this with json?
also im almost done sorry
Yes, that is with the following JSON:
{“lobbyId”:11223344,“hostJobId”:“9tv3h7estkcxe7r5tdvjk5gti8d5g”,“players”:[{“userId”:12342332,“jobid”:““4e89d”},{“userId”:12342332,“jobid”:”“4e89d”},{“userId”:12342332,“jobid”:““4e89d”},{“userId”:12342332,“jobid”:”“4e89d”},{“userId”:12342332,“jobid”:““4e89d”},{“userId”:12342332,“jobid”:”“4e89d”},{“userId”:12342332,“jobid”:““4e89d”},{“userId”:12342332,“jobid”:”“4e89d”},{“userId”:12342332,“jobid”:““4e89d”},{“userId”:12342332,“jobid”:”“4e89d”},{“userId”:12342332,“jobid”:““4e89d”},{“userId”:12342332,“jobid”:”“4e89d”},{“userId”:12342332,“jobid”:““4e89d”},{“userId”:12342332,“jobid”:”“4e89d”},{“userId”:12342332,“jobid”:““4e89d”},{“userId”:12342332,“jobid”:”“4e89d”},{“userId”:12342332,“jobid”:““4e89d”},{“userId”:12342332,“jobid”:”“4e89d”},{“userId”:12342332,“jobid”:““4e89d”},{“userId”:12342332,“jobid”:”“4e89d”},{“userId”:12342332,“jobid”:““4e89d”},{“userId”:12342332,“jobid”:”“4e89d”}],“gamemode”:{“name”:“Testing Gamemode”,“maxplayers”:50},“status”:{“current”:“waiting_for_players”,“timerStart”:-1,“timerLength”:-1}}
Also I want to note, I want to be able to add more data to gamemode later on.
i didnt test this but:
local function serialize(plrs)
local buff=bitbuffer()
buff:ushort(#plrs)
for _,t in next,plrs do
buff:ulong(t.userId)
local len=#t.jobId
buff:uint(len)
for i=1,len do
buff:alphanum(t.jobId:sub(i,i))
end
end
return buff:ds()
end
local function deserialize(s)
local buff=bitbuffer.ds(s)
local plrs={}
for i=1,buff:ushort()do
local uid=buff:ulong()
local len=buff:uint()
local job=''
for i=1,len do
job=job..buff:alphanum()
end
plrs[i]={
userId=uid,
username=game:GetService'Players':GetNameFromUserIdAsync(uid),
jobId=job,
}
end
end
with this bitbuffer module:
How about taking the entire JSON-encoded array and running it through the bitbuffer module? How well could that work out?
Edit: This is why Roblox needs to raise the message size limit to 5kb, it wouldn’t hurt to allow bigger messages.
dont run strings through bitbuffers, that is really bad
_G.reload()
local bitbuffer=_G.bitbuffer
local function serialize(plrs)
local buff=bitbuffer()
buff:ushort(#plrs)
for _,t in next,plrs do
buff:ulong(t.userId)
local len=#t.jobId
buff:uint(len)
for i=1,len do
buff:alphanum(t.jobId:sub(i,i))
end
end
return buff:dump()
end
local function deserialize(s)
local buff=bitbuffer(s)
local plrs={}
for i=1,buff:ushort()do
local uid=buff:ulong()
local len=buff:uint()
local job=''
for i=1,len do
job=job..buff:alphanum()
end
plrs[i]={
userId=uid,
username=game:GetService'Players':GetNameFromUserIdAsync(uid),
jobId=job,
}
end
end
local plrs={}
for i=1,22 do
plrs[i]={userId=113926330,username='CoreDev',jobId='test3483fh3487fgw983rte497t'}
end
print(#serialize(plrs))
print(#_G.http:JSONEncode(plrs))
22 players with bitbuffer is:
682
json is:
1761
but I’m using all 256 characters now not datastore friendly so I’m not sure if this works for you?
I don’t care about datastores that much (I have my own data storing solution) and the only thing that matters is getting this message compressed enough that I can get it under 1kb. I need to be able to store 50 players inside of this (at max).
what api are you using
some apis dont support all 256 characters (datastore for ex)
you should test it
do you also know if jobids are always 27 characters long?
I use HttpService which contacts my own service that I host.
And I don’t know if they are always 27 chars long. The only thing the compression is for is for MessagingService.
Edit: Unless someone makes a wrapper for MessagingService to send messages in parts and put them together automatically?
ok so
_G.reload()
local bitbuffer=_G.bitbuffer
local function serialize(plrs)
local buff=bitbuffer()
buff:ushort(#plrs)
local pos={}
for i=1,10 do
pos[i]=math.random()
end
local function ulong(job)
local x=0
for i=1,10 do
local c=math.floor(pos[i]*#job)+1
c=job:sub(c)
local n=tonumber(c)
n=n or string.byte(c)-string.byte'a'+10
x=x*36+n
end
return x
end
for _,t in next,plrs do
buff:ulong(t.userId)
buff:ulong(ulong(t.jobId))
end
return buff:dump()
end
local function deserialize(s)
local buff=bitbuffer(s)
local plrs={}
for i=1,buff:ushort()do
local uid=buff:ulong()
plrs[i]={
userId=uid,
username=game:GetService'Players':GetNameFromUserIdAsync(uid),
jobId=buff:ulong()..'',
}
end
return plrs
end
local plrs={}
for i=1,50 do
plrs[i]={userId=113926330,username='CoreDev',jobId='test3483fh3487fgw983rte497t'}
end
print(#serialize(plrs))
print(#_G.http:JSONEncode(plrs))
its 665 characters
it does something similar to taking the 5 characters you mentioned, but instead it takes 10 (because that is just enough to fit in an integer double w/o losing precision) random characters but that are the same for all of the job ids in this serialize call that have the same length (if they have different lengths the characters taken will be different, nothing to do about that unless we take FIRST five characters but I would rather not trust that Roblox’s job ids are completely random)
using these new hashed job ids, you are still guaranteed that players who had the same job ids before still have the same job ids
the only downside is that with astronomically low probability players in separate servers may be said to have the same job id
@CoreDev I’m not sure if I modified the bitbuffer so I’ll link the newest version: --[[ BitBuffer read/write mode is determined by whether a string is passe - Pastebin.com
Does your code optimize the user ids in any way by any chance? (Also you can remove the username code since I removed it from mine)
userids are optimally and losslessly stored
here it is w/o username (removed one line, didnt affect the size)
_G.reload()
local bitbuffer=_G.bitbuffer
local function serialize(plrs)
local buff=bitbuffer()
buff:ushort(#plrs)
local pos={}
for i=1,10 do
pos[i]=math.random()
end
local function ulong(job)
local x=0
for i=1,10 do
local c=math.floor(pos[i]*#job)+1
c=job:sub(c)
local n=tonumber(c)
n=n or string.byte(c)-string.byte'a'+10
x=x*36+n
end
return x
end
for _,t in next,plrs do
buff:ulong(t.userId)
buff:ulong(ulong(t.jobId))
end
return buff:dump()
end
local function deserialize(s)
local buff=bitbuffer(s)
local plrs={}
for i=1,buff:ushort()do
local uid=buff:ulong()
plrs[i]={
userId=uid,
jobId=buff:ulong()..'',
}
end
return plrs
end
local plrs={}
for i=1,50 do
plrs[i]={userId=113926330,username='CoreDev',jobId='test3483fh3487fgw983rte497t'}
end
print(#serialize(plrs))
print(#_G.http:JSONEncode(plrs))
Trying out your code now, my studio is crashing so its taking a bit.
Edit @Acreol – I’m having a problem. How am I supposed to require the BitBuffer?
make it a modulescript and require it?
Not working for me. I did with the code
local bitbuffer = require(script.Parent.BitBuffer)
And yes, I removed the _G.reload()
oh sorry:
_G.reload()
local bitbuffer=_G.bitbuffer
local function serialize(plrs)
local buff=bitbuffer.new()
buff:ushort(#plrs)
local pos={}
for i=1,10 do
pos[i]=math.random()
end
local function ulong(job)
local x=0
for i=1,10 do
local c=math.floor(pos[i]*#job)+1
c=job:sub(c)
local n=tonumber(c)
n=n or string.byte(c)-string.byte'a'+10
x=x*36+n
end
return x
end
for _,t in next,plrs do
buff:ulong(t.userId)
buff:ulong(ulong(t.jobId))
end
return buff:dump()
end
local function deserialize(s)
local buff=bitbuffer.new(s)
local plrs={}
for i=1,buff:ushort()do
local uid=buff:ulong()
plrs[i]={
userId=uid,
jobId=buff:ulong()..'',
}
end
return plrs
end
local plrs={}
for i=1,50 do
plrs[i]={userId=113926330,jobId='test3483fh3487fgw983rte497t'}
end
print(#serialize(plrs))
print(#_G.http:JSONEncode(plrs))
in my framework I have the tables have the module tables have __call linked to their .new constructor
You might want to fix the _G.http too, but let me try it
Edit: It works! (I’m assuming the numbers are the size in bytes?)
Great! Now let me try implementing this into my game (I still don’t have access to messaging service so I can’t test it in game, but I can add it.)
Also I need to fix the spacing on your code since it bugs me 
But thanks so much for helping me out with fixing this. I really appreciate it.
In case the code above errors because messagingservice doesn’t support all 256 characters, use this version (its 824 characters):
_G.reload()
local bitbuffer=_G.bitbuffer
local function serialize(plrs)
local buff=bitbuffer.new()
buff:ushort(#plrs)
local pos={}
for i=1,10 do
pos[i]=math.random()
end
local function ulong(job)
local x=0
for i=1,10 do
local c=math.floor(pos[i]*#job)+1
c=job:sub(c)
local n=tonumber(c)
n=n or string.byte(c)-string.byte'a'+10
x=x*36+n
end
return x
end
for _,t in next,plrs do
buff:ulong(t.userId)
buff:ulong(ulong(t.jobId))
end
return buff:ds()
end
local function deserialize(s)
local buff=bitbuffer.ds(s)
local plrs={}
for i=1,buff:ushort()do
local uid=buff:ulong()
plrs[i]={
userId=uid,
jobId=buff:ulong()..'',
}
end
return plrs
end
local plrs={}
for i=1,50 do
plrs[i]={userId=113926330,jobId='test3483fh3487fgw983rte497t'}
end
print(#serialize(plrs))
print(#_G.http:JSONEncode(plrs))
also use this version of bitbuffer because I fixed some things (well they didn’t really affect you but in case you decide to use bitbuffer for more things) --[[ BitBuffer read/write mode is determined by whether a string is passe - Pastebin.com
Swapped out BitBuffer, now I need to re-space the new code you just send for the functions