That being said, I am happy to see that work on Warp has resumed and that a rewrite is actively underway. I hope the project goes well, however, I have moved on due to the long period of inactivity and the numerous issues I encountered during that time. Wish you good luck
Sorry for the misinformation, will correct my post immediately.
Hopefully Warp improves
# 1.1.0 | pre-release6
In this version, the bandwidth gets significantly reduced.
# Bandwidth
| Entities | StructOfNumbers | Booleans | |
|---|---|---|---|
| Warp (previous) | 55.48KB/s | 18.50KB/s | 3.69KB/s |
| Warp (w/ schema) | 5.50KB/s | 3.07KB/s | 3.21KB/s |
| Warp (w/o schema) | 5.47KB/s | 3.33KB/s | 3.29KB/s |
| Packet | 46.60KB/s | 24.65KB/s | 5.83KB/s |
| Zap (v0.6.28) | 51.12KB/s | 30.61KB/s | 10.71KB/s |
| Blink (v0.18.6) | 54.60KB/s | 37.75KB/s | 17.94KB/s |
# FPS
| Entities | StructOfNumbers | Booleans | |
|---|---|---|---|
| Warp (previous) | 18.4 | 44.4 | 240 |
| Warp (w/ schema) | 18.6 | 44.5 | 244 |
| Warp (w/o schema) | 11.1 | 23.3 | 245 |
| Packet | 15.8 | 42.4 | 192 |
| Zap (v0.6.28) | 34.1 | 55.2 | 239 |
| Blink (v0.18.6) | 36.4 | 70.8 | 242 |
2 Likes
rare footage of package benchmarks not being “adjusted”
1 Like
They are probably adjusted NGL + apparently there is a hybrid schema/runtime structure? Yeah that’s pretty weird I think but let’s see maybe Warp is finally stable
1 Like
keep working on it i love it so much
here is the newer benchmark test
cc to @Khrrs for the template
1 Like
So your telling me Warp has gotten better then ByteNetMax? Well as long as this isn’t fake I’m in support!
if you like to repro the benchmark on ur end, here is the file:
File’s not opening is this a backdoor or smth
what are you talking about, its totally works fine for me, prob just ur studio
no you have to drag the flie to studio right like idk
In what way is this criticism actually helpful? Is your end goal to degrade OP until he doesn’t have the motivation to work on his module anymore? Jolt and ByteNetMax are stable releases, and this was a pre-release, not meant for use in production yet. How is that fair?
Warp seems to have a memory issue. I watched my memory hit 6GB, and the only times it meaningfully increased were when Warp was running
One thing to say, please read the post from top to bottom and you will see how unreliable and confusing this network module was. I am actually happy to see real improvement to the library as are many other people. But if you had read the post, you’d see the OP was very defensive about their work and made tons of false claims which lead to lots of developers being disappointed and often times having to re-implement netcode.
Also,
Bro… Bro… The entire resource ever since jan 2024 was always pre-release quality but advertised like it’s not. That is the entire point. Come on now!
I fixed multiple issues with the benchmark logic, with percentile and other issues, and retested.
What I noticed
- Warp was sending buffers full of zeros. Due to your use of XOR, you were not sending anything since the data is the same, artificially reducing your bandwidth usage
- Warp has a huge memory issue. My memory starts at 2GB in Roblox Studio test and reaches a max of 6GB by the end of the benchmark, entirely from warp; the other libraries temporarily increase memory for a second or don’t at all, unlike warp
- Warp sends a huge packet at the start frame, crashing my packet profiler plugin as shown below.
From just watching memory, the main culprits are Roblox, warp dynamic and warp in that order.
23:20:38.702 ================================================================================================================================ - Client - Client:250
23:20:38.702 BENCHMARK RESULTS (Client → Server, 100 fires/frame, 10s) - Client - Client:251
23:20:38.702 ================================================================================================================================ - Client - Client:252
23:20:38.702
Entities - Client - Client:255
23:20:38.702 ┌──────────────┬───────┬────────────┬──────────┬────────────┬────────────┬────────────┐ - Client - Client:256
23:20:38.702 │ Tool │ P50FPS│ P50 Kbps │ Loss │ After Stop │ Tail (srv) │ Span (srv) │ - Client - Client:257
23:20:38.702 ├──────────────┼───────┼────────────┼──────────┼────────────┼────────────┼────────────┤ - Client - Client:258
23:20:38.702 │ roblox │ 55 │ 355917.25 │ 0.0% │ 0.00% │ 0.000 │ 17.483 │ - Client - Client:269
23:20:38.702 │ blink │ 167 │ 10060.28 │ 0.0% │ 0.06% │ 0.003 │ 10.000 │ - Client - Client:269
23:20:38.702 │ bytenetmax │ 160 │ 9748.79 │ 0.0% │ 0.06% │ 0.003 │ 9.997 │ - Client - Client:269
23:20:38.702 │ zap │ 144 │ 8759.16 │ 0.0% │ 0.07% │ 0.003 │ 9.995 │ - Client - Client:269
23:20:38.702 │ NetRay │ 189 │ 11287.18 │ 0.0% │ 0.05% │ 0.003 │ 10.005 │ - Client - Client:269
23:20:38.702 │ warp_dynamic │ 63 │ 15485.29 │ 0.0% │ 0.00% │ 0.000 │ 9.954 │ - Client - Client:269
23:20:38.702 │ warp │ 125 │ 4562.28 │ 0.0% │ 0.00% │ 0.000 │ 9.998 │ - Client - Client:269
23:20:38.702 └──────────────┴───────┴────────────┴──────────┴────────────┴────────────┴────────────┘ - Client - Client:281
23:20:38.702
Booleans - Client - Client:255
23:20:38.702 ┌──────────────┬───────┬────────────┬──────────┬────────────┬────────────┬────────────┐ - Client - Client:256
23:20:38.702 │ Tool │ P50FPS│ P50 Kbps │ Loss │ After Stop │ Tail (srv) │ Span (srv) │ - Client - Client:257
23:20:38.702 ├──────────────┼───────┼────────────┼──────────┼────────────┼────────────┼────────────┤ - Client - Client:258
23:20:38.702 │ roblox │ 45 │ 136437.78 │ 0.0% │ 0.00% │ 0.000 │ 17.511 │ - Client - Client:269
23:20:38.702 │ blink │ 227 │ 54.14 │ 0.0% │ 0.04% │ 0.001 │ 10.000 │ - Client - Client:269
23:20:38.702 │ bytenetmax │ 170 │ 40.26 │ 0.0% │ 0.06% │ 0.003 │ 10.001 │ - Client - Client:269
23:20:38.702 │ zap │ 215 │ 50.49 │ 0.0% │ 0.05% │ 0.002 │ 10.003 │ - Client - Client:269
23:20:38.703 │ NetRay │ 228 │ 43.35 │ 0.0% │ 0.05% │ 0.001 │ 10.000 │ - Client - Client:269
23:20:38.703 │ warp_dynamic │ 204 │ 54.18 │ 0.0% │ 0.05% │ 0.010 │ 9.987 │ - Client - Client:269
23:20:38.703 │ warp │ 227 │ 60.80 │ 0.0% │ 0.09% │ 0.012 │ 9.985 │ - Client - Client:269
23:20:38.703 └──────────────┴───────┴────────────┴──────────┴────────────┴────────────┴────────────┘ - Client - Client:281
23:20:38.703 - Client - Client:284
23:20:38.716 Generated results - Server - Server:183
I added NetRay Compile as a comparison
Based on the benchmark, I’d rank it as
- NetRay
- Zap
- Blink
- ByteNet Max
- Warp / Warp Dynamic
- Roblox
I am sceptical about Zap’s and Bytenetmax P50 Kbps, as when I inspected the raw byte size, Zap and Bytenetmax were bigger for both benches but had a lower P50, most likely due to batching logic, as they have a lag after stop (although % small). Due to that, you could place Blink above both of them.
Warp is good at sending identical data repeatedly due to its XOR; otherwise, it’s not bad but has some issues like memory usage/leak, FPS tank. It’s still better than default Roblox in both benches.
yeah, it was because the xor currently stores the prev buffer & players buffer len. also currently the xor buffer size never get shrinks, and keep growing when the data size get bigger than previous.
its still early development for this xor feature.
Also are you willing to share your modified file, so then i could reproduce it? thanks!
1 Like
I see!
Yes, here is the fixed benchmark place
test_benchmark_1.rbxl (132.8 KB)
Let me know if of any issues
not sure why devforum didnt give a notifcation for your reply
unfortunately i could not reproduce the memory issue you had with & without my newest xor patch. my memory start at 2.98GB and at the end of benchmark its about 2.99GB.
here is the result what i got:
16:38:33.104 ================================================================================================================================ - Client - Client:250
16:38:33.104 BENCHMARK RESULTS (Client → Server, 100 fires/frame, 10s) - Client - Client:251
16:38:33.104 ================================================================================================================================ - Client - Client:252
16:38:33.105
Entities - Client - Client:255
16:38:33.105 ┌──────────────┬───────┬────────────┬──────────┬────────────┬────────────┬────────────┐ - Client - Client:256
16:38:33.105 │ Tool │ P50FPS│ P50 Kbps │ Loss │ After Stop │ Tail (srv) │ Span (srv) │ - Client - Client:257
16:38:33.105 ├──────────────┼───────┼────────────┼──────────┼────────────┼────────────┼────────────┤ - Client - Client:258
16:38:33.105 │ roblox │ 54 │ 262311.75 │ 0.0% │ 0.00% │ 0.000 │ 12.822 │ - Client - Client:269
16:38:33.105 │ blink │ 181 │ 11069.28 │ 0.0% │ 0.05% │ 0.003 │ 9.997 │ - Client - Client:269
16:38:33.105 │ bytenetmax │ 185 │ 11227.82 │ 0.0% │ 0.05% │ 0.003 │ 9.991 │ - Client - Client:269
16:38:33.105 │ zap │ 190 │ 11652.29 │ 0.0% │ 0.05% │ 0.003 │ 9.999 │ - Client - Client:269
16:38:33.105 │ NetRay │ 185 │ 11067.10 │ 0.0% │ 0.05% │ 0.003 │ 9.992 │ - Client - Client:269
16:38:33.105 │ warp_dynamic │ 65 │ 15402.03 │ 0.0% │ 0.00% │ 0.000 │ 9.966 │ - Client - Client:269
16:38:33.105 │ warp │ 134 │ 4915.07 │ 0.0% │ 0.15% │ 0.017 │ 10.007 │ - Client - Client:269
16:38:33.105 └──────────────┴───────┴────────────┴──────────┴────────────┴────────────┴────────────┘ - Client - Client:281
16:38:33.105
Booleans - Client - Client:255
16:38:33.105 ┌──────────────┬───────┬────────────┬──────────┬────────────┬────────────┬────────────┐ - Client - Client:256
16:38:33.105 │ Tool │ P50FPS│ P50 Kbps │ Loss │ After Stop │ Tail (srv) │ Span (srv) │ - Client - Client:257
16:38:33.105 ├──────────────┼───────┼────────────┼──────────┼────────────┼────────────┼────────────┤ - Client - Client:258
16:38:33.105 │ roblox │ 124 │ 189205.61 │ 0.0% │ 0.00% │ 0.000 │ 10.006 │ - Client - Client:269
16:38:33.105 │ blink │ 232 │ 55.02 │ 0.0% │ 0.05% │ 0.001 │ 10.001 │ - Client - Client:269
16:38:33.105 │ bytenetmax │ 177 │ 42.16 │ 0.0% │ 0.06% │ 0.003 │ 10.007 │ - Client - Client:269
16:38:33.105 │ zap │ 232 │ 53.92 │ 0.0% │ 0.04% │ 0.001 │ 10.003 │ - Client - Client:269
16:38:33.105 │ NetRay │ 236 │ 45.72 │ 0.0% │ 0.04% │ 0.001 │ 9.994 │ - Client - Client:269
16:38:33.105 │ warp_dynamic │ 202 │ 52.13 │ 0.0% │ 0.10% │ 0.022 │ 10.008 │ - Client - Client:269
16:38:33.105 │ warp │ 207 │ 57.02 │ 0.0% │ 0.00% │ 0.000 │ 9.999 │ - Client - Client:269
16:38:33.105 └──────────────┴───────┴────────────┴──────────┴────────────┴────────────┴────────────┘ - Client - Client:281
16:38:33.105 - Client - Client:284
16:38:33.112 Generated results - Server - Server:183
Did you edit the benches? Or just apply the XOR patch?