Benchmarker Plugin - Compare function speeds with graphs, percentiles, and more!

Help and Feedback Creations Feedback
83

Just to make sure I understand; I’m supposed to report here, right?

I believe you misunderstood me. With displays I meant that the flamechart displays with text the values, but the chart itself doesn’t display/resemble those values correctly.

Here’s an example:

image
image1638×907 88.8 KB

As you can see, the difference between the sizes for TweenPlus and TweenService are not even close to the 48μs which is the 50th-percentile difference. Why does this happen? Because the flamechart is not using the 50th-percentile — what it’s then using I can’t tell when looking through the source code, because that part is so unreadable.

I don’t think reproduction steps are necesssary, because it’s just a general issue, although only clear to the eye in some scenarios.

Oh sorry, I wasn’t aware. I thought all updates were posted here, so that’s why.
I appreciate you maintaining it.

The plugin has proven useful, but it’s missing a lot of polish.

Additionally, I have two super simple feature requests, which I have already added manually, but would love official support for:

An Iterations parameter.

This is a parameter, just like Functions, BeforeEach etc.

What it allows you to do is control the amount of iterations of the for-loop.
By default (in the current official version of Benchmarker) it is 1.000 iterations. I’d like to be able to control this, as there are a few scenarios where you’d want to increase/decrease the iterations for better accuracy or , without influencing the actual function run-time with e.g. a manual for-loop.

The parameter is of course of type number.

A Runs parameter

This is a parameter, just like Functions, BeforeEach etc.

What it allows you to do is to repeatedly run each function multiple times.

But it doesn’t work quite like just a manual for-loop inside of the functions.
This is because it allows you to run the BeforeEach before every function run, and without taking the measurement of the BeforeEach — because if you just did some BeforeEach logic in a manual loop, before the actual thing you want to measure, inside of each function, well then you’d have the BeforeEach logic be apart of the measurement of each function (the for-loop would also be apart of the measurement).

Additionally, it’s a time-saving feature, because you don’t have to manually create for-loops inside of every single function.

The parameter is of course of type number.
​
​
​
If you have any questions, please let me know!

84

Here’s another small but annoying bug with the flamechart:

image
image1635×933 112 KB

The flamechart code uses an unsanitized key and looks it up in a table of sanitized keys, so for any keys that contain . like in the example image, the calculated order value will simply be replaced with keyCount due to or keyCount. This order value is used to calculate the color, which results in all functions, whose name contain ., having the same exact color, just like in the example image.

85

Oh and another very small issue:

image
image808×454 8.75 KB

As you can see, some of the blue graph is overlapping the red, and sometimes not.
I believe the ZIndex is just random. You should group each graph’s individual lines so that they share the same ZIndex.

86

Thank you for the reports and requests. :+1: I will take a deeper look into these when I have a chance.

Iterations and others were actually in older versions of Benchmarker but I removed them because people were shooting their foot and getting unreliable results using improper configs. I felt it best to force reasonable standards so that results shared around would be trustworthy and not heavily dependent on user configuration.
I quite like the Runs idea, seems like a great QoL improvement for a common pattern.

1 Like
87

It’s been a few months. Are you still going to fix these bugs and perhaps add a few features?

If you have any issues, let me know!

88

I’ve been quite busy but rest assured I do intend to do this when I can.

3 Likes
89

Hello, I would really appreciate it if this plugin would be accessible using WeChat pay (Ya, no credit card to connect on Itch)

90

This feels a bit weird to say given you market the plugin as “used by top developers and Roblox engineers”, and that it is a “power user tool”. Why remove something that a power user will know how to use properly?

Like, dude, come on. This is not an industry-standard tool. If it’s really that big of a concern, why not add a warning that lowering the iterations will naturally reduce the quality of the data? Or perhaps show the iteration/run parameters of the test in the benchmark UI?

For my benchmarks where I am only looking at a guesstimate of time taken (so I can tweak parameters and try again), I now have to wait minutes for everything to complete on every single run. Please add back a way to control how long benchmarks take. Don’t coddle your paying users.
I’m writing this as I look for a way to downgrade my current Benchmarker version so I can get those options back.

2 Likes