Measuring performance in a hardware-agnostic way
Measuring performance in a hardware-agnostic way
Is there a way to measure performance without depending on the hardware, i.e. two entirely different computers get the same score for the same code?
I could probably run the program on a server or something, but something local feels more reliable.
Yes, you can consider the algorithmic complexity of your code, which is independent of any hardware.
Sure, but that makes a lot of optimizations nonexistent (e.g. cloning the input on an O(n) algorithm is "free")
You don't need to use big-O. You can calculate the full complexity in algebraic notation. It's just a lot more work as you don't get to discard terms.
Im theory you could try counting CPU cycles or something analogous, or algorithmic complexity as the other commenter mentioned. You could also try to measure it in comparison to different code.
Outside of that, and in any practical sense, no you can't. Performance measurements are dependent on the machine as much as the software.
If a car is hardware, and the driver is software, how would you measure the performance of the driver in the different cars? The only way I can think of is if you had 2 different drivers and could compare their times in both cars. If driver 1 is 2x as fast in car 1, and 2x as fast in car 2, you could say driver 1 has a score 2x higher than driver 2.
CPU cycles depend on the instruction set of the hardware. This would also require compilation without any optimization to avoid even software dependency.
One option could be to emulate a full CPU and run it on that, and count the cycles on that. One issue is linking/syscalls. Maybe compiling to smth like WASM could work?
perf and valgrind might be good places to start. Although, some programs aren't going to have the exact same executed instruction count between runs, and it's possible that executed instruction count can depend on the exact CPU that's running. You can probably mitigate the latter by running valgrind and the program inside of QEMU.
Thank you! That's exactly the kinda thing I was looking for!
I believe everyone gets different input and needs to produce a different result. There can be multiple solutions. Do you process as you are going, or do you parse and build a data structure to process later so you loop through the results multiple times.