I just expanded the features of
(ref:gauche.time). Now it has convenience procedures
to run benchmarks and compare results.
gosh> (time-these/report '(cpu 5.0) `((native-map! . ,(cut map (cut + 1 <>) input)) (scheme-map! . ,(cut maps!(cut + 1 <>) input)) (native-map . ,(cut mapa (cut + 1 <>) input)) (scheme-map . ,(cut maps (cut + 1 <>) input)))) Benchmark: ran native-map!, scheme-map!, native-map, scheme-map, each for at least 5.0 cpu seconds. native-map!: 5.267 real, 5.250 cpu (5.230 user + 0.020 sys)@86.86/s n=456 scheme-map!: 5.268 real, 5.260 cpu (5.260 user + 0.000 sys)@85.55/s n=450 native-map: 5.348 real, 5.330 cpu (5.330 user + 0.000 sys)@63.41/s n=338 scheme-map: 5.338 real, 5.340 cpu (5.330 user + 0.010 sys)@62.92/s n=336 Rate native-map! scheme-map! native-map scheme-map native-map! 87/s -- 1.015 1.370 1.380 scheme-map! 86/s 0.985 -- 1.349 1.360 native-map 63/s 0.730 0.741 -- 1.008 scheme-map 63/s 0.724 0.735 0.992 -- #<undef>
If you know Perl, the output would look familiar.
Yes, I blatantly stole the idea of Perl's
It still lacks the features such as cache control and
flexible output formatting seen in Perl,
but I'm happy to have this in the standard Gauche toolbox.
The rest of a little story I ended up having this.
When I add libraries to Gauche, I tend to make them as a set of relatively simple, solid, and mutually orthogonal components, rather than specialized, complex chunk of code each of which is designed for a particular application.
I guess that's rooted from the Unix culture and/or functional programming, and not very uncommon tendency. It is natural to expect it to be easy to assemble a new tool with those simple, orthogonal components when I face a new problem. Probably as the side effect of the tendency, I tend to be overly skeptical to a tool that is specialized to handle a single specific problem.
When I began thinking of enhancing
(I had been running benchmarks for optimization for some
time then, and was getting bored with the tedious process),
I started off with functional components that could be combined
to achieve common task of benchmarking.
It took not long to realize that I couldn't decide a good interface in the intermediate layer, though. If I kept it simple, I feared there would be cases that it was not enough and the whole suite should be scrapped. If I made it general enough, the interface looked too complicated for typical usage.
I stepped back and thought what I wanted to do at the beginning. Basically almost always the task consists of running several pieces of code, measure their speed, and compare. Perl has a module specifically designed for that, and it looked to do just what I want. And it seemed that it's straightforward to copy its main features.
It could be constructed with orthogonal compoents; e.g. creating the layout of comparison matrix could be a separete component. But I don't understand the problem enough to design a clean interface to make it independent. So I hard-coded that.
I may rewrite them in future, once I have a solid idea on how to design generic text table formatting API. In the meantime, it is more useful to shortcut and have some working code, than wait for clean, neat, and composable components. Specialized tools aren't necessarily bad, if the problem is well defined and common enough.
Here are several procedures added for benchmarking. See the git repo for the details.
time-this: benchmark single code in thunk. returns
time-result-: operations on
time-these: benchmark several code to compare and returns list of results.
report-time-result: format the result of
time-these/report: combination of above two.
Tags: gauche.time, Perl, benchmarking
Post a comment