shootout: Expression Performance Comparison

2:0

shootout: Expression Performance Comparison🔗ℹ

Neil Van Dyke

License: LGPLv3 Web: http://www.neilvandyke.org/racket/shootout/

1 Introduction🔗ℹ

This package provides the shootout syntax, which is a small tool to help compare the performance of multiple Racket expressions. This can be useful when hand-optimizing performance-sensitive Racket code.

For example, the program:

(define (va x (y "bbb") (z "ccc"))
  (expt 42 42)
  (string-append x y z))

(define (fa x y z)
  (expt 42 42)
  (string-append x y z))

(shootout (va "aaa")
          (fa "aaa" "bbb" "ccc")
          #:repeat 1000000)

might produce a log similar to:

SHOOTOUT-SAMPLING-GC-BASE

SHOOTOUT-BEGIN (REPEAT 1000000, GC-BASE-MS 2498, GC-BASE-GCMS 2496)

SHOOTOUT-EXPRESSION 1:

(va "aaa")

SHOOTOUT-TIMING (RUN 275, RUN-GCMS 12, COST 274, COST-GCMS 12, CLEANUP-DIFF -2.0)

SHOOTOUT-EXPRESSION 2:

(fa "aaa" "bbb" "ccc")

SHOOTOUT-TIMING (RUN 255, RUN-GCMS 12, COST 254, COST-GCMS 13, CLEANUP-DIFF -1.0)

SHOOTOUT-END

The most meaningful numbers are “RUN” and “RUN-GCMS”. RUN is the real time in milliseconds during which the expression ran for the specificed number of iteractions (the “REPEAT” number). “RUN-GMCS” is the number of milliseconds of garbage collection (GC) time that occurred during the run. Neither of these numbers includes the GC “cleanup” time that shootout performs in an attempt to gauge any lingering GC cost of the expression.

Note that multiple runs of shootout may give substantially different results. So, you may wish to run multiple times and take the variation into consideration before drawing any conclusions.

Note that shootout is only an aid for performance-comparison, and does not control for all variables, nor are its measurements necessarily accurate.

Now that Racket has a time feature, you might prefer that, though that alone is not a complete replacement for shootout.

2 Interface🔗ℹ

syntax
(shootout expr ... maybe-repeat)

maybe-repeat =
| #:repeat repeat

expr : any/c
repeat : exact-nonnegative-integer?

Performs repeat iterations of each expr and reports performance information in a log.

Note that the format of this log and the information is reports is likely to change in future versions of this package.

3 Known Issues🔗ℹ

Rework to use new Racket time-apply feature, instead of old kludges.

4 History🔗ℹ

Version 2:0 — 2016-02-29
- Moving from PLaneT to new package system.
Version 1:1 — 2011-08-22
- Fixed typo in legal notice.
Version 1:0 — 2010-12-21
- Initial release.

5 Legal🔗ℹ

Copyright 2010, 2011, 2016 Neil Van Dyke. This program is Free Software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See http://www.gnu.org/licenses/ for details. For other licenses and consulting, please contact the author.