Tag Archives: performance

Useful Key Performance Indicators

I’ve seen a lot of business statistic graphs over the years and many of them are useless for communicating trends.  Key Performance Indicators (KPIs) should be ubiquitous and understandable to everyone in a business.

Rules to follow

  • Absolute numbers (e.g. page views per day, widgets sold per day) tell us how a business is doing.
  • Ratios of X/unit tell us how systems are doing.
  • Ratios of Units/Profit tell us how much they cost and bridge the gap between Biz metrics and Tech metrics.
  • Simple KPIs with concrete units are easy understand. KPIs without well-understood units require too much explanation.
  • KPI plots must have the actual units specified in the legends.
  • KPI plots with the same numerator should be in the same graphic.
  • Plot Y axies should be appropriately scaled and must have a 0-origin.
  • The collection of the KPI data does not need to be exact, it must only be agreed upon.
  • KPIs must be versioned; if a KPI collection method is changed, the KPI must have a new version.

Examples

The KPIs below are relatively easy to compute and simple enough for anyone to craft given access to basic operational data.

Assume we are selling Widgets, for each Application and collectively for all Applications, sampled for each day:

Biz Metrics

  1. W = number of new widgets sold
  2. P = total money in-flow – out-flow (gross profit)
  3. WP = projected new widgets gross profit (estimated)

Applications (internal and externally facing)

  1. P = page views
  2. PE = page errors
  3. PEP = PE / P = errors / pages
  4. PT = total page time
  5. PTP = PT / P = page time / pages
  6. PW = P / W = pages / new widget sold
  7. PTW = PT / W = page time / new widgets sold
  8. PTWP = PT / WP = page time / new widgets sold gross profit

Offline Processing – Batches

  1. B = batch items
  2. BE = batch item errors
  3. BEB = BE / B = errors / items
  4. BT = total batch item time
  5. BTB = BT / B = item time / item
  6. BTP = BT / P = item time / gross profit

ChicagoRuby Ruby Code Tweaks slides, code and video

The slides from my ChicagoRuby 2010/05/04 presentation :

http://kurtstephens.com/pub/ruby/ruby_code_tweaks/

All the raw data used to generate the graph should be referenced in the slides.

The code used to generate the slides is here:

http://github.com/kstephens/ruby_code_tweaks

I’m looking to increase the set of code “Problems” to cover other tiny code idioms and platform issues, for example: regular expressions, numerics, etc. If you have ideas, take a look at the code and contact me.

Justin Love gave a fantastic presentation on lambda and closure.

Thanks to everyone who came — hope it was helpful.

Video from the talk:

Ruby Code Performance Tweaks by Kurt Stephens from ChicagoRuby on Vimeo.

Ruby: Performance of Symbol Construction

Measurements of Symbol Constructor Expressions.

n=10_000_000

ruby 1.8.6 (2008-08-08 patchlevel 286) [i686-linux]:

> /cnu/bin/ruby symbol_benchmark.rb
                               user     system      total        real
Null Test                  0.740000   0.000000   0.740000 (  0.742914)
'foo_bar'                  1.670000   0.000000   1.670000 (  1.661374)
"foo_bar"                  1.620000   0.000000   1.620000 (  1.625221)
:foo_bar                   0.890000   0.000000   0.890000 (  0.886903)
:'foo_bar'                 0.880000   0.000000   0.880000 (  0.878555)
:"foo_bar"                 0.860000   0.000000   0.860000 (  0.867110)
"foo_bar".to_sym           2.830000   0.000000   2.830000 (  2.830536)
str.to_sym                 2.050000   0.000000   2.050000 (  2.052756)
:"{str}"                   0.880000   0.000000   0.880000 (  0.881367)
:"foo_#{'bar'}"            0.860000   0.000000   0.860000 (  0.854944)
"foo_#{'bar'}".to_sym      2.880000   0.000000   2.880000 (  2.942499)
:"foo_#{bar}"              4.280000   0.000000   4.280000 (  4.290880)
"foo_#{bar}".to_sym        4.930000   0.000000   4.930000 (  4.929801)

ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]:

> /usr/bin/ruby symbol_benchmark.rb
                               user     system      total        real
Null Test                  0.780000   0.000000   0.780000 (  0.782060)
'foo_bar'                  3.480000   0.770000   4.250000 (  4.263535)
"foo_bar"                  3.400000   0.820000   4.220000 (  4.224401)
:foo_bar                   2.280000   0.950000   3.230000 (  3.223590)
:'foo_bar'                 2.370000   0.860000   3.230000 (  3.249133)
:"foo_bar"                 2.310000   0.940000   3.250000 (  3.283796)
"foo_bar".to_sym           5.010000   0.980000   5.990000 (  6.029690)
str.to_sym                 3.740000   0.930000   4.670000 (  4.706827)
:"{str}"                   2.440000   0.810000   3.250000 (  3.254253)
:"foo_#{'bar'}"            2.470000   0.780000   3.250000 (  3.245739)
"foo_#{'bar'}".to_sym      4.800000   0.970000   5.770000 (  5.767003)
:"foo_#{bar}"              7.490000   0.990000   8.480000 (  8.506735)
"foo_#{bar}".to_sym        8.640000   0.890000   9.530000 (  9.532481)

ruby 1.9.0 (2008-11-25 revision 20347) [i686-linux]:

> ~/local/ruby/trunk/bin/ruby symbol_benchmark.rb
                               user     system      total        real
Null Test                  0.690000   0.000000   0.690000 (  0.695087)
'foo_bar'                  1.760000   0.000000   1.760000 (  1.772417)
"foo_bar"                  1.780000   0.000000   1.780000 (  1.772031)
:foo_bar                   0.720000   0.000000   0.720000 (  0.764095)
:'foo_bar'                 0.710000   0.000000   0.710000 (  0.710952)
:"foo_bar"                 0.720000   0.000000   0.720000 (  0.717041)
"foo_bar".to_sym           3.370000   0.000000   3.370000 (  3.371836)
str.to_sym                 2.340000   0.010000   2.350000 (  2.347026)
:"{str}"                   0.720000   0.000000   0.720000 (  0.723032)
:"foo_#{'bar'}"            0.710000   0.000000   0.710000 (  0.712813)
"foo_#{'bar'}".to_sym      3.370000   0.020000   3.390000 (  3.399618)
:"foo_#{bar}"              5.030000   0.000000   5.030000 (  5.037542)
"foo_#{bar}".to_sym        5.060000   0.010000   5.070000 (  5.112535)

rubinius 0.10.0 (ruby 1.8.6 compatible) (c458077eb) (12/31/2009) [i686-pc-linux-gnu]:

> ~/local/ruby/rubininus/bin/rbx symbol_benchmark.rb
                               user     system      total        real
Null Test                  2.369585   0.000000   2.369585 (  2.369602)
'foo_bar'                  3.616918   0.000000   3.616918 (  3.616949)
"foo_bar"                  3.672719   0.000000   3.672719 (  3.672752)
:foo_bar                   2.407764   0.000000   2.407764 (  2.407796)
:'foo_bar'                 2.462789   0.000000   2.462789 (  2.462824)
:"foo_bar"                 2.401784   0.000000   2.401784 (  2.401812)
"foo_bar".to_sym           7.127442   0.000000   7.127442 (  7.127472)
str.to_sym                15.128944   0.000000  15.128944 ( 15.128974)
:"{str}"                   2.411760   0.000000   2.411760 (  2.411789)
:"foo_#{'bar'}"            2.437595   0.000000   2.437595 (  2.437629)
"foo_#{'bar'}".to_sym      7.120185   0.000000   7.120185 (  7.120209)
:"foo_#{bar}"             20.144837   0.000000  20.144837 ( 20.144864)
"foo_#{bar}".to_sym       20.222530   0.000000  20.222530 ( 20.222891)

jruby 1.1.5 (ruby 1.8.6 patchlevel 114) (2008-11-03 rev 7996) [i386-java]:

> ~/local/ruby/jruby-1.1.5/bin/jruby symbol_benchmark.rb
                               user     system      total        real
Null Test                  0.936000   0.000000   0.936000 (  0.936172)
'foo_bar'                  1.886000   0.000000   1.886000 (  1.886079)
"foo_bar"                  1.880000   0.000000   1.880000 (  1.880344)
:foo_bar                   1.379000   0.000000   1.379000 (  1.379957)
:'foo_bar'                 5.842000   0.000000   5.842000 (  5.842242)
:"foo_bar"                 5.834000   0.000000   5.834000 (  5.833263)
"foo_bar".to_sym           2.597000   0.000000   2.597000 (  2.597294)
str.to_sym                 2.306000   0.000000   2.306000 (  2.306082)
:"{str}"                   5.655000   0.000000   5.655000 (  5.655672)
:"foo_#{'bar'}"            5.785000   0.000000   5.785000 (  5.785285)
"foo_#{'bar'}".to_sym      2.585000   0.000000   2.585000 (  2.584877)
:"foo_#{bar}"              7.532000   0.000000   7.532000 (  7.531846)
"foo_#{bar}".to_sym        8.361000   0.000000   8.361000 (  8.360630)

symbol_benchmark.rb:

require 'benchmark'

def bm_expr(x, expr, title = nil)
  e = <<"END"
  n = 10_000_000
  str = 'foo_bar'
  bar = 'bar'
  #begin; GC.enable; GC.start; GC.disable; rescue nil; end
  begin; GC.start; rescue nil; end
  x.report(#{(title || expr).inspect}) {
    n.times { 
      #{expr}
    }
  }
END
  eval e

end

Benchmark.bm(25) { | x |
  bm_expr x, "", 'Null Test' 
  bm_expr x, "'foo_bar'" 
  bm_expr x, '"foo_bar"' 
  bm_expr x, ':foo_bar' 
  bm_expr x, ":'foo_bar'" 
  bm_expr x, ':"foo_bar"' 
  bm_expr x, '"foo_bar".to_sym' 
  bm_expr x, 'str.to_sym' 
  bm_expr x, ':"{str}"' 
  bm_expr x, ':"foo_#{\'bar\'}"'
  bm_expr x, '"foo_#{\'bar\'}".to_sym'
  bm_expr x, ':"foo_#{bar}"'
  bm_expr x, '"foo_#{bar}".to_sym'
}

First time I ran the benchmarks without forcing and disabling GC :'foo_bar' appeared faster than :foo_bar. Reminder to self: always disable GC around benchmarks, unless you are benchmarking GC.