Michael Truog (okeuday) wrote,
Michael Truog
okeuday

More Erlang Benchmark Results

I added a way to automate all the interesting benchmarks for the Erlang data structures (including a trie). Below are the results for non-HiPE R14B01 on Linux version 2.6.32-21, Ubuntu 10.04 with an Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz.

TEST string_key
N == 1000 (10 runs)
              aadict get:      812.0 µs (  1.3), set:     2207.6 µs (  1.4)
                dict get:     1051.2 µs (  1.7), set:     3267.3 µs (  2.0)
           ets (set) get:      888.7 µs (  1.5), set:     1611.8 µs (  1.0)
       ets x10 (set) get:     1092.5 µs (  1.8)
            gb_trees get:      938.1 µs (  1.6), set:     2803.5 µs (  1.7)
             orddict get:    25313.7 µs ( 41.9), set:    22926.3 µs ( 14.2)
  process dictionary get:      604.2 µs (  1.0), set:     1623.2 µs (  1.0)
              rbdict get:      801.3 µs (  1.3), set:     1769.9 µs (  1.1)
                trie get:      711.0 µs (  1.2), set:     2528.4 µs (  1.6)
N == 10000 (10 runs)
              aadict get:    22826.8 µs (  2.6), set:    44371.5 µs (  1.8)
                dict get:    18529.8 µs (  2.1), set:    61161.0 µs (  2.5)
           ets (set) get:    18280.8 µs (  2.1), set:    24070.6 µs (  1.0)
       ets x10 (set) get:    22830.2 µs (  2.6)
            gb_trees get:    24896.9 µs (  2.8), set:    50116.9 µs (  2.1)
             orddict get:  2674456.1 µs (302.2), set:  2442526.4 µs (101.5)
  process dictionary get:    13625.8 µs (  1.5), set:    26584.9 µs (  1.1)
              rbdict get:    22842.5 µs (  2.6), set:    39537.7 µs (  1.6)
                trie get:     8848.6 µs (  1.0), set:    35448.0 µs (  1.5)
N == 50000 (10 runs)
              aadict get:   177210.4 µs (  2.8), set:   350989.2 µs (  2.5)
                dict get:   107799.7 µs (  1.7), set:   638809.5 µs (  4.6)
           ets (set) get:    92473.9 µs (  1.5), set:   139183.1 µs (  1.0)
       ets x10 (set) get:   108796.4 µs (  1.7)
            gb_trees get:   194868.6 µs (  3.1), set:   374895.5 µs (  2.7)
             orddict get: 80504982.5 µs (*****), set: 68700235.0 µs (493.6)
  process dictionary get:    63323.8 µs (  1.0), set:   141767.2 µs (  1.0)
              rbdict get:   182692.8 µs (  2.9), set:   288624.4 µs (  2.1)
                trie get:    71009.8 µs (  1.1), set:   354443.1 µs (  2.5)
TEST integer_key
N == 1000 (10 runs)
              aadict get:      481.9 µs (  5.5), set:     2202.4 µs ( 16.9)
     array (dynamic) get:      379.6 µs (  4.3), set:      805.8 µs (  6.2)
       array (fixed) get:      384.7 µs (  4.4), set:      850.3 µs (  6.5)
                dict get:      459.1 µs (  5.2), set:     2122.6 µs ( 16.3)
           ets (set) get:      337.0 µs (  3.8), set:      611.3 µs (  4.7)
       ets x10 (set) get:      203.9 µs (  2.3)
            gb_trees get:      489.8 µs (  5.5), set:     4817.7 µs ( 37.1)
             orddict get:    16415.9 µs (185.9), set:      130.0 µs (  1.0)
  process dictionary get:      131.8 µs (  1.5), set:      157.0 µs (  1.2)
              rbdict get:      483.5 µs (  5.5), set:     2270.6 µs ( 17.5)
               tuple get:       88.3 µs (  1.0), set:    11688.3 µs ( 89.9)
N == 10000 (10 runs)
              aadict get:     7085.1 µs (  8.2), set:    23029.4 µs ( 18.2)
     array (dynamic) get:     5056.6 µs (  5.9), set:    12024.1 µs (  9.5)
       array (fixed) get:     5086.3 µs (  5.9), set:    11836.4 µs (  9.4)
                dict get:     5362.6 µs (  6.2), set:    32728.6 µs ( 25.9)
           ets (set) get:     3768.3 µs (  4.4), set:     7688.5 µs (  6.1)
       ets x10 (set) get:     1984.2 µs (  2.3)
            gb_trees get:     6315.3 µs (  7.3), set:    62973.9 µs ( 49.8)
             orddict get:  1651408.1 µs (*****), set:     1264.6 µs (  1.0)
  process dictionary get:     1329.4 µs (  1.5), set:     1614.8 µs (  1.3)
              rbdict get:     7058.8 µs (  8.2), set:    21787.7 µs ( 17.2)
               tuple get:      861.5 µs (  1.0), set:  1023597.5 µs (809.4)
N == 50000 (10 runs)
              aadict get:    37768.6 µs (  9.3), set:   163942.9 µs ( 27.2)
     array (dynamic) get:    30625.4 µs (  7.6), set:    67480.3 µs ( 11.2)
       array (fixed) get:    30555.5 µs (  7.5), set:    83706.3 µs ( 13.9)
                dict get:    29288.6 µs (  7.2), set:   435990.0 µs ( 72.2)
           ets (set) get:    26412.3 µs (  6.5), set:    55838.4 µs (  9.2)
       ets x10 (set) get:    12812.3 µs (  3.2)
            gb_trees get:    36219.4 µs (  8.9), set:   396159.8 µs ( 65.6)
             orddict get: 42714046.1 µs (*****), set:     6037.3 µs (  1.0)
  process dictionary get:     6583.2 µs (  1.6), set:     9550.8 µs (  1.6)
              rbdict get:    37226.9 µs (  9.2), set:   146739.7 µs ( 24.3)
               tuple get:     4052.0 µs (  1.0), set: 19153655.7 µs (*****)


HiPE results:
TEST string_key
N == 1000 (10 runs)
              aadict get:      839.5 µs (  1.5), set:     2558.9 µs (  1.6)
                dict get:      946.7 µs (  1.7), set:     3233.6 µs (  2.0)
           ets (set) get:      836.5 µs (  1.5), set:     1654.7 µs (  1.0)
       ets x10 (set) get:      914.8 µs (  1.6)
            gb_trees get:      945.8 µs (  1.7), set:     3183.7 µs (  2.0)
             orddict get:    23305.9 µs ( 40.7), set:    23718.0 µs ( 15.0)
  process dictionary get:      573.2 µs (  1.0), set:     1582.6 µs (  1.0)
              rbdict get:      831.6 µs (  1.5), set:     2134.6 µs (  1.3)
                trie get:      652.9 µs (  1.1), set:     2379.5 µs (  1.5)
N == 10000 (10 runs)
              aadict get:    21463.1 µs (  2.4), set:    43555.5 µs (  2.0)
                dict get:    18086.4 µs (  2.1), set:    60505.8 µs (  2.7)
           ets (set) get:    17702.1 µs (  2.0), set:    22067.5 µs (  1.0)
       ets x10 (set) get:    21480.2 µs (  2.4)
            gb_trees get:    22833.0 µs (  2.6), set:    50190.2 µs (  2.3)
             orddict get:  2467552.8 µs (280.5), set:  2334416.3 µs (105.8)
  process dictionary get:    13326.6 µs (  1.5), set:    26943.1 µs (  1.2)
              rbdict get:    21257.1 µs (  2.4), set:    36856.8 µs (  1.7)
                trie get:     8796.2 µs (  1.0), set:    35143.5 µs (  1.6)
N == 50000 (10 runs)
              aadict get:   178153.0 µs (  2.8), set:   328912.0 µs (  2.4)
                dict get:   108123.8 µs (  1.7), set:   641260.2 µs (  4.6)
           ets (set) get:    95263.1 µs (  1.5), set:   138832.9 µs (  1.0)
       ets x10 (set) get:   109103.1 µs (  1.7)
            gb_trees get:   195767.6 µs (  3.1), set:   372523.4 µs (  2.7)
             orddict get: 71658356.8 µs (*****), set: 66819443.2 µs (481.3)
  process dictionary get:    63528.8 µs (  1.0), set:   141625.8 µs (  1.0)
              rbdict get:   179620.9 µs (  2.8), set:   289155.1 µs (  2.1)
                trie get:    69662.6 µs (  1.1), set:   306391.0 µs (  2.2)
TEST integer_key
N == 1000 (10 runs)
              aadict get:      490.9 µs (  5.6), set:     2064.5 µs ( 14.4)
     array (dynamic) get:      379.1 µs (  4.3), set:    10328.3 µs ( 72.1)
       array (fixed) get:      387.9 µs (  4.4), set:      859.7 µs (  6.0)
                dict get:      473.7 µs (  5.4), set:     2312.0 µs ( 16.1)
           ets (set) get:      324.6 µs (  3.7), set:      572.9 µs (  4.0)
       ets x10 (set) get:      253.0 µs (  2.9)
            gb_trees get:     1501.1 µs ( 17.0), set:     5331.8 µs ( 37.2)
             orddict get:    16627.7 µs (188.1), set:      143.3 µs (  1.0)
  process dictionary get:      130.6 µs (  1.5), set:      165.4 µs (  1.2)
              rbdict get:      484.5 µs (  5.5), set:     1858.8 µs ( 13.0)
               tuple get:       88.4 µs (  1.0), set:    10606.2 µs ( 74.0)
N == 10000 (10 runs)
              aadict get:     7629.1 µs (  9.5), set:    22593.4 µs ( 19.4)
     array (dynamic) get:     5058.1 µs (  6.3), set:    13388.1 µs ( 11.5)
       array (fixed) get:     5442.5 µs (  6.8), set:    12669.2 µs ( 10.9)
                dict get:     5522.5 µs (  6.9), set:    34435.3 µs ( 29.5)
           ets (set) get:     3692.8 µs (  4.6), set:     7651.2 µs (  6.6)
       ets x10 (set) get:     2204.8 µs (  2.7)
            gb_trees get:     6485.1 µs (  8.1), set:    66586.3 µs ( 57.0)
             orddict get:  1635810.3 µs (*****), set:     1167.3 µs (  1.0)
  process dictionary get:     1301.4 µs (  1.6), set:     1923.7 µs (  1.6)
              rbdict get:     7530.5 µs (  9.4), set:    20872.8 µs ( 17.9)
               tuple get:      803.8 µs (  1.0), set:  1025256.6 µs (878.3)
N == 50000 (10 runs)
              aadict get:    38563.9 µs (  9.6), set:   147089.5 µs ( 24.1)
     array (dynamic) get:    34494.8 µs (  8.6), set:    84619.5 µs ( 13.9)
       array (fixed) get:    31674.2 µs (  7.9), set:    86212.3 µs ( 14.1)
                dict get:    31817.1 µs (  7.9), set:   549230.6 µs ( 90.0)
           ets (set) get:    26095.5 µs (  6.5), set:    56195.0 µs (  9.2)
       ets x10 (set) get:    14836.9 µs (  3.7)
            gb_trees get:    36555.7 µs (  9.1), set:   397905.1 µs ( 65.2)
             orddict get: 42921037.8 µs (*****), set:     6101.3 µs (  1.0)
  process dictionary get:     6665.2 µs (  1.7), set:    11555.6 µs (  1.9)
              rbdict get:    37324.8 µs (  9.3), set:   137461.0 µs ( 22.5)
               tuple get:     4025.4 µs (  1.0), set: 27603206.1 µs (*****)
Subscribe
  • Post a new comment

    Error

    default userpic

    Your reply will be screened

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.
  • 0 comments