performance - How to do fast data deserialization in Haskell -

benchmarking shows cereal library takes 100x longer deserialize data structure of mine (detailed below) takes read same data off drive:

benchmarking read mean: 465.7050 us, lb 460.9873 us, ub 471.0938 us, ci 0.950 std dev: 25.79706 us, lb 22.19820 us, ub 30.81870 us, ci 0.950 found 4 outliers among 100 samples (4.0%)   4 (4.0%) high mild variance introduced outliers: 53.460% variance severely inflated outliers  benchmarking read + decode collecting 100 samples, 1 iterations each, in estimated 6.356502 s mean: 68.85135 ms, lb 67.65992 ms, ub 70.05832 ms, ci 0.950 std dev: 6.134430 ms, lb 5.607914 ms, ub 6.755639 ms, ci 0.950 variance introduced outliers: 74.863% variance severely inflated outliers 

this supported profiling typical deserialization usage of data structure in program of mine 98% of time spent deserializing data , 1% io plus core algorithm:

cost centre                    module               %time %alloc  getword8                       data.serialize.get    30.5   40.4 unget                          data.serialize.get    29.5   17.9 getword64be                    data.serialize.get    14.0   10.7 getlistof                      data.serialize.get    10.2   12.8 roll                           data.serialize         8.2   11.5 shiftl_w64                     data.serialize.get     3.4    2.9 decode                         data.serialize         2.9    3.1 main                           main                   1.3    0.6 

the data structure i'm deserializing intmap [triplet atom] , definitions of component types given below:

type triplet = (a, a, a)  data point = point {     _x :: {-# unpack #-} !double ,     _y :: {-# unpack #-} !double ,     _z :: {-# unpack #-} !double }  data atom = atom {     _serial :: {-# unpack #-} !int    ,     _r      :: {-# unpack #-} !point  ,     _n      :: {-# unpack #-} !word64 } 

i'm using default intmap, (,,) , [] instances provided cereal, , following types , instances custom types:

instance serialize point     put (point x y z) =         put x         put y         put z     = point <$> <*> <*>  instance serialize atom     put (atom s r n) =         put s         put r         put n     = atom <$> <*> <*> 

so questions are:

1. why deserialization slow in general?
2. is there way change data structure (i.e. intmap/[]) make deserialization go faster?
3. is there way change data types (i.e. atom/point) make deserialization go faster?
4. are there faster alternatives cereal within haskell, or should store data structure in c-land more rapid deserialization (i.e. use mmap)?

these files deserializing being used sub-indices search engine since full index cannot fit in memory target computer (which consumer-grade desktop), store each sub-index on disk , read+decode sub-indices pointed initial global index residing in memory. also, i'm not concerned serialization speed since searching index bottle-neck end user , current serialization performance of cereal satisfactory generating , updating index.

edit:

tried out don's suggestion of using space-efficient triplet, , quadrupled speed:

benchmarking read mean: 468.9671 us, lb 464.2564 us, ub 473.8867 us, ci 0.950 std dev: 24.67863 us, lb 21.71392 us, ub 28.39479 us, ci 0.950 found 2 outliers among 100 samples (2.0%)   2 (2.0%) high mild variance introduced outliers: 50.474% variance severely inflated outliers  benchmarking read + decode mean: 15.04670 ms, lb 14.99097 ms, ub 15.10520 ms, ci 0.950 std dev: 292.7815 us, lb 278.8742 us, ub 308.1960 us, ci 0.950 variance introduced outliers: 12.303% variance moderately inflated outliers 

however, still remains bottleneck using 25x time io. also, can explain why don's suggestion works? mean if switched other list (like array?) might give improvement, too?

edit #2: switched latest haskell platform , reran profiling cereal. information considerably more detailed , i've provided hpaste of it.

ok. answer summary of advice. fast deserialization of data:

• use cereal (strict bytestring output) or binary (lazy bytetring output)
• make sure you're compiling -o2, these libraries rely on inlining remove overhead
• use dense data types, such replacing polymorphic tuple unpacked, specialized form.
• avoid converting data types lists serialize them. if have bytestrings, taken care of. unpacked array types, fast io, worth double checking instances
• you may able use mmap'd io
• for double-heavy data consider more efficient double reader.
• use modern array , container types tuned performance, more recent ghc versions.