0001 ================================================================================================
0002 Benchmark to measure CSV read/write performance
0003 ================================================================================================
0004
0005 OpenJDK 64-Bit Server VM 1.8.0_252-8u252-b09-1~18.04-b09 on Linux 4.15.0-1063-aws
0006 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0007 Parsing quoted values: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0008 ------------------------------------------------------------------------------------------------------------------------
0009 One quoted string 45457 45731 344 0.0 909136.8 1.0X
0010
0011 OpenJDK 64-Bit Server VM 1.8.0_252-8u252-b09-1~18.04-b09 on Linux 4.15.0-1063-aws
0012 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0013 Wide rows with 1000 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0014 ------------------------------------------------------------------------------------------------------------------------
0015 Select 1000 columns 129646 130527 1412 0.0 129646.3 1.0X
0016 Select 100 columns 42444 42551 119 0.0 42444.0 3.1X
0017 Select one column 35415 35428 20 0.0 35414.6 3.7X
0018 count() 11114 11128 16 0.1 11113.6 11.7X
0019 Select 100 columns, one bad input field 93353 93670 275 0.0 93352.6 1.4X
0020 Select 100 columns, corrupt record field 113569 113952 373 0.0 113568.8 1.1X
0021
0022 OpenJDK 64-Bit Server VM 1.8.0_252-8u252-b09-1~18.04-b09 on Linux 4.15.0-1063-aws
0023 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0024 Count a dataset with 10 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0025 ------------------------------------------------------------------------------------------------------------------------
0026 Select 10 columns + count() 18498 18589 87 0.5 1849.8 1.0X
0027 Select 1 column + count() 11078 11095 27 0.9 1107.8 1.7X
0028 count() 3928 3950 22 2.5 392.8 4.7X
0029
0030 OpenJDK 64-Bit Server VM 1.8.0_252-8u252-b09-1~18.04-b09 on Linux 4.15.0-1063-aws
0031 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0032 Write dates and timestamps: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0033 ------------------------------------------------------------------------------------------------------------------------
0034 Create a dataset of timestamps 1933 1940 11 5.2 193.3 1.0X
0035 to_csv(timestamp) 18078 18243 255 0.6 1807.8 0.1X
0036 write timestamps to files 12668 12786 134 0.8 1266.8 0.2X
0037 Create a dataset of dates 2196 2201 5 4.6 219.6 0.9X
0038 to_csv(date) 9583 9597 21 1.0 958.3 0.2X
0039 write dates to files 7091 7110 20 1.4 709.1 0.3X
0040
0041 OpenJDK 64-Bit Server VM 1.8.0_252-8u252-b09-1~18.04-b09 on Linux 4.15.0-1063-aws
0042 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0043 Read dates and timestamps: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0044 ------------------------------------------------------------------------------------------------------------------------
0045 read timestamp text from files 2166 2177 10 4.6 216.6 1.0X
0046 read timestamps from files 53212 53402 281 0.2 5321.2 0.0X
0047 infer timestamps from files 109788 110372 570 0.1 10978.8 0.0X
0048 read date text from files 1921 1929 8 5.2 192.1 1.1X
0049 read date from files 25470 25499 25 0.4 2547.0 0.1X
0050 infer date from files 27201 27342 134 0.4 2720.1 0.1X
0051 timestamp strings 3638 3653 19 2.7 363.8 0.6X
0052 parse timestamps from Dataset[String] 61894 62532 555 0.2 6189.4 0.0X
0053 infer timestamps from Dataset[String] 125171 125430 236 0.1 12517.1 0.0X
0054 date strings 3736 3749 14 2.7 373.6 0.6X
0055 parse dates from Dataset[String] 30787 30829 43 0.3 3078.7 0.1X
0056 from_csv(timestamp) 60842 61035 209 0.2 6084.2 0.0X
0057 from_csv(date) 30123 30196 95 0.3 3012.3 0.1X
0058
0059 OpenJDK 64-Bit Server VM 1.8.0_252-8u252-b09-1~18.04-b09 on Linux 4.15.0-1063-aws
0060 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0061 Filters pushdown: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0062 ------------------------------------------------------------------------------------------------------------------------
0063 w/o filters 28985 29042 80 0.0 289852.9 1.0X
0064 pushdown disabled 29080 29146 58 0.0 290799.4 1.0X
0065 w/ filters 2072 2084 17 0.0 20722.3 14.0X
0066
0067
