0001 ================================================================================================
0002 Benchmark to measure CSV read/write performance
0003 ================================================================================================
0004
0005 OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
0006 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0007 Parsing quoted values: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0008 ------------------------------------------------------------------------------------------------------------------------
0009 One quoted string 46568 46683 198 0.0 931358.6 1.0X
0010
0011 OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
0012 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0013 Wide rows with 1000 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0014 ------------------------------------------------------------------------------------------------------------------------
0015 Select 1000 columns 129836 130796 1404 0.0 129836.0 1.0X
0016 Select 100 columns 40444 40679 261 0.0 40443.5 3.2X
0017 Select one column 33429 33475 73 0.0 33428.6 3.9X
0018 count() 7967 8047 73 0.1 7966.7 16.3X
0019 Select 100 columns, one bad input field 90639 90832 266 0.0 90638.6 1.4X
0020 Select 100 columns, corrupt record field 109023 109084 74 0.0 109023.3 1.2X
0021
0022 OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
0023 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0024 Count a dataset with 10 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0025 ------------------------------------------------------------------------------------------------------------------------
0026 Select 10 columns + count() 20685 20707 35 0.5 2068.5 1.0X
0027 Select 1 column + count() 13096 13149 49 0.8 1309.6 1.6X
0028 count() 3994 4001 7 2.5 399.4 5.2X
0029
0030 OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
0031 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0032 Write dates and timestamps: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0033 ------------------------------------------------------------------------------------------------------------------------
0034 Create a dataset of timestamps 2169 2203 32 4.6 216.9 1.0X
0035 to_csv(timestamp) 14401 14591 168 0.7 1440.1 0.2X
0036 write timestamps to files 13209 13276 59 0.8 1320.9 0.2X
0037 Create a dataset of dates 2231 2248 17 4.5 223.1 1.0X
0038 to_csv(date) 10406 10473 68 1.0 1040.6 0.2X
0039 write dates to files 7970 7976 9 1.3 797.0 0.3X
0040
0041 OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
0042 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0043 Read dates and timestamps: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0044 ------------------------------------------------------------------------------------------------------------------------
0045 read timestamp text from files 2387 2391 6 4.2 238.7 1.0X
0046 read timestamps from files 53503 53593 124 0.2 5350.3 0.0X
0047 infer timestamps from files 107988 108668 647 0.1 10798.8 0.0X
0048 read date text from files 2121 2133 12 4.7 212.1 1.1X
0049 read date from files 29983 30039 48 0.3 2998.3 0.1X
0050 infer date from files 30196 30436 218 0.3 3019.6 0.1X
0051 timestamp strings 3098 3109 10 3.2 309.8 0.8X
0052 parse timestamps from Dataset[String] 63331 63426 84 0.2 6333.1 0.0X
0053 infer timestamps from Dataset[String] 124003 124463 490 0.1 12400.3 0.0X
0054 date strings 3423 3429 11 2.9 342.3 0.7X
0055 parse dates from Dataset[String] 34235 34314 76 0.3 3423.5 0.1X
0056 from_csv(timestamp) 60829 61600 668 0.2 6082.9 0.0X
0057 from_csv(date) 33047 33173 139 0.3 3304.7 0.1X
0058
0059 OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
0060 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0061 Filters pushdown: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0062 ------------------------------------------------------------------------------------------------------------------------
0063 w/o filters 28752 28765 16 0.0 287516.5 1.0X
0064 pushdown disabled 28856 28880 22 0.0 288556.3 1.0X
0065 w/ filters 1714 1731 15 0.1 17137.3 16.8X
0066
0067
