0001 ================================================================================================
0002 Dataset Benchmark
0003 ================================================================================================
0004
0005 OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64
0006 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0007 back-to-back map long: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0008 ------------------------------------------------------------------------------------------------------------------------
0009 RDD 12720 12777 80 7.9 127.2 1.0X
0010 DataFrame 2242 2501 366 44.6 22.4 5.7X
0011 Dataset 3040 3174 189 32.9 30.4 4.2X
0012
0013 OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64
0014 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0015 back-to-back map: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0016 ------------------------------------------------------------------------------------------------------------------------
0017 RDD 15865 15922 82 6.3 158.6 1.0X
0018 DataFrame 8423 8476 75 11.9 84.2 1.9X
0019 Dataset 17180 18142 1361 5.8 171.8 0.9X
0020
0021 OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64
0022 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0023 back-to-back filter Long: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0024 ------------------------------------------------------------------------------------------------------------------------
0025 RDD 2928 3009 114 34.1 29.3 1.0X
0026 DataFrame 1386 1427 59 72.2 13.9 2.1X
0027 Dataset 3448 3451 5 29.0 34.5 0.8X
0028
0029 OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64
0030 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0031 back-to-back filter: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0032 ------------------------------------------------------------------------------------------------------------------------
0033 RDD 5476 5483 10 18.3 54.8 1.0X
0034 DataFrame 209 235 23 479.1 2.1 26.2X
0035 Dataset 9433 9549 163 10.6 94.3 0.6X
0036
0037 OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64
0038 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
0039 aggregate: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
0040 ------------------------------------------------------------------------------------------------------------------------
0041 RDD sum 5146 5239 132 19.4 51.5 1.0X
0042 DataFrame sum 84 99 15 1196.9 0.8 61.6X
0043 Dataset sum using Aggregator 8944 9021 109 11.2 89.4 0.6X
0044 Dataset complex Aggregator 12832 13141 436 7.8 128.3 0.4X
0045
0046
