Skip to content

Optimize slice appending by using slice expansion - #53

Merged
lukasmalkmus merged 3 commits into
axiomhq:mainfrom
HurSungYun:tiny_optimizations
Mar 13, 2025
Merged

Optimize slice appending by using slice expansion#53
lukasmalkmus merged 3 commits into
axiomhq:mainfrom
HurSungYun:tiny_optimizations

Conversation

@HurSungYun

@HurSungYun HurSungYun commented Mar 12, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

Optimize slice appending by using slice expansion (append(data, v...)) instead of a loop.

The benchmark results show a minor improvement in execution time for sparse sketches.

goos: darwin
goarch: arm64
pkg: github.com/axiomhq/hyperloglog
cpu: Apple M3
                                                    │   old.txt    │               new.txt                │
                                                    │    sec/op    │    sec/op     vs base                │
_MarshalBinary/Precision_12/Size_100/Sparse-8         350.0n ± 19%   250.0n ± 60%  -28.57% (p=0.014 n=10)
_MarshalBinary/Precision_12/Size_100/NoSparse-8       1.534µ ±  9%   1.575µ ±  6%        ~ (p=0.541 n=10)
_MarshalBinary/Precision_12/Size_10000/Sparse-8       1.554µ ±  6%   1.542µ ±  4%        ~ (p=0.323 n=10)
_MarshalBinary/Precision_12/Size_10000/NoSparse-8     1.525µ ±  3%   1.512µ ± 14%        ~ (p=0.491 n=10)
_MarshalBinary/Precision_12/Size_1000000/Sparse-8     1.533µ ±  3%   1.508µ ±  1%   -1.63% (p=0.029 n=10)
_MarshalBinary/Precision_12/Size_1000000/NoSparse-8   1.525µ ±  2%   1.529µ ± 17%        ~ (p=0.616 n=10)
_MarshalBinary/Precision_14/Size_100/Sparse-8         579.2n ± 15%   654.1n ± 18%        ~ (p=0.122 n=10)
_MarshalBinary/Precision_14/Size_100/NoSparse-8       5.466µ ±  6%   5.484µ ±  4%        ~ (p=0.643 n=10)
_MarshalBinary/Precision_14/Size_10000/Sparse-8       5.904µ ±  8%   5.821µ ±  2%        ~ (p=0.541 n=10)
_MarshalBinary/Precision_14/Size_10000/NoSparse-8     5.980µ ±  4%   5.667µ ±  2%   -5.23% (p=0.002 n=10)
_MarshalBinary/Precision_14/Size_1000000/Sparse-8     5.863µ ±  8%   5.804µ ±  5%        ~ (p=0.362 n=10)
_MarshalBinary/Precision_14/Size_1000000/NoSparse-8   5.854µ ±  4%   5.792µ ±  7%        ~ (p=0.927 n=10)
_MarshalBinary/Precision_16/Size_100/Sparse-8         608.3n ± 58%   604.1n ± 23%        ~ (p=0.698 n=10)
_MarshalBinary/Precision_16/Size_100/NoSparse-8       21.12µ ±  3%   21.33µ ±  4%        ~ (p=0.529 n=10)
_MarshalBinary/Precision_16/Size_10000/Sparse-8       7.900µ ±  3%   1.296µ ± 65%  -83.60% (p=0.000 n=10)
_MarshalBinary/Precision_16/Size_10000/NoSparse-8     20.70µ ±  1%   20.37µ ±  2%        ~ (p=0.072 n=10)
_MarshalBinary/Precision_16/Size_1000000/Sparse-8     20.66µ ±  3%   20.38µ ±  4%   -1.33% (p=0.045 n=10)
_MarshalBinary/Precision_16/Size_1000000/NoSparse-8   20.75µ ±  2%   20.34µ ±  1%   -1.99% (p=0.015 n=10)
geomean                                               3.599µ         3.192µ        -11.30%
@HurSungYun HurSungYun changed the title Optimize slice appending by using variadic syntax Mar 12, 2025
@lukasmalkmus
lukasmalkmus requested a review from seiflotfy March 13, 2025 10:26
@lukasmalkmus

lukasmalkmus commented Mar 13, 2025

Copy link
Copy Markdown
Contributor

Looks good! Here is another set of benchmarks I ran locally, as well:

goos: darwin
goarch: arm64
pkg: github.com/axiomhq/hyperloglog
cpu: Apple M1 Pro
                                                     │   old.out   │               new.out               │
                                                     │   sec/op    │   sec/op     vs base                │
_MarshalBinary/Precision_12/Size_100/Sparse-10         282.6n ± 0%   195.4n ± 3%  -30.85% (p=0.000 n=20)
_MarshalBinary/Precision_12/Size_100/NoSparse-10       1.789µ ± 0%   1.794µ ± 2%        ~ (p=0.101 n=20)
_MarshalBinary/Precision_12/Size_10000/Sparse-10       1.787µ ± 0%   1.812µ ± 5%   +1.40% (p=0.010 n=20)
_MarshalBinary/Precision_12/Size_10000/NoSparse-10     1.807µ ± 4%   1.809µ ± 7%        ~ (p=0.344 n=20)
_MarshalBinary/Precision_12/Size_1000000/Sparse-10     1.854µ ± 3%   1.810µ ± 1%   -2.40% (p=0.004 n=20)
_MarshalBinary/Precision_12/Size_1000000/NoSparse-10   1.820µ ± 3%   1.830µ ± 2%        ~ (p=0.743 n=20)
_MarshalBinary/Precision_14/Size_100/Sparse-10         560.4n ± 2%   579.0n ± 3%   +3.32% (p=0.025 n=20)
_MarshalBinary/Precision_14/Size_100/NoSparse-10       6.945µ ± 2%   6.877µ ± 1%   -0.97% (p=0.021 n=20)
_MarshalBinary/Precision_14/Size_10000/Sparse-10       7.029µ ± 1%   7.263µ ± 3%   +3.33% (p=0.026 n=20)
_MarshalBinary/Precision_14/Size_10000/NoSparse-10     6.992µ ± 1%   7.019µ ± 4%        ~ (p=0.794 n=20)
_MarshalBinary/Precision_14/Size_1000000/Sparse-10     6.968µ ± 1%   7.067µ ± 1%   +1.43% (p=0.007 n=20)
_MarshalBinary/Precision_14/Size_1000000/NoSparse-10   6.897µ ± 1%   6.897µ ± 1%        ~ (p=0.846 n=20)
_MarshalBinary/Precision_16/Size_100/Sparse-10         561.4n ± 0%   570.3n ± 5%   +1.59% (p=0.033 n=20)
_MarshalBinary/Precision_16/Size_100/NoSparse-10       27.87µ ± 0%   28.27µ ± 2%   +1.45% (p=0.024 n=20)
_MarshalBinary/Precision_16/Size_10000/Sparse-10       9.419µ ± 2%   2.547µ ± 1%  -72.96% (p=0.000 n=20)
_MarshalBinary/Precision_16/Size_10000/NoSparse-10     27.52µ ± 0%   27.53µ ± 0%        ~ (p=0.984 n=20)
_MarshalBinary/Precision_16/Size_1000000/Sparse-10     27.13µ ± 1%   27.17µ ± 0%        ~ (p=0.588 n=20)
_MarshalBinary/Precision_16/Size_1000000/NoSparse-10   28.79µ ± 4%   27.83µ ± 3%   -3.32% (p=0.005 n=20)
geomean                                                4.193µ        3.835µ        -8.53%

                                                     │   old.out    │                new.out                │
                                                     │     B/op     │     B/op      vs base                 │
_MarshalBinary/Precision_12/Size_100/Sparse-10           600.0 ± 0%     600.0 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_12/Size_100/NoSparse-10       4.750Ki ± 0%   4.750Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_12/Size_10000/Sparse-10       4.750Ki ± 0%   4.750Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_12/Size_10000/NoSparse-10     4.750Ki ± 0%   4.750Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_12/Size_1000000/Sparse-10     4.750Ki ± 0%   4.750Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_12/Size_1000000/NoSparse-10   4.750Ki ± 0%   4.750Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_14/Size_100/Sparse-10         1.164Ki ± 0%   1.164Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_14/Size_100/NoSparse-10       18.00Ki ± 0%   18.00Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_14/Size_10000/Sparse-10       18.00Ki ± 0%   18.00Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_14/Size_10000/NoSparse-10     18.00Ki ± 0%   18.00Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_14/Size_1000000/Sparse-10     18.00Ki ± 0%   18.00Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_14/Size_1000000/NoSparse-10   18.00Ki ± 0%   18.00Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_16/Size_100/Sparse-10         1.164Ki ± 0%   1.164Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_16/Size_100/NoSparse-10       72.00Ki ± 0%   72.00Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_16/Size_10000/Sparse-10       21.95Ki ± 0%   21.95Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_16/Size_10000/NoSparse-10     72.00Ki ± 0%   72.00Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_16/Size_1000000/Sparse-10     72.00Ki ± 0%   72.00Ki ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_16/Size_1000000/NoSparse-10   72.00Ki ± 0%   72.00Ki ± 0%       ~ (p=1.000 n=20) ¹
geomean                                                10.43Ki        10.43Ki       +0.00%
¹ all samples are equal

                                                     │  old.out   │               new.out               │
                                                     │ allocs/op  │ allocs/op   vs base                 │
_MarshalBinary/Precision_12/Size_100/Sparse-10         4.000 ± 0%   4.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_12/Size_100/NoSparse-10       1.000 ± 0%   1.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_12/Size_10000/Sparse-10       1.000 ± 0%   1.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_12/Size_10000/NoSparse-10     1.000 ± 0%   1.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_12/Size_1000000/Sparse-10     1.000 ± 0%   1.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_12/Size_1000000/NoSparse-10   1.000 ± 0%   1.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_14/Size_100/Sparse-10         3.000 ± 0%   3.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_14/Size_100/NoSparse-10       1.000 ± 0%   1.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_14/Size_10000/Sparse-10       1.000 ± 0%   1.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_14/Size_10000/NoSparse-10     1.000 ± 0%   1.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_14/Size_1000000/Sparse-10     1.000 ± 0%   1.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_14/Size_1000000/NoSparse-10   1.000 ± 0%   1.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_16/Size_100/Sparse-10         3.000 ± 0%   3.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_16/Size_100/NoSparse-10       1.000 ± 0%   1.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_16/Size_10000/Sparse-10       3.000 ± 0%   3.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_16/Size_10000/NoSparse-10     1.000 ± 0%   1.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_16/Size_1000000/Sparse-10     1.000 ± 0%   1.000 ± 0%       ~ (p=1.000 n=20) ¹
_MarshalBinary/Precision_16/Size_1000000/NoSparse-10   1.000 ± 0%   1.000 ± 0%       ~ (p=1.000 n=20) ¹
geomean                                                1.297        1.297       +0.00%
¹ all samples are equal
@lukasmalkmus
lukasmalkmus merged commit b7cebb6 into axiomhq:main Mar 13, 2025
@HurSungYun
HurSungYun deleted the tiny_optimizations branch March 14, 2025 07:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants

@HurSungYun @lukasmalkmus @seiflotfy