Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: axiomhq/hyperloglog
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.2.3
Choose a base ref
...
head repository: axiomhq/hyperloglog
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v0.2.4
Choose a head ref
  • 10 commits
  • 4 files changed
  • 3 contributors

Commits on Jan 10, 2025

  1. chore: re-organize comment in docs-friendly way

    @HurSungYun
    HurSungYun committed Jan 10, 2025
    Configuration menu
    Copy the full SHA
    72727e9 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2025

  1. Merge pull request #48 from HurSungYun/docs_friendly_comment 

    chore: reorganize comments in docs-friendly way
    @lukasmalkmus
    lukasmalkmus authored Jan 19, 2025
    Configuration menu
    Copy the full SHA
    771e45e View commit details
    Browse the repository at this point in the history

Commits on Feb 11, 2025

  1. speed up genData 

    The `genData` function used in benchmarks now generates one big slice of
random bytes at once and breaks it up into chunks of 8-byte slices. This
speeds up `genData` significantly by eliminating the overhead of `num`
invocations of `crypto/rand.Read` and `num` allocations of 8-byte
slices. On a M1 Pro a single run of `Benchmark_Add_100000000` previously
took ~13 seconds and it now takes ~3 seconds.
    @mgartner
    mgartner committed Feb 11, 2025
    Configuration menu
    Copy the full SHA
    d3a13c7 View commit details
    Browse the repository at this point in the history

  2. simplify and speed up (*set).add 

    The `(*set).add` method has been simplified to no longer call its
internal set's `Has` method. This speeds up insertions for sketches with
few values.

```
name               old time/op    new time/op    delta
_Add_100-10          1.26µs ± 1%    1.00µs ± 0%  -20.64%  (p=0.000 n=9+10)
_Add_1000-10          123µs ± 0%     120µs ± 0%   -2.45%  (p=0.000 n=9+9)
_Add_10000-10        2.57ms ± 0%    2.50ms ± 0%   -2.62%  (p=0.000 n=10+10)
_Add_100000-10        629µs ± 0%     630µs ± 0%   +0.05%  (p=0.019 n=9+9)
_Add_1000000-10      6.29ms ± 0%    6.29ms ± 0%     ~     (p=0.167 n=8+9)
_Add_10000000-10     63.0ms ± 0%    63.0ms ± 0%     ~     (p=0.113 n=10+9)
_Add_100000000-10     630ms ± 0%     629ms ± 0%   -0.08%  (p=0.011 n=9+9)

name               old alloc/op   new alloc/op   delta
_Add_100-10           0.00B          0.00B          ~     (all equal)
_Add_1000-10         38.6kB ± 0%    38.6kB ± 0%     ~     (all equal)
_Add_10000-10         666kB ± 0%     666kB ± 0%   +0.00%  (p=0.002 n=10+9)
_Add_100000-10       1.92kB ± 0%    1.91kB ± 1%   -0.68%  (p=0.006 n=9+10)
_Add_1000000-10      16.5kB ± 0%    16.5kB ± 0%     ~     (p=0.059 n=9+8)
_Add_10000000-10      171kB ± 0%     171kB ± 0%     ~     (all equal)
_Add_100000000-10    1.54MB ± 0%    1.54MB ± 0%     ~     (all equal)

name               old allocs/op  new allocs/op  delta
_Add_100-10            0.00           0.00          ~     (all equal)
_Add_1000-10           24.0 ± 0%      24.0 ± 0%     ~     (all equal)
_Add_10000-10           243 ± 0%       243 ± 0%     ~     (all equal)
_Add_100000-10         0.00           0.00          ~     (all equal)
_Add_1000000-10        4.00 ± 0%      4.00 ± 0%     ~     (all equal)
_Add_10000000-10       46.0 ± 0%      46.0 ± 0%     ~     (all equal)
_Add_100000000-10       416 ± 0%       416 ± 0%     ~     (all equal)
```
    @mgartner
    mgartner committed Feb 11, 2025
    Configuration menu
    Copy the full SHA
    83f039e View commit details
    Browse the repository at this point in the history

  3. replace sort.Sort with slices.Sort 

    Replacing `sort.Sort` with `slices.Sort` speeds up insertion in some
cases.

```
name               old time/op    new time/op    delta
_Add_100-10          1.02µs ± 0%    1.03µs ± 0%   +0.95%  (p=0.000 n=9+9)
_Add_1000-10          119µs ± 0%      95µs ± 0%  -20.23%  (p=0.000 n=10+10)
_Add_10000-10        2.50ms ± 0%    2.26ms ± 0%   -9.50%  (p=0.000 n=9+9)
_Add_100000-10        630µs ± 0%     629µs ± 0%   -0.20%  (p=0.000 n=10+9)
_Add_1000000-10      6.30ms ± 0%    6.29ms ± 0%   -0.16%  (p=0.006 n=9+9)
_Add_10000000-10     63.1ms ± 1%    62.9ms ± 0%   -0.31%  (p=0.010 n=10+9)
_Add_100000000-10     629ms ± 0%     630ms ± 1%     ~     (p=0.173 n=8+10)

name               old alloc/op   new alloc/op   delta
_Add_100-10           0.00B          0.00B          ~     (all equal)
_Add_1000-10         38.6kB ± 0%    38.5kB ± 0%   -0.09%  (p=0.000 n=9+8)
_Add_10000-10         666kB ± 0%     666kB ± 0%   -0.05%  (p=0.000 n=10+10)
_Add_100000-10       1.91kB ± 1%    1.87kB ± 0%   -1.80%  (p=0.000 n=10+10)
_Add_1000000-10      16.5kB ± 0%    16.5kB ± 0%   -0.20%  (p=0.000 n=10+10)
_Add_10000000-10      171kB ± 0%     171kB ± 0%   -0.04%  (p=0.000 n=10+10)
_Add_100000000-10    1.54MB ± 0%    1.54MB ± 0%   -0.04%  (p=0.000 n=10+10)

name               old allocs/op  new allocs/op  delta
_Add_100-10            0.00           0.00          ~     (all equal)
_Add_1000-10           24.0 ± 0%      22.0 ± 0%   -8.33%  (p=0.000 n=10+10)
_Add_10000-10           243 ± 0%       228 ± 0%   -6.17%  (p=0.000 n=10+10)
_Add_100000-10         0.00           0.00          ~     (all equal)
_Add_1000000-10        4.00 ± 0%      4.00 ± 0%     ~     (all equal)
_Add_10000000-10       46.0 ± 0%      43.0 ± 0%   -6.52%  (p=0.000 n=10+10)
_Add_100000000-10       416 ± 0%       390 ± 0%   -6.25%  (p=0.000 n=10+10)
```
    @mgartner
    mgartner committed Feb 11, 2025
    Configuration menu
    Copy the full SHA
    f7de2fd View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2025

  1. avoid re-decoding after calling (*iterator).Peek 

    Both `(*iterator).Peek` and (*iterator).Next` decode the next element of
the iterator. `mergeSparse` now saves the results of decoding from
`Peek` and calls a new iterator method, `Advance`, instead of `Next` to
avoid re-decoding the same element. This speeds up insertion into
sketches with ~1-10k items.

```
name               old time/op    new time/op    delta
_Add_100-10          1.03µs ± 0%    1.06µs ± 0%   +3.34%  (p=0.000 n=9+10)
_Add_1000-10         95.2µs ± 0%    90.9µs ± 0%   -4.51%  (p=0.000 n=10+10)
_Add_10000-10        2.26ms ± 0%    1.93ms ± 0%  -14.57%  (p=0.000 n=9+10)
_Add_100000-10        629µs ± 0%     629µs ± 1%     ~     (p=0.050 n=9+9)
_Add_1000000-10      6.29ms ± 0%    6.29ms ± 0%     ~     (p=0.065 n=9+10)
_Add_10000000-10     62.9ms ± 0%    63.0ms ± 1%     ~     (p=0.905 n=9+10)
_Add_100000000-10     630ms ± 1%     632ms ± 1%     ~     (p=0.497 n=10+9)

name               old alloc/op   new alloc/op   delta
_Add_100-10           0.00B          0.00B          ~     (all equal)
_Add_1000-10         38.5kB ± 0%    38.5kB ± 0%     ~     (all equal)
_Add_10000-10         666kB ± 0%     666kB ± 0%   +0.01%  (p=0.000 n=10+10)
_Add_100000-10       1.87kB ± 0%    1.83kB ± 1%   -2.44%  (p=0.000 n=10+10)
_Add_1000000-10      16.5kB ± 0%    16.4kB ± 0%   -0.53%  (p=0.000 n=10+10)
_Add_10000000-10      171kB ± 0%     177kB ± 8%     ~     (p=0.211 n=10+10)
_Add_100000000-10    1.54MB ± 0%    1.54MB ± 0%     ~     (all equal)

name               old allocs/op  new allocs/op  delta
_Add_100-10            0.00           0.00          ~     (all equal)
_Add_1000-10           22.0 ± 0%      22.0 ± 0%     ~     (all equal)
_Add_10000-10           228 ± 0%       228 ± 0%     ~     (all equal)
_Add_100000-10         0.00           0.00          ~     (all equal)
_Add_1000000-10        4.00 ± 0%      4.00 ± 0%     ~     (all equal)
_Add_10000000-10       43.0 ± 0%      44.5 ± 8%     ~     (p=0.211 n=10+10)
_Add_100000000-10       390 ± 0%       390 ± 0%     ~     (all equal)
```
    @mgartner
    mgartner committed Feb 12, 2025
    Configuration menu
    Copy the full SHA
    7c218ad View commit details
    Browse the repository at this point in the history

  2. remove iterable interface 

    The `iterable` interface was unnecessary so it has been removed.

```
name               old time/op    new time/op    delta
_Add_100-10          1.06µs ± 0%    1.02µs ± 0%  -4.32%  (p=0.000 n=10+8)
_Add_1000-10         90.9µs ± 0%    90.6µs ± 0%  -0.35%  (p=0.004 n=10+9)
_Add_10000-10        1.93ms ± 0%    1.77ms ± 0%  -8.18%  (p=0.000 n=10+9)
_Add_100000-10        629µs ± 1%     627µs ± 0%  -0.37%  (p=0.000 n=9+9)
_Add_1000000-10      6.29ms ± 0%    6.27ms ± 0%  -0.17%  (p=0.002 n=10+8)
_Add_10000000-10     63.0ms ± 1%    62.8ms ± 0%  -0.28%  (p=0.004 n=10+9)
_Add_100000000-10     632ms ± 1%     628ms ± 0%  -0.63%  (p=0.001 n=9+9)

name               old alloc/op   new alloc/op   delta
_Add_100-10           0.00B          0.00B         ~     (all equal)
_Add_1000-10         38.5kB ± 0%    38.5kB ± 0%    ~     (all equal)
_Add_10000-10         666kB ± 0%     666kB ± 0%  +0.00%  (p=0.005 n=10+7)
_Add_100000-10       1.83kB ± 1%    1.82kB ± 1%  -0.47%  (p=0.012 n=10+10)
_Add_1000000-10      16.4kB ± 0%    16.3kB ± 0%  -0.32%  (p=0.011 n=10+10)
_Add_10000000-10      177kB ± 8%     192kB ± 0%  +8.43%  (p=0.003 n=10+10)
_Add_100000000-10    1.54MB ± 0%    1.54MB ± 0%    ~     (all equal)

name               old allocs/op  new allocs/op  delta
_Add_100-10            0.00           0.00         ~     (all equal)
_Add_1000-10           22.0 ± 0%      22.0 ± 0%    ~     (all equal)
_Add_10000-10           228 ± 0%       228 ± 0%    ~     (all equal)
_Add_100000-10         0.00           0.00         ~     (all equal)
_Add_1000000-10        4.00 ± 0%      4.00 ± 0%    ~     (all equal)
_Add_10000000-10       44.5 ± 8%      48.0 ± 0%  +7.87%  (p=0.003 n=10+10)
_Add_100000000-10       390 ± 0%       390 ± 0%    ~     (all equal)
```
    @mgartner
    mgartner committed Feb 12, 2025
    Configuration menu
    Copy the full SHA
    a0dcd48 View commit details
    Browse the repository at this point in the history

  3. reduce set allocations 

    The extra indirection incurred by using `*set` has been eliminated by
using `set`.

```
name               old time/op    new time/op    delta
_Add_100-10          1.02µs ± 0%    1.08µs ± 0%   +6.23%  (p=0.000 n=8+10)
_Add_1000-10         90.6µs ± 0%    90.0µs ± 0%   -0.71%  (p=0.000 n=9+9)
_Add_10000-10        1.77ms ± 0%    1.76ms ± 0%   -0.40%  (p=0.000 n=9+9)
_Add_100000-10        627µs ± 0%     627µs ± 0%     ~     (p=0.211 n=9+10)
_Add_1000000-10      6.27ms ± 0%    6.27ms ± 0%     ~     (p=0.236 n=8+9)
_Add_10000000-10     62.8ms ± 0%    62.9ms ± 0%     ~     (p=0.340 n=9+9)
_Add_100000000-10     628ms ± 0%     628ms ± 0%     ~     (p=0.605 n=9+9)

name               old alloc/op   new alloc/op   delta
_Add_100-10           0.00B          0.00B          ~     (all equal)
_Add_1000-10         38.5kB ± 0%    38.5kB ± 0%   -0.03%  (p=0.000 n=9+9)
_Add_10000-10         666kB ± 0%     666kB ± 0%   -0.02%  (p=0.000 n=7+10)
_Add_100000-10       1.82kB ± 1%    1.80kB ± 0%   -1.03%  (p=0.000 n=10+10)
_Add_1000000-10      16.3kB ± 0%    16.3kB ± 0%   -0.05%  (p=0.023 n=10+9)
_Add_10000000-10      192kB ± 0%     192kB ± 0%   -0.01%  (p=0.000 n=10+10)
_Add_100000000-10    1.54MB ± 0%    1.54MB ± 0%   -0.01%  (p=0.000 n=10+10)

name               old allocs/op  new allocs/op  delta
_Add_100-10            0.00           0.00          ~     (all equal)
_Add_1000-10           22.0 ± 0%      21.0 ± 0%   -4.55%  (p=0.000 n=10+10)
_Add_10000-10           228 ± 0%       213 ± 0%   -6.58%  (p=0.000 n=10+10)
_Add_100000-10         0.00           0.00          ~     (all equal)
_Add_1000000-10        4.00 ± 0%      3.00 ± 0%  -25.00%  (p=0.000 n=10+9)
_Add_10000000-10       48.0 ± 0%      45.0 ± 0%   -6.25%  (p=0.000 n=10+10)
_Add_100000000-10       390 ± 0%       364 ± 0%   -6.67%  (p=0.000 n=10+10)
```
    @mgartner
    mgartner committed Feb 12, 2025
    Configuration menu
    Copy the full SHA
    6fef577 View commit details
    Browse the repository at this point in the history

  4. remove unused parameter in variableLengthList.decode

    @mgartner
    mgartner committed Feb 12, 2025
    Configuration menu
    Copy the full SHA
    bc95329 View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2025

  1. Merge pull request #50 from mgartner/opt 

    assorted optimizations
    @lukasmalkmus
    lukasmalkmus authored Feb 28, 2025
    Configuration menu
    Copy the full SHA
    ac34a16 View commit details
    Browse the repository at this point in the history
Loading