Skip to content

chore: Add alternative int64 encoder to dataobj #18202

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

benclive
Copy link
Contributor

What this PR does / why we need it:
Adds an alternative implementation encoding for int64s in dataobjects.

I think the faster decoding speed is really beneficial to query perf, the other improvements are just bonuses. I haven't hooked this up in any builders yet, but I can do that afterwards if we want to use this.

Raw benchmark results:

goos: darwin
goarch: arm64
pkg: github.com/grafana/loki/v3/pkg/dataobj/internal/dataset
cpu: Apple M3 Max
Benchmark_intCompEncoder_Encode/Sequential-14         	277998073	         4.297 ns/op	         1.862 GB/s	       0 B/op	       0 allocs/op
Benchmark_intCompEncoder_Encode/Largest_delta-14      	233474617	         5.122 ns/op	         1.562 GB/s	       0 B/op	       0 allocs/op
Benchmark_intCompEncoder_Encode/Random-14             	192020827	         6.072 ns/op	         1.317 GB/s	       0 B/op	       0 allocs/op
Benchmark_deltaEncoder_Encode/Sequential-14         	222049738	         5.392 ns/op	         1.484 GB/s	       0 B/op	       0 allocs/op
Benchmark_deltaEncoder_Encode/Largest_delta-14      	87491991	        13.73 ns/op	         0.5825 GB/s	       0 B/op	       0 allocs/op
Benchmark_deltaEncoder_Encode/Random-14             	77057061	        15.66 ns/op	         0.5108 GB/s	       0 B/op	       0 allocs/op
PASS
ok  	github.com/grafana/loki/v3/pkg/dataobj/internal/dataset	4.382s

goos: darwin
goarch: arm64
pkg: github.com/grafana/loki/v3/pkg/dataobj/internal/dataset
cpu: Apple M3 Max
Benchmark_intCompDecoder_Decode/Sequential-14         	705083431	         1.682 ns/op	         4.758 GB/s	         0.2539 bytes/value	       0 B/op	       0 allocs/op
Benchmark_intCompDecoder_Decode/Largest_delta-14      	510498273	         2.337 ns/op	         3.423 GB/s	         8.133 bytes/value	       0 B/op	       0 allocs/op
Benchmark_intCompDecoder_Decode/Random-14             	516526146	         2.392 ns/op	         3.344 GB/s	         8.133 bytes/value	       0 B/op	       0 allocs/op
Benchmark_deltaDecoder_Decode/Sequential-14         	259016820	         4.577 ns/op	         1.748 GB/s	         1.000 bytes/value	       0 B/op	       0 allocs/op
Benchmark_deltaDecoder_Decode/Largest_delta-14      	66355470	        18.16 ns/op	         0.4405 GB/s	        10.00 bytes/value	       0 B/op	       0 allocs/op
Benchmark_deltaDecoder_Decode/Random-14             	64460536	        18.93 ns/op	         0.4226 GB/s	         9.242 bytes/value	       0 B/op	       0 allocs/op
PASS
ok  	github.com/grafana/loki/v3/pkg/dataobj/internal/dataset	9.529s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1 participant