Skip to content

Conversation

@philippgille
Copy link
Owner

@philippgille philippgille commented Mar 12, 2024

  • Normalized vectors
  • Only query similarity
  • No metadata or content filter

With the current state of the project, not performance optimized except some parallelization with goroutines, on my first generation Framework Laptop 13, on performance mode "Balanced":

$ go test -benchmem -run=^$ -bench .
goos: linux
goarch: amd64
pkg: github.com/philippgille/chromem-go
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
BenchmarkCollection_Query_NoContent_100-8      	    3082	    401136 ns/op	 1240100 B/op	     238 allocs/op
BenchmarkCollection_Query_NoContent_1000-8     	     435	   2735553 ns/op	12372113 B/op	    2038 allocs/op
BenchmarkCollection_Query_NoContent_5000-8     	      79	  12975637 ns/op	61843650 B/op	   10039 allocs/op
BenchmarkCollection_Query_NoContent_25000-8    	      18	  65804944 ns/op	309209240 B/op	   50038 allocs/op
BenchmarkCollection_Query_NoContent_100000-8   	       4	 279518934 ns/op	1236805992 B/op	  200041 allocs/op
BenchmarkCollection_Query_100-8                	    2860	    414340 ns/op	 1240069 B/op	     238 allocs/op
BenchmarkCollection_Query_1000-8               	     444	   2770227 ns/op	12372085 B/op	    2038 allocs/op
BenchmarkCollection_Query_5000-8               	      79	  12908316 ns/op	61843567 B/op	   10038 allocs/op
BenchmarkCollection_Query_25000-8              	      15	  67794886 ns/op	309209185 B/op	   50038 allocs/op
BenchmarkCollection_Query_100000-8             	       4	 287507129 ns/op	1236805700 B/op	  200038 allocs/op
PASS
ok  	github.com/philippgille/chromem-go	31.496s

This is to establish a baseline, from which we can start optimizing while making sure nothing that one might assume is an optimization is actually making things worse. For example passing thousands of values through a channel can lead to a lot of data copying, and locking with a mutex can actually be faster. (Example from a recent One Billion Row Challenge (in Go) article)

- Normalized vectors
- Only query similarity
- No metadata or content filter
- More relevant for basic chromem-go usage
- String copying during querying might be very relevant for performance
@philippgille philippgille merged commit f59e9dc into main Mar 12, 2024
@philippgille philippgille deleted the add-query-benchmark branch March 12, 2024 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants