Add a benchmark workflow for running sift/gist and more

[yuzhoumao]

What problem does this PR solve?

Currently the tests workflow runs the sift benchmark during CI and after commit. This commit creates a benchmark workflow that allows more diverse benchmarks (e.g. gist and many others) to be run automatically (e.g. once per day) that will help catch performance changes early.

For now, a new workflow is created. In future this workflow can be combined into the slow_tests workflow, once the failures in slow_tests are resolved.

Issue link: #951

Type of change

• Test cases

[JinHai-CN]

This is the directory structure of benchmark dataset, right now.

infiniflow@inf128:~/runners_work/benchmark$ tree
.
├── db_tmp
│   ├── benchmark
│   │   └── sift_1m -> /home/infiniflow/runners_work/benchmark/sift1M
│   └── infinity_conf.toml
├── enwiki
│   └── enwiki-10w.csv
├── golden_benchmark_sift_1_thread.log
├── golden_benchmark_sift_8_threads.log
├── infinity_conf.toml
├── knn_import_benchmark
├── knn_query_benchmark
└── sift1M
    ├── query.fvecs
    ├── sift_base.fvecs
    ├── sift_groundtruth.ivecs
    ├── sift_learn.fvecs
    └── sift_query.fvecs

We have the gist dataset and please let me know which directory name you want.

[yuzhoumao]

Thanks! Shall we follow the same pattern for sift, and make the tree look like this?

infiniflow@inf128:~/runners_work/benchmark$ tree
.
├── db_tmp
│   ├── benchmark
│   │   └── sift_1m -> /home/infiniflow/runners_work/benchmark/sift1M
│   └── infinity_conf.toml
├── enwiki
│   └── enwiki-10w.csv
├── golden_benchmark_sift_1_thread.log
├── golden_benchmark_sift_8_threads.log
├── infinity_conf.toml
├── knn_import_benchmark
├── knn_query_benchmark
└── sift1M
    ├── query.fvecs
    ├── sift_base.fvecs
    ├── sift_groundtruth.ivecs
    ├── sift_learn.fvecs
    └── sift_query.fvecs
└── gist1M                                           <- directory to add
    ├── query.fvecs
    ├── gist_base.fvecs
    ├── gist_groundtruth.ivecs
    ├── gist_learn.fvecs
    └── gist_query.fvecs

[JinHai-CN]

Done.

infiniflow@inf128:~/runners_work$ tree benchmark/
benchmark/
├── db_tmp
│   ├── benchmark
│   │   └── sift_1m -> /home/infiniflow/runners_work/benchmark/sift1M
│   └── infinity_conf.toml
├── enwiki
│   └── enwiki-10w.csv
├── gist1M
│   ├── gist_base.fvecs
│   ├── gist_groundtruth.ivecs
│   ├── gist_query.fvecs
│   └── ip_groundtruth.ivecs
├── golden_benchmark_sift_1_thread.log
├── golden_benchmark_sift_8_threads.log
├── infinity_conf.toml
├── knn_import_benchmark
├── knn_query_benchmark
└── sift1M
    ├── query.fvecs
    ├── sift_base.fvecs
    ├── sift_groundtruth.ivecs
    ├── sift_learn.fvecs
    └── sift_query.fvecs