perf(stream): improve file stream range reading and caching mechanism

[j2rong4cn]

完善配置选项max_buffer_limitMB的使用场景

配置选项 max_buffer_limitMB

• 限制单个buffer的大小
  自动： -1 (默认)：5%的总内存
  关闭：0
  自定义：大于0 单位为MB
• 用于上传、分片下载、解压缩缓存

close #787
close #1039

[Copilot]

Pull Request Overview

This PR improves the file stream range reading and caching mechanism by refactoring the buffer management system and enhancing the max_buffer_limitMB configuration option usage. The changes focus on streamlining caching operations, improving memory management, and providing better progress tracking for file operations.

• Refactors CacheFullInTempFile to CacheFullAndWriter with enhanced functionality and progress tracking
• Introduces a new Bytes buffer type for efficient non-reusable byte slice storage
• Updates method signatures across drivers to use pointer-based progress tracking and renamed section reader methods

Reviewed Changes

Copilot reviewed 45 out of 45 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
pkg/errgroup/errgroup.go	Minor comment updates for clarity
pkg/buffer/bytes.go	New buffer implementation for managing multiple byte slices
pkg/buffer/bytes_test.go	Comprehensive tests for the new Bytes buffer functionality
internal/stream/stream.go	Major refactoring of FileStream with new caching logic and buffer management
internal/stream/util.go	Updated utility functions and renamed RecycleSectionReader to FreeSectionReader
internal/stream/stream_test.go	Tests for the refactored stream functionality
internal/model/obj.go	Updated FileStreamer interface with new method signatures
internal/conf/var.go	Set default MaxBufferLimit value
internal/bootstrap/config.go	Improved logging format for buffer limit
drivers/*	Updated all drivers to use new CacheFullAndWriter method and pointer-based progress tracking

[bitxeno]

@j2rong4cn 大佬，有部分网盘会获取preHash，如115和阿里，但获取preHash后，再调用CacheFullAndHash返回的hash值是错的，能不能修复下，下面这个测试能重现：

func TestFileStream_With_PreHash(t *testing.T) {
	conf.MaxBufferLimit = 16 * 1024 * 1024
	type args struct {
		httpRange http_range.Range
	}
	buf := []byte("github.com/OpenListTeam/OpenList")
	f := &FileStream{
		Obj: &model.Object{
			Size: int64(len(buf)),
		},
		Reader: io.NopCloser(bytes.NewReader(buf)),
	}

	const hashSize int64 = 20
	reader, _ := f.RangeRead(http_range.Range{Start: 0, Length: hashSize})
	preHash, _ := utils.HashReader(utils.SHA1, reader)
	if preHash == "" {
		t.Error("preHash is empty")
	}
	tmpF, fullHash, _ := CacheFullAndHash(f, nil, utils.SHA1)
	fmt.Println(fullHash)
	fileFullHash, _ := utils.HashFile(utils.SHA1, tmpF)
	fmt.Println(fileFullHash)
	if fullHash != fileFullHash {
		t.Errorf("fullHash and fileFullHash should match: fullHash=%s fileFullHash=%s", fullHash, fileFullHash)
	}
}

去掉前面的preHash调用，才能获取到正确的hash值

[j2rong4cn]

@bitxeno 发现这么隐蔽的bug，真的帮大忙了，感谢