[Go] [Parquet] pqarrow file-writer & row-group-writer tracking total & total compressed bytes

[DuanWeiFan]

Rationale for this change

As discussed in issue, pqarrow parquet writer when writing more than 1 row group, it cannot track the bytesWritten & compressedbytesWritten for closed row group.
The method RowGroupTotalCompressedBytes & RowGroupTotalBytesWritten only returns the current row group.
The ideal is to introduce TotalCompressedBytes() & TotalBytesWritten() as an enhancement for user to track all row group written bytes & compressed bytes.

What changes are included in this PR?

As part of making this change, I notice the totalCompressedBytes is not being populated correctly in row_group_writer.go.
It should follow the same approach as bytesWritten where it tracks all the closed column_writer.

Are these changes tested?

Yes. I added test cases to ensure those metrics are populated correctly

Are there any user-facing changes?

Yes. There are two new methods introduced:

1. TotalCompressedBytes()
2. TotalBytesWritten()

It won't impact on existing users.

[zeroshade]

In FileWriter.Close when we write the metadata, we should probably add the size of the metadata we write to the total bytes written, yea?

[DuanWeiFan]

makes a lot of sense! I've added that to Close() so we correctly track the last row group's bytes written

[zeroshade]

I was referring to the Close method in file/file_writer.go specifically, inside of FlushWithFooter. Currently when we close the file, we'll cover all of the bytes except for the metadata footer which is written at the end of the file. Make sense?

[zeroshade]

Sorry for the delay here, I was travelling for a conference last week

[zeroshade]

Any updates @DuanWeiFan ?

[DuanWeiFan]

sorry for the delay I lost track of this.
I think if we are considering the metadata (header/footer..), instead of rowGroup level, I think we would need to move
total bytes written to file/file_writer.go so it has the complete picture.

[zeroshade]

All looks good now, thanks!