fix(parquet/pqarrow): unsupported dictionary types in pqarrow

[zeroshade]

Rationale for this change

fixes #490

What changes are included in this PR?

When using PutDictionary to write an arrow dictionary array to Parquet directly, we will now normalize the data type to a parquet physical storage type to avoid type problems. e.g. A dictionary with int8 values will get cast to int32 values before we process it so that all of the remaining encoding code can continue to be based on the physical storage type. This alleviates panics when using pqarrow to write Dictionary arrays.

Are these changes tested?

Yes, the unit tests are updated to include tests for int8/uint8/etc. as dictionary value types.

Are there any user-facing changes?

No, just fixing panics.

[lidavidm]

That's kind of annoying, right? You have to branch. Maybe it should always require releasing (normalize can call retain)?

[zeroshade]

fair enough, i'll update to that.

[amoeba]

Two tangential notes:

• Would it be good to title PRs like this fix(pqarrow): ... so users can skim changelogs for parquet changes?
• Does pqarrow smuggle Arrow schemas inside Parquet metadata so users can round-trip? This PR doesn't introduce any breakage of course but it strikes me that arrow -> parquet -> arrow won't round-trip well if the arrow input has to be cast like this.

[zeroshade]

Does pqarrow smuggle Arrow schemas inside Parquet metadata so users can round-trip?

If you set the StoreSchema option, then yes it does add the Arrow schema inside the parquet metadata.

This PR doesn't introduce any breakage of course but it strikes me that arrow -> parquet -> arrow won't round-trip well if the arrow input has to be cast like this.

This is the case no matter what type you pass (even if it's not dictionary encoded) because Parquet only has int32/int64 as physical storage types. In the case where the original Arrow schema is in the metadata, we properly cast on the way out so there isn't an issue.

[amoeba]

Thanks for the answers to my questions.