[common] Reject non-struct inner fields in variant shredding schema - #9518
[common] Reject non-struct inner fields in variant shredding schema#9518LuciferYang wants to merge 1 commit into
Conversation
buildVariantSchema's ROW branch cast every field of an object's typed_value to RowType without checking it, so a physical schema whose inner field is a scalar died on a bare ClassCastException instead of the invalid-schema error the method raises for every other malformed shape. Check the type and raise that error. The two checks removed alongside it were dead. `!(dataType instanceof RowType)` sits inside `case ROW:` of a switch on getTypeRoot(), and RowType is the only type with that root. The other check tested the outer field list: its emptiness half is already rejected at the top of the method, and its duplicate half cannot hold because RowType's constructor calls validateFields, which rejects duplicate field names outright. An empty inner struct stays legal, as before, and produces a zero-length objectSchema. Assisted-by: GLM-5.3
|
The I pulled the job log: of the 826 test classes it started, exactly one never reported a result, For the part that is mine, I re-ran every variant and shredding test locally on JDK 11 (Zulu 11.0.30), A re-run should clear the job. |
Purpose
close #9517
In a variant shredding schema, every field of an object's
typed_valueis itself a group ofvalue/typed_value.buildVariantSchemacast each inner field toRowTypewithout checking it, so a physical schema whose inner field is a scalar died on a bareClassCastExceptionnaming two class names, instead of theInvalid variant shredding schema: <rowType>error the method raises for every other malformed shape. That path now checks the type and raises the same error as the rest.Two checks in the same branch are removed because they cannot fire:
if (!(dataType instanceof RowType))sits insidecase ROW:of a switch onfield.type().getTypeRoot(), andRowTypeis the only type in paimon-api withDataTypeRoot.ROW.if (fields.isEmpty() || fields.stream().distinct().count() != fields.size())tests the outer field list, not the inner struct it appears to guard. Emptiness is already rejected at the top of the method; duplicates cannot exist becauseRowType's constructor callsvalidateFields, which throws on duplicate field names. Duplicatetyped_value/value/metadataare caught by thetypedIdx != -1style checks, and any other outer name falls into the throwingdefault:of the name switch.An empty inner struct stays legal, exactly as before: the removed check never guarded the inner list, so nothing changes there. It yields a zero-length
objectSchema, which is whatShreddingUtilsandVariantShreddingWriteralready consume by looping on.length.Tests
Both cases live in
PaimonShreddingUtilsTest.testBuildVariantSchemaRejectsNonStructInnerFieldbuilds a schema that is valid except for a scalar inner field and asserts the message is the invalid-schema one. Verified red before the change, where it fails withClassCastException: org.apache.paimon.types.IntType cannot be cast to org.apache.paimon.types.RowType. The message is what separates the two outcomes, sinceClassCastExceptionis itself aRuntimeException.testBuildVariantSchemaAcceptsEmptyInnerStructpins that an emptytyped_valuestruct produces a zero-lengthobjectSchemarather than null, which is the shape the two consumers above depend on. This one passes before the change as well; it is there to hold that behavior still.mvn -pl paimon-common teston JDK 8: 12467 tests, 0 failures, 0 errors. checkstyle, spotless, enforcer and rat run clean.