Search before asking
Paimon version
master @ da71a66 (2.1-SNAPSHOT)
Compute Engine
Engine-agnostic (Hadoop FileSystem SDK, paimon-vfs-hadoop)
Minimal reproduce step
PaimonVirtualFileSystem.initialize() (paimon-vfs/paimon-vfs-hadoop/src/main/java/org/apache/paimon/vfs/hadoop/PaimonVirtualFileSystem.java line 70) sets workingDirectory = new Path(uri) from the raw URI Hadoop passes in, while line 74 normalizes this.uri to scheme://authority/.
Following the documented usage in docs/docs/concepts/rest/pvfs.md lines 61-62:
Path path = new Path("pvfs://catalog/db_a/tbl_a/a.csv");
FileSystem fs = path.getFileSystem(conf);
fs.create(new Path("db_b/tbl_b/b.csv"));
What doesn't meet your expectations?
Expected: the working directory is the catalog root, so a relative path resolves under the table it names. Actual: the instance contradicts itself — getUri() returns pvfs://catalog/ but getWorkingDirectory() returns pvfs://catalog/db_a/tbl_a/a.csv. The create above raises no exception and writes into db_a/tbl_a.
Anything else?
Which table receives the write is non-deterministic: the Hadoop FileSystem cache key is scheme+authority+ugi, so it depends on which path first opened a pvfs FileSystem in that JVM.
Are you willing to submit a PR?
Search before asking
Paimon version
master @ da71a66 (2.1-SNAPSHOT)
Compute Engine
Engine-agnostic (Hadoop FileSystem SDK,
paimon-vfs-hadoop)Minimal reproduce step
PaimonVirtualFileSystem.initialize()(paimon-vfs/paimon-vfs-hadoop/src/main/java/org/apache/paimon/vfs/hadoop/PaimonVirtualFileSystem.java line 70) setsworkingDirectory = new Path(uri)from the raw URI Hadoop passes in, while line 74 normalizesthis.uritoscheme://authority/.Following the documented usage in
docs/docs/concepts/rest/pvfs.mdlines 61-62:What doesn't meet your expectations?
Expected: the working directory is the catalog root, so a relative path resolves under the table it names. Actual: the instance contradicts itself —
getUri()returnspvfs://catalog/butgetWorkingDirectory()returnspvfs://catalog/db_a/tbl_a/a.csv. Thecreateabove raises no exception and writes intodb_a/tbl_a.Anything else?
Which table receives the write is non-deterministic: the Hadoop FileSystem cache key is scheme+authority+ugi, so it depends on which path first opened a pvfs FileSystem in that JVM.
Are you willing to submit a PR?