You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This project brings delta-sharing capabilities to java.
13
+
This project brings Delta Sharing capabilities to Java.
14
+
14
15
The Java connector follows the Delta Sharing protocol to read shared tables from a Delta Sharing Server. To further reduce and limit egress costs on the Data Provider side, we implemented a persistent cache to reduce and limit the egress costs on the Data Provider side by removing any unnecessary reads.
15
16
16
17
- The data is served to the connector via persisted cache to limit the egress costs whenever possible.
@@ -19,7 +20,7 @@ The Java connector follows the Delta Sharing protocol to read shared tables from
19
20
- Data records are provided as a set of Avro GenericRecords that provide a good balance between the flexibility of representation and integrational capabilities. GenericRecords can easily be exported to JSON and/or other formats using EncoderFactory in Avro.
20
21
21
22
- Every time the data access is requested the connector will check for the metadata updates and refresh the table data in case of any metadata changes.
22
-
- The connector requests the metadata for the table based on its coordinate from the provider. The table coordinate is the profile file path following with `#` and the fully qualified name of a table (<share-name>.<schema-name>.<table-name>)
23
+
- The connector requests the metadata for the table based on its coordinate from the provider. The table coordinate is the profile file path following with `#` and the fully qualified name of a table (`<share-name>.<schema-name>.<table-name>`)
23
24
- A lookup of table to metadata is maintained inside the JVM. The connector then compares the received metadata with the last metadata snapshot. If there is no change, then the existing table data is served from cache. Otherwise, the connector will refresh the table data in the cache.
24
25
25
26
- When the metadata changes are detected both the data and the metadata will be updated.
0 commit comments