Repository navigation
Document v3 types - #14888
Document v3 types#14888
Conversation
| | **`timestamp_ns`** | Timestamp without timezone, nanosecond precision | Stored as nanoseconds; added in v3 | | ||
| | **`timestamptz_ns`** | Timestamp with timezone, nanosecond precision | Stored as nanoseconds; added in v3 | | ||
| | **`string`** | Arbitrary-length character sequences | Encoded with UTF-8 | | ||
| | **`uuid`** | Universally unique identifiers | Should use 16-byte fixed | |
There was a problem hiding this comment.
I wonder if the note here should just like out to the UUID RFC https://datatracker.ietf.org/doc/html/rfc9562.html, I'd also be ok with no note here.
| | **`uuid`** | Universally unique identifiers | Should use 16-byte fixed | | ||
| | **`fixed(L)`** | Fixed-length byte array of length L | | | ||
| | **`binary`** | Arbitrary-length byte array | | | ||
| | **`variant`** | Semi-structured data (JSON-like) | Added in v3; requires Spark 4.0+ | |
There was a problem hiding this comment.
I'd remove the "Spark" bit here, since that's more of an engine specific detail and we tend not to keep those in this document.
| | **`fixed(L)`** | Fixed-length byte array of length L | | | ||
| | **`binary`** | Arbitrary-length byte array | | | ||
| | **`variant`** | Semi-structured data (JSON-like) | Added in v3; requires Spark 4.0+ | | ||
| | **`geometry(C)`** | Geospatial features with CRS parameter | Planar calculations; added in v3 | |
There was a problem hiding this comment.
not sure "calculations" is the right word here, I don't know the Geo Space well enough but I would have guessed maybe kernel? Or Topography
| ListType list = ListType.ofRequired(1, IntegerType.get()); | ||
| ``` | ||
|
|
||
| ## Using Version 3 Types |
There was a problem hiding this comment.
Not sure we need this section, do you think it adds something that isn't explained in the above examples?
There was a problem hiding this comment.
@evindj Can we please remove these doc changes? I just don't think they add very much here, "nanosecond" is just another primitive that we've already gone over. The Variant and Geospatial examples don't really show much either. I think we are good with just updating the other page
| | **`fixed(L)`** | Fixed-length byte array of length L | | | ||
| | **`binary`** | Arbitrary-length byte array | | | ||
| | **`variant`** | Semi-structured data (JSON-like) | Added in v3 | | ||
| | **`geometry(C)`** | Geospatial features with CRS parameter | Planar kernel; added in v3 | |
There was a problem hiding this comment.
@jiayuasu Does Kernel fit here? I know you are the expert here and this was just my guess
There was a problem hiding this comment.
Let's use Linear edge-interpolation
| | **`binary`** | Arbitrary-length byte array | | | ||
| | **`variant`** | Semi-structured data (JSON-like) | Added in v3 | | ||
| | **`geometry(C)`** | Geospatial features with CRS parameter | Planar kernel; added in v3 | | ||
| | **`geography(C, A)`** | Geospatial features with CRS and edge algorithm | Spherical kernel; added in v3 | |
There was a problem hiding this comment.
| | **`geography(C, A)`** | Geospatial features with CRS and edge algorithm | Spherical kernel; added in v3 | | |
| | **`geography(C, A)`** | Geospatial features with CRS and edge algorithm | Non-linear edge-interpolation such as spherical; added in v3 | |
| | **`fixed(L)`** | Fixed-length byte array of length L | | | ||
| | **`binary`** | Arbitrary-length byte array | | | ||
| | **`variant`** | Semi-structured data (JSON-like) | Added in v3 | | ||
| | **`geometry(C)`** | Geospatial features with CRS parameter | Linear edge-interpolation such as planar; added in v3 | |
There was a problem hiding this comment.
I think we drop "such as planar", but the spherical comment should stay
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
|
@RussellSpitzer @jiayuasu Do you have any more comments? |
|
Ah sorry, missed this got updated. Let's merge!~ |
4 participants
Summary
This PR addresses Issues #14874 adds documentation for new types including timestamp(with and without timezone support), UUID, Variant, Geometry and Unknown. It is not clear to me that all the types mentioned here where added v3, so please let me know if I have missed anything.
Testing
tested with
cd site && make serveverified http://127.0.0.1:8000/docs/nightly/api/#using-version-3-types and http://127.0.0.1:8000/docs/nightly/schemas/