How-to Guides¶
- Read Row by Row:
RowReader, typed accessors, nested structs / lists / maps. - Read Column by Column:
ColumnReaderandColumnReaders, the layer model, hot-loop patterns. - Filter, Project, Limit, and Split: predicate pushdown, column projection, row limits, split-aware reading. Apply to both reader types.
- Read Multiple Files as One Dataset:
Hardwood.openAll(...)with cross-file prefetching and a shared thread pool. - Read into Avro GenericRecords:
AvroRowReaderand schema conversion (hardwood-avromodule). - Read from S3: object-store reading without Hadoop (
hardwood-s3module). - Read with the parquet-java API: drop-in
org.apache.parquet.*replacement (experimental). - Read Variant Columns:
getVariantand thePqVariantAPI. - Read Geospatial Columns: GEOMETRY / GEOGRAPHY columns, bounding-box filter pushdown.
- Write Row by Row:
RowWriter, typed setters, nested structs / lists / maps. - Write Column by Column:
ColumnWriter, typed arrays, nulls, per-layer offsets. - Handle Write Failures: what
close()does after a failed write, andabort(). - Inspect File Metadata: file metadata, row groups, column chunks, schema introspection.
Runnable examples
Most guides link to a matching standalone example in the hardwood-examples repository. Every example is a complete project you can clone and run with a single command. To start, see Hello Hardwood.
For detailed class-level documentation, see the JavaDoc.
Choosing a Reader¶
Hardwood provides two reader APIs:
RowReader: row-oriented access with typed getters, including nested structs, lists, and maps. Best for general-purpose reading where you process one row at a time.ColumnReader: batch-oriented columnar access with typed primitive arrays. Best for analytical workloads where you process columns independently (e.g. summing a column, computing statistics).
For the reasoning behind the two APIs and the ergonomics-versus-throughput trade-off, see RowReader vs. ColumnReader.
Both support column projection and predicate pushdown. Each reader has a no-arg shortcut for default reads and a builder form for filtered or limited reads:
| Reader | Shortcut | Builder |
|---|---|---|
RowReader |
reader.rowReader() |
reader.buildRowReader().…build() |
ColumnReader (single) |
reader.columnReader("id") |
reader.buildColumnReader("id").…build() |
ColumnReaders (multiple) |
reader.columnReaders(projection) |
reader.buildColumnReaders(projection).…build() |
Choosing a Writer¶
Writing mirrors the same split. Both APIs come from one ParquetFileWriter and produce the same layout, but a file is written through one of them, not both:
RowWriter: record-oriented, obtained fromwriter.rowWriter(). Fields are addressed by name, and logical-type values are written as the Java types the reader returns for them. Best when you hold records.ColumnWriter: batch-oriented, obtained fromwriter.columnWriter(). EachwriteBatchcall takes one typed array per leaf column, plus per-layer validity and offsets for nested columns. Best when you already hold columns.
For the model behind the file the writer produces, see The Write Model.
For the exceptions the readers and the writer can throw and when, see Error Handling.