Skip to content

Configuration

Faster GZIP with libdeflate (Java 22+)

Hardwood can use libdeflate for GZIP decompression instead of the built-in Java implementation. This feature requires Java 22 or newer (it uses the Foreign Function & Memory API).

1. JVM flag (allow Hardwood to bind native functions):

--enable-native-access=ALL-UNNAMED

2. Native library — install on your system:

brew install libdeflate

apt install libdeflate-dev

dnf install libdeflate-devel

vcpkg install libdeflate

Or download from GitHub releases.

When libdeflate is installed and available on the library path, Hardwood will automatically use it for GZIP decompression. To disable libdeflate and use the built-in Java implementation instead, set the system property:

-Dhardwood.uselibdeflate=false

SIMD Acceleration with Vector API (Java 22+)

Hardwood can use the Java Vector API (SIMD) to accelerate counting the non-null values of a page. SIMD requires Java 22 or newer and the Vector API incubator module, which is added with this JVM argument:

--add-modules jdk.incubator.vector

With the module added, Hardwood uses SIMD when the CPU's preferred vector width holds at least four 32-bit values (128 bits). It uses the preferred width of the CPU: 128-bit for SSE/NEON, 256-bit for AVX2, 512-bit for AVX-512. On Java 21, without the module, or on a CPU with narrower vectors, Hardwood uses scalar implementations that produce the same results.

To run the scalar implementations (for debugging or comparison), omit the --add-modules jdk.incubator.vector argument.

JFR (Java Flight Recorder) Events

Hardwood emits JFR events during file reading. Start a JFR recording to capture them:

java -XX:StartFlightRecording=filename=recording.jfr,settings=profile ...

Or attach dynamically via jcmd <pid> JFR.start.

Available events:

Event Category Description
dev.hardwood.FileOpened I/O File opened and metadata read. Fields: file, fileSize, rowGroupCount, columnCount
dev.hardwood.FileMapping I/O Memory-mapping of a file region. Fields: file, offset, size
dev.hardwood.RowGroupScanned Decode Page boundaries scanned in a column chunk. Fields: file, rowGroupIndex, column, pageCount, scanStrategy (sequential or offset-index)
dev.hardwood.PageDecoded Decode Single data page decoded. Fields: column, compressedSize, uncompressedSize
dev.hardwood.RowGroupFilter Filter Row groups dropped by statistics predicate pushdown. Fields: file, totalRowGroups, rowGroupsKept, rowGroupsSkipped, rowGroupsFullyMatching
dev.hardwood.RowGroupBloomFilter Filter Row group dropped by bloom filter predicate pushdown when the read reaches it, one event per row group dropped. RowGroupFilter counts that row group as kept. Fields: file, rowGroupIndex
dev.hardwood.RowGroupDictionaryFilter Filter Row group dropped by dictionary predicate pushdown when the read reaches it, one event per row group dropped. RowGroupFilter counts that row group as kept. Fields: file, rowGroupIndex
dev.hardwood.RowGroupByteRangeFilter Filter Row groups selected by a byte-range split predicate (split-aware reading). Fields: file, totalRowGroups, rowGroupsKept, rowGroupsSkipped
dev.hardwood.PageFilter Filter Pages filtered by Column Index predicate pushdown within a kept row group, one event per column chunk the Column Index narrowed; a chunk without a Column Index has its pages dropped from page-header statistics, which no event reports. Fields: file, rowGroupIndex, column, totalPages, pagesKept, pagesSkipped
dev.hardwood.RecordFilter Filter Records filtered by record-level predicate evaluation, one event per file read. Fields: file, totalRecords, recordsKept, recordsSkipped
dev.hardwood.BatchWait Pipeline Consumer blocked waiting for the assembly pipeline; the event's duration is the stall. Fields: column

Events appear under the Hardwood category in JDK Mission Control (JMC) or any JFR analysis tool. Use them to identify:

  • I/O bottlenecks — large FileMapping durations
  • Filter effectiveness: RowGroupFilter, RowGroupBloomFilter, RowGroupDictionaryFilter, RowGroupByteRangeFilter, PageFilter and RecordFilter counts
  • Decode hotspots — PageDecoded events with large uncompressed sizes or high frequency
  • Pipeline stalls — BatchWait events indicate the reader is waiting for decoded data

Reader and Writer Options

Read-time behaviour is configured with a ReaderConfig passed to ParquetFileReader.open(...); see the Reader Reference. Write-time behaviour is configured with a WriterConfig passed to ParquetFileWriter.create(...); see the Writer Reference.

System Properties Reference

Property Default Description
hardwood.uselibdeflate true Set to false to disable libdeflate for GZIP decompression