Decouple spine from array reserved regions by LinHu2016 · Pull Request #22116 · eclipse-openj9/openj9

LinHu2016 · 2025-06-18T21:51:02Z

For off-heap enabled case, the Large size Array(larger than
region size) will be allocated on sparse heap, but we still
reserve leaf regions for preventing over usage on heap.
Currently we have to allocate/reserve whole region for the
remaining bytes(remainder of array size from region size), it
potentially generate fragmentation on heap.

1, Decouple spine with reserved regions for off-heap enanbled
case.
2, find spines via SparseDataTable in recycling reserved regions.
3, use GC_HashTableIterator to get current spine in
doObjectInVirtualLargeObjectHeap() and pass the iterator for
UnmapFromHeapObject to avoid conflict between hashTable iterator and
hashTableRemove().
4, recycle reserved regions during doObjectInVirtualLargeObjectHeap
instead of during post collection(copyforward collector) or sweepProcess
(global collector) for off-heap case.

new global _arryReservedRegionList in MM_AllocationContextBalanced
for keeping reserved regions for off-heap case(reuse
region->_allocateData._nextArrayletLeafRegion for linking next).
retrieve reserved regions via _arryReservedRegionList instead of
iterating whole regions to find reserved regions during recycling
reserved regions for off-heap case.
all of reserved regions are the same, can pick any reserved region in
the _arryReservedRegionList for recycling, so we only reuse single link list
(region->_allocateData._nextArrayletLeafRegion) for maintaining the list.
Replace arrayletLeafSize with regionSize
Rename push/popRegionToLeafRegionList() with
push/popRegionToArrayReservedRegionList()
push/popRegionToArrayReservedRegionList in/from common allocation
context.
Using lockCommon/unlockCommon for synchronizing multithreads
pushRegionToArrayReservedRegionList.
Rename _leafRegionList with _arrayReservedRegionList
Rename recycleLeafRegionsForVirtualLargeObjectHeap() with
recycleReservedRegionsForVirtualLargeObjectHeap()
Rename arrayletLeafCount with reservedRegionCount

#depends on eclipse-omr/omr#7748 and
eclipse-omr/omr#7792
#reference #21817

amicic · 2025-07-10T16:17:10Z

 				}
-			} else if(region->isArrayletLeaf()) {
+			} else if(region->isArrayletLeaf() && !_extensions->isVirtualLargeObjectHeapEnabled) {
 				MM_HeapRegionDescriptorVLHGC *parentRegion = (MM_HeapRegionDescriptorVLHGC *)_regionManager->regionDescriptorForAddress((void *)region->_allocateData.getSpine());


put a comment that recycling of regions reserved for offheap objects is already done in root clearable phase at the end of GMP

sorry, not here, but in Sweep code

amicic · 2025-07-10T16:31:05Z

 	UDATA _freeProcessorNodeCount;	/**< The length, in elements, of the _freeProcessorNodes array (always at least 1 after startup) */

+#if defined(J9VM_GC_SPARSE_HEAP_ALLOCATION)
+	MM_HeapRegionDescriptorVLHGC *_leafRegionList; /** for Off-heap case only, adding and removing via region->_allocateData.pushRegionToLeafRegionList/popRegionFromLeafRegionList */


this is more descriptive of what this really is:
the list of regions reserved and de-committed by all large array allocations, instead of which some other contiguous memory (offheap) is actually committed and used by those arrays

we could even rename it as: _arrayReservedRegionList (@dmitripivkine opinion?)

we should start retiring terms like arraylet and even leaf, in offheap specific paths

I agree. We should change terminology, at least in sparse heap related path. _arrayReservedRegionLis sounds good.

amicic · 2025-07-10T16:46:04Z

+			MM_SparseDataTableEntry *sparseDataEntry = NULL;
+			GC_HashTableIterator iterator(table);
+			while (NULL != (sparseDataEntry = (MM_SparseDataTableEntry *)iterator.nextSlot())) {
+				J9Object *spineObject = (J9Object *)sparseDataEntry->_proxyObjPtr;


Another questionable term is spine. For offheap it's far less then that (no arroid pointers, no last leaf). In the map we call it proxyObject.
@dmitripivkine, opinion, which one is better for you? Or anything else? For example, something simpler like ' object header'?

Also the type should be J9IndexableObject, for both spineObject (of whatever we rename it to) and the parameter of doObjectInVirtualLargeObjectHeap (what should trickle down a lot more changes to change the type).

I have no problem with "spine" despite it is "empty spine" for off-heap object. If we can get better self-explainable name we can change it.

amicic · 2025-07-11T16:28:53Z


 			/* If forwarded object is NULL, free the sparse region occupied by the data of the indexable object */
 			if (NULL == fwdOjectPtr) {
+				const uintptr_t arrayletLeafSize = env->getOmrVM()->_arrayletLeafSize;


we really reserve regions form the main heap when we allocate offheap objects (and in near future, it will be fractional reservation of regions)
so we should be fetching region size here, not arrayletLeafSize

as per earlier comment, we should be changing terminology in offheap specific path
here, it's even more then just terminology - we should not even deal with this field

amicic · 2025-07-11T16:30:35Z


+#if defined(J9VM_GC_SPARSE_HEAP_ALLOCATION)
+void
+MM_CopyForwardScheme::recycleLeafRegionsForVirtualLargeObjectHeap(MM_EnvironmentVLHGC *env, uintptr_t arrayletLeafCount)


I'd rename the method name to recycleReservedRegionsForVirtualLargeObjectHeap
and use reservedRegionCount instead of arrayletLeafCount

amicic · 2025-07-11T16:32:23Z


+#if defined(J9VM_GC_SPARSE_HEAP_ALLOCATION)
 void 
+MM_HeapRegionDataForAllocate::pushRegionToLeafRegionList(MM_EnvironmentVLHGC *env, MM_HeapRegionDescriptorVLHGC **head)


I'd rename this to push/popRegionFromArrayReservedRegionList

it's ok to reuse existing _nextArrayletLeafRegion pointer (rather then introducing new one), even though the name does not fully fit

amicic · 2025-07-11T16:32:49Z

+
+	MM_GCExtensions *extensions = MM_GCExtensions::getExtensions(env);
+	/* Restore/Recommit arraylet leaves that have been previously decommitted. */
+	const UDATA arrayletLeafSize = env->getOmrVM()->_arrayletLeafSize;


we should deal with the region size

amicic · 2025-07-11T16:35:56Z

-	 * we already hold is not sufficient any more. We need to additionally acquire common context' common lock, since multiple spines from different ACs could have come into this state,
-	 * and worse multiple spines originally allocated from different ACs may end up in a single common context region.
-	 */
+	if (!MM_GCExtensions::getExtensions(env)->isVirtualLargeObjectHeapEnabled) {


this should be below just before we add/push region to the list
the only thing that we change is that list is AC local (nor array local)
same problem of spine changing context to common likely exists with offheap

#if defined(J9VM_GC_SPARSE_HEAP_ALLOCATION) if (MM_GCExtensions::getExtensions(env)->isVirtualLargeObjectHeapEnabled) { leafAllocateData->pushRegionToLeafRegionList(env, &_leafRegionList); } else #endif /* defined(J9VM_GC_SPARSE_HEAP_ALLOCATION) */ { leafAllocateData->addToArrayletLeafList(spineRegion); }

amicic · 2025-07-11T16:39:12Z

 		, _freeProcessorNodes(NULL)
 		, _freeProcessorNodeCount(0)
+#if defined(J9VM_GC_SPARSE_HEAP_ALLOCATION)
+		, _leafRegionList(NULL)


rename to _arrayReservedRegionList

amicic · 2025-07-11T16:40:07Z

+		if (_extensions->isVirtualLargeObjectHeapEnabled) {
+			const uintptr_t arrayletLeafSize = env->getOmrVM()->_arrayletLeafSize;
+			MM_SparseVirtualMemory *largeObjectVirtualMemory = _extensions->largeObjectVirtualMemory;
+			uintptr_t arrayletLeafCount = 0;


should deal with region size, and arrayRegionReservedCount

amicic · 2025-07-11T16:49:37Z

+			J9Object *spineObject = (J9Object *)sparseDataEntry->_proxyObjPtr;
+
+			if (_extensions->objectModel.isObjectArray(spineObject)) {
+				_liveSetBytesAfterPartialCollect += MM_Math::roundToCeiling(arrayletLeafSize, sparseDataEntry->_size);


round to regions size, but this should be updated once we start using fractional regions reservation
(maybe still round up, but only to the OS page size that offheap uses!?)

amicic · 2025-07-11T16:51:55Z

+				stats->_reclaimStats._regionCountAfter += arrayletLeafCount;
+				stats->_reclaimStats._regionCountArrayletLeafAfter += arrayletLeafCount;
+
+				if(!parentRegion->_sweepData._alreadySwept) {


space after 'if'
(not only here, but a few spots elsewhere)

In PR eclipse-openj9#22116 and eclipse-openj9#22546, _arrayReservedRegionList and _sharedArrayReservedRegionsBytesUsed has been introduced to share fraction of reserved regions. _arrayReservedRegionList is set in commonAllocationContext(single global list), For numa case, the reserved regions for large array could be allocated from different allocation context(allocation context per numa node and in case there is no free region for the numa note, it could borrow from neighbor numa node). during recycling, there are no extra information indicate which reserved regions are for recycling off-heap array, so potentially it might cause imbalance free memory among numa notes, then affect runtime performance in some cases. Undate to manage reserved region list per allocation context(numa note) to avoid imbalance free memory caused by reserved regions. 1, for reducing complexity, shared fraction reserved regions still be managed in _arrayReservedRegionList of commom context. set _sharedReserved in AllocateDescriptor for shared fraction reserved region allocation. 2, new _allocationContextArray in SparseVirtualMemory(omr PR eclipse-omr/omr#7956) to store/retrieve allocation context for the reserved regions of large array. 3, handle allocation failure during getSparseAddressAndDecommitLeaves(). Signed-off-by: lhu <linhu@ca.ibm.com>

In PR eclipse-openj9#22116 and eclipse-openj9#22546, _arrayReservedRegionList and _sharedArrayReservedRegionsBytesUsed has been introduced to share fraction of reserved regions. _arrayReservedRegionList is set in commonAllocationContext(single global list), For numa case, the reserved regions for large array could be allocated from different allocation context(allocation context per numa node and in case there is no free region for the numa note, it could borrow from neighbor numa node). during recycling, there are no extra information indicate which reserved regions are for recycling off-heap array, so potentially it might cause imbalance free memory among numa notes, then affect runtime performance in some cases. Undate to manage reserved region list per allocation context(numa note) to avoid imbalance free memory caused by reserved regions. 1, for reducing complexity, shared fraction reserved regions still be managed in _arrayReservedRegionList of commom context. set _sharedReserved in AllocateDescriptor for shared fraction reserved region allocation. 2, new _allocationContextArray in SparseVirtualMemory(omr PR eclipse-omr/omr#7956) to store/retrieve allocation context for the reserved regions of large array. 3, handle allocation failure during getSparseAddressAndDecommitLeaves(). Change-Id: I84938d68346ce45b00d2c2fa0a351cc0925bebfe Signed-off-by: lhu <linhu@ca.ibm.com>

In PR eclipse-openj9#22116 and eclipse-openj9#22546, _arrayReservedRegionList and _sharedArrayReservedRegionsBytesUsed has been introduced to share fraction of reserved regions. _arrayReservedRegionList is set in commonAllocationContext(single global list), For numa case, the reserved regions for large array could be allocated from different allocation context(allocation context per numa node and in case there is no free region for the numa note, it could borrow from neighbor numa node). during recycling, there are no extra information indicate which reserved regions are for recycling off-heap array, so potentially it might cause imbalance free memory among numa notes, then affect runtime performance in some cases. Undate to manage reserved region list per allocation context(numa note) to avoid imbalance free memory caused by reserved regions. 1, for reducing complexity, shared fraction reserved regions still be managed in _arrayReservedRegionList of commom context. set _sharedReserved in AllocateDescriptor for shared fraction reserved region allocation. 2, new _allocationContextArray in SparseVirtualMemory(omr PR eclipse-omr/omr#7956) to store/retrieve allocation context for the reserved regions of large array. 3, handle allocation failure during getSparseAddressAndDecommitLeaves(). Change-Id: Ib10d11d71780c5b6d0337d74b65616a73b7f0299 Signed-off-by: lhu <linhu@ca.ibm.com>

In PR eclipse-openj9#22116 and eclipse-openj9#22546, _arrayReservedRegionList and _sharedArrayReservedRegionsBytesUsed has been introduced to share fraction of reserved regions. _arrayReservedRegionList is set in commonAllocationContext(single global list), For numa case, the reserved regions for large array could be allocated from different allocation context(allocation context per numa node and in case there is no free region for the numa note, it could borrow from neighbor numa node). during recycling, there are no extra information indicate which reserved regions are for recycling off-heap array, so potentially it might cause imbalance free memory among numa notes, then affect runtime performance in some cases. Undate to manage reserved region list per allocation context(numa note) to avoid imbalance free memory caused by reserved regions. 1, new _allocationContextArray in SparseVirtualMemory(omr PR eclipse-omr/omr#8070) to store allocation context for the reserved regions of large array in getSparseAddressAndDecommitLeaves(). 2, during recycling the large array, retrieve the related allocation context for recycling the reserved regions. 3, handle allocation failure during getSparseAddressAndDecommitLeaves(). 3, new APIs allocateFromSharedArrayReservedRegion and allocateFromSharedReservedRegionForNode to handle sharing fraction and allocating shared reserved regions in the context(numa node). 4, Using new allocation Type ALLOCATION_TYPE_SHARED_RESERVED to handle the different re-allocate processing after GC. 5, Via new field _allocationContext in MM_AllocateDescription to pass the related allocation context back to the caller in order to keep the common interfaces signature unchanged. 6, In allocateFromSharedArrayReservedRegion() use local variable instead of instance variable for keeping nextToSteal (ToBorrowFreeRegionFromForeignNode) to avoid race condition. Signed-off-by: lhu <linhu@ca.ibm.com>

In PR eclipse-openj9#22116 and eclipse-openj9#22546, _arrayReservedRegionList and _sharedArrayReservedRegionsBytesUsed has been introduced to share fraction of reserved regions. _arrayReservedRegionList is set in commonAllocationContext(single global list), For numa case, the reserved regions for large array could be allocated from different allocation context(allocation context per numa node and in case there is no free region for the numa note, it could borrow from neighbor numa node). during recycling, there are no extra information indicate which reserved regions are for recycling off-heap array, so potentially it might cause imbalance free memory among numa notes, then affect runtime performance in some cases. Undate to manage reserved region list per allocation context(numa note) to avoid imbalance free memory caused by reserved regions. 1, new _allocationContextArray in SparseVirtualMemory(omr PR eclipse-omr/omr#8070) to store allocation context for the reserved regions of large array in getSparseAddressAndDecommitLeaves(). 2, during recycling the large array, retrieve the related allocation context for recycling the reserved regions. 3, handle allocation failure during getSparseAddressAndDecommitLeaves(). 3, new APIs allocateFromSharedArrayReservedRegion and allocateFromSharedReservedRegionForNode to handle sharing fraction and allocating shared reserved regions in the context(numa node). 4, Using new allocation Type ALLOCATION_TYPE_SHARED_RESERVED to handle the different re-allocate processing after GC. 5, Via new field _allocationContext in MM_AllocateDescription to pass the related allocation context back to the caller in order to keep the common interfaces signature unchanged. 6, In allocateFromSharedArrayReservedRegion() use local variable instead of instance variable for keeping nextToSteal (ToBorrowFreeRegionFromForeignNode) to avoid race condition. Change-Id: I6a8070bdf758b801cd17f5bd357250f36060110b Signed-off-by: lhu <linhu@ca.ibm.com>

In PR eclipse-openj9#22116 and eclipse-openj9#22546, _arrayReservedRegionList and _sharedArrayReservedRegionsBytesUsed has been introduced to share fraction of reserved regions. _arrayReservedRegionList is set in commonAllocationContext(single global list), For numa case, the reserved regions for large array could be allocated from different allocation context(allocation context per numa node and in case there is no free region for the numa note, it could borrow from neighbor numa node). during recycling, there are no extra information indicate which reserved regions are for recycling off-heap array, so potentially it might cause imbalance free memory among numa notes, then affect runtime performance in some cases. Undate to manage reserved region list per allocation context(numa note) to avoid imbalance free memory caused by reserved regions. 1, new _allocationContextArray in SparseVirtualMemory(omr PR eclipse-omr/omr#8070) to store allocation context for the reserved regions of large array in getSparseAddressAndDecommitLeaves(). 2, during recycling the large array, retrieve the related allocation context for recycling the reserved regions. 3, handle allocation failure during getSparseAddressAndDecommitLeaves(). 3, new APIs allocateFromSharedArrayReservedRegion and allocateFromSharedReservedRegionForNode to handle sharing fraction and allocating shared reserved regions in the context(numa node). 4, Using new allocation Type ALLOCATION_TYPE_SHARED_RESERVED to handle the different re-allocate processing after GC. 5, Via new field _allocationContext in MM_AllocateDescription to pass the related allocation context back to the caller in order to keep the common interfaces signature unchanged. 6, In allocateFromSharedArrayReservedRegion() use local variable instead of instance variable for keeping nextToSteal (ToBorrowFreeRegionFromForeignNode) to avoid race condition. Signed-off-by: lhu <linhu@ca.ibm.com>

LinHu2016 force-pushed the defrag-offheap-1.5 branch 3 times, most recently from 90c7b07 to 7392bc1 Compare June 20, 2025 14:53

LinHu2016 force-pushed the defrag-offheap-1.5 branch from 7392bc1 to c18dd54 Compare July 2, 2025 15:46

amicic added the comp:gc label Jul 8, 2025

LinHu2016 force-pushed the defrag-offheap-1.5 branch from c18dd54 to 5620c3c Compare July 8, 2025 20:32

amicic reviewed Jul 10, 2025

View reviewed changes

LinHu2016 force-pushed the defrag-offheap-1.5 branch 2 times, most recently from ce9463e to 53943fd Compare July 10, 2025 21:07

amicic reviewed Jul 11, 2025

View reviewed changes

LinHu2016 force-pushed the defrag-offheap-1.5 branch 10 times, most recently from 9af3783 to 97ab503 Compare July 15, 2025 15:12

LinHu2016 mentioned this pull request Sep 29, 2025

Handling multiple reserved region lists for numa off-heap case #22694

Closed

LinHu2016 mentioned this pull request Dec 8, 2025

Handling multiple reserved region lists for numa off-heap case #23075

Merged

Uh oh!

Conversation

LinHu2016 commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amicic Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amicic Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amicic Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

LinHu2016 commented Jun 18, 2025 •

edited

Loading

amicic Jul 10, 2025 •

edited

Loading

amicic Jul 10, 2025 •

edited

Loading

amicic Jul 11, 2025 •

edited

Loading