Utilize partially used TLABs
Currently, once a mutator tries to allocate an object bigger than the
TLAB size, it attempts to acquire another TLAB. The previous TLAB is not
utilized again. This leads to blocking GCs when mutators get created and
killed very frequently, as could happen in the case of Zygote.
In this change, we maintain a separate list of partially used TLABs
which can be reused whenever any mutator attempts to allocate a new TLAB
from region space.
Test: forrest hermatic test
Bug: 146706834
Change-Id: I8076663628e49fc10e33f30de937833f6812fdca
diff --git a/runtime/gc/collector/concurrent_copying.cc b/runtime/gc/collector/concurrent_copying.cc
index d15fdad..1f50c27 100644
--- a/runtime/gc/collector/concurrent_copying.cc
+++ b/runtime/gc/collector/concurrent_copying.cc
@@ -444,15 +444,17 @@
<< thread->GetState() << " thread " << thread << " self " << self;
thread->SetIsGcMarkingAndUpdateEntrypoints(true);
if (use_tlab_ && thread->HasTlab()) {
+ // We should not reuse the partially utilized TLABs revoked here as they
+ // are going to be part of from-space.
if (ConcurrentCopying::kEnableFromSpaceAccountingCheck) {
// This must come before the revoke.
size_t thread_local_objects = thread->GetThreadLocalObjectsAllocated();
- concurrent_copying_->region_space_->RevokeThreadLocalBuffers(thread);
+ concurrent_copying_->region_space_->RevokeThreadLocalBuffers(thread, /*reuse=*/ false);
reinterpret_cast<Atomic<size_t>*>(
&concurrent_copying_->from_space_num_objects_at_first_pause_)->
fetch_add(thread_local_objects, std::memory_order_relaxed);
} else {
- concurrent_copying_->region_space_->RevokeThreadLocalBuffers(thread);
+ concurrent_copying_->region_space_->RevokeThreadLocalBuffers(thread, /*reuse=*/ false);
}
}
if (kUseThreadLocalAllocationStack) {