When a pool resource is constructed with max_blocks_per_chunk=1 it ends
up creating a pool with blocks_per_chunk=0 which means it never
allocates anything. Instead it returns null pointers, which should be
impossible.
To avoid this problem, round the max_blocks_per_chunk value to a
multiple of four, so it's never smaller than four.
libstdc++-v3/ChangeLog:
PR libstdc++/94160
* src/c++17/memory_resource.cc (munge_options): Round
max_blocks_per_chunk to a multiple of four.
(__pool_resource::_M_alloc_pools()): Simplify slightly.
* testsuite/20_util/unsynchronized_pool_resource/allocate.cc:
Check that valid pointers are returned when small values are
used for max_blocks_per_chunk.
The primary reason for this change is to reduce the size of buffers
allocated by std::pmr::monotonic_buffer_resource. Previously, a new
buffer would always add the size of the linked list node (11 bytes) and
then round up to the next power of two. This results in a huge increase
if the expected size of the next buffer is already a power of two. For
example, if the resource is constructed with a desired initial size of
4096 the first buffer it allocates will be std::bit_ceil(4096+11) which
is 8192. If the user has carefully selected the initial size to match
their expected memory requirements then allocating double that amount
wastes a lot of memory.
After this patch the allocated size will be rounded up to a 64-byte
boundary, instead of to a power of two. This means for an initial size
of 4096 only 4160 bytes get allocated.
Previously only the base-2 logarithm of the size was stored, which could
be stored in a single 8-bit integer. Now that the size isn't always a
power of two we need to use more bits to store it. As the size is always
a multiple of 64 the low six bits are not needed, and so we can use the
same approach that the pool resources already use of storing the base-2
logarithm of the alignment in the low bits that are not used for the
size. To avoid code duplication, a new aligned_size<N> helper class is
introduced by this patch, which is then used by both the pool resources'
big_block type and the monotonic_buffer_resource::_Chunk type.
Originally the big_block type used two bit-fields to store the size and
alignment in the space of a single size_t member. The aligned_size type
uses a single size_t member and uses masks and bitwise operations to
manipulate the size and alignment values. This results in better code
than the old version, because the bit-fields weren't optimally ordered
for little endian architectures, so the alignment was actually stored in
the high bits, not the unused low bits, requiring additional shifts to
calculate the values. Using bitwise operations directly avoids needing
to reorder the bit-fields depending on the endianness.
While adapting the _Chunk and big_block types to use aligned_size<N> I
also added checks for size overflows (technically, unsigned wraparound).
The memory resources now ensure that when they require an allocation
that is too large to represent in size_t they will request SIZE_MAX
bytes from the upstream resource, rather than requesting a small value
that results from wrapround. The testsuite is enhanced to verify this.
libstdc++-v3/ChangeLog:
PR libstdc++/96942
* include/std/memory_resource (monotonic_buffer_resource::do_allocate):
Use __builtin_expect when checking if a new buffer needs to be
allocated from the upstream resource, and for checks for edge
cases like zero sized buffers and allocations.
* src/c++17/memory_resource.cc (aligned_size): New class template.
(aligned_ceil): New helper function to round up to a given
alignment.
(monotonic_buffer_resource::chunk): Replace _M_size and _M_align
with an aligned_size member. Remove _M_canary member. Change _M_next
to pointer instead of unaligned buffer.
(monotonic_buffer_resource::chunk::allocate): Round up to multiple
of 64 instead of to power of two. Check for size overflow. Remove
redundant check for minimum required alignment.
(monotonic_buffer_resource::chunk::release): Adjust for changes
to data members.
(monotonic_buffer_resource::_M_new_buffer): Use aligned_ceil.
(big_block): Replace _M_size and _M_align with aligned_size
member.
(big_block::big_block): Check for size overflow.
(big_block::size, big_block::align): Adjust to use aligned_size.
(big_block::alloc_size): Use aligned_ceil.
(munge_options): Use aligned_ceil.
(__pool_resource::allocate): Use big_block::align for alignment.
* testsuite/20_util/monotonic_buffer_resource/allocate.cc: Check
upstream resource gets expected values for impossible sizes.
* testsuite/20_util/unsynchronized_pool_resource/allocate.cc:
Likewise. Adjust checks for expected alignment in existing test.
A recent change caused a performance regression. This restores the
previous performance and adds a performance test.
* scripts/check_performance: Allow tests to choose a -std flag.
* src/c++17/memory_resource.cc (bitset::get_first_unset()): Use local
variables of the right types. Call update_next_word() unconditionally.
* testsuite/20_util/unsynchronized_pool_resource/cons.cc: New test.
* testsuite/performance/20_util/memory_resource/pools.cc: New test.
* testsuite/util/testsuite_performance.h (time_counter): Allow
timer to be restarted.
From-SVN: r266164
Make the munge_options function round the largest_required_pool_block
value to a multiple of the smallest pool size (currently 8 bytes) to
avoid pools with odd sizes.
Ensure there is a pool large enough for blocks of the requested size.
Previously when largest_required_pool_block was exactly equal to one of
the pool_sizes[] values there would be no pool of that size. This patch
increases _M_npools by one, so there is a pool at least as large as the
requested value. It also reduces the size of the largest pool to be no
larger than needed.
* src/c++17/memory_resource.cc (munge_options): Round up value of
largest_required_pool_block to multiple of smallest pool size. Round
excessively large values down to largest pool size.
(select_num_pools): Increase number of pools by one unless it exactly
matches requested largest_required_pool_block.
(__pool_resource::_M_alloc_pools()): Make largest pool size equal
largest_required_pool_block.
* testsuite/20_util/unsynchronized_pool_resource/options.cc: Check
that pool_options::largest_required_pool_block is set appropriately.
From-SVN: r266089
Since a big_block rounds up the size to a multiple of big_block::min it
is wrong to assert that the supplied number of bytes equals the
big_block's size(). Add big_block::alloc_size(size_t) to calculate the
allocated size consistently, and add comments to the code.
* src/c++17/memory_resource.cc (big_block): Improve comments.
(big_block::all_ones): Remove.
(big_block::big_block(size_t, size_t)): Use alloc_size.
(big_block::size()): Add comment, replace all_ones with equivalent
expression.
(big_block::align()): Shift value of correct type.
(big_block::alloc_size(size_t)): New function to round up size.
(__pool_resource::allocate(size_t, size_t)): Add comment.
(__pool_resource::deallocate(void*, size_t, size_t)): Likewise. Fix
incorrect assertion by using big_block::alloc_size(size_t).
* testsuite/20_util/unsynchronized_pool_resource/allocate.cc: Add
more tests for unpooled allocations.
From-SVN: r266088
* src/c++17/memory_resource.cc (bitset::full()): Handle edge case
for _M_next_word maximum value.
(bitset::get_first_unset(), bitset::set(size_type)): Use
update_next_word() to update _M_next_word.
(bitset::update_next_word()): New function, avoiding wraparound of
unsigned _M_next_word member.
(bitset::max_word_index()): New function.
(chunk::chunk(void*, uint32_t, void*, size_t)): Add assertion.
(chunk::max_bytes_per_chunk()): New function.
(pool::replenish(memory_resource*, const pool_options&)): Prevent
_M_blocks_per_chunk from exceeding max_blocks_per_chunk or from
causing chunk::max_bytes_per_chunk() to be exceeded.
* testsuite/20_util/unsynchronized_pool_resource/allocate-max-chunks.cc:
New test.
From-SVN: r266087
