This commit introduces memory tracker with jemalloc extent hooks which fully works in case when jemalloc config is following:
MALLOC_CONF="retain:false,percpu_arena:percpu,oversize_threshold:1000000000000,muzzy_decay_ms:0,dirty_decay_ms:0" \
./configure \
--disable-cxx \
$COMMON_CONFIGURE_FLAGS \
--with-malloc-conf="retain:false,percpu_arena:percpu,oversize_threshold:1000000000000,muzzy_decay_ms:0,dirty_decay_ms:0"
This config will for jemalloc not to use lazy purge or MADV_FREE(muzzy_decay_ms=0 and dirty_decay_ms=0), it will force jemalloc not to use
custom arena for huge allocations (oversize_threshold) and it will force jemalloc not extend virtual memory indefinitely (retain=false)
and therefore call alloc hook when allocation actually takes place.
Only problem is if we do huge allocations which are not mapped on alloc directly, in that case jemalloc uses cache and we can overcounter
allocation size.
It is possible for multiple read only queries to be accessing the same
sequence of vertices/edges. The reader mode of the spin lock will ensure
multiple threads can make progress at the same time.
Add timestamp to DELETE_DESERIALIZED_OBJECT delta at which this object was created.
RocksDB currently doesn't provide timestamp() functionality in iterators of TransationDB.
Because of that we are using constant "0" timestamp for DELETE_DESERIALIZED_OBJECT.
Concurrent access to the same query module had a race condition on the
pointer that was used to handle the custom memory management. With this
commit, a mapping has been added to keep information about what
thread used the pointer to handle the memory resources. This should be
fine since the respected query executions are running on a dedicated
thread. Access to the mapping itself is threadsafe. A simple RAII
wrapper for the mapping container has also been added for simpler
client-side use.
Add supernode vertex cache to account for long delta chains and modifications in the same module being independent of scanning of the nodes in the next iteration of the pulling mechanism.
