Commit Graph

3510 Commits

Author SHA1 Message Date
antoniofilipovic
40e94dc524 add poc memory tracker per procedure 2023-10-11 14:05:12 +02:00
antoniofilipovic
8ba34ef8f0 revert back cmakelists 2023-10-11 13:57:31 +02:00
antoniofilipovic
df312387cc revert back cmakelists 2023-10-11 13:56:49 +02:00
antoniofilipovic
94d5e22dcd introduce tracking per thread, add working test 2023-10-11 13:55:42 +02:00
antoniofilipovic
11ee19516a add limits 2023-10-10 18:51:24 +02:00
antoniofilipovic
e6f854396c enable memory tracker per thread id 2023-10-10 16:26:56 +02:00
antoniofilipovic
afdf38e0c8 add per query memory limit 2023-10-09 12:13:25 +02:00
antoniofilipovic
956b95a95c remove unnecessary includes 2023-10-06 16:59:46 +02:00
antoniofilipovic
46df649c65 revert checks 2023-10-06 16:55:08 +02:00
antoniofilipovic
379ab47866 cleanup memory_control, remove per query tracker 2023-10-06 16:42:44 +02:00
antoniofilipovic
80689a8337 remove virtual memory tracker 2023-10-06 16:33:57 +02:00
antoniofilipovic
771982be05 remove old memory tracker 2023-10-06 16:30:55 +02:00
antoniofilipovic
ba1a9d3045 fix cmakelists, remove comments 2023-10-06 16:26:23 +02:00
antoniofilipovic
0fc2355d8d fix tests 2023-10-06 13:46:01 +02:00
antoniofilipovic
76d4790b81 comment out interpreter jemalloc stats 2023-10-06 12:22:18 +02:00
antoniofilipovic
32682fa463 merge master 2023-10-06 10:56:06 +02:00
antoniofilipovic
f4e4fdb754 add jemalloc to libs 2023-10-06 10:47:47 +02:00
Andi
2fd34489af Add mgbench support for disk storage and analytical mode (#1286)
* Add mgbench support for disk storage and analytical mode
2023-10-06 10:19:29 +02:00
Gareth Andrew Lloyd
3cc2bc2791 Refactor interpreter to support multiple distributed clocks (Part 1) (#1281)
* Interpreter transaction ID decoupled from storage transaction ID
* Transactional scope for indices, statistics and constraints
* Storage::Accessor now has 2 modes (unique and shared)
* Introduced ResourceLock to fix pthread mutex problems
* Split InfoQuery in two: non-transactional SystemInfoQuery and transactional DatabaseInfoQuery
* Replicable and durable statistics
* Bumped WAL/Snapshot versions
* Initial implementation of the Lamport clock

---------

Co-authored-by: Andreja Tonev <andreja.tonev@memgraph.io>
2023-10-05 16:58:39 +02:00
antoniofilipovic
2c2c55abf4 test memory control 2023-10-05 09:42:10 +02:00
antoniofilipovic
32e744f09f comment unnecessary atomics 2023-10-04 10:07:09 +02:00
antoniofilipovic
e10daca67a add few improvements 2023-09-29 19:22:11 +02:00
Gareth Andrew Lloyd
d71b6a5007 Refactor replication client/server (#1311) 2023-09-29 11:21:42 +01:00
antoniofilipovic
5659ff21a8 fix flakly behavior on sigterm on tests 2023-09-28 17:35:55 +02:00
antoniofilipovic
817688d63d add working versions for asan tests 2023-09-28 15:54:13 +02:00
antoniofilipovic
3e2a5c2744 clean up memory control 2023-09-27 14:14:35 +02:00
antoniofilipovic
4f9c8b661e add arenas to cmakelist 2023-09-26 14:50:48 +02:00
antoniofilipovic
9e83a37873 add working tracker 2023-09-26 14:50:01 +02:00
Andi
61ac7e1b11 Add --storage-mode flag (#1282)
* Add --storage-mode flag
2023-09-26 14:47:30 +02:00
Andi
efdf7baea0 Refactor mgbench 2023-09-22 19:05:16 +02:00
antoniofilipovic
a0047672cb Add fully working version with jemalloc hook memory tracker
This commit introduces memory tracker with jemalloc extent hooks which fully works in case when jemalloc config is following:
MALLOC_CONF="retain:false,percpu_arena:percpu,oversize_threshold:1000000000000,muzzy_decay_ms:0,dirty_decay_ms:0" \
./configure \
    --disable-cxx \
    $COMMON_CONFIGURE_FLAGS \
    --with-malloc-conf="retain:false,percpu_arena:percpu,oversize_threshold:1000000000000,muzzy_decay_ms:0,dirty_decay_ms:0"

This config will for jemalloc not to use lazy purge or MADV_FREE(muzzy_decay_ms=0 and dirty_decay_ms=0), it will force jemalloc not to use
custom arena for huge allocations (oversize_threshold) and it will force jemalloc not extend virtual memory indefinitely (retain=false)
and therefore call alloc hook when allocation actually takes place.

Only problem is if we do huge allocations which are not mapped on alloc directly, in that case jemalloc uses cache and we can overcounter
allocation size.
2023-09-21 12:52:17 +02:00
Gareth Andrew Lloyd
eb4e2b019d Fix distinct, now doesn't impacts other aggregates (#1235)
Before a distinct on one aggregate would impact distinct on another
aggregate. Fixed the logical error and at the same time did some memory
optimisations.
2023-09-20 16:45:55 +01:00
Andi
1553fcb958 Improve deserialization performance
* Change std::stoull to std::from_chars
---------

Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
2023-09-20 14:25:17 +02:00
andrejtonev
bce48361ca Decoupling Interpreter from Storage (#1186)
Unique/global InterpreterContext that is Storage agnostic (has a reference to the DbmsHandler instead)

* InterpreterContext is no longer the owner of Storage
* New Database structure that handles Storage, Triggers, Streams
* Renamed SessinContextHandler to DbmsHandler and simplified the multi-tenant logic
* Added Gatekeeper and updated handlers to use it

---------

Co-authored-by: Gareth Lloyd <gareth.lloyd@memgraph.io>
2023-09-20 13:13:54 +02:00
antoniofilipovic
6a4780d2ac remove reducing memory usage on lazy purge 2023-09-20 12:06:40 +02:00
antoniofilipovic
362cbe8338 remove reducing memory usage on lazy purge 2023-09-20 12:05:35 +02:00
antoniofilipovic
e7dd60b1f0 add better version than current tracking 2023-09-19 17:02:47 +02:00
imilinovic
404cdf05d3 Add path pop to mgp API (#1249) 2023-09-19 12:37:55 +02:00
antoniofilipovic
fd9b653de9 add basic working version 1 2023-09-14 14:18:26 +02:00
Marko Barišić
b719f0744f Update v2.11 license date (#1247) v2.11.0 2023-09-13 07:19:11 -04:00
Josipmrden
440838c0e9 Add dependency check for e2e tests (#1240) 2023-09-12 11:34:08 -04:00
Josipmrden
79a3c5af8e Add manual performance benchmark execution (#1239) 2023-09-12 11:33:05 -04:00
Josipmrden
bf03b38e39 Remove gqlalchemy from stress tests (#1245) 2023-09-12 11:32:16 -04:00
antoniofilipovic
596760e655 add non working version of hooks alloc 2023-09-12 14:02:57 +02:00
gvolfing
fd63944493 Add --query-callable-mappings-path package default (#1203) 2023-09-11 15:12:14 -04:00
Gareth Andrew Lloyd
6694de2dfa Fix libkrb5 TRUE and FALSE macros leakage (#1243)
Co-authored-by: Marko Budiselić <marko.budiselic@memgraph.com>
2023-09-11 12:46:40 -04:00
andrejtonev
5e5f4ffc5d Add more runtime configurable settings (#1183)
server name, query timeout settings, log.level, log.to_stderr
2023-09-11 11:30:54 -04:00
antoniofilipovic
0f8ef3cdb2 add initial version of extent_hooks 2023-09-11 16:58:49 +02:00
Ante Pušić
060b9d1c16 [master < T1204] Add detailed operator info to PROFILE and EXPLAIN (#1204) 2023-09-11 14:34:27 +02:00
Ante Pušić
29a505cb38 Property lookup caching (#1168) 2023-09-11 13:03:54 +02:00