Commit Graph

2487 Commits

Author SHA1 Message Date
Matej Ferencevic
974e0d388d Bump version and update changelog 2019-10-23 10:02:50 +02:00
Matej Ferencevic
fd36263b07 Reduce memory blowup on each Pull
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2509
2019-10-23 10:01:39 +02:00
Matej Ferencevic
3d02f29bc3 Bump version and update changelog 2019-10-15 15:48:47 +02:00
Matej Ferencevic
7b7ce25608 Fix MonotonicBufferResource buffer growth
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2483
2019-10-15 15:41:27 +02:00
Lovro Lugovic
936dbbd34e An aggregate element's input expression can be NULL
Summary: For example, the aggregate element produced for `COUNT(*)` has its `value` set to `NULL`.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2463
2019-10-15 15:41:23 +02:00
Teon Banek
2acd5d80af Set -Werror=return-stack-address in CMakeLists
Summary:
This is a clang only flag, it promotes warnings to errors when returning
an address to something on the stack

Reviewers: mferencevic, ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2394
2019-10-15 15:41:19 +02:00
Teon Banek
702144bf09 Add missing include to scoped_profile.hpp
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2370
2019-10-15 15:41:16 +02:00
Teon Banek
50872cf893 Check for aggregations when planning RegexMatch
Reviewers: mferencevic, ipaljak, dsantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2344
2019-10-15 15:41:08 +02:00
Matej Ferencevic
6fad1cc43e Improve memory allocations in execution
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2340
2019-10-15 15:32:52 +02:00
Teon Banek
913bfb832b Correctly release PoolResource memory
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2341
2019-10-15 15:29:02 +02:00
Matej Ferencevic
851a0fcc36 Allocate initial execution memory
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2316
2019-10-15 15:28:54 +02:00
Teon Banek
84e5d70e2c Add PoolResource benchmarking in execution and skiplist
Summary:
With a pool allocator, lookups in STL set and map are up to 50% faster.
This is probably due to contiguous memory of pooled objects, i.e. nodes
of those containers. In some cases, the lookup outperforms the SkipList.
Insertions are also faster, though not as dramatically, up to 30%. This
does make a significant difference when the STL containers are used in a
single thread as they outperform the SkipList significantly.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2326
2019-10-15 15:24:07 +02:00
Teon Banek
dd853ded31 Fix a bug in Pool::Deallocate
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2325
2019-10-15 15:24:03 +02:00
Teon Banek
7cd8e1c079 Prepend MonotonicBufferResource::Buffer to an allocation
Summary:
This is a different scheme for setting up a bookkeeping object while
still supporting arbitrary allocation alignment requests. The previous
scheme was simpler as it always allocated a power of 2 bytes, but the
trade-off was increased memory usage. This should waste less memory.

Reviewers: mtomic, mferencevic, ipaljak

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2321
2019-10-15 15:24:00 +02:00
Teon Banek
d6e4f1e3b6 Add utilities for type checking awesome MG functions
Reviewers: mtomic, msantl

Reviewed By: mtomic, msantl

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2259
2019-10-15 15:23:53 +02:00
Teon Banek
d942934746 Update storage API docs with thrown exceptions
Summary:
The documentation includes `std` exceptions like `std::bad_alloc` or
`std::system_error`, for which there's probably nothing we can do. This
may seem unnecessary, but it will be really helpful when writing the C
API for interfacing with custom modules and plugins, as well as when
switching to storage v2 API.

In general, we should start updating the documentation of functions
which may throw exceptions. This ought to be enforced in code review, so
that the implementation and documentation are kept in sync.

Reviewers: mferencevic, mtomic, msantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2288
2019-10-15 15:22:09 +02:00
Matej Ferencevic
b5018b364b Replace bswap with proper endian functions
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2290
2019-10-15 15:22:03 +02:00
Matej Ferencevic
1cd26f40a2 Remove copy constructor/assignment from SkipList accessors
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2230
2019-10-15 15:21:51 +02:00
Teon Banek
e3f43f0212 Prepare utils/memory for switching to libstdc++-9
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2246
2019-10-15 15:21:21 +02:00
Teon Banek
750098548f cmake: Allow absolute paths in define_add_lcp
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2240
2019-10-15 15:21:17 +02:00
Teon Banek
b4435b801a Add non-template Value getters to PropertyValue
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2216
2019-10-15 15:21:10 +02:00
Matej Ferencevic
90ab54c35b Cleanup locks
Summary: Move RWLock and replace exceptions with `CHECK`s

Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2205
2019-10-15 15:20:29 +02:00
Matej Ferencevic
82883d90b6 Remove leftover audit config files 2019-07-17 15:06:33 +02:00
Matej Ferencevic
91afe2910c Fix compilation on CentOS 2019-07-17 14:45:17 +02:00
Matej Ferencevic
4024fc96df Prepare release v0.15.0
Reviewers: teon.banek, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2211
2019-07-17 14:06:27 +02:00
Matej Ferencevic
1bc062a946 Add changelog for version 0.15
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2210
2019-07-16 17:32:36 +02:00
Ivan Paljak
4acad5795b Expose the status of transaction within Raft
Summary:
For proper client interaction, we need to expose the (term_id, log_index)
pair for the transaction that's about to be replicated and we need to be able
to retrieve the status of a transaction defined by that pair. Transaction
status can be one of the following:

  1) REPLICATED (self-explanatory)
  2) WAITING (waiting for replication)
  3) ABORTED (self-explanatory)
  4) INVALID (received request with either invalid term_id or invalid log_index)

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2201
2019-07-16 13:53:16 +02:00
Matej Ferencevic
d5106c7233 Handle zero increment in counters
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2208
2019-07-16 12:35:35 +02:00
Marin Tomic
e7f897984f Fix wrong settings in gc bench
Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2207
2019-07-15 17:53:03 +02:00
Teon Banek
bf4f69f2e5 storage/v2: Reduce the (In|Out)Edges tuple to only EdgeAccessor
Summary:
The first 2 tuple elements are redundant as they are available through
EdgeAccessor and they needlessly complicate the usage of the API.

Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2200
2019-07-12 15:30:24 +02:00
Teon Banek
d4df7d9d60 Add helper methods for new/delete to Allocator
Summary:
The newly added methods are modeled after the ones being added in C++20 to
`std::pmr::polymorphic_allocator`.

Reviewers: mtomic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2197
2019-07-12 09:53:03 +02:00
Matej Ferencevic
15c959dfb5 Fix transaction abort object deletion in storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2198
2019-07-11 15:59:48 +02:00
Matej Ferencevic
d9a6775656 Move transaction to stack in storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2196
2019-07-11 15:04:10 +02:00
Teon Banek
c4c6febbc4 Move Parsing utils from query/common to frontend/parsing
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2195
2019-07-11 12:12:59 +02:00
Marin Tomic
8414479abe [StorageV2] Implement GC
Summary:
Here are some numbers from the benchmark:

```
(TOOLCHAIN) mtomic@poso:~/memgraph/build_release$ tests/benchmark/storage_v2_gc --num-threads 8
Config: NoGc, Time: 25.9836
Config: OnFinishGc, Time: 49.012
Config: 100msPeriodicGc, Time: 45.9856
Config: 1000msPeriodicGc, Time: 40.3094
```

```
(TOOLCHAIN) mtomic@poso:~/memgraph/build_release$ tests/benchmark/storage_v2_gc --num-threads 7
Config: NoGc, Time: 20.4256
Config: OnFinishGc, Time: 39.6669
Config: 100msPeriodicGc, Time: 30.7956
Config: 1000msPeriodicGc, Time: 35.128
```

It is not that bad if there is a core dedicated to doing garbage collection.

Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2168
2019-07-10 16:12:08 +02:00
Teon Banek
a0e1175634 Remove unused includes from operator.cpp
Reviewers: mtomic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2194
2019-07-10 15:15:53 +02:00
Marin Tomic
a4da2f123c [StorageV2] Fix object deletion in Abort function
Summary:
`lock_guard` is holding vertex lock while we're deleting the vertex,
and then it might try to unlock it in its destructor and access freed memory.

Reviewers: teon.banek, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2192
2019-07-10 10:20:58 +02:00
Matej Ferencevic
18e2f9eeda Make previous pointer atomic in storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2191
2019-07-09 16:26:10 +02:00
Matej Ferencevic
4d94aa740a Implement multiple previous pointer in storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2190
2019-07-09 14:24:28 +02:00
Teon Banek
5bc0be50c5 Remove GraphDbAccessor parameter from MakeCursor
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2189
2019-07-09 14:22:55 +02:00
Teon Banek
3a34d6fcdd Support all power of 2 alignments in MonotonicBufferResource
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2188
2019-07-09 14:22:05 +02:00
Ivan Paljak
2936312612 Add memgraph ha normal operation long running test
Summary:
The functionality of the test is the same as for single node Memgraph.
Locally, it seems to work fine. I'll update the apollo related files when I feel
a bit more certain that everything works locally.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2135
2019-07-09 13:59:56 +02:00
Teon Banek
76d8020169 Implement a PoolResource for allocations
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2147
2019-07-09 09:31:16 +02:00
Teon Banek
a417ef36f1 Use MemoryResource in SkipList
Summary:
This is a preparation step in case we want to have a custom allocator in
SkipList. For example, pool based allocator for SkipListNode.
Introduction of MemoryResource and removal of `calloc` has reduced the
performance a bit according to micro benchmarks. This performance hit is
not visible on benchmarks which do more concurrent operations.

Reviewers: mferencevic, mtomic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2140
2019-07-09 09:31:15 +02:00
Matej Ferencevic
b1f7bbf051 Implement properties for edges in storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2187
2019-07-08 16:55:47 +02:00
Matej Ferencevic
5b8e7ff432 Implement edges in storage v2
Summary:
This change implements full edges support in storage v2. Edges can be created
and deleted. Support for detach-deleting vertices is added and regular vertex
deletion verifies existance of edges.

Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2180
2019-07-08 16:06:57 +02:00
Matej Ferencevic
a2c5b5eac6 Temporarily disable LDBC test
Reviewers: mtomic, buda

Reviewed By: buda

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2178
2019-07-05 14:04:38 +02:00
Matej Ferencevic
069000a92a Fix CommitLog namespace in storage v2
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2179
2019-07-05 13:53:32 +02:00
Matej Ferencevic
26f14443c2 Move vertex deletion to storage accessor
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2174
2019-07-04 14:47:14 +02:00
Matej Ferencevic
6e6dff81e0 Split storage and vertex accessor into cpp in storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2172
2019-07-04 12:56:06 +02:00