1e0ac8ab8f
Summary: My dear fellow Memgraphians. It's friday afternoon, and I am as ready to pop as WAL is to get reviewed... What's done: - Vertices and Edges have global IDs, stored in `VersionList`. Main storage is now a concurrent map ID->vlist_ptr. - WriteAheadLog class added. It's based around buffering WAL::Op objects (elementraly DB changes) and periodically serializing and flusing them to disk. - Snapshot recovery refactored, WAL recovery added. Snapshot format changed again to include necessary info. - Durability testing completely reworked. What's not done (and should be when we decide how): - Old WAL file purging. - Config refactor (naming and organization). Will do when we discuss what we want. - Changelog and new feature documentation (both depending on the point above). - Better error handling and recovery feedback. Currently it's all returning bools, which is not fine-grained enough (neither for errors nor partial successes, also EOF is reported as a failure at the moment). - Moving the implementation of WAL stuff to .cpp where possible. - Not sure if there are transactions being created outside of `GraphDbAccessor` and it's `BuildIndex`. Need to look into. - True write-ahead logic (flag controlled): not committing a DB transaction if the WAL has not flushed it's data. We can discuss the gain/effort ratio for this feature. Reviewers: buda, mislav.bradac, teon.banek, dgleich Reviewed By: dgleich Subscribers: mtomic, pullbot Differential Revision: https://phabricator.memgraph.io/D958
92 lines
2.3 KiB
C++
92 lines
2.3 KiB
C++
#pragma once
|
|
|
|
#include <atomic>
|
|
#include <chrono>
|
|
#include <experimental/optional>
|
|
#include <mutex>
|
|
#include <thread>
|
|
#include <utility>
|
|
|
|
#include "glog/logging.h"
|
|
|
|
#include "threading/sync/spinlock.hpp"
|
|
|
|
/**
|
|
* A thread-safe ring buffer. Multi-producer, multi-consumer. Producers get
|
|
* blocked if the buffer is full. Consumers get returnd a nullopt. First in
|
|
* first out.
|
|
*
|
|
* @tparam TElement - type of element the buffer tracks.
|
|
*/
|
|
template <typename TElement>
|
|
class RingBuffer {
|
|
public:
|
|
RingBuffer(int capacity) : capacity_(capacity) {
|
|
buffer_ = new TElement[capacity_];
|
|
}
|
|
|
|
RingBuffer(const RingBuffer &) = delete;
|
|
RingBuffer(RingBuffer &&) = delete;
|
|
RingBuffer &operator=(const RingBuffer &) = delete;
|
|
RingBuffer &operator=(RingBuffer &&) = delete;
|
|
|
|
~RingBuffer() {
|
|
delete[] buffer_;
|
|
}
|
|
|
|
/**
|
|
* Emplaces a new element into the buffer. This call blocks until space in the
|
|
* buffer is available. If multiple threads are waiting for space to become
|
|
* available, there are no order-of-entrace guarantees.
|
|
*/
|
|
template <typename... TArgs>
|
|
void emplace(TArgs &&... args) {
|
|
while (true) {
|
|
{
|
|
std::lock_guard<SpinLock> guard(lock_);
|
|
if (size_ < capacity_) {
|
|
buffer_[write_pos_++] = TElement(std::forward<TArgs>(args)...);
|
|
write_pos_ %= capacity_;
|
|
size_++;
|
|
return;
|
|
}
|
|
}
|
|
|
|
// Log a warning approximately once per second if buffer is full.
|
|
LOG_EVERY_N(WARNING, 4000) << "RingBuffer full: worker waiting";
|
|
// Sleep time determined using tests/benchmark/ring_buffer.cpp
|
|
std::this_thread::sleep_for(std::chrono::microseconds(250));
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Removes and returns the oldest element from the buffer. If the buffer is
|
|
* empty, nullopt is returned.
|
|
*/
|
|
std::experimental::optional<TElement> pop() {
|
|
std::lock_guard<SpinLock> guard(lock_);
|
|
if (size_ == 0) return std::experimental::nullopt;
|
|
size_--;
|
|
std::experimental::optional<TElement> result(
|
|
std::move(buffer_[read_pos_++]));
|
|
read_pos_ %= capacity_;
|
|
return result;
|
|
}
|
|
|
|
/** Removes all elements from the buffer. */
|
|
void clear() {
|
|
std::lock_guard<SpinLock> guard(lock_);
|
|
read_pos_ = 0;
|
|
write_pos_ = 0;
|
|
size_ = 0;
|
|
}
|
|
|
|
private:
|
|
int capacity_;
|
|
TElement *buffer_;
|
|
SpinLock lock_;
|
|
int read_pos_{0};
|
|
int write_pos_{0};
|
|
int size_{0};
|
|
};
|