Compare commits

..

32 Commits

Author SHA1 Message Date
antoniofilipovic
40e94dc524 add poc memory tracker per procedure 2023-10-11 14:05:12 +02:00
antoniofilipovic
8ba34ef8f0 revert back cmakelists 2023-10-11 13:57:31 +02:00
antoniofilipovic
df312387cc revert back cmakelists 2023-10-11 13:56:49 +02:00
antoniofilipovic
94d5e22dcd introduce tracking per thread, add working test 2023-10-11 13:55:42 +02:00
antoniofilipovic
11ee19516a add limits 2023-10-10 18:51:24 +02:00
antoniofilipovic
e6f854396c enable memory tracker per thread id 2023-10-10 16:26:56 +02:00
antoniofilipovic
afdf38e0c8 add per query memory limit 2023-10-09 12:13:25 +02:00
antoniofilipovic
956b95a95c remove unnecessary includes 2023-10-06 16:59:46 +02:00
antoniofilipovic
46df649c65 revert checks 2023-10-06 16:55:08 +02:00
antoniofilipovic
379ab47866 cleanup memory_control, remove per query tracker 2023-10-06 16:42:44 +02:00
antoniofilipovic
80689a8337 remove virtual memory tracker 2023-10-06 16:33:57 +02:00
antoniofilipovic
771982be05 remove old memory tracker 2023-10-06 16:30:55 +02:00
antoniofilipovic
ba1a9d3045 fix cmakelists, remove comments 2023-10-06 16:26:23 +02:00
antoniofilipovic
0fc2355d8d fix tests 2023-10-06 13:46:01 +02:00
antoniofilipovic
76d4790b81 comment out interpreter jemalloc stats 2023-10-06 12:22:18 +02:00
antoniofilipovic
32682fa463 merge master 2023-10-06 10:56:06 +02:00
antoniofilipovic
f4e4fdb754 add jemalloc to libs 2023-10-06 10:47:47 +02:00
antoniofilipovic
2c2c55abf4 test memory control 2023-10-05 09:42:10 +02:00
antoniofilipovic
32e744f09f comment unnecessary atomics 2023-10-04 10:07:09 +02:00
antoniofilipovic
e10daca67a add few improvements 2023-09-29 19:22:11 +02:00
antoniofilipovic
5659ff21a8 fix flakly behavior on sigterm on tests 2023-09-28 17:35:55 +02:00
antoniofilipovic
817688d63d add working versions for asan tests 2023-09-28 15:54:13 +02:00
antoniofilipovic
3e2a5c2744 clean up memory control 2023-09-27 14:14:35 +02:00
antoniofilipovic
4f9c8b661e add arenas to cmakelist 2023-09-26 14:50:48 +02:00
antoniofilipovic
9e83a37873 add working tracker 2023-09-26 14:50:01 +02:00
antoniofilipovic
a0047672cb Add fully working version with jemalloc hook memory tracker
This commit introduces memory tracker with jemalloc extent hooks which fully works in case when jemalloc config is following:
MALLOC_CONF="retain:false,percpu_arena:percpu,oversize_threshold:1000000000000,muzzy_decay_ms:0,dirty_decay_ms:0" \
./configure \
    --disable-cxx \
    $COMMON_CONFIGURE_FLAGS \
    --with-malloc-conf="retain:false,percpu_arena:percpu,oversize_threshold:1000000000000,muzzy_decay_ms:0,dirty_decay_ms:0"

This config will for jemalloc not to use lazy purge or MADV_FREE(muzzy_decay_ms=0 and dirty_decay_ms=0), it will force jemalloc not to use
custom arena for huge allocations (oversize_threshold) and it will force jemalloc not extend virtual memory indefinitely (retain=false)
and therefore call alloc hook when allocation actually takes place.

Only problem is if we do huge allocations which are not mapped on alloc directly, in that case jemalloc uses cache and we can overcounter
allocation size.
2023-09-21 12:52:17 +02:00
antoniofilipovic
6a4780d2ac remove reducing memory usage on lazy purge 2023-09-20 12:06:40 +02:00
antoniofilipovic
362cbe8338 remove reducing memory usage on lazy purge 2023-09-20 12:05:35 +02:00
antoniofilipovic
e7dd60b1f0 add better version than current tracking 2023-09-19 17:02:47 +02:00
antoniofilipovic
fd9b653de9 add basic working version 1 2023-09-14 14:18:26 +02:00
antoniofilipovic
596760e655 add non working version of hooks alloc 2023-09-12 14:02:57 +02:00
antoniofilipovic
0f8ef3cdb2 add initial version of extent_hooks 2023-09-11 16:58:49 +02:00
860 changed files with 12874 additions and 52871 deletions

View File

@@ -6,7 +6,6 @@ Checks: '*,
-altera-unroll-loops, -altera-unroll-loops,
-android-*, -android-*,
-cert-err58-cpp, -cert-err58-cpp,
-cppcoreguidelines-avoid-do-while,
-cppcoreguidelines-avoid-c-arrays, -cppcoreguidelines-avoid-c-arrays,
-cppcoreguidelines-avoid-goto, -cppcoreguidelines-avoid-goto,
-cppcoreguidelines-avoid-magic-numbers, -cppcoreguidelines-avoid-magic-numbers,
@@ -61,7 +60,6 @@ Checks: '*,
-readability-implicit-bool-conversion, -readability-implicit-bool-conversion,
-readability-magic-numbers, -readability-magic-numbers,
-readability-named-parameter, -readability-named-parameter,
-readability-identifier-length,
-misc-no-recursion, -misc-no-recursion,
-concurrency-mt-unsafe, -concurrency-mt-unsafe,
-bugprone-easily-swappable-parameters' -bugprone-easily-swappable-parameters'

View File

@@ -33,4 +33,4 @@ for file in $modified_files; do
fi fi
done; done;
exit ${FAIL} return ${FAIL}

View File

@@ -1,18 +1,19 @@
--- ---
name: Bug report name: Bug report
about: Create a report to help us improve about: Create a report to help us improve
title: "" title: "[BUG] "
labels: bug labels: bug
assignees: gitbuda assignees: gitbuda, antonio2368
--- ---
**Memgraph version** **Memgraph version**
Which version did you use? Which version did you use?
**Environment** **Environment**
Some information about the environment you are using Memgraph on: operating Some information about the environment you are using Memgraph on: operating
system, architecture (ARM, x86), how do you connect, with or without docker, system, how do you connect, with or without docker, which driver etc.
which driver etc.
**Describe the bug** **Describe the bug**
A clear and concise description of what the bug is. A clear and concise description of what the bug is.
@@ -21,7 +22,6 @@ A clear and concise description of what the bug is.
Steps to reproduce the behavior: Steps to reproduce the behavior:
1. Run the following query '...' 1. Run the following query '...'
2. Click on '....' 2. Click on '....'
3. ... IDEALLY: link to the workload info (DATASET & QUERIES) ...
**Expected behavior** **Expected behavior**
A clear and concise description of what you expected to happen. A clear and concise description of what you expected to happen.
@@ -32,11 +32,3 @@ your problem.
**Additional context** **Additional context**
Add any other context about the problem here. Add any other context about the problem here.
**Verification Environment**
Once we fix it, what do you need to verify the fix?
Do you need:
* Plain memgraph package -> for which Linux?
* Plain memgraph Docker image?
* Which architecture do you use ARM | x86?
* Full Memgraph platform?

View File

@@ -16,7 +16,7 @@ jobs:
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -59,7 +59,7 @@ jobs:
source ve3/bin/activate source ve3/bin/activate
pip install -r requirements.txt pip install -r requirements.txt
./main.py --benchmark-name "macro_benchmark" \ ./main.py --benchmark-name "macro_benchmark" \
--benchmark-results "../../tests/macro_benchmark/.harness_summary" \ --benchmark-results-path "../../tests/macro_benchmark/.harness_summary" \
--github-run-id "${{ github.run_id }}" \ --github-run-id "${{ github.run_id }}" \
--github-run-number "${{ github.run_number }}" \ --github-run-number "${{ github.run_number }}" \
--head-branch-name "${{ env.BRANCH_NAME }}" --head-branch-name "${{ env.BRANCH_NAME }}"
@@ -73,8 +73,6 @@ jobs:
./benchmark.py vendor-native --num-workers-for-benchmark 1 --export-results benchmark_high_write_set_property.json high_write_set_property ./benchmark.py vendor-native --num-workers-for-benchmark 1 --export-results benchmark_high_write_set_property.json high_write_set_property
./benchmark.py vendor-native --num-workers-for-benchmark 12 --export-results cartesian.json cartesian
- name: Upload mgbench results - name: Upload mgbench results
run: | run: |
cd tools/bench-graph-client cd tools/bench-graph-client
@@ -82,25 +80,19 @@ jobs:
source ve3/bin/activate source ve3/bin/activate
pip install -r requirements.txt pip install -r requirements.txt
./main.py --benchmark-name "mgbench" \ ./main.py --benchmark-name "mgbench" \
--benchmark-results "../../tests/mgbench/benchmark_pokec.json" \ --benchmark-results-path "../../tests/mgbench/benchmark_pokec.json" \
--github-run-id "${{ github.run_id }}" \ --github-run-id "${{ github.run_id }}" \
--github-run-number "${{ github.run_number }}" \ --github-run-number "${{ github.run_number }}" \
--head-branch-name "${{ env.BRANCH_NAME }}" --head-branch-name "${{ env.BRANCH_NAME }}"
./main.py --benchmark-name "supernode" \ ./main.py --benchmark-name "supernode" \
--benchmark-results "../../tests/mgbench/benchmark_supernode.json" \ --benchmark-results-path "../../tests/mgbench/benchmark_supernode.json" \
--github-run-id "${{ github.run_id }}" \ --github-run-id "${{ github.run_id }}" \
--github-run-number "${{ github.run_number }}" \ --github-run-number "${{ github.run_number }}" \
--head-branch-name "${{ env.BRANCH_NAME }}" --head-branch-name "${{ env.BRANCH_NAME }}"
./main.py --benchmark-name "high_write_set_property" \ ./main.py --benchmark-name "high_write_set_property" \
--benchmark-results "../../tests/mgbench/benchmark_high_write_set_property.json" \ --benchmark-results-path "../../tests/mgbench/benchmark_high_write_set_property.json" \
--github-run-id "${{ github.run_id }}" \
--github-run-number "${{ github.run_number }}" \
--head-branch-name "${{ env.BRANCH_NAME }}"
./main.py --benchmark-name "cartesian" \
--benchmark-results "../../tests/mgbench/cartesian.json" \
--github-run-id "${{ github.run_id }}" \ --github-run-id "${{ github.run_id }}" \
--github-run-number "${{ github.run_number }}" \ --github-run-number "${{ github.run_number }}" \
--head-branch-name "${{ env.BRANCH_NAME }}" --head-branch-name "${{ env.BRANCH_NAME }}"

View File

@@ -27,7 +27,7 @@ jobs:
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -43,7 +43,7 @@ jobs:
# Build community binaries. # Build community binaries.
cd build cd build
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DMG_ENTERPRISE=OFF .. cmake -DCMAKE_BUILD_TYPE=release -DMG_ENTERPRISE=OFF ..
make -j$THREADS make -j$THREADS
- name: Run unit tests - name: Run unit tests
@@ -65,7 +65,7 @@ jobs:
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -96,7 +96,7 @@ jobs:
- name: Python code analysis - name: Python code analysis
run: | run: |
CHANGED_FILES=$(git diff -U0 ${{ env.BASE_BRANCH }}... --name-only --diff-filter=d) CHANGED_FILES=$(git diff -U0 ${{ env.BASE_BRANCH }}... --name-only)
for file in ${CHANGED_FILES}; do for file in ${CHANGED_FILES}; do
echo ${file} echo ${file}
if [[ ${file} == *.py ]]; then if [[ ${file} == *.py ]]; then
@@ -137,9 +137,9 @@ jobs:
tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu
- name: Save code coverage - name: Save code coverage
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: "Code coverage(Code analysis)" name: "Code coverage"
path: tools/github/generated/code_coverage.tar.gz path: tools/github/generated/code_coverage.tar.gz
- name: Run clang-tidy - name: Run clang-tidy
@@ -162,7 +162,7 @@ jobs:
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -208,9 +208,9 @@ jobs:
./cppcheck_and_clang_format diff ./cppcheck_and_clang_format diff
- name: Save cppcheck and clang-format errors - name: Save cppcheck and clang-format errors
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: "Code coverage(Debug build)" name: "Code coverage"
path: tools/github/cppcheck_and_clang_format.txt path: tools/github/cppcheck_and_clang_format.txt
release_build: release_build:
@@ -223,12 +223,16 @@ jobs:
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
fetch-depth: 0 fetch-depth: 0
- name: Check e2e service dependencies
run: |
cd tests/e2e
./dependency_check.sh
- name: Build release binaries - name: Build release binaries
run: | run: |
@@ -240,18 +244,16 @@ jobs:
# Build release binaries. # Build release binaries.
cd build cd build
cmake -DCMAKE_BUILD_TYPE=Release .. cmake -DCMAKE_BUILD_TYPE=release ..
make -j$THREADS make -j$THREADS
- name: Run GQL Behave tests - name: Run GQL Behave tests
run: | run: |
cd tests cd tests/gql_behave
./setup.sh /opt/toolchain-v4/activate
cd gql_behave
./continuous_integration ./continuous_integration
- name: Save quality assurance status - name: Save quality assurance status
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: "GQL Behave Status" name: "GQL Behave Status"
path: | path: |
@@ -267,13 +269,6 @@ jobs:
cd build cd build
ctest -R memgraph__unit --output-on-failure -j$THREADS ctest -R memgraph__unit --output-on-failure -j$THREADS
- name: Ensure Kafka and Pulsar are up
run: |
cd tests/e2e/streams/kafka
docker-compose up -d
cd ../pulsar
docker-compose up -d
- name: Run e2e tests - name: Run e2e tests
run: | run: |
cd tests cd tests
@@ -282,13 +277,6 @@ jobs:
cd e2e cd e2e
./run.sh ./run.sh
- name: Ensure Kafka and Pulsar are down
run: |
cd tests/e2e/streams/kafka
docker-compose down
cd ../pulsar
docker-compose down
- name: Run stress test (plain) - name: Run stress test (plain)
run: | run: |
cd tests/stress cd tests/stress
@@ -322,128 +310,16 @@ jobs:
cpack -G DEB --config ../CPackConfig.cmake cpack -G DEB --config ../CPackConfig.cmake
- name: Save enterprise DEB package - name: Save enterprise DEB package
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: "Enterprise DEB package" name: "Enterprise DEB package"
path: build/output/memgraph*.deb path: build/output/memgraph*.deb
- name: Save test data - name: Save test data
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
if: always() if: always()
with: with:
name: "Test data(Release build)" name: "Test data"
path: |
# multiple paths could be defined
build/logs
experimental_build_ha:
name: "High availability build"
runs-on: [self-hosted, Linux, X64, Diff]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v4
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build release binaries
run: |
source /opt/toolchain-v4/activate
./init
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DMG_EXPERIMENTAL_HIGH_AVAILABILITY=ON ..
make -j$THREADS
- name: Run unit tests
run: |
source /opt/toolchain-v4/activate
cd build
ctest -R memgraph__unit --output-on-failure -j$THREADS
- name: Run e2e tests
if: false
run: |
cd tests
./setup.sh /opt/toolchain-v4/activate
source ve3/bin/activate_e2e
cd e2e
./run.sh "Coordinator"
./run.sh "Client initiated failover"
./run.sh "Uninitialized cluster"
- name: Save test data
uses: actions/upload-artifact@v4
if: always()
with:
name: "Test data(High availability build)"
path: |
# multiple paths could be defined
build/logs
experimental_build_mt:
name: "MultiTenancy replication build"
runs-on: [self-hosted, Linux, X64, Diff]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v4
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build MT replication experimental binaries.
cd build
cmake -DCMAKE_BUILD_TYPE=Release -D MG_EXPERIMENTAL_REPLICATION_MULTITENANCY=ON ..
make -j$THREADS
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
ctest -R memgraph__unit --output-on-failure -j$THREADS
- name: Run e2e tests
if: false
run: |
cd tests
./setup.sh /opt/toolchain-v4/activate
source ve3/bin/activate_e2e
cd e2e
# Just the replication based e2e tests
./run.sh "Replicate multitenancy"
./run.sh "Show"
./run.sh "Show while creating invalid state"
./run.sh "Delete edge replication"
./run.sh "Read-write benchmark"
./run.sh "Index replication"
./run.sh "Constraints"
- name: Save test data
uses: actions/upload-artifact@v4
if: always()
with:
name: "Test data(MultiTenancy replication build)"
path: | path: |
# multiple paths could be defined # multiple paths could be defined
build/logs build/logs
@@ -459,7 +335,7 @@ jobs:
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -473,21 +349,16 @@ jobs:
./init ./init
# Build only memgraph release binarie. # Build only memgraph release binarie.
cd build cd build
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo .. cmake -DCMAKE_BUILD_TYPE=release ..
make -j$THREADS memgraph make -j$THREADS memgraph
- name: Refresh Jepsen Cluster
run: |
cd tests/jepsen
./run.sh cluster-refresh
- name: Run Jepsen tests - name: Run Jepsen tests
run: | run: |
cd tests/jepsen cd tests/jepsen
./run.sh test-all-individually --binary ../../build/memgraph --ignore-run-stdout-logs --ignore-run-stderr-logs ./run.sh test-all-individually --binary ../../build/memgraph --ignore-run-stdout-logs --ignore-run-stderr-logs
- name: Save Jepsen report - name: Save Jepsen report
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
if: ${{ always() }} if: ${{ always() }}
with: with:
name: "Jepsen Report" name: "Jepsen Report"
@@ -503,7 +374,7 @@ jobs:
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)

View File

@@ -14,7 +14,7 @@ jobs:
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)

View File

@@ -1,4 +1,4 @@
name: Package memgraph name: Package All
# TODO(gitbuda): Cleanup docker container if GHA job was canceled. # TODO(gitbuda): Cleanup docker container if GHA job was canceled.
@@ -6,268 +6,229 @@ on:
workflow_dispatch: workflow_dispatch:
inputs: inputs:
memgraph_version: memgraph_version:
description: "Memgraph version to upload as. Leave this field empty if you don't want to upload binaries to S3. Format: 'X.Y.Z'" description: "Memgraph version to upload as. If empty upload is skipped. Format: 'X.Y.Z'"
required: false required: false
build_type:
type: choice
description: "Memgraph Build type. Default value is Release"
default: 'Release'
options:
- Release
- RelWithDebInfo
target_os:
type: choice
description: "Target OS for which memgraph will be packaged. Select 'all' if you want to package for every listed OS. Default is Ubuntu 22.04"
default: 'ubuntu-22_04'
options:
- all
- amzn-2
- centos-7
- centos-9
- debian-10
- debian-11
- debian-11-arm
- debian-11-platform
- docker
- fedora-36
- ubuntu-18_04
- ubuntu-20_04
- ubuntu-22_04
- ubuntu-22_04-arm
jobs: jobs:
amzn-2:
if: ${{ github.event.inputs.target_os == 'amzn-2' || github.event.inputs.target_os == 'all' }}
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v4
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package amzn-2 ${{ github.event.inputs.build_type }}
- name: "Upload package"
uses: actions/upload-artifact@v4
with:
name: amzn-2
path: build/output/amzn-2/memgraph*.rpm
centos-7: centos-7:
if: ${{ github.event.inputs.target_os == 'centos-7' || github.event.inputs.target_os == 'all' }}
runs-on: [self-hosted, DockerMgBuild, X64] runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60 timeout-minutes: 60
steps: steps:
- name: "Set up repository" - name: "Set up repository"
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
fetch-depth: 0 # Required because of release/get_version.py fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package" - name: "Build package"
run: | run: |
./release/package/run.sh package centos-7 ${{ github.event.inputs.build_type }} ./release/package/run.sh package centos-7
- name: "Upload package" - name: "Upload package"
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: centos-7 name: centos-7
path: build/output/centos-7/memgraph*.rpm path: build/output/centos-7/memgraph*.rpm
centos-9: centos-9:
if: ${{ github.event.inputs.target_os == 'centos-9' || github.event.inputs.target_os == 'all' }}
runs-on: [self-hosted, DockerMgBuild, X64] runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60 timeout-minutes: 60
steps: steps:
- name: "Set up repository" - name: "Set up repository"
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
fetch-depth: 0 # Required because of release/get_version.py fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package" - name: "Build package"
run: | run: |
./release/package/run.sh package centos-9 ${{ github.event.inputs.build_type }} ./release/package/run.sh package centos-9
- name: "Upload package" - name: "Upload package"
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: centos-9 name: centos-9
path: build/output/centos-9/memgraph*.rpm path: build/output/centos-9/memgraph*.rpm
debian-10: debian-10:
if: ${{ github.event.inputs.target_os == 'debian-10' || github.event.inputs.target_os == 'all' }}
runs-on: [self-hosted, DockerMgBuild, X64] runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60 timeout-minutes: 60
steps: steps:
- name: "Set up repository" - name: "Set up repository"
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
fetch-depth: 0 # Required because of release/get_version.py fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package" - name: "Build package"
run: | run: |
./release/package/run.sh package debian-10 ${{ github.event.inputs.build_type }} ./release/package/run.sh package debian-10
- name: "Upload package" - name: "Upload package"
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: debian-10 name: debian-10
path: build/output/debian-10/memgraph*.deb path: build/output/debian-10/memgraph*.deb
debian-11: debian-11:
if: ${{ github.event.inputs.target_os == 'debian-11' || github.event.inputs.target_os == 'all' }}
runs-on: [self-hosted, DockerMgBuild, X64] runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60 timeout-minutes: 60
steps: steps:
- name: "Set up repository" - name: "Set up repository"
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
fetch-depth: 0 # Required because of release/get_version.py fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package" - name: "Build package"
run: | run: |
./release/package/run.sh package debian-11 ${{ github.event.inputs.build_type }} ./release/package/run.sh package debian-11
- name: "Upload package" - name: "Upload package"
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: debian-11 name: debian-11
path: build/output/debian-11/memgraph*.deb path: build/output/debian-11/memgraph*.deb
debian-11-arm:
if: ${{ github.event.inputs.target_os == 'debian-11-arm' || github.event.inputs.target_os == 'all' }}
runs-on: [self-hosted, DockerMgBuild, ARM64, strange]
timeout-minutes: 120
steps:
- name: "Set up repository"
uses: actions/checkout@v4
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package debian-11-arm ${{ github.event.inputs.build_type }}
- name: "Upload package"
uses: actions/upload-artifact@v4
with:
name: debian-11-aarch64
path: build/output/debian-11-arm/memgraph*.deb
debian-11-platform:
if: ${{ github.event.inputs.target_os == 'debian-11-platform' || github.event.inputs.target_os == 'all' }}
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v4
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package debian-11 ${{ github.event.inputs.build_type }} --for-platform
- name: "Upload package"
uses: actions/upload-artifact@v4
with:
name: debian-11-platform
path: build/output/debian-11/memgraph*.deb
docker: docker:
if: ${{ github.event.inputs.target_os == 'docker' || github.event.inputs.target_os == 'all' }}
runs-on: [self-hosted, DockerMgBuild, X64] runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60 timeout-minutes: 60
steps: steps:
- name: "Set up repository" - name: "Set up repository"
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
fetch-depth: 0 # Required because of release/get_version.py fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package" - name: "Build package"
run: | run: |
cd release/package cd release/package
./run.sh package debian-11 ${{ github.event.inputs.build_type }} --for-docker ./run.sh package debian-11 --for-docker
./run.sh docker ./run.sh docker
- name: "Upload package" - name: "Upload package"
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: docker name: docker
path: build/output/docker/memgraph*.tar.gz path: build/output/docker/memgraph*.tar.gz
fedora-36: ubuntu-1804:
if: ${{ github.event.inputs.target_os == 'fedora-36' || github.event.inputs.target_os == 'all' }}
runs-on: [self-hosted, DockerMgBuild, X64] runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60 timeout-minutes: 60
steps: steps:
- name: "Set up repository" - name: "Set up repository"
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
fetch-depth: 0 # Required because of release/get_version.py fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package" - name: "Build package"
run: | run: |
./release/package/run.sh package fedora-36 ${{ github.event.inputs.build_type }} ./release/package/run.sh package ubuntu-18.04
- name: "Upload package" - name: "Upload package"
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with:
name: fedora-36
path: build/output/fedora-36/memgraph*.rpm
ubuntu-18_04:
if: ${{ github.event.inputs.target_os == 'ubuntu-18_04' || github.event.inputs.target_os == 'all' }}
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v4
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package ubuntu-18.04 ${{ github.event.inputs.build_type }}
- name: "Upload package"
uses: actions/upload-artifact@v4
with: with:
name: ubuntu-18.04 name: ubuntu-18.04
path: build/output/ubuntu-18.04/memgraph*.deb path: build/output/ubuntu-18.04/memgraph*.deb
ubuntu-20_04: ubuntu-2004:
if: ${{ github.event.inputs.target_os == 'ubuntu-20_04' || github.event.inputs.target_os == 'all' }}
runs-on: [self-hosted, DockerMgBuild, X64] runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60 timeout-minutes: 60
steps: steps:
- name: "Set up repository" - name: "Set up repository"
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
fetch-depth: 0 # Required because of release/get_version.py fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package" - name: "Build package"
run: | run: |
./release/package/run.sh package ubuntu-20.04 ${{ github.event.inputs.build_type }} ./release/package/run.sh package ubuntu-20.04
- name: "Upload package" - name: "Upload package"
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: ubuntu-20.04 name: ubuntu-20.04
path: build/output/ubuntu-20.04/memgraph*.deb path: build/output/ubuntu-20.04/memgraph*.deb
ubuntu-22_04: ubuntu-2204:
if: ${{ github.event.inputs.target_os == 'ubuntu-22_04' || github.event.inputs.target_os == 'all' }}
runs-on: [self-hosted, DockerMgBuild, X64] runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60 timeout-minutes: 60
steps: steps:
- name: "Set up repository" - name: "Set up repository"
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
fetch-depth: 0 # Required because of release/get_version.py fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package" - name: "Build package"
run: | run: |
./release/package/run.sh package ubuntu-22.04 ${{ github.event.inputs.build_type }} ./release/package/run.sh package ubuntu-22.04
- name: "Upload package" - name: "Upload package"
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: ubuntu-22.04 name: ubuntu-22.04
path: build/output/ubuntu-22.04/memgraph*.deb path: build/output/ubuntu-22.04/memgraph*.deb
ubuntu-22_04-arm: debian-11-platform:
if: ${{ github.event.inputs.target_os == 'ubuntu-22_04-arm' || github.event.inputs.target_os == 'all' }} runs-on: [self-hosted, DockerMgBuild, X64]
runs-on: [self-hosted, DockerMgBuild, ARM64, strange] timeout-minutes: 60
timeout-minutes: 120
steps: steps:
- name: "Set up repository" - name: "Set up repository"
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
fetch-depth: 0 # Required because of release/get_version.py fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package" - name: "Build package"
run: | run: |
./release/package/run.sh package ubuntu-22.04-arm ${{ github.event.inputs.build_type }} ./release/package/run.sh package debian-11 --for-platform
- name: "Upload package" - name: "Upload package"
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with:
name: debian-11-platform
path: build/output/debian-11/memgraph*.deb
fedora-36:
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package fedora-36
- name: "Upload package"
uses: actions/upload-artifact@v3
with:
name: fedora-36
path: build/output/fedora-36/memgraph*.rpm
amzn-2:
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package amzn-2
- name: "Upload package"
uses: actions/upload-artifact@v3
with:
name: amzn-2
path: build/output/amzn-2/memgraph*.rpm
debian-11-arm:
runs-on: [self-hosted, DockerMgBuild, ARM64, strange]
timeout-minutes: 120
steps:
- name: "Set up repository"
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package debian-11-arm
- name: "Upload package"
uses: actions/upload-artifact@v3
with:
name: debian-11-aarch64
path: build/output/debian-11-arm/memgraph*.deb
ubuntu-2204-arm:
runs-on: [self-hosted, DockerMgBuild, ARM64, strange]
timeout-minutes: 120
steps:
- name: "Set up repository"
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package ubuntu-22.04-arm
- name: "Upload package"
uses: actions/upload-artifact@v3
with: with:
name: ubuntu-22.04-aarch64 name: ubuntu-22.04-aarch64
path: build/output/ubuntu-22.04-arm/memgraph*.deb path: build/output/ubuntu-22.04-arm/memgraph*.deb
@@ -275,11 +236,11 @@ jobs:
upload-to-s3: upload-to-s3:
# only run upload if we specified version. Allows for runs without upload # only run upload if we specified version. Allows for runs without upload
if: "${{ github.event.inputs.memgraph_version != '' }}" if: "${{ github.event.inputs.memgraph_version != '' }}"
needs: [amzn-2, centos-7, centos-9, debian-10, debian-11, debian-11-arm, debian-11-platform, docker, fedora-36, ubuntu-18_04, ubuntu-20_04, ubuntu-22_04, ubuntu-22_04-arm] needs: [centos-7, centos-9, debian-10, debian-11, docker, ubuntu-1804, ubuntu-2004, ubuntu-2204, debian-11-platform, fedora-36, amzn-2, debian-11-arm, ubuntu-2204-arm]
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Download artifacts - name: Download artifacts
uses: actions/download-artifact@v4 uses: actions/download-artifact@v3
with: with:
# name: # if name input parameter is not provided, all artifacts are downloaded # name: # if name input parameter is not provided, all artifacts are downloaded
# and put in directories named after each one. # and put in directories named after each one.

View File

@@ -14,7 +14,7 @@ jobs:
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -30,7 +30,7 @@ jobs:
# Build only memgraph release binaries. # Build only memgraph release binaries.
cd build cd build
cmake -DCMAKE_BUILD_TYPE=release .. cmake -DCMAKE_BUILD_TYPE=Release ..
make -j$THREADS make -j$THREADS
- name: Get branch name (merge) - name: Get branch name (merge)
@@ -52,8 +52,6 @@ jobs:
./benchmark.py vendor-native --num-workers-for-benchmark 1 --export-results benchmark_high_write_set_property.json high_write_set_property ./benchmark.py vendor-native --num-workers-for-benchmark 1 --export-results benchmark_high_write_set_property.json high_write_set_property
./benchmark.py vendor-native --num-workers-for-benchmark 12 --export-results benchmark_cartesian.json cartesian
- name: Upload benchmark results - name: Upload benchmark results
run: | run: |
cd tools/bench-graph-client cd tools/bench-graph-client
@@ -61,25 +59,19 @@ jobs:
source ve3/bin/activate source ve3/bin/activate
pip install -r requirements.txt pip install -r requirements.txt
./main.py --benchmark-name "mgbench" \ ./main.py --benchmark-name "mgbench" \
--benchmark-results "../../tests/mgbench/benchmark_result.json" \ --benchmark-results-path "../../tests/mgbench/benchmark_result.json" \
--github-run-id "${{ github.run_id }}" \ --github-run-id "${{ github.run_id }}" \
--github-run-number "${{ github.run_number }}" \ --github-run-number "${{ github.run_number }}" \
--head-branch-name "${{ env.BRANCH_NAME }}" --head-branch-name "${{ env.BRANCH_NAME }}"
./main.py --benchmark-name "supernode" \ ./main.py --benchmark-name "supernode" \
--benchmark-results "../../tests/mgbench/benchmark_supernode.json" \ --benchmark-results-path "../../tests/mgbench/benchmark_supernode.json" \
--github-run-id "${{ github.run_id }}" \ --github-run-id "${{ github.run_id }}" \
--github-run-number "${{ github.run_number }}" \ --github-run-number "${{ github.run_number }}" \
--head-branch-name "${{ env.BRANCH_NAME }}" --head-branch-name "${{ env.BRANCH_NAME }}"
./main.py --benchmark-name "high_write_set_property" \ ./main.py --benchmark-name "high_write_set_property" \
--benchmark-results "../../tests/mgbench/benchmark_high_write_set_property.json" \ --benchmark-results-path "../../tests/mgbench/benchmark_high_write_set_property.json" \
--github-run-id "${{ github.run_id }}" \
--github-run-number "${{ github.run_number }}" \
--head-branch-name "${{ env.BRANCH_NAME }}"
./main.py --benchmark-name "cartesian" \
--benchmark-results "../../tests/mgbench/cartesian.json" \
--github-run-id "${{ github.run_id }}" \ --github-run-id "${{ github.run_id }}" \
--github-run-number "${{ github.run_number }}" \ --github-run-number "${{ github.run_number }}" \
--head-branch-name "${{ env.BRANCH_NAME }}" --head-branch-name "${{ env.BRANCH_NAME }}"

299
.github/workflows/release_centos8.yaml vendored Normal file
View File

@@ -0,0 +1,299 @@
name: Release CentOS 8
on:
workflow_dispatch:
schedule:
- cron: "0 22 * * *"
jobs:
community_build:
name: "Community build"
runs-on: [self-hosted, Linux, X64, CentOS8]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 960
steps:
- name: Set up repository
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build community binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build community binaries.
cd build
cmake -DCMAKE_BUILD_TYPE=release -DMG_ENTERPRISE=OFF ..
make -j$THREADS
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
ctest -R memgraph__unit --output-on-failure
coverage_build:
name: "Coverage build"
runs-on: [self-hosted, Linux, X64, CentOS8]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build coverage binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build coverage binaries.
cd build
cmake -DTEST_COVERAGE=ON ..
make -j$THREADS memgraph__unit
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
ctest -R memgraph__unit --output-on-failure
- name: Compute code coverage
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Compute code coverage.
cd tools/github
./coverage_convert
# Package code coverage.
cd generated
tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu
- name: Save code coverage
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/generated/code_coverage.tar.gz
debug_build:
name: "Debug build"
runs-on: [self-hosted, Linux, X64, CentOS8]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build debug binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build debug binaries.
cd build
cmake ..
make -j$THREADS
- name: Run leftover CTest tests
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Run leftover CTest tests (all except unit and benchmark tests).
cd build
ctest -E "(memgraph__unit|memgraph__benchmark)" --output-on-failure
- name: Run drivers tests
run: |
./tests/drivers/run.sh
- name: Run integration tests
run: |
tests/integration/run.sh
- name: Run cppcheck and clang-format
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Run cppcheck and clang-format.
cd tools/github
./cppcheck_and_clang_format diff
- name: Save cppcheck and clang-format errors
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/cppcheck_and_clang_format.txt
release_build:
name: "Release build"
runs-on: [self-hosted, Linux, X64, CentOS8]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 960
steps:
- name: Set up repository
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build release binaries.
cd build
cmake -DCMAKE_BUILD_TYPE=release ..
make -j$THREADS
- name: Create enterprise RPM package
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
cd build
# create mgconsole
# we use the -B to force the build
make -j$THREADS -B mgconsole
# Create enterprise RPM package.
mkdir output && cd output
cpack -G RPM --config ../CPackConfig.cmake
rpmlint memgraph*.rpm
- name: Save enterprise RPM package
uses: actions/upload-artifact@v3
with:
name: "Enterprise RPM package"
path: build/output/memgraph*.rpm
- name: Run micro benchmark tests
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Run micro benchmark tests.
cd build
# The `eval` benchmark needs a large stack limit.
ulimit -s 262144
ctest -R memgraph__benchmark -V
- name: Run macro benchmark tests
run: |
cd tests/macro_benchmark
./harness QuerySuite MemgraphRunner \
--groups aggregation 1000_create unwind_create dense_expand match \
--no-strict
- name: Run parallel macro benchmark tests
run: |
cd tests/macro_benchmark
./harness QueryParallelSuite MemgraphRunner \
--groups aggregation_parallel create_parallel bfs_parallel \
--num-database-workers 9 --num-clients-workers 30 \
--no-strict
- name: Run GQL Behave tests
run: |
cd tests/gql_behave
./continuous_integration
- name: Save quality assurance status
uses: actions/upload-artifact@v3
with:
name: "GQL Behave Status"
path: |
tests/gql_behave/gql_behave_status.csv
tests/gql_behave/gql_behave_status.html
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
ctest -R memgraph__unit --output-on-failure
- name: Run e2e tests
run: |
cd tests
./setup.sh /opt/toolchain-v4/activate
source ve3/bin/activate_e2e
cd e2e
./run.sh
- name: Run stress test (plain)
run: |
cd tests/stress
./continuous_integration
- name: Run stress test (SSL)
run: |
cd tests/stress
./continuous_integration --use-ssl
- name: Run stress test (large)
run: |
cd tests/stress
./continuous_integration --large-dataset
- name: Run durability test (plain)
run: |
cd tests/stress
source ve3/bin/activate
python3 durability --num-steps 5
- name: Run durability test (large)
run: |
cd tests/stress
source ve3/bin/activate
python3 durability --num-steps 20

View File

@@ -2,33 +2,22 @@ name: Release Debian 10
on: on:
workflow_dispatch: workflow_dispatch:
inputs:
build_type:
type: choice
description: "Memgraph Build type. Default value is Release."
default: 'Release'
options:
- Release
- RelWithDebInfo
schedule: schedule:
- cron: "0 22 * * *" - cron: "0 22 * * *"
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
BUILD_TYPE: ${{ github.event.inputs.build_type || 'Release' }}
jobs: jobs:
community_build: community_build:
name: "Community build" name: "Community build"
runs-on: [self-hosted, Linux, X64, Debian10] runs-on: [self-hosted, Linux, X64, Debian10]
timeout-minutes: 60 env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 960
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -44,7 +33,7 @@ jobs:
# Build community binaries. # Build community binaries.
cd build cd build
cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE -DMG_ENTERPRISE=OFF .. cmake -DCMAKE_BUILD_TYPE=release -DMG_ENTERPRISE=OFF ..
make -j$THREADS make -j$THREADS
- name: Run unit tests - name: Run unit tests
@@ -63,11 +52,10 @@ jobs:
THREADS: 24 THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }} MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }} MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 60
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -109,19 +97,22 @@ jobs:
tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu
- name: Save code coverage - name: Save code coverage
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: "Code coverage(Coverage build)" name: "Code coverage"
path: tools/github/generated/code_coverage.tar.gz path: tools/github/generated/code_coverage.tar.gz
debug_build: debug_build:
name: "Debug build" name: "Debug build"
runs-on: [self-hosted, Linux, X64, Debian10] runs-on: [self-hosted, Linux, X64, Debian10]
timeout-minutes: 60 env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -153,6 +144,10 @@ jobs:
run: | run: |
./tests/drivers/run.sh ./tests/drivers/run.sh
- name: Run integration tests
run: |
tests/integration/run.sh
- name: Run cppcheck and clang-format - name: Run cppcheck and clang-format
run: | run: |
# Activate toolchain. # Activate toolchain.
@@ -163,49 +158,23 @@ jobs:
./cppcheck_and_clang_format diff ./cppcheck_and_clang_format diff
- name: Save cppcheck and clang-format errors - name: Save cppcheck and clang-format errors
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: "Code coverage(Debug build)" name: "Code coverage"
path: tools/github/cppcheck_and_clang_format.txt path: tools/github/cppcheck_and_clang_format.txt
debug_integration_test:
name: "Debug integration tests"
runs-on: [self-hosted, Linux, X64, Debian10]
timeout-minutes: 60
steps:
- name: Set up repository
uses: actions/checkout@v4
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build debug binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build debug binaries.
cd build
cmake ..
make -j$THREADS
- name: Run integration tests
run: |
tests/integration/run.sh
release_build: release_build:
name: "Release build" name: "Release build"
runs-on: [self-hosted, Linux, X64, Debian10] runs-on: [self-hosted, Linux, X64, Debian10]
timeout-minutes: 60 env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 960
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -221,7 +190,7 @@ jobs:
# Build release binaries. # Build release binaries.
cd build cd build
cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE .. cmake -DCMAKE_BUILD_TYPE=release ..
make -j$THREADS make -j$THREADS
- name: Create enterprise DEB package - name: Create enterprise DEB package
@@ -240,60 +209,11 @@ jobs:
cpack -G DEB --config ../CPackConfig.cmake cpack -G DEB --config ../CPackConfig.cmake
- name: Save enterprise DEB package - name: Save enterprise DEB package
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: "Enterprise DEB package" name: "Enterprise DEB package"
path: build/output/memgraph*.deb path: build/output/memgraph*.deb
- name: Run GQL Behave tests
run: |
cd tests
./setup.sh /opt/toolchain-v4/activate
cd gql_behave
./continuous_integration
- name: Save quality assurance status
uses: actions/upload-artifact@v4
with:
name: "GQL Behave Status"
path: |
tests/gql_behave/gql_behave_status.csv
tests/gql_behave/gql_behave_status.html
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
ctest -R memgraph__unit --output-on-failure
release_benchmark_tests:
name: "Release Benchmark Tests"
runs-on: [self-hosted, Linux, X64, Debian10]
timeout-minutes: 60
steps:
- name: Set up repository
uses: actions/checkout@v4
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build release binaries
cd build
cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE ..
make -j$THREADS
- name: Run micro benchmark tests - name: Run micro benchmark tests
run: | run: |
# Activate toolchain. # Activate toolchain.
@@ -320,38 +240,27 @@ jobs:
--num-database-workers 9 --num-clients-workers 30 \ --num-database-workers 9 --num-clients-workers 30 \
--no-strict --no-strict
release_e2e_test: - name: Run GQL Behave tests
if: false run: |
name: "Release End-to-end Test" cd tests/gql_behave
runs-on: [self-hosted, Linux, X64, Debian10] ./continuous_integration
timeout-minutes: 60
steps: - name: Save quality assurance status
- name: Set up repository uses: actions/upload-artifact@v3
uses: actions/checkout@v4
with: with:
# Number of commits to fetch. `0` indicates all history for all name: "GQL Behave Status"
# branches and tags. (default: 1) path: |
fetch-depth: 0 tests/gql_behave/gql_behave_status.csv
tests/gql_behave/gql_behave_status.html
- name: Build release binaries - name: Run unit tests
run: | run: |
# Activate toolchain. # Activate toolchain.
source /opt/toolchain-v4/activate source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build release binaries # Run unit tests.
cd build cd build
cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE .. ctest -R memgraph__unit --output-on-failure
make -j$THREADS
- name: Ensure Kafka and Pulsar are up
run: |
cd tests/e2e/streams/kafka
docker-compose up -d
cd ../pulsar
docker-compose up -d
- name: Run e2e tests - name: Run e2e tests
run: | run: |
@@ -361,40 +270,6 @@ jobs:
cd e2e cd e2e
./run.sh ./run.sh
- name: Ensure Kafka and Pulsar are down
if: always()
run: |
cd tests/e2e/streams/kafka
docker-compose down
cd ../pulsar
docker-compose down
release_durability_stress_tests:
name: "Release durability and stress tests"
runs-on: [self-hosted, Linux, X64, Debian10]
timeout-minutes: 60
steps:
- name: Set up repository
uses: actions/checkout@v4
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build release binaries.
cd build
cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE ..
make -j$THREADS
- name: Run stress test (plain) - name: Run stress test (plain)
run: | run: |
cd tests/stress cd tests/stress
@@ -405,6 +280,11 @@ jobs:
cd tests/stress cd tests/stress
./continuous_integration --use-ssl ./continuous_integration --use-ssl
- name: Run stress test (large)
run: |
cd tests/stress
./continuous_integration --large-dataset
- name: Run durability test (plain) - name: Run durability test (plain)
run: | run: |
cd tests/stress cd tests/stress
@@ -420,11 +300,15 @@ jobs:
release_jepsen_test: release_jepsen_test:
name: "Release Jepsen Test" name: "Release Jepsen Test"
runs-on: [self-hosted, Linux, X64, Debian10, JepsenControl] runs-on: [self-hosted, Linux, X64, Debian10, JepsenControl]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 60 timeout-minutes: 60
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -436,24 +320,18 @@ jobs:
source /opt/toolchain-v4/activate source /opt/toolchain-v4/activate
# Initialize dependencies. # Initialize dependencies.
./init ./init
# Build only memgraph release binary. # Build only memgraph release binary.
cd build cd build
cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE .. cmake -DCMAKE_BUILD_TYPE=release ..
make -j$THREADS memgraph make -j$THREADS memgraph
- name: Refresh Jepsen Cluster
run: |
cd tests/jepsen
./run.sh cluster-refresh
- name: Run Jepsen tests - name: Run Jepsen tests
run: | run: |
cd tests/jepsen cd tests/jepsen
./run.sh test-all-individually --binary ../../build/memgraph --ignore-run-stdout-logs --ignore-run-stderr-logs ./run.sh test-all-individually --binary ../../build/memgraph --ignore-run-stdout-logs --ignore-run-stderr-logs
- name: Save Jepsen report - name: Save Jepsen report
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
if: ${{ always() }} if: ${{ always() }}
with: with:
name: "Jepsen Report" name: "Jepsen Report"

View File

@@ -19,7 +19,7 @@ jobs:
DOCKER_REPOSITORY_NAME: memgraph DOCKER_REPOSITORY_NAME: memgraph
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v3
- name: Set up QEMU - name: Set up QEMU
uses: docker/setup-qemu-action@v2 uses: docker/setup-qemu-action@v2

View File

@@ -20,7 +20,7 @@ jobs:
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v3
- name: Set up QEMU - name: Set up QEMU
uses: docker/setup-qemu-action@v2 uses: docker/setup-qemu-action@v2

View File

@@ -2,33 +2,22 @@ name: Release Ubuntu 20.04
on: on:
workflow_dispatch: workflow_dispatch:
inputs:
build_type:
type: choice
description: "Memgraph Build type. Default value is Release."
default: 'Release'
options:
- Release
- RelWithDebInfo
schedule: schedule:
- cron: "0 22 * * *" - cron: "0 22 * * *"
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
BUILD_TYPE: ${{ github.event.inputs.build_type || 'Release' }}
jobs: jobs:
community_build: community_build:
name: "Community build" name: "Community build"
runs-on: [self-hosted, Linux, X64, Ubuntu20.04] runs-on: [self-hosted, Linux, X64, Ubuntu20.04]
timeout-minutes: 60 env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 960
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -44,7 +33,7 @@ jobs:
# Build community binaries. # Build community binaries.
cd build cd build
cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE -DMG_ENTERPRISE=OFF .. cmake -DCMAKE_BUILD_TYPE=release -DMG_ENTERPRISE=OFF ..
make -j$THREADS make -j$THREADS
- name: Run unit tests - name: Run unit tests
@@ -59,11 +48,14 @@ jobs:
coverage_build: coverage_build:
name: "Coverage build" name: "Coverage build"
runs-on: [self-hosted, Linux, X64, Ubuntu20.04] runs-on: [self-hosted, Linux, X64, Ubuntu20.04]
timeout-minutes: 60 env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -105,19 +97,22 @@ jobs:
tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu
- name: Save code coverage - name: Save code coverage
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: "Code coverage(Coverage build)" name: "Code coverage"
path: tools/github/generated/code_coverage.tar.gz path: tools/github/generated/code_coverage.tar.gz
debug_build: debug_build:
name: "Debug build" name: "Debug build"
runs-on: [self-hosted, Linux, X64, Ubuntu20.04] runs-on: [self-hosted, Linux, X64, Ubuntu20.04]
timeout-minutes: 60 env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -149,6 +144,10 @@ jobs:
run: | run: |
./tests/drivers/run.sh ./tests/drivers/run.sh
- name: Run integration tests
run: |
tests/integration/run.sh
- name: Run cppcheck and clang-format - name: Run cppcheck and clang-format
run: | run: |
# Activate toolchain. # Activate toolchain.
@@ -159,49 +158,23 @@ jobs:
./cppcheck_and_clang_format diff ./cppcheck_and_clang_format diff
- name: Save cppcheck and clang-format errors - name: Save cppcheck and clang-format errors
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: "Code coverage(Debug build)" name: "Code coverage"
path: tools/github/cppcheck_and_clang_format.txt path: tools/github/cppcheck_and_clang_format.txt
debug_integration_test:
name: "Debug integration tests"
runs-on: [self-hosted, Linux, X64, Ubuntu20.04]
timeout-minutes: 60
steps:
- name: Set up repository
uses: actions/checkout@v4
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build debug binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build debug binaries.
cd build
cmake ..
make -j$THREADS
- name: Run integration tests
run: |
tests/integration/run.sh
release_build: release_build:
name: "Release build" name: "Release build"
runs-on: [self-hosted, Linux, X64, Ubuntu20.04] runs-on: [self-hosted, Linux, X64, Ubuntu20.04]
timeout-minutes: 60 env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 960
steps: steps:
- name: Set up repository - name: Set up repository
uses: actions/checkout@v4 uses: actions/checkout@v3
with: with:
# Number of commits to fetch. `0` indicates all history for all # Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1) # branches and tags. (default: 1)
@@ -217,7 +190,7 @@ jobs:
# Build release binaries. # Build release binaries.
cd build cd build
cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE .. cmake -DCMAKE_BUILD_TYPE=release ..
make -j$THREADS make -j$THREADS
- name: Create enterprise DEB package - name: Create enterprise DEB package
@@ -236,60 +209,11 @@ jobs:
cpack -G DEB --config ../CPackConfig.cmake cpack -G DEB --config ../CPackConfig.cmake
- name: Save enterprise DEB package - name: Save enterprise DEB package
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v3
with: with:
name: "Enterprise DEB package" name: "Enterprise DEB package"
path: build/output/memgraph*.deb path: build/output/memgraph*.deb
- name: Run GQL Behave tests
run: |
cd tests
./setup.sh /opt/toolchain-v4/activate
cd gql_behave
./continuous_integration
- name: Save quality assurance status
uses: actions/upload-artifact@v4
with:
name: "GQL Behave Status"
path: |
tests/gql_behave/gql_behave_status.csv
tests/gql_behave/gql_behave_status.html
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
ctest -R memgraph__unit --output-on-failure
release_benchmark_tests:
name: "Release Benchmark Tests"
runs-on: [self-hosted, Linux, X64, Ubuntu20.04]
timeout-minutes: 60
steps:
- name: Set up repository
uses: actions/checkout@v4
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build release binaries
cd build
cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE ..
make -j$THREADS
- name: Run micro benchmark tests - name: Run micro benchmark tests
run: | run: |
# Activate toolchain. # Activate toolchain.
@@ -316,38 +240,27 @@ jobs:
--num-database-workers 9 --num-clients-workers 30 \ --num-database-workers 9 --num-clients-workers 30 \
--no-strict --no-strict
release_e2e_test: - name: Run GQL Behave tests
if: false run: |
name: "Release End-to-end Test" cd tests/gql_behave
runs-on: [self-hosted, Linux, X64, Ubuntu20.04] ./continuous_integration
timeout-minutes: 60
steps: - name: Save quality assurance status
- name: Set up repository uses: actions/upload-artifact@v3
uses: actions/checkout@v4
with: with:
# Number of commits to fetch. `0` indicates all history for all name: "GQL Behave Status"
# branches and tags. (default: 1) path: |
fetch-depth: 0 tests/gql_behave/gql_behave_status.csv
tests/gql_behave/gql_behave_status.html
- name: Build release binaries - name: Run unit tests
run: | run: |
# Activate toolchain. # Activate toolchain.
source /opt/toolchain-v4/activate source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build release binaries # Run unit tests.
cd build cd build
cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE .. ctest -R memgraph__unit --output-on-failure
make -j$THREADS
- name: Ensure Kafka and Pulsar are up
run: |
cd tests/e2e/streams/kafka
docker-compose up -d
cd ../pulsar
docker-compose up -d
- name: Run e2e tests - name: Run e2e tests
run: | run: |
@@ -357,40 +270,6 @@ jobs:
cd e2e cd e2e
./run.sh ./run.sh
- name: Ensure Kafka and Pulsar are down
if: always()
run: |
cd tests/e2e/streams/kafka
docker-compose down
cd ../pulsar
docker-compose down
release_durability_stress_tests:
name: "Release durability and stress tests"
runs-on: [self-hosted, Linux, X64, Ubuntu20.04]
timeout-minutes: 60
steps:
- name: Set up repository
uses: actions/checkout@v4
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build release binaries.
cd build
cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE ..
make -j$THREADS
- name: Run stress test (plain) - name: Run stress test (plain)
run: | run: |
cd tests/stress cd tests/stress
@@ -401,6 +280,11 @@ jobs:
cd tests/stress cd tests/stress
./continuous_integration --use-ssl ./continuous_integration --use-ssl
- name: Run stress test (large)
run: |
cd tests/stress
./continuous_integration --large-dataset
- name: Run durability test (plain) - name: Run durability test (plain)
run: | run: |
cd tests/stress cd tests/stress

View File

@@ -1,62 +0,0 @@
name: Stress test large
on:
workflow_dispatch:
inputs:
build_type:
type: choice
description: "Memgraph Build type. Default value is Release."
default: 'Release'
options:
- Release
- RelWithDebInfo
schedule:
- cron: "0 22 * * *"
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
BUILD_TYPE: ${{ github.event.inputs.build_type || 'Release' }}
jobs:
stress_test_large:
name: "Stress test large"
timeout-minutes: 720
strategy:
matrix:
os: [Debian10, Ubuntu20.04]
extra: [BigMemory, Gen8]
exclude:
- os: Debian10
extra: Gen8
- os: Ubuntu20.04
extra: BigMemory
runs-on: [self-hosted, Linux, X64, "${{ matrix.os }}", "${{ matrix.extra }}"]
steps:
- name: Set up repository
uses: actions/checkout@v4
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build release binaries.
cd build
cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE ..
make -j$THREADS
- name: Run stress test (large)
run: |
cd tests/stress
./continuous_integration --large-dataset

View File

@@ -15,7 +15,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Download artifacts - name: Download artifacts
uses: dawidd6/action-download-artifact@v4 uses: dawidd6/action-download-artifact@v2
with: with:
workflow: package_all.yaml workflow: package_all.yaml
workflow_conclusion: success workflow_conclusion: success

2
.gitignore vendored
View File

@@ -60,5 +60,3 @@ src/storage/distributed/rpc/concurrent_id_mapper_rpc_messages.hpp
src/transactions/distributed/engine_rpc_messages.hpp src/transactions/distributed/engine_rpc_messages.hpp
/tests/manual/js/transaction_timeout/package-lock.json /tests/manual/js/transaction_timeout/package-lock.json
/tests/manual/js/transaction_timeout/node_modules/ /tests/manual/js/transaction_timeout/node_modules/
.vscode/
src/query/frontend/opencypher/grammar/.antlr/*

View File

@@ -1,22 +0,0 @@
# Path to sources
sonar.sources = .
# sonar.exclusions=
sonar.inclusions=src,include,query_modules
# Path to tests
sonar.tests = tests/
# sonar.test.exclusions=
# sonar.test.inclusions=
# Source encoding
# sonar.sourceEncoding=
# Exclusions for copy-paste detection
# sonar.cpd.exclusions=
# Python version (for python projects only)
# sonar.python.version=
# C++ standard version (for C++ projects only)
# If not specified, it defaults to the latest supported standard
# sonar.cfamily.reportingCppStandardOverride=c++98|c++11|c++14|c++17|c++20

View File

@@ -1,32 +0,0 @@
# Tantivy ADR
**Author**
Marko Budiselic (github.com/gitbuda)
**Status**
APPROVED
**Date**
January 5, 2024
**Problem**
For some of Memgraph workloads, text search is a required feature. We don't
want to build a new text search engine because that's not Memgraph's core
value.
**Criteria**
- easy integration with our C++ codebase
- ability to operate in-memory and on-disk
- sufficient features (regex, full-text search, fuzzy search, aggregations over
text data)
- production-ready
**Decision**
All known C++ libraries are not production-ready. Recent Rust libraries, in
particular [Tantivy](https://github.com/quickwit-oss/tantivy), seem to provide
much more features, it is production ready. The way how we'll integrate Tantivy
into the current Memgraph codebase is via
[cxx](https://github.com/dtolnay/cxx). **We select Tantivy.**

View File

@@ -1,34 +0,0 @@
# NuRaft ADR
**Author**
Marko Budiselic (github.com/gitbuda)
**Status**
PROPOSED
**Date**
January 10, 2024
**Problem**
In order to enhance Memgraph to have High Availability features as requested by
customers, we want to have reliable coordinators backed by RAFT consensus algorithm. Implementing
RAFT to be correct and performant is a very challenging task. Skillful Memgraph
engineers already tried 3 times and failed to deliver in a reasonable timeframe
all three times (approximately 4 person-weeks of engineering work each time).
**Criteria**
- easy integration with our C++ codebase
- heavily tested in production environments
- implementation of performance optimizations on top of the canonical Raft
implementation
**Decision**
There are a few, robust C++ implementations of Raft but as a part of other
projects or bigger libraries. **We select
[NuRaft](https://github.com/eBay/NuRaft)** because it focuses on delivering
Raft without bloatware, and it's used by
[Clickhouse](https://github.com/ClickHouse/ClickHouse) (an comparable peer to
Memgraph, a very well-established product).

View File

@@ -1,38 +0,0 @@
# RocksDB ADR
**Author**
Marko Budiselic (github.com/gitbuda)
**Status**
ACCEPTED
**Date**
January 23, 2024
**Problem**
Interacting with data (reads and writes) on disk in a concurrent, safe, and
fast way is a challenging task. Implementing all low-level primitives to
interact with various disk hardware efficiently consumes significant
engineering people. Whenever Memgraph has to store data on disk (or any
other colder than RAM storage system), the problem is how to do that in the
least amount of development time while satisfying all functional requirements
(often performance).
**Criteria**
- working efficiently in a highly concurrent environment
- easy integration with Memgraph's C++ codebase
- providing low-level key-value API
- heavily tested in production environments
- providing abstractions for the storage hardware (even for cloud-based
storages like S3)
**Decision**
There are a few robust key-value stores, but finding one that is
production-ready and compatible with Memgraph's C++ codebase is challenging.
**We select [RocksDB](https://github.com/facebook/rocksdb)** because it
delivers robust API to manage data on disk; it's battle-tested in many
production environments (many databases systems are embedding RocksDB), and
it's the most compatible one.

View File

@@ -1,67 +0,0 @@
# Architecture Decision Records
Also known as ADRs. This practice has become widespread in many
high performing engineering teams. It is a technique for communicating
between software engineers. ADRs provide a clear and documented
history of architectural choices, ensuring that everyone on the
team is on the same page. This improves communication and reduces
misunderstandings. The act of recording decisions encourages
thoughtful consideration before making choices. This can lead to
more robust and better-informed architectural decisions.
Links must be created, pointing both to and from the Github Issues
and/or the Notion Program Management "Initiative" database.
ADRs are complimentary to any tech specs that get written while
designing a solution. ADRs are very short and to the point, while
tech specs will include diagrams and can be quite verbose.
## HOWTO
Each ADR will be assigned a monotonically increasing unique numeric
identifier, which will be zero-padded to 3 digits. Each ADR will
be in a single markdown file containing no more than one page of
text, and the filename will start with that unique identifier,
followed by a camel case phrase summarizing the problem. For
example: `001_architecture_decision_records.md` or
`002_big_integration_cap_theorem.md`.
We want to use an ADR when:
1. Significant Impact: This includes choices that affect scalability, performance, or fundamental design principles.
1. Long-Term Ramifications: When a decision is expected to have long-term ramifications or is difficult to reverse.
1. Architectural Principles: ADRs are suitable for documenting decisions related to architectural principles, frameworks, or patterns that shape the system's structure.
1. Controversial Choices: When a decision is likely to be controversial or may require justification in the future.
The most senior engineer on a project will evaluate and decide
whether or not an ADR is needed.
## Do
1. Keep them brief and concise.
1. Explain the trade-offs.
1. Each ADR should be about one AD, not multiple ADs
1. Don't alter existing information in an ADR. Instead, amend the ADR by adding new information, or supersede the ADR by creating a new ADR.
1. Explain your organization's situation and business priorities.
1. Include rationale and considerations based on social and skills makeups of your teams.
1. Include pros and cons that are relevant, and describe them in terms that align with your needs and goals.
1. Explain what follows from making the decision. This can include the effects, outcomes, outputs, follow ups, and more.
## Don't
1. Try to guess what the executive leader wants, and then attempt to please them. Be objective.
1. Try to solve everything all at once. A pretty good solution now is MUCH BETTER than a perfect solution later. Carpe diem!
1. Hide any doubts or unanswered questions.
1. Make it a sales pitch. Everything has upsides and downsides - be authentic and honest about them.
1. Perform merely a superficial investigation. If an ADR doesn't call for some deep thinking, then it probably shouldn't exist.
1. Ignore the long-term costs such as performance, tech debt or hardware and maintenance.
1. Get tunnel vision where creative or surprising approaches are not explored.
# Template - use the format below for each new ADR
1. **Author** - who has written the ADR
1. **Status** - one of: PROPOSED, ACCEPTED, REJECTED, SUPERSEDED-BY or DEPRECATED
1. **Date** - when the status was most recently updated
1. **Problem** - a concise paragraph explaining the context
1. **Criteria** - a list of the two or three metrics by which the solution was evaluated, and their relative weights (importance)
1. **Decision** - what was chosen as the way forward, and what the consequences are of the decision

View File

@@ -42,6 +42,10 @@ endif()
project(memgraph LANGUAGES C CXX) project(memgraph LANGUAGES C CXX)
# NOTE: once in a while this needs to be toggled to check headers are
# correct and PCH isn't masking any include issues
set(CMAKE_DISABLE_PRECOMPILE_HEADERS OFF)
#TODO: upgrade to cmake 3.24 + CheckIPOSupported #TODO: upgrade to cmake 3.24 + CheckIPOSupported
#cmake_policy(SET CMP0138 NEW) #cmake_policy(SET CMP0138 NEW)
#include(CheckIPOSupported) #include(CheckIPOSupported)
@@ -235,6 +239,7 @@ else()
endif() endif()
# ----------------------------------------------------------------------------- # -----------------------------------------------------------------------------
# default build type is debug # default build type is debug
if (NOT CMAKE_BUILD_TYPE) if (NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE "Debug") set(CMAKE_BUILD_TYPE "Debug")
@@ -271,17 +276,6 @@ endif()
set(libs_dir ${CMAKE_SOURCE_DIR}/libs) set(libs_dir ${CMAKE_SOURCE_DIR}/libs)
add_subdirectory(libs EXCLUDE_FROM_ALL) add_subdirectory(libs EXCLUDE_FROM_ALL)
option(MG_EXPERIMENTAL_HIGH_AVAILABILITY "Feature flag for experimental high availability" OFF)
if (NOT MG_ENTERPRISE AND MG_EXPERIMENTAL_HIGH_AVAILABILITY)
set(MG_EXPERIMENTAL_HIGH_AVAILABILITY OFF)
message(FATAL_ERROR "MG_EXPERIMENTAL_HIGH_AVAILABILITY can only be used with enterpise version of the code.")
endif ()
if (MG_EXPERIMENTAL_HIGH_AVAILABILITY)
add_compile_definitions(MG_EXPERIMENTAL_HIGH_AVAILABILITY)
endif ()
# Optional subproject configuration ------------------------------------------- # Optional subproject configuration -------------------------------------------
option(TEST_COVERAGE "Generate coverage reports from running memgraph" OFF) option(TEST_COVERAGE "Generate coverage reports from running memgraph" OFF)
option(TOOLS "Build tools binaries" ON) option(TOOLS "Build tools binaries" ON)
@@ -290,18 +284,6 @@ option(ASAN "Build with Address Sanitizer. To get a reasonable performance optio
option(TSAN "Build with Thread Sanitizer. To get a reasonable performance option should be used only in Release or RelWithDebInfo build " OFF) option(TSAN "Build with Thread Sanitizer. To get a reasonable performance option should be used only in Release or RelWithDebInfo build " OFF)
option(UBSAN "Build with Undefined Behaviour Sanitizer" OFF) option(UBSAN "Build with Undefined Behaviour Sanitizer" OFF)
# Build feature flags
option(MG_EXPERIMENTAL_REPLICATION_MULTITENANCY "Feature flag for experimental replicaition of multitenacy" OFF)
if (NOT MG_ENTERPRISE AND MG_EXPERIMENTAL_REPLICATION_MULTITENANCY)
set(MG_EXPERIMENTAL_REPLICATION_MULTITENANCY OFF)
message(FATAL_ERROR "MG_EXPERIMENTAL_REPLICATION_MULTITENANCY with community edition build isn't possible")
endif ()
if (MG_EXPERIMENTAL_REPLICATION_MULTITENANCY)
add_compile_definitions(MG_EXPERIMENTAL_REPLICATION_MULTITENANCY)
endif ()
if (TEST_COVERAGE) if (TEST_COVERAGE)
string(TOLOWER ${CMAKE_BUILD_TYPE} lower_build_type) string(TOLOWER ${CMAKE_BUILD_TYPE} lower_build_type)
if (NOT lower_build_type STREQUAL "debug") if (NOT lower_build_type STREQUAL "debug")
@@ -315,7 +297,7 @@ if (MG_ENTERPRISE)
add_definitions(-DMG_ENTERPRISE) add_definitions(-DMG_ENTERPRISE)
endif() endif()
option(ENABLE_JEMALLOC "Use jemalloc" ON) set(ENABLE_JEMALLOC ON)
if (ASAN) if (ASAN)
message(WARNING "Disabling jemalloc as it doesn't work well with ASAN") message(WARNING "Disabling jemalloc as it doesn't work well with ASAN")

View File

@@ -1,5 +1,5 @@
<p align="center"> <p align="center">
<img src="https://public-assets.memgraph.com/github-readme-images/github-memgraph-repo-banner.png"> <img width="400px" src="https://uploads-ssl.webflow.com/5e7ceb09657a69bdab054b3a/5e7ceb09657a6937ab054bba_Black_Original%20_Logo.png">
</p> </p>
--- ---
@@ -141,20 +141,29 @@ guide](https://memgraph.com/docs/memgraph/reference-guide/configuration).
## :trophy: Contributing ## :trophy: Contributing
Welcome to the heart of Memgraph development! We're on a mission to supercharge Memgraph, making it faster, more user-friendly, and even more powerful. We owe a big thanks to our fantastic community of contributors who help us fix bugs and bring incredible improvements to life. If you're passionate about databases and open source, here's your chance to make a difference! The main purpose of this repository is to continue evolving Memgraph, making it
faster and easier to use. Development of Memgraph happens in the open on GitHub,
### Explore Memgraph Internals and we are grateful to the community for contributing bug fixes and
improvements. Read below to learn how you can take part in improving Memgraph.
Interested in the nuts and bolts of Memgraph? Our [internals documentation](https://memgraph.notion.site/Memgraph-Internals-12b69132d67a417898972927d6870bd2) is where you can uncover the inner workings of Memgraph's architecture, learn how to build the project from scratch, and discover the secrets of effective contributions. Dive deep into the database!
### Dive into the Contributing Guide
Ready to jump into the action? Explore our [contributing guide](CONTRIBUTING.md) to get the inside scoop on how we develop Memgraph. It's your roadmap for suggesting bug fixes and enhancements. Contribute your skills and ideas!
### Code of Conduct ### Code of Conduct
Our commitment to a respectful and professional community is unwavering. Every participant in Memgraph is expected to adhere to a stringent Code of Conduct. Please carefully review [the complete text](CODE_OF_CONDUCT.md) to gain a comprehensive understanding of the behaviors that are both expected and explicitly prohibited. Memgraph has adopted a Code of Conduct that we expect project participants to
adhere to. Please read [the full text](CODE_OF_CONDUCT.md) so that you can
understand what actions will and will not be tolerated.
We maintain a zero-tolerance policy towards any violations. Our shared commitment to this Code of Conduct ensures that Memgraph remains a place where integrity and excellence are paramount. ### Contributing Guide
Read our [contributing guide](CONTRIBUTING.md) to learn about our development
process and how to propose bug fixes and improvements.
### Internals
Read our
[internal](https://memgraph.notion.site/Memgraph-Internals-12b69132d67a417898972927d6870bd2)
docs to learn more about Memgraph's architecture, how to build the project from
source and how to start contributing. All information related to the database,
can be found in the aforementioned docs.
### :scroll: License ### :scroll: License

View File

@@ -111,14 +111,6 @@ modifications:
value: "false" value: "false"
override: true override: true
- name: "storage_parallel_schema_recovery"
value: "false"
override: true
- name: "storage_enable_schema_metadata"
value: "false"
override: true
- name: "query_callable_mappings_path" - name: "query_callable_mappings_path"
value: "/etc/memgraph/apoc_compatibility_mappings.json" value: "/etc/memgraph/apoc_compatibility_mappings.json"
override: true override: true

View File

@@ -1,8 +1,8 @@
{ {
"dbms.components": "mgps.components", "dbms.components": "mgps.components",
"apoc.util.validate": "mgps.validate", "apoc.util.validate": "mgps.validate",
"db.schema.nodeTypeProperties": "schema.NodeTypeOroperties", "db.schema.nodeTypeProperties":"schema.node_type_properties",
"db.schema.relTypeProperties": "schema.RelTypeProperties", "db.schema.relTypeProperties":"schema.rel_type_properties",
"apoc.coll.contains": "collections.contains", "apoc.coll.contains": "collections.contains",
"apoc.coll.partition": "collections.partition", "apoc.coll.partition": "collections.partition",
"apoc.coll.toSet": "collections.to_set", "apoc.coll.toSet": "collections.to_set",
@@ -23,4 +23,5 @@
"apoc.refactor.cloneSubgraph": "refactor.clone_subgraph", "apoc.refactor.cloneSubgraph": "refactor.clone_subgraph",
"apoc.refactor.cloneSubgraphFromPath": "refactor.clone_subgraph_from_path", "apoc.refactor.cloneSubgraphFromPath": "refactor.clone_subgraph_from_path",
"apoc.label.exists": "label.exists" "apoc.label.exists": "label.exists"
} }

2
environment/.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
archives
build

View File

@@ -1,10 +1,5 @@
# Memgraph Operating Environments # Memgraph Operating Environments
## Issues related to build toolchain
* GCC 11.2 (toolchain-v4) doesn't compile on Fedora 38, multiple definitions of enum issue
* spdlog 1.10/11 doesn't work with fmt 10.0.0
## os ## os
Under the `os` directory, you can find scripts to install all required system Under the `os` directory, you can find scripts to install all required system

View File

@@ -18,7 +18,7 @@ TOOLCHAIN_BUILD_DEPS=(
curl libcurl-devel # for cmake curl libcurl-devel # for cmake
readline-devel # for cmake and llvm readline-devel # for cmake and llvm
libffi-devel libxml2-devel # for llvm libffi-devel libxml2-devel # for llvm
libedit-devel pcre-devel pcre2-devel automake bison # for swig libedit-devel pcre-devel automake bison # for swig
file file
openssl-devel openssl-devel
gmp-devel gmp-devel
@@ -158,9 +158,9 @@ install() {
continue continue
fi fi
if [ "$pkg" == nodejs ]; then if [ "$pkg" == nodejs ]; then
curl -sL https://rpm.nodesource.com/setup_16.x | bash -
if ! yum list installed nodejs >/dev/null 2>/dev/null; then if ! yum list installed nodejs >/dev/null 2>/dev/null; then
yum install https://rpm.nodesource.com/pub_16.x/nodistro/repo/nodesource-release-nodistro-1.noarch.rpm -y yum install -y nodejs
yum install nodejs -y --setopt=nodesource-nodejs.module_hotfixes=1
fi fi
continue continue
fi fi
@@ -172,6 +172,13 @@ install() {
fi fi
continue continue
fi fi
if [ "$pkg" == nodejs ]; then
curl -sL https://rpm.nodesource.com/setup_16.x | bash -
if ! yum list installed nodejs >/dev/null 2>/dev/null; then
yum install -y nodejs
fi
continue
fi
if [ "$pkg" == java-11-openjdk ]; then if [ "$pkg" == java-11-openjdk ]; then
amazon-linux-extras install -y java-openjdk11 amazon-linux-extras install -y java-openjdk11
continue continue

View File

@@ -20,7 +20,7 @@ TOOLCHAIN_BUILD_DEPS=(
curl # snappy curl # snappy
readline-devel # cmake and llvm readline-devel # cmake and llvm
libffi-devel libxml2-devel perl-Digest-MD5 # llvm libffi-devel libxml2-devel perl-Digest-MD5 # llvm
libedit-devel pcre-devel pcre2-devel automake bison # swig libedit-devel pcre-devel automake bison # swig
file file
openssl-devel openssl-devel
gmp-devel gmp-devel

View File

@@ -17,7 +17,7 @@ TOOLCHAIN_BUILD_DEPS=(
expat-devel xz-devel python3-devel texinfo libbabeltrace-devel # for gdb expat-devel xz-devel python3-devel texinfo libbabeltrace-devel # for gdb
readline-devel # for cmake and llvm readline-devel # for cmake and llvm
libffi-devel libxml2-devel # for llvm libffi-devel libxml2-devel # for llvm
libedit-devel pcre-devel pcre2-devel automake bison # for swig libedit-devel pcre-devel automake bison # for swig
file file
openssl-devel openssl-devel
gmp-devel gmp-devel

View File

@@ -24,7 +24,7 @@ TOOLCHAIN_BUILD_DEPS=(
libgmp-dev # for gdb libgmp-dev # for gdb
gperf # for proxygen gperf # for proxygen
git # for fbthrift git # for fbthrift
libedit-dev libpcre2-dev libpcre3-dev automake bison # for swig libedit-dev libpcre3-dev automake bison # for swig
) )
TOOLCHAIN_RUN_DEPS=( TOOLCHAIN_RUN_DEPS=(

View File

@@ -18,7 +18,7 @@ TOOLCHAIN_BUILD_DEPS=(
libcurl4-openssl-dev # for cmake libcurl4-openssl-dev # for cmake
libreadline-dev # for cmake and llvm libreadline-dev # for cmake and llvm
libffi-dev libxml2-dev # for llvm libffi-dev libxml2-dev # for llvm
libedit-dev libpcre2-dev libpcre3-dev automake bison # for swig libedit-dev libpcre3-dev automake bison # for swig
curl # snappy curl # snappy
file # for libunwind file # for libunwind
libssl-dev # for libevent libssl-dev # for libevent

View File

@@ -18,7 +18,7 @@ TOOLCHAIN_BUILD_DEPS=(
libcurl4-openssl-dev # for cmake libcurl4-openssl-dev # for cmake
libreadline-dev # for cmake and llvm libreadline-dev # for cmake and llvm
libffi-dev libxml2-dev # for llvm libffi-dev libxml2-dev # for llvm
libedit-dev libpcre2-dev libpcre3-dev automake bison # for swig libedit-dev libpcre3-dev automake bison # for swig
curl # snappy curl # snappy
file # for libunwind file # for libunwind
libssl-dev # for libevent libssl-dev # for libevent

View File

@@ -18,7 +18,7 @@ TOOLCHAIN_BUILD_DEPS=(
curl libcurl-devel # for cmake curl libcurl-devel # for cmake
readline-devel # for cmake and llvm readline-devel # for cmake and llvm
libffi-devel libxml2-devel # for llvm libffi-devel libxml2-devel # for llvm
libedit-devel pcre-devel pcre2-devel automake bison # for swig libedit-devel pcre-devel automake bison # for swig
file file
openssl-devel openssl-devel
gmp-devel gmp-devel

View File

@@ -1,108 +0,0 @@
#!/bin/bash
set -Eeuo pipefail
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
source "$DIR/../util.sh"
check_operating_system "fedora-38"
check_architecture "x86_64"
TOOLCHAIN_BUILD_DEPS=(
coreutils-common gcc gcc-c++ make # generic build tools
wget # used for archive download
gnupg2 # used for archive signature verification
tar gzip bzip2 xz unzip # used for archive unpacking
zlib-devel # zlib library used for all builds
expat-devel xz-devel python3-devel texinfo libbabeltrace-devel # for gdb
curl libcurl-devel # for cmake
readline-devel # for cmake and llvm
libffi-devel libxml2-devel # for llvm
libedit-devel pcre-devel pcre2-devel automake bison # for swig
file
openssl-devel
gmp-devel
gperf
diffutils
libipt libipt-devel # intel
patch
perl # for openssl
)
TOOLCHAIN_RUN_DEPS=(
make # generic build tools
tar gzip bzip2 xz # used for archive unpacking
zlib # zlib library used for all builds
expat xz-libs python3 # for gdb
readline # for cmake and llvm
libffi libxml2 # for llvm
openssl-devel
)
MEMGRAPH_BUILD_DEPS=(
git # source code control
make pkgconf-pkg-config # build system
wget # for downloading libs
libuuid-devel java-11-openjdk # required by antlr
readline-devel # for memgraph console
python3-devel # for query modules
openssl-devel
libseccomp-devel
python3 python3-pip python3-virtualenv python3-virtualenvwrapper python3-pyyaml nmap-ncat # for tests
libcurl-devel # mg-requests
rpm-build rpmlint # for RPM package building
doxygen graphviz # source documentation generators
which nodejs golang zip unzip java-11-openjdk-devel # for driver tests
sbcl # for custom Lisp C++ preprocessing
autoconf # for jemalloc code generation
libtool # for protobuf code generation
)
list() {
echo "$1"
}
check() {
if [ -v LD_LIBRARY_PATH ]; then
# On Fedora 38 yum/dnf and python11 use newer glibc which is not compatible
# with ours, so we need to momentarely disable env
local OLD_LD_LIBRARY_PATH=${LD_LIBRARY_PATH}
LD_LIBRARY_PATH=""
fi
local missing=""
for pkg in $1; do
if ! dnf list installed "$pkg" >/dev/null 2>/dev/null; then
missing="$pkg $missing"
fi
done
if [ "$missing" != "" ]; then
echo "MISSING PACKAGES: $missing"
exit 1
fi
if [ -v OLD_LD_LIBRARY_PATH ]; then
echo "Restoring LD_LIBRARY_PATH..."
LD_LIBRARY_PATH=${OLD_LD_LIBRARY_PATH}
fi
}
install() {
cd "$DIR"
if [ "$EUID" -ne 0 ]; then
echo "Please run as root."
exit 1
fi
# If GitHub Actions runner is installed, append LANG to the environment.
# Python related tests don't work without the LANG export.
if [ -d "/home/gh/actions-runner" ]; then
echo "LANG=en_US.utf8" >> /home/gh/actions-runner/.env
else
echo "NOTE: export LANG=en_US.utf8"
fi
dnf update -y
for pkg in $1; do
dnf install -y "$pkg"
done
}
deps=$2"[*]"
"$1" "${!deps}"

View File

@@ -25,7 +25,7 @@ TOOLCHAIN_BUILD_DEPS=(
libgmp-dev # for gdb libgmp-dev # for gdb
gperf # for proxygen gperf # for proxygen
libssl-dev libssl-dev
libedit-dev libpcre2-dev libpcre3-dev automake bison # swig libedit-dev libpcre3-dev automake bison # swig
) )
TOOLCHAIN_RUN_DEPS=( TOOLCHAIN_RUN_DEPS=(

View File

@@ -24,7 +24,7 @@ TOOLCHAIN_BUILD_DEPS=(
libgmp-dev # for gdb libgmp-dev # for gdb
gperf # for proxygen gperf # for proxygen
libssl-dev libssl-dev
libedit-dev libpcre2-dev libpcre3-dev automake bison # for swig libedit-dev libpcre3-dev automake bison # for swig
) )
TOOLCHAIN_RUN_DEPS=( TOOLCHAIN_RUN_DEPS=(

View File

@@ -24,7 +24,7 @@ TOOLCHAIN_BUILD_DEPS=(
libgmp-dev # for gdb libgmp-dev # for gdb
gperf # for proxygen gperf # for proxygen
libssl-dev libssl-dev
libedit-dev libpcre2-dev libpcre3-dev automake bison # for swig libedit-dev libpcre3-dev automake bison # for swig
) )
TOOLCHAIN_RUN_DEPS=( TOOLCHAIN_RUN_DEPS=(

View File

@@ -24,7 +24,7 @@ TOOLCHAIN_BUILD_DEPS=(
libgmp-dev # for gdb libgmp-dev # for gdb
gperf # for proxygen gperf # for proxygen
libssl-dev libssl-dev
libedit-dev libpcre2-dev libpcre3-dev automake bison # for swig libedit-dev libpcre3-dev automake bison # for swig
) )
TOOLCHAIN_RUN_DEPS=( TOOLCHAIN_RUN_DEPS=(

View File

@@ -1,4 +1 @@
archives
build
output
*.tar.gz *.tar.gz

View File

@@ -4,7 +4,7 @@ diff -ur a/CMakeLists.txt b/CMakeLists.txt
@@ -52,9 +52,9 @@ @@ -52,9 +52,9 @@
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /EHs-c-") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /EHs-c-")
add_definitions(-D_HAS_EXCEPTIONS=0) add_definitions(-D_HAS_EXCEPTIONS=0)
- # Disable RTTI. - # Disable RTTI.
- string(REGEX REPLACE "/GR" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}") - string(REGEX REPLACE "/GR" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
- set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /GR-") - set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /GR-")
@@ -17,7 +17,7 @@ diff -ur a/CMakeLists.txt b/CMakeLists.txt
@@ -77,9 +77,9 @@ @@ -77,9 +77,9 @@
string(REGEX REPLACE "-fexceptions" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}") string(REGEX REPLACE "-fexceptions" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-exceptions") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-exceptions")
- # Disable RTTI. - # Disable RTTI.
- string(REGEX REPLACE "-frtti" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}") - string(REGEX REPLACE "-frtti" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
- set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-rtti") - set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-rtti")
@@ -25,5 +25,5 @@ diff -ur a/CMakeLists.txt b/CMakeLists.txt
+ # string(REGEX REPLACE "-frtti" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}") + # string(REGEX REPLACE "-frtti" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
+ # set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-rtti") + # set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-rtti")
endif(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC") endif(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# BUILD_SHARED_LIBS is a standard CMake variable, but we declare it here to make # BUILD_SHARED_LIBS is a standard CMake variable, but we declare it here to make

View File

@@ -7,7 +7,7 @@ DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
CPUS=$( grep -c processor < /proc/cpuinfo ) CPUS=$( grep -c processor < /proc/cpuinfo )
cd "$DIR" cd "$DIR"
source "$DIR/../../util.sh" source "$DIR/../util.sh"
DISTRO="$(operating_system)" DISTRO="$(operating_system)"
# toolchain version # toolchain version
@@ -30,10 +30,10 @@ LLVM_VERSION=11.0.0
SWIG_VERSION=4.0.2 # used only for LLVM compilation SWIG_VERSION=4.0.2 # used only for LLVM compilation
# Check for the dependencies. # Check for the dependencies.
echo "ALL BUILD PACKAGES: $($DIR/../../os/$DISTRO.sh list TOOLCHAIN_BUILD_DEPS)" echo "ALL BUILD PACKAGES: $($DIR/../os/$DISTRO.sh list TOOLCHAIN_BUILD_DEPS)"
$DIR/../../os/$DISTRO.sh check TOOLCHAIN_BUILD_DEPS $DIR/../os/$DISTRO.sh check TOOLCHAIN_BUILD_DEPS
echo "ALL RUN PACKAGES: $($DIR/../../os/$DISTRO.sh list TOOLCHAIN_RUN_DEPS)" echo "ALL RUN PACKAGES: $($DIR/../os/$DISTRO.sh list TOOLCHAIN_RUN_DEPS)"
$DIR/../../os/$DISTRO.sh check TOOLCHAIN_RUN_DEPS $DIR/../os/$DISTRO.sh check TOOLCHAIN_RUN_DEPS
# check installation directory # check installation directory
NAME=toolchain-v$TOOLCHAIN_VERSION NAME=toolchain-v$TOOLCHAIN_VERSION
@@ -442,7 +442,7 @@ In order to be able to run all of these tools you should install the following
packages: packages:
\`\`\` \`\`\`
$($DIR/../../os/$DISTRO.sh list TOOLCHAIN_RUN_DEPS) $($DIR/../os/$DISTRO.sh list TOOLCHAIN_RUN_DEPS)
\`\`\` \`\`\`
## Usage ## Usage

View File

@@ -7,7 +7,7 @@ DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
CPUS=$( grep -c processor < /proc/cpuinfo ) CPUS=$( grep -c processor < /proc/cpuinfo )
cd "$DIR" cd "$DIR"
source "$DIR/../../util.sh" source "$DIR/../util.sh"
DISTRO="$(operating_system)" DISTRO="$(operating_system)"
# toolchain version # toolchain version
@@ -31,10 +31,10 @@ LLVM_VERSION_LONG=12.0.1-rc4
SWIG_VERSION=4.0.2 # used only for LLVM compilation SWIG_VERSION=4.0.2 # used only for LLVM compilation
# Check for the dependencies. # Check for the dependencies.
echo "ALL BUILD PACKAGES: $($DIR/../../os/$DISTRO.sh list TOOLCHAIN_BUILD_DEPS)" echo "ALL BUILD PACKAGES: $($DIR/../os/$DISTRO.sh list TOOLCHAIN_BUILD_DEPS)"
$DIR/../../os/$DISTRO.sh check TOOLCHAIN_BUILD_DEPS $DIR/../os/$DISTRO.sh check TOOLCHAIN_BUILD_DEPS
echo "ALL RUN PACKAGES: $($DIR/../../os/$DISTRO.sh list TOOLCHAIN_RUN_DEPS)" echo "ALL RUN PACKAGES: $($DIR/../os/$DISTRO.sh list TOOLCHAIN_RUN_DEPS)"
$DIR/../../os/$DISTRO.sh check TOOLCHAIN_RUN_DEPS $DIR/../os/$DISTRO.sh check TOOLCHAIN_RUN_DEPS
# check installation directory # check installation directory
NAME=toolchain-v$TOOLCHAIN_VERSION NAME=toolchain-v$TOOLCHAIN_VERSION
@@ -452,7 +452,7 @@ In order to be able to run all of these tools you should install the following
packages: packages:
\`\`\` \`\`\`
$($DIR/../../os/$DISTRO.sh list TOOLCHAIN_RUN_DEPS) $($DIR/../os/$DISTRO.sh list TOOLCHAIN_RUN_DEPS)
\`\`\` \`\`\`
## Usage ## Usage

View File

@@ -7,7 +7,7 @@ DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
CPUS=$( grep -c processor < /proc/cpuinfo ) CPUS=$( grep -c processor < /proc/cpuinfo )
cd "$DIR" cd "$DIR"
source "$DIR/../../util.sh" source "$DIR/../util.sh"
DISTRO="$(operating_system)" DISTRO="$(operating_system)"
function log_tool_name () { function log_tool_name () {
@@ -51,13 +51,17 @@ CPPCHECK_VERSION=2.6
LLVM_VERSION=13.0.0 LLVM_VERSION=13.0.0
SWIG_VERSION=4.0.2 # used only for LLVM compilation SWIG_VERSION=4.0.2 # used only for LLVM compilation
# Set the right operating system setup script. # Set the right env script
ENV_SCRIPT="$DIR/../../os/$DISTRO.sh" ENV_SCRIPT="$DIR/../os/$DISTRO.sh"
if [[ "$for_arm" = true ]]; then if [[ "$for_arm" = true ]]; then
ENV_SCRIPT="$DIR/../../os/$DISTRO-arm.sh" ENV_SCRIPT="$DIR/../os/$DISTRO-arm.sh"
fi fi
# Check for the toolchain build dependencies.
echo "ALL BUILD PACKAGES: $(${ENV_SCRIPT} list TOOLCHAIN_BUILD_DEPS)" echo "ALL BUILD PACKAGES: $(${ENV_SCRIPT} list TOOLCHAIN_BUILD_DEPS)"
${ENV_SCRIPT} check TOOLCHAIN_BUILD_DEPS ${ENV_SCRIPT} check TOOLCHAIN_BUILD_DEPS
# Check for the toolchain run dependencies.
echo "ALL RUN PACKAGES: $(${ENV_SCRIPT} list TOOLCHAIN_RUN_DEPS)" echo "ALL RUN PACKAGES: $(${ENV_SCRIPT} list TOOLCHAIN_RUN_DEPS)"
${ENV_SCRIPT} check TOOLCHAIN_RUN_DEPS ${ENV_SCRIPT} check TOOLCHAIN_RUN_DEPS
@@ -654,7 +658,7 @@ In order to be able to run all of these tools you should install the following
packages: packages:
\`\`\` \`\`\`
$($DIR/../../os/$ENV_SCRIPT.sh list TOOLCHAIN_RUN_DEPS) $($DIR/../os/$ENV_SCRIPT.sh list TOOLCHAIN_RUN_DEPS)
\`\`\` \`\`\`
## Usage ## Usage

View File

@@ -1,67 +0,0 @@
#!/bin/bash -e
# helpers
pushd () { command pushd "$@" > /dev/null; }
popd () { command popd "$@" > /dev/null; }
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
CPUS=$( grep -c processor < /proc/cpuinfo )
cd "$DIR"
source "$DIR/../../util.sh"
DISTRO="$(operating_system)"
function log_tool_name () {
echo ""
echo ""
echo "#### $1 ####"
echo ""
echo ""
}
TOOLCHAIN_VERSION=4
NAME=toolchain-v$TOOLCHAIN_VERSION
PREFIX=/opt/$NAME
rm -rf "$PREFIX/include/zstd.h"
# create archives directory
mkdir -p archives && pushd archives
ZSTD_VERSION=1.5.5
if [ ! -f zstd-$ZSTD_VERSION.tar.gz ]; then
wget https://github.com/facebook/zstd/releases/download/v$ZSTD_VERSION/zstd-$ZSTD_VERSION.tar.gz -O zstd-$ZSTD_VERSION.tar.gz
fi
popd
# create build directory and activate toolchain
mkdir -p build
pushd build
source $PREFIX/activate
export CC=$PREFIX/bin/clang
export CXX=$PREFIX/bin/clang++
export CFLAGS="$CFLAGS -fPIC"
export PATH=$PREFIX/bin:$PATH
export LD_LIBRARY_PATH=$PREFIX/lib64
COMMON_CMAKE_FLAGS="-DCMAKE_INSTALL_PREFIX=$PREFIX
-DCMAKE_PREFIX_PATH=$PREFIX
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_C_COMPILER=$CC
-DCMAKE_CXX_COMPILER=$CXX
-DBUILD_SHARED_LIBS=OFF
-DCMAKE_CXX_STANDARD=20
-DBUILD_TESTING=OFF
-DCMAKE_REQUIRED_INCLUDES=$PREFIX/include
-DCMAKE_POSITION_INDEPENDENT_CODE=ON"
log_tool_name "zstd $ZSTD_VERSION"
if [ ! -f $PREFIX/include/zstd.h ]; then
if [ -d zstd-$ZSTD_VERSION ]; then
rm -rf zstd-$ZSTD_VERSION
fi
tar -xzf ../archives/zstd-$ZSTD_VERSION.tar.gz
pushd zstd-$ZSTD_VERSION
# build is used by facebook builder
mkdir _build
pushd _build
cmake ../build/cmake $COMMON_CMAKE_FLAGS -DZSTD_BUILD_SHARED=OFF
make -j$CPUS install
popd && popd
fi

File diff suppressed because it is too large Load Diff

View File

@@ -1,42 +0,0 @@
#!/bin/bash -ex
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
PREFIX=/opt/toolchain-v5
# NOTE: Often times when versions in the build script are changes, something
# doesn't work. To avoid rebuild of the whole toolchain but rebuild specific
# lib from 0, just comment specific line under this cript and run it. Don't
# forget to comment back to avoid unnecessary deletes next time your run this
# cript.
# rm -rf "$DIR/build"
# rm -rf "$DIR/output"
# rm -rf "$PREFIX/bin/gcc"
# rm -rf "$PREFIX/bin/ld.gold"
# rm -rf "$PREFIX/bin/gdb"
# rm -rf "$PREFIX/bin/cmake"
# rm -rf "$PREFIX/bin/clang"
# rm -rf "$PREFIX/include/bzlib.h"
# rm -rf "$PREFIX/include/fmt"
# rm -rf "$PREFIX/include/lz4.h"
# rm -rf "$PREFIX/include/lzma.h"
# rm -rf "$PREFIX/include/zlib.h"
# rm -rf "$PREFIX/include/zstd.h"
# rm -rf "$PREFIX/include/jemalloc"
# rm -rf "$PREFIX/include/boost"
# rm -rf "$PREFIX/include/double-conversion"
# rm -rf "$PREFIX/include/gflags"
# rm -rf "$PREFIX/include/libunwind.h"
# rm -rf "$PREFIX/include/glog"
# rm -rf "$PREFIX/include/event2"
# rm -rf "$PREFIX/include/sodium.h"
# rm -rf "$PREFIX/include/libaio.h"
# rm -rf "$PREFIX/include/FlexLexer.h"
# rm -rf "$PREFIX/include/snappy.h"
# rm -rf "$PREFIX/include/fizz"
# rm -rf "$PREFIX/include/folly"
# rm -rf "$PREFIX/include/proxygen"
# rm -rf "$PREFIX/include/wangle"
# rm -rf "$PREFIX/include/thrift"
# rm -rf "$PREFIX"

View File

@@ -1,41 +0,0 @@
diff -ur a/folly/CMakeLists.txt b/folly/CMakeLists.txt
--- a/folly/CMakeLists.txt 2021-12-12 23:10:42.000000000 +0100
+++ b/folly/CMakeLists.txt 2022-02-03 15:19:41.349693134 +0100
@@ -28,7 +28,6 @@
)
add_subdirectory(experimental/exception_tracer)
-add_subdirectory(logging/example)
if (PYTHON_EXTENSIONS)
# Create tree of symbolic links in structure required for successful
diff -ur a/folly/experimental/exception_tracer/ExceptionTracerLib.cpp b/folly/experimental/exception_tracer/ExceptionTracerLib.cpp
--- a/folly/experimental/exception_tracer/ExceptionTracerLib.cpp 2021-12-12 23:10:42.000000000 +0100
+++ b/folly/experimental/exception_tracer/ExceptionTracerLib.cpp 2022-02-03 15:19:11.003368891 +0100
@@ -96,6 +96,7 @@
#define __builtin_unreachable()
#endif
+#if 0
namespace __cxxabiv1 {
void __cxa_throw(
@@ -154,5 +155,5 @@
}
} // namespace std
-
+#endif
#endif // defined(__GLIBCXX__)
diff -ur a/folly/Portability.h b/folly/Portability.h
--- a/folly/Portability.h 2021-12-12 23:10:42.000000000 +0100
+++ b/folly/Portability.h 2022-02-03 15:19:11.003368891 +0100
@@ -566,7 +566,7 @@
#define FOLLY_HAS_COROUTINES 0
#elif (__cpp_coroutines >= 201703L || __cpp_impl_coroutine >= 201902L) && \
(__has_include(<coroutine>) || __has_include(<experimental/coroutine>))
-#define FOLLY_HAS_COROUTINES 1
+#define FOLLY_HAS_COROUTINES 0
// This is mainly to workaround bugs triggered by LTO, when stack allocated
// variables in await_suspend end up on a coroutine frame.
#define FOLLY_CORO_AWAIT_SUSPEND_NONTRIVIAL_ATTRIBUTES FOLLY_NOINLINE

View File

@@ -1,26 +0,0 @@
diff --git a/folly/CMakeLists.txt b/folly/CMakeLists.txt
index e0e16df..471131e 100644
--- a/folly/CMakeLists.txt
+++ b/folly/CMakeLists.txt
@@ -28,7 +28,7 @@ install(
)
add_subdirectory(experimental/exception_tracer)
-add_subdirectory(logging/example)
+# add_subdirectory(logging/example)
if (PYTHON_EXTENSIONS)
# Create tree of symbolic links in structure required for successful
diff --git a/folly/Portability.h b/folly/Portability.h
index 365ef1b..42d24b8 100644
--- a/folly/Portability.h
+++ b/folly/Portability.h
@@ -560,7 +560,7 @@ constexpr auto kCpplibVer = 0;
(defined(__cpp_coroutines) && __cpp_coroutines >= 201703L) || \
(defined(__cpp_impl_coroutine) && __cpp_impl_coroutine >= 201902L)) && \
(__has_include(<coroutine>) || __has_include(<experimental/coroutine>))
-#define FOLLY_HAS_COROUTINES 1
+#define FOLLY_HAS_COROUTINES 0
// This is mainly to workaround bugs triggered by LTO, when stack allocated
// variables in await_suspend end up on a coroutine frame.
#define FOLLY_CORO_AWAIT_SUSPEND_NONTRIVIAL_ATTRIBUTES FOLLY_NOINLINE

View File

@@ -1,29 +0,0 @@
diff -ur a/CMakeLists.txt b/CMakeLists.txt
--- a/CMakeLists.txt 2021-05-05 00:53:34.000000000 +0200
+++ b/CMakeLists.txt 2022-01-27 17:18:34.758302398 +0100
@@ -52,9 +52,9 @@
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /EHs-c-")
add_definitions(-D_HAS_EXCEPTIONS=0)
- # Disable RTTI.
- string(REGEX REPLACE "/GR" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
- set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /GR-")
+ # # Disable RTTI.
+ # string(REGEX REPLACE "/GR" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
+ # set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /GR-")
else(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# Use -Wall for clang and gcc.
if(NOT CMAKE_CXX_FLAGS MATCHES "-Wall")
@@ -77,9 +77,9 @@
string(REGEX REPLACE "-fexceptions" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-exceptions")
- # Disable RTTI.
- string(REGEX REPLACE "-frtti" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
- set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-rtti")
+ # # Disable RTTI.
+ # string(REGEX REPLACE "-frtti" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
+ # set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-rtti")
endif(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# BUILD_SHARED_LIBS is a standard CMake variable, but we declare it here to make

View File

@@ -1,75 +0,0 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBEzEOZIBEACxg/IuXERlDB48JBWmF4NxNUuuup1IhJAJyFGFSKh3OGAO2Ard
sNuRLjANsFXA7m7P5eTFcG+BoHHuAVYmKnI3PPZtHVLnUt4pGItPczQZ2BE1WpcI
ayjGTBJeKItX3Npqg9D/odO9WWS1i3FQPVdrLn0YH37/BA66jeMQCRo7g7GLpaNf
IrvYGsqTbxCwsmA37rpE7oyU4Yrf74HT091WBsRIoq/MelhbxTDMR8eu/dUGZQVc
Kj3lN55RepwWwUUKyqarY0zMt4HkFJ7v7yRL+Cvzy92Ouv4Wf2FlhNtEs5LE4Tax
W0PO5AEmUoKjX87SezQK0f652018b4u6Ex52cY7p+n5TII/UyoowH6+tY8UHo9yb
fStrqgNE/mY2bhA6+AwCaOUGsFzVVPTbjtxL3HacUP/jlA1h78V8VTvTs5d55iG7
jSqR9o05wje8rwNiXXK0xtiJahyNzL97Kn/DgPSqPIi45G+8nxWSPFM5eunBKRl9
vAnsvwrdPRsR6YR3uMHTuVhQX9/CY891MHkaZJ6wydWtKt3yQwJLYqwo5d4DwnUX
CduUwSKv+6RmtWI5ZmTQYOcBRcZyGKml9X9Q8iSbm6cnpFXmLrNQwCJN+D3SiYGc
MtbltZo0ysPMa6Xj5xFaYqWk/BI4iLb2Gs+ByGo/+a0Eq4XYBMOpitNniQARAQAB
tCdMYXNzZSBDb2xsaW4gPGxhc3NlLmNvbGxpbkB0dWthYW5pLm9yZz6JAlEEEwEK
ADsCGwMCHgECF4AECwkIBwMVCggFFgIDAQAWIQQ2kMJAzlG0Zw0wrRw47nV9aRhG
IAUCYEt9dQUJFxeR4wAKCRA47nV9aRhGIBNDEACxD6vJ+enZwe3IgkJh5JtLsC9b
MWCQRlPW1EVMsg96Cb5Rtron1eN1pp1TlzENJu1/C7C/VEsr9WwOPg26Men7fNf/
O21QM9IBWd/uB0Pu333WqKh92ESS5x9ST9DrG39nVGSPkQQBMuia72VrA+crPnwT
/h/u1IN6/sff5VDIU24rUiqW2Npy733dANruj7Ny0scRXVPltnVdhqwPHt6qNjC1
t+/cCnwHgW1BR1RYXBPpB42z/m29dL9rPrG0YPGWs2Bc+EATUICfEE6eIvwfciue
IJTjKT9Y9DrogJC2AYFhjC7N04OKdCB2hFs4BjexJwr4X0GJO7LhFl03c951AsIE
GHwrucRPB5bo2vmvQ8IvZn7CmtdUJzXv9JlyU6p+MIK1pz7TK6GgSOSffQIXZn6e
nUPtm9mEwuncOfmW8/ODYPs1gCWYgyiFJx8h7eEu+M4MxHSFBs7MwXf/Ae2fSp+M
P/p198qB8fC5oVBnF95qb0Qi0uc1D+Gb+gpBF+ymMb+s/VBOR3QWiym7AzBrJ62g
UnbC9jMLGnSRI+7p7raUfMTgXr5/oQoBw7ExJVltSSRrim2YH/t4CV47mO6dR9J3
1RtsTFIRNhz+07XPsETcuCV/dgqeC8fOFLt9MY17Sufhb1DcGy4urZBOIhXcpTV7
vHVj5IYH5nYOT49NRYkCOAQTAQIAIgUCTMQ5kgIbAwYLCQgHAwIGFQgCCQoLBBYC
AwECHgECF4AACgkQOO51fWkYRiAg4A/7BXKwoRaXrMbMPOW7vuVF7c2IKB2Yqzn1
vLBCwuEHkqY237lDcXY4/5LR+1gcZ3Duw1n/BRSm0FBdvyX/JTWiWNSDUkKAO/0l
T2Tg44YLrDT3bzwu8dbU9xQt6kH+SCOHvv5Oe4k79l5mro6fF3H1M0bN63x/YoFY
ojy09D7/JptY82oR4f/VdKnfZLJcCViCb0wp8SD2NkDAudKg+K+7PD8HlTWklQQg
TZdRXxVZKIJeU42aJDqnRbAhJd64YHyClhqut9F5LUmiP5qfLfNhkKDhNOwk2Blr
BGBJkSd7wPyzcX4Mun/L6YspHjbeVMt9TD7HQlo+OOd2OjAHCx6pqwkXnzeLPEaE
cPdQ1SHgrBViAxX3DNPubLP0Knw8XwFu96EuhHZgexE1W7bB4LFsJyXAc5k1PqPD
CLsAauxmvI2OfI7opG/8wyxDvNgoPjG8fZNAgY0REqPC0JnTXChH31IxUmhNotH8
tD3DDTZOHw05n5MwwUrEE9xiETVDfFQcMLfxZ9KLz+BC2g1t5LYublRgnCMNJzFg
sNUMM02CphABzl/LCLnumr0eyQQ/weV4twEhLwSDmqLYHL0EdYW0Y3CnnU9vmYxQ
cXKbstS71sEJJYBBmSBbf9GxkOY8BRNtwVwY0kPgxv1WqdVBiAFvfB+pyAsrax9B
3UeB7ZSwRD6JAhwEEAEKAAYFAlS25GwACgkQlbYYGy0z6ew92Q//ZA9/6piQtoW4
PwP/1DtWGyKU8hwR+9FG669iPk/dAG+yoEJtFMOUpg/FUFmCX8Bc4oEHsCVyLxKt
DcCVUIRcYNSFi5hTZaBEbwsOlDT37gtlfIIu34hhHRccKaLnN/N9gNMNw8wGh9xg
Q/KtxZwcbk/bZIlDkKTJkFBRAekdEGAFDWb/AZOy+LQxS8ZAh1eWkfV0i8opmK9k
gPXtLE0WSsqtYyGs58z+BFE9NH3tEUwK6jSvtuLwQl4UrICNbKthcpb8WwH6UXzb
q3QNSYVOpf/cqRdBJA6bvb/ku/xyKVL08lGmxD9v1b137R7mafDAFPTsvH2Mt/0V
YuhtWav3r1Bl9QksDxt2DTS8wiWDUBetGqOVdcw7vBrXPEWDNBmxeJXsiJ7zJlR+
9wrJOm6RV2+l1IPxu96EaPS+kTNBijKrhxb67bww8BTEWTd0wcdJmgWRkM8SIstp
IKqd0L2TFYph2/NtrBhRg+DIEPJPpSTGsUMcCEXCZPQ+cIdlQKsWpk0tZ62DlvEl
r7E+wgUSQolRfx5KrpZifiS2zQlhzdXv28CJhsVbLyw5fUAWUKIH/dCo5NKsNLk2
Lc5DH9VWnFgxAAtW290FqeK/4ulMq7Vs1dQSwyHM2Ni3QqqeaiOrh8gbSY5CMLFN
Y3HYRwuTYPa3AobsozCzBj0Zdf/6AFe5Ag0ETMQ5kgEQAL/FwKdjxgPxtSpgq1SM
zgZtTTyLqhgGD3NZfadHWHYRIL38NDV3JeTA79Y2zj2dj7KQPDT+0aqeizTV2E3j
P3iCQ53VOT4consBaQAgKexpptnS+T1DobtICFJ0GGzf0HRj6KO2zSOuOitWPWlU
wbvX7M0LLI2+hqlx0jTPqbJFZ/Za6KTtbS6xdCPVUpUqYZQpokEZcwQmUp8Q+lGo
JD2sNYCZyap63X/aAOgCGr2RXYddOH5e8vGzGW+mwtCv+WQ9Ay35mGqI5MqkbZd1
Qbuv2b1647E/QEEucfRHVbJVKGGPpFMUJtcItyyIt5jo+r9CCL4Cs47dF/9/RNwu
NvpvHXUyqMBQdWNZRMx4k/NGD/WviPi9m6mIMui6rOQsSOaqYdcUX4Nq2Orr3Oaz
2JPQdUfeI23iot1vK8hxvUCQTV3HfJghizN6spVl0yQOKBiE8miJRgrjHilH3hTb
xoo42xDkNAq+CQo3QAm1ibDxKCDq0RcWPjcCRAN/Q5MmpcodpdKkzV0yGIS4g7s5
frVrgV/kox2r4/Yxsr8K909+4H82AjTKGX/BmsQFCTAqBk6p7I0zxjIqJ/w33TZB
Q0Pn4r3WIlUPafzY6a9/LAvN1fHRxf9SpCByJsszD03Qu5f5TB8gthsdnVmTo7jj
iordEKMtw2aEMLzdWWTQ/TNVABEBAAGJAjwEGAEKACYCGwwWIQQ2kMJAzlG0Zw0w
rRw47nV9aRhGIAUCYEt9YAUJFxeRzgAKCRA47nV9aRhGIMLtD/9HuKM4pngImcuz
YwzQmdv4j26YYyh4jVsKEmVWTiRcehEgUIlrWkCu3qzd5NK+RetS7kJ8MPnzEUfj
YbpdC6yrF6n1mSrZZ4VJMkV2ev37bIgXM+Wp1mCAGbjNxQnjn9RabT/gjIqmGuRn
AP7RsSeOSuO/gO9h2Pteciz23ussTilB+8cTooQEQQZe6Kv/zukvL+ccSehLHsZ7
qVfRUAmtt8nFkXXE+s8jfLfhqstaI2/RJu5witaPcXM8Mnz2E95aASAbZy0eQot9
0Pvf07n9yuC3tueTvzvlXx3h5U3yT44tIOmzANIQjay1TGdm+RBJ2ZYyhyLawlZ2
NVUXXSp4QZZXPA0UWbF+pb7Q9cdKDNFVuvGBljuea0Yd0T2o+ibDq43HziX9ll+l
SXk9mqvW1UcDOaxWrSsm1Gc1O9g3wqH5xHAhtY8GPh/7VgAawskPkmnlkMW6pYPy
zibbeISJL1gd1jIT63y6aoVrtNoo+wYJm280ROflh4+5QOo6QJ+jm70fkXSG/qJ5
a8/qCPTHkJc/rpkL6/TDQAJURi9RhDAC0gb40HtusbN1LZEA+i0cWTmYXap+DB4Y
R4pApilpaG87M+VUokR4xpnx7vTb2MPa7Mdenvi9FEGnKXadmT8038vlfzz5GGUT
MlVin9BQPTpdA+PpRiJvKJgVDeAFOg==
=asTC
-----END PGP PUBLIC KEY BLOCK-----

View File

@@ -1,4 +1,4 @@
// Copyright 2024 Memgraph Ltd. // Copyright 2023 Memgraph Ltd.
// //
// Use of this software is governed by the Business Source License // Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source // included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
@@ -234,58 +234,8 @@ inline mgp_type *type_duration() { return MgInvoke<mgp_type *>(mgp_type_duration
inline mgp_type *type_nullable(mgp_type *type) { return MgInvoke<mgp_type *>(mgp_type_nullable, type); } inline mgp_type *type_nullable(mgp_type *type) { return MgInvoke<mgp_type *>(mgp_type_nullable, type); }
inline bool create_label_index(mgp_graph *graph, const char *label) {
return MgInvoke<int>(mgp_create_label_index, graph, label);
}
inline bool drop_label_index(mgp_graph *graph, const char *label) {
return MgInvoke<int>(mgp_drop_label_index, graph, label);
}
inline mgp_list *list_all_label_indices(mgp_graph *graph, mgp_memory *memory) {
return MgInvoke<mgp_list *>(mgp_list_all_label_indices, graph, memory);
}
inline bool create_label_property_index(mgp_graph *graph, const char *label, const char *property) {
return MgInvoke<int>(mgp_create_label_property_index, graph, label, property);
}
inline bool drop_label_property_index(mgp_graph *graph, const char *label, const char *property) {
return MgInvoke<int>(mgp_drop_label_property_index, graph, label, property);
}
inline mgp_list *list_all_label_property_indices(mgp_graph *graph, mgp_memory *memory) {
return MgInvoke<mgp_list *>(mgp_list_all_label_property_indices, graph, memory);
}
inline bool create_existence_constraint(mgp_graph *graph, const char *label, const char *property) {
return MgInvoke<int>(mgp_create_existence_constraint, graph, label, property);
}
inline bool drop_existence_constraint(mgp_graph *graph, const char *label, const char *property) {
return MgInvoke<int>(mgp_drop_existence_constraint, graph, label, property);
}
inline mgp_list *list_all_existence_constraints(mgp_graph *graph, mgp_memory *memory) {
return MgInvoke<mgp_list *>(mgp_list_all_existence_constraints, graph, memory);
}
inline bool create_unique_constraint(mgp_graph *memgraph_graph, const char *label, mgp_value *properties) {
return MgInvoke<int>(mgp_create_unique_constraint, memgraph_graph, label, properties);
}
inline bool drop_unique_constraint(mgp_graph *memgraph_graph, const char *label, mgp_value *properties) {
return MgInvoke<int>(mgp_drop_unique_constraint, memgraph_graph, label, properties);
}
inline mgp_list *list_all_unique_constraints(mgp_graph *graph, mgp_memory *memory) {
return MgInvoke<mgp_list *>(mgp_list_all_unique_constraints, graph, memory);
}
// mgp_graph // mgp_graph
inline bool graph_is_transactional(mgp_graph *graph) { return MgInvoke<int>(mgp_graph_is_transactional, graph); }
inline bool graph_is_mutable(mgp_graph *graph) { return MgInvoke<int>(mgp_graph_is_mutable, graph); } inline bool graph_is_mutable(mgp_graph *graph) { return MgInvoke<int>(mgp_graph_is_mutable, graph); }
inline mgp_vertex *graph_create_vertex(mgp_graph *graph, mgp_memory *memory) { inline mgp_vertex *graph_create_vertex(mgp_graph *graph, mgp_memory *memory) {
@@ -315,31 +265,12 @@ inline mgp_edge *graph_edge_set_to(struct mgp_graph *graph, struct mgp_edge *e,
return MgInvoke<mgp_edge *>(mgp_graph_edge_set_to, graph, e, new_to, memory); return MgInvoke<mgp_edge *>(mgp_graph_edge_set_to, graph, e, new_to, memory);
} }
inline mgp_edge *graph_edge_change_type(struct mgp_graph *graph, struct mgp_edge *e, struct mgp_edge_type new_type,
mgp_memory *memory) {
return MgInvoke<mgp_edge *>(mgp_graph_edge_change_type, graph, e, new_type, memory);
}
inline void graph_delete_edge(mgp_graph *graph, mgp_edge *edge) { MgInvokeVoid(mgp_graph_delete_edge, graph, edge); } inline void graph_delete_edge(mgp_graph *graph, mgp_edge *edge) { MgInvokeVoid(mgp_graph_delete_edge, graph, edge); }
inline mgp_vertex *graph_get_vertex_by_id(mgp_graph *g, mgp_vertex_id id, mgp_memory *memory) { inline mgp_vertex *graph_get_vertex_by_id(mgp_graph *g, mgp_vertex_id id, mgp_memory *memory) {
return MgInvoke<mgp_vertex *>(mgp_graph_get_vertex_by_id, g, id, memory); return MgInvoke<mgp_vertex *>(mgp_graph_get_vertex_by_id, g, id, memory);
} }
inline bool graph_has_text_index(mgp_graph *graph, const char *index_name) {
return MgInvoke<int>(mgp_graph_has_text_index, graph, index_name);
}
inline mgp_map *graph_search_text_index(mgp_graph *graph, const char *index_name, const char *search_query,
mgp_memory *memory) {
return MgInvoke<mgp_map *>(mgp_graph_search_text_index, graph, index_name, search_query, memory);
}
inline mgp_map *graph_regex_search_text_index(mgp_graph *graph, const char *index_name, const char *search_query,
mgp_memory *memory) {
return MgInvoke<mgp_map *>(mgp_graph_regex_search_text_index, graph, index_name, search_query, memory);
}
inline mgp_vertices_iterator *graph_iter_vertices(mgp_graph *g, mgp_memory *memory) { inline mgp_vertices_iterator *graph_iter_vertices(mgp_graph *g, mgp_memory *memory) {
return MgInvoke<mgp_vertices_iterator *>(mgp_graph_iter_vertices, g, memory); return MgInvoke<mgp_vertices_iterator *>(mgp_graph_iter_vertices, g, memory);
} }
@@ -392,8 +323,6 @@ inline mgp_list *list_copy(mgp_list *list, mgp_memory *memory) {
inline void list_destroy(mgp_list *list) { mgp_list_destroy(list); } inline void list_destroy(mgp_list *list) { mgp_list_destroy(list); }
inline bool list_contains_deleted(mgp_list *list) { return MgInvoke<int>(mgp_list_contains_deleted, list); }
inline void list_append(mgp_list *list, mgp_value *val) { MgInvokeVoid(mgp_list_append, list, val); } inline void list_append(mgp_list *list, mgp_value *val) { MgInvokeVoid(mgp_list_append, list, val); }
inline void list_append_extend(mgp_list *list, mgp_value *val) { MgInvokeVoid(mgp_list_append_extend, list, val); } inline void list_append_extend(mgp_list *list, mgp_value *val) { MgInvokeVoid(mgp_list_append_extend, list, val); }
@@ -412,8 +341,6 @@ inline mgp_map *map_copy(mgp_map *map, mgp_memory *memory) { return MgInvoke<mgp
inline void map_destroy(mgp_map *map) { mgp_map_destroy(map); } inline void map_destroy(mgp_map *map) { mgp_map_destroy(map); }
inline bool map_contains_deleted(mgp_map *map) { return MgInvoke<int>(mgp_map_contains_deleted, map); }
inline void map_insert(mgp_map *map, const char *key, mgp_value *value) { inline void map_insert(mgp_map *map, const char *key, mgp_value *value) {
MgInvokeVoid(mgp_map_insert, map, key, value); MgInvokeVoid(mgp_map_insert, map, key, value);
} }
@@ -428,8 +355,6 @@ inline size_t map_size(mgp_map *map) { return MgInvoke<size_t>(mgp_map_size, map
inline mgp_value *map_at(mgp_map *map, const char *key) { return MgInvoke<mgp_value *>(mgp_map_at, map, key); } inline mgp_value *map_at(mgp_map *map, const char *key) { return MgInvoke<mgp_value *>(mgp_map_at, map, key); }
inline bool key_exists(mgp_map *map, const char *key) { return MgInvoke<int>(mgp_key_exists, map, key); }
inline const char *map_item_key(mgp_map_item *item) { return MgInvoke<const char *>(mgp_map_item_key, item); } inline const char *map_item_key(mgp_map_item *item) { return MgInvoke<const char *>(mgp_map_item_key, item); }
inline mgp_value *map_item_value(mgp_map_item *item) { return MgInvoke<mgp_value *>(mgp_map_item_value, item); } inline mgp_value *map_item_value(mgp_map_item *item) { return MgInvoke<mgp_value *>(mgp_map_item_value, item); }
@@ -462,8 +387,6 @@ inline mgp_vertex *vertex_copy(mgp_vertex *v, mgp_memory *memory) {
inline void vertex_destroy(mgp_vertex *v) { mgp_vertex_destroy(v); } inline void vertex_destroy(mgp_vertex *v) { mgp_vertex_destroy(v); }
inline bool vertex_is_deleted(mgp_vertex *v) { return MgInvoke<int>(mgp_vertex_is_deleted, v); }
inline bool vertex_equal(mgp_vertex *v1, mgp_vertex *v2) { return MgInvoke<int>(mgp_vertex_equal, v1, v2); } inline bool vertex_equal(mgp_vertex *v1, mgp_vertex *v2) { return MgInvoke<int>(mgp_vertex_equal, v1, v2); }
inline size_t vertex_labels_count(mgp_vertex *v) { return MgInvoke<size_t>(mgp_vertex_labels_count, v); } inline size_t vertex_labels_count(mgp_vertex *v) { return MgInvoke<size_t>(mgp_vertex_labels_count, v); }
@@ -516,8 +439,6 @@ inline mgp_edge *edge_copy(mgp_edge *e, mgp_memory *memory) { return MgInvoke<mg
inline void edge_destroy(mgp_edge *e) { mgp_edge_destroy(e); } inline void edge_destroy(mgp_edge *e) { mgp_edge_destroy(e); }
inline bool edge_is_deleted(mgp_edge *e) { return MgInvoke<int>(mgp_edge_is_deleted, e); }
inline bool edge_equal(mgp_edge *e1, mgp_edge *e2) { return MgInvoke<int>(mgp_edge_equal, e1, e2); } inline bool edge_equal(mgp_edge *e1, mgp_edge *e2) { return MgInvoke<int>(mgp_edge_equal, e1, e2); }
inline mgp_edge_type edge_get_type(mgp_edge *e) { return MgInvoke<mgp_edge_type>(mgp_edge_get_type, e); } inline mgp_edge_type edge_get_type(mgp_edge *e) { return MgInvoke<mgp_edge_type>(mgp_edge_get_type, e); }
@@ -554,8 +475,6 @@ inline mgp_path *path_copy(mgp_path *path, mgp_memory *memory) {
inline void path_destroy(mgp_path *path) { mgp_path_destroy(path); } inline void path_destroy(mgp_path *path) { mgp_path_destroy(path); }
inline bool path_contains_deleted(mgp_path *path) { return MgInvoke<int>(mgp_path_contains_deleted, path); }
inline void path_expand(mgp_path *path, mgp_edge *edge) { MgInvokeVoid(mgp_path_expand, path, edge); } inline void path_expand(mgp_path *path, mgp_edge *edge) { MgInvokeVoid(mgp_path_expand, path, edge); }
inline void path_pop(mgp_path *path) { MgInvokeVoid(mgp_path_pop, path); } inline void path_pop(mgp_path *path) { MgInvokeVoid(mgp_path_pop, path); }

View File

@@ -1,4 +1,4 @@
// Copyright 2024 Memgraph Ltd. // Copyright 2023 Memgraph Ltd.
// //
// Use of this software is governed by the Business Source License // Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source // included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
@@ -25,7 +25,7 @@ namespace mg_exception {
template <typename FirstArg, typename... Args> template <typename FirstArg, typename... Args>
std::string StringSerialize(FirstArg &&firstArg, Args &&...args) { std::string StringSerialize(FirstArg &&firstArg, Args &&...args) {
std::stringstream stream; std::stringstream stream;
stream << std::forward<FirstArg>(firstArg); stream << firstArg;
((stream << " " << args), ...); ((stream << " " << args), ...);
return stream.str(); return stream.str();
} }

View File

@@ -1,4 +1,4 @@
// Copyright 2024 Memgraph Ltd. // Copyright 2023 Memgraph Ltd.
// //
// Use of this software is governed by the Business Source License // Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source // included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
@@ -115,6 +115,19 @@ void mgp_global_free(void *p);
/// State of the graph database. /// State of the graph database.
struct mgp_graph; struct mgp_graph;
/// Allocations are tracked only for master thread. If new threads are spawned
/// inside procedure, by calling following function with thread id
/// you can start tracking allocations for that thread too. This
/// is important if you need query memory limit to work
/// for given procedure or per procedure memory limit.
enum mgp_error mgp_track_thread_allocations(struct mgp_graph *graph, const char *thread_id);
/// Once allocations are tracked for custom thread, you need to stop tracking allocations
/// for given thread, before thread finishes with execution, or is detached.
/// Otherwise it might result in slowdown of system due to unnecessary tracking of
/// allocations.
enum mgp_error mgp_untrack_thread_allocations(struct mgp_graph *graph, const char *thread_id);
/// Allocations are tracked only for master thread. If new threads are spawned /// Allocations are tracked only for master thread. If new threads are spawned
/// inside procedure, by calling following function /// inside procedure, by calling following function
/// you can start tracking allocations for current thread too. This /// you can start tracking allocations for current thread too. This
@@ -429,9 +442,6 @@ enum mgp_error mgp_list_copy(struct mgp_list *list, struct mgp_memory *memory, s
/// Free the memory used by the given mgp_list and contained elements. /// Free the memory used by the given mgp_list and contained elements.
void mgp_list_destroy(struct mgp_list *list); void mgp_list_destroy(struct mgp_list *list);
/// Return whether the given mgp_list contains any deleted values.
enum mgp_error mgp_list_contains_deleted(struct mgp_list *list, int *result);
/// Append a copy of mgp_value to mgp_list if capacity allows. /// Append a copy of mgp_value to mgp_list if capacity allows.
/// The list copies the given value and therefore does not take ownership of the /// The list copies the given value and therefore does not take ownership of the
/// original value. You still need to call mgp_value_destroy to free the /// original value. You still need to call mgp_value_destroy to free the
@@ -472,9 +482,6 @@ enum mgp_error mgp_map_copy(struct mgp_map *map, struct mgp_memory *memory, stru
/// Free the memory used by the given mgp_map and contained items. /// Free the memory used by the given mgp_map and contained items.
void mgp_map_destroy(struct mgp_map *map); void mgp_map_destroy(struct mgp_map *map);
/// Return whether the given mgp_map contains any deleted values.
enum mgp_error mgp_map_contains_deleted(struct mgp_map *map, int *result);
/// Insert a new mapping from a NULL terminated character string to a value. /// Insert a new mapping from a NULL terminated character string to a value.
/// If a mapping with the same key already exists, it is *not* replaced. /// If a mapping with the same key already exists, it is *not* replaced.
/// In case of insertion, both the string and the value are copied into the map. /// In case of insertion, both the string and the value are copied into the map.
@@ -504,9 +511,6 @@ enum mgp_error mgp_map_size(struct mgp_map *map, size_t *result);
/// Result is NULL if no mapping exists. /// Result is NULL if no mapping exists.
enum mgp_error mgp_map_at(struct mgp_map *map, const char *key, struct mgp_value **result); enum mgp_error mgp_map_at(struct mgp_map *map, const char *key, struct mgp_value **result);
/// Returns true if key in map.
enum mgp_error mgp_key_exists(struct mgp_map *map, const char *key, int *result);
/// An item in the mgp_map. /// An item in the mgp_map.
struct mgp_map_item; struct mgp_map_item;
@@ -558,9 +562,6 @@ enum mgp_error mgp_path_copy(struct mgp_path *path, struct mgp_memory *memory, s
/// Free the memory used by the given mgp_path and contained vertices and edges. /// Free the memory used by the given mgp_path and contained vertices and edges.
void mgp_path_destroy(struct mgp_path *path); void mgp_path_destroy(struct mgp_path *path);
/// Return whether the given mgp_path contains any deleted values.
enum mgp_error mgp_path_contains_deleted(struct mgp_path *path, int *result);
/// Append an edge continuing from the last vertex on the path. /// Append an edge continuing from the last vertex on the path.
/// The edge is copied into the path. Therefore, the path does not take /// The edge is copied into the path. Therefore, the path does not take
/// ownership of the original edge, so you still need to free the edge memory /// ownership of the original edge, so you still need to free the edge memory
@@ -734,9 +735,6 @@ enum mgp_error mgp_vertex_copy(struct mgp_vertex *v, struct mgp_memory *memory,
/// Free the memory used by a mgp_vertex. /// Free the memory used by a mgp_vertex.
void mgp_vertex_destroy(struct mgp_vertex *v); void mgp_vertex_destroy(struct mgp_vertex *v);
/// Return whether the given mgp_vertex is deleted.
enum mgp_error mgp_vertex_is_deleted(struct mgp_vertex *v, int *result);
/// Result is non-zero if given vertices are equal, otherwise 0. /// Result is non-zero if given vertices are equal, otherwise 0.
enum mgp_error mgp_vertex_equal(struct mgp_vertex *v1, struct mgp_vertex *v2, int *result); enum mgp_error mgp_vertex_equal(struct mgp_vertex *v1, struct mgp_vertex *v2, int *result);
@@ -831,9 +829,6 @@ enum mgp_error mgp_edge_copy(struct mgp_edge *e, struct mgp_memory *memory, stru
/// Free the memory used by a mgp_edge. /// Free the memory used by a mgp_edge.
void mgp_edge_destroy(struct mgp_edge *e); void mgp_edge_destroy(struct mgp_edge *e);
/// Return whether the given mgp_edge is deleted.
enum mgp_error mgp_edge_is_deleted(struct mgp_edge *e, int *result);
/// Result is non-zero if given edges are equal, otherwise 0. /// Result is non-zero if given edges are equal, otherwise 0.
enum mgp_error mgp_edge_equal(struct mgp_edge *e1, struct mgp_edge *e2, int *result); enum mgp_error mgp_edge_equal(struct mgp_edge *e1, struct mgp_edge *e2, int *result);
@@ -891,94 +886,12 @@ enum mgp_error mgp_edge_iter_properties(struct mgp_edge *e, struct mgp_memory *m
enum mgp_error mgp_graph_get_vertex_by_id(struct mgp_graph *g, struct mgp_vertex_id id, struct mgp_memory *memory, enum mgp_error mgp_graph_get_vertex_by_id(struct mgp_graph *g, struct mgp_vertex_id id, struct mgp_memory *memory,
struct mgp_vertex **result); struct mgp_vertex **result);
/// Result is non-zero if the index with the given name exists.
/// Current implementation always returns without errors.
enum mgp_error mgp_graph_has_text_index(struct mgp_graph *graph, const char *index_name, int *result);
/// Search the named text index for the given query. The result is a list of the vertices whose text properties match
/// the given query.
/// Return mgp_error::MGP_ERROR_UNABLE_TO_ALLOCATE if unable to allocate search result vertices.
enum mgp_error mgp_graph_search_text_index(struct mgp_graph *graph, const char *index_name, const char *search_query,
struct mgp_memory *memory, struct mgp_map **result);
/// Search the named text index for the given regex. The result is a list of the vertices whose text properties match
/// the given query.
/// Return mgp_error::MGP_ERROR_UNABLE_TO_ALLOCATE if unable to allocate search result vertices.
enum mgp_error mgp_graph_regex_search_text_index(struct mgp_graph *graph, const char *index_name,
const char *search_query, struct mgp_memory *memory,
struct mgp_map **result);
/// Creates label index for given label.
/// mgp_error::MGP_ERROR_NO_ERROR is always returned.
/// if label index already exists, result will be 0, otherwise 1.
enum mgp_error mgp_create_label_index(struct mgp_graph *graph, const char *label, int *result);
/// Drop label index.
enum mgp_error mgp_drop_label_index(struct mgp_graph *graph, const char *label, int *result);
/// List all label indices.
enum mgp_error mgp_list_all_label_indices(struct mgp_graph *graph, struct mgp_memory *memory, struct mgp_list **result);
/// Creates label-property index for given label and propery.
/// mgp_error::MGP_ERROR_NO_ERROR is always returned.
/// if label property index already exists, result will be 0, otherwise 1.
enum mgp_error mgp_create_label_property_index(struct mgp_graph *graph, const char *label, const char *property,
int *result);
/// Drops label-property index for given label and propery.
/// mgp_error::MGP_ERROR_NO_ERROR is always returned.
/// if dropping label property index failed, result will be 0, otherwise 1.
enum mgp_error mgp_drop_label_property_index(struct mgp_graph *graph, const char *label, const char *property,
int *result);
/// List all label+property indices.
enum mgp_error mgp_list_all_label_property_indices(struct mgp_graph *graph, struct mgp_memory *memory,
struct mgp_list **result);
/// Creates existence constraint for given label and property.
/// mgp_error::MGP_ERROR_NO_ERROR is always returned.
/// if creating existence constraint failed, result will be 0, otherwise 1.
enum mgp_error mgp_create_existence_constraint(struct mgp_graph *graph, const char *label, const char *property,
int *result);
/// Drops existence constraint for given label and property.
/// mgp_error::MGP_ERROR_NO_ERROR is always returned.
/// if dropping existence constraint failed, result will be 0, otherwise 1.
enum mgp_error mgp_drop_existence_constraint(struct mgp_graph *graph, const char *label, const char *property,
int *result);
/// List all existence constraints.
enum mgp_error mgp_list_all_existence_constraints(struct mgp_graph *graph, struct mgp_memory *memory,
struct mgp_list **result);
/// Creates unique constraint for given label and properties.
/// mgp_error::MGP_ERROR_NO_ERROR is always returned.
/// if creating unique constraint failed, result will be 0, otherwise 1.
enum mgp_error mgp_create_unique_constraint(struct mgp_graph *graph, const char *label, struct mgp_value *properties,
int *result);
/// Drops unique constraint for given label and properties.
/// mgp_error::MGP_ERROR_NO_ERROR is always returned.
/// if dropping unique constraint failed, result will be 0, otherwise 1.
enum mgp_error mgp_drop_unique_constraint(struct mgp_graph *graph, const char *label, struct mgp_value *properties,
int *result);
/// List all unique constraints
enum mgp_error mgp_list_all_unique_constraints(struct mgp_graph *graph, struct mgp_memory *memory,
struct mgp_list **result);
/// Result is non-zero if the graph can be modified. /// Result is non-zero if the graph can be modified.
/// If a graph is immutable, then vertices cannot be created or deleted, and all of the returned vertices will be /// If a graph is immutable, then vertices cannot be created or deleted, and all of the returned vertices will be
/// immutable also. The same applies for edges. /// immutable also. The same applies for edges.
/// Current implementation always returns without errors. /// Current implementation always returns without errors.
enum mgp_error mgp_graph_is_mutable(struct mgp_graph *graph, int *result); enum mgp_error mgp_graph_is_mutable(struct mgp_graph *graph, int *result);
/// Result is non-zero if the graph is in transactional storage mode.
/// If a graph is not in transactional mode (i.e. analytical mode), then vertices and edges can be missing
/// because changes from other transactions are visible.
/// Current implementation always returns without errors.
enum mgp_error mgp_graph_is_transactional(struct mgp_graph *graph, int *result);
/// Add a new vertex to the graph. /// Add a new vertex to the graph.
/// Resulting vertex must be freed using mgp_vertex_destroy. /// Resulting vertex must be freed using mgp_vertex_destroy.
/// Return mgp_error::MGP_ERROR_IMMUTABLE_OBJECT if `graph` is immutable. /// Return mgp_error::MGP_ERROR_IMMUTABLE_OBJECT if `graph` is immutable.
@@ -1021,13 +934,6 @@ enum mgp_error mgp_graph_edge_set_from(struct mgp_graph *graph, struct mgp_edge
enum mgp_error mgp_graph_edge_set_to(struct mgp_graph *graph, struct mgp_edge *e, struct mgp_vertex *new_to, enum mgp_error mgp_graph_edge_set_to(struct mgp_graph *graph, struct mgp_edge *e, struct mgp_vertex *new_to,
struct mgp_memory *memory, struct mgp_edge **result); struct mgp_memory *memory, struct mgp_edge **result);
/// Change edge type
/// Return mgp_error::MGP_ERROR_IMMUTABLE_OBJECT if `graph` is immutable.
/// Return mgp_error::MGP_ERROR_SERIALIZATION_ERROR if `edge`, its source or destination vertex has been modified by
/// another transaction.
enum mgp_error mgp_graph_edge_change_type(struct mgp_graph *graph, struct mgp_edge *e, struct mgp_edge_type new_type,
struct mgp_memory *memory, struct mgp_edge **result);
/// Delete an edge from the graph. /// Delete an edge from the graph.
/// Return mgp_error::MGP_ERROR_IMMUTABLE_OBJECT if `graph` is immutable. /// Return mgp_error::MGP_ERROR_IMMUTABLE_OBJECT if `graph` is immutable.
/// Return mgp_error::MGP_ERROR_SERIALIZATION_ERROR if `edge`, its source or destination vertex has been modified by /// Return mgp_error::MGP_ERROR_SERIALIZATION_ERROR if `edge`, its source or destination vertex has been modified by

File diff suppressed because it is too large Load Diff

View File

@@ -4,8 +4,7 @@ include(GNUInstallDirs)
include(ProcessorCount) include(ProcessorCount)
ProcessorCount(NPROC) ProcessorCount(NPROC)
if (NPROC EQUAL 0)
if(NPROC EQUAL 0)
set(NPROC 1) set(NPROC 1)
endif() endif()
@@ -13,7 +12,6 @@ find_package(Boost 1.78 REQUIRED)
find_package(BZip2 1.0.6 REQUIRED) find_package(BZip2 1.0.6 REQUIRED)
find_package(Threads REQUIRED) find_package(Threads REQUIRED)
set(GFLAGS_NOTHREADS OFF) set(GFLAGS_NOTHREADS OFF)
# NOTE: config/generate.py depends on the gflags help XML format. # NOTE: config/generate.py depends on the gflags help XML format.
find_package(gflags REQUIRED) find_package(gflags REQUIRED)
find_package(fmt 8.0.1) find_package(fmt 8.0.1)
@@ -25,27 +23,24 @@ set(LIB_DIR ${CMAKE_CURRENT_SOURCE_DIR})
function(import_header_library name include_dir) function(import_header_library name include_dir)
add_library(${name} INTERFACE IMPORTED GLOBAL) add_library(${name} INTERFACE IMPORTED GLOBAL)
set_property(TARGET ${name} PROPERTY set_property(TARGET ${name} PROPERTY
INTERFACE_INCLUDE_DIRECTORIES ${include_dir}) INTERFACE_INCLUDE_DIRECTORIES ${include_dir})
string(TOUPPER ${name} _upper_name) string(TOUPPER ${name} _upper_name)
set(${_upper_name}_INCLUDE_DIR ${include_dir} CACHE FILEPATH set(${_upper_name}_INCLUDE_DIR ${include_dir} CACHE FILEPATH
"Path to ${name} include directory" FORCE) "Path to ${name} include directory" FORCE)
mark_as_advanced(${_upper_name}_INCLUDE_DIR) mark_as_advanced(${_upper_name}_INCLUDE_DIR)
add_library(lib::${name} ALIAS ${name}) add_library(lib::${name} ALIAS ${name})
endfunction(import_header_library) endfunction(import_header_library)
function(import_library name type location include_dir) function(import_library name type location include_dir)
add_library(${name} ${type} IMPORTED GLOBAL) add_library(${name} ${type} IMPORTED GLOBAL)
if (${ARGN})
if(${ARGN})
# Optional argument is the name of the external project that we need to # Optional argument is the name of the external project that we need to
# depend on. # depend on.
add_dependencies(${name} ${ARGN0}) add_dependencies(${name} ${ARGN0})
else() else()
add_dependencies(${name} ${name}-proj) add_dependencies(${name} ${name}-proj)
endif() endif()
set_property(TARGET ${name} PROPERTY IMPORTED_LOCATION ${location}) set_property(TARGET ${name} PROPERTY IMPORTED_LOCATION ${location})
# We need to create the include directory first in order to be able to add it # We need to create the include directory first in order to be able to add it
# as an include directory. The header files in the include directory will be # as an include directory. The header files in the include directory will be
# generated later during the build process. # generated later during the build process.
@@ -65,34 +60,29 @@ function(add_external_project name)
set(options NO_C_COMPILER) set(options NO_C_COMPILER)
set(one_value_kwargs SOURCE_DIR BUILD_IN_SOURCE) set(one_value_kwargs SOURCE_DIR BUILD_IN_SOURCE)
set(multi_value_kwargs CMAKE_ARGS DEPENDS INSTALL_COMMAND BUILD_COMMAND set(multi_value_kwargs CMAKE_ARGS DEPENDS INSTALL_COMMAND BUILD_COMMAND
CONFIGURE_COMMAND) CONFIGURE_COMMAND)
cmake_parse_arguments(KW "${options}" "${one_value_kwargs}" "${multi_value_kwargs}" ${ARGN}) cmake_parse_arguments(KW "${options}" "${one_value_kwargs}" "${multi_value_kwargs}" ${ARGN})
set(source_dir ${CMAKE_CURRENT_SOURCE_DIR}/${name}) set(source_dir ${CMAKE_CURRENT_SOURCE_DIR}/${name})
if (KW_SOURCE_DIR)
if(KW_SOURCE_DIR)
set(source_dir ${KW_SOURCE_DIR}) set(source_dir ${KW_SOURCE_DIR})
endif() endif()
set(build_in_source 0) set(build_in_source 0)
if (KW_BUILD_IN_SOURCE)
if(KW_BUILD_IN_SOURCE)
set(build_in_source ${KW_BUILD_IN_SOURCE}) set(build_in_source ${KW_BUILD_IN_SOURCE})
endif() endif()
if (NOT KW_NO_C_COMPILER)
if(NOT KW_NO_C_COMPILER)
set(KW_CMAKE_ARGS -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER} ${KW_CMAKE_ARGS}) set(KW_CMAKE_ARGS -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER} ${KW_CMAKE_ARGS})
endif() endif()
ExternalProject_Add(${name}-proj DEPENDS ${KW_DEPENDS} ExternalProject_Add(${name}-proj DEPENDS ${KW_DEPENDS}
PREFIX ${source_dir} SOURCE_DIR ${source_dir} PREFIX ${source_dir} SOURCE_DIR ${source_dir}
BUILD_IN_SOURCE ${build_in_source} BUILD_IN_SOURCE ${build_in_source}
CONFIGURE_COMMAND ${KW_CONFIGURE_COMMAND} CONFIGURE_COMMAND ${KW_CONFIGURE_COMMAND}
CMAKE_ARGS -DCMAKE_BUILD_TYPE=Release CMAKE_ARGS -DCMAKE_BUILD_TYPE=Release
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER} -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_INSTALL_PREFIX=${source_dir} -DCMAKE_INSTALL_PREFIX=${source_dir}
${KW_CMAKE_ARGS} ${KW_CMAKE_ARGS}
INSTALL_COMMAND ${KW_INSTALL_COMMAND} INSTALL_COMMAND ${KW_INSTALL_COMMAND}
BUILD_COMMAND ${KW_BUILD_COMMAND}) BUILD_COMMAND ${KW_BUILD_COMMAND})
endfunction(add_external_project) endfunction(add_external_project)
# Calls `add_external_project`, sets NAME_LIBRARY, NAME_INCLUDE_DIR variables # Calls `add_external_project`, sets NAME_LIBRARY, NAME_INCLUDE_DIR variables
@@ -101,9 +91,9 @@ macro(import_external_library name type library_location include_dir)
add_external_project(${name} ${ARGN}) add_external_project(${name} ${ARGN})
string(TOUPPER ${name} _upper_name) string(TOUPPER ${name} _upper_name)
set(${_upper_name}_LIBRARY ${library_location} CACHE FILEPATH set(${_upper_name}_LIBRARY ${library_location} CACHE FILEPATH
"Path to ${name} library" FORCE) "Path to ${name} library" FORCE)
set(${_upper_name}_INCLUDE_DIR ${include_dir} CACHE FILEPATH set(${_upper_name}_INCLUDE_DIR ${include_dir} CACHE FILEPATH
"Path to ${name} include directory" FORCE) "Path to ${name} include directory" FORCE)
mark_as_advanced(${_upper_name}_LIBRARY ${_upper_name}_INCLUDE_DIR) mark_as_advanced(${_upper_name}_LIBRARY ${_upper_name}_INCLUDE_DIR)
import_library(${name} ${type} ${${_upper_name}_LIBRARY} ${${_upper_name}_INCLUDE_DIR}) import_library(${name} ${type} ${${_upper_name}_LIBRARY} ${${_upper_name}_INCLUDE_DIR})
endmacro(import_external_library) endmacro(import_external_library)
@@ -125,10 +115,10 @@ import_external_library(antlr4 STATIC
${CMAKE_CURRENT_SOURCE_DIR}/antlr4/runtime/Cpp/include/antlr4-runtime ${CMAKE_CURRENT_SOURCE_DIR}/antlr4/runtime/Cpp/include/antlr4-runtime
SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/antlr4/runtime/Cpp SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/antlr4/runtime/Cpp
CMAKE_ARGS # http://stackoverflow.com/questions/37096062/get-a-basic-c-program-to-compile-using-clang-on-ubuntu-16/38385967#38385967 CMAKE_ARGS # http://stackoverflow.com/questions/37096062/get-a-basic-c-program-to-compile-using-clang-on-ubuntu-16/38385967#38385967
-DWITH_LIBCXX=OFF # because of debian bug -DWITH_LIBCXX=OFF # because of debian bug
-DCMAKE_SKIP_INSTALL_ALL_DEPENDENCY=true -DCMAKE_SKIP_INSTALL_ALL_DEPENDENCY=true
-DCMAKE_CXX_STANDARD=20 -DCMAKE_CXX_STANDARD=20
-DANTLR_BUILD_CPP_TESTS=OFF -DANTLR_BUILD_CPP_TESTS=OFF
BUILD_COMMAND $(MAKE) antlr4_static BUILD_COMMAND $(MAKE) antlr4_static
INSTALL_COMMAND $(MAKE) install) INSTALL_COMMAND $(MAKE) install)
@@ -136,7 +126,6 @@ import_external_library(antlr4 STATIC
import_external_library(benchmark STATIC import_external_library(benchmark STATIC
${CMAKE_CURRENT_SOURCE_DIR}/benchmark/${CMAKE_INSTALL_LIBDIR}/libbenchmark.a ${CMAKE_CURRENT_SOURCE_DIR}/benchmark/${CMAKE_INSTALL_LIBDIR}/libbenchmark.a
${CMAKE_CURRENT_SOURCE_DIR}/benchmark/include ${CMAKE_CURRENT_SOURCE_DIR}/benchmark/include
# Skip testing. The tests don't compile with Clang 8. # Skip testing. The tests don't compile with Clang 8.
CMAKE_ARGS -DBENCHMARK_ENABLE_TESTING=OFF) CMAKE_ARGS -DBENCHMARK_ENABLE_TESTING=OFF)
@@ -152,15 +141,15 @@ add_subdirectory(rapidcheck EXCLUDE_FROM_ALL)
# setup google test # setup google test
add_external_project(gtest SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/googletest) add_external_project(gtest SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/googletest)
set(GTEST_INCLUDE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/googletest/include set(GTEST_INCLUDE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/googletest/include
CACHE PATH "Path to gtest and gmock include directory" FORCE) CACHE PATH "Path to gtest and gmock include directory" FORCE)
set(GMOCK_LIBRARY ${CMAKE_CURRENT_SOURCE_DIR}/googletest/lib/libgmock.a set(GMOCK_LIBRARY ${CMAKE_CURRENT_SOURCE_DIR}/googletest/lib/libgmock.a
CACHE FILEPATH "Path to gmock library" FORCE) CACHE FILEPATH "Path to gmock library" FORCE)
set(GMOCK_MAIN_LIBRARY ${CMAKE_CURRENT_SOURCE_DIR}/googletest/lib/libgmock_main.a set(GMOCK_MAIN_LIBRARY ${CMAKE_CURRENT_SOURCE_DIR}/googletest/lib/libgmock_main.a
CACHE FILEPATH "Path to gmock_main library" FORCE) CACHE FILEPATH "Path to gmock_main library" FORCE)
set(GTEST_LIBRARY ${CMAKE_CURRENT_SOURCE_DIR}/googletest/lib/libgtest.a set(GTEST_LIBRARY ${CMAKE_CURRENT_SOURCE_DIR}/googletest/lib/libgtest.a
CACHE FILEPATH "Path to gtest library" FORCE) CACHE FILEPATH "Path to gtest library" FORCE)
set(GTEST_MAIN_LIBRARY ${CMAKE_CURRENT_SOURCE_DIR}/googletest/lib/libgtest_main.a set(GTEST_MAIN_LIBRARY ${CMAKE_CURRENT_SOURCE_DIR}/googletest/lib/libgtest_main.a
CACHE FILEPATH "Path to gtest_main library" FORCE) CACHE FILEPATH "Path to gtest_main library" FORCE)
mark_as_advanced(GTEST_INCLUDE_DIR GMOCK_LIBRARY GMOCK_MAIN_LIBRARY GTEST_LIBRARY GTEST_MAIN_LIBRARY) mark_as_advanced(GTEST_INCLUDE_DIR GMOCK_LIBRARY GMOCK_MAIN_LIBRARY GTEST_LIBRARY GTEST_MAIN_LIBRARY)
import_library(gtest STATIC ${GTEST_LIBRARY} ${GTEST_INCLUDE_DIR} gtest-proj) import_library(gtest STATIC ${GTEST_LIBRARY} ${GTEST_INCLUDE_DIR} gtest-proj)
import_library(gtest_main STATIC ${GTEST_MAIN_LIBRARY} ${GTEST_INCLUDE_DIR} gtest-proj) import_library(gtest_main STATIC ${GTEST_MAIN_LIBRARY} ${GTEST_INCLUDE_DIR} gtest-proj)
@@ -178,10 +167,10 @@ import_external_library(rocksdb STATIC
${CMAKE_CURRENT_SOURCE_DIR}/rocksdb/lib/librocksdb.a ${CMAKE_CURRENT_SOURCE_DIR}/rocksdb/lib/librocksdb.a
${CMAKE_CURRENT_SOURCE_DIR}/rocksdb/include ${CMAKE_CURRENT_SOURCE_DIR}/rocksdb/include
CMAKE_ARGS -DUSE_RTTI=ON CMAKE_ARGS -DUSE_RTTI=ON
-DWITH_TESTS=OFF -DWITH_TESTS=OFF
-DGFLAGS_NOTHREADS=OFF -DGFLAGS_NOTHREADS=OFF
-DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_LIBDIR=lib
-DCMAKE_SKIP_INSTALL_ALL_DEPENDENCY=true -DCMAKE_SKIP_INSTALL_ALL_DEPENDENCY=true
BUILD_COMMAND $(MAKE) rocksdb) BUILD_COMMAND $(MAKE) rocksdb)
# Setup libbcrypt # Setup libbcrypt
@@ -190,8 +179,8 @@ import_external_library(libbcrypt STATIC
${CMAKE_CURRENT_SOURCE_DIR}/libbcrypt ${CMAKE_CURRENT_SOURCE_DIR}/libbcrypt
CONFIGURE_COMMAND sed s/-Wcast-align// -i ${CMAKE_CURRENT_SOURCE_DIR}/libbcrypt/crypt_blowfish/Makefile CONFIGURE_COMMAND sed s/-Wcast-align// -i ${CMAKE_CURRENT_SOURCE_DIR}/libbcrypt/crypt_blowfish/Makefile
BUILD_COMMAND make -C ${CMAKE_CURRENT_SOURCE_DIR}/libbcrypt BUILD_COMMAND make -C ${CMAKE_CURRENT_SOURCE_DIR}/libbcrypt
CC=${CMAKE_C_COMPILER} CC=${CMAKE_C_COMPILER}
CXX=${CMAKE_CXX_COMPILER} CXX=${CMAKE_CXX_COMPILER}
INSTALL_COMMAND true) INSTALL_COMMAND true)
# Setup mgclient # Setup mgclient
@@ -199,16 +188,16 @@ import_external_library(mgclient STATIC
${CMAKE_CURRENT_SOURCE_DIR}/mgclient/lib/libmgclient.a ${CMAKE_CURRENT_SOURCE_DIR}/mgclient/lib/libmgclient.a
${CMAKE_CURRENT_SOURCE_DIR}/mgclient/include ${CMAKE_CURRENT_SOURCE_DIR}/mgclient/include
CMAKE_ARGS -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER} CMAKE_ARGS -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER} -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DBUILD_TESTING=OFF -DBUILD_TESTING=OFF
-DBUILD_CPP_BINDINGS=ON) -DBUILD_CPP_BINDINGS=ON)
find_package(OpenSSL REQUIRED) find_package(OpenSSL REQUIRED)
target_link_libraries(mgclient INTERFACE ${OPENSSL_LIBRARIES}) target_link_libraries(mgclient INTERFACE ${OPENSSL_LIBRARIES})
add_external_project(mgconsole add_external_project(mgconsole
SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/mgconsole SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/mgconsole
CMAKE_ARGS CMAKE_ARGS
-DCMAKE_INSTALL_PREFIX:PATH=${CMAKE_BINARY_DIR} -DCMAKE_INSTALL_PREFIX:PATH=${CMAKE_BINARY_DIR}
BUILD_COMMAND $(MAKE) mgconsole) BUILD_COMMAND $(MAKE) mgconsole)
add_custom_target(mgconsole DEPENDS mgconsole-proj) add_custom_target(mgconsole DEPENDS mgconsole-proj)
@@ -225,15 +214,14 @@ import_external_library(librdkafka STATIC
${CMAKE_CURRENT_SOURCE_DIR}/librdkafka/lib/librdkafka.a ${CMAKE_CURRENT_SOURCE_DIR}/librdkafka/lib/librdkafka.a
${CMAKE_CURRENT_SOURCE_DIR}/librdkafka/include ${CMAKE_CURRENT_SOURCE_DIR}/librdkafka/include
CMAKE_ARGS -DRDKAFKA_BUILD_STATIC=ON CMAKE_ARGS -DRDKAFKA_BUILD_STATIC=ON
-DRDKAFKA_BUILD_EXAMPLES=OFF -DRDKAFKA_BUILD_EXAMPLES=OFF
-DRDKAFKA_BUILD_TESTS=OFF -DRDKAFKA_BUILD_TESTS=OFF
-DWITH_ZSTD=OFF -DWITH_ZSTD=OFF
-DENABLE_LZ4_EXT=OFF -DENABLE_LZ4_EXT=OFF
-DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_LIBDIR=lib
-DWITH_SSL=ON -DWITH_SSL=ON
# If we want SASL, we need to install it on build machines
# If we want SASL, we need to install it on build machines -DWITH_SASL=OFF)
-DWITH_SASL=OFF)
target_link_libraries(librdkafka INTERFACE ${OPENSSL_LIBRARIES} ZLIB::ZLIB) target_link_libraries(librdkafka INTERFACE ${OPENSSL_LIBRARIES} ZLIB::ZLIB)
import_library(librdkafka++ STATIC import_library(librdkafka++ STATIC
@@ -254,24 +242,24 @@ import_external_library(pulsar STATIC
${CMAKE_CURRENT_SOURCE_DIR}/pulsar/install/include ${CMAKE_CURRENT_SOURCE_DIR}/pulsar/install/include
BUILD_IN_SOURCE 1 BUILD_IN_SOURCE 1
CONFIGURE_COMMAND cmake pulsar-client-cpp CONFIGURE_COMMAND cmake pulsar-client-cpp
-DCMAKE_INSTALL_PREFIX=${CMAKE_CURRENT_SOURCE_DIR}/pulsar/install -DCMAKE_INSTALL_PREFIX=${CMAKE_CURRENT_SOURCE_DIR}/pulsar/install
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER} -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER} -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
-DBUILD_DYNAMIC_LIB=OFF -DBUILD_DYNAMIC_LIB=OFF
-DBUILD_STATIC_LIB=ON -DBUILD_STATIC_LIB=ON
-DBUILD_TESTS=OFF -DBUILD_TESTS=OFF
-DLINK_STATIC=ON -DLINK_STATIC=ON
-DPROTOC_PATH=${PROTOBUF_ROOT}/bin/protoc -DPROTOC_PATH=${PROTOBUF_ROOT}/bin/protoc
-DBOOST_ROOT=${BOOST_ROOT} -DBOOST_ROOT=${BOOST_ROOT}
-DCMAKE_PREFIX_PATH=${PROTOBUF_ROOT} -DCMAKE_PREFIX_PATH=${PROTOBUF_ROOT}
-DProtobuf_INCLUDE_DIRS=${PROTOBUF_ROOT}/include -DProtobuf_INCLUDE_DIRS=${PROTOBUF_ROOT}/include
-DBUILD_PYTHON_WRAPPER=OFF -DBUILD_PYTHON_WRAPPER=OFF
-DBUILD_PERF_TOOLS=OFF -DBUILD_PERF_TOOLS=OFF
-DUSE_LOG4CXX=OFF -DUSE_LOG4CXX=OFF
BUILD_COMMAND $(MAKE) pulsarStaticWithDeps) BUILD_COMMAND $(MAKE) pulsarStaticWithDeps)
add_dependencies(pulsar-proj protobuf) add_dependencies(pulsar-proj protobuf)
if(${MG_ARCH} STREQUAL "ARM64") if (${MG_ARCH} STREQUAL "ARM64")
set(MG_LIBRDTSC_CMAKE_ARGS -DLIBRDTSC_ARCH_x86=OFF -DLIBRDTSC_ARCH_ARM64=ON) set(MG_LIBRDTSC_CMAKE_ARGS -DLIBRDTSC_ARCH_x86=OFF -DLIBRDTSC_ARCH_ARM64=ON)
endif() endif()
@@ -292,46 +280,3 @@ add_subdirectory(absl EXCLUDE_FROM_ALL)
set_path_external_library(jemalloc STATIC set_path_external_library(jemalloc STATIC
${CMAKE_CURRENT_SOURCE_DIR}/jemalloc/lib/libjemalloc.a ${CMAKE_CURRENT_SOURCE_DIR}/jemalloc/lib/libjemalloc.a
${CMAKE_CURRENT_SOURCE_DIR}/jemalloc/include/) ${CMAKE_CURRENT_SOURCE_DIR}/jemalloc/include/)
import_header_library(rangev3 ${CMAKE_CURRENT_SOURCE_DIR}/rangev3/include)
if(NOT DEFINED MGCXX_GIT_TAG)
set(MGCXX_GIT_TAG "v0.0.3" CACHE STRING "mgcxx git tag")
else()
set(MGCXX_GIT_TAG "${MGCXX_GIT_TAG}" CACHE STRING "mgcxx git tag")
endif()
ExternalProject_Add(mgcxx-proj
PREFIX mgcxx-proj
GIT_REPOSITORY https://github.com/memgraph/mgcxx
GIT_TAG "origin/regex-queries"
CMAKE_ARGS
"-DCMAKE_INSTALL_PREFIX=<INSTALL_DIR>"
"-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}"
"-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}"
"-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}"
"-DENABLE_TESTS=OFF"
INSTALL_DIR "${PROJECT_BINARY_DIR}/mgcxx"
)
ExternalProject_Get_Property(mgcxx-proj install_dir)
set(MGCXX_ROOT ${install_dir})
add_library(tantivy_text_search STATIC IMPORTED GLOBAL)
add_dependencies(tantivy_text_search mgcxx-proj)
set_property(TARGET tantivy_text_search PROPERTY IMPORTED_LOCATION ${MGCXX_ROOT}/lib/libtantivy_text_search.a)
add_library(mgcxx_text_search STATIC IMPORTED GLOBAL)
add_dependencies(mgcxx_text_search mgcxx-proj)
set_property(TARGET mgcxx_text_search PROPERTY IMPORTED_LOCATION ${MGCXX_ROOT}/lib/libmgcxx_text_search.a)
# We need to create the include directory first in order to be able to add it
# as an include directory. The header files in the include directory will be
# generated later during the build process.
file(MAKE_DIRECTORY ${MGCXX_ROOT}/include)
set_property(TARGET mgcxx_text_search PROPERTY INTERFACE_INCLUDE_DIRECTORIES ${MGCXX_ROOT}/include)
# Setup NuRaft
import_external_library(nuraft STATIC
${CMAKE_CURRENT_SOURCE_DIR}/nuraft/lib/libnuraft.a
${CMAKE_CURRENT_SOURCE_DIR}/nuraft/include/)
find_package(OpenSSL REQUIRED)
target_link_libraries(nuraft INTERFACE ${OPENSSL_LIBRARIES})

View File

@@ -123,11 +123,8 @@ declare -A primary_urls=(
["pulsar"]="http://$local_cache_host/git/pulsar.git" ["pulsar"]="http://$local_cache_host/git/pulsar.git"
["librdtsc"]="http://$local_cache_host/git/librdtsc.git" ["librdtsc"]="http://$local_cache_host/git/librdtsc.git"
["ctre"]="http://$local_cache_host/file/hanickadot/compile-time-regular-expressions/v3.7.2/single-header/ctre.hpp" ["ctre"]="http://$local_cache_host/file/hanickadot/compile-time-regular-expressions/v3.7.2/single-header/ctre.hpp"
["absl"]="http://$local_cache_host/git/abseil-cpp.git" ["absl"]="https://$local_cache_host/git/abseil-cpp.git"
["jemalloc"]="http://$local_cache_host/git/jemalloc.git" ["jemalloc"]="https://$local_cache_host/git/jemalloc.git"
["range-v3"]="http://$local_cache_host/git/range-v3.git"
["nuraft"]="http://$local_cache_host/git/NuRaft.git"
["zstd"]="http://$local_cache_host/git/zstd.git"
) )
# The goal of secondary urls is to have links to the "source of truth" of # The goal of secondary urls is to have links to the "source of truth" of
@@ -156,9 +153,6 @@ declare -A secondary_urls=(
["ctre"]="https://raw.githubusercontent.com/hanickadot/compile-time-regular-expressions/v3.7.2/single-header/ctre.hpp" ["ctre"]="https://raw.githubusercontent.com/hanickadot/compile-time-regular-expressions/v3.7.2/single-header/ctre.hpp"
["absl"]="https://github.com/abseil/abseil-cpp.git" ["absl"]="https://github.com/abseil/abseil-cpp.git"
["jemalloc"]="https://github.com/jemalloc/jemalloc.git" ["jemalloc"]="https://github.com/jemalloc/jemalloc.git"
["range-v3"]="https://github.com/ericniebler/range-v3.git"
["nuraft"]="https://github.com/eBay/NuRaft.git"
["zstd"]="https://github.com/facebook/zstd.git"
) )
# antlr # antlr
@@ -261,9 +255,10 @@ cd ..
absl_ref="20230125.3" absl_ref="20230125.3"
repo_clone_try_double "${primary_urls[absl]}" "${secondary_urls[absl]}" "absl" "$absl_ref" repo_clone_try_double "${primary_urls[absl]}" "${secondary_urls[absl]}" "absl" "$absl_ref"
# jemalloc ea6b3e973b477b8061e0076bb257dbd7f3faa756 # jemalloc ea6b3e973b477b8061e0076bb257dbd7f3faa756
JEMALLOC_COMMIT_VERSION="5.2.1" JEMALLOC_COMMIT_VERSION="5.2.1"
repo_clone_try_double "${primary_urls[jemalloc]}" "${secondary_urls[jemalloc]}" "jemalloc" "$JEMALLOC_COMMIT_VERSION" repo_clone_try_double "${secondary_urls[jemalloc]}" "${secondary_urls[jemalloc]}" "jemalloc" "$JEMALLOC_COMMIT_VERSION"
# this is hack for cmake in libs to set path, and for FindJemalloc to use Jemalloc_INCLUDE_DIR # this is hack for cmake in libs to set path, and for FindJemalloc to use Jemalloc_INCLUDE_DIR
pushd jemalloc pushd jemalloc
@@ -272,25 +267,8 @@ pushd jemalloc
MALLOC_CONF="retain:false,percpu_arena:percpu,oversize_threshold:0,muzzy_decay_ms:5000,dirty_decay_ms:5000" \ MALLOC_CONF="retain:false,percpu_arena:percpu,oversize_threshold:0,muzzy_decay_ms:5000,dirty_decay_ms:5000" \
./configure \ ./configure \
--disable-cxx \ --disable-cxx \
--with-lg-page=12 \
--with-lg-hugepage=21 \
--enable-shared=no --prefix=$working_dir \ --enable-shared=no --prefix=$working_dir \
--with-malloc-conf="retain:false,percpu_arena:percpu,oversize_threshold:0,muzzy_decay_ms:5000,dirty_decay_ms:5000" --with-malloc-conf="retain:false,percpu_arena:percpu,oversize_threshold:0,muzzy_decay_ms:5000,dirty_decay_ms:5000"
make -j$CPUS install make -j$CPUS install
popd popd
#range-v3 release-0.12.0
range_v3_ref="release-0.12.0"
repo_clone_try_double "${primary_urls[range-v3]}" "${secondary_urls[range-v3]}" "rangev3" "$range_v3_ref"
# NuRaft
nuraft_tag="v2.1.0"
repo_clone_try_double "${primary_urls[nuraft]}" "${secondary_urls[nuraft]}" "nuraft" "$nuraft_tag" true
pushd nuraft
./prepare.sh
popd
# zstd
zstd_tag="v1.5.5"
repo_clone_try_double "${primary_urls[zstd]}" "${secondary_urls[zstd]}" "zstd" "$zstd_tag" true

View File

@@ -36,7 +36,7 @@ ADDITIONAL USE GRANT: You may use the Licensed Work in accordance with the
3. using the Licensed Work to create a work or solution 3. using the Licensed Work to create a work or solution
which competes (or might reasonably be expected to which competes (or might reasonably be expected to
compete) with the Licensed Work. compete) with the Licensed Work.
CHANGE DATE: 2028-21-01 CHANGE DATE: 2027-13-09
CHANGE LICENSE: Apache License, Version 2.0 CHANGE LICENSE: Apache License, Version 2.0
For information about alternative licensing arrangements, please visit: https://memgraph.com/legal. For information about alternative licensing arrangements, please visit: https://memgraph.com/legal.

View File

@@ -13,7 +13,6 @@ string(TOLOWER ${CMAKE_BUILD_TYPE} lower_build_type)
add_library(example_c SHARED example.c) add_library(example_c SHARED example.c)
target_include_directories(example_c PRIVATE ${CMAKE_SOURCE_DIR}/include) target_include_directories(example_c PRIVATE ${CMAKE_SOURCE_DIR}/include)
target_compile_options(example_c PRIVATE -Wall) target_compile_options(example_c PRIVATE -Wall)
target_link_libraries(example_c PRIVATE -static-libgcc -static-libstdc++)
# Strip C example in release build. # Strip C example in release build.
if (lower_build_type STREQUAL "release") if (lower_build_type STREQUAL "release")
add_custom_command(TARGET example_c POST_BUILD add_custom_command(TARGET example_c POST_BUILD
@@ -29,7 +28,6 @@ install(FILES example.c DESTINATION lib/memgraph/query_modules/src)
add_library(example_cpp SHARED example.cpp) add_library(example_cpp SHARED example.cpp)
target_include_directories(example_cpp PRIVATE ${CMAKE_SOURCE_DIR}/include) target_include_directories(example_cpp PRIVATE ${CMAKE_SOURCE_DIR}/include)
target_compile_options(example_cpp PRIVATE -Wall) target_compile_options(example_cpp PRIVATE -Wall)
target_link_libraries(example_cpp PRIVATE -static-libgcc -static-libstdc++)
# Strip C++ example in release build. # Strip C++ example in release build.
if (lower_build_type STREQUAL "release") if (lower_build_type STREQUAL "release")
add_custom_command(TARGET example_cpp POST_BUILD add_custom_command(TARGET example_cpp POST_BUILD
@@ -42,43 +40,9 @@ install(PROGRAMS $<TARGET_FILE:example_cpp>
# Also install the source of the example, so user can read it. # Also install the source of the example, so user can read it.
install(FILES example.cpp DESTINATION lib/memgraph/query_modules/src) install(FILES example.cpp DESTINATION lib/memgraph/query_modules/src)
add_library(schema SHARED schema.cpp)
target_include_directories(schema PRIVATE ${CMAKE_SOURCE_DIR}/include)
target_compile_options(schema PRIVATE -Wall)
target_link_libraries(schema PRIVATE -static-libgcc -static-libstdc++)
# Strip C++ example in release build.
if (lower_build_type STREQUAL "release")
add_custom_command(TARGET schema POST_BUILD
COMMAND strip -s $<TARGET_FILE:schema>
COMMENT "Stripping symbols and sections from the C++ schema module")
endif()
install(PROGRAMS $<TARGET_FILE:schema>
DESTINATION lib/memgraph/query_modules
RENAME schema.so)
# Also install the source of the example, so user can read it.
install(FILES schema.cpp DESTINATION lib/memgraph/query_modules/src)
add_library(text SHARED text_search_module.cpp)
target_include_directories(text PRIVATE ${CMAKE_SOURCE_DIR}/include)
target_compile_options(text PRIVATE -Wall)
target_link_libraries(text PRIVATE -static-libgcc -static-libstdc++)
# Strip C++ example in release build.
if (lower_build_type STREQUAL "release")
add_custom_command(TARGET text POST_BUILD
COMMAND strip -s $<TARGET_FILE:text>
COMMENT "Stripping symbols and sections from the C++ text_search module")
endif()
install(PROGRAMS $<TARGET_FILE:text>
DESTINATION lib/memgraph/query_modules
RENAME text.so)
# Also install the source of the example, so user can read it.
install(FILES text_search_module.cpp DESTINATION lib/memgraph/query_modules/src)
# Install the Python example and modules # Install the Python example and modules
install(FILES example.py DESTINATION lib/memgraph/query_modules RENAME py_example.py) install(FILES example.py DESTINATION lib/memgraph/query_modules RENAME py_example.py)
install(FILES graph_analyzer.py DESTINATION lib/memgraph/query_modules) install(FILES graph_analyzer.py DESTINATION lib/memgraph/query_modules)
install(FILES mgp_networkx.py DESTINATION lib/memgraph/query_modules) install(FILES mgp_networkx.py DESTINATION lib/memgraph/query_modules)
install(FILES nxalg.py DESTINATION lib/memgraph/query_modules) install(FILES nxalg.py DESTINATION lib/memgraph/query_modules)
install(FILES wcc.py DESTINATION lib/memgraph/query_modules) install(FILES wcc.py DESTINATION lib/memgraph/query_modules)
install(FILES mgps.py DESTINATION lib/memgraph/query_modules)
install(FILES convert.py DESTINATION lib/memgraph/query_modules)

View File

@@ -1,10 +0,0 @@
from json import loads
import mgp
@mgp.function
def str2object(string: str) -> mgp.Any:
if string:
return loads(string)
return None

View File

@@ -1,8 +0,0 @@
import mgp
@mgp.read_proc
def components(
context: mgp.ProcCtx,
) -> mgp.Record(versions=list, edition=str, name=str):
return mgp.Record(versions=["5.9.0"], edition="community", name="Memgraph")

View File

@@ -1,678 +0,0 @@
// Copyright 2023 Memgraph Ltd.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
// License, and you may not use this file except in compliance with the Business Source License.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#include <mgp.hpp>
#include "utils/string.hpp"
#include <optional>
namespace Schema {
constexpr std::string_view kStatusKept = "Kept";
constexpr std::string_view kStatusCreated = "Created";
constexpr std::string_view kStatusDropped = "Dropped";
constexpr std::string_view kReturnNodeType = "nodeType";
constexpr std::string_view kProcedureNodeType = "node_type_properties";
constexpr std::string_view kProcedureRelType = "rel_type_properties";
constexpr std::string_view kProcedureAssert = "assert";
constexpr std::string_view kReturnLabels = "nodeLabels";
constexpr std::string_view kReturnRelType = "relType";
constexpr std::string_view kReturnPropertyName = "propertyName";
constexpr std::string_view kReturnPropertyType = "propertyTypes";
constexpr std::string_view kReturnMandatory = "mandatory";
constexpr std::string_view kReturnLabel = "label";
constexpr std::string_view kReturnKey = "key";
constexpr std::string_view kReturnKeys = "keys";
constexpr std::string_view kReturnUnique = "unique";
constexpr std::string_view kReturnAction = "action";
constexpr std::string_view kParameterIndices = "indices";
constexpr std::string_view kParameterUniqueConstraints = "unique_constraints";
constexpr std::string_view kParameterExistenceConstraints = "existence_constraints";
constexpr std::string_view kParameterDropExisting = "drop_existing";
std::string TypeOf(const mgp::Type &type);
template <typename T>
void ProcessPropertiesNode(mgp::Record &record, const std::string &type, const mgp::List &labels,
const std::string &propertyName, const T &propertyType, const bool &mandatory);
template <typename T>
void ProcessPropertiesRel(mgp::Record &record, const std::string_view &type, const std::string &propertyName,
const T &propertyType, const bool &mandatory);
void NodeTypeProperties(mgp_list *args, mgp_graph *memgraph_graph, mgp_result *result, mgp_memory *memory);
void RelTypeProperties(mgp_list *args, mgp_graph *memgraph_graph, mgp_result *result, mgp_memory *memory);
void Assert(mgp_list *args, mgp_graph *memgraph_graph, mgp_result *result, mgp_memory *memory);
} // namespace Schema
/*we have << operator for type in Cpp API, but in it we return somewhat different strings than I would like in this
module, so I implemented a small function here*/
std::string Schema::TypeOf(const mgp::Type &type) {
switch (type) {
case mgp::Type::Null:
return "Null";
case mgp::Type::Bool:
return "Bool";
case mgp::Type::Int:
return "Int";
case mgp::Type::Double:
return "Double";
case mgp::Type::String:
return "String";
case mgp::Type::List:
return "List[Any]";
case mgp::Type::Map:
return "Map[Any]";
case mgp::Type::Node:
return "Vertex";
case mgp::Type::Relationship:
return "Edge";
case mgp::Type::Path:
return "Path";
case mgp::Type::Date:
return "Date";
case mgp::Type::LocalTime:
return "LocalTime";
case mgp::Type::LocalDateTime:
return "LocalDateTime";
case mgp::Type::Duration:
return "Duration";
default:
throw mgp::ValueException("Unsupported type");
}
}
template <typename T>
void Schema::ProcessPropertiesNode(mgp::Record &record, const std::string &type, const mgp::List &labels,
const std::string &propertyName, const T &propertyType, const bool &mandatory) {
record.Insert(std::string(kReturnNodeType).c_str(), type);
record.Insert(std::string(kReturnLabels).c_str(), labels);
record.Insert(std::string(kReturnPropertyName).c_str(), propertyName);
record.Insert(std::string(kReturnPropertyType).c_str(), propertyType);
record.Insert(std::string(kReturnMandatory).c_str(), mandatory);
}
template <typename T>
void Schema::ProcessPropertiesRel(mgp::Record &record, const std::string_view &type, const std::string &propertyName,
const T &propertyType, const bool &mandatory) {
record.Insert(std::string(kReturnRelType).c_str(), type);
record.Insert(std::string(kReturnPropertyName).c_str(), propertyName);
record.Insert(std::string(kReturnPropertyType).c_str(), propertyType);
record.Insert(std::string(kReturnMandatory).c_str(), mandatory);
}
struct Property {
std::string name;
mgp::Value value;
Property(const std::string &name, mgp::Value &&value) : name(name), value(std::move(value)) {}
};
struct LabelsHash {
std::size_t operator()(const std::set<std::string> &set) const {
std::size_t seed = set.size();
for (const auto &i : set) {
seed ^= std::hash<std::string>{}(i) + 0x9e3779b9 + (seed << 6) + (seed >> 2);
}
return seed;
}
};
struct LabelsComparator {
bool operator()(const std::set<std::string> &lhs, const std::set<std::string> &rhs) const { return lhs == rhs; }
};
struct PropertyComparator {
bool operator()(const Property &lhs, const Property &rhs) const { return lhs.name < rhs.name; }
};
struct PropertyInfo {
std::set<Property, PropertyComparator> properties;
bool mandatory;
};
void Schema::NodeTypeProperties(mgp_list * /*args*/, mgp_graph *memgraph_graph, mgp_result *result,
mgp_memory *memory) {
mgp::MemoryDispatcherGuard guard{memory};
const auto record_factory = mgp::RecordFactory(result);
try {
std::unordered_map<std::set<std::string>, PropertyInfo, LabelsHash, LabelsComparator> node_types_properties;
for (auto node : mgp::Graph(memgraph_graph).Nodes()) {
std::set<std::string> labels_set = {};
for (auto label : node.Labels()) {
labels_set.emplace(label);
}
if (node_types_properties.find(labels_set) == node_types_properties.end()) {
node_types_properties[labels_set] = PropertyInfo{std::set<Property, PropertyComparator>(), true};
}
if (node.Properties().empty()) {
node_types_properties[labels_set].mandatory = false; // if there is node with no property, it is not mandatory
continue;
}
auto &property_info = node_types_properties.at(labels_set);
for (auto &[key, prop] : node.Properties()) {
property_info.properties.emplace(key, std::move(prop));
if (property_info.mandatory) {
property_info.mandatory =
property_info.properties.size() == 1; // if there is only one property, it is mandatory
}
}
}
for (auto &[labels, property_info] : node_types_properties) {
std::string label_type;
mgp::List labels_list = mgp::List();
for (auto const &label : labels) {
label_type += ":`" + std::string(label) + "`";
labels_list.AppendExtend(mgp::Value(label));
}
for (auto const &prop : property_info.properties) {
auto record = record_factory.NewRecord();
ProcessPropertiesNode(record, label_type, labels_list, prop.name, TypeOf(prop.value.Type()),
property_info.mandatory);
}
if (property_info.properties.empty()) {
auto record = record_factory.NewRecord();
ProcessPropertiesNode<std::string>(record, label_type, labels_list, "", "", false);
}
}
} catch (const std::exception &e) {
record_factory.SetErrorMessage(e.what());
return;
}
}
void Schema::RelTypeProperties(mgp_list * /*args*/, mgp_graph *memgraph_graph, mgp_result *result, mgp_memory *memory) {
mgp::MemoryDispatcherGuard guard{memory};
std::unordered_map<std::string, PropertyInfo> rel_types_properties;
const auto record_factory = mgp::RecordFactory(result);
try {
const mgp::Graph graph = mgp::Graph(memgraph_graph);
for (auto rel : graph.Relationships()) {
std::string rel_type = std::string(rel.Type());
if (rel_types_properties.find(rel_type) == rel_types_properties.end()) {
rel_types_properties[rel_type] = PropertyInfo{std::set<Property, PropertyComparator>(), true};
}
if (rel.Properties().empty()) {
rel_types_properties[rel_type].mandatory = false; // if there is rel with no property, it is not mandatory
continue;
}
auto &property_info = rel_types_properties.at(rel_type);
for (auto &[key, prop] : rel.Properties()) {
property_info.properties.emplace(key, std::move(prop));
if (property_info.mandatory) {
property_info.mandatory =
property_info.properties.size() == 1; // if there is only one property, it is mandatory
}
}
}
for (auto &[type, property_info] : rel_types_properties) {
std::string type_str = ":`" + std::string(type) + "`";
for (auto const &prop : property_info.properties) {
auto record = record_factory.NewRecord();
ProcessPropertiesRel(record, type_str, prop.name, TypeOf(prop.value.Type()), property_info.mandatory);
}
if (property_info.properties.empty()) {
auto record = record_factory.NewRecord();
ProcessPropertiesRel<std::string>(record, type_str, "", "", false);
}
}
} catch (const std::exception &e) {
record_factory.SetErrorMessage(e.what());
return;
}
}
void InsertRecordForLabelIndex(const auto &record_factory, const std::string_view label,
const std::string_view status) {
auto record = record_factory.NewRecord();
record.Insert(std::string(Schema::kReturnLabel).c_str(), label);
record.Insert(std::string(Schema::kReturnKey).c_str(), "");
record.Insert(std::string(Schema::kReturnKeys).c_str(), mgp::List());
record.Insert(std::string(Schema::kReturnUnique).c_str(), false);
record.Insert(std::string(Schema::kReturnAction).c_str(), status);
}
void InsertRecordForUniqueConstraint(const auto &record_factory, const std::string_view label,
const mgp::List &properties, const std::string_view status) {
auto record = record_factory.NewRecord();
record.Insert(std::string(Schema::kReturnLabel).c_str(), label);
record.Insert(std::string(Schema::kReturnKey).c_str(), properties.ToString());
record.Insert(std::string(Schema::kReturnKeys).c_str(), properties);
record.Insert(std::string(Schema::kReturnUnique).c_str(), true);
record.Insert(std::string(Schema::kReturnAction).c_str(), status);
}
void InsertRecordForLabelPropertyIndexAndExistenceConstraint(const auto &record_factory, const std::string_view label,
const std::string_view property,
const std::string_view status) {
auto record = record_factory.NewRecord();
record.Insert(std::string(Schema::kReturnLabel).c_str(), label);
record.Insert(std::string(Schema::kReturnKey).c_str(), property);
record.Insert(std::string(Schema::kReturnKeys).c_str(), mgp::List({mgp::Value(property)}));
record.Insert(std::string(Schema::kReturnUnique).c_str(), false);
record.Insert(std::string(Schema::kReturnAction).c_str(), status);
}
void ProcessCreatingLabelIndex(const std::string_view label, const std::set<std::string_view> &existing_label_indices,
mgp_graph *memgraph_graph, const auto &record_factory) {
if (existing_label_indices.contains(label)) {
InsertRecordForLabelIndex(record_factory, label, Schema::kStatusKept);
} else if (mgp::CreateLabelIndex(memgraph_graph, label)) {
InsertRecordForLabelIndex(record_factory, label, Schema::kStatusCreated);
}
}
template <typename TFunc>
void ProcessCreatingLabelPropertyIndexAndExistenceConstraint(const std::string_view label,
const std::string_view property,
const std::set<std::string_view> &existing_collection,
const TFunc &func_creation, mgp_graph *memgraph_graph,
const auto &record_factory) {
const auto label_property_search_key = std::string(label) + ":" + std::string(property);
if (existing_collection.contains(label_property_search_key)) {
InsertRecordForLabelPropertyIndexAndExistenceConstraint(record_factory, label, property, Schema::kStatusKept);
} else if (func_creation(memgraph_graph, label, property)) {
InsertRecordForLabelPropertyIndexAndExistenceConstraint(record_factory, label, property, Schema::kStatusCreated);
}
}
/// We collect properties for which index was created.
using AssertedIndices = std::set<std::string, std::less<>>;
AssertedIndices CreateIndicesForLabel(const std::string_view label, const mgp::Value &properties_val,
mgp_graph *memgraph_graph, const auto &record_factory,
const std::set<std::string_view> &existing_label_indices,
const std::set<std::string_view> &existing_label_property_indices) {
AssertedIndices asserted_indices;
if (!properties_val.IsList()) {
return {};
}
if (const auto properties = properties_val.ValueList();
properties.Empty() && mgp::CreateLabelIndex(memgraph_graph, label)) {
InsertRecordForLabelIndex(record_factory, label, Schema::kStatusCreated);
asserted_indices.emplace("");
} else {
std::for_each(properties.begin(), properties.end(),
[&label, &existing_label_indices, &existing_label_property_indices, &memgraph_graph, &record_factory,
&asserted_indices](const mgp::Value &property) {
if (!property.IsString()) {
return;
}
const auto property_str = property.ValueString();
if (property_str.empty()) {
ProcessCreatingLabelIndex(label, existing_label_indices, memgraph_graph, record_factory);
asserted_indices.emplace("");
} else {
ProcessCreatingLabelPropertyIndexAndExistenceConstraint(
label, property_str, existing_label_property_indices, mgp::CreateLabelPropertyIndex,
memgraph_graph, record_factory);
asserted_indices.emplace(property_str);
}
});
}
return asserted_indices;
}
void ProcessIndices(const mgp::Map &indices_map, mgp_graph *memgraph_graph, const auto &record_factory,
bool drop_existing) {
auto mgp_existing_label_indices = mgp::ListAllLabelIndices(memgraph_graph);
auto mgp_existing_label_property_indices = mgp::ListAllLabelPropertyIndices(memgraph_graph);
std::set<std::string_view> existing_label_indices;
std::transform(mgp_existing_label_indices.begin(), mgp_existing_label_indices.end(),
std::inserter(existing_label_indices, existing_label_indices.begin()),
[](const mgp::Value &index) { return index.ValueString(); });
std::set<std::string_view> existing_label_property_indices;
std::transform(mgp_existing_label_property_indices.begin(), mgp_existing_label_property_indices.end(),
std::inserter(existing_label_property_indices, existing_label_property_indices.begin()),
[](const mgp::Value &index) { return index.ValueString(); });
std::set<std::string> asserted_label_indices;
std::set<std::string> asserted_label_property_indices;
auto merge_label_property = [](const std::string &label, const std::string &property) {
return label + ":" + property;
};
for (const auto &index : indices_map) {
const std::string_view label = index.key;
const mgp::Value &properties_val = index.value;
AssertedIndices asserted_indices_new = CreateIndicesForLabel(
label, properties_val, memgraph_graph, record_factory, existing_label_indices, existing_label_property_indices);
if (!drop_existing) {
continue;
}
std::ranges::for_each(asserted_indices_new, [&asserted_label_indices, &asserted_label_property_indices, label,
&merge_label_property](const std::string &property) {
if (property.empty()) {
asserted_label_indices.emplace(label);
} else {
asserted_label_property_indices.emplace(merge_label_property(std::string(label), property));
}
});
}
if (!drop_existing) {
return;
}
std::set<std::string_view> label_indices_to_drop;
std::ranges::set_difference(existing_label_indices, asserted_label_indices,
std::inserter(label_indices_to_drop, label_indices_to_drop.begin()));
std::ranges::for_each(label_indices_to_drop, [memgraph_graph, &record_factory](const std::string_view label) {
if (mgp::DropLabelIndex(memgraph_graph, label)) {
InsertRecordForLabelIndex(record_factory, label, Schema::kStatusDropped);
}
});
std::set<std::string_view> label_property_indices_to_drop;
std::ranges::set_difference(existing_label_property_indices, asserted_label_property_indices,
std::inserter(label_property_indices_to_drop, label_property_indices_to_drop.begin()));
auto decouple_label_property = [](const std::string_view label_property) {
const auto label_size = label_property.find(':');
const auto label = std::string(label_property.substr(0, label_size));
const auto property = std::string(label_property.substr(label_size + 1));
return std::make_pair(label, property);
};
std::ranges::for_each(label_property_indices_to_drop, [memgraph_graph, &record_factory, decouple_label_property](
const std::string_view label_property) {
const auto [label, property] = decouple_label_property(label_property);
if (mgp::DropLabelPropertyIndex(memgraph_graph, label, property)) {
InsertRecordForLabelPropertyIndexAndExistenceConstraint(record_factory, label, property, Schema::kStatusDropped);
}
});
}
using ExistenceConstraintsStorage = std::set<std::string_view>;
ExistenceConstraintsStorage CreateExistenceConstraintsForLabel(
const std::string_view label, const mgp::Value &properties_val, mgp_graph *memgraph_graph,
const auto &record_factory, const std::set<std::string_view> &existing_existence_constraints) {
ExistenceConstraintsStorage asserted_existence_constraints;
if (!properties_val.IsList()) {
return asserted_existence_constraints;
}
auto validate_property = [](const mgp::Value &property) -> bool {
return property.IsString() && !property.ValueString().empty();
};
const auto &properties = properties_val.ValueList();
std::for_each(properties.begin(), properties.end(),
[&label, &existing_existence_constraints, &asserted_existence_constraints, &memgraph_graph,
&record_factory, &validate_property](const mgp::Value &property) {
if (!validate_property(property)) {
return;
}
const std::string_view property_str = property.ValueString();
asserted_existence_constraints.emplace(property_str);
ProcessCreatingLabelPropertyIndexAndExistenceConstraint(
label, property_str, existing_existence_constraints, mgp::CreateExistenceConstraint,
memgraph_graph, record_factory);
});
return asserted_existence_constraints;
}
void ProcessExistenceConstraints(const mgp::Map &existence_constraints_map, mgp_graph *memgraph_graph,
const auto &record_factory, bool drop_existing) {
auto mgp_existing_existence_constraints = mgp::ListAllExistenceConstraints(memgraph_graph);
std::set<std::string_view> existing_existence_constraints;
std::transform(mgp_existing_existence_constraints.begin(), mgp_existing_existence_constraints.end(),
std::inserter(existing_existence_constraints, existing_existence_constraints.begin()),
[](const mgp::Value &constraint) { return constraint.ValueString(); });
auto merge_label_property = [](const std::string_view label, const std::string_view property) {
auto str = std::string(label) + ":";
str += property;
return str;
};
ExistenceConstraintsStorage asserted_existence_constraints;
for (const auto &existing_constraint : existence_constraints_map) {
const std::string_view label = existing_constraint.key;
const mgp::Value &properties_val = existing_constraint.value;
auto asserted_existence_constraints_new = CreateExistenceConstraintsForLabel(
label, properties_val, memgraph_graph, record_factory, existing_existence_constraints);
if (!drop_existing) {
continue;
}
std::ranges::for_each(asserted_existence_constraints_new, [&asserted_existence_constraints, &merge_label_property,
label](const std::string_view property) {
asserted_existence_constraints.emplace(merge_label_property(label, property));
});
}
if (!drop_existing) {
return;
}
std::set<std::string_view> existence_constraints_to_drop;
std::ranges::set_difference(existing_existence_constraints, asserted_existence_constraints,
std::inserter(existence_constraints_to_drop, existence_constraints_to_drop.begin()));
auto decouple_label_property = [](const std::string_view label_property) {
const auto label_size = label_property.find(':');
const auto label = std::string(label_property.substr(0, label_size));
const auto property = std::string(label_property.substr(label_size + 1));
return std::make_pair(label, property);
};
std::ranges::for_each(existence_constraints_to_drop, [&](const std::string_view label_property) {
const auto [label, property] = decouple_label_property(label_property);
if (mgp::DropExistenceConstraint(memgraph_graph, label, property)) {
InsertRecordForLabelPropertyIndexAndExistenceConstraint(record_factory, label, property, Schema::kStatusDropped);
}
});
}
using AssertedUniqueConstraintsStorage = std::set<std::set<std::string_view>>;
AssertedUniqueConstraintsStorage CreateUniqueConstraintsForLabel(
const std::string_view label, const mgp::Value &unique_props_nested,
const std::map<std::string_view, AssertedUniqueConstraintsStorage> &existing_unique_constraints,
mgp_graph *memgraph_graph, const auto &record_factory) {
AssertedUniqueConstraintsStorage asserted_unique_constraints;
if (!unique_props_nested.IsList()) {
return asserted_unique_constraints;
}
auto validate_unique_constraint_props = [](const mgp::Value &properties) -> bool {
if (!properties.IsList()) {
return false;
}
const auto &properties_list = properties.ValueList();
if (properties_list.Empty()) {
return false;
}
return std::all_of(properties_list.begin(), properties_list.end(), [](const mgp::Value &property) {
return property.IsString() && !property.ValueString().empty();
});
};
auto unique_constraint_exists =
[](const std::string_view label, const std::set<std::string_view> &properties,
const std::map<std::string_view, AssertedUniqueConstraintsStorage> &existing_unique_constraints) -> bool {
auto iter = existing_unique_constraints.find(label);
if (iter == existing_unique_constraints.end()) {
return false;
}
return iter->second.find(properties) != iter->second.end();
};
for (const auto unique_props_nested_list = unique_props_nested.ValueList();
const auto &properties : unique_props_nested_list) {
if (!validate_unique_constraint_props(properties)) {
continue;
}
const auto properties_list = properties.ValueList();
std::set<std::string_view> properties_coll;
std::transform(properties_list.begin(), properties_list.end(),
std::inserter(properties_coll, properties_coll.begin()),
[](const mgp::Value &property) { return property.ValueString(); });
if (unique_constraint_exists(label, properties_coll, existing_unique_constraints)) {
InsertRecordForUniqueConstraint(record_factory, label, properties_list, Schema::kStatusKept);
} else if (mgp::CreateUniqueConstraint(memgraph_graph, label, properties.ptr())) {
InsertRecordForUniqueConstraint(record_factory, label, properties_list, Schema::kStatusCreated);
}
asserted_unique_constraints.emplace(std::move(properties_coll));
}
return asserted_unique_constraints;
}
void ProcessUniqueConstraints(const mgp::Map &unique_constraints_map, mgp_graph *memgraph_graph,
const auto &record_factory, bool drop_existing) {
auto mgp_existing_unique_constraints = mgp::ListAllUniqueConstraints(memgraph_graph);
// label-unique_constraints pair
std::map<std::string_view, AssertedUniqueConstraintsStorage> existing_unique_constraints;
for (const auto &constraint : mgp_existing_unique_constraints) {
auto constraint_list = constraint.ValueList();
std::set<std::string_view> properties;
for (int i = 1; i < constraint_list.Size(); i++) {
properties.emplace(constraint_list[i].ValueString());
}
const std::string_view label = constraint_list[0].ValueString();
auto [it, inserted] = existing_unique_constraints.try_emplace(label, AssertedUniqueConstraintsStorage{properties});
if (!inserted) {
it->second.emplace(std::move(properties));
}
}
std::map<std::string_view, AssertedUniqueConstraintsStorage> asserted_unique_constraints;
for (const auto &[label, unique_props_nested] : unique_constraints_map) {
auto asserted_unique_constraints_new = CreateUniqueConstraintsForLabel(
label, unique_props_nested, existing_unique_constraints, memgraph_graph, record_factory);
if (drop_existing) {
asserted_unique_constraints.emplace(label, std::move(asserted_unique_constraints_new));
}
}
if (!drop_existing) {
return;
}
std::vector<std::pair<std::string_view, std::set<std::string_view>>> unique_constraints_to_drop;
// Check for each label for we found existing constraint in the DB whether it was asserted.
// If no unique constraint was found with label, we can drop all unique constraints for this label. (if branch)
// If some unique constraint was found with label, we can drop only those unique constraints that were not asserted.
// (else branch.)
std::ranges::for_each(existing_unique_constraints, [&asserted_unique_constraints, &unique_constraints_to_drop](
const auto &existing_label_unique_constraints) {
const auto &label = existing_label_unique_constraints.first;
const auto &existing_unique_constraints_for_label = existing_label_unique_constraints.second;
const auto &asserted_unique_constraints_for_label = asserted_unique_constraints.find(label);
if (asserted_unique_constraints_for_label == asserted_unique_constraints.end()) {
std::ranges::for_each(
std::make_move_iterator(existing_unique_constraints_for_label.begin()),
std::make_move_iterator(existing_unique_constraints_for_label.end()),
[&unique_constraints_to_drop, &label](std::set<std::string_view> existing_unique_constraint_for_label) {
unique_constraints_to_drop.emplace_back(label, std::move(existing_unique_constraint_for_label));
});
} else {
const auto &asserted_unique_constraints_for_label_coll = asserted_unique_constraints_for_label->second;
std::ranges::for_each(
std::make_move_iterator(existing_unique_constraints_for_label.begin()),
std::make_move_iterator(existing_unique_constraints_for_label.end()),
[&unique_constraints_to_drop, &label, &asserted_unique_constraints_for_label_coll](
std::set<std::string_view> existing_unique_constraint_for_label) {
if (!asserted_unique_constraints_for_label_coll.contains(existing_unique_constraint_for_label)) {
unique_constraints_to_drop.emplace_back(label, std::move(existing_unique_constraint_for_label));
}
});
}
});
std::ranges::for_each(
unique_constraints_to_drop, [memgraph_graph, &record_factory](const auto &label_unique_constraint) {
const auto &[label, unique_constraint] = label_unique_constraint;
auto unique_constraint_list = mgp::List();
std::ranges::for_each(unique_constraint, [&unique_constraint_list](const std::string_view &property) {
unique_constraint_list.AppendExtend(mgp::Value(property));
});
if (mgp::DropUniqueConstraint(memgraph_graph, label, mgp::Value(unique_constraint_list).ptr())) {
InsertRecordForUniqueConstraint(record_factory, label, unique_constraint_list, Schema::kStatusDropped);
}
});
}
void Schema::Assert(mgp_list *args, mgp_graph *memgraph_graph, mgp_result *result, mgp_memory *memory) {
mgp::MemoryDispatcherGuard guard{memory};
const auto record_factory = mgp::RecordFactory(result);
auto arguments = mgp::List(args);
auto indices_map = arguments[0].ValueMap();
auto unique_constraints_map = arguments[1].ValueMap();
auto existence_constraints_map = arguments[2].ValueMap();
auto drop_existing = arguments[3].ValueBool();
ProcessIndices(indices_map, memgraph_graph, record_factory, drop_existing);
ProcessExistenceConstraints(existence_constraints_map, memgraph_graph, record_factory, drop_existing);
ProcessUniqueConstraints(unique_constraints_map, memgraph_graph, record_factory, drop_existing);
}
extern "C" int mgp_init_module(struct mgp_module *module, struct mgp_memory *memory) {
try {
mgp::MemoryDispatcherGuard guard{memory};
AddProcedure(Schema::NodeTypeProperties, Schema::kProcedureNodeType, mgp::ProcedureType::Read, {},
{mgp::Return(Schema::kReturnNodeType, mgp::Type::String),
mgp::Return(Schema::kReturnLabels, {mgp::Type::List, mgp::Type::String}),
mgp::Return(Schema::kReturnPropertyName, mgp::Type::String),
mgp::Return(Schema::kReturnPropertyType, mgp::Type::Any),
mgp::Return(Schema::kReturnMandatory, mgp::Type::Bool)},
module, memory);
AddProcedure(Schema::RelTypeProperties, Schema::kProcedureRelType, mgp::ProcedureType::Read, {},
{mgp::Return(Schema::kReturnRelType, mgp::Type::String),
mgp::Return(Schema::kReturnPropertyName, mgp::Type::String),
mgp::Return(Schema::kReturnPropertyType, mgp::Type::Any),
mgp::Return(Schema::kReturnMandatory, mgp::Type::Bool)},
module, memory);
AddProcedure(
Schema::Assert, Schema::kProcedureAssert, mgp::ProcedureType::Read,
{
mgp::Parameter(Schema::kParameterIndices, {mgp::Type::Map, mgp::Type::Any}),
mgp::Parameter(Schema::kParameterUniqueConstraints, {mgp::Type::Map, mgp::Type::Any}),
mgp::Parameter(Schema::kParameterExistenceConstraints, {mgp::Type::Map, mgp::Type::Any},
mgp::Value(mgp::Map{})),
mgp::Parameter(Schema::kParameterDropExisting, mgp::Type::Bool, mgp::Value(true)),
},
{mgp::Return(Schema::kReturnLabel, mgp::Type::String), mgp::Return(Schema::kReturnKey, mgp::Type::String),
mgp::Return(Schema::kReturnKeys, {mgp::Type::List, mgp::Type::String}),
mgp::Return(Schema::kReturnUnique, mgp::Type::Bool), mgp::Return(Schema::kReturnAction, mgp::Type::String)},
module, memory);
} catch (const std::exception &e) {
std::cerr << "Error while initializing query module: " << e.what() << std::endl;
return 1;
}
return 0;
}
extern "C" int mgp_shutdown_module() { return 0; }

View File

@@ -1,105 +0,0 @@
// Copyright 2024 Memgraph Ltd.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
// License, and you may not use this file except in compliance with the Business Source License.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#include <string>
#include <string_view>
#include <fmt/format.h>
#include <mgp.hpp>
namespace TextSearch {
constexpr std::string_view kProcedureSearch = "search";
constexpr std::string_view kProcedureRegexSearch = "regex_search";
constexpr std::string_view kParameterIndexName = "index_name";
constexpr std::string_view kParameterSearchString = "search_query";
constexpr std::string_view kReturnNode = "node";
void Search(mgp_list *args, mgp_graph *memgraph_graph, mgp_result *result, mgp_memory *memory);
void RegexSearch(mgp_list *args, mgp_graph *memgraph_graph, mgp_result *result, mgp_memory *memory);
} // namespace TextSearch
void TextSearch::Search(mgp_list *args, mgp_graph *memgraph_graph, mgp_result *result, mgp_memory *memory) {
mgp::MemoryDispatcherGuard guard{memory};
const auto record_factory = mgp::RecordFactory(result);
auto arguments = mgp::List(args);
try {
const auto *index_name = arguments[0].ValueString().data();
const auto *search_query = arguments[1].ValueString().data();
// 1. See if the given index_name is text-indexed
if (!mgp::graph_has_text_index(memgraph_graph, index_name)) {
record_factory.SetErrorMessage(fmt::format("Text index \"{}\" doesnt exist.", index_name));
return;
}
// 2. Run a text search of that index and return the search results
for (const auto &node : mgp::RunTextSearchQuery(memgraph_graph, index_name, search_query)) {
auto record = record_factory.NewRecord();
record.Insert(TextSearch::kReturnNode.data(), node.ValueNode());
}
} catch (const std::exception &e) {
record_factory.SetErrorMessage(e.what());
}
}
void TextSearch::RegexSearch(mgp_list *args, mgp_graph *memgraph_graph, mgp_result *result, mgp_memory *memory) {
mgp::MemoryDispatcherGuard guard{memory};
const auto record_factory = mgp::RecordFactory(result);
auto arguments = mgp::List(args);
try {
const auto *index_name = arguments[0].ValueString().data();
const auto *search_query = arguments[1].ValueString().data();
// 1. See if the given index_name is text-indexed
if (!mgp::graph_has_text_index(memgraph_graph, index_name)) {
record_factory.SetErrorMessage(fmt::format("Text index \"{}\" doesnt exist.", index_name));
return;
}
// 2. Run a text search of that index and return the search results
for (const auto &node : mgp::RunTextRegexSearchQuery(memgraph_graph, index_name, search_query)) {
auto record = record_factory.NewRecord();
record.Insert(TextSearch::kReturnNode.data(), node.ValueNode());
}
} catch (const std::exception &e) {
record_factory.SetErrorMessage(e.what());
}
}
extern "C" int mgp_init_module(struct mgp_module *module, struct mgp_memory *memory) {
try {
mgp::MemoryDispatcherGuard guard{memory};
AddProcedure(TextSearch::Search, TextSearch::kProcedureSearch, mgp::ProcedureType::Read,
{
mgp::Parameter(TextSearch::kParameterIndexName, mgp::Type::String),
mgp::Parameter(TextSearch::kParameterSearchString, mgp::Type::String),
},
{mgp::Return(TextSearch::kReturnNode, mgp::Type::Node)}, module, memory);
AddProcedure(TextSearch::RegexSearch, TextSearch::kProcedureRegexSearch, mgp::ProcedureType::Read,
{
mgp::Parameter(TextSearch::kParameterIndexName, mgp::Type::String),
mgp::Parameter(TextSearch::kParameterSearchString, mgp::Type::String),
},
{mgp::Return(TextSearch::kReturnNode, mgp::Type::Node)}, module, memory);
} catch (const std::exception &e) {
std::cerr << "Error while initializing query module: " << e.what() << std::endl;
return 1;
}
return 0;
}
extern "C" int mgp_shutdown_module() { return 0; }

View File

@@ -104,9 +104,7 @@ def retry(retry_limit, timeout=100):
except Exception: except Exception:
time.sleep(timeout) time.sleep(timeout)
return func(*args, **kwargs) return func(*args, **kwargs)
return wrapper return wrapper
return inner_func return inner_func
@@ -202,19 +200,19 @@ if args.version:
try: try:
current_branch = get_output("git", "rev-parse", "--abbrev-ref", "HEAD") current_branch = get_output("git", "rev-parse", "--abbrev-ref", "HEAD")
if current_branch != "master": if current_branch != "master":
branches = get_output("git", "branch", "-r", "--list", "origin/master") branches = get_output("git", "branch")
if "origin/master" in branches: if "master" in branches:
# If master is present locally, the fetch is allowed to fail # If master is present locally, the fetch is allowed to fail
# because this script will still be able to compare against the # because this script will still be able to compare against the
# master branch. # master branch.
try: try:
get_output("git", "fetch", "origin", "master") get_output("git", "fetch", "origin", "master:master")
except Exception: except Exception:
pass pass
else: else:
# If master is not present locally, the fetch command has to # If master is not present locally, the fetch command has to
# succeed because something else will fail otherwise. # succeed because something else will fail otherwise.
get_output("git", "fetch", "origin", "master") get_output("git", "fetch", "origin", "master:master")
except Exception: except Exception:
print("Fatal error while ensuring local master branch.") print("Fatal error while ensuring local master branch.")
sys.exit(1) sys.exit(1)
@@ -234,7 +232,7 @@ for branch in branches:
match = branch_regex.match(branch) match = branch_regex.match(branch)
if match is not None: if match is not None:
version = tuple(map(int, match.group(1).split("."))) version = tuple(map(int, match.group(1).split(".")))
master_branch_merge = get_output("git", "merge-base", "origin/master", branch) master_branch_merge = get_output("git", "merge-base", "master", branch)
versions.append((version, branch, master_branch_merge)) versions.append((version, branch, master_branch_merge))
versions.sort(reverse=True) versions.sort(reverse=True)
@@ -245,7 +243,7 @@ current_version = None
for version in versions: for version in versions:
version_tuple, branch, master_branch_merge = version version_tuple, branch, master_branch_merge = version
current_branch_merge = get_output("git", "merge-base", current_hash, branch) current_branch_merge = get_output("git", "merge-base", current_hash, branch)
master_current_merge = get_output("git", "merge-base", current_hash, "origin/master") master_current_merge = get_output("git", "merge-base", current_hash, "master")
# The first check checks whether this commit is a child of `master` and # The first check checks whether this commit is a child of `master` and
# the version branch was created before us. # the version branch was created before us.
# The second check checks whether this commit is a child of the version # The second check checks whether this commit is a child of the version

View File

@@ -1,27 +1,12 @@
[Unit] [Unit]
Description=Memgraph: High performance, in-memory, transactional graph database Description=Memgraph: High performance, in-memory, transactional graph database
# Auto-restart config
#StartLimitIntervalSec=300
#StartLimitBurst=5
[Service] [Service]
User=memgraph User=memgraph
Group=memgraph Group=memgraph
ExecStart=/usr/lib/memgraph/memgraph ExecStart=/usr/lib/memgraph/memgraph
# Uncomment this if Memgraph needs more time to write the snapshot on exit. # Uncomment this if Memgraph needs more time to write the snapshot on exit.
#TimeoutStopSec=5min #TimeoutStopSec=5min
# Auto-restart config
#Restart=on-failure
#RestartSec=10s
[Install] [Install]
WantedBy=multi-user.target WantedBy=multi-user.target
# Checks processes' memory usage and force kills it when it is taking up enough
# memory to make the system unstable / about to crash, e.g. on Debian-based OS:
# apt install systemd-oomd
# systemctl enable systemd-oomd
# systemctl start systemd-oomd

View File

@@ -11,12 +11,6 @@ SUPPORTED_OS=(
amzn-2 amzn-2
) )
SUPPORTED_BUILD_TYPES=(
Debug
Release
RelWithDebInfo
)
PROJECT_ROOT="$SCRIPT_DIR/../.." PROJECT_ROOT="$SCRIPT_DIR/../.."
TOOLCHAIN_VERSION="toolchain-v4" TOOLCHAIN_VERSION="toolchain-v4"
ACTIVATE_TOOLCHAIN="source /opt/${TOOLCHAIN_VERSION}/activate" ACTIVATE_TOOLCHAIN="source /opt/${TOOLCHAIN_VERSION}/activate"
@@ -24,16 +18,14 @@ HOST_OUTPUT_DIR="$PROJECT_ROOT/build/output"
print_help () { print_help () {
# TODO(gitbuda): Update the release/package/run.sh help # TODO(gitbuda): Update the release/package/run.sh help
echo "$0 init|package|docker|test {os} {build_type} [--for-docker|--for-platform]" echo "$0 init|package|docker|test {os} [--for-docker|--for-platform]"
echo "" echo ""
echo " OSs: ${SUPPORTED_OS[*]}" echo " OSs: ${SUPPORTED_OS[*]}"
echo " Build types: ${SUPPORTED_BUILD_TYPES[*]}"
exit 1 exit 1
} }
make_package () { make_package () {
os="$1" os="$1"
build_type="$2"
build_container="mgbuild_$os" build_container="mgbuild_$os"
echo "Building Memgraph for $os on $build_container..." echo "Building Memgraph for $os on $build_container..."
@@ -52,10 +44,10 @@ make_package () {
package_command=" cpack -G DEB --config ../CPackConfig.cmake " package_command=" cpack -G DEB --config ../CPackConfig.cmake "
fi fi
telemetry_id_override_flag="" telemetry_id_override_flag=""
if [[ "$#" -gt 2 ]]; then if [[ "$#" -gt 1 ]]; then
if [[ "$3" == "--for-docker" ]]; then if [[ "$2" == "--for-docker" ]]; then
telemetry_id_override_flag=" -DMG_TELEMETRY_ID_OVERRIDE=DOCKER " telemetry_id_override_flag=" -DMG_TELEMETRY_ID_OVERRIDE=DOCKER "
elif [[ "$3" == "--for-platform" ]]; then elif [[ "$2" == "--for-platform" ]]; then
telemetry_id_override_flag=" -DMG_TELEMETRY_ID_OVERRIDE=DOCKER-PLATFORM" telemetry_id_override_flag=" -DMG_TELEMETRY_ID_OVERRIDE=DOCKER-PLATFORM"
else else
print_help print_help
@@ -72,10 +64,6 @@ make_package () {
if [[ "$(git rev-parse --abbrev-ref HEAD)" != "master" ]]; then if [[ "$(git rev-parse --abbrev-ref HEAD)" != "master" ]]; then
git fetch origin master:master git fetch origin master:master
fi fi
# Ensure we have a clean build directory
docker exec "$build_container" rm -rf /memgraph
docker exec "$build_container" mkdir -p /memgraph docker exec "$build_container" mkdir -p /memgraph
# TODO(gitbuda): Revisit copying the whole repo -> makese sense under CI. # TODO(gitbuda): Revisit copying the whole repo -> makese sense under CI.
docker cp "$PROJECT_ROOT/." "$build_container:/memgraph/" docker cp "$PROJECT_ROOT/." "$build_container:/memgraph/"
@@ -101,9 +89,9 @@ make_package () {
docker exec "$build_container" bash -c "cd $container_build_dir && rm -rf ./*" docker exec "$build_container" bash -c "cd $container_build_dir && rm -rf ./*"
# TODO(gitbuda): cmake fails locally if remote is clone via ssh because of the key -> FIX # TODO(gitbuda): cmake fails locally if remote is clone via ssh because of the key -> FIX
if [[ "$os" =~ "-arm" ]]; then if [[ "$os" =~ "-arm" ]]; then
docker exec "$build_container" bash -c "cd $container_build_dir && $ACTIVATE_TOOLCHAIN && cmake -DCMAKE_BUILD_TYPE=$build_type -DMG_ARCH="ARM64" $telemetry_id_override_flag .." docker exec "$build_container" bash -c "cd $container_build_dir && $ACTIVATE_TOOLCHAIN && cmake -DCMAKE_BUILD_TYPE=release -DMG_ARCH="ARM64" $telemetry_id_override_flag .."
else else
docker exec "$build_container" bash -c "cd $container_build_dir && $ACTIVATE_TOOLCHAIN && cmake -DCMAKE_BUILD_TYPE=$build_type $telemetry_id_override_flag .." docker exec "$build_container" bash -c "cd $container_build_dir && $ACTIVATE_TOOLCHAIN && cmake -DCMAKE_BUILD_TYPE=release $telemetry_id_override_flag .."
fi fi
# ' is used instead of " because we need to run make within the allowed # ' is used instead of " because we need to run make within the allowed
# container resources. # container resources.
@@ -153,34 +141,20 @@ case "$1" in
package) package)
shift 1 shift 1
if [[ "$#" -lt 2 ]]; then if [[ "$#" -lt 1 ]]; then
print_help print_help
fi fi
os="$1" os="$1"
build_type="$2" shift 1
shift 2
is_os_ok=false is_os_ok=false
for supported_os in "${SUPPORTED_OS[@]}"; do for supported_os in "${SUPPORTED_OS[@]}"; do
if [[ "$supported_os" == "${os}" ]]; then if [[ "$supported_os" == "${os}" ]]; then
is_os_ok=true is_os_ok=true
break
fi fi
done done
is_build_type_ok=false if [[ "$is_os_ok" == true ]]; then
for supported_build_type in "${SUPPORTED_BUILD_TYPES[@]}"; do make_package "$os" "$@"
if [[ "$supported_build_type" == "${build_type}" ]]; then
is_build_type_ok=true
break
fi
done
if [[ "$is_os_ok" == true && "$is_build_type_ok" == true ]]; then
make_package "$os" "$build_type" "$@"
else else
if [[ "$is_os_ok" == false ]]; then
echo "Unsupported OS: $os"
elif [[ "$is_build_type_ok" == false ]]; then
echo "Unsupported build type: $build_type"
fi
print_help print_help
fi fi
;; ;;

View File

@@ -21,11 +21,6 @@ add_subdirectory(audit)
add_subdirectory(dbms) add_subdirectory(dbms)
add_subdirectory(flags) add_subdirectory(flags)
add_subdirectory(distributed) add_subdirectory(distributed)
add_subdirectory(replication)
add_subdirectory(replication_handler)
add_subdirectory(coordination)
add_subdirectory(replication_coordination_glue)
add_subdirectory(system)
string(TOLOWER ${CMAKE_BUILD_TYPE} lower_build_type) string(TOLOWER ${CMAKE_BUILD_TYPE} lower_build_type)
@@ -45,10 +40,10 @@ set(mg_single_node_v2_sources
add_executable(memgraph ${mg_single_node_v2_sources}) add_executable(memgraph ${mg_single_node_v2_sources})
target_include_directories(memgraph PUBLIC ${CMAKE_SOURCE_DIR}/include) target_include_directories(memgraph PUBLIC ${CMAKE_SOURCE_DIR}/include)
target_link_libraries(memgraph stdc++fs Threads::Threads target_link_libraries(memgraph stdc++fs Threads::Threads
mg-telemetry mgcxx_text_search tantivy_text_search mg-communication mg-communication-metrics mg-memory mg-utils mg-license mg-settings mg-glue mg-flags mg::system mg::replication_handler) mg-telemetry mg-communication mg-memory mg-utils mg-license mg-settings mg-glue mg-flags)
# NOTE: `include/mg_procedure.syms` describes a pattern match for symbols which # NOTE: `include/mg_procedure.syms` describes a pattern match for symbols which
# should be dynamically exported, so that `dlopen` can correctly link th # should be dynamically exported, so that `dlopen` can correctly link the
# symbols in custom procedure module libraries. # symbols in custom procedure module libraries.
target_link_libraries(memgraph "-Wl,--dynamic-list=${CMAKE_SOURCE_DIR}/include/mg_procedure.syms") target_link_libraries(memgraph "-Wl,--dynamic-list=${CMAKE_SOURCE_DIR}/include/mg_procedure.syms")
set_target_properties(memgraph PROPERTIES set_target_properties(memgraph PROPERTIES
@@ -57,10 +52,7 @@ set_target_properties(memgraph PROPERTIES
OUTPUT_NAME "memgraph-${MEMGRAPH_VERSION}_${CMAKE_BUILD_TYPE}" OUTPUT_NAME "memgraph-${MEMGRAPH_VERSION}_${CMAKE_BUILD_TYPE}"
# Output the executable in main binary dir. # Output the executable in main binary dir.
RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR} RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
POSITION_INDEPENDENT_CODE ON
)
# Create symlink to the built executable. # Create symlink to the built executable.
add_custom_command(TARGET memgraph POST_BUILD add_custom_command(TARGET memgraph POST_BUILD
@@ -150,7 +142,7 @@ install(CODE "file(MAKE_DIRECTORY \$ENV{DESTDIR}/var/log/memgraph
# Memgraph CSV Import Tool Executable # Memgraph CSV Import Tool Executable
# ---------------------------------------------------------------------------- # ----------------------------------------------------------------------------
add_executable(mg_import_csv mg_import_csv.cpp) add_executable(mg_import_csv mg_import_csv.cpp)
target_link_libraries(mg_import_csv mg-storage-v2 mg-dbms) target_link_libraries(mg_import_csv mg-storage-v2)
# Strip the executable in release build. # Strip the executable in release build.
if(lower_build_type STREQUAL "release") if(lower_build_type STREQUAL "release")

View File

@@ -13,7 +13,6 @@
#include <fmt/format.h> #include <fmt/format.h>
#include <json/json.hpp> #include <json/json.hpp>
#include <utility>
#include "storage/v2/temporal.hpp" #include "storage/v2/temporal.hpp"
#include "utils/logging.hpp" #include "utils/logging.hpp"
@@ -88,8 +87,8 @@ inline nlohmann::json PropertyValueToJson(const storage::PropertyValue &pv) {
return ret; return ret;
} }
Log::Log(std::filesystem::path storage_directory, int32_t buffer_size, int32_t buffer_flush_interval_millis) Log::Log(const std::filesystem::path &storage_directory, int32_t buffer_size, int32_t buffer_flush_interval_millis)
: storage_directory_(std::move(storage_directory)), : storage_directory_(storage_directory),
buffer_size_(buffer_size), buffer_size_(buffer_size),
buffer_flush_interval_millis_(buffer_flush_interval_millis), buffer_flush_interval_millis_(buffer_flush_interval_millis),
started_(false) {} started_(false) {}

View File

@@ -36,7 +36,7 @@ class Log {
}; };
public: public:
Log(std::filesystem::path storage_directory, int32_t buffer_size, int32_t buffer_flush_interval_millis); Log(const std::filesystem::path &storage_directory, int32_t buffer_size, int32_t buffer_flush_interval_millis);
~Log(); ~Log();

View File

@@ -2,9 +2,7 @@ set(auth_src_files
auth.cpp auth.cpp
crypto.cpp crypto.cpp
models.cpp models.cpp
module.cpp module.cpp)
rpc.cpp
replication_handlers.cpp)
find_package(Seccomp REQUIRED) find_package(Seccomp REQUIRED)
find_package(fmt REQUIRED) find_package(fmt REQUIRED)
@@ -13,7 +11,7 @@ find_package(gflags REQUIRED)
add_library(mg-auth STATIC ${auth_src_files}) add_library(mg-auth STATIC ${auth_src_files})
target_link_libraries(mg-auth json libbcrypt gflags fmt::fmt) target_link_libraries(mg-auth json libbcrypt gflags fmt::fmt)
target_link_libraries(mg-auth mg-utils mg-kvstore mg-license mg::system mg-replication) target_link_libraries(mg-auth mg-utils mg-kvstore mg-license )
target_link_libraries(mg-auth ${Seccomp_LIBRARIES}) target_link_libraries(mg-auth ${Seccomp_LIBRARIES})
target_include_directories(mg-auth SYSTEM PRIVATE ${Seccomp_INCLUDE_DIRS}) target_include_directories(mg-auth SYSTEM PRIVATE ${Seccomp_INCLUDE_DIRS})

View File

@@ -1,4 +1,4 @@
// Copyright 2024 Memgraph Ltd. // Copyright 2023 Memgraph Ltd.
// //
// Licensed as a Memgraph Enterprise file under the Memgraph Enterprise // Licensed as a Memgraph Enterprise file under the Memgraph Enterprise
// License (the "License"); by using this file, you agree to be bound by the terms of the License, and you may not use // License (the "License"); by using this file, you agree to be bound by the terms of the License, and you may not use
@@ -8,18 +8,17 @@
#include "auth/auth.hpp" #include "auth/auth.hpp"
#include <cstring>
#include <iostream> #include <iostream>
#include <optional> #include <limits>
#include <utility> #include <utility>
#include <fmt/format.h> #include <fmt/format.h>
#include "auth/crypto.hpp"
#include "auth/exceptions.hpp" #include "auth/exceptions.hpp"
#include "auth/rpc.hpp"
#include "license/license.hpp" #include "license/license.hpp"
#include "system/transaction.hpp"
#include "utils/flag_validation.hpp" #include "utils/flag_validation.hpp"
#include "utils/logging.hpp"
#include "utils/message.hpp" #include "utils/message.hpp"
#include "utils/settings.hpp" #include "utils/settings.hpp"
#include "utils/string.hpp" #include "utils/string.hpp"
@@ -44,89 +43,9 @@ DEFINE_VALIDATED_int32(auth_module_timeout_ms, 10000,
FLAG_IN_RANGE(100, 1800000)); FLAG_IN_RANGE(100, 1800000));
namespace memgraph::auth { namespace memgraph::auth {
namespace {
#ifdef MG_ENTERPRISE
/**
* REPLICATION SYSTEM ACTION IMPLEMENTATIONS
*/
struct UpdateAuthData : memgraph::system::ISystemAction {
explicit UpdateAuthData(User user) : user_{std::move(user)}, role_{std::nullopt} {}
explicit UpdateAuthData(Role role) : user_{std::nullopt}, role_{std::move(role)} {}
void DoDurability() override { /* Done during Auth execution */
}
bool DoReplication(replication::ReplicationClient &client, const utils::UUID &main_uuid,
replication::ReplicationEpoch const &epoch,
memgraph::system::Transaction const &txn) const override {
auto check_response = [](const replication::UpdateAuthDataRes &response) { return response.success; };
if (user_) {
return client.SteamAndFinalizeDelta<replication::UpdateAuthDataRpc>(
check_response, main_uuid, std::string{epoch.id()}, txn.last_committed_system_timestamp(), txn.timestamp(),
*user_);
}
if (role_) {
return client.SteamAndFinalizeDelta<replication::UpdateAuthDataRpc>(
check_response, main_uuid, std::string{epoch.id()}, txn.last_committed_system_timestamp(), txn.timestamp(),
*role_);
}
// Should never get here
MG_ASSERT(false, "Trying to update auth data that is not a user nor a role");
return {};
}
void PostReplication(replication::RoleMainData &mainData) const override {}
private:
std::optional<User> user_;
std::optional<Role> role_;
};
struct DropAuthData : memgraph::system::ISystemAction {
enum class AuthDataType { USER, ROLE };
explicit DropAuthData(AuthDataType type, std::string_view name) : type_{type}, name_{name} {}
void DoDurability() override { /* Done during Auth execution */
}
bool DoReplication(replication::ReplicationClient &client, const utils::UUID &main_uuid,
replication::ReplicationEpoch const &epoch,
memgraph::system::Transaction const &txn) const override {
auto check_response = [](const replication::DropAuthDataRes &response) { return response.success; };
memgraph::replication::DropAuthDataReq::DataType type{};
switch (type_) {
case AuthDataType::USER:
type = memgraph::replication::DropAuthDataReq::DataType::USER;
break;
case AuthDataType::ROLE:
type = memgraph::replication::DropAuthDataReq::DataType::ROLE;
break;
}
return client.SteamAndFinalizeDelta<replication::DropAuthDataRpc>(
check_response, main_uuid, std::string{epoch.id()}, txn.last_committed_system_timestamp(), txn.timestamp(),
type, name_);
}
void PostReplication(replication::RoleMainData &mainData) const override {}
private:
AuthDataType type_;
std::string name_;
};
#endif
/**
* CONSTANTS
*/
const std::string kUserPrefix = "user:"; const std::string kUserPrefix = "user:";
const std::string kRolePrefix = "role:"; const std::string kRolePrefix = "role:";
const std::string kLinkPrefix = "link:"; const std::string kLinkPrefix = "link:";
const std::string kVersion = "version";
static constexpr auto kVersionV1 = "V1";
} // namespace
/** /**
* All data stored in the `Auth` storage is stored in an underlying * All data stored in the `Auth` storage is stored in an underlying
@@ -145,59 +64,7 @@ static constexpr auto kVersionV1 = "V1";
* key="link:<username>", value="<rolename>" * key="link:<username>", value="<rolename>"
*/ */
namespace { Auth::Auth(const std::string &storage_directory) : storage_(storage_directory), module_(FLAGS_auth_module_executable) {}
void MigrateVersions(kvstore::KVStore &store) {
static constexpr auto kPasswordHashV0V1 = "password_hash";
auto version_str = store.Get(kVersion);
if (!version_str) {
using namespace std::string_literals;
// pre versioning, add version to the store
auto puts = std::map<std::string, std::string>{{kVersion, kVersionV1}};
// also add hash kind into durability
auto it = store.begin(kUserPrefix);
auto const e = store.end(kUserPrefix);
if (it != e) {
const auto hash_algo = CurrentHashAlgorithm();
spdlog::info("Updating auth durability, assuming previously stored as {}", AsString(hash_algo));
for (; it != e; ++it) {
auto const &[key, value] = *it;
try {
auto user_data = nlohmann::json::parse(value);
auto password_hash = user_data[kPasswordHashV0V1];
if (!password_hash.is_string()) {
throw AuthException("Couldn't load user data!");
}
// upgrade the password_hash to include the hash algortihm
if (password_hash.empty()) {
user_data[kPasswordHashV0V1] = nullptr;
} else {
user_data[kPasswordHashV0V1] = HashedPassword{hash_algo, password_hash};
}
puts.emplace(key, user_data.dump());
} catch (const nlohmann::json::parse_error &e) {
throw AuthException("Couldn't load user data!");
}
}
}
// Perform migration to V1
store.PutMultiple(puts);
version_str = kVersionV1;
}
}
}; // namespace
Auth::Auth(std::string storage_directory, Config config)
: storage_(std::move(storage_directory)), module_(FLAGS_auth_module_executable), config_{std::move(config)} {
MigrateVersions(storage_);
}
std::optional<User> Auth::Authenticate(const std::string &username, const std::string &password) { std::optional<User> Auth::Authenticate(const std::string &username, const std::string &password) {
if (module_.IsUsed()) { if (module_.IsUsed()) {
@@ -228,12 +95,6 @@ std::optional<User> Auth::Authenticate(const std::string &username, const std::s
// Authenticate the user. // Authenticate the user.
if (!is_authenticated) return std::nullopt; if (!is_authenticated) return std::nullopt;
/**
* TODO
* The auth module should not update auth data.
* There is now way to replicate it and we should not be storing sensitive data if we don't have to.
*/
// Find or create the user and return it. // Find or create the user and return it.
auto user = GetUser(username); auto user = GetUser(username);
if (!user) { if (!user) {
@@ -252,7 +113,7 @@ std::optional<User> Auth::Authenticate(const std::string &username, const std::s
return std::nullopt; return std::nullopt;
} }
} else { } else {
UpdatePassword(*user, password); user->UpdatePassword(password);
} }
if (FLAGS_auth_module_manage_roles) { if (FLAGS_auth_module_manage_roles) {
if (!rolename.empty()) { if (!rolename.empty()) {
@@ -294,10 +155,6 @@ std::optional<User> Auth::Authenticate(const std::string &username, const std::s
username, "https://memgr.ph/auth")); username, "https://memgr.ph/auth"));
return std::nullopt; return std::nullopt;
} }
if (user->UpgradeHash(password)) {
SaveUser(*user);
}
return user; return user;
} }
} }
@@ -326,7 +183,7 @@ std::optional<User> Auth::GetUser(const std::string &username_orig) const {
return user; return user;
} }
void Auth::SaveUser(const User &user, system::Transaction *system_tx) { void Auth::SaveUser(const User &user) {
bool success = false; bool success = false;
if (const auto *role = user.role(); role != nullptr) { if (const auto *role = user.role(); role != nullptr) {
success = storage_.PutMultiple( success = storage_.PutMultiple(
@@ -338,72 +195,26 @@ void Auth::SaveUser(const User &user, system::Transaction *system_tx) {
if (!success) { if (!success) {
throw AuthException("Couldn't save user '{}'!", user.username()); throw AuthException("Couldn't save user '{}'!", user.username());
} }
// All changes to the user end up calling this function, so no need to add a delta anywhere else
if (system_tx) {
#ifdef MG_ENTERPRISE
system_tx->AddAction<UpdateAuthData>(user);
#endif
}
} }
void Auth::UpdatePassword(auth::User &user, const std::optional<std::string> &password) { std::optional<User> Auth::AddUser(const std::string &username, const std::optional<std::string> &password) {
// Check if null
if (!password) {
if (!config_.password_permit_null) {
throw AuthException("Null passwords aren't permitted!");
}
} else {
// Check if compliant with our filter
if (config_.custom_password_regex) {
if (const auto license_check_result = license::global_license_checker.IsEnterpriseValid(utils::global_settings);
license_check_result.HasError()) {
throw AuthException(
"Custom password regex is a Memgraph Enterprise feature. Please set the config "
"(\"--auth-password-strength-regex\") to its default value (\"{}\") or remove the flag.\n{}",
glue::kDefaultPasswordRegex,
license::LicenseCheckErrorToString(license_check_result.GetError(), "password regex"));
}
}
if (!std::regex_match(*password, config_.password_regex)) {
throw AuthException(
"The user password doesn't conform to the required strength! Regex: "
"\"{}\"",
config_.password_regex_str);
}
}
// All checks passed; update
user.UpdatePassword(password);
}
std::optional<User> Auth::AddUser(const std::string &username, const std::optional<std::string> &password,
system::Transaction *system_tx) {
if (!NameRegexMatch(username)) {
throw AuthException("Invalid user name.");
}
auto existing_user = GetUser(username); auto existing_user = GetUser(username);
if (existing_user) return std::nullopt; if (existing_user) return std::nullopt;
auto existing_role = GetRole(username); auto existing_role = GetRole(username);
if (existing_role) return std::nullopt; if (existing_role) return std::nullopt;
auto new_user = User(username); auto new_user = User(username);
UpdatePassword(new_user, password); new_user.UpdatePassword(password);
SaveUser(new_user, system_tx); SaveUser(new_user);
return new_user; return new_user;
} }
bool Auth::RemoveUser(const std::string &username_orig, system::Transaction *system_tx) { bool Auth::RemoveUser(const std::string &username_orig) {
auto username = utils::ToLowerCase(username_orig); auto username = utils::ToLowerCase(username_orig);
if (!storage_.Get(kUserPrefix + username)) return false; if (!storage_.Get(kUserPrefix + username)) return false;
std::vector<std::string> keys({kLinkPrefix + username, kUserPrefix + username}); std::vector<std::string> keys({kLinkPrefix + username, kUserPrefix + username});
if (!storage_.DeleteMultiple(keys)) { if (!storage_.DeleteMultiple(keys)) {
throw AuthException("Couldn't remove user '{}'!", username); throw AuthException("Couldn't remove user '{}'!", username);
} }
// Handling drop user delta
if (system_tx) {
#ifdef MG_ENTERPRISE
system_tx->AddAction<DropAuthData>(DropAuthData::AuthDataType::USER, username);
#endif
}
return true; return true;
} }
@@ -420,19 +231,6 @@ std::vector<auth::User> Auth::AllUsers() const {
return ret; return ret;
} }
std::vector<std::string> Auth::AllUsernames() const {
std::vector<std::string> ret;
for (auto it = storage_.begin(kUserPrefix); it != storage_.end(kUserPrefix); ++it) {
auto username = it->first.substr(kUserPrefix.size());
if (username != utils::ToLowerCase(username)) continue;
auto user = GetUser(username);
if (user) {
ret.push_back(username);
}
}
return ret;
}
bool Auth::HasUsers() const { return storage_.begin(kUserPrefix) != storage_.end(kUserPrefix); } bool Auth::HasUsers() const { return storage_.begin(kUserPrefix) != storage_.end(kUserPrefix); }
std::optional<Role> Auth::GetRole(const std::string &rolename_orig) const { std::optional<Role> Auth::GetRole(const std::string &rolename_orig) const {
@@ -450,30 +248,23 @@ std::optional<Role> Auth::GetRole(const std::string &rolename_orig) const {
return Role::Deserialize(data); return Role::Deserialize(data);
} }
void Auth::SaveRole(const Role &role, system::Transaction *system_tx) { void Auth::SaveRole(const Role &role) {
if (!storage_.Put(kRolePrefix + role.rolename(), role.Serialize().dump())) { if (!storage_.Put(kRolePrefix + role.rolename(), role.Serialize().dump())) {
throw AuthException("Couldn't save role '{}'!", role.rolename()); throw AuthException("Couldn't save role '{}'!", role.rolename());
} }
// All changes to the role end up calling this function, so no need to add a delta anywhere else
if (system_tx) {
#ifdef MG_ENTERPRISE
system_tx->AddAction<UpdateAuthData>(role);
#endif
}
} }
std::optional<Role> Auth::AddRole(const std::string &rolename, system::Transaction *system_tx) { std::optional<Role> Auth::AddRole(const std::string &rolename) {
if (!NameRegexMatch(rolename)) { auto existing_role = GetRole(rolename);
throw AuthException("Invalid role name."); if (existing_role) return std::nullopt;
} auto existing_user = GetUser(rolename);
if (auto existing_role = GetRole(rolename)) return std::nullopt; if (existing_user) return std::nullopt;
if (auto existing_user = GetUser(rolename)) return std::nullopt;
auto new_role = Role(rolename); auto new_role = Role(rolename);
SaveRole(new_role, system_tx); SaveRole(new_role);
return new_role; return new_role;
} }
bool Auth::RemoveRole(const std::string &rolename_orig, system::Transaction *system_tx) { bool Auth::RemoveRole(const std::string &rolename_orig) {
auto rolename = utils::ToLowerCase(rolename_orig); auto rolename = utils::ToLowerCase(rolename_orig);
if (!storage_.Get(kRolePrefix + rolename)) return false; if (!storage_.Get(kRolePrefix + rolename)) return false;
std::vector<std::string> keys; std::vector<std::string> keys;
@@ -486,12 +277,6 @@ bool Auth::RemoveRole(const std::string &rolename_orig, system::Transaction *sys
if (!storage_.DeleteMultiple(keys)) { if (!storage_.DeleteMultiple(keys)) {
throw AuthException("Couldn't remove role '{}'!", rolename); throw AuthException("Couldn't remove role '{}'!", rolename);
} }
// Handling drop role delta
if (system_tx) {
#ifdef MG_ENTERPRISE
system_tx->AddAction<DropAuthData>(DropAuthData::AuthDataType::ROLE, rolename);
#endif
}
return true; return true;
} }
@@ -500,7 +285,8 @@ std::vector<auth::Role> Auth::AllRoles() const {
for (auto it = storage_.begin(kRolePrefix); it != storage_.end(kRolePrefix); ++it) { for (auto it = storage_.begin(kRolePrefix); it != storage_.end(kRolePrefix); ++it) {
auto rolename = it->first.substr(kRolePrefix.size()); auto rolename = it->first.substr(kRolePrefix.size());
if (rolename != utils::ToLowerCase(rolename)) continue; if (rolename != utils::ToLowerCase(rolename)) continue;
if (auto role = GetRole(rolename)) { auto role = GetRole(rolename);
if (role) {
ret.push_back(*role); ret.push_back(*role);
} else { } else {
throw AuthException("Couldn't load role '{}'!", rolename); throw AuthException("Couldn't load role '{}'!", rolename);
@@ -509,27 +295,16 @@ std::vector<auth::Role> Auth::AllRoles() const {
return ret; return ret;
} }
std::vector<std::string> Auth::AllRolenames() const {
std::vector<std::string> ret;
for (auto it = storage_.begin(kRolePrefix); it != storage_.end(kRolePrefix); ++it) {
auto rolename = it->first.substr(kRolePrefix.size());
if (rolename != utils::ToLowerCase(rolename)) continue;
if (auto role = GetRole(rolename)) {
ret.push_back(rolename);
}
}
return ret;
}
std::vector<auth::User> Auth::AllUsersForRole(const std::string &rolename_orig) const { std::vector<auth::User> Auth::AllUsersForRole(const std::string &rolename_orig) const {
const auto rolename = utils::ToLowerCase(rolename_orig); auto rolename = utils::ToLowerCase(rolename_orig);
std::vector<auth::User> ret; std::vector<auth::User> ret;
for (auto it = storage_.begin(kLinkPrefix); it != storage_.end(kLinkPrefix); ++it) { for (auto it = storage_.begin(kLinkPrefix); it != storage_.end(kLinkPrefix); ++it) {
auto username = it->first.substr(kLinkPrefix.size()); auto username = it->first.substr(kLinkPrefix.size());
if (username != utils::ToLowerCase(username)) continue; if (username != utils::ToLowerCase(username)) continue;
if (it->second != utils::ToLowerCase(it->second)) continue; if (it->second != utils::ToLowerCase(it->second)) continue;
if (it->second == rolename) { if (it->second == rolename) {
if (auto user = GetUser(username)) { auto user = GetUser(username);
if (user) {
ret.push_back(std::move(*user)); ret.push_back(std::move(*user));
} else { } else {
throw AuthException("Couldn't load user '{}'!", username); throw AuthException("Couldn't load user '{}'!", username);
@@ -540,67 +315,56 @@ std::vector<auth::User> Auth::AllUsersForRole(const std::string &rolename_orig)
} }
#ifdef MG_ENTERPRISE #ifdef MG_ENTERPRISE
bool Auth::GrantDatabaseToUser(const std::string &db, const std::string &name, system::Transaction *system_tx) { bool Auth::GrantDatabaseToUser(const std::string &db, const std::string &name) {
if (auto user = GetUser(name)) { auto user = GetUser(name);
if (user) {
if (db == kAllDatabases) { if (db == kAllDatabases) {
user->db_access().GrantAll(); user->db_access().GrantAll();
} else { } else {
user->db_access().Add(db); user->db_access().Add(db);
} }
SaveUser(*user, system_tx); SaveUser(*user);
return true; return true;
} }
return false; return false;
} }
bool Auth::RevokeDatabaseFromUser(const std::string &db, const std::string &name, system::Transaction *system_tx) { bool Auth::RevokeDatabaseFromUser(const std::string &db, const std::string &name) {
if (auto user = GetUser(name)) { auto user = GetUser(name);
if (user) {
if (db == kAllDatabases) { if (db == kAllDatabases) {
user->db_access().DenyAll(); user->db_access().DenyAll();
} else { } else {
user->db_access().Remove(db); user->db_access().Remove(db);
} }
SaveUser(*user, system_tx); SaveUser(*user);
return true; return true;
} }
return false; return false;
} }
void Auth::DeleteDatabase(const std::string &db, system::Transaction *system_tx) { void Auth::DeleteDatabase(const std::string &db) {
for (auto it = storage_.begin(kUserPrefix); it != storage_.end(kUserPrefix); ++it) { for (auto it = storage_.begin(kUserPrefix); it != storage_.end(kUserPrefix); ++it) {
auto username = it->first.substr(kUserPrefix.size()); auto username = it->first.substr(kUserPrefix.size());
if (auto user = GetUser(username)) { auto user = GetUser(username);
if (user) {
user->db_access().Delete(db); user->db_access().Delete(db);
SaveUser(*user, system_tx); SaveUser(*user);
} }
} }
} }
bool Auth::SetMainDatabase(std::string_view db, const std::string &name, system::Transaction *system_tx) { bool Auth::SetMainDatabase(const std::string &db, const std::string &name) {
if (auto user = GetUser(name)) { auto user = GetUser(name);
if (user) {
if (!user->db_access().SetDefault(db)) { if (!user->db_access().SetDefault(db)) {
throw AuthException("Couldn't set default database '{}' for user '{}'!", db, name); throw AuthException("Couldn't set default database '{}' for user '{}'!", db, name);
} }
SaveUser(*user, system_tx); SaveUser(*user);
return true; return true;
} }
return false; return false;
} }
#endif #endif
bool Auth::NameRegexMatch(const std::string &user_or_role) const {
if (config_.custom_name_regex) {
if (const auto license_check_result =
memgraph::license::global_license_checker.IsEnterpriseValid(memgraph::utils::global_settings);
license_check_result.HasError()) {
throw memgraph::auth::AuthException(
"Custom user/role regex is a Memgraph Enterprise feature. Please set the config "
"(\"--auth-user-or-role-name-regex\") to its default value (\"{}\") or remove the flag.\n{}",
glue::kDefaultUserRoleRegex,
memgraph::license::LicenseCheckErrorToString(license_check_result.GetError(), "user/role regex"));
}
}
return std::regex_match(user_or_role, config_.name_regex);
}
} // namespace memgraph::auth } // namespace memgraph::auth

View File

@@ -1,4 +1,4 @@
// Copyright 2024 Memgraph Ltd. // Copyright 2023 Memgraph Ltd.
// //
// Licensed as a Memgraph Enterprise file under the Memgraph Enterprise // Licensed as a Memgraph Enterprise file under the Memgraph Enterprise
// License (the "License"); by using this file, you agree to be bound by the terms of the License, and you may not use // License (the "License"); by using this file, you agree to be bound by the terms of the License, and you may not use
@@ -10,23 +10,16 @@
#include <mutex> #include <mutex>
#include <optional> #include <optional>
#include <regex>
#include <vector> #include <vector>
#include "auth/exceptions.hpp" #include "auth/exceptions.hpp"
#include "auth/models.hpp" #include "auth/models.hpp"
#include "auth/module.hpp" #include "auth/module.hpp"
#include "glue/auth_global.hpp"
#include "kvstore/kvstore.hpp" #include "kvstore/kvstore.hpp"
#include "system/action.hpp"
#include "utils/settings.hpp" #include "utils/settings.hpp"
#include "utils/synchronized.hpp"
namespace memgraph::auth { namespace memgraph::auth {
class Auth;
using SynchedAuth = memgraph::utils::Synchronized<memgraph::auth::Auth, memgraph::utils::WritePrioritizedRWLock>;
static const constexpr char *const kAllDatabases = "*"; static const constexpr char *const kAllDatabases = "*";
/** /**
@@ -38,47 +31,7 @@ static const constexpr char *const kAllDatabases = "*";
*/ */
class Auth final { class Auth final {
public: public:
struct Config { explicit Auth(const std::string &storage_directory);
Config() {}
Config(std::string name_regex, std::string password_regex, bool password_permit_null)
: name_regex_str{std::move(name_regex)},
password_regex_str{std::move(password_regex)},
password_permit_null{password_permit_null},
custom_name_regex{name_regex_str != glue::kDefaultUserRoleRegex},
name_regex{name_regex_str},
custom_password_regex{password_regex_str != glue::kDefaultPasswordRegex},
password_regex{password_regex_str} {}
std::string name_regex_str{glue::kDefaultUserRoleRegex};
std::string password_regex_str{glue::kDefaultPasswordRegex};
bool password_permit_null{true};
private:
friend class Auth;
bool custom_name_regex{false};
std::regex name_regex{name_regex_str};
bool custom_password_regex{false};
std::regex password_regex{password_regex_str};
};
explicit Auth(std::string storage_directory, Config config);
/**
* @brief Set the Config object
*
* @param config
*/
void SetConfig(Config config) {
// NOTE: The Auth class itself is not thread-safe, higher-level code needs to synchronize it when using it.
config_ = std::move(config);
}
/**
* @brief
*
* @return Config
*/
Config GetConfig() const { return config_; }
/** /**
* Authenticates a user using his username and password. * Authenticates a user using his username and password.
@@ -108,7 +61,7 @@ class Auth final {
* *
* @throw AuthException if unable to save the user. * @throw AuthException if unable to save the user.
*/ */
void SaveUser(const User &user, system::Transaction *system_tx = nullptr); void SaveUser(const User &user);
/** /**
* Creates a user if the user doesn't exist. * Creates a user if the user doesn't exist.
@@ -119,8 +72,7 @@ class Auth final {
* @return a user when the user is created, nullopt if the user exists * @return a user when the user is created, nullopt if the user exists
* @throw AuthException if unable to save the user. * @throw AuthException if unable to save the user.
*/ */
std::optional<User> AddUser(const std::string &username, const std::optional<std::string> &password = std::nullopt, std::optional<User> AddUser(const std::string &username, const std::optional<std::string> &password = std::nullopt);
system::Transaction *system_tx = nullptr);
/** /**
* Removes a user from the storage. * Removes a user from the storage.
@@ -131,15 +83,7 @@ class Auth final {
* doesn't exist * doesn't exist
* @throw AuthException if unable to remove the user. * @throw AuthException if unable to remove the user.
*/ */
bool RemoveUser(const std::string &username, system::Transaction *system_tx = nullptr); bool RemoveUser(const std::string &username);
/**
* @brief
*
* @param user
* @param password
*/
void UpdatePassword(auth::User &user, const std::optional<std::string> &password);
/** /**
* Gets all users from the storage. * Gets all users from the storage.
@@ -149,13 +93,6 @@ class Auth final {
*/ */
std::vector<User> AllUsers() const; std::vector<User> AllUsers() const;
/**
* @brief
*
* @return std::vector<std::string>
*/
std::vector<std::string> AllUsernames() const;
/** /**
* Returns whether there are users in the storage. * Returns whether there are users in the storage.
* *
@@ -180,7 +117,7 @@ class Auth final {
* *
* @throw AuthException if unable to save the role. * @throw AuthException if unable to save the role.
*/ */
void SaveRole(const Role &role, system::Transaction *system_tx = nullptr); void SaveRole(const Role &role);
/** /**
* Creates a role if the role doesn't exist. * Creates a role if the role doesn't exist.
@@ -190,7 +127,7 @@ class Auth final {
* @return a role when the role is created, nullopt if the role exists * @return a role when the role is created, nullopt if the role exists
* @throw AuthException if unable to save the role. * @throw AuthException if unable to save the role.
*/ */
std::optional<Role> AddRole(const std::string &rolename, system::Transaction *system_tx = nullptr); std::optional<Role> AddRole(const std::string &rolename);
/** /**
* Removes a role from the storage. * Removes a role from the storage.
@@ -201,7 +138,7 @@ class Auth final {
* doesn't exist * doesn't exist
* @throw AuthException if unable to remove the role. * @throw AuthException if unable to remove the role.
*/ */
bool RemoveRole(const std::string &rolename, system::Transaction *system_tx = nullptr); bool RemoveRole(const std::string &rolename);
/** /**
* Gets all roles from the storage. * Gets all roles from the storage.
@@ -211,13 +148,6 @@ class Auth final {
*/ */
std::vector<Role> AllRoles() const; std::vector<Role> AllRoles() const;
/**
* @brief
*
* @return std::vector<std::string>
*/
std::vector<std::string> AllRolenames() const;
/** /**
* Gets all users for a role from the storage. * Gets all users for a role from the storage.
* *
@@ -237,7 +167,7 @@ class Auth final {
* @return true on success * @return true on success
* @throw AuthException if unable to find or update the user * @throw AuthException if unable to find or update the user
*/ */
bool RevokeDatabaseFromUser(const std::string &db, const std::string &name, system::Transaction *system_tx = nullptr); bool RevokeDatabaseFromUser(const std::string &db, const std::string &name);
/** /**
* @brief Grant access to individual database for a user. * @brief Grant access to individual database for a user.
@@ -247,7 +177,7 @@ class Auth final {
* @return true on success * @return true on success
* @throw AuthException if unable to find or update the user * @throw AuthException if unable to find or update the user
*/ */
bool GrantDatabaseToUser(const std::string &db, const std::string &name, system::Transaction *system_tx = nullptr); bool GrantDatabaseToUser(const std::string &db, const std::string &name);
/** /**
* @brief Delete a database from all users. * @brief Delete a database from all users.
@@ -255,7 +185,7 @@ class Auth final {
* @param db name of the database to delete * @param db name of the database to delete
* @throw AuthException if unable to read data * @throw AuthException if unable to read data
*/ */
void DeleteDatabase(const std::string &db, system::Transaction *system_tx = nullptr); void DeleteDatabase(const std::string &db);
/** /**
* @brief Set main database for an individual user. * @brief Set main database for an individual user.
@@ -265,24 +195,14 @@ class Auth final {
* @return true on success * @return true on success
* @throw AuthException if unable to find or update the user * @throw AuthException if unable to find or update the user
*/ */
bool SetMainDatabase(std::string_view db, const std::string &name, system::Transaction *system_tx = nullptr); bool SetMainDatabase(const std::string &db, const std::string &name);
#endif #endif
private: private:
/**
* @brief
*
* @param user_or_role
* @return true
* @return false
*/
bool NameRegexMatch(const std::string &user_or_role) const;
// Even though the `kvstore::KVStore` class is guaranteed to be thread-safe, // Even though the `kvstore::KVStore` class is guaranteed to be thread-safe,
// Auth is not thread-safe because modifying users and roles might require // Auth is not thread-safe because modifying users and roles might require
// more than one operation on the storage. // more than one operation on the storage.
kvstore::KVStore storage_; kvstore::KVStore storage_;
auth::Module module_; auth::Module module_;
Config config_;
}; };
} // namespace memgraph::auth } // namespace memgraph::auth

View File

@@ -1,4 +1,4 @@
// Copyright 2024 Memgraph Ltd. // Copyright 2023 Memgraph Ltd.
// //
// Licensed as a Memgraph Enterprise file under the Memgraph Enterprise // Licensed as a Memgraph Enterprise file under the Memgraph Enterprise
// License (the "License"); by using this file, you agree to be bound by the terms of the License, and you may not use // License (the "License"); by using this file, you agree to be bound by the terms of the License, and you may not use
@@ -22,14 +22,10 @@
namespace { namespace {
using namespace std::literals; using namespace std::literals;
inline constexpr std::array password_encryption_mappings{
constexpr auto kHashAlgo = "hash_algo"; std::pair{"bcrypt"sv, memgraph::auth::PasswordEncryptionAlgorithm::BCRYPT},
constexpr auto kPasswordHash = "password_hash"; std::pair{"sha256"sv, memgraph::auth::PasswordEncryptionAlgorithm::SHA256},
std::pair{"sha256-multiple"sv, memgraph::auth::PasswordEncryptionAlgorithm::SHA256_MULTIPLE}};
inline constexpr std::array password_hash_mappings{
std::pair{"bcrypt"sv, memgraph::auth::PasswordHashAlgorithm::BCRYPT},
std::pair{"sha256"sv, memgraph::auth::PasswordHashAlgorithm::SHA256},
std::pair{"sha256-multiple"sv, memgraph::auth::PasswordHashAlgorithm::SHA256_MULTIPLE}};
inline constexpr uint64_t ONE_SHA_ITERATION = 1; inline constexpr uint64_t ONE_SHA_ITERATION = 1;
inline constexpr uint64_t MULTIPLE_SHA_ITERATIONS = 1024; inline constexpr uint64_t MULTIPLE_SHA_ITERATIONS = 1024;
@@ -39,7 +35,7 @@ inline constexpr uint64_t MULTIPLE_SHA_ITERATIONS = 1024;
DEFINE_VALIDATED_string(password_encryption_algorithm, "bcrypt", DEFINE_VALIDATED_string(password_encryption_algorithm, "bcrypt",
"The password encryption algorithm used for authentication.", { "The password encryption algorithm used for authentication.", {
if (const auto result = if (const auto result =
memgraph::utils::IsValidEnumValueString(value, password_hash_mappings); memgraph::utils::IsValidEnumValueString(value, password_encryption_mappings);
result.HasError()) { result.HasError()) {
const auto error = result.GetError(); const auto error = result.GetError();
switch (error) { switch (error) {
@@ -49,7 +45,7 @@ DEFINE_VALIDATED_string(password_encryption_algorithm, "bcrypt",
} }
case memgraph::utils::ValidationError::InvalidValue: { case memgraph::utils::ValidationError::InvalidValue: {
std::cout << "Invalid value for password encryption algorithm. Allowed values: " std::cout << "Invalid value for password encryption algorithm. Allowed values: "
<< memgraph::utils::GetAllowedEnumValuesString(password_hash_mappings) << memgraph::utils::GetAllowedEnumValuesString(password_encryption_mappings)
<< std::endl; << std::endl;
break; break;
} }
@@ -62,7 +58,7 @@ DEFINE_VALIDATED_string(password_encryption_algorithm, "bcrypt",
namespace memgraph::auth { namespace memgraph::auth {
namespace BCrypt { namespace BCrypt {
std::string HashPassword(const std::string &password) { std::string EncryptPassword(const std::string &password) {
char salt[BCRYPT_HASHSIZE]; char salt[BCRYPT_HASHSIZE];
char hash[BCRYPT_HASHSIZE]; char hash[BCRYPT_HASHSIZE];
@@ -90,30 +86,16 @@ bool VerifyPassword(const std::string &password, const std::string &hash) {
} // namespace BCrypt } // namespace BCrypt
namespace SHA { namespace SHA {
namespace {
constexpr auto SHA_LENGTH = 64U;
constexpr auto SALT_SIZE = 16U;
constexpr auto SALT_SIZE_DURABLE = SALT_SIZE * 2;
#if OPENSSL_VERSION_MAJOR >= 3 #if OPENSSL_VERSION_MAJOR >= 3
std::string HashPasswordOpenSSL3(std::string_view password, const uint64_t number_of_iterations, std::string EncryptPasswordOpenSSL3(const std::string &password, const uint64_t number_of_iterations) {
std::string_view salt) {
unsigned char hash[SHA256_DIGEST_LENGTH]; unsigned char hash[SHA256_DIGEST_LENGTH];
EVP_MD_CTX *ctx = EVP_MD_CTX_new(); EVP_MD_CTX *ctx = EVP_MD_CTX_new();
EVP_MD *md = EVP_MD_fetch(nullptr, "SHA2-256", nullptr); EVP_MD *md = EVP_MD_fetch(nullptr, "SHA2-256", nullptr);
EVP_DigestInit_ex(ctx, md, nullptr); EVP_DigestInit_ex(ctx, md, nullptr);
if (!salt.empty()) {
DMG_ASSERT(salt.size() == SALT_SIZE);
EVP_DigestUpdate(ctx, salt.data(), salt.size());
}
for (auto i = 0; i < number_of_iterations; i++) { for (auto i = 0; i < number_of_iterations; i++) {
EVP_DigestUpdate(ctx, password.data(), password.size()); EVP_DigestUpdate(ctx, password.c_str(), password.size());
} }
EVP_DigestFinal_ex(ctx, hash, nullptr); EVP_DigestFinal_ex(ctx, hash, nullptr);
@@ -121,11 +103,6 @@ std::string HashPasswordOpenSSL3(std::string_view password, const uint64_t numbe
EVP_MD_CTX_free(ctx); EVP_MD_CTX_free(ctx);
std::stringstream result_stream; std::stringstream result_stream;
for (unsigned char salt_char : salt) {
result_stream << std::hex << std::setw(2) << std::setfill('0') << (((unsigned int)salt_char) & 0xFFU);
}
for (auto hash_char : hash) { for (auto hash_char : hash) {
result_stream << std::hex << std::setw(2) << std::setfill('0') << (int)hash_char; result_stream << std::hex << std::setw(2) << std::setfill('0') << (int)hash_char;
} }
@@ -133,27 +110,17 @@ std::string HashPasswordOpenSSL3(std::string_view password, const uint64_t numbe
return result_stream.str(); return result_stream.str();
} }
#else #else
std::string HashPasswordOpenSSL1_1(std::string_view password, const uint64_t number_of_iterations, std::string EncryptPasswordOpenSSL1_1(const std::string &password, const uint64_t number_of_iterations) {
std::string_view salt) {
unsigned char hash[SHA256_DIGEST_LENGTH]; unsigned char hash[SHA256_DIGEST_LENGTH];
SHA256_CTX sha256; SHA256_CTX sha256;
SHA256_Init(&sha256); SHA256_Init(&sha256);
if (!salt.empty()) {
DMG_ASSERT(salt.size() == SALT_SIZE);
SHA256_Update(&sha256, salt.data(), salt.size());
}
for (auto i = 0; i < number_of_iterations; i++) { for (auto i = 0; i < number_of_iterations; i++) {
SHA256_Update(&sha256, password.data(), password.size()); SHA256_Update(&sha256, password.c_str(), password.size());
} }
SHA256_Final(hash, &sha256); SHA256_Final(hash, &sha256);
std::stringstream ss; std::stringstream ss;
for (unsigned char salt_char : salt) {
ss << std::hex << std::setw(2) << std::setfill('0') << (((unsigned int)salt_char) & 0xFFU);
}
for (auto hash_char : hash) { for (auto hash_char : hash) {
ss << std::hex << std::setw(2) << std::setfill('0') << (int)hash_char; ss << std::hex << std::setw(2) << std::setfill('0') << (int)hash_char;
} }
@@ -162,144 +129,55 @@ std::string HashPasswordOpenSSL1_1(std::string_view password, const uint64_t num
} }
#endif #endif
std::string HashPassword(std::string_view password, const uint64_t number_of_iterations, std::string_view salt) { std::string EncryptPassword(const std::string &password, const uint64_t number_of_iterations) {
#if OPENSSL_VERSION_MAJOR >= 3 #if OPENSSL_VERSION_MAJOR >= 3
return HashPasswordOpenSSL3(password, number_of_iterations, salt); return EncryptPasswordOpenSSL3(password, number_of_iterations);
#else #else
return HashPasswordOpenSSL1_1(password, number_of_iterations, salt); return EncryptPasswordOpenSSL1_1(password, number_of_iterations);
#endif #endif
} }
auto ExtractSalt(std::string_view salt_durable) -> std::array<char, SALT_SIZE> { bool VerifyPassword(const std::string &password, const std::string &hash, const uint64_t number_of_iterations) {
static_assert(SALT_SIZE_DURABLE % 2 == 0); auto password_hash = EncryptPassword(password, number_of_iterations);
static_assert(SALT_SIZE_DURABLE / 2 == SALT_SIZE);
MG_ASSERT(salt_durable.size() == SALT_SIZE_DURABLE);
auto const *b = salt_durable.cbegin();
auto const *const e = salt_durable.cend();
auto salt = std::array<char, SALT_SIZE>{};
auto *inserter = salt.begin();
auto const toval = [](char a) -> uint8_t {
if ('0' <= a && a <= '9') {
return a - '0';
}
if ('a' <= a && a <= 'f') {
return 10 + (a - 'a');
}
MG_ASSERT(false, "Currupt hash, can't extract salt");
__builtin_unreachable();
};
for (; b != e; b += 2, ++inserter) {
*inserter = static_cast<char>(static_cast<uint8_t>(toval(b[0]) << 4U) | toval(b[1]));
}
return salt;
}
bool IsSalted(std::string_view hash) { return hash.size() == SHA_LENGTH + SALT_SIZE_DURABLE; }
bool VerifyPassword(std::string_view password, std::string_view hash, const uint64_t number_of_iterations) {
auto password_hash = std::invoke([&] {
if (hash.size() == SHA_LENGTH) [[unlikely]] {
// Just SHA256
return HashPassword(password, number_of_iterations, {});
} else {
// SHA256 + SALT
MG_ASSERT(IsSalted(hash));
auto const salt_durable = std::string_view{hash.data(), SALT_SIZE_DURABLE};
std::array<char, SALT_SIZE> salt = ExtractSalt(salt_durable);
return HashPassword(password, number_of_iterations, {salt.data(), salt.size()});
}
});
return password_hash == hash; return password_hash == hash;
} }
} // namespace
} // namespace SHA } // namespace SHA
HashedPassword HashPassword(const std::string &password, std::optional<PasswordHashAlgorithm> override_algo) { bool VerifyPassword(const std::string &password, const std::string &hash) {
auto const hash_algo = override_algo.value_or(CurrentHashAlgorithm()); const auto password_encryption_algorithm = utils::StringToEnum<PasswordEncryptionAlgorithm>(
auto password_hash = std::invoke([&] { FLAGS_password_encryption_algorithm, password_encryption_mappings);
switch (hash_algo) {
case PasswordHashAlgorithm::BCRYPT: {
return BCrypt::HashPassword(password);
}
case PasswordHashAlgorithm::SHA256:
case PasswordHashAlgorithm::SHA256_MULTIPLE: {
auto gen = std::mt19937(std::random_device{}());
auto salt = std::array<char, SHA::SALT_SIZE>{};
auto dis = std::uniform_int_distribution<unsigned char>(0, 255);
std::generate(salt.begin(), salt.end(), [&]() { return dis(gen); });
auto iterations = (hash_algo == PasswordHashAlgorithm::SHA256) ? ONE_SHA_ITERATION : MULTIPLE_SHA_ITERATIONS;
return SHA::HashPassword(password, iterations, {salt.data(), salt.size()});
}
}
});
return HashedPassword{hash_algo, std::move(password_hash)};
};
namespace { if (!password_encryption_algorithm.has_value()) {
throw AuthException("Invalid password encryption flag '{}'!", FLAGS_password_encryption_algorithm);
auto InternalParseHashAlgorithm(std::string_view algo) -> PasswordHashAlgorithm {
auto maybe_parsed = utils::StringToEnum<PasswordHashAlgorithm>(algo, password_hash_mappings);
if (!maybe_parsed) {
throw AuthException("Invalid password encryption '{}'!", algo);
} }
return *maybe_parsed;
}
PasswordHashAlgorithm &InternalCurrentHashAlgorithm() { switch (password_encryption_algorithm.value()) {
static auto current = PasswordHashAlgorithm::BCRYPT; case PasswordEncryptionAlgorithm::BCRYPT:
static std::once_flag flag; return BCrypt::VerifyPassword(password, hash);
std::call_once(flag, [] { current = InternalParseHashAlgorithm(FLAGS_password_encryption_algorithm); }); case PasswordEncryptionAlgorithm::SHA256:
return current; return SHA::VerifyPassword(password, hash, ONE_SHA_ITERATION);
} case PasswordEncryptionAlgorithm::SHA256_MULTIPLE:
} // namespace return SHA::VerifyPassword(password, hash, MULTIPLE_SHA_ITERATIONS);
auto CurrentHashAlgorithm() -> PasswordHashAlgorithm { return InternalCurrentHashAlgorithm(); }
void SetHashAlgorithm(std::string_view algo) {
auto &current = InternalCurrentHashAlgorithm();
current = InternalParseHashAlgorithm(algo);
}
auto AsString(PasswordHashAlgorithm hash_algo) -> std::string_view {
return *utils::EnumToString<PasswordHashAlgorithm>(hash_algo, password_hash_mappings);
}
bool HashedPassword::VerifyPassword(const std::string &password) {
switch (hash_algo) {
case PasswordHashAlgorithm::BCRYPT:
return BCrypt::VerifyPassword(password, password_hash);
case PasswordHashAlgorithm::SHA256:
return SHA::VerifyPassword(password, password_hash, ONE_SHA_ITERATION);
case PasswordHashAlgorithm::SHA256_MULTIPLE:
return SHA::VerifyPassword(password, password_hash, MULTIPLE_SHA_ITERATIONS);
} }
throw AuthException("Invalid password encryption flag '{}'!", FLAGS_password_encryption_algorithm);
} }
void to_json(nlohmann::json &j, const HashedPassword &p) { std::string EncryptPassword(const std::string &password) {
j = nlohmann::json{{kHashAlgo, p.hash_algo}, {kPasswordHash, p.password_hash}}; const auto password_encryption_algorithm = utils::StringToEnum<PasswordEncryptionAlgorithm>(
} FLAGS_password_encryption_algorithm, password_encryption_mappings);
void from_json(const nlohmann::json &j, HashedPassword &p) { if (!password_encryption_algorithm.has_value()) {
// NOLINTNEXTLINE(cppcoreguidelines-init-variables) throw AuthException("Invalid password encryption flag '{}'!", FLAGS_password_encryption_algorithm);
PasswordHashAlgorithm hash_algo; }
j.at(kHashAlgo).get_to(hash_algo);
auto password_hash = j.value(kPasswordHash, std::string());
p = HashedPassword{hash_algo, std::move(password_hash)};
}
bool HashedPassword::IsSalted() const { switch (password_encryption_algorithm.value()) {
switch (hash_algo) { case PasswordEncryptionAlgorithm::BCRYPT:
case PasswordHashAlgorithm::BCRYPT: return BCrypt::EncryptPassword(password);
return true; case PasswordEncryptionAlgorithm::SHA256:
case PasswordHashAlgorithm::SHA256: return SHA::EncryptPassword(password, ONE_SHA_ITERATION);
case PasswordHashAlgorithm::SHA256_MULTIPLE: case PasswordEncryptionAlgorithm::SHA256_MULTIPLE:
return SHA::IsSalted(password_hash); return SHA::EncryptPassword(password, MULTIPLE_SHA_ITERATIONS);
} }
} }

View File

@@ -1,4 +1,4 @@
// Copyright 2024 Memgraph Ltd. // Copyright 2023 Memgraph Ltd.
// //
// Licensed as a Memgraph Enterprise file under the Memgraph Enterprise // Licensed as a Memgraph Enterprise file under the Memgraph Enterprise
// License (the "License"); by using this file, you agree to be bound by the terms of the License, and you may not use // License (the "License"); by using this file, you agree to be bound by the terms of the License, and you may not use
@@ -8,45 +8,14 @@
#pragma once #pragma once
#include <json/json.hpp>
#include <optional>
#include <string> #include <string>
namespace memgraph::auth { namespace memgraph::auth {
/// Need to be stable, auth durability depends on this enum class PasswordEncryptionAlgorithm : uint8_t { BCRYPT, SHA256, SHA256_MULTIPLE };
enum class PasswordHashAlgorithm : uint8_t { BCRYPT = 0, SHA256 = 1, SHA256_MULTIPLE = 2 };
void SetHashAlgorithm(std::string_view algo); /// @throw AuthException if unable to encrypt the password.
std::string EncryptPassword(const std::string &password);
auto CurrentHashAlgorithm() -> PasswordHashAlgorithm; /// @throw AuthException if unable to verify the password.
bool VerifyPassword(const std::string &password, const std::string &hash);
auto AsString(PasswordHashAlgorithm hash_algo) -> std::string_view;
struct HashedPassword {
HashedPassword() = default;
HashedPassword(PasswordHashAlgorithm hash_algo, std::string password_hash)
: hash_algo{hash_algo}, password_hash{std::move(password_hash)} {}
HashedPassword(HashedPassword const &) = default;
HashedPassword(HashedPassword &&) = default;
HashedPassword &operator=(HashedPassword const &) = default;
HashedPassword &operator=(HashedPassword &&) = default;
friend bool operator==(HashedPassword const &, HashedPassword const &) = default;
bool VerifyPassword(const std::string &password);
bool IsSalted() const;
auto HashAlgo() const -> PasswordHashAlgorithm { return hash_algo; }
friend void to_json(nlohmann::json &j, const HashedPassword &p);
friend void from_json(const nlohmann::json &j, HashedPassword &p);
private:
PasswordHashAlgorithm hash_algo{PasswordHashAlgorithm::BCRYPT};
std::string password_hash{};
};
/// @throw AuthException if unable to hash the password.
HashedPassword HashPassword(const std::string &password, std::optional<PasswordHashAlgorithm> override_algo = {});
} // namespace memgraph::auth } // namespace memgraph::auth

View File

@@ -1,4 +1,4 @@
// Copyright 2023 Memgraph Ltd. // Copyright 2022 Memgraph Ltd.
// //
// Use of this software is governed by the Business Source License // Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source // included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
@@ -21,6 +21,5 @@ namespace memgraph::auth {
class AuthException : public utils::BasicException { class AuthException : public utils::BasicException {
public: public:
using utils::BasicException::BasicException; using utils::BasicException::BasicException;
SPECIALIZE_GET_EXCEPTION_NAME(AuthException)
}; };
} // namespace memgraph::auth } // namespace memgraph::auth

View File

@@ -1,4 +1,4 @@
// Copyright 2024 Memgraph Ltd. // Copyright 2023 Memgraph Ltd.
// //
// Licensed as a Memgraph Enterprise file under the Memgraph Enterprise // Licensed as a Memgraph Enterprise file under the Memgraph Enterprise
// License (the "License"); by using this file, you agree to be bound by the terms of the License, and you may not use // License (the "License"); by using this file, you agree to be bound by the terms of the License, and you may not use
@@ -9,7 +9,7 @@
#include "auth/models.hpp" #include "auth/models.hpp"
#include <cstdint> #include <cstdint>
#include <utility> #include <regex>
#include <gflags/gflags.h> #include <gflags/gflags.h>
@@ -20,26 +20,22 @@
#include "query/constants.hpp" #include "query/constants.hpp"
#include "spdlog/spdlog.h" #include "spdlog/spdlog.h"
#include "utils/cast.hpp" #include "utils/cast.hpp"
#include "utils/logging.hpp"
#include "utils/settings.hpp"
#include "utils/string.hpp" #include "utils/string.hpp"
// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)
DEFINE_bool(auth_password_permit_null, true, "Set to false to disable null passwords.");
inline constexpr std::string_view default_password_regex = ".+";
// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)
DEFINE_string(auth_password_strength_regex, default_password_regex.data(),
"The regular expression that should be used to match the entire "
"entered password to ensure its strength.");
namespace memgraph::auth { namespace memgraph::auth {
namespace { namespace {
constexpr auto kRoleName = "rolename";
constexpr auto kPermissions = "permissions";
constexpr auto kGrants = "grants";
constexpr auto kDenies = "denies";
constexpr auto kUsername = "username";
constexpr auto kPasswordHash = "password_hash";
#ifdef MG_ENTERPRISE
constexpr auto kGlobalPermission = "global_permission";
constexpr auto kFineGrainedAccessHandler = "fine_grained_access_handler";
constexpr auto kAllowAll = "allow_all";
constexpr auto kDefault = "default";
constexpr auto kDatabases = "databases";
#endif
// Constant list of all available permissions. // Constant list of all available permissions.
const std::vector<Permission> kPermissionsAll = {Permission::MATCH, const std::vector<Permission> kPermissionsAll = {Permission::MATCH,
Permission::CREATE, Permission::CREATE,
@@ -65,8 +61,7 @@ const std::vector<Permission> kPermissionsAll = {Permission::MATCH,
Permission::TRANSACTION_MANAGEMENT, Permission::TRANSACTION_MANAGEMENT,
Permission::STORAGE_MODE, Permission::STORAGE_MODE,
Permission::MULTI_DATABASE_EDIT, Permission::MULTI_DATABASE_EDIT,
Permission::MULTI_DATABASE_USE, Permission::MULTI_DATABASE_USE};
Permission::COORDINATOR};
} // namespace } // namespace
@@ -122,8 +117,6 @@ std::string PermissionToString(Permission permission) {
return "MULTI_DATABASE_EDIT"; return "MULTI_DATABASE_EDIT";
case Permission::MULTI_DATABASE_USE: case Permission::MULTI_DATABASE_USE:
return "MULTI_DATABASE_USE"; return "MULTI_DATABASE_USE";
case Permission::COORDINATOR:
return "COORDINATOR";
} }
} }
@@ -248,9 +241,8 @@ std::vector<Permission> Permissions::GetDenies() const {
nlohmann::json Permissions::Serialize() const { nlohmann::json Permissions::Serialize() const {
nlohmann::json data = nlohmann::json::object(); nlohmann::json data = nlohmann::json::object();
data["grants"] = grants_;
data[kGrants] = grants_; data["denies"] = denies_;
data[kDenies] = denies_;
return data; return data;
} }
@@ -258,10 +250,10 @@ Permissions Permissions::Deserialize(const nlohmann::json &data) {
if (!data.is_object()) { if (!data.is_object()) {
throw AuthException("Couldn't load permissions data!"); throw AuthException("Couldn't load permissions data!");
} }
if (!data[kGrants].is_number_unsigned() || !data[kDenies].is_number_unsigned()) { if (!data["grants"].is_number_unsigned() || !data["denies"].is_number_unsigned()) {
throw AuthException("Couldn't load permissions data!"); throw AuthException("Couldn't load permissions data!");
} }
return Permissions{data[kGrants], data[kDenies]}; return Permissions{data["grants"], data["denies"]};
} }
uint64_t Permissions::grants() const { return grants_; } uint64_t Permissions::grants() const { return grants_; }
@@ -323,8 +315,8 @@ nlohmann::json FineGrainedAccessPermissions::Serialize() const {
return {}; return {};
} }
nlohmann::json data = nlohmann::json::object(); nlohmann::json data = nlohmann::json::object();
data[kPermissions] = permissions_; data["permissions"] = permissions_;
data[kGlobalPermission] = global_permission_.has_value() ? global_permission_.value() : -1; data["global_permission"] = global_permission_.has_value() ? global_permission_.value() : -1;
return data; return data;
} }
@@ -337,13 +329,13 @@ FineGrainedAccessPermissions FineGrainedAccessPermissions::Deserialize(const nlo
} }
std::optional<uint64_t> global_permission; std::optional<uint64_t> global_permission;
if (data[kGlobalPermission].empty() || data[kGlobalPermission] == -1) { if (data["global_permission"].empty() || data["global_permission"] == -1) {
global_permission = std::nullopt; global_permission = std::nullopt;
} else { } else {
global_permission = data[kGlobalPermission]; global_permission = data["global_permission"];
} }
return FineGrainedAccessPermissions(data[kPermissions], global_permission); return FineGrainedAccessPermissions(data["permissions"], global_permission);
} }
const std::unordered_map<std::string, uint64_t> &FineGrainedAccessPermissions::GetPermissions() const { const std::unordered_map<std::string, uint64_t> &FineGrainedAccessPermissions::GetPermissions() const {
@@ -449,13 +441,13 @@ const FineGrainedAccessPermissions &Role::GetFineGrainedAccessEdgeTypePermission
nlohmann::json Role::Serialize() const { nlohmann::json Role::Serialize() const {
nlohmann::json data = nlohmann::json::object(); nlohmann::json data = nlohmann::json::object();
data[kRoleName] = rolename_; data["rolename"] = rolename_;
data[kPermissions] = permissions_.Serialize(); data["permissions"] = permissions_.Serialize();
#ifdef MG_ENTERPRISE #ifdef MG_ENTERPRISE
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) { if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
data[kFineGrainedAccessHandler] = fine_grained_access_handler_.Serialize(); data["fine_grained_access_handler"] = fine_grained_access_handler_.Serialize();
} else { } else {
data[kFineGrainedAccessHandler] = {}; data["fine_grained_access_handler"] = {};
} }
#endif #endif
return data; return data;
@@ -465,21 +457,21 @@ Role Role::Deserialize(const nlohmann::json &data) {
if (!data.is_object()) { if (!data.is_object()) {
throw AuthException("Couldn't load role data!"); throw AuthException("Couldn't load role data!");
} }
if (!data[kRoleName].is_string() || !data[kPermissions].is_object()) { if (!data["rolename"].is_string() || !data["permissions"].is_object()) {
throw AuthException("Couldn't load role data!"); throw AuthException("Couldn't load role data!");
} }
auto permissions = Permissions::Deserialize(data[kPermissions]); auto permissions = Permissions::Deserialize(data["permissions"]);
#ifdef MG_ENTERPRISE #ifdef MG_ENTERPRISE
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) { if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
FineGrainedAccessHandler fine_grained_access_handler; FineGrainedAccessHandler fine_grained_access_handler;
// We can have an empty fine_grained if the user was created without a valid license // We can have an empty fine_grained if the user was created without a valid license
if (data[kFineGrainedAccessHandler].is_object()) { if (data["fine_grained_access_handler"].is_object()) {
fine_grained_access_handler = FineGrainedAccessHandler::Deserialize(data[kFineGrainedAccessHandler]); fine_grained_access_handler = FineGrainedAccessHandler::Deserialize(data["fine_grained_access_handler"]);
} }
return {data[kRoleName], permissions, std::move(fine_grained_access_handler)}; return {data["rolename"], permissions, std::move(fine_grained_access_handler)};
} }
#endif #endif
return {data[kRoleName], permissions}; return {data["rolename"], permissions};
} }
bool operator==(const Role &first, const Role &second) { bool operator==(const Role &first, const Role &second) {
@@ -493,13 +485,13 @@ bool operator==(const Role &first, const Role &second) {
} }
#ifdef MG_ENTERPRISE #ifdef MG_ENTERPRISE
void Databases::Add(std::string_view db) { void Databases::Add(const std::string &db) {
if (allow_all_) { if (allow_all_) {
grants_dbs_.clear(); grants_dbs_.clear();
allow_all_ = false; allow_all_ = false;
} }
grants_dbs_.emplace(db); grants_dbs_.emplace(db);
denies_dbs_.erase(std::string{db}); // TODO: C++23 use transparent key compare denies_dbs_.erase(db);
} }
void Databases::Remove(const std::string &db) { void Databases::Remove(const std::string &db) {
@@ -530,13 +522,13 @@ void Databases::DenyAll() {
denies_dbs_.clear(); denies_dbs_.clear();
} }
bool Databases::SetDefault(std::string_view db) { bool Databases::SetDefault(const std::string &db) {
if (!Contains(db)) return false; if (!Contains(db)) return false;
default_db_ = db; default_db_ = db;
return true; return true;
} }
[[nodiscard]] bool Databases::Contains(std::string_view db) const { [[nodiscard]] bool Databases::Contains(const std::string &db) const {
return !denies_dbs_.contains(db) && (allow_all_ || grants_dbs_.contains(db)); return !denies_dbs_.contains(db) && (allow_all_ || grants_dbs_.contains(db));
} }
@@ -549,10 +541,10 @@ const std::string &Databases::GetDefault() const {
nlohmann::json Databases::Serialize() const { nlohmann::json Databases::Serialize() const {
nlohmann::json data = nlohmann::json::object(); nlohmann::json data = nlohmann::json::object();
data[kGrants] = grants_dbs_; data["grants"] = grants_dbs_;
data[kDenies] = denies_dbs_; data["denies"] = denies_dbs_;
data[kAllowAll] = allow_all_; data["allow_all"] = allow_all_;
data[kDefault] = default_db_; data["default"] = default_db_;
return data; return data;
} }
@@ -560,41 +552,63 @@ Databases Databases::Deserialize(const nlohmann::json &data) {
if (!data.is_object()) { if (!data.is_object()) {
throw AuthException("Couldn't load database data!"); throw AuthException("Couldn't load database data!");
} }
if (!data[kGrants].is_structured() || !data[kDenies].is_structured() || !data[kAllowAll].is_boolean() || if (!data["grants"].is_structured() || !data["denies"].is_structured() || !data["allow_all"].is_boolean() ||
!data[kDefault].is_string()) { !data["default"].is_string()) {
throw AuthException("Couldn't load database data!"); throw AuthException("Couldn't load database data!");
} }
return {data[kAllowAll], data[kGrants], data[kDenies], data[kDefault]}; return {data["allow_all"], data["grants"], data["denies"], data["default"]};
} }
#endif #endif
User::User() = default; User::User() {}
User::User(const std::string &username) : username_(utils::ToLowerCase(username)) {} User::User(const std::string &username) : username_(utils::ToLowerCase(username)) {}
User::User(const std::string &username, std::optional<HashedPassword> password_hash, const Permissions &permissions) User::User(const std::string &username, const std::string &password_hash, const Permissions &permissions)
: username_(utils::ToLowerCase(username)), password_hash_(std::move(password_hash)), permissions_(permissions) {} : username_(utils::ToLowerCase(username)), password_hash_(password_hash), permissions_(permissions) {}
#ifdef MG_ENTERPRISE #ifdef MG_ENTERPRISE
User::User(const std::string &username, std::optional<HashedPassword> password_hash, const Permissions &permissions, User::User(const std::string &username, const std::string &password_hash, const Permissions &permissions,
FineGrainedAccessHandler fine_grained_access_handler, Databases db_access) FineGrainedAccessHandler fine_grained_access_handler, Databases db_access)
: username_(utils::ToLowerCase(username)), : username_(utils::ToLowerCase(username)),
password_hash_(std::move(password_hash)), password_hash_(password_hash),
permissions_(permissions), permissions_(permissions),
fine_grained_access_handler_(std::move(fine_grained_access_handler)), fine_grained_access_handler_(std::move(fine_grained_access_handler)),
database_access_(std::move(db_access)) {} database_access_(db_access) {}
#endif #endif
bool User::CheckPassword(const std::string &password) { bool User::CheckPassword(const std::string &password) {
return password_hash_ ? password_hash_->VerifyPassword(password) : true; if (password_hash_.empty()) return true;
return VerifyPassword(password, password_hash_);
} }
void User::UpdatePassword(const std::optional<std::string> &password, void User::UpdatePassword(const std::optional<std::string> &password) {
std::optional<PasswordHashAlgorithm> algo_override) {
if (!password) { if (!password) {
password_hash_.reset(); if (!FLAGS_auth_password_permit_null) {
throw AuthException("Null passwords aren't permitted!");
}
password_hash_ = "";
return; return;
} }
password_hash_ = HashPassword(*password, algo_override);
if (FLAGS_auth_password_strength_regex != default_password_regex) {
if (const auto license_check_result = license::global_license_checker.IsEnterpriseValid(utils::global_settings);
license_check_result.HasError()) {
throw AuthException(
"Custom password regex is a Memgraph Enterprise feature. Please set the config "
"(\"--auth-password-strength-regex\") to its default value (\"{}\") or remove the flag.\n{}",
default_password_regex,
license::LicenseCheckErrorToString(license_check_result.GetError(), "password regex"));
}
}
std::regex re(FLAGS_auth_password_strength_regex);
if (!std::regex_match(*password, re)) {
throw AuthException(
"The user password doesn't conform to the required strength! Regex: "
"\"{}\"",
FLAGS_auth_password_strength_regex);
}
password_hash_ = EncryptPassword(*password);
} }
void User::SetRole(const Role &role) { role_.emplace(role); } void User::SetRole(const Role &role) { role_.emplace(role); }
@@ -611,49 +625,27 @@ Permissions User::GetPermissions() const {
#ifdef MG_ENTERPRISE #ifdef MG_ENTERPRISE
FineGrainedAccessPermissions User::GetFineGrainedAccessLabelPermissions() const { FineGrainedAccessPermissions User::GetFineGrainedAccessLabelPermissions() const {
return Merge(GetUserFineGrainedAccessLabelPermissions(), GetRoleFineGrainedAccessLabelPermissions());
}
FineGrainedAccessPermissions User::GetFineGrainedAccessEdgeTypePermissions() const {
return Merge(GetUserFineGrainedAccessEdgeTypePermissions(), GetRoleFineGrainedAccessEdgeTypePermissions());
}
FineGrainedAccessPermissions User::GetUserFineGrainedAccessEdgeTypePermissions() const {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) { if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return FineGrainedAccessPermissions{}; return FineGrainedAccessPermissions{};
} }
return fine_grained_access_handler_.edge_type_permissions(); if (role_) {
} return Merge(role()->fine_grained_access_handler().label_permissions(),
fine_grained_access_handler_.label_permissions());
FineGrainedAccessPermissions User::GetUserFineGrainedAccessLabelPermissions() const {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return FineGrainedAccessPermissions{};
} }
return fine_grained_access_handler_.label_permissions(); return fine_grained_access_handler_.label_permissions();
} }
FineGrainedAccessPermissions User::GetRoleFineGrainedAccessEdgeTypePermissions() const { FineGrainedAccessPermissions User::GetFineGrainedAccessEdgeTypePermissions() const {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) { if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return FineGrainedAccessPermissions{}; return FineGrainedAccessPermissions{};
} }
if (role_) { if (role_) {
return role()->fine_grained_access_handler().edge_type_permissions(); return Merge(role()->fine_grained_access_handler().edge_type_permissions(),
fine_grained_access_handler_.edge_type_permissions());
} }
return FineGrainedAccessPermissions{}; return fine_grained_access_handler_.edge_type_permissions();
}
FineGrainedAccessPermissions User::GetRoleFineGrainedAccessLabelPermissions() const {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return FineGrainedAccessPermissions{};
}
if (role_) {
return role()->fine_grained_access_handler().label_permissions();
}
return FineGrainedAccessPermissions{};
} }
#endif #endif
@@ -675,20 +667,16 @@ const Role *User::role() const {
nlohmann::json User::Serialize() const { nlohmann::json User::Serialize() const {
nlohmann::json data = nlohmann::json::object(); nlohmann::json data = nlohmann::json::object();
data[kUsername] = username_; data["username"] = username_;
if (password_hash_.has_value()) { data["password_hash"] = password_hash_;
data[kPasswordHash] = *password_hash_; data["permissions"] = permissions_.Serialize();
} else {
data[kPasswordHash] = nullptr;
}
data[kPermissions] = permissions_.Serialize();
#ifdef MG_ENTERPRISE #ifdef MG_ENTERPRISE
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) { if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
data[kFineGrainedAccessHandler] = fine_grained_access_handler_.Serialize(); data["fine_grained_access_handler"] = fine_grained_access_handler_.Serialize();
data[kDatabases] = database_access_.Serialize(); data["databases"] = database_access_.Serialize();
} else { } else {
data[kFineGrainedAccessHandler] = {}; data["fine_grained_access_handler"] = {};
data[kDatabases] = {}; data["databases"] = {};
} }
#endif #endif
// The role shouldn't be serialized here, it is stored as a foreign key. // The role shouldn't be serialized here, it is stored as a foreign key.
@@ -699,23 +687,15 @@ User User::Deserialize(const nlohmann::json &data) {
if (!data.is_object()) { if (!data.is_object()) {
throw AuthException("Couldn't load user data!"); throw AuthException("Couldn't load user data!");
} }
auto password_hash_json = data[kPasswordHash]; if (!data["username"].is_string() || !data["password_hash"].is_string() || !data["permissions"].is_object()) {
if (!data[kUsername].is_string() || !(password_hash_json.is_object() || password_hash_json.is_null()) ||
!data[kPermissions].is_object()) {
throw AuthException("Couldn't load user data!"); throw AuthException("Couldn't load user data!");
} }
auto permissions = Permissions::Deserialize(data["permissions"]);
std::optional<HashedPassword> password_hash{};
if (password_hash_json.is_object()) {
password_hash = password_hash_json.get<HashedPassword>();
}
auto permissions = Permissions::Deserialize(data[kPermissions]);
#ifdef MG_ENTERPRISE #ifdef MG_ENTERPRISE
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) { if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
Databases db_access; Databases db_access;
if (data[kDatabases].is_structured()) { if (data["databases"].is_structured()) {
db_access = Databases::Deserialize(data[kDatabases]); db_access = Databases::Deserialize(data["databases"]);
} else { } else {
// Back-compatibility // Back-compatibility
spdlog::warn("User without specified database access. Given access to the default database."); spdlog::warn("User without specified database access. Given access to the default database.");
@@ -724,13 +704,13 @@ User User::Deserialize(const nlohmann::json &data) {
} }
FineGrainedAccessHandler fine_grained_access_handler; FineGrainedAccessHandler fine_grained_access_handler;
// We can have an empty fine_grained if the user was created without a valid license // We can have an empty fine_grained if the user was created without a valid license
if (data[kFineGrainedAccessHandler].is_object()) { if (data["fine_grained_access_handler"].is_object()) {
fine_grained_access_handler = FineGrainedAccessHandler::Deserialize(data[kFineGrainedAccessHandler]); fine_grained_access_handler = FineGrainedAccessHandler::Deserialize(data["fine_grained_access_handler"]);
} }
return {data[kUsername], std::move(password_hash), permissions, std::move(fine_grained_access_handler), db_access}; return {data["username"], data["password_hash"], permissions, std::move(fine_grained_access_handler), db_access};
} }
#endif #endif
return {data[kUsername], std::move(password_hash), permissions}; return {data["username"], data["password_hash"], permissions};
} }
bool operator==(const User &first, const User &second) { bool operator==(const User &first, const User &second) {

View File

@@ -1,4 +1,4 @@
// Copyright 2024 Memgraph Ltd. // Copyright 2023 Memgraph Ltd.
// //
// Licensed as a Memgraph Enterprise file under the Memgraph Enterprise // Licensed as a Memgraph Enterprise file under the Memgraph Enterprise
// License (the "License"); by using this file, you agree to be bound by the terms of the License, and you may not use // License (the "License"); by using this file, you agree to be bound by the terms of the License, and you may not use
@@ -14,8 +14,6 @@
#include <unordered_map> #include <unordered_map>
#include <json/json.hpp> #include <json/json.hpp>
#include <utility>
#include "crypto.hpp"
#include "dbms/constants.hpp" #include "dbms/constants.hpp"
#include "utils/logging.hpp" #include "utils/logging.hpp"
@@ -49,7 +47,6 @@ enum class Permission : uint64_t {
STORAGE_MODE = 1U << 22U, STORAGE_MODE = 1U << 22U,
MULTI_DATABASE_EDIT = 1U << 23U, MULTI_DATABASE_EDIT = 1U << 23U,
MULTI_DATABASE_USE = 1U << 24U, MULTI_DATABASE_USE = 1U << 24U,
COORDINATOR = 1U << 25U,
}; };
// clang-format on // clang-format on
@@ -207,8 +204,6 @@ bool operator==(const FineGrainedAccessHandler &first, const FineGrainedAccessHa
class Role final { class Role final {
public: public:
Role() = default;
explicit Role(const std::string &rolename); explicit Role(const std::string &rolename);
Role(const std::string &rolename, const Permissions &permissions); Role(const std::string &rolename, const Permissions &permissions);
#ifdef MG_ENTERPRISE #ifdef MG_ENTERPRISE
@@ -250,7 +245,7 @@ bool operator==(const Role &first, const Role &second);
#ifdef MG_ENTERPRISE #ifdef MG_ENTERPRISE
class Databases final { class Databases final {
public: public:
Databases() : grants_dbs_{std::string{dbms::kDefaultDB}}, allow_all_(false), default_db_(dbms::kDefaultDB) {} Databases() : grants_dbs_({dbms::kDefaultDB}), allow_all_(false), default_db_(dbms::kDefaultDB) {}
Databases(const Databases &) = default; Databases(const Databases &) = default;
Databases &operator=(const Databases &) = default; Databases &operator=(const Databases &) = default;
@@ -263,7 +258,7 @@ class Databases final {
* *
* @param db name of the database to grant access to * @param db name of the database to grant access to
*/ */
void Add(std::string_view db); void Add(const std::string &db);
/** /**
* @brief Remove database to the list of granted access. * @brief Remove database to the list of granted access.
@@ -295,7 +290,7 @@ class Databases final {
/** /**
* @brief Set the default database. * @brief Set the default database.
*/ */
bool SetDefault(std::string_view db); bool SetDefault(const std::string &db);
/** /**
* @brief Checks if access is grated to the database. * @brief Checks if access is grated to the database.
@@ -303,11 +298,11 @@ class Databases final {
* @param db name of the database * @param db name of the database
* @return true if allow_all and not denied or granted * @return true if allow_all and not denied or granted
*/ */
bool Contains(std::string_view db) const; bool Contains(const std::string &db) const;
bool GetAllowAll() const { return allow_all_; } bool GetAllowAll() const { return allow_all_; }
const std::set<std::string, std::less<>> &GetGrants() const { return grants_dbs_; } const std::set<std::string> &GetGrants() const { return grants_dbs_; }
const std::set<std::string, std::less<>> &GetDenies() const { return denies_dbs_; } const std::set<std::string> &GetDenies() const { return denies_dbs_; }
const std::string &GetDefault() const; const std::string &GetDefault() const;
nlohmann::json Serialize() const; nlohmann::json Serialize() const;
@@ -315,17 +310,14 @@ class Databases final {
static Databases Deserialize(const nlohmann::json &data); static Databases Deserialize(const nlohmann::json &data);
private: private:
Databases(bool allow_all, std::set<std::string, std::less<>> grant, std::set<std::string, std::less<>> deny, Databases(bool allow_all, std::set<std::string> grant, std::set<std::string> deny,
std::string default_db = std::string{dbms::kDefaultDB}) const std::string &default_db = dbms::kDefaultDB)
: grants_dbs_(std::move(grant)), : grants_dbs_(grant), denies_dbs_(deny), allow_all_(allow_all), default_db_(default_db) {}
denies_dbs_(std::move(deny)),
allow_all_(allow_all),
default_db_(std::move(default_db)) {}
std::set<std::string, std::less<>> grants_dbs_; //!< set of databases with granted access std::set<std::string> grants_dbs_; //!< set of databases with granted access
std::set<std::string, std::less<>> denies_dbs_; //!< set of databases with denied access std::set<std::string> denies_dbs_; //!< set of databases with denied access
bool allow_all_; //!< flag to allow access to everything (denied overrides this) bool allow_all_; //!< flag to allow access to everything (denied overrides this)
std::string default_db_; //!< user's default database std::string default_db_; //!< user's default database
}; };
#endif #endif
@@ -335,9 +327,9 @@ class User final {
User(); User();
explicit User(const std::string &username); explicit User(const std::string &username);
User(const std::string &username, std::optional<HashedPassword> password_hash, const Permissions &permissions); User(const std::string &username, const std::string &password_hash, const Permissions &permissions);
#ifdef MG_ENTERPRISE #ifdef MG_ENTERPRISE
User(const std::string &username, std::optional<HashedPassword> password_hash, const Permissions &permissions, User(const std::string &username, const std::string &password_hash, const Permissions &permissions,
FineGrainedAccessHandler fine_grained_access_handler, Databases db_access = {}); FineGrainedAccessHandler fine_grained_access_handler, Databases db_access = {});
#endif #endif
User(const User &) = default; User(const User &) = default;
@@ -349,18 +341,8 @@ class User final {
/// @throw AuthException if unable to verify the password. /// @throw AuthException if unable to verify the password.
bool CheckPassword(const std::string &password); bool CheckPassword(const std::string &password);
bool UpgradeHash(const std::string password) {
if (!password_hash_) return false;
if (password_hash_->IsSalted()) return false;
auto const algo = password_hash_->HashAlgo();
UpdatePassword(password, algo);
return true;
}
/// @throw AuthException if unable to set the password. /// @throw AuthException if unable to set the password.
void UpdatePassword(const std::optional<std::string> &password = {}, void UpdatePassword(const std::optional<std::string> &password = std::nullopt);
std::optional<PasswordHashAlgorithm> algo_override = std::nullopt);
void SetRole(const Role &role); void SetRole(const Role &role);
@@ -371,10 +353,6 @@ class User final {
#ifdef MG_ENTERPRISE #ifdef MG_ENTERPRISE
FineGrainedAccessPermissions GetFineGrainedAccessLabelPermissions() const; FineGrainedAccessPermissions GetFineGrainedAccessLabelPermissions() const;
FineGrainedAccessPermissions GetFineGrainedAccessEdgeTypePermissions() const; FineGrainedAccessPermissions GetFineGrainedAccessEdgeTypePermissions() const;
FineGrainedAccessPermissions GetUserFineGrainedAccessLabelPermissions() const;
FineGrainedAccessPermissions GetUserFineGrainedAccessEdgeTypePermissions() const;
FineGrainedAccessPermissions GetRoleFineGrainedAccessLabelPermissions() const;
FineGrainedAccessPermissions GetRoleFineGrainedAccessEdgeTypePermissions() const;
const FineGrainedAccessHandler &fine_grained_access_handler() const; const FineGrainedAccessHandler &fine_grained_access_handler() const;
FineGrainedAccessHandler &fine_grained_access_handler(); FineGrainedAccessHandler &fine_grained_access_handler();
#endif #endif
@@ -399,7 +377,7 @@ class User final {
private: private:
std::string username_; std::string username_;
std::optional<HashedPassword> password_hash_; std::string password_hash_;
Permissions permissions_; Permissions permissions_;
#ifdef MG_ENTERPRISE #ifdef MG_ENTERPRISE
FineGrainedAccessHandler fine_grained_access_handler_; FineGrainedAccessHandler fine_grained_access_handler_;

View File

@@ -1,190 +0,0 @@
// Copyright 2024 Memgraph Ltd.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
// License, and you may not use this file except in compliance with the Business Source License.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#include "auth/replication_handlers.hpp"
#include "auth/auth.hpp"
#include "auth/rpc.hpp"
#include "license/license.hpp"
namespace memgraph::auth {
void LogWrongMain(const std::optional<utils::UUID> &current_main_uuid, const utils::UUID &main_req_id,
std::string_view rpc_req) {
spdlog::error(fmt::format("Received {} with main_id: {} != current_main_uuid: {}", rpc_req, std::string(main_req_id),
current_main_uuid.has_value() ? std::string(current_main_uuid.value()) : ""));
}
#ifdef MG_ENTERPRISE
void UpdateAuthDataHandler(memgraph::system::ReplicaHandlerAccessToState &system_state_access,
const std::optional<utils::UUID> &current_main_uuid, auth::SynchedAuth &auth,
slk::Reader *req_reader, slk::Builder *res_builder) {
replication::UpdateAuthDataReq req;
memgraph::slk::Load(&req, req_reader);
using memgraph::replication::UpdateAuthDataRes;
UpdateAuthDataRes res(false);
if (!current_main_uuid.has_value() || req.main_uuid != current_main_uuid) [[unlikely]] {
LogWrongMain(current_main_uuid, req.main_uuid, replication::UpdateAuthDataReq::kType.name);
memgraph::slk::Save(res, res_builder);
return;
}
// Note: No need to check epoch, recovery mechanism is done by a full uptodate snapshot
// of the set of databases. Hence no history exists to maintain regarding epoch change.
// If MAIN has changed we need to check this new group_timestamp is consistent with
// what we have so far.
if (req.expected_group_timestamp != system_state_access.LastCommitedTS()) {
spdlog::debug("UpdateAuthDataHandler: bad expected timestamp {},{}", req.expected_group_timestamp,
system_state_access.LastCommitedTS());
memgraph::slk::Save(res, res_builder);
return;
}
try {
// Update
if (req.user) auth->SaveUser(*req.user);
if (req.role) auth->SaveRole(*req.role);
// Success
system_state_access.SetLastCommitedTS(req.new_group_timestamp);
res = UpdateAuthDataRes(true);
spdlog::debug("UpdateAuthDataHandler: SUCCESS updated LCTS to {}", req.new_group_timestamp);
} catch (const auth::AuthException & /* not used */) {
// Failure
}
memgraph::slk::Save(res, res_builder);
}
void DropAuthDataHandler(memgraph::system::ReplicaHandlerAccessToState &system_state_access,
const std::optional<utils::UUID> &current_main_uuid, auth::SynchedAuth &auth,
slk::Reader *req_reader, slk::Builder *res_builder) {
replication::DropAuthDataReq req;
memgraph::slk::Load(&req, req_reader);
using memgraph::replication::DropAuthDataRes;
DropAuthDataRes res(false);
if (!current_main_uuid.has_value() || req.main_uuid != current_main_uuid) [[unlikely]] {
LogWrongMain(current_main_uuid, req.main_uuid, replication::DropAuthDataRes::kType.name);
memgraph::slk::Save(res, res_builder);
return;
}
// Note: No need to check epoch, recovery mechanism is done by a full uptodate snapshot
// of the set of databases. Hence no history exists to maintain regarding epoch change.
// If MAIN has changed we need to check this new group_timestamp is consistent with
// what we have so far.
if (req.expected_group_timestamp != system_state_access.LastCommitedTS()) {
spdlog::debug("DropAuthDataHandler: bad expected timestamp {},{}", req.expected_group_timestamp,
system_state_access.LastCommitedTS());
memgraph::slk::Save(res, res_builder);
return;
}
try {
// Remove
switch (req.type) {
case replication::DropAuthDataReq::DataType::USER:
auth->RemoveUser(req.name);
break;
case replication::DropAuthDataReq::DataType::ROLE:
auth->RemoveRole(req.name);
break;
}
// Success
system_state_access.SetLastCommitedTS(req.new_group_timestamp);
res = DropAuthDataRes(true);
spdlog::debug("DropAuthDataHandler: SUCCESS updated LCTS to {}", req.new_group_timestamp);
} catch (const auth::AuthException & /* not used */) {
// Failure
}
memgraph::slk::Save(res, res_builder);
}
bool SystemRecoveryHandler(auth::SynchedAuth &auth, auth::Auth::Config auth_config,
const std::vector<auth::User> &users, const std::vector<auth::Role> &roles) {
return auth.WithLock([&](auto &locked_auth) {
// Update config
locked_auth.SetConfig(std::move(auth_config));
// Get all current users
auto old_users = locked_auth.AllUsernames();
// Save incoming users
for (const auto &user : users) {
// Missing users
try {
locked_auth.SaveUser(user);
} catch (const auth::AuthException &) {
spdlog::debug("SystemRecoveryHandler: Failed to save user");
return false;
}
const auto it = std::find(old_users.begin(), old_users.end(), user.username());
if (it != old_users.end()) old_users.erase(it);
}
// Delete all the leftover users
for (const auto &user : old_users) {
if (!locked_auth.RemoveUser(user)) {
spdlog::debug("SystemRecoveryHandler: Failed to remove user \"{}\".", user);
return false;
}
}
// Roles are only supported with a license
if (license::global_license_checker.IsEnterpriseValidFast()) {
// Get all current roles
auto old_roles = locked_auth.AllRolenames();
// Save incoming users
for (const auto &role : roles) {
// Missing users
try {
locked_auth.SaveRole(role);
} catch (const auth::AuthException &) {
spdlog::debug("SystemRecoveryHandler: Failed to save user");
return false;
}
const auto it = std::find(old_roles.begin(), old_roles.end(), role.rolename());
if (it != old_roles.end()) old_roles.erase(it);
}
// Delete all the leftover users
for (const auto &role : old_roles) {
if (!locked_auth.RemoveRole(role)) {
spdlog::debug("SystemRecoveryHandler: Failed to remove user \"{}\".", role);
return false;
}
}
}
// Success
return true;
});
}
void Register(replication::RoleReplicaData const &data, system::ReplicaHandlerAccessToState &system_state_access,
auth::SynchedAuth &auth) {
// NOTE: Register even without license as the user could add a license at run-time
data.server->rpc_server_.Register<replication::UpdateAuthDataRpc>(
[&data, system_state_access, &auth](auto *req_reader, auto *res_builder) mutable {
spdlog::debug("Received UpdateAuthDataRpc");
UpdateAuthDataHandler(system_state_access, data.uuid_, auth, req_reader, res_builder);
});
data.server->rpc_server_.Register<replication::DropAuthDataRpc>(
[&data, system_state_access, &auth](auto *req_reader, auto *res_builder) mutable {
spdlog::debug("Received DropAuthDataRpc");
DropAuthDataHandler(system_state_access, data.uuid_, auth, req_reader, res_builder);
});
}
#endif
} // namespace memgraph::auth

View File

@@ -1,37 +0,0 @@
// Copyright 2024 Memgraph Ltd.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
// License, and you may not use this file except in compliance with the Business Source License.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#pragma once
#include "auth/auth.hpp"
#include "replication/state.hpp"
#include "slk/streams.hpp"
#include "system/state.hpp"
namespace memgraph::auth {
void LogWrongMain(const std::optional<utils::UUID> &current_main_uuid, const utils::UUID &main_req_id,
std::string_view rpc_req);
#ifdef MG_ENTERPRISE
void UpdateAuthDataHandler(system::ReplicaHandlerAccessToState &system_state_access,
const std::optional<utils::UUID> &current_main_uuid, auth::SynchedAuth &auth,
slk::Reader *req_reader, slk::Builder *res_builder);
void DropAuthDataHandler(system::ReplicaHandlerAccessToState &system_state_access,
const std::optional<utils::UUID> &current_main_uuid, auth::SynchedAuth &auth,
slk::Reader *req_reader, slk::Builder *res_builder);
bool SystemRecoveryHandler(auth::SynchedAuth &auth, auth::Auth::Config auth_config,
const std::vector<auth::User> &users, const std::vector<auth::Role> &roles);
void Register(replication::RoleReplicaData const &data, system::ReplicaHandlerAccessToState &system_state_access,
auth::SynchedAuth &auth);
#endif
} // namespace memgraph::auth

View File

@@ -1,182 +0,0 @@
// Copyright 2024 Memgraph Ltd.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
// License, and you may not use this file except in compliance with the Business Source License.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#include "auth/rpc.hpp"
#include <json/json.hpp>
#include "auth/auth.hpp"
#include "slk/serialization.hpp"
#include "slk/streams.hpp"
#include "utils/enum.hpp"
namespace memgraph::slk {
// Serialize code for auth::Role
void Save(const auth::Role &self, memgraph::slk::Builder *builder) {
memgraph::slk::Save(self.Serialize().dump(), builder);
}
namespace {
auth::Role LoadAuthRole(memgraph::slk::Reader *reader) {
std::string tmp;
memgraph::slk::Load(&tmp, reader);
const auto json = nlohmann::json::parse(tmp);
return memgraph::auth::Role::Deserialize(json);
}
} // namespace
// Deserialize code for auth::Role
void Load(auth::Role *self, memgraph::slk::Reader *reader) { *self = LoadAuthRole(reader); }
// Special case for optional<Role>
template <>
inline void Load<auth::Role>(std::optional<auth::Role> *obj, Reader *reader) {
bool exists = false;
Load(&exists, reader);
if (exists) {
obj->emplace(LoadAuthRole(reader));
} else {
*obj = std::nullopt;
}
}
// Serialize code for auth::User
void Save(const auth::User &self, memgraph::slk::Builder *builder) {
memgraph::slk::Save(self.Serialize().dump(), builder);
std::optional<auth::Role> role{};
if (const auto *role_ptr = self.role(); role_ptr) {
role.emplace(*role_ptr);
}
memgraph::slk::Save(role, builder);
}
// Deserialize code for auth::User
void Load(auth::User *self, memgraph::slk::Reader *reader) {
std::string tmp;
memgraph::slk::Load(&tmp, reader);
const auto json = nlohmann::json::parse(tmp);
*self = memgraph::auth::User::Deserialize(json);
std::optional<auth::Role> role{};
memgraph::slk::Load(&role, reader);
if (role)
self->SetRole(*role);
else
self->ClearRole();
}
// Serialize code for auth::Auth::Config
void Save(const auth::Auth::Config &self, memgraph::slk::Builder *builder) {
memgraph::slk::Save(self.name_regex_str, builder);
memgraph::slk::Save(self.password_regex_str, builder);
memgraph::slk::Save(self.password_permit_null, builder);
}
// Deserialize code for auth::Auth::Config
void Load(auth::Auth::Config *self, memgraph::slk::Reader *reader) {
std::string name_regex_str{};
std::string password_regex_str{};
bool password_permit_null{};
memgraph::slk::Load(&name_regex_str, reader);
memgraph::slk::Load(&password_regex_str, reader);
memgraph::slk::Load(&password_permit_null, reader);
*self = auth::Auth::Config{std::move(name_regex_str), std::move(password_regex_str), password_permit_null};
}
// Serialize code for UpdateAuthDataReq
void Save(const memgraph::replication::UpdateAuthDataReq &self, memgraph::slk::Builder *builder) {
memgraph::slk::Save(self.main_uuid, builder);
memgraph::slk::Save(self.epoch_id, builder);
memgraph::slk::Save(self.expected_group_timestamp, builder);
memgraph::slk::Save(self.new_group_timestamp, builder);
memgraph::slk::Save(self.user, builder);
memgraph::slk::Save(self.role, builder);
}
void Load(memgraph::replication::UpdateAuthDataReq *self, memgraph::slk::Reader *reader) {
memgraph::slk::Load(&self->main_uuid, reader);
memgraph::slk::Load(&self->epoch_id, reader);
memgraph::slk::Load(&self->expected_group_timestamp, reader);
memgraph::slk::Load(&self->new_group_timestamp, reader);
memgraph::slk::Load(&self->user, reader);
memgraph::slk::Load(&self->role, reader);
}
// Serialize code for UpdateAuthDataRes
void Save(const memgraph::replication::UpdateAuthDataRes &self, memgraph::slk::Builder *builder) {
memgraph::slk::Save(self.success, builder);
}
void Load(memgraph::replication::UpdateAuthDataRes *self, memgraph::slk::Reader *reader) {
memgraph::slk::Load(&self->success, reader);
}
// Serialize code for DropAuthDataReq
void Save(const memgraph::replication::DropAuthDataReq &self, memgraph::slk::Builder *builder) {
memgraph::slk::Save(self.main_uuid, builder);
memgraph::slk::Save(self.epoch_id, builder);
memgraph::slk::Save(self.expected_group_timestamp, builder);
memgraph::slk::Save(self.new_group_timestamp, builder);
memgraph::slk::Save(utils::EnumToNum<2, uint8_t>(self.type), builder);
memgraph::slk::Save(self.name, builder);
}
void Load(memgraph::replication::DropAuthDataReq *self, memgraph::slk::Reader *reader) {
memgraph::slk::Load(&self->main_uuid, reader);
memgraph::slk::Load(&self->epoch_id, reader);
memgraph::slk::Load(&self->expected_group_timestamp, reader);
memgraph::slk::Load(&self->new_group_timestamp, reader);
uint8_t type_tmp = 0;
memgraph::slk::Load(&type_tmp, reader);
if (!utils::NumToEnum<2>(type_tmp, self->type)) {
throw SlkReaderException("Unexpected result line:{}!", __LINE__);
}
memgraph::slk::Load(&self->name, reader);
}
// Serialize code for DropAuthDataRes
void Save(const memgraph::replication::DropAuthDataRes &self, memgraph::slk::Builder *builder) {
memgraph::slk::Save(self.success, builder);
}
void Load(memgraph::replication::DropAuthDataRes *self, memgraph::slk::Reader *reader) {
memgraph::slk::Load(&self->success, reader);
}
} // namespace memgraph::slk
namespace memgraph::replication {
constexpr utils::TypeInfo UpdateAuthDataReq::kType{utils::TypeId::REP_UPDATE_AUTH_DATA_REQ, "UpdateAuthDataReq",
nullptr};
constexpr utils::TypeInfo UpdateAuthDataRes::kType{utils::TypeId::REP_UPDATE_AUTH_DATA_RES, "UpdateAuthDataRes",
nullptr};
constexpr utils::TypeInfo DropAuthDataReq::kType{utils::TypeId::REP_DROP_AUTH_DATA_REQ, "DropAuthDataReq", nullptr};
constexpr utils::TypeInfo DropAuthDataRes::kType{utils::TypeId::REP_DROP_AUTH_DATA_RES, "DropAuthDataRes", nullptr};
void UpdateAuthDataReq::Save(const UpdateAuthDataReq &self, memgraph::slk::Builder *builder) {
memgraph::slk::Save(self, builder);
}
void UpdateAuthDataReq::Load(UpdateAuthDataReq *self, memgraph::slk::Reader *reader) {
memgraph::slk::Load(self, reader);
}
void UpdateAuthDataRes::Save(const UpdateAuthDataRes &self, memgraph::slk::Builder *builder) {
memgraph::slk::Save(self, builder);
}
void UpdateAuthDataRes::Load(UpdateAuthDataRes *self, memgraph::slk::Reader *reader) {
memgraph::slk::Load(self, reader);
}
void DropAuthDataReq::Save(const DropAuthDataReq &self, memgraph::slk::Builder *builder) {
memgraph::slk::Save(self, builder);
}
void DropAuthDataReq::Load(DropAuthDataReq *self, memgraph::slk::Reader *reader) { memgraph::slk::Load(self, reader); }
void DropAuthDataRes::Save(const DropAuthDataRes &self, memgraph::slk::Builder *builder) {
memgraph::slk::Save(self, builder);
}
void DropAuthDataRes::Load(DropAuthDataRes *self, memgraph::slk::Reader *reader) { memgraph::slk::Load(self, reader); }
} // namespace memgraph::replication

View File

@@ -1,127 +0,0 @@
// Copyright 2024 Memgraph Ltd.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
// License, and you may not use this file except in compliance with the Business Source License.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#pragma once
#include <optional>
#include "auth/auth.hpp"
#include "auth/models.hpp"
#include "rpc/messages.hpp"
#include "slk/streams.hpp"
namespace memgraph::replication {
struct UpdateAuthDataReq {
static const utils::TypeInfo kType;
static const utils::TypeInfo &GetTypeInfo() { return kType; }
static void Load(UpdateAuthDataReq *self, memgraph::slk::Reader *reader);
static void Save(const UpdateAuthDataReq &self, memgraph::slk::Builder *builder);
UpdateAuthDataReq() = default;
UpdateAuthDataReq(const utils::UUID &main_uuid, std::string epoch_id, uint64_t expected_ts, uint64_t new_ts,
auth::User user)
: main_uuid(main_uuid),
epoch_id{std::move(epoch_id)},
expected_group_timestamp{expected_ts},
new_group_timestamp{new_ts},
user{std::move(user)} {}
UpdateAuthDataReq(const utils::UUID &main_uuid, std::string epoch_id, uint64_t expected_ts, uint64_t new_ts,
auth::Role role)
: main_uuid(main_uuid),
epoch_id{std::move(epoch_id)},
expected_group_timestamp{expected_ts},
new_group_timestamp{new_ts},
role{std::move(role)} {}
utils::UUID main_uuid;
std::string epoch_id;
uint64_t expected_group_timestamp;
uint64_t new_group_timestamp;
std::optional<auth::User> user;
std::optional<auth::Role> role;
};
struct UpdateAuthDataRes {
static const utils::TypeInfo kType;
static const utils::TypeInfo &GetTypeInfo() { return kType; }
static void Load(UpdateAuthDataRes *self, memgraph::slk::Reader *reader);
static void Save(const UpdateAuthDataRes &self, memgraph::slk::Builder *builder);
UpdateAuthDataRes() = default;
explicit UpdateAuthDataRes(bool success) : success{success} {}
bool success;
};
using UpdateAuthDataRpc = rpc::RequestResponse<UpdateAuthDataReq, UpdateAuthDataRes>;
struct DropAuthDataReq {
static const utils::TypeInfo kType;
static const utils::TypeInfo &GetTypeInfo() { return kType; }
static void Load(DropAuthDataReq *self, memgraph::slk::Reader *reader);
static void Save(const DropAuthDataReq &self, memgraph::slk::Builder *builder);
DropAuthDataReq() = default;
enum class DataType { USER, ROLE };
DropAuthDataReq(const utils::UUID &main_uuid, std::string epoch_id, uint64_t expected_ts, uint64_t new_ts,
DataType type, std::string_view name)
: main_uuid(main_uuid),
epoch_id{std::move(epoch_id)},
expected_group_timestamp{expected_ts},
new_group_timestamp{new_ts},
type{type},
name{name} {}
utils::UUID main_uuid;
std::string epoch_id;
uint64_t expected_group_timestamp;
uint64_t new_group_timestamp;
DataType type;
std::string name;
};
struct DropAuthDataRes {
static const utils::TypeInfo kType;
static const utils::TypeInfo &GetTypeInfo() { return kType; }
static void Load(DropAuthDataRes *self, memgraph::slk::Reader *reader);
static void Save(const DropAuthDataRes &self, memgraph::slk::Builder *builder);
DropAuthDataRes() = default;
explicit DropAuthDataRes(bool success) : success{success} {}
bool success;
};
using DropAuthDataRpc = rpc::RequestResponse<DropAuthDataReq, DropAuthDataRes>;
} // namespace memgraph::replication
namespace memgraph::slk {
void Save(const auth::Role &self, memgraph::slk::Builder *builder);
void Load(auth::Role *self, memgraph::slk::Reader *reader);
void Save(const auth::User &self, memgraph::slk::Builder *builder);
void Load(auth::User *self, memgraph::slk::Reader *reader);
void Save(const auth::Auth::Config &self, memgraph::slk::Builder *builder);
void Load(auth::Auth::Config *self, memgraph::slk::Reader *reader);
void Save(const memgraph::replication::UpdateAuthDataRes &self, memgraph::slk::Builder *builder);
void Load(memgraph::replication::UpdateAuthDataRes *self, memgraph::slk::Reader *reader);
void Save(const memgraph::replication::UpdateAuthDataReq & /*self*/, memgraph::slk::Builder * /*builder*/);
void Load(memgraph::replication::UpdateAuthDataReq * /*self*/, memgraph::slk::Reader * /*reader*/);
void Save(const memgraph::replication::DropAuthDataRes &self, memgraph::slk::Builder *builder);
void Load(memgraph::replication::DropAuthDataRes *self, memgraph::slk::Reader *reader);
void Save(const memgraph::replication::DropAuthDataReq & /*self*/, memgraph::slk::Builder * /*builder*/);
void Load(memgraph::replication::DropAuthDataReq * /*self*/, memgraph::slk::Reader * /*reader*/);
} // namespace memgraph::slk

View File

@@ -16,12 +16,10 @@ set(communication_src_files
find_package(Boost REQUIRED) find_package(Boost REQUIRED)
add_library(mg-communication-metrics STATIC metrics.cpp)
target_link_libraries(mg-communication-metrics json)
add_library(mg-communication STATIC ${communication_src_files}) add_library(mg-communication STATIC ${communication_src_files})
target_link_libraries(mg-communication Boost::headers Threads::Threads mg-utils mg-io mg-auth fmt::fmt gflags mg-communication-metrics mg-events) target_link_libraries(mg-communication Boost::headers Threads::Threads mg-utils mg-io mg-auth fmt::fmt gflags)
find_package(OpenSSL REQUIRED) find_package(OpenSSL REQUIRED)
target_link_libraries(mg-communication ${OPENSSL_LIBRARIES}) target_link_libraries(mg-communication ${OPENSSL_LIBRARIES})
target_include_directories(mg-communication SYSTEM PUBLIC ${OPENSSL_INCLUDE_DIR}) target_include_directories(mg-communication SYSTEM PUBLIC ${OPENSSL_INCLUDE_DIR})
target_precompile_headers(mg-communication INTERFACE http/server.hpp <boost/beast/websocket.hpp> bolt/v1/session.hpp)

View File

@@ -1,4 +1,4 @@
// Copyright 2023 Memgraph Ltd. // Copyright 2022 Memgraph Ltd.
// //
// Use of this software is governed by the Business Source License // Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source // included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
@@ -264,5 +264,4 @@ bool Client::ReadMessageData(Marker marker, Value &ret) {
} }
return false; return false;
} }
} // namespace memgraph::communication::bolt } // namespace memgraph::communication::bolt

View File

@@ -1,4 +1,4 @@
// Copyright 2023 Memgraph Ltd. // Copyright 2022 Memgraph Ltd.
// //
// Use of this software is governed by the Business Source License // Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source // included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
@@ -14,7 +14,6 @@
#include <map> #include <map>
#include <optional> #include <optional>
#include <string> #include <string>
#include <utility>
#include <vector> #include <vector>
#include "communication/bolt/v1/codes.hpp" #include "communication/bolt/v1/codes.hpp"
@@ -35,11 +34,10 @@ class FailureResponseException : public utils::BasicException {
explicit FailureResponseException(const std::string &message) : utils::BasicException{message} {} explicit FailureResponseException(const std::string &message) : utils::BasicException{message} {}
FailureResponseException(std::string code, const std::string &message) FailureResponseException(const std::string &code, const std::string &message)
: utils::BasicException{message}, code_{std::move(code)} {} : utils::BasicException{message}, code_{code} {}
const std::string &code() const { return code_; } const std::string &code() const { return code_; }
SPECIALIZE_GET_EXCEPTION_NAME(FailureResponseException)
private: private:
std::string code_; std::string code_;
@@ -51,7 +49,6 @@ class FailureResponseException : public utils::BasicException {
class ClientQueryException : public FailureResponseException { class ClientQueryException : public FailureResponseException {
public: public:
using FailureResponseException::FailureResponseException; using FailureResponseException::FailureResponseException;
SPECIALIZE_GET_EXCEPTION_NAME(ClientQueryException)
}; };
/// This exception is thrown whenever a fatal error occurs during query /// This exception is thrown whenever a fatal error occurs during query
@@ -60,7 +57,6 @@ class ClientQueryException : public FailureResponseException {
class ClientFatalException : public utils::BasicException { class ClientFatalException : public utils::BasicException {
public: public:
using utils::BasicException::BasicException; using utils::BasicException::BasicException;
SPECIALIZE_GET_EXCEPTION_NAME(ClientFatalException)
}; };
// Internal exception used whenever a communication error occurs. You should // Internal exception used whenever a communication error occurs. You should
@@ -68,7 +64,6 @@ class ClientFatalException : public utils::BasicException {
class ServerCommunicationException : public ClientFatalException { class ServerCommunicationException : public ClientFatalException {
public: public:
ServerCommunicationException() : ClientFatalException("Couldn't communicate with the server!") {} ServerCommunicationException() : ClientFatalException("Couldn't communicate with the server!") {}
SPECIALIZE_GET_EXCEPTION_NAME(ServerCommunicationException)
}; };
// Internal exception used whenever a malformed data error occurs. You should // Internal exception used whenever a malformed data error occurs. You should
@@ -76,7 +71,6 @@ class ServerCommunicationException : public ClientFatalException {
class ServerMalformedDataException : public ClientFatalException { class ServerMalformedDataException : public ClientFatalException {
public: public:
ServerMalformedDataException() : ClientFatalException("The server sent malformed data!") {} ServerMalformedDataException() : ClientFatalException("The server sent malformed data!") {}
SPECIALIZE_GET_EXCEPTION_NAME(ServerMalformedDataException)
}; };
/// Structure that is used to return results from an executed query. /// Structure that is used to return results from an executed query.
@@ -161,5 +155,4 @@ class Client final {
ChunkedEncoderBuffer<communication::ClientOutputStream> encoder_buffer_{output_stream_}; ChunkedEncoderBuffer<communication::ClientOutputStream> encoder_buffer_{output_stream_};
ClientEncoder encoder_{encoder_buffer_}; ClientEncoder encoder_{encoder_buffer_};
}; };
} // namespace memgraph::communication::bolt } // namespace memgraph::communication::bolt

View File

@@ -1,55 +0,0 @@
// Copyright 2023 Memgraph Ltd.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
// License, and you may not use this file except in compliance with the Business Source License.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#pragma once
#include "communication/bolt/v1/value.hpp"
#include "communication/metrics.hpp"
namespace memgraph::communication::bolt {
template <typename TSession>
inline void RegisterNewSession(TSession &session, Value &metadata) {
auto &data = metadata.ValueMap();
session.metrics_ = bolt_metrics.Add(data.contains("user_agent") ? data["user_agent"].ValueString() : "unknown",
fmt::format("{}.{}", session.version_.major, session.version_.minor),
session.client_supported_bolt_versions_);
++session.metrics_.value()->sessions;
auto conn_type = (!data.contains("scheme") || data["scheme"].ValueString() == "none")
? BoltMetrics::ConnectionType::kAnonymous
: BoltMetrics::ConnectionType::kBasic;
++session.metrics_.value()->connection_types[(int)conn_type];
}
template <typename TSession>
inline void TouchNewSession(TSession &session, Value &metadata) {
auto &data = metadata.ValueMap();
session.metrics_ = bolt_metrics.Add(data.contains("user_agent") ? data["user_agent"].ValueString() : "unknown");
}
template <typename TSession>
inline void UpdateNewSession(TSession &session, Value &metadata) {
auto &data = metadata.ValueMap();
session.metrics_.value()->bolt_v = fmt::format("{}.{}", session.version_.major, session.version_.minor);
session.metrics_.value()->supported_bolt_v = session.client_supported_bolt_versions_;
++session.metrics_.value()->sessions;
auto conn_type = (!data.contains("scheme") || data["scheme"].ValueString() == "none")
? BoltMetrics::ConnectionType::kAnonymous
: BoltMetrics::ConnectionType::kBasic;
++session.metrics_.value()->connection_types[(int)conn_type];
}
template <typename TSession>
inline void IncrementQueryMetrics(TSession &session) {
++session.metrics_.value()->queries;
}
} // namespace memgraph::communication::bolt

View File

@@ -1,4 +1,4 @@
// Copyright 2023 Memgraph Ltd. // Copyright 2022 Memgraph Ltd.
// //
// Use of this software is governed by the Business Source License // Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source // included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
@@ -51,7 +51,7 @@ enum class ChunkState : uint8_t {
template <typename TBuffer> template <typename TBuffer>
class ChunkedDecoderBuffer { class ChunkedDecoderBuffer {
public: public:
explicit ChunkedDecoderBuffer(TBuffer &buffer) : buffer_(buffer) { data_.reserve(kChunkMaxDataSize); } ChunkedDecoderBuffer(TBuffer &buffer) : buffer_(buffer) { data_.reserve(kChunkMaxDataSize); }
/** /**
* Reads data from the internal buffer. * Reads data from the internal buffer.

View File

@@ -401,11 +401,11 @@ class Decoder {
} }
auto &labels = dv.ValueList(); auto &labels = dv.ValueList();
vertex.labels.reserve(labels.size()); vertex.labels.reserve(labels.size());
for (auto &label : labels) { for (size_t i = 0; i < labels.size(); ++i) {
if (label.type() != Value::Type::String) { if (labels[i].type() != Value::Type::String) {
return false; return false;
} }
vertex.labels.emplace_back(std::move(label.ValueString())); vertex.labels.emplace_back(std::move(labels[i].ValueString()));
} }
// read properties // read properties

View File

@@ -111,12 +111,12 @@ class BaseEncoder {
void WriteList(const std::vector<Value> &value) { void WriteList(const std::vector<Value> &value) {
WriteTypeSize(value.size(), MarkerList); WriteTypeSize(value.size(), MarkerList);
for (const auto &x : value) WriteValue(x); for (auto &x : value) WriteValue(x);
} }
void WriteMap(const std::map<std::string, Value> &value) { void WriteMap(const std::map<std::string, Value> &value) {
WriteTypeSize(value.size(), MarkerMap); WriteTypeSize(value.size(), MarkerMap);
for (const auto &x : value) { for (auto &x : value) {
WriteString(x.first); WriteString(x.first);
WriteValue(x.second); WriteValue(x.second);
} }
@@ -205,11 +205,11 @@ class BaseEncoder {
WriteRAW(utils::UnderlyingCast(Marker::TinyStruct) + 3); WriteRAW(utils::UnderlyingCast(Marker::TinyStruct) + 3);
WriteRAW(utils::UnderlyingCast(Signature::Path)); WriteRAW(utils::UnderlyingCast(Signature::Path));
WriteTypeSize(path.vertices.size(), MarkerList); WriteTypeSize(path.vertices.size(), MarkerList);
for (const auto &v : path.vertices) WriteVertex(v); for (auto &v : path.vertices) WriteVertex(v);
WriteTypeSize(path.edges.size(), MarkerList); WriteTypeSize(path.edges.size(), MarkerList);
for (const auto &e : path.edges) WriteEdge(e); for (auto &e : path.edges) WriteEdge(e);
WriteTypeSize(path.indices.size(), MarkerList); WriteTypeSize(path.indices.size(), MarkerList);
for (const auto &i : path.indices) WriteInt(i); for (auto &i : path.indices) WriteInt(i);
} }
void WriteDate(const utils::Date &date) { void WriteDate(const utils::Date &date) {

View File

@@ -1,4 +1,4 @@
// Copyright 2023 Memgraph Ltd. // Copyright 2022 Memgraph Ltd.
// //
// Use of this software is governed by the Business Source License // Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source // included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
@@ -48,7 +48,7 @@ namespace memgraph::communication::bolt {
template <class TOutputStream> template <class TOutputStream>
class ChunkedEncoderBuffer { class ChunkedEncoderBuffer {
public: public:
explicit ChunkedEncoderBuffer(TOutputStream &output_stream) : output_stream_(output_stream) {} ChunkedEncoderBuffer(TOutputStream &output_stream) : output_stream_(output_stream) {}
/** /**
* Writes n values into the buffer. If n is bigger than whole chunk size * Writes n values into the buffer. If n is bigger than whole chunk size

View File

@@ -39,7 +39,7 @@ class ClientEncoder : private BaseEncoder<Buffer> {
using BaseEncoder<Buffer>::buffer_; using BaseEncoder<Buffer>::buffer_;
public: public:
explicit ClientEncoder(Buffer &buffer) : BaseEncoder<Buffer>(buffer) {} ClientEncoder(Buffer &buffer) : BaseEncoder<Buffer>(buffer) {}
using BaseEncoder<Buffer>::UpdateVersion; using BaseEncoder<Buffer>::UpdateVersion;

View File

@@ -32,7 +32,7 @@ class Encoder : private BaseEncoder<Buffer> {
using BaseEncoder<Buffer>::buffer_; using BaseEncoder<Buffer>::buffer_;
public: public:
explicit Encoder(Buffer &buffer) : BaseEncoder<Buffer>(buffer) {} Encoder(Buffer &buffer) : BaseEncoder<Buffer>(buffer) {}
using BaseEncoder<Buffer>::UpdateVersion; using BaseEncoder<Buffer>::UpdateVersion;

View File

@@ -1,4 +1,4 @@
// Copyright 2023 Memgraph Ltd. // Copyright 2022 Memgraph Ltd.
// //
// Use of this software is governed by the Business Source License // Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source // included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
@@ -30,7 +30,6 @@ namespace memgraph::communication::bolt {
class ClientError : public utils::BasicException { class ClientError : public utils::BasicException {
public: public:
using utils::BasicException::BasicException; using utils::BasicException::BasicException;
SPECIALIZE_GET_EXCEPTION_NAME(ClientError)
}; };
/** /**
@@ -68,7 +67,6 @@ class VerboseError : public utils::BasicException {
code_(fmt::format("Memgraph.{}.{}.{}", ClassificationToString(classification), category, title)) {} code_(fmt::format("Memgraph.{}.{}.{}", ClassificationToString(classification), category, title)) {}
const std::string &code() const noexcept { return code_; } const std::string &code() const noexcept { return code_; }
SPECIALIZE_GET_EXCEPTION_NAME(VerboseError)
private: private:
std::string ClassificationToString(Classification classification) { std::string ClassificationToString(Classification classification) {

View File

@@ -27,7 +27,6 @@
#include "communication/bolt/v1/states/handshake.hpp" #include "communication/bolt/v1/states/handshake.hpp"
#include "communication/bolt/v1/states/init.hpp" #include "communication/bolt/v1/states/init.hpp"
#include "communication/bolt/v1/value.hpp" #include "communication/bolt/v1/value.hpp"
#include "communication/metrics.hpp"
#include "dbms/constants.hpp" #include "dbms/constants.hpp"
#include "dbms/global.hpp" #include "dbms/global.hpp"
#include "utils/exceptions.hpp" #include "utils/exceptions.hpp"
@@ -44,7 +43,6 @@ namespace memgraph::communication::bolt {
class SessionException : public utils::BasicException { class SessionException : public utils::BasicException {
public: public:
using utils::BasicException::BasicException; using utils::BasicException::BasicException;
SPECIALIZE_GET_EXCEPTION_NAME(SessionException)
}; };
/** /**
@@ -209,8 +207,6 @@ class Session {
}; };
Version version_; Version version_;
std::vector<std::string> client_supported_bolt_versions_;
std::optional<BoltMetrics::Metrics> metrics_;
virtual std::string GetCurrentDB() const = 0; virtual std::string GetCurrentDB() const = 0;
std::string UUID() const { return session_uuid_; } std::string UUID() const { return session_uuid_; }

View File

@@ -1,4 +1,4 @@
// Copyright 2024 Memgraph Ltd. // Copyright 2023 Memgraph Ltd.
// //
// Use of this software is governed by the Business Source License // Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source // included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
@@ -18,7 +18,6 @@
#include <string_view> #include <string_view>
#include <vector> #include <vector>
#include "communication/bolt/metrics.hpp"
#include "communication/bolt/v1/codes.hpp" #include "communication/bolt/v1/codes.hpp"
#include "communication/bolt/v1/constants.hpp" #include "communication/bolt/v1/constants.hpp"
#include "communication/bolt/v1/exceptions.hpp" #include "communication/bolt/v1/exceptions.hpp"
@@ -93,7 +92,7 @@ State HandlePullDiscard(TSession &session, std::optional<int> n, std::optional<i
return State::Close; return State::Close;
} }
if (summary.contains("has_more") && summary.at("has_more").ValueBool()) { if (summary.count("has_more") && summary.at("has_more").ValueBool()) {
return State::Result; return State::Result;
} }
@@ -148,13 +147,13 @@ State HandlePullDiscardV4(TSession &session, const State state, const Marker mar
spdlog::trace("Couldn't read extra field!"); spdlog::trace("Couldn't read extra field!");
} }
const auto &extra_map = extra.ValueMap(); const auto &extra_map = extra.ValueMap();
if (extra_map.contains("n")) { if (extra_map.count("n")) {
if (const auto n_value = extra_map.at("n").ValueInt(); n_value != kPullAll) { if (const auto n_value = extra_map.at("n").ValueInt(); n_value != kPullAll) {
n = n_value; n = n_value;
} }
} }
if (extra_map.contains("qid")) { if (extra_map.count("qid")) {
if (const auto qid_value = extra_map.at("qid").ValueInt(); qid_value != kPullLast) { if (const auto qid_value = extra_map.at("qid").ValueInt(); qid_value != kPullLast) {
qid = qid_value; qid = qid_value;
} }
@@ -170,7 +169,6 @@ inline State HandleFailure(TSession &session, const std::exception &e) {
spdlog::trace("Error trace: {}", p->trace()); spdlog::trace("Error trace: {}", p->trace());
} }
session.encoder_buffer_.Clear(); session.encoder_buffer_.Clear();
auto code_message = ExceptionToErrorMessage(e); auto code_message = ExceptionToErrorMessage(e);
bool fail_sent = session.encoder_.MessageFailure({{"code", code_message.first}, {"message", code_message.second}}); bool fail_sent = session.encoder_.MessageFailure({{"code", code_message.first}, {"message", code_message.second}});
if (!fail_sent) { if (!fail_sent) {
@@ -210,14 +208,7 @@ State HandleRunV1(TSession &session, const State state, const Marker marker) {
DMG_ASSERT(!session.encoder_buffer_.HasData(), "There should be no data to write in this state"); DMG_ASSERT(!session.encoder_buffer_.HasData(), "There should be no data to write in this state");
#if MG_ENTERPRISE
spdlog::debug("[Run - {}] '{}'", session.GetCurrentDB(), query.ValueString()); spdlog::debug("[Run - {}] '{}'", session.GetCurrentDB(), query.ValueString());
#else
spdlog::debug("[Run] '{}'", query.ValueString());
#endif
// Increment number of queries in the metrics
IncrementQueryMetrics(session);
try { try {
// Interpret can throw. // Interpret can throw.
@@ -281,14 +272,7 @@ State HandleRunV4(TSession &session, const State state, const Marker marker) {
return HandleFailure(session, e); return HandleFailure(session, e);
} }
#if MG_ENTERPRISE
spdlog::debug("[Run - {}] '{}'", session.GetCurrentDB(), query.ValueString()); spdlog::debug("[Run - {}] '{}'", session.GetCurrentDB(), query.ValueString());
#else
spdlog::debug("[Run] '{}'", query.ValueString());
#endif
// Increment number of queries in the metrics
IncrementQueryMetrics(session);
try { try {
// Interpret can throw. // Interpret can throw.
@@ -368,16 +352,14 @@ State HandleReset(TSession &session, const Marker marker) {
return State::Close; return State::Close;
} }
try { if (!session.encoder_.MessageSuccess()) {
session.Abort(); spdlog::trace("Couldn't send success message!");
if (!session.encoder_.MessageSuccess({})) { return State::Close;
spdlog::trace("Couldn't send success message!");
return State::Close;
}
return State::Idle;
} catch (const std::exception &e) {
return HandleFailure(session, e);
} }
session.Abort();
return State::Idle;
} }
template <typename TSession> template <typename TSession>
@@ -400,17 +382,19 @@ State HandleBegin(TSession &session, const State state, const Marker marker) {
DMG_ASSERT(!session.encoder_buffer_.HasData(), "There should be no data to write in this state"); DMG_ASSERT(!session.encoder_buffer_.HasData(), "There should be no data to write in this state");
if (!session.encoder_.MessageSuccess({})) {
spdlog::trace("Couldn't send success message!");
return State::Close;
}
try { try {
session.Configure(extra.ValueMap()); session.Configure(extra.ValueMap());
session.BeginTransaction(extra.ValueMap()); session.BeginTransaction(extra.ValueMap());
if (!session.encoder_.MessageSuccess({})) {
spdlog::trace("Couldn't send success message!");
return State::Close;
}
return State::Idle;
} catch (const std::exception &e) { } catch (const std::exception &e) {
return HandleFailure(session, e); return HandleFailure(session, e);
} }
return State::Idle;
} }
template <typename TSession> template <typename TSession>
@@ -428,11 +412,11 @@ State HandleCommit(TSession &session, const State state, const Marker marker) {
DMG_ASSERT(!session.encoder_buffer_.HasData(), "There should be no data to write in this state"); DMG_ASSERT(!session.encoder_buffer_.HasData(), "There should be no data to write in this state");
try { try {
session.CommitTransaction();
if (!session.encoder_.MessageSuccess({})) { if (!session.encoder_.MessageSuccess({})) {
spdlog::trace("Couldn't send success message!"); spdlog::trace("Couldn't send success message!");
return State::Close; return State::Close;
} }
session.CommitTransaction();
return State::Idle; return State::Idle;
} catch (const std::exception &e) { } catch (const std::exception &e) {
return HandleFailure(session, e); return HandleFailure(session, e);
@@ -454,11 +438,11 @@ State HandleRollback(TSession &session, const State state, const Marker marker)
DMG_ASSERT(!session.encoder_buffer_.HasData(), "There should be no data to write in this state"); DMG_ASSERT(!session.encoder_buffer_.HasData(), "There should be no data to write in this state");
try { try {
session.RollbackTransaction();
if (!session.encoder_.MessageSuccess({})) { if (!session.encoder_.MessageSuccess({})) {
spdlog::trace("Couldn't send success message!"); spdlog::trace("Couldn't send success message!");
return State::Close; return State::Close;
} }
session.RollbackTransaction();
return State::Idle; return State::Idle;
} catch (const std::exception &e) { } catch (const std::exception &e) {
return HandleFailure(session, e); return HandleFailure(session, e);

View File

@@ -1,4 +1,4 @@
// Copyright 2023 Memgraph Ltd. // Copyright 2022 Memgraph Ltd.
// //
// Use of this software is governed by the Business Source License // Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source // included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
@@ -24,31 +24,6 @@
namespace memgraph::communication::bolt { namespace memgraph::communication::bolt {
inline std::vector<std::string> StringifySupportedVersions(uint8_t *data) {
std::vector<std::string> res;
uint8_t range_i = 1;
uint8_t minor_i = 2;
uint8_t major_i = 3;
uint8_t chunk_size = 4;
uint8_t n_chunks = 4;
auto stringify_version = [](uint8_t major, uint8_t minor) { return fmt::format("{}.{}", major, minor); };
for (uint8_t i = 0; i < n_chunks; ++i) {
const uint32_t full_version = *((uint32_t *)data);
if (full_version == 0) break;
if (data[1] != 0) { // Supports a range of versions
uint8_t range = data[range_i];
auto max_minor = data[minor_i];
for (uint8_t r = 0; r <= range; ++r) {
res.push_back(stringify_version(data[major_i], max_minor - r));
}
} else {
res.push_back(stringify_version(data[major_i], data[minor_i]));
}
data += chunk_size;
}
return res;
}
inline bool CopyProtocolInformationIfSupported(uint16_t version, uint8_t *protocol) { inline bool CopyProtocolInformationIfSupported(uint16_t version, uint8_t *protocol) {
const auto *supported_version = std::find(std::begin(kSupportedVersions), std::end(kSupportedVersions), version); const auto *supported_version = std::find(std::begin(kSupportedVersions), std::end(kSupportedVersions), version);
if (supported_version != std::end(kSupportedVersions)) { if (supported_version != std::end(kSupportedVersions)) {
@@ -96,8 +71,6 @@ State StateHandshakeRun(TSession &session) {
auto dataPosition = session.input_stream_.data() + sizeof(kPreamble); auto dataPosition = session.input_stream_.data() + sizeof(kPreamble);
uint8_t protocol[4] = {0x00}; uint8_t protocol[4] = {0x00};
session.client_supported_bolt_versions_ = std::move(StringifySupportedVersions(dataPosition));
for (int i = 0; i < 4 && !protocol[3]; ++i) { for (int i = 0; i < 4 && !protocol[3]; ++i) {
// If there is an offset defined (e.g. 0x00 0x03 0x03 0x04) the second byte // If there is an offset defined (e.g. 0x00 0x03 0x03 0x04) the second byte
// That would enable the client to pick between 4.0 and 4.3 versions // That would enable the client to pick between 4.0 and 4.3 versions

View File

@@ -11,7 +11,6 @@
#pragma once #pragma once
#include <fmt/core.h>
#include <fmt/format.h> #include <fmt/format.h>
#include <optional> #include <optional>
@@ -19,7 +18,6 @@
#include "communication/bolt/v1/state.hpp" #include "communication/bolt/v1/state.hpp"
#include "communication/bolt/v1/value.hpp" #include "communication/bolt/v1/value.hpp"
#include "communication/exceptions.hpp" #include "communication/exceptions.hpp"
#include "communication/metrics.hpp"
#include "spdlog/spdlog.h" #include "spdlog/spdlog.h"
#include "utils/likely.hpp" #include "utils/likely.hpp"
#include "utils/logging.hpp" #include "utils/logging.hpp"
@@ -42,11 +40,11 @@ std::optional<State> AuthenticateUser(TSession &session, Value &metadata) {
std::string username; std::string username;
std::string password; std::string password;
if (data["scheme"].ValueString() == "basic") { if (data["scheme"].ValueString() == "basic") {
if (!data.contains("principal")) { // Special case principal = "" if (!data.count("principal")) { // Special case principal = ""
spdlog::warn("The client didn't supply the principal field! Trying with \"\"..."); spdlog::warn("The client didn't supply the principal field! Trying with \"\"...");
data["principal"] = ""; data["principal"] = "";
} }
if (!data.contains("credentials")) { // Special case credentials = "" if (!data.count("credentials")) { // Special case credentials = ""
spdlog::warn("The client didn't supply the credentials field! Trying with \"\"..."); spdlog::warn("The client didn't supply the credentials field! Trying with \"\"...");
data["credentials"] = ""; data["credentials"] = "";
} }
@@ -118,7 +116,7 @@ std::optional<Value> GetMetadataV4(TSession &session, const Marker marker) {
} }
auto &data = metadata.ValueMap(); auto &data = metadata.ValueMap();
if (!data.contains("user_agent")) { if (!data.count("user_agent")) {
spdlog::warn("The client didn't supply the user agent!"); spdlog::warn("The client didn't supply the user agent!");
return std::nullopt; return std::nullopt;
} }
@@ -142,7 +140,7 @@ std::optional<Value> GetInitDataV5(TSession &session, const Marker marker) {
} }
const auto &data = metadata.ValueMap(); const auto &data = metadata.ValueMap();
if (!data.contains("user_agent")) { if (!data.count("user_agent")) {
spdlog::warn("The client didn't supply the user agent!"); spdlog::warn("The client didn't supply the user agent!");
return std::nullopt; return std::nullopt;
} }
@@ -203,9 +201,6 @@ State StateInitRunV1(TSession &session, const Marker marker, const Signature sig
return result.value(); return result.value();
} }
// Register session to metrics
RegisterNewSession(session, *maybeMetadata);
return SendSuccessMessage(session); return SendSuccessMessage(session);
} }
@@ -232,9 +227,6 @@ State StateInitRunV4(TSession &session, Marker marker, Signature signature) {
return result.value(); return result.value();
} }
// Register session to metrics
RegisterNewSession(session, *maybeMetadata);
return SendSuccessMessage(session); return SendSuccessMessage(session);
} }
@@ -255,10 +247,6 @@ State StateInitRunV5(TSession &session, Marker marker, Signature signature) {
if (SendSuccessMessage(session) == State::Close) { if (SendSuccessMessage(session) == State::Close) {
return State::Close; return State::Close;
} }
// Register session to metrics
TouchNewSession(session, *maybeMetadata);
// Stay in Init // Stay in Init
return State::Init; return State::Init;
} }
@@ -286,10 +274,6 @@ State StateInitRunV5(TSession &session, Marker marker, Signature signature) {
if (SendSuccessMessage(session) == State::Close) { if (SendSuccessMessage(session) == State::Close) {
return State::Close; return State::Close;
} }
// Register session to metrics
UpdateNewSession(session, *maybeMetadata);
return State::Idle; return State::Idle;
} }

Some files were not shown because too many files have changed in this diff Show More