Compare commits

...

28 Commits

Author SHA1 Message Date
jbajic
ff739d2dea Reduce requirement for py tools because of Ubuntu18.04 and CentOS7 2022-12-13 16:15:25 +01:00
jbajic
858a79d311 Add licenses directory to ignore paths 2022-12-13 13:50:43 +01:00
Andi
310e305cfb Fix python module reloading (#706) 2022-12-12 21:11:13 +01:00
Marko Budiselić
9d6a23b6bd Add init-file and init-data-file capabilities (#696) 2022-12-09 18:50:33 +01:00
Andi
f2d5ab61c4 Fix Python submodules reloading (#653) 2022-12-09 14:30:41 +01:00
Andi
0f77c85824 Fix cursor exhaustion by adding EmptyResult operator (#667) 2022-12-09 11:44:07 +01:00
niko4299
d6d4153fb7 Fix graph projection bug (#697) 2022-12-08 13:45:20 +01:00
Tyler Neely
7d6a5e5b9c Add support for -h to show help in addition to --help (#682) 2022-12-07 16:51:32 +01:00
Vlasta
c529d52664 Add community links to the README (#693) 2022-12-07 12:42:48 +01:00
Ante Pušić
45451bae3b Fix C++ query modules API bugs (#688) 2022-12-06 16:57:50 +01:00
niko4299
3e11f38548 Add aggregation distinct (#654) (#665) 2022-12-03 13:48:44 +02:00
Jure Bajic
6e4047a847 Bump mgconsole version (#660) 2022-12-01 13:10:08 +01:00
Ante Javor
8febdc12fb Update tests/mgbench README (#679) 2022-11-30 12:43:57 +01:00
Kruno Golubic
3f23a10f44 Update README with one new paragraph and link (#675) 2022-11-29 14:24:02 +01:00
Ante Javor
11300960de Add mixed workload and Neo4j client to mgbench (#566)
* Fix bolt bug inside the C++ client
* Add tail latency stats
* Add hot run option
* Add query caching
* Add jcmd memory tracking
2022-11-28 08:47:22 +01:00
Jure Bajic
1d5f387ddd Add python check (#643) 2022-11-09 11:48:34 +02:00
Marko Budiselić
c4c3a254bf Remove tools/check-build-system (#642) 2022-11-07 18:54:22 +01:00
Jure Bajic
b4beb0fc86 Update license for release 2.4.2 (#641) 2022-11-07 13:07:20 +01:00
Bruno Sačarić
58e6097664 Fix ALLSHORTEST combined with id function (#636) 2022-11-04 19:36:03 +01:00
Jure Bajic
ff21c0705c Add multiple license support (#618)
Make license info available through LicenseChecker
Add LicenseInfoSender
Move license library from utils
Rename telemetry_lib to mg-telemetry
2022-11-04 15:23:43 +01:00
Katarina Supe
2a2b99b02a Release mgp 1.1.0 (#620)
- Updated its README, authors, and published it with the added
   AuthorizationError that was a part of Memgraph 2.4.0 release.
 - Added missing SOURCE_TYPE_KAFKA and SOURCE_TYPE_PULSAR variables in _mgp.
2022-11-02 14:14:12 +01:00
Antonio Filipovic
3daab6ce97 Fix bug in the C API in-edges iterator (#613) 2022-10-25 19:18:44 +02:00
Marko Budiselić
fbd7274c95 Add Fedora 36 as an OS (#599) 2022-10-21 15:22:50 +02:00
Marko Budiselić
6efc84f022 Add libc++ option to the toolchain (#567) 2022-10-20 07:52:59 +02:00
Jure Bajic
287c2e94d1 Update license date (#586) 2022-10-07 14:42:52 +02:00
Antonio Filipovic
417cf4b30b Fix bug related to EdgeType and Label getters in query modules (#582)
Co-authored-by: Kostas Kyrimis <kostaskyrim@gmail.com>
2022-10-06 21:21:11 +02:00
Jure Bajic
68e7fd3d36 Fix architecture check (#583) 2022-10-06 15:55:23 +02:00
Bruno Sačarić
5261d82063 Fix passing user's fine_grained_access_handler instead of role's (#579)
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
2022-09-30 18:27:47 +02:00
127 changed files with 6493 additions and 1727 deletions

View File

@@ -16,7 +16,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)

View File

@@ -14,6 +14,7 @@ on:
- "**/*.md"
- ".clang-format"
- "CODEOWNERS"
- licenses/**
jobs:
community_build:
@@ -26,7 +27,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -64,7 +65,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -75,7 +76,7 @@ jobs:
- name: Fetch all history for all tags and branches
run: git fetch
- name: Build combined ASAN, UBSAN and coverage binaries
- name: Initialize deps
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
@@ -83,6 +84,32 @@ jobs:
# Initialize dependencies.
./init
- name: Set base branch
if: ${{ github.event_name == 'pull_request' }}
run: |
echo "BASE_BRANCH=origin/${{ github.base_ref }}" >> $GITHUB_ENV
- name: Set base branch # if we manually dispatch or push to master
if: ${{ github.event_name != 'pull_request' }}
run: |
echo "BASE_BRANCH=origin/master" >> $GITHUB_ENV
- name: Python code analysis
run: |
CHANGED_FILES=$(git diff -U0 ${{ env.BASE_BRANCH }}... --name-only)
for file in ${CHANGED_FILES}; do
echo ${file}
if [[ ${file} == *.py ]]; then
python3 -m black --check --diff ${file}
python3 -m isort --check-only --diff ${file}
fi
done
- name: Build combined ASAN, UBSAN and coverage binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
cd build
cmake -DTEST_COVERAGE=ON -DASAN=ON -DUBSAN=ON ..
make -j$THREADS memgraph__unit
@@ -110,21 +137,11 @@ jobs:
tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu
- name: Save code coverage
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/generated/code_coverage.tar.gz
- name: Set base branch
if: ${{ github.event_name == 'pull_request' }}
run: |
echo "BASE_BRANCH=origin/${{ github.base_ref }}" >> $GITHUB_ENV
- name: Set base branch # if we manually dispatch or push to master
if: ${{ github.event_name != 'pull_request' }}
run: |
echo "BASE_BRANCH=origin/master" >> $GITHUB_ENV
- name: Run clang-tidy
run: |
source /opt/toolchain-v4/activate
@@ -145,7 +162,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -206,7 +223,7 @@ jobs:
./cppcheck_and_clang_format diff
- name: Save cppcheck and clang-format errors
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/cppcheck_and_clang_format.txt
@@ -221,7 +238,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -246,7 +263,7 @@ jobs:
./continuous_integration
- name: Save quality assurance status
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "GQL Behave Status"
path: |
@@ -303,13 +320,13 @@ jobs:
cpack -G DEB --config ../CPackConfig.cmake
- name: Save enterprise DEB package
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Enterprise DEB package"
path: build/output/memgraph*.deb
- name: Save test data
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
if: always()
with:
name: "Test data"
@@ -328,7 +345,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -353,7 +370,7 @@ jobs:
./run.sh test --binary ../../build/memgraph --run-args "test-all --node-configs resources/node-config.edn" --ignore-run-stdout-logs --ignore-run-stderr-logs
- name: Save Jepsen report
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
if: ${{ always() }}
with:
name: "Jepsen Report"
@@ -369,7 +386,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)

View File

@@ -14,7 +14,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)

View File

@@ -17,7 +17,7 @@ jobs:
run: |
./release/package/run.sh package centos-7
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: centos-7
path: build/output/centos-7/memgraph*.rpm
@@ -34,7 +34,7 @@ jobs:
run: |
./release/package/run.sh package centos-9
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: centos-9
path: build/output/centos-9/memgraph*.rpm
@@ -51,7 +51,7 @@ jobs:
run: |
./release/package/run.sh package debian-10
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: debian-10
path: build/output/debian-10/memgraph*.deb
@@ -68,7 +68,7 @@ jobs:
run: |
./release/package/run.sh package debian-11
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: debian-11
path: build/output/debian-11/memgraph*.deb
@@ -87,7 +87,7 @@ jobs:
./run.sh package debian-11 --for-docker
./run.sh docker
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: docker
path: build/output/docker/memgraph*.tar.gz
@@ -104,7 +104,7 @@ jobs:
run: |
./release/package/run.sh package ubuntu-18.04
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: ubuntu-1804
path: build/output/ubuntu-18.04/memgraph*.deb
@@ -121,7 +121,7 @@ jobs:
run: |
./release/package/run.sh package ubuntu-20.04
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: ubuntu-2004
path: build/output/ubuntu-20.04/memgraph*.deb
@@ -138,7 +138,7 @@ jobs:
run: |
./release/package/run.sh package ubuntu-22.04
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: ubuntu-2204
path: build/output/ubuntu-22.04/memgraph*.deb
@@ -155,7 +155,7 @@ jobs:
run: |
./release/package/run.sh package debian-11 --for-platform
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: debian-11-platform
path: build/output/debian-11/memgraph*.deb
@@ -172,7 +172,7 @@ jobs:
run: |
./release/package/run.sh package debian-11-arm
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: debian-11-arm
path: build/output/debian-11-arm/memgraph*.deb

View File

@@ -17,7 +17,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -55,7 +55,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -97,7 +97,7 @@ jobs:
tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu
- name: Save code coverage
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/generated/code_coverage.tar.gz
@@ -112,7 +112,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -173,7 +173,7 @@ jobs:
./cppcheck_and_clang_format diff
- name: Save cppcheck and clang-format errors
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/cppcheck_and_clang_format.txt
@@ -189,7 +189,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -225,7 +225,7 @@ jobs:
rpmlint memgraph*.rpm
- name: Save enterprise RPM package
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Enterprise RPM package"
path: build/output/memgraph*.rpm
@@ -262,7 +262,7 @@ jobs:
./continuous_integration
- name: Save quality assurance status
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "GQL Behave Status"
path: |

View File

@@ -17,7 +17,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -55,7 +55,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -97,7 +97,7 @@ jobs:
tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu
- name: Save code coverage
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/generated/code_coverage.tar.gz
@@ -112,7 +112,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -173,7 +173,7 @@ jobs:
./cppcheck_and_clang_format diff
- name: Save cppcheck and clang-format errors
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/cppcheck_and_clang_format.txt
@@ -189,7 +189,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -224,7 +224,7 @@ jobs:
cpack -G DEB --config ../CPackConfig.cmake
- name: Save enterprise DEB package
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Enterprise DEB package"
path: build/output/memgraph*.deb
@@ -261,7 +261,7 @@ jobs:
./continuous_integration
- name: Save quality assurance status
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "GQL Behave Status"
path: |
@@ -324,7 +324,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -349,7 +349,7 @@ jobs:
./run.sh test --binary ../../build/memgraph --run-args "test-all --node-configs resources/node-config.edn" --ignore-run-stdout-logs --ignore-run-stderr-logs
- name: Save Jepsen report
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
if: ${{ always() }}
with:
name: "Jepsen Report"

View File

@@ -19,17 +19,17 @@ jobs:
DOCKER_REPOSITORY_NAME: memgraph
steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v3
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v2
- name: Log in to Docker Hub
uses: docker/login-action@v1
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

View File

@@ -17,7 +17,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -55,7 +55,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -97,7 +97,7 @@ jobs:
tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu
- name: Save code coverage
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/generated/code_coverage.tar.gz
@@ -112,7 +112,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -173,7 +173,7 @@ jobs:
./cppcheck_and_clang_format diff
- name: Save cppcheck and clang-format errors
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/cppcheck_and_clang_format.txt
@@ -189,7 +189,7 @@ jobs:
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -224,7 +224,7 @@ jobs:
cpack -G DEB --config ../CPackConfig.cmake
- name: Save enterprise DEB package
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Enterprise DEB package"
path: build/output/memgraph*.deb
@@ -261,7 +261,7 @@ jobs:
./continuous_integration
- name: Save quality assurance status
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "GQL Behave Status"
path: |

View File

@@ -6,18 +6,14 @@ repos:
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/psf/black
rev: 22.3.0
rev: 22.10.0
hooks:
- id: black
args: # arguments to configure black
- --line-length=120
- --include='\.pyi?$'
# these folders wont be formatted by black
- --exclude="""\.git |
\.__pycache__|
build|
libs|
.cache"""
- repo: https://github.com/pycqa/isort
rev: 5.10.1
hooks:
- id: isort
name: isort (python)
- repo: https://github.com/pre-commit/mirrors-clang-format
rev: v13.0.0
hooks:

View File

@@ -37,9 +37,10 @@ Build modern, graph-based applications on top of your streaming data in minutes.
## :clipboard: Description
Memgraph is a streaming graph application platform that helps you wrangle your
streaming data, build sophisticated models that you can query in real-time, and
develop graph applications.
Memgraph is an open source graph database built for real-time streaming and
compatible with Neo4j. Whether you're a developer or a data scientist with
interconnected data, Memgraph will get you the immediate actionable insights
fast.
Memgraph directly connects to your streaming infrastructure. You can ingest data
from sources like Kafka, SQL, or plain CSV files. Memgraph provides a standard
@@ -51,8 +52,9 @@ natural and effective way to model many real-world problems without relying on
complex SQL schemas.
Memgraph is implemented in C/C++ and leverages an in-memory first architecture
to ensure that youre getting the best possible performance consistently and
without surprises. Its also ACID-compliant and highly available.
to ensure that youre getting the [best possible
performance](http://memgraph.com/benchgraph) consistently and without surprises.
Its also ACID-compliant and highly available.
## :video_game: Memgraph Playground
@@ -141,6 +143,15 @@ Memgraph Community is available under the [BSL
license](./licenses/BSL.txt).</br> Memgraph Enterprise is available under the
[MEL license](./licenses/MEL.txt).
## 🙋 Community
- :purple_heart: [**Discord**](https://discord.gg/memgraph)
- :busts_in_silhouette: [**Discourse forum**](https://discourse.memgraph.com/)
- :open_file_folder: [**Memgraph GitHub**](https://github.com/memgraph)
- :bird: [**Twitter**](https://twitter.com/memgraphdb)
- :movie_camera:
[**YouTube**](https://www.youtube.com/channel/UCZ3HOJvHGxtQ_JHxOselBYg)
<p align="center">
<a href="#">
<img src="https://img.shields.io/badge/⬆back_to_top_⬆-white" alt="Back to top" title="Back to top"/>

View File

@@ -5,12 +5,10 @@ import os
import subprocess
import sys
import textwrap
import xml.etree.ElementTree as ET
import yaml
SCRIPT_DIR = os.path.dirname(os.path.realpath(__file__))
CONFIG_FILE = os.path.join(SCRIPT_DIR, "flags.yaml")
WIDTH = 80
@@ -18,14 +16,13 @@ WIDTH = 80
def wrap_text(s, initial_indent="# "):
return "\n#\n".join(
map(lambda x: textwrap.fill(x, WIDTH, initial_indent=initial_indent,
subsequent_indent="# "), s.split("\n")))
map(lambda x: textwrap.fill(x, WIDTH, initial_indent=initial_indent, subsequent_indent="# "), s.split("\n"))
)
def extract_flags(binary_path):
ret = {}
data = subprocess.run([binary_path, "--help-xml"],
stdout=subprocess.PIPE).stdout.decode("utf-8")
data = subprocess.run([binary_path, "--help-xml"], stdout=subprocess.PIPE).stdout.decode("utf-8")
root = ET.fromstring(data)
for child in root:
if child.tag == "usage" and child.text.lower().count("warning"):
@@ -46,8 +43,7 @@ def apply_config_to_flags(config, flags):
for modification in config["modifications"]:
name = modification["name"]
if name not in flags:
print("WARNING: Flag '" + name + "' missing from binary!",
file=sys.stderr)
print("WARNING: Flag '" + name + "' missing from binary!", file=sys.stderr)
continue
flags[name]["default"] = modification["value"]
flags[name]["override"] = modification["override"]
@@ -75,8 +71,9 @@ def extract_sections(flags):
else:
sections.append((current_section, current_flags))
sections.append(("other", other))
assert set(sum(map(lambda x: x[1], sections), [])) == set(flags.keys()), \
"The section extraction algorithm lost some flags!"
assert set(sum(map(lambda x: x[1], sections), [])) == set(
flags.keys()
), "The section extraction algorithm lost some flags!"
return sections
@@ -89,8 +86,7 @@ def generate_config_file(sections, flags):
helpstr = flag["meaning"] + " [" + flag["type"] + "]"
ret += wrap_text(helpstr) + "\n"
prefix = "# " if not flag["override"] else ""
ret += prefix + "--" + flag["name"].replace("_", "-") + \
"=" + flag["default"] + "\n\n"
ret += prefix + "--" + flag["name"].replace("_", "-") + "=" + flag["default"] + "\n\n"
ret += "\n"
ret += wrap_text(config["footer"])
return ret.strip() + "\n"
@@ -98,13 +94,9 @@ def generate_config_file(sections, flags):
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("memgraph_binary",
help="path to Memgraph binary")
parser.add_argument("output_file",
help="path where to store the generated Memgraph "
"configuration file")
parser.add_argument("--config-file", default=CONFIG_FILE,
help="path to generator configuration file")
parser.add_argument("memgraph_binary", help="path to Memgraph binary")
parser.add_argument("output_file", help="path where to store the generated Memgraph " "configuration file")
parser.add_argument("--config-file", default=CONFIG_FILE, help="path to generator configuration file")
args = parser.parse_args()
flags = extract_flags(args.memgraph_binary)

98
environment/os/fedora-36.sh Executable file
View File

@@ -0,0 +1,98 @@
#!/bin/bash
set -Eeuo pipefail
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
source "$DIR/../util.sh"
check_operating_system "fedora-36"
check_architecture "x86_64"
TOOLCHAIN_BUILD_DEPS=(
coreutils-common gcc gcc-c++ make # generic build tools
wget # used for archive download
gnupg2 # used for archive signature verification
tar gzip bzip2 xz unzip # used for archive unpacking
zlib-devel # zlib library used for all builds
expat-devel xz-devel python3-devel texinfo libbabeltrace-devel # for gdb
curl libcurl-devel # for cmake
readline-devel # for cmake and llvm
libffi-devel libxml2-devel # for llvm
libedit-devel pcre-devel automake bison # for swig
file
openssl-devel
gmp-devel
gperf
diffutils
libipt libipt-devel # intel
patch
)
TOOLCHAIN_RUN_DEPS=(
make # generic build tools
tar gzip bzip2 xz # used for archive unpacking
zlib # zlib library used for all builds
expat xz-libs python3 # for gdb
readline # for cmake and llvm
libffi libxml2 # for llvm
openssl-devel
perl # for openssl
)
MEMGRAPH_BUILD_DEPS=(
git # source code control
make pkgconf-pkg-config # build system
wget # for downloading libs
libuuid-devel java-11-openjdk # required by antlr
readline-devel # for memgraph console
python3-devel # for query modules
openssl-devel
libseccomp-devel
python3 python3-pip python3-virtualenv python3-virtualenvwrapper python3-pyyaml nmap-ncat # for tests
libcurl-devel # mg-requests
rpm-build rpmlint # for RPM package building
doxygen graphviz # source documentation generators
which nodejs golang zip unzip java-11-openjdk-devel # for driver tests
sbcl # for custom Lisp C++ preprocessing
autoconf # for jemalloc code generation
libtool # for protobuf code generation
)
list() {
echo "$1"
}
check() {
local missing=""
for pkg in $1; do
if ! dnf list installed "$pkg" >/dev/null 2>/dev/null; then
missing="$pkg $missing"
fi
done
if [ "$missing" != "" ]; then
echo "MISSING PACKAGES: $missing"
exit 1
fi
}
install() {
cd "$DIR"
if [ "$EUID" -ne 0 ]; then
echo "Please run as root."
exit 1
fi
# If GitHub Actions runner is installed, append LANG to the environment.
# Python related tests don't work without the LANG export.
if [ -d "/home/gh/actions-runner" ]; then
echo "LANG=en_US.utf8" >> /home/gh/actions-runner/.env
else
echo "NOTE: export LANG=en_US.utf8"
fi
dnf update -y
for pkg in $1; do
dnf install -y "$pkg"
done
}
deps=$2"[*]"
"$1" "${!deps}"

1
environment/toolchain/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
*.tar.gz

View File

@@ -10,6 +10,14 @@ cd "$DIR"
source "$DIR/../util.sh"
DISTRO="$(operating_system)"
function log_tool_name () {
echo ""
echo ""
echo "#### $1 ####"
echo ""
echo ""
}
for_arm=false
if [[ "$#" -eq 1 ]]; then
if [[ "$1" == "--for-arm" ]]; then
@@ -20,9 +28,11 @@ if [[ "$#" -eq 1 ]]; then
fi
fi
os="$1"
# toolchain version
TOOLCHAIN_STDCXX="${TOOLCHAIN_STDCXX:-libstdc++}"
if [[ "$TOOLCHAIN_STDCXX" != "libstdc++" && "$TOOLCHAIN_STDCXX" != "libc++" ]]; then
echo "Only GCC (libstdc++) or LLVM (libc++) C++ standard library implementations are supported."
exit 1
fi
TOOLCHAIN_VERSION=4
# package versions used
@@ -99,6 +109,8 @@ if [ ! -f llvm-$LLVM_VERSION.src.tar.xz ]; then
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/clang-tools-extra-$LLVM_VERSION.src.tar.xz
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/compiler-rt-$LLVM_VERSION.src.tar.xz
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/libunwind-$LLVM_VERSION.src.tar.xz
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/libcxx-$LLVM_VERSION.src.tar.xz
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/libcxxabi-$LLVM_VERSION.src.tar.xz
fi
if [ ! -f pahole-gdb-master.zip ]; then
wget https://github.com/PhilArmstrong/pahole-gdb/archive/master.zip -O pahole-gdb-master.zip
@@ -156,6 +168,8 @@ if [ ! -f llvm-$LLVM_VERSION.src.tar.xz.sig ]; then
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/clang-tools-extra-$LLVM_VERSION.src.tar.xz.sig
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/compiler-rt-$LLVM_VERSION.src.tar.xz.sig
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/libunwind-$LLVM_VERSION.src.tar.xz.sig
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/libcxx-$LLVM_VERSION.src.tar.xz.sig
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/libcxxabi-$LLVM_VERSION.src.tar.xz.sig
fi
# list of valid llvm gnupg keys: https://releases.llvm.org/download.html
$GPG --keyserver $KEYSERVER --recv-keys 0x474E22316ABF4785A88C6E8EA2C794A986419D8A
@@ -165,6 +179,8 @@ $GPG --verify lld-$LLVM_VERSION.src.tar.xz.sig lld-$LLVM_VERSION.src.tar.xz
$GPG --verify clang-tools-extra-$LLVM_VERSION.src.tar.xz.sig clang-tools-extra-$LLVM_VERSION.src.tar.xz
$GPG --verify compiler-rt-$LLVM_VERSION.src.tar.xz.sig compiler-rt-$LLVM_VERSION.src.tar.xz
$GPG --verify libunwind-$LLVM_VERSION.src.tar.xz.sig libunwind-$LLVM_VERSION.src.tar.xz
$GPG --verify libcxx-$LLVM_VERSION.src.tar.xz.sig libcxx-$LLVM_VERSION.src.tar.xz
$GPG --verify libcxxabi-$LLVM_VERSION.src.tar.xz.sig libcxxabi-$LLVM_VERSION.src.tar.xz
popd
@@ -172,7 +188,7 @@ popd
mkdir -p build
pushd build
# compile gcc
log_tool_name "GCC $GCC_VERSION"
if [ ! -f $PREFIX/bin/gcc ]; then
if [ -d gcc-$GCC_VERSION ]; then
rm -rf gcc-$GCC_VERSION
@@ -263,7 +279,7 @@ fi
export PATH=$PREFIX/bin:$PATH
export LD_LIBRARY_PATH=$PREFIX/lib64
# compile binutils
log_tool_name "binutils $BINUTILS_VERSION"
if [ ! -f $PREFIX/bin/ld.gold ]; then
if [ -d binutils-$BINUTILS_VERSION ]; then
rm -rf binutils-$BINUTILS_VERSION
@@ -327,7 +343,7 @@ if [ ! -f $PREFIX/bin/ld.gold ]; then
popd && popd
fi
# compile gdb
log_tool_name "GDB $GDB_VERSION"
if [ ! -f $PREFIX/bin/gdb ]; then
if [ -d gdb-$GDB_VERSION ]; then
rm -rf gdb-$GDB_VERSION
@@ -398,13 +414,13 @@ if [ ! -f $PREFIX/bin/gdb ]; then
popd && popd
fi
# install pahole
log_tool_name "install pahole"
if [ ! -d $PREFIX/share/pahole-gdb ]; then
unzip ../archives/pahole-gdb-master.zip
mv pahole-gdb-master $PREFIX/share/pahole-gdb
fi
# setup system gdbinit
log_tool_name "setup system gdbinit"
if [ ! -f $PREFIX/etc/gdb/gdbinit ]; then
mkdir -p $PREFIX/etc/gdb
cat >$PREFIX/etc/gdb/gdbinit <<EOF
@@ -430,7 +446,7 @@ end
EOF
fi
# compile cmake
log_tool_name "cmake $CMAKE_VERSION"
if [ ! -f $PREFIX/bin/cmake ]; then
if [ -d cmake-$CMAKE_VERSION ]; then
rm -rf cmake-$CMAKE_VERSION
@@ -456,7 +472,7 @@ if [ ! -f $PREFIX/bin/cmake ]; then
popd && popd
fi
# compile cppcheck
log_tool_name "cppcheck $CPPCHECK_VERSION"
if [ ! -f $PREFIX/bin/cppcheck ]; then
if [ -d cppcheck-$CPPCHECK_VERSION ]; then
rm -rf cppcheck-$CPPCHECK_VERSION
@@ -480,7 +496,7 @@ if [ ! -f $PREFIX/bin/cppcheck ]; then
popd
fi
# compile swig
log_tool_name "swig $SWIG_VERSION"
if [ ! -d swig-$SWIG_VERSION/install ]; then
if [ -d swig-$SWIG_VERSION ]; then
rm -rf swig-$SWIG_VERSION
@@ -496,7 +512,7 @@ if [ ! -d swig-$SWIG_VERSION/install ]; then
popd && popd
fi
# compile llvm
log_tool_name "LLVM $LLVM_VERSION"
if [ ! -f $PREFIX/bin/clang ]; then
if [ -d llvm-$LLVM_VERSION ]; then
rm -rf llvm-$LLVM_VERSION
@@ -513,8 +529,19 @@ if [ ! -f $PREFIX/bin/clang ]; then
mv compiler-rt-$LLVM_VERSION.src/ llvm-$LLVM_VERSION/projects/compiler-rt
tar -xvf ../archives/libunwind-$LLVM_VERSION.src.tar.xz
mv libunwind-$LLVM_VERSION.src/include/mach-o llvm-$LLVM_VERSION/tools/lld/include
# The following is required because of libc++
tar -xvf ../archives/libcxx-$LLVM_VERSION.src.tar.xz
mv libcxx-$LLVM_VERSION.src llvm-$LLVM_VERSION/projects/libcxx
tar -xvf ../archives/libcxxabi-$LLVM_VERSION.src.tar.xz
mv libcxxabi-$LLVM_VERSION.src llvm-$LLVM_VERSION/projects/libcxxabi
# NOTE: We moved part of the libunwind in one of the previous step.
rm -r libunwind-$LLVM_VERSION.src
tar -xvf ../archives/libunwind-$LLVM_VERSION.src.tar.xz
mv libunwind-$LLVM_VERSION.src llvm-$LLVM_VERSION/projects/libunwind
pushd llvm-$LLVM_VERSION
mkdir build && pushd build
mkdir -p build && pushd build
# activate swig
export PATH=$DIR/build/swig-$SWIG_VERSION/install/bin:$PATH
# influenced by: https://buildd.debian.org/status/fetch.php?pkg=llvm-toolchain-7&arch=amd64&ver=1%3A7.0.1%7E%2Brc2-1%7Eexp1&stamp=1541506173&raw=0
@@ -820,7 +847,11 @@ source $PREFIX/activate
export CC=$PREFIX/bin/clang
export CXX=$PREFIX/bin/clang++
export CFLAGS="$CFLAGS -fPIC"
export CXXFLAGS="$CXXFLAGS -fPIC"
if [ "$TOOLCHAIN_STDCXX" = "libstdc++" ]; then
export CXXFLAGS="$CXXFLAGS -fPIC"
else
export CXXFLAGS="$CXXFLAGS -fPIC -stdlib=libc++"
fi
COMMON_CMAKE_FLAGS="-DCMAKE_INSTALL_PREFIX=$PREFIX
-DCMAKE_PREFIX_PATH=$PREFIX
-DCMAKE_BUILD_TYPE=Release
@@ -834,7 +865,7 @@ COMMON_CMAKE_FLAGS="-DCMAKE_INSTALL_PREFIX=$PREFIX
COMMON_CONFIGURE_FLAGS="--enable-shared=no --prefix=$PREFIX"
COMMON_MAKE_INSTALL_FLAGS="-j$CPUS BUILD_SHARED=no PREFIX=$PREFIX install"
# install bzip2
log_tool_name "bzip2 $BZIP2_VERSION"
if [ ! -f $PREFIX/include/bzlib.h ]; then
if [ -d bzip2-$BZIP2_VERSION ]; then
rm -rf bzip2-$BZIP2_VERSION
@@ -845,7 +876,7 @@ if [ ! -f $PREFIX/include/bzlib.h ]; then
popd
fi
# install fmt
log_tool_name "fmt $FMT_VERSION"
if [ ! -d $PREFIX/include/fmt ]; then
if [ -d fmt-$FMT_VERSION ]; then
rm -rf fmt-$FMT_VERSION
@@ -858,7 +889,7 @@ if [ ! -d $PREFIX/include/fmt ]; then
popd && popd
fi
# install lz4
log_tool_name "lz4 $LZ4_VERSION"
if [ ! -f $PREFIX/include/lz4.h ]; then
if [ -d lz4-$LZ4_VERSION ]; then
rm -rf lz4-$LZ4_VERSION
@@ -869,7 +900,7 @@ if [ ! -f $PREFIX/include/lz4.h ]; then
popd
fi
# install xz
log_tool_name "xz $XZ_VERSION"
if [ ! -f $PREFIX/include/lzma.h ]; then
if [ -d xz-$XZ_VERSION ]; then
rm -rf xz-$XZ_VERSION
@@ -881,7 +912,7 @@ if [ ! -f $PREFIX/include/lzma.h ]; then
popd
fi
# install zlib
log_tool_name "zlib $ZLIB_VERSION"
if [ ! -f $PREFIX/include/zlib.h ]; then
if [ -d zlib-$ZLIB_VERSION ]; then
rm -rf zlib-$ZLIB_VERSION
@@ -895,7 +926,7 @@ if [ ! -f $PREFIX/include/zlib.h ]; then
popd && popd
fi
# install zstd
log_tool_name "zstd $ZSTD_VERSION"
if [ ! -f $PREFIX/include/zstd.h ]; then
if [ -d zstd-$ZSTD_VERSION ]; then
rm -rf zstd-$ZSTD_VERSION
@@ -910,7 +941,8 @@ if [ ! -f $PREFIX/include/zstd.h ]; then
popd && popd
fi
#install jemalloc
# TODO(gitbuda): Freeze jmalloc version.
log_tool_name "jmalloc"
if [ ! -d $PREFIX/include/jemalloc ]; then
if [ -d jemalloc ]; then
rm -rf jemalloc
@@ -927,7 +959,7 @@ if [ ! -d $PREFIX/include/jemalloc ]; then
popd
fi
# install boost
log_tool_name "BOOST $BOOST_VERSION"
if [ ! -d $PREFIX/include/boost ]; then
if [ -d boost_$BOOST_VERSION_UNDERSCORES ]; then
rm -rf boost_$BOOST_VERSION_UNDERSCORES
@@ -935,15 +967,24 @@ if [ ! -d $PREFIX/include/boost ]; then
tar -xzf ../archives/boost_$BOOST_VERSION_UNDERSCORES.tar.gz
pushd boost_$BOOST_VERSION_UNDERSCORES
./bootstrap.sh --prefix=$PREFIX --with-toolset=clang --with-python=python3 --without-icu
./b2 toolset=clang -j$CPUS install variant=release link=static cxxstd=20 --disable-icu \
-sZLIB_SOURCE="$PREFIX" -sZLIB_INCLUDE="$PREFIX/include" -sZLIB_LIBPATH="$PREFIX/lib" \
-sBZIP2_SOURCE="$PREFIX" -sBZIP2_INCLUDE="$PREFIX/include" -sBZIP2_LIBPATH="$PREFIX/lib" \
-sLZMA_SOURCE="$PREFIX" -sLZMA_INCLUDE="$PREFIX/include" -sLZMA_LIBPATH="$PREFIX/lib" \
-sZSTD_SOURCE="$PREFIX" -sZSTD_INCLUDE="$PREFIX/include" -sZSTD_LIBPATH="$PREFIX/lib"
if [ "$TOOLCHAIN_STDCXX" = "libstdc++" ]; then
./b2 toolset=clang -j$CPUS install variant=release link=static cxxstd=20 --disable-icu \
-sZLIB_SOURCE="$PREFIX" -sZLIB_INCLUDE="$PREFIX/include" -sZLIB_LIBPATH="$PREFIX/lib" \
-sBZIP2_SOURCE="$PREFIX" -sBZIP2_INCLUDE="$PREFIX/include" -sBZIP2_LIBPATH="$PREFIX/lib" \
-sLZMA_SOURCE="$PREFIX" -sLZMA_INCLUDE="$PREFIX/include" -sLZMA_LIBPATH="$PREFIX/lib" \
-sZSTD_SOURCE="$PREFIX" -sZSTD_INCLUDE="$PREFIX/include" -sZSTD_LIBPATH="$PREFIX/lib"
else
./b2 toolset=clang -j$CPUS install variant=release link=static cxxstd=20 --disable-icu \
cxxflags="-stdlib=libc++" linkflags="-stdlib=libc++" \
-sZLIB_SOURCE="$PREFIX" -sZLIB_INCLUDE="$PREFIX/include" -sZLIB_LIBPATH="$PREFIX/lib" \
-sBZIP2_SOURCE="$PREFIX" -sBZIP2_INCLUDE="$PREFIX/include" -sBZIP2_LIBPATH="$PREFIX/lib" \
-sLZMA_SOURCE="$PREFIX" -sLZMA_INCLUDE="$PREFIX/include" -sLZMA_LIBPATH="$PREFIX/lib" \
-sZSTD_SOURCE="$PREFIX" -sZSTD_INCLUDE="$PREFIX/include" -sZSTD_LIBPATH="$PREFIX/lib"
fi
popd
fi
# install double-conversion
log_tool_name "double-conversion $DOUBLE_CONVERSION_VERSION"
if [ ! -d $PREFIX/include/double-conversion ]; then
if [ -d double-conversion-$DOUBLE_CONVERSION_VERSION ]; then
rm -rf double-conversion-$DOUBLE_CONVERSION_VERSION
@@ -958,7 +999,8 @@ if [ ! -d $PREFIX/include/double-conversion ]; then
popd && popd
fi
# install gflags
# TODO(gitbuda): Freeze gflags version.
log_tool_name "gflags"
if [ ! -d $PREFIX/include/gflags ]; then
if [ -d gflags ]; then
rm -rf gflags
@@ -977,7 +1019,7 @@ if [ ! -d $PREFIX/include/gflags ]; then
popd && popd
fi
# install libunwind
log_tool_name "libunwind $LIBUNWIND_VERSION"
if [ ! -f $PREFIX/include/libunwind.h ]; then
if [ -d libunwind-$LIBUNWIND_VERSION ]; then
rm -rf libunwind-$LIBUNWIND_VERSION
@@ -990,7 +1032,7 @@ if [ ! -f $PREFIX/include/libunwind.h ]; then
popd
fi
# install glog
log_tool_name "glog $GLOG_VERSION"
if [ ! -d $PREFIX/include/glog ]; then
if [ -d glog-$GLOG_VERSION ]; then
rm -rf glog-$GLOG_VERSION
@@ -1004,7 +1046,7 @@ if [ ! -d $PREFIX/include/glog ]; then
popd && popd
fi
# install libevent
log_tool_name "libevent $LIBEVENT_VERSION"
if [ ! -d $PREFIX/include/event2 ]; then
if [ -d libevent-$LIBEVENT_VERSION ]; then
rm -rf libevent-$LIBEVENT_VERSION
@@ -1023,7 +1065,7 @@ if [ ! -d $PREFIX/include/event2 ]; then
popd && popd
fi
# install snappy
log_tool_name "snappy $SNAPPY_VERSION"
if [ ! -f $PREFIX/include/snappy.h ]; then
if [ -d snappy-$SNAPPY_VERSION ]; then
rm -rf snappy-$SNAPPY_VERSION
@@ -1041,7 +1083,7 @@ if [ ! -f $PREFIX/include/snappy.h ]; then
popd && popd
fi
# install libsodium
log_tool_name "libsodium $LIBSODIUM_VERSION"
if [ ! -f $PREFIX/include/sodium.h ]; then
if [ -d libsodium-$LIBSODIUM_VERSION ]; then
rm -rf libsodium-$LIBSODIUM_VERSION
@@ -1053,7 +1095,7 @@ if [ ! -f $PREFIX/include/sodium.h ]; then
popd
fi
# install libaio
log_tool_name "libaio $LIBAIO_VERSION"
if [ ! -f $PREFIX/include/libaio.h ]; then
if [ -d libaio-$LIBAIO_VERSION ]; then
rm -rf libaio-$LIBAIO_VERSION
@@ -1064,7 +1106,7 @@ if [ ! -f $PREFIX/include/libaio.h ]; then
popd
fi
# install folly
log_tool_name "folly $FBLIBS_VERSION"
if [ ! -d $PREFIX/include/folly ]; then
if [ -d folly-$FBLIBS_VERSION ]; then
rm -rf folly-$FBLIBS_VERSION
@@ -1085,7 +1127,7 @@ if [ ! -d $PREFIX/include/folly ]; then
popd && popd
fi
# install fizz
log_tool_name "fizz $FBLIBS_VERSION"
if [ ! -d $PREFIX/include/fizz ]; then
if [ -d fizz-$FBLIBS_VERSION ]; then
rm -rf fizz-$FBLIBS_VERSION
@@ -1104,7 +1146,7 @@ if [ ! -d $PREFIX/include/fizz ]; then
popd && popd
fi
# install wangle
log_tool_name "wangle FBLIBS_VERSION"
if [ ! -d $PREFIX/include/wangle ]; then
if [ -d wangle-$FBLIBS_VERSION ]; then
rm -rf wangle-$FBLIBS_VERSION
@@ -1123,7 +1165,7 @@ if [ ! -d $PREFIX/include/wangle ]; then
popd && popd
fi
# install proxygen
log_tool_name "proxygen $FBLIBS_VERSION"
if [ ! -d $PREFIX/include/proxygen ]; then
if [ -d proxygen-$FBLIBS_VERSION ]; then
rm -rf proxygen-$FBLIBS_VERSION
@@ -1144,7 +1186,7 @@ if [ ! -d $PREFIX/include/proxygen ]; then
popd && popd
fi
# install flex
log_tool_name "flex $FBLIBS_VERSION"
if [ ! -f $PREFIX/include/FlexLexer.h ]; then
if [ -d flex-$FLEX_VERSION ]; then
rm -rf flex-$FLEX_VERSION
@@ -1156,7 +1198,7 @@ if [ ! -f $PREFIX/include/FlexLexer.h ]; then
popd
fi
# install fbthrift
log_tool_name "fbthrift $FBLIBS_VERSION"
if [ ! -d $PREFIX/include/thrift ]; then
if [ -d fbthrift-$FBLIBS_VERSION ]; then
rm -rf fbthrift-$FBLIBS_VERSION
@@ -1166,10 +1208,15 @@ if [ ! -d $PREFIX/include/thrift ]; then
# build is used by facebook builder
mkdir _build
pushd _build
if [ "$TOOLCHAIN_STDCXX" = "libstdc++" ]; then
CMAKE_CXX_FLAGS="-fsized-deallocation"
else
CMAKE_CXX_FLAGS="-fsized-deallocation -stdlib=libc++"
fi
cmake .. $COMMON_CMAKE_FLAGS \
-Denable_tests=OFF \
-DGFLAGS_NOTHREADS=OFF \
-DCMAKE_CXX_FLAGS=-fsized-deallocation
-DCMAKE_CXX_FLAGS="$CMAKE_CXX_FLAGS"
make -j$CPUS install
popd
fi
@@ -1192,7 +1239,12 @@ if [ ! -f $NAME-binaries-$DISTRO.tar.gz ]; then
DISTRO_FULL_NAME="$DISTRO_FULL_NAME-amd64"
fi
fi
if [ "$TOOLCHAIN_STDCXX" = "libstdc++" ]; then
# Pass because infra scripts assume there is not C++ standard lib in the name.
echo "NOTE: Not adding anything to the archive name that GCC C++ standard lib is used."
else
DISTRO_FULL_NAME="$DISTRO_FULL_NAME-libc++"
fi
tar --owner=root --group=root -cpvzf $NAME-binaries-$DISTRO_FULL_NAME.tar.gz -C /opt $NAME
fi

View File

@@ -22,7 +22,7 @@ check_architecture() {
for arch in "$@"; do
if [ "$(architecture)" = "$arch" ]; then
echo "The right architecture!"
exit 0
return 0
fi
done
echo "Not the right architecture!"

View File

@@ -22,7 +22,7 @@
namespace mgp {
namespace {
void MgExceptionHandle(mgp_error result_code) {
inline void MgExceptionHandle(mgp_error result_code) {
switch (result_code) {
case mgp_error::MGP_ERROR_UNKNOWN_ERROR:
throw mg_exception::UnknownException();
@@ -62,7 +62,7 @@ TResult MgInvoke(TFunc func, TArgs... args) {
}
template <typename TFunc, typename... TArgs>
void MgInvokeVoid(TFunc func, TArgs... args) {
inline void MgInvokeVoid(TFunc func, TArgs... args) {
auto result_code = func(args...);
MgExceptionHandle(result_code);
}
@@ -72,211 +72,221 @@ void MgInvokeVoid(TFunc func, TArgs... args) {
// Make value
mgp_value *value_make_null(mgp_memory *memory) { return MgInvoke<mgp_value *>(mgp_value_make_null, memory); }
inline mgp_value *value_make_null(mgp_memory *memory) { return MgInvoke<mgp_value *>(mgp_value_make_null, memory); }
mgp_value *value_make_bool(int val, mgp_memory *memory) {
inline mgp_value *value_make_bool(int val, mgp_memory *memory) {
return MgInvoke<mgp_value *>(mgp_value_make_bool, val, memory);
}
mgp_value *value_make_int(int64_t val, mgp_memory *memory) {
inline mgp_value *value_make_int(int64_t val, mgp_memory *memory) {
return MgInvoke<mgp_value *>(mgp_value_make_int, val, memory);
}
mgp_value *value_make_double(double val, mgp_memory *memory) {
inline mgp_value *value_make_double(double val, mgp_memory *memory) {
return MgInvoke<mgp_value *>(mgp_value_make_double, val, memory);
}
mgp_value *value_make_string(const char *val, mgp_memory *memory) {
inline mgp_value *value_make_string(const char *val, mgp_memory *memory) {
return MgInvoke<mgp_value *>(mgp_value_make_string, val, memory);
}
mgp_value *value_make_list(mgp_list *val) { return MgInvoke<mgp_value *>(mgp_value_make_list, val); }
inline mgp_value *value_make_list(mgp_list *val) { return MgInvoke<mgp_value *>(mgp_value_make_list, val); }
mgp_value *value_make_map(mgp_map *val) { return MgInvoke<mgp_value *>(mgp_value_make_map, val); }
inline mgp_value *value_make_map(mgp_map *val) { return MgInvoke<mgp_value *>(mgp_value_make_map, val); }
mgp_value *value_make_vertex(mgp_vertex *val) { return MgInvoke<mgp_value *>(mgp_value_make_vertex, val); }
inline mgp_value *value_make_vertex(mgp_vertex *val) { return MgInvoke<mgp_value *>(mgp_value_make_vertex, val); }
mgp_value *value_make_edge(mgp_edge *val) { return MgInvoke<mgp_value *>(mgp_value_make_edge, val); }
inline mgp_value *value_make_edge(mgp_edge *val) { return MgInvoke<mgp_value *>(mgp_value_make_edge, val); }
mgp_value *value_make_path(mgp_path *val) { return MgInvoke<mgp_value *>(mgp_value_make_path, val); }
inline mgp_value *value_make_path(mgp_path *val) { return MgInvoke<mgp_value *>(mgp_value_make_path, val); }
mgp_value *value_make_date(mgp_date *val) { return MgInvoke<mgp_value *>(mgp_value_make_date, val); }
inline mgp_value *value_make_date(mgp_date *val) { return MgInvoke<mgp_value *>(mgp_value_make_date, val); }
mgp_value *value_make_local_time(mgp_local_time *val) { return MgInvoke<mgp_value *>(mgp_value_make_local_time, val); }
inline mgp_value *value_make_local_time(mgp_local_time *val) {
return MgInvoke<mgp_value *>(mgp_value_make_local_time, val);
}
mgp_value *value_make_local_date_time(mgp_local_date_time *val) {
inline mgp_value *value_make_local_date_time(mgp_local_date_time *val) {
return MgInvoke<mgp_value *>(mgp_value_make_local_date_time, val);
}
mgp_value *value_make_duration(mgp_duration *val) { return MgInvoke<mgp_value *>(mgp_value_make_duration, val); }
inline mgp_value *value_make_duration(mgp_duration *val) { return MgInvoke<mgp_value *>(mgp_value_make_duration, val); }
// Copy value
// TODO: implement within MGP API
// with primitive types ({bool, int, double, string}), create a new identical value
// otherwise call mgp_##TYPE_copy and convert tpye
mgp_value *value_copy(mgp_value *val, mgp_memory *memory) { return MgInvoke<mgp_value *>(mgp_value_copy, val, memory); }
inline mgp_value *value_copy(mgp_value *val, mgp_memory *memory) {
return MgInvoke<mgp_value *>(mgp_value_copy, val, memory);
}
// Destroy value
void value_destroy(mgp_value *val) { mgp_value_destroy(val); }
inline void value_destroy(mgp_value *val) { mgp_value_destroy(val); }
// Get value of type
mgp_value_type value_get_type(mgp_value *val) { return MgInvoke<mgp_value_type>(mgp_value_get_type, val); }
inline mgp_value_type value_get_type(mgp_value *val) { return MgInvoke<mgp_value_type>(mgp_value_get_type, val); }
bool value_get_bool(mgp_value *val) { return MgInvoke<int>(mgp_value_get_bool, val); }
inline bool value_get_bool(mgp_value *val) { return MgInvoke<int>(mgp_value_get_bool, val); }
int64_t value_get_int(mgp_value *val) { return MgInvoke<int64_t>(mgp_value_get_int, val); }
inline int64_t value_get_int(mgp_value *val) { return MgInvoke<int64_t>(mgp_value_get_int, val); }
double value_get_double(mgp_value *val) { return MgInvoke<double>(mgp_value_get_double, val); }
inline double value_get_double(mgp_value *val) { return MgInvoke<double>(mgp_value_get_double, val); }
const char *value_get_string(mgp_value *val) { return MgInvoke<const char *>(mgp_value_get_string, val); }
inline const char *value_get_string(mgp_value *val) { return MgInvoke<const char *>(mgp_value_get_string, val); }
mgp_list *value_get_list(mgp_value *val) { return MgInvoke<mgp_list *>(mgp_value_get_list, val); }
inline mgp_list *value_get_list(mgp_value *val) { return MgInvoke<mgp_list *>(mgp_value_get_list, val); }
mgp_map *value_get_map(mgp_value *val) { return MgInvoke<mgp_map *>(mgp_value_get_map, val); }
inline mgp_map *value_get_map(mgp_value *val) { return MgInvoke<mgp_map *>(mgp_value_get_map, val); }
mgp_vertex *value_get_vertex(mgp_value *val) { return MgInvoke<mgp_vertex *>(mgp_value_get_vertex, val); }
inline mgp_vertex *value_get_vertex(mgp_value *val) { return MgInvoke<mgp_vertex *>(mgp_value_get_vertex, val); }
mgp_edge *value_get_edge(mgp_value *val) { return MgInvoke<mgp_edge *>(mgp_value_get_edge, val); }
inline mgp_edge *value_get_edge(mgp_value *val) { return MgInvoke<mgp_edge *>(mgp_value_get_edge, val); }
mgp_path *value_get_path(mgp_value *val) { return MgInvoke<mgp_path *>(mgp_value_get_path, val); }
inline mgp_path *value_get_path(mgp_value *val) { return MgInvoke<mgp_path *>(mgp_value_get_path, val); }
mgp_date *value_get_date(mgp_value *val) { return MgInvoke<mgp_date *>(mgp_value_get_date, val); }
inline mgp_date *value_get_date(mgp_value *val) { return MgInvoke<mgp_date *>(mgp_value_get_date, val); }
mgp_local_time *value_get_local_time(mgp_value *val) {
inline mgp_local_time *value_get_local_time(mgp_value *val) {
return MgInvoke<mgp_local_time *>(mgp_value_get_local_time, val);
}
mgp_local_date_time *value_get_local_date_time(mgp_value *val) {
inline mgp_local_date_time *value_get_local_date_time(mgp_value *val) {
return MgInvoke<mgp_local_date_time *>(mgp_value_get_local_date_time, val);
}
mgp_duration *value_get_duration(mgp_value *val) { return MgInvoke<mgp_duration *>(mgp_value_get_duration, val); }
inline mgp_duration *value_get_duration(mgp_value *val) {
return MgInvoke<mgp_duration *>(mgp_value_get_duration, val);
}
// Check type of value
bool value_is_null(mgp_value *val) { return MgInvoke<int>(mgp_value_is_null, val); }
inline bool value_is_null(mgp_value *val) { return MgInvoke<int>(mgp_value_is_null, val); }
bool value_is_bool(mgp_value *val) { return MgInvoke<int>(mgp_value_is_bool, val); }
inline bool value_is_bool(mgp_value *val) { return MgInvoke<int>(mgp_value_is_bool, val); }
bool value_is_int(mgp_value *val) { return MgInvoke<int>(mgp_value_is_int, val); }
inline bool value_is_int(mgp_value *val) { return MgInvoke<int>(mgp_value_is_int, val); }
bool value_is_double(mgp_value *val) { return MgInvoke<int>(mgp_value_is_double, val); }
inline bool value_is_double(mgp_value *val) { return MgInvoke<int>(mgp_value_is_double, val); }
bool value_is_string(mgp_value *val) { return MgInvoke<int>(mgp_value_is_string, val); }
inline bool value_is_string(mgp_value *val) { return MgInvoke<int>(mgp_value_is_string, val); }
bool value_is_list(mgp_value *val) { return MgInvoke<int>(mgp_value_is_list, val); }
inline bool value_is_list(mgp_value *val) { return MgInvoke<int>(mgp_value_is_list, val); }
bool value_is_map(mgp_value *val) { return MgInvoke<int>(mgp_value_is_map, val); }
inline bool value_is_map(mgp_value *val) { return MgInvoke<int>(mgp_value_is_map, val); }
bool value_is_vertex(mgp_value *val) { return MgInvoke<int>(mgp_value_is_vertex, val); }
inline bool value_is_vertex(mgp_value *val) { return MgInvoke<int>(mgp_value_is_vertex, val); }
bool value_is_edge(mgp_value *val) { return MgInvoke<int>(mgp_value_is_edge, val); }
inline bool value_is_edge(mgp_value *val) { return MgInvoke<int>(mgp_value_is_edge, val); }
bool value_is_path(mgp_value *val) { return MgInvoke<int>(mgp_value_is_path, val); }
inline bool value_is_path(mgp_value *val) { return MgInvoke<int>(mgp_value_is_path, val); }
bool value_is_date(mgp_value *val) { return MgInvoke<int>(mgp_value_is_date, val); }
inline bool value_is_date(mgp_value *val) { return MgInvoke<int>(mgp_value_is_date, val); }
bool value_is_local_time(mgp_value *val) { return MgInvoke<int>(mgp_value_is_local_time, val); }
inline bool value_is_local_time(mgp_value *val) { return MgInvoke<int>(mgp_value_is_local_time, val); }
bool value_is_local_date_time(mgp_value *val) { return MgInvoke<int>(mgp_value_is_local_date_time, val); }
inline bool value_is_local_date_time(mgp_value *val) { return MgInvoke<int>(mgp_value_is_local_date_time, val); }
bool value_is_duration(mgp_value *val) { return MgInvoke<int>(mgp_value_is_duration, val); }
inline bool value_is_duration(mgp_value *val) { return MgInvoke<int>(mgp_value_is_duration, val); }
// Get type
mgp_type *type_any() { return MgInvoke<mgp_type *>(mgp_type_any); }
inline mgp_type *type_any() { return MgInvoke<mgp_type *>(mgp_type_any); }
mgp_type *type_bool() { return MgInvoke<mgp_type *>(mgp_type_bool); }
inline mgp_type *type_bool() { return MgInvoke<mgp_type *>(mgp_type_bool); }
mgp_type *type_string() { return MgInvoke<mgp_type *>(mgp_type_string); }
inline mgp_type *type_string() { return MgInvoke<mgp_type *>(mgp_type_string); }
mgp_type *type_int() { return MgInvoke<mgp_type *>(mgp_type_int); }
inline mgp_type *type_int() { return MgInvoke<mgp_type *>(mgp_type_int); }
mgp_type *type_float() { return MgInvoke<mgp_type *>(mgp_type_float); }
inline mgp_type *type_float() { return MgInvoke<mgp_type *>(mgp_type_float); }
mgp_type *type_number() { return MgInvoke<mgp_type *>(mgp_type_number); }
inline mgp_type *type_number() { return MgInvoke<mgp_type *>(mgp_type_number); }
mgp_type *type_list(mgp_type *element_type) { return MgInvoke<mgp_type *>(mgp_type_list, element_type); }
inline mgp_type *type_list(mgp_type *element_type) { return MgInvoke<mgp_type *>(mgp_type_list, element_type); }
mgp_type *type_map() { return MgInvoke<mgp_type *>(mgp_type_map); }
inline mgp_type *type_map() { return MgInvoke<mgp_type *>(mgp_type_map); }
mgp_type *type_node() { return MgInvoke<mgp_type *>(mgp_type_node); }
inline mgp_type *type_node() { return MgInvoke<mgp_type *>(mgp_type_node); }
mgp_type *type_relationship() { return MgInvoke<mgp_type *>(mgp_type_relationship); }
inline mgp_type *type_relationship() { return MgInvoke<mgp_type *>(mgp_type_relationship); }
mgp_type *type_path() { return MgInvoke<mgp_type *>(mgp_type_path); }
inline mgp_type *type_path() { return MgInvoke<mgp_type *>(mgp_type_path); }
mgp_type *type_date() { return MgInvoke<mgp_type *>(mgp_type_date); }
inline mgp_type *type_date() { return MgInvoke<mgp_type *>(mgp_type_date); }
mgp_type *type_local_time() { return MgInvoke<mgp_type *>(mgp_type_local_time); }
inline mgp_type *type_local_time() { return MgInvoke<mgp_type *>(mgp_type_local_time); }
mgp_type *type_local_date_time() { return MgInvoke<mgp_type *>(mgp_type_local_date_time); }
inline mgp_type *type_local_date_time() { return MgInvoke<mgp_type *>(mgp_type_local_date_time); }
mgp_type *type_duration() { return MgInvoke<mgp_type *>(mgp_type_duration); }
inline mgp_type *type_duration() { return MgInvoke<mgp_type *>(mgp_type_duration); }
mgp_type *type_nullable(mgp_type *type) { return MgInvoke<mgp_type *>(mgp_type_nullable, type); }
inline mgp_type *type_nullable(mgp_type *type) { return MgInvoke<mgp_type *>(mgp_type_nullable, type); }
// mgp_graph
bool graph_is_mutable(mgp_graph *graph) { return MgInvoke<int>(mgp_graph_is_mutable, graph); }
inline bool graph_is_mutable(mgp_graph *graph) { return MgInvoke<int>(mgp_graph_is_mutable, graph); }
mgp_vertex *graph_create_vertex(mgp_graph *graph, mgp_memory *memory) {
inline mgp_vertex *graph_create_vertex(mgp_graph *graph, mgp_memory *memory) {
return MgInvoke<mgp_vertex *>(mgp_graph_create_vertex, graph, memory);
}
void graph_delete_vertex(mgp_graph *graph, mgp_vertex *vertex) { MgInvokeVoid(mgp_graph_delete_vertex, graph, vertex); }
inline void graph_delete_vertex(mgp_graph *graph, mgp_vertex *vertex) {
MgInvokeVoid(mgp_graph_delete_vertex, graph, vertex);
}
void graph_detach_delete_vertex(mgp_graph *graph, mgp_vertex *vertex) {
inline void graph_detach_delete_vertex(mgp_graph *graph, mgp_vertex *vertex) {
MgInvokeVoid(mgp_graph_detach_delete_vertex, graph, vertex);
}
mgp_edge *graph_create_edge(mgp_graph *graph, mgp_vertex *from, mgp_vertex *to, mgp_edge_type type,
mgp_memory *memory) {
inline mgp_edge *graph_create_edge(mgp_graph *graph, mgp_vertex *from, mgp_vertex *to, mgp_edge_type type,
mgp_memory *memory) {
return MgInvoke<mgp_edge *>(mgp_graph_create_edge, graph, from, to, type, memory);
}
void graph_delete_edge(mgp_graph *graph, mgp_edge *edge) { MgInvokeVoid(mgp_graph_delete_edge, graph, edge); }
inline void graph_delete_edge(mgp_graph *graph, mgp_edge *edge) { MgInvokeVoid(mgp_graph_delete_edge, graph, edge); }
mgp_vertex *graph_get_vertex_by_id(mgp_graph *g, mgp_vertex_id id, mgp_memory *memory) {
inline mgp_vertex *graph_get_vertex_by_id(mgp_graph *g, mgp_vertex_id id, mgp_memory *memory) {
return MgInvoke<mgp_vertex *>(mgp_graph_get_vertex_by_id, g, id, memory);
}
mgp_vertices_iterator *graph_iter_vertices(mgp_graph *g, mgp_memory *memory) {
inline mgp_vertices_iterator *graph_iter_vertices(mgp_graph *g, mgp_memory *memory) {
return MgInvoke<mgp_vertices_iterator *>(mgp_graph_iter_vertices, g, memory);
}
// mgp_vertices_iterator
void vertices_iterator_destroy(mgp_vertices_iterator *it) { mgp_vertices_iterator_destroy(it); }
inline void vertices_iterator_destroy(mgp_vertices_iterator *it) { mgp_vertices_iterator_destroy(it); }
mgp_vertex *vertices_iterator_get(mgp_vertices_iterator *it) {
inline mgp_vertex *vertices_iterator_get(mgp_vertices_iterator *it) {
return MgInvoke<mgp_vertex *>(mgp_vertices_iterator_get, it);
}
mgp_vertex *vertices_iterator_next(mgp_vertices_iterator *it) {
inline mgp_vertex *vertices_iterator_next(mgp_vertices_iterator *it) {
return MgInvoke<mgp_vertex *>(mgp_vertices_iterator_next, it);
}
// mgp_edges_iterator
void edges_iterator_destroy(mgp_edges_iterator *it) { mgp_edges_iterator_destroy(it); }
inline void edges_iterator_destroy(mgp_edges_iterator *it) { mgp_edges_iterator_destroy(it); }
mgp_edge *edges_iterator_get(mgp_edges_iterator *it) { return MgInvoke<mgp_edge *>(mgp_edges_iterator_get, it); }
inline mgp_edge *edges_iterator_get(mgp_edges_iterator *it) { return MgInvoke<mgp_edge *>(mgp_edges_iterator_get, it); }
mgp_edge *edges_iterator_next(mgp_edges_iterator *it) { return MgInvoke<mgp_edge *>(mgp_edges_iterator_next, it); }
inline mgp_edge *edges_iterator_next(mgp_edges_iterator *it) {
return MgInvoke<mgp_edge *>(mgp_edges_iterator_next, it);
}
// mgp_properties_iterator
void properties_iterator_destroy(mgp_properties_iterator *it) { mgp_properties_iterator_destroy(it); }
inline void properties_iterator_destroy(mgp_properties_iterator *it) { mgp_properties_iterator_destroy(it); }
mgp_property *properties_iterator_get(mgp_properties_iterator *it) {
inline mgp_property *properties_iterator_get(mgp_properties_iterator *it) {
return MgInvoke<mgp_property *>(mgp_properties_iterator_get, it);
}
mgp_property *properties_iterator_next(mgp_properties_iterator *it) {
inline mgp_property *properties_iterator_next(mgp_properties_iterator *it) {
return MgInvoke<mgp_property *>(mgp_properties_iterator_next, it);
}
@@ -284,409 +294,432 @@ mgp_property *properties_iterator_next(mgp_properties_iterator *it) {
// mgp_list
mgp_list *list_make_empty(size_t capacity, mgp_memory *memory) {
inline mgp_list *list_make_empty(size_t capacity, mgp_memory *memory) {
return MgInvoke<mgp_list *>(mgp_list_make_empty, capacity, memory);
}
mgp_list *list_copy(mgp_list *list, mgp_memory *memory) { return MgInvoke<mgp_list *>(mgp_list_copy, list, memory); }
inline mgp_list *list_copy(mgp_list *list, mgp_memory *memory) {
return MgInvoke<mgp_list *>(mgp_list_copy, list, memory);
}
void list_destroy(mgp_list *list) { mgp_list_destroy(list); }
inline void list_destroy(mgp_list *list) { mgp_list_destroy(list); }
void list_append(mgp_list *list, mgp_value *val) { MgInvokeVoid(mgp_list_append, list, val); }
inline void list_append(mgp_list *list, mgp_value *val) { MgInvokeVoid(mgp_list_append, list, val); }
void list_append_extend(mgp_list *list, mgp_value *val) { MgInvokeVoid(mgp_list_append_extend, list, val); }
inline void list_append_extend(mgp_list *list, mgp_value *val) { MgInvokeVoid(mgp_list_append_extend, list, val); }
size_t list_size(mgp_list *list) { return MgInvoke<size_t>(mgp_list_size, list); }
inline size_t list_size(mgp_list *list) { return MgInvoke<size_t>(mgp_list_size, list); }
size_t list_capacity(mgp_list *list) { return MgInvoke<size_t>(mgp_list_capacity, list); }
inline size_t list_capacity(mgp_list *list) { return MgInvoke<size_t>(mgp_list_capacity, list); }
mgp_value *list_at(mgp_list *list, size_t index) { return MgInvoke<mgp_value *>(mgp_list_at, list, index); }
inline mgp_value *list_at(mgp_list *list, size_t index) { return MgInvoke<mgp_value *>(mgp_list_at, list, index); }
// mgp_map
mgp_map *map_make_empty(mgp_memory *memory) { return MgInvoke<mgp_map *>(mgp_map_make_empty, memory); }
inline mgp_map *map_make_empty(mgp_memory *memory) { return MgInvoke<mgp_map *>(mgp_map_make_empty, memory); }
mgp_map *map_copy(mgp_map *map, mgp_memory *memory) { return MgInvoke<mgp_map *>(mgp_map_copy, map, memory); }
inline mgp_map *map_copy(mgp_map *map, mgp_memory *memory) { return MgInvoke<mgp_map *>(mgp_map_copy, map, memory); }
void map_destroy(mgp_map *map) { mgp_map_destroy(map); }
inline void map_destroy(mgp_map *map) { mgp_map_destroy(map); }
void map_insert(mgp_map *map, const char *key, mgp_value *value) { MgInvokeVoid(mgp_map_insert, map, key, value); }
inline void map_insert(mgp_map *map, const char *key, mgp_value *value) {
MgInvokeVoid(mgp_map_insert, map, key, value);
}
size_t map_size(mgp_map *map) { return MgInvoke<size_t>(mgp_map_size, map); }
inline size_t map_size(mgp_map *map) { return MgInvoke<size_t>(mgp_map_size, map); }
mgp_value *map_at(mgp_map *map, const char *key) { return MgInvoke<mgp_value *>(mgp_map_at, map, key); }
inline mgp_value *map_at(mgp_map *map, const char *key) { return MgInvoke<mgp_value *>(mgp_map_at, map, key); }
const char *map_item_key(mgp_map_item *item) { return MgInvoke<const char *>(mgp_map_item_key, item); }
inline const char *map_item_key(mgp_map_item *item) { return MgInvoke<const char *>(mgp_map_item_key, item); }
mgp_value *map_item_value(mgp_map_item *item) { return MgInvoke<mgp_value *>(mgp_map_item_value, item); }
inline mgp_value *map_item_value(mgp_map_item *item) { return MgInvoke<mgp_value *>(mgp_map_item_value, item); }
mgp_map_items_iterator *map_iter_items(mgp_map *map, mgp_memory *memory) {
inline mgp_map_items_iterator *map_iter_items(mgp_map *map, mgp_memory *memory) {
return MgInvoke<mgp_map_items_iterator *>(mgp_map_iter_items, map, memory);
}
void map_items_iterator_destroy(mgp_map_items_iterator *it) { mgp_map_items_iterator_destroy(it); }
inline void map_items_iterator_destroy(mgp_map_items_iterator *it) { mgp_map_items_iterator_destroy(it); }
mgp_map_item *map_items_iterator_get(mgp_map_items_iterator *it) {
inline mgp_map_item *map_items_iterator_get(mgp_map_items_iterator *it) {
return MgInvoke<mgp_map_item *>(mgp_map_items_iterator_get, it);
}
mgp_map_item *map_items_iterator_next(mgp_map_items_iterator *it) {
inline mgp_map_item *map_items_iterator_next(mgp_map_items_iterator *it) {
return MgInvoke<mgp_map_item *>(mgp_map_items_iterator_next, it);
}
// mgp_vertex
mgp_vertex_id vertex_get_id(mgp_vertex *v) { return MgInvoke<mgp_vertex_id>(mgp_vertex_get_id, v); }
inline mgp_vertex_id vertex_get_id(mgp_vertex *v) { return MgInvoke<mgp_vertex_id>(mgp_vertex_get_id, v); }
mgp_vertex *vertex_copy(mgp_vertex *v, mgp_memory *memory) {
inline mgp_vertex *vertex_copy(mgp_vertex *v, mgp_memory *memory) {
return MgInvoke<mgp_vertex *>(mgp_vertex_copy, v, memory);
}
void vertex_destroy(mgp_vertex *v) { mgp_vertex_destroy(v); }
inline void vertex_destroy(mgp_vertex *v) { mgp_vertex_destroy(v); }
bool vertex_equal(mgp_vertex *v1, mgp_vertex *v2) { return MgInvoke<int>(mgp_vertex_equal, v1, v2); }
inline bool vertex_equal(mgp_vertex *v1, mgp_vertex *v2) { return MgInvoke<int>(mgp_vertex_equal, v1, v2); }
size_t vertex_labels_count(mgp_vertex *v) { return MgInvoke<size_t>(mgp_vertex_labels_count, v); }
inline size_t vertex_labels_count(mgp_vertex *v) { return MgInvoke<size_t>(mgp_vertex_labels_count, v); }
mgp_label vertex_label_at(mgp_vertex *v, size_t index) { return MgInvoke<mgp_label>(mgp_vertex_label_at, v, index); }
inline mgp_label vertex_label_at(mgp_vertex *v, size_t index) {
return MgInvoke<mgp_label>(mgp_vertex_label_at, v, index);
}
bool vertex_has_label(mgp_vertex *v, mgp_label label) { return MgInvoke<int>(mgp_vertex_has_label, v, label); }
inline bool vertex_has_label(mgp_vertex *v, mgp_label label) { return MgInvoke<int>(mgp_vertex_has_label, v, label); }
bool vertex_has_label_named(mgp_vertex *v, const char *label_name) {
inline bool vertex_has_label_named(mgp_vertex *v, const char *label_name) {
return MgInvoke<int>(mgp_vertex_has_label_named, v, label_name);
}
void vertex_add_label(mgp_vertex *vertex, mgp_label label) { MgInvokeVoid(mgp_vertex_add_label, vertex, label); }
inline void vertex_add_label(mgp_vertex *vertex, mgp_label label) { MgInvokeVoid(mgp_vertex_add_label, vertex, label); }
mgp_value *vertex_get_property(mgp_vertex *v, const char *property_name, mgp_memory *memory) {
inline mgp_value *vertex_get_property(mgp_vertex *v, const char *property_name, mgp_memory *memory) {
return MgInvoke<mgp_value *>(mgp_vertex_get_property, v, property_name, memory);
}
mgp_properties_iterator *vertex_iter_properties(mgp_vertex *v, mgp_memory *memory) {
inline mgp_properties_iterator *vertex_iter_properties(mgp_vertex *v, mgp_memory *memory) {
return MgInvoke<mgp_properties_iterator *>(mgp_vertex_iter_properties, v, memory);
}
mgp_edges_iterator *vertex_iter_in_edges(mgp_vertex *v, mgp_memory *memory) {
inline mgp_edges_iterator *vertex_iter_in_edges(mgp_vertex *v, mgp_memory *memory) {
return MgInvoke<mgp_edges_iterator *>(mgp_vertex_iter_in_edges, v, memory);
}
mgp_edges_iterator *vertex_iter_out_edges(mgp_vertex *v, mgp_memory *memory) {
inline mgp_edges_iterator *vertex_iter_out_edges(mgp_vertex *v, mgp_memory *memory) {
return MgInvoke<mgp_edges_iterator *>(mgp_vertex_iter_out_edges, v, memory);
}
// mgp_edge
mgp_edge_id edge_get_id(mgp_edge *e) { return MgInvoke<mgp_edge_id>(mgp_edge_get_id, e); }
inline mgp_edge_id edge_get_id(mgp_edge *e) { return MgInvoke<mgp_edge_id>(mgp_edge_get_id, e); }
mgp_edge *edge_copy(mgp_edge *e, mgp_memory *memory) { return MgInvoke<mgp_edge *>(mgp_edge_copy, e, memory); }
inline mgp_edge *edge_copy(mgp_edge *e, mgp_memory *memory) { return MgInvoke<mgp_edge *>(mgp_edge_copy, e, memory); }
void edge_destroy(mgp_edge *e) { mgp_edge_destroy(e); }
inline void edge_destroy(mgp_edge *e) { mgp_edge_destroy(e); }
bool edge_equal(mgp_edge *e1, mgp_edge *e2) { return MgInvoke<int>(mgp_edge_equal, e1, e2); }
inline bool edge_equal(mgp_edge *e1, mgp_edge *e2) { return MgInvoke<int>(mgp_edge_equal, e1, e2); }
mgp_edge_type edge_get_type(mgp_edge *e) { return MgInvoke<mgp_edge_type>(mgp_edge_get_type, e); }
inline mgp_edge_type edge_get_type(mgp_edge *e) { return MgInvoke<mgp_edge_type>(mgp_edge_get_type, e); }
mgp_vertex *edge_get_from(mgp_edge *e) { return MgInvoke<mgp_vertex *>(mgp_edge_get_from, e); }
inline mgp_vertex *edge_get_from(mgp_edge *e) { return MgInvoke<mgp_vertex *>(mgp_edge_get_from, e); }
mgp_vertex *edge_get_to(mgp_edge *e) { return MgInvoke<mgp_vertex *>(mgp_edge_get_to, e); }
inline mgp_vertex *edge_get_to(mgp_edge *e) { return MgInvoke<mgp_vertex *>(mgp_edge_get_to, e); }
mgp_value *edge_get_property(mgp_edge *e, const char *property_name, mgp_memory *memory) {
inline mgp_value *edge_get_property(mgp_edge *e, const char *property_name, mgp_memory *memory) {
return MgInvoke<mgp_value *>(mgp_edge_get_property, e, property_name, memory);
}
mgp_properties_iterator *edge_iter_properties(mgp_edge *e, mgp_memory *memory) {
inline mgp_properties_iterator *edge_iter_properties(mgp_edge *e, mgp_memory *memory) {
return MgInvoke<mgp_properties_iterator *>(mgp_edge_iter_properties, e, memory);
}
// mgp_path
mgp_path *path_make_with_start(mgp_vertex *vertex, mgp_memory *memory) {
inline mgp_path *path_make_with_start(mgp_vertex *vertex, mgp_memory *memory) {
return MgInvoke<mgp_path *>(mgp_path_make_with_start, vertex, memory);
}
mgp_path *path_copy(mgp_path *path, mgp_memory *memory) { return MgInvoke<mgp_path *>(mgp_path_copy, path, memory); }
inline mgp_path *path_copy(mgp_path *path, mgp_memory *memory) {
return MgInvoke<mgp_path *>(mgp_path_copy, path, memory);
}
void path_destroy(mgp_path *path) { mgp_path_destroy(path); }
inline void path_destroy(mgp_path *path) { mgp_path_destroy(path); }
void path_expand(mgp_path *path, mgp_edge *edge) { MgInvokeVoid(mgp_path_expand, path, edge); }
inline void path_expand(mgp_path *path, mgp_edge *edge) { MgInvokeVoid(mgp_path_expand, path, edge); }
size_t path_size(mgp_path *path) { return MgInvoke<size_t>(mgp_path_size, path); }
inline size_t path_size(mgp_path *path) { return MgInvoke<size_t>(mgp_path_size, path); }
mgp_vertex *path_vertex_at(mgp_path *path, size_t index) {
inline mgp_vertex *path_vertex_at(mgp_path *path, size_t index) {
return MgInvoke<mgp_vertex *>(mgp_path_vertex_at, path, index);
}
mgp_edge *path_edge_at(mgp_path *path, size_t index) { return MgInvoke<mgp_edge *>(mgp_path_edge_at, path, index); }
inline mgp_edge *path_edge_at(mgp_path *path, size_t index) {
return MgInvoke<mgp_edge *>(mgp_path_edge_at, path, index);
}
bool path_equal(mgp_path *p1, mgp_path *p2) { return MgInvoke<int>(mgp_path_equal, p1, p2); }
inline bool path_equal(mgp_path *p1, mgp_path *p2) { return MgInvoke<int>(mgp_path_equal, p1, p2); }
// Temporal type {mgp_date, mgp_local_time, mgp_local_date_time, mgp_duration} methods
// mgp_date
mgp_date *date_from_string(const char *string, mgp_memory *memory) {
inline mgp_date *date_from_string(const char *string, mgp_memory *memory) {
return MgInvoke<mgp_date *>(mgp_date_from_string, string, memory);
}
mgp_date *date_from_parameters(mgp_date_parameters *parameters, mgp_memory *memory) {
inline mgp_date *date_from_parameters(mgp_date_parameters *parameters, mgp_memory *memory) {
return MgInvoke<mgp_date *>(mgp_date_from_parameters, parameters, memory);
}
mgp_date *date_copy(mgp_date *date, mgp_memory *memory) { return MgInvoke<mgp_date *>(mgp_date_copy, date, memory); }
inline mgp_date *date_copy(mgp_date *date, mgp_memory *memory) {
return MgInvoke<mgp_date *>(mgp_date_copy, date, memory);
}
void date_destroy(mgp_date *date) { mgp_date_destroy(date); }
inline void date_destroy(mgp_date *date) { mgp_date_destroy(date); }
bool date_equal(mgp_date *first, mgp_date *second) { return MgInvoke<int>(mgp_date_equal, first, second); }
inline bool date_equal(mgp_date *first, mgp_date *second) { return MgInvoke<int>(mgp_date_equal, first, second); }
int date_get_year(mgp_date *date) { return MgInvoke<int>(mgp_date_get_year, date); }
inline int date_get_year(mgp_date *date) { return MgInvoke<int>(mgp_date_get_year, date); }
int date_get_month(mgp_date *date) { return MgInvoke<int>(mgp_date_get_month, date); }
inline int date_get_month(mgp_date *date) { return MgInvoke<int>(mgp_date_get_month, date); }
int date_get_day(mgp_date *date) { return MgInvoke<int>(mgp_date_get_day, date); }
inline int date_get_day(mgp_date *date) { return MgInvoke<int>(mgp_date_get_day, date); }
int64_t date_timestamp(mgp_date *date) { return MgInvoke<int64_t>(mgp_date_timestamp, date); }
inline int64_t date_timestamp(mgp_date *date) { return MgInvoke<int64_t>(mgp_date_timestamp, date); }
mgp_date *date_now(mgp_memory *memory) { return MgInvoke<mgp_date *>(mgp_date_now, memory); }
inline mgp_date *date_now(mgp_memory *memory) { return MgInvoke<mgp_date *>(mgp_date_now, memory); }
mgp_date *date_add_duration(mgp_date *date, mgp_duration *dur, mgp_memory *memory) {
inline mgp_date *date_add_duration(mgp_date *date, mgp_duration *dur, mgp_memory *memory) {
return MgInvoke<mgp_date *>(mgp_date_add_duration, date, dur, memory);
}
mgp_date *date_sub_duration(mgp_date *date, mgp_duration *dur, mgp_memory *memory) {
inline mgp_date *date_sub_duration(mgp_date *date, mgp_duration *dur, mgp_memory *memory) {
return MgInvoke<mgp_date *>(mgp_date_sub_duration, date, dur, memory);
}
mgp_duration *date_diff(mgp_date *first, mgp_date *second, mgp_memory *memory) {
inline mgp_duration *date_diff(mgp_date *first, mgp_date *second, mgp_memory *memory) {
return MgInvoke<mgp_duration *>(mgp_date_diff, first, second, memory);
}
// mgp_local_time
mgp_local_time *local_time_from_string(const char *string, mgp_memory *memory) {
inline mgp_local_time *local_time_from_string(const char *string, mgp_memory *memory) {
return MgInvoke<mgp_local_time *>(mgp_local_time_from_string, string, memory);
}
mgp_local_time *local_time_from_parameters(mgp_local_time_parameters *parameters, mgp_memory *memory) {
inline mgp_local_time *local_time_from_parameters(mgp_local_time_parameters *parameters, mgp_memory *memory) {
return MgInvoke<mgp_local_time *>(mgp_local_time_from_parameters, parameters, memory);
}
mgp_local_time *local_time_copy(mgp_local_time *local_time, mgp_memory *memory) {
inline mgp_local_time *local_time_copy(mgp_local_time *local_time, mgp_memory *memory) {
return MgInvoke<mgp_local_time *>(mgp_local_time_copy, local_time, memory);
}
void local_time_destroy(mgp_local_time *local_time) { mgp_local_time_destroy(local_time); }
inline void local_time_destroy(mgp_local_time *local_time) { mgp_local_time_destroy(local_time); }
bool local_time_equal(mgp_local_time *first, mgp_local_time *second) {
inline bool local_time_equal(mgp_local_time *first, mgp_local_time *second) {
return MgInvoke<int>(mgp_local_time_equal, first, second);
}
int local_time_get_hour(mgp_local_time *local_time) { return MgInvoke<int>(mgp_local_time_get_hour, local_time); }
inline int local_time_get_hour(mgp_local_time *local_time) {
return MgInvoke<int>(mgp_local_time_get_hour, local_time);
}
int local_time_get_minute(mgp_local_time *local_time) { return MgInvoke<int>(mgp_local_time_get_minute, local_time); }
inline int local_time_get_minute(mgp_local_time *local_time) {
return MgInvoke<int>(mgp_local_time_get_minute, local_time);
}
int local_time_get_second(mgp_local_time *local_time) { return MgInvoke<int>(mgp_local_time_get_second, local_time); }
inline int local_time_get_second(mgp_local_time *local_time) {
return MgInvoke<int>(mgp_local_time_get_second, local_time);
}
int local_time_get_millisecond(mgp_local_time *local_time) {
inline int local_time_get_millisecond(mgp_local_time *local_time) {
return MgInvoke<int>(mgp_local_time_get_millisecond, local_time);
}
int local_time_get_microsecond(mgp_local_time *local_time) {
inline int local_time_get_microsecond(mgp_local_time *local_time) {
return MgInvoke<int>(mgp_local_time_get_microsecond, local_time);
}
int64_t local_time_timestamp(mgp_local_time *local_time) {
inline int64_t local_time_timestamp(mgp_local_time *local_time) {
return MgInvoke<int64_t>(mgp_local_time_timestamp, local_time);
}
mgp_local_time *local_time_now(mgp_memory *memory) { return MgInvoke<mgp_local_time *>(mgp_local_time_now, memory); }
inline mgp_local_time *local_time_now(mgp_memory *memory) {
return MgInvoke<mgp_local_time *>(mgp_local_time_now, memory);
}
mgp_local_time *local_time_add_duration(mgp_local_time *local_time, mgp_duration *dur, mgp_memory *memory) {
inline mgp_local_time *local_time_add_duration(mgp_local_time *local_time, mgp_duration *dur, mgp_memory *memory) {
return MgInvoke<mgp_local_time *>(mgp_local_time_add_duration, local_time, dur, memory);
}
mgp_local_time *local_time_sub_duration(mgp_local_time *local_time, mgp_duration *dur, mgp_memory *memory) {
inline mgp_local_time *local_time_sub_duration(mgp_local_time *local_time, mgp_duration *dur, mgp_memory *memory) {
return MgInvoke<mgp_local_time *>(mgp_local_time_sub_duration, local_time, dur, memory);
}
mgp_duration *local_time_diff(mgp_local_time *first, mgp_local_time *second, mgp_memory *memory) {
inline mgp_duration *local_time_diff(mgp_local_time *first, mgp_local_time *second, mgp_memory *memory) {
return MgInvoke<mgp_duration *>(mgp_local_time_diff, first, second, memory);
}
// mgp_local_date_time
mgp_local_date_time *local_date_time_from_string(const char *string, mgp_memory *memory) {
inline mgp_local_date_time *local_date_time_from_string(const char *string, mgp_memory *memory) {
return MgInvoke<mgp_local_date_time *>(mgp_local_date_time_from_string, string, memory);
}
mgp_local_date_time *local_date_time_from_parameters(mgp_local_date_time_parameters *parameters, mgp_memory *memory) {
inline mgp_local_date_time *local_date_time_from_parameters(mgp_local_date_time_parameters *parameters,
mgp_memory *memory) {
return MgInvoke<mgp_local_date_time *>(mgp_local_date_time_from_parameters, parameters, memory);
}
mgp_local_date_time *local_date_time_copy(mgp_local_date_time *local_date_time, mgp_memory *memory) {
inline mgp_local_date_time *local_date_time_copy(mgp_local_date_time *local_date_time, mgp_memory *memory) {
return MgInvoke<mgp_local_date_time *>(mgp_local_date_time_copy, local_date_time, memory);
}
void local_date_time_destroy(mgp_local_date_time *local_date_time) { mgp_local_date_time_destroy(local_date_time); }
inline void local_date_time_destroy(mgp_local_date_time *local_date_time) {
mgp_local_date_time_destroy(local_date_time);
}
bool local_date_time_equal(mgp_local_date_time *first, mgp_local_date_time *second) {
inline bool local_date_time_equal(mgp_local_date_time *first, mgp_local_date_time *second) {
return MgInvoke<int>(mgp_local_date_time_equal, first, second);
}
int local_date_time_get_year(mgp_local_date_time *local_date_time) {
inline int local_date_time_get_year(mgp_local_date_time *local_date_time) {
return MgInvoke<int>(mgp_local_date_time_get_year, local_date_time);
}
int local_date_time_get_month(mgp_local_date_time *local_date_time) {
inline int local_date_time_get_month(mgp_local_date_time *local_date_time) {
return MgInvoke<int>(mgp_local_date_time_get_month, local_date_time);
}
int local_date_time_get_day(mgp_local_date_time *local_date_time) {
inline int local_date_time_get_day(mgp_local_date_time *local_date_time) {
return MgInvoke<int>(mgp_local_date_time_get_day, local_date_time);
}
int local_date_time_get_hour(mgp_local_date_time *local_date_time) {
inline int local_date_time_get_hour(mgp_local_date_time *local_date_time) {
return MgInvoke<int>(mgp_local_date_time_get_hour, local_date_time);
}
int local_date_time_get_minute(mgp_local_date_time *local_date_time) {
inline int local_date_time_get_minute(mgp_local_date_time *local_date_time) {
return MgInvoke<int>(mgp_local_date_time_get_minute, local_date_time);
}
int local_date_time_get_second(mgp_local_date_time *local_date_time) {
inline int local_date_time_get_second(mgp_local_date_time *local_date_time) {
return MgInvoke<int>(mgp_local_date_time_get_second, local_date_time);
}
int local_date_time_get_millisecond(mgp_local_date_time *local_date_time) {
inline int local_date_time_get_millisecond(mgp_local_date_time *local_date_time) {
return MgInvoke<int>(mgp_local_date_time_get_millisecond, local_date_time);
}
int local_date_time_get_microsecond(mgp_local_date_time *local_date_time) {
inline int local_date_time_get_microsecond(mgp_local_date_time *local_date_time) {
return MgInvoke<int>(mgp_local_date_time_get_microsecond, local_date_time);
}
int64_t local_date_time_timestamp(mgp_local_date_time *local_date_time) {
inline int64_t local_date_time_timestamp(mgp_local_date_time *local_date_time) {
return MgInvoke<int64_t>(mgp_local_date_time_timestamp, local_date_time);
}
mgp_local_date_time *local_date_time_now(mgp_memory *memory) {
inline mgp_local_date_time *local_date_time_now(mgp_memory *memory) {
return MgInvoke<mgp_local_date_time *>(mgp_local_date_time_now, memory);
}
mgp_local_date_time *local_date_time_add_duration(mgp_local_date_time *local_date_time, mgp_duration *dur,
mgp_memory *memory) {
inline mgp_local_date_time *local_date_time_add_duration(mgp_local_date_time *local_date_time, mgp_duration *dur,
mgp_memory *memory) {
return MgInvoke<mgp_local_date_time *>(mgp_local_date_time_add_duration, local_date_time, dur, memory);
}
mgp_local_date_time *local_date_time_sub_duration(mgp_local_date_time *local_date_time, mgp_duration *dur,
mgp_memory *memory) {
inline mgp_local_date_time *local_date_time_sub_duration(mgp_local_date_time *local_date_time, mgp_duration *dur,
mgp_memory *memory) {
return MgInvoke<mgp_local_date_time *>(mgp_local_date_time_sub_duration, local_date_time, dur, memory);
}
mgp_duration *local_date_time_diff(mgp_local_date_time *first, mgp_local_date_time *second, mgp_memory *memory) {
inline mgp_duration *local_date_time_diff(mgp_local_date_time *first, mgp_local_date_time *second, mgp_memory *memory) {
return MgInvoke<mgp_duration *>(mgp_local_date_time_diff, first, second, memory);
}
// mgp_duration
mgp_duration *duration_from_string(const char *string, mgp_memory *memory) {
inline mgp_duration *duration_from_string(const char *string, mgp_memory *memory) {
return MgInvoke<mgp_duration *>(mgp_duration_from_string, string, memory);
}
mgp_duration *duration_from_parameters(mgp_duration_parameters *parameters, mgp_memory *memory) {
inline mgp_duration *duration_from_parameters(mgp_duration_parameters *parameters, mgp_memory *memory) {
return MgInvoke<mgp_duration *>(mgp_duration_from_parameters, parameters, memory);
}
mgp_duration *duration_from_microseconds(int64_t microseconds, mgp_memory *memory) {
inline mgp_duration *duration_from_microseconds(int64_t microseconds, mgp_memory *memory) {
return MgInvoke<mgp_duration *>(mgp_duration_from_microseconds, microseconds, memory);
}
mgp_duration *duration_copy(mgp_duration *duration, mgp_memory *memory) {
inline mgp_duration *duration_copy(mgp_duration *duration, mgp_memory *memory) {
return MgInvoke<mgp_duration *>(mgp_duration_copy, duration, memory);
}
void duration_destroy(mgp_duration *duration) { mgp_duration_destroy(duration); }
inline void duration_destroy(mgp_duration *duration) { mgp_duration_destroy(duration); }
int64_t duration_get_microseconds(mgp_duration *duration) {
inline int64_t duration_get_microseconds(mgp_duration *duration) {
return MgInvoke<int64_t>(mgp_duration_get_microseconds, duration);
}
bool duration_equal(mgp_duration *first, mgp_duration *second) {
inline bool duration_equal(mgp_duration *first, mgp_duration *second) {
return MgInvoke<int>(mgp_duration_equal, first, second);
}
mgp_duration *duration_neg(mgp_duration *duration, mgp_memory *memory) {
inline mgp_duration *duration_neg(mgp_duration *duration, mgp_memory *memory) {
return MgInvoke<mgp_duration *>(mgp_duration_neg, duration, memory);
}
mgp_duration *duration_add(mgp_duration *first, mgp_duration *second, mgp_memory *memory) {
inline mgp_duration *duration_add(mgp_duration *first, mgp_duration *second, mgp_memory *memory) {
return MgInvoke<mgp_duration *>(mgp_duration_add, first, second, memory);
}
mgp_duration *duration_sub(mgp_duration *first, mgp_duration *second, mgp_memory *memory) {
inline mgp_duration *duration_sub(mgp_duration *first, mgp_duration *second, mgp_memory *memory) {
return MgInvoke<mgp_duration *>(mgp_duration_sub, first, second, memory);
}
// Procedure
mgp_proc *module_add_read_procedure(mgp_module *module, const char *name, mgp_proc_cb cb) {
inline mgp_proc *module_add_read_procedure(mgp_module *module, const char *name, mgp_proc_cb cb) {
return MgInvoke<mgp_proc *>(mgp_module_add_read_procedure, module, name, cb);
}
mgp_proc *module_add_write_procedure(mgp_module *module, const char *name, mgp_proc_cb cb) {
inline mgp_proc *module_add_write_procedure(mgp_module *module, const char *name, mgp_proc_cb cb) {
return MgInvoke<mgp_proc *>(mgp_module_add_write_procedure, module, name, cb);
}
void proc_add_arg(mgp_proc *proc, const char *name, mgp_type *type) {
inline void proc_add_arg(mgp_proc *proc, const char *name, mgp_type *type) {
MgInvokeVoid(mgp_proc_add_arg, proc, name, type);
}
void proc_add_opt_arg(mgp_proc *proc, const char *name, mgp_type *type, mgp_value *default_value) {
inline void proc_add_opt_arg(mgp_proc *proc, const char *name, mgp_type *type, mgp_value *default_value) {
MgInvokeVoid(mgp_proc_add_opt_arg, proc, name, type, default_value);
}
void proc_add_result(mgp_proc *proc, const char *name, mgp_type *type) {
inline void proc_add_result(mgp_proc *proc, const char *name, mgp_type *type) {
MgInvokeVoid(mgp_proc_add_result, proc, name, type);
}
void proc_add_deprecated_result(mgp_proc *proc, const char *name, mgp_type *type) {
inline void proc_add_deprecated_result(mgp_proc *proc, const char *name, mgp_type *type) {
MgInvokeVoid(mgp_proc_add_deprecated_result, proc, name, type);
}
bool must_abort(mgp_graph *graph) { return mgp_must_abort(graph); }
inline bool must_abort(mgp_graph *graph) { return mgp_must_abort(graph); }
// mgp_result
void result_set_error_msg(mgp_result *res, const char *error_msg) {
inline void result_set_error_msg(mgp_result *res, const char *error_msg) {
MgInvokeVoid(mgp_result_set_error_msg, res, error_msg);
}
mgp_result_record *result_new_record(mgp_result *res) {
inline mgp_result_record *result_new_record(mgp_result *res) {
return MgInvoke<mgp_result_record *>(mgp_result_new_record, res);
}
void result_record_insert(mgp_result_record *record, const char *field_name, mgp_value *val) {
inline void result_record_insert(mgp_result_record *record, const char *field_name, mgp_value *val) {
MgInvokeVoid(mgp_result_record_insert, record, field_name, val);
}
// Function
mgp_func *module_add_function(mgp_module *module, const char *name, mgp_func_cb cb) {
inline mgp_func *module_add_function(mgp_module *module, const char *name, mgp_func_cb cb) {
return MgInvoke<mgp_func *>(mgp_module_add_function, module, name, cb);
}
void func_add_arg(mgp_func *func, const char *name, mgp_type *type) {
inline void func_add_arg(mgp_func *func, const char *name, mgp_type *type) {
MgInvokeVoid(mgp_func_add_arg, func, name, type);
}
void func_add_opt_arg(mgp_func *func, const char *name, mgp_type *type, mgp_value *default_value) {
inline void func_add_opt_arg(mgp_func *func, const char *name, mgp_type *type, mgp_value *default_value) {
MgInvokeVoid(mgp_func_add_opt_arg, func, name, type, default_value);
}
void func_result_set_error_msg(mgp_func_result *res, const char *msg, mgp_memory *memory) {
inline void func_result_set_error_msg(mgp_func_result *res, const char *msg, mgp_memory *memory) {
MgInvokeVoid(mgp_func_result_set_error_msg, res, msg, memory);
}
void func_result_set_value(mgp_func_result *res, mgp_value *value, mgp_memory *memory) {
inline void func_result_set_value(mgp_func_result *res, mgp_value *value, mgp_memory *memory) {
MgInvokeVoid(mgp_func_result_set_value, res, value, memory);
}

File diff suppressed because it is too large Load Diff

6
init
View File

@@ -147,5 +147,11 @@ done;
python3 -m pip install pre-commit
python3 -m pre_commit install
# Install py format tools
echo "Install black formatter"
python3 -m pip install black==22.*
echo "Install isort"
python3 -m pip install isort==5.*
# Link `include/mgp.py` with `release/mgp/mgp.py`
ln -v -f include/mgp.py release/mgp/mgp.py

View File

@@ -208,7 +208,7 @@ pymgclient_tag="4f85c179e56302d46a1e3e2cf43509db65f062b3" # (2021-01-15)
repo_clone_try_double "${primary_urls[pymgclient]}" "${secondary_urls[pymgclient]}" "pymgclient" "$pymgclient_tag"
# mgconsole
mgconsole_tag="v1.1.0" # (2021-10-07)
mgconsole_tag="v1.3.0" # (2022-11-20)
repo_clone_try_double "${primary_urls[mgconsole]}" "${secondary_urls[mgconsole]}" "mgconsole" "$mgconsole_tag" true
spdlog_tag="v1.9.2" # (2021-08-12)

View File

@@ -36,7 +36,7 @@ ADDITIONAL USE GRANT: You may use the Licensed Work in accordance with the
3. using the Licensed Work to create a work or solution
which competes (or might reasonably be expected to
compete) with the Licensed Work.
CHANGE DATE: 2026-20-09
CHANGE DATE: 2026-07-11
CHANGE LICENSE: Apache License, Version 2.0
For information about alternative licensing arrangements, please visit: https://memgraph.com/legal.

View File

@@ -2,8 +2,8 @@ MEMGRAPH
ENTERPRISE LICENCE AGREEMENT
Memgraph Limited is registered in England under registration 10195084 and has its registered office at Suite 4,
Ironstone House, Ironstone Way, Brixworth, Northampton, NN6 9UD (Memgraph).
Memgraph Limited is registered in England under registration 10195084 and has its registered office at 90a High Street,
Hertfordshire, Berkhamsted, HP4 2BL United Kingdom ("Memgraph").
Memgraph agrees to license and/or grant you (the “Customer”) access to the Software ( as defined below) and provide

12
pyproject.toml Normal file
View File

@@ -0,0 +1,12 @@
[tool.black]
line-length = 120
include = '\.pyi?$'
extend-exclude = '''
/(
| .git
| .__pycache__
| build
| libs
| .cache
)/
'''

View File

@@ -69,7 +69,7 @@ extern "C" int mgp_init_module(struct mgp_module *module, struct mgp_memory *mem
try {
mgp::memory = memory;
AddProcedure(SampleReadProc, "return_true", mgp::ProdecureType::Read,
AddProcedure(SampleReadProc, "return_true", mgp::ProcedureType::Read,
{mgp::Parameter("param_1", mgp::Type::Int), mgp::Parameter("param_2", mgp::Type::Double, 2.3)},
{mgp::Return("out", mgp::Type::Bool)}, module, memory);
} catch (const std::exception &e) {
@@ -79,7 +79,7 @@ extern "C" int mgp_init_module(struct mgp_module *module, struct mgp_memory *mem
try {
mgp::memory = memory;
mgp::AddProcedure(AddXNodes, "add_x_nodes", mgp::ProdecureType::Write, {mgp::Parameter("param_1", mgp::Type::Int)},
mgp::AddProcedure(AddXNodes, "add_x_nodes", mgp::ProcedureType::Write, {mgp::Parameter("param_1", mgp::Type::Int)},
{}, module, memory);
} catch (const std::exception &e) {

View File

@@ -1,4 +1,11 @@
# mgp
PyPi package used for type hinting when creating MAGE modules. The get started
using MAGE repository checkout the repository here: https://github.com/memgraph/mage.
PyPi package used for type hinting when creating query modules. Repository of already available query modules is called [MAGE](https://github.com/memgraph/mage).
## 🎬 Get started
To learn more, head over to the [docs for the query modules Python API](https://memgraph.com/docs/memgraph/reference-guide/query-modules/api/python-api). To get started with query modules, check out the [how-to guide](https://memgraph.com/docs/memgraph/how-to-guides/query-modules) on Memgraph docs.
## 🔢 Versioning
- mgp v1.1 is compatible with Memgraph >= 2.4.0

View File

@@ -257,3 +257,11 @@ class _MODULE:
@staticmethod
def add_function(wrapper):
pass
class SOURCE_TYPE_KAFKA:
pass
class SOURCE_TYPE_PULSAR:
pass

View File

@@ -1,13 +1,13 @@
[tool.poetry]
name = "mgp"
version = "1.0.0"
version = "1.1.0"
description = "Memgraph's module for developing MAGE modules. Used only for type hinting!"
authors = [
"MasterMedo <mislav.vuletic@gmail.com>",
"jbajic <jure.bajic@memgraph.io>",
"katarinasupe <katarina.supe@memgraph.io>",
"jbajic <jure.bajic@memgraph.io>",
"antejavor <ante.javor@memgraph.io>",
"antaljanosbenjamin <benjamin.antal@memgraph.io>",
"MasterMedo <mislav.vuletic@gmail.com>",
]
license = "Apache-2.0"
readme = "README.md"

View File

@@ -15,10 +15,11 @@ add_subdirectory(query)
add_subdirectory(glue)
add_subdirectory(slk)
add_subdirectory(rpc)
add_subdirectory(license)
add_subdirectory(auth)
if (MG_ENTERPRISE)
add_subdirectory(audit)
if(MG_ENTERPRISE)
add_subdirectory(audit)
endif()
string(TOLOWER ${CMAKE_BUILD_TYPE} lower_build_type)
@@ -36,68 +37,76 @@ set(mg_single_node_v2_sources
)
set(mg_single_node_v2_libs stdc++fs Threads::Threads
telemetry_lib mg-query mg-communication mg-memory mg-utils mg-auth mg-license mg-settings mg-glue)
if (MG_ENTERPRISE)
# These are enterprise subsystems
set(mg_single_node_v2_libs ${mg_single_node_v2_libs} mg-audit)
mg-telemetry mg-query mg-communication mg-memory mg-utils mg-auth mg-license mg-settings mg-glue)
if(MG_ENTERPRISE)
# These are enterprise subsystems
set(mg_single_node_v2_libs ${mg_single_node_v2_libs} mg-audit)
endif()
# memgraph main executable
add_executable(memgraph ${mg_single_node_v2_sources})
target_include_directories(memgraph PUBLIC ${CMAKE_SOURCE_DIR}/include)
target_link_libraries(memgraph ${mg_single_node_v2_libs})
# NOTE: `include/mg_procedure.syms` describes a pattern match for symbols which
# should be dynamically exported, so that `dlopen` can correctly link the
# symbols in custom procedure module libraries.
target_link_libraries(memgraph "-Wl,--dynamic-list=${CMAKE_SOURCE_DIR}/include/mg_procedure.syms")
set_target_properties(memgraph PROPERTIES
# Set the executable output name to include version information.
OUTPUT_NAME "memgraph-${MEMGRAPH_VERSION}_${CMAKE_BUILD_TYPE}"
# Output the executable in main binary dir.
RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
# Set the executable output name to include version information.
OUTPUT_NAME "memgraph-${MEMGRAPH_VERSION}_${CMAKE_BUILD_TYPE}"
# Output the executable in main binary dir.
RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
# Create symlink to the built executable.
add_custom_command(TARGET memgraph POST_BUILD
COMMAND ${CMAKE_COMMAND} -E create_symlink $<TARGET_FILE:memgraph> ${CMAKE_BINARY_DIR}/memgraph
BYPRODUCTS ${CMAKE_BINARY_DIR}/memgraph
COMMENT "Creating symlink to memgraph executable")
COMMAND ${CMAKE_COMMAND} -E create_symlink $<TARGET_FILE:memgraph> ${CMAKE_BINARY_DIR}/memgraph
BYPRODUCTS ${CMAKE_BINARY_DIR}/memgraph
COMMENT "Creating symlink to memgraph executable")
# Emulate the installed python_support, by creating a symlink
add_custom_command(TARGET memgraph POST_BUILD
COMMAND ${CMAKE_COMMAND} -E create_symlink ${CMAKE_SOURCE_DIR}/include ${CMAKE_BINARY_DIR}/python_support
BYPRODUCTS ${CMAKE_BINARY_DIR}/python_support
COMMENT "Creating symlink for python_support")
COMMAND ${CMAKE_COMMAND} -E create_symlink ${CMAKE_SOURCE_DIR}/include ${CMAKE_BINARY_DIR}/python_support
BYPRODUCTS ${CMAKE_BINARY_DIR}/python_support
COMMENT "Creating symlink for python_support")
# Strip the executable in release build.
if (lower_build_type STREQUAL "release")
add_custom_command(TARGET memgraph POST_BUILD
COMMAND strip -s $<TARGET_FILE:memgraph>
COMMENT "Stripping symbols and sections from memgraph")
if(lower_build_type STREQUAL "release")
add_custom_command(TARGET memgraph POST_BUILD
COMMAND strip -s $<TARGET_FILE:memgraph>
COMMENT "Stripping symbols and sections from memgraph")
endif()
# Generate the configuration file.
add_custom_command(TARGET memgraph POST_BUILD
COMMAND ${CMAKE_SOURCE_DIR}/config/generate.py
${CMAKE_BINARY_DIR}/memgraph
${CMAKE_BINARY_DIR}/config/memgraph.conf
DEPENDS ${CMAKE_SOURCE_DIR}/config/generate.py
${CMAKE_SOURCE_DIR}/config/flags.yaml
BYPRODUCTS ${CMAKE_BINARY_DIR}/config/memgraph.conf
COMMENT "Generating memgraph configuration file")
COMMAND ${CMAKE_SOURCE_DIR}/config/generate.py
${CMAKE_BINARY_DIR}/memgraph
${CMAKE_BINARY_DIR}/config/memgraph.conf
DEPENDS ${CMAKE_SOURCE_DIR}/config/generate.py
${CMAKE_SOURCE_DIR}/config/flags.yaml
BYPRODUCTS ${CMAKE_BINARY_DIR}/config/memgraph.conf
COMMENT "Generating memgraph configuration file")
# Everything here is under "memgraph" install component.
set(CMAKE_INSTALL_DEFAULT_COMPONENT_NAME "memgraph")
# TODO: Default directory permissions to 755
# NOTE: This is added in CMake 3.11, so enable it then
#set(CMAKE_INSTALL_DEFAULT_DIRECTORY_PERMISSIONS
# OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ WORLD_READ)
# set(CMAKE_INSTALL_DEFAULT_DIRECTORY_PERMISSIONS
# OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ WORLD_READ)
# Install and rename executable to just 'memgraph' Since we have to rename,
# we cannot use the recommended `install(TARGETS ...)`.
install(PROGRAMS $<TARGET_FILE:memgraph>
DESTINATION lib/memgraph RENAME memgraph)
# Install Python source for supporting our embedded Python.
install(FILES ${CMAKE_SOURCE_DIR}/include/mgp.py
DESTINATION lib/memgraph/python_support)
# Install the includes file for writing custom procedures in C and C++>
install(FILES ${CMAKE_SOURCE_DIR}/include/mg_procedure.h
DESTINATION include/memgraph)
@@ -107,9 +116,11 @@ install(FILES ${CMAKE_SOURCE_DIR}/include/mg_exceptions.hpp
DESTINATION include/memgraph)
install(FILES ${CMAKE_SOURCE_DIR}/include/mgp.hpp
DESTINATION include/memgraph)
# Install the config file (must use absolute path).
install(FILES ${CMAKE_BINARY_DIR}/config/memgraph.conf
DESTINATION /etc/memgraph RENAME memgraph.conf)
# Install logrotate configuration (must use absolute path).
install(FILES ${CMAKE_SOURCE_DIR}/release/logrotate.conf
DESTINATION /etc/logrotate.d RENAME memgraph)
@@ -125,15 +136,14 @@ install(CODE "file(MAKE_DIRECTORY \$ENV{DESTDIR}/var/log/memgraph
# ----------------------------------------------------------------------------
# Memgraph CSV Import Tool Executable
# ----------------------------------------------------------------------------
add_executable(mg_import_csv mg_import_csv.cpp)
target_link_libraries(mg_import_csv mg-storage-v2)
# Strip the executable in release build.
if (lower_build_type STREQUAL "release")
add_custom_command(TARGET mg_import_csv POST_BUILD
COMMAND strip -s mg_import_csv
COMMENT "Stripping symbols and sections from mg_import_csv")
if(lower_build_type STREQUAL "release")
add_custom_command(TARGET mg_import_csv POST_BUILD
COMMAND strip -s mg_import_csv
COMMENT "Stripping symbols and sections from mg_import_csv")
endif()
install(TARGETS mg_import_csv RUNTIME DESTINATION bin)

View File

@@ -16,8 +16,8 @@
#include <fmt/format.h>
#include "auth/exceptions.hpp"
#include "license/license.hpp"
#include "utils/flag_validation.hpp"
#include "utils/license.hpp"
#include "utils/logging.hpp"
#include "utils/message.hpp"
#include "utils/settings.hpp"
@@ -68,10 +68,9 @@ Auth::Auth(const std::string &storage_directory) : storage_(storage_directory),
std::optional<User> Auth::Authenticate(const std::string &username, const std::string &password) {
if (module_.IsUsed()) {
const auto license_check_result = utils::license::global_license_checker.IsValidLicense(utils::global_settings);
const auto license_check_result = license::global_license_checker.IsEnterpriseValid(utils::global_settings);
if (license_check_result.HasError()) {
spdlog::warn(
utils::license::LicenseCheckErrorToString(license_check_result.GetError(), "authentication modules"));
spdlog::warn(license::LicenseCheckErrorToString(license_check_result.GetError(), "authentication modules"));
return std::nullopt;
}

View File

@@ -15,8 +15,8 @@
#include "auth/crypto.hpp"
#include "auth/exceptions.hpp"
#include "license/license.hpp"
#include "utils/cast.hpp"
#include "utils/license.hpp"
#include "utils/logging.hpp"
#include "utils/settings.hpp"
#include "utils/string.hpp"
@@ -242,7 +242,7 @@ FineGrainedAccessPermissions::FineGrainedAccessPermissions(const std::unordered_
PermissionLevel FineGrainedAccessPermissions::Has(const std::string &permission,
const FineGrainedPermission fine_grained_permission) const {
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return PermissionLevel::GRANT;
}
const auto concrete_permission = std::invoke([&]() -> uint64_t {
@@ -281,7 +281,7 @@ void FineGrainedAccessPermissions::Revoke(const std::string &permission) {
}
nlohmann::json FineGrainedAccessPermissions::Serialize() const {
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return {};
}
nlohmann::json data = nlohmann::json::object();
@@ -294,7 +294,7 @@ FineGrainedAccessPermissions FineGrainedAccessPermissions::Deserialize(const nlo
if (!data.is_object()) {
throw AuthException("Couldn't load permissions data!");
}
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return FineGrainedAccessPermissions{};
}
std::optional<uint64_t> global_permission;
@@ -347,7 +347,7 @@ const FineGrainedAccessPermissions &FineGrainedAccessHandler::edge_type_permissi
FineGrainedAccessPermissions &FineGrainedAccessHandler::edge_type_permissions() { return edge_type_permissions_; }
nlohmann::json FineGrainedAccessHandler::Serialize() const {
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return {};
}
nlohmann::json data = nlohmann::json::object();
@@ -363,7 +363,7 @@ FineGrainedAccessHandler FineGrainedAccessHandler::Deserialize(const nlohmann::j
if (!data["label_permissions"].is_object() || !data["edge_type_permissions"].is_object()) {
throw AuthException("Couldn't load label_permissions or edge_type_permissions data!");
}
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return FineGrainedAccessHandler{};
}
auto label_permissions = FineGrainedAccessPermissions::Deserialize(data["label_permissions"]);
@@ -414,7 +414,7 @@ nlohmann::json Role::Serialize() const {
data["rolename"] = rolename_;
data["permissions"] = permissions_.Serialize();
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
data["fine_grained_access_handler"] = fine_grained_access_handler_.Serialize();
} else {
data["fine_grained_access_handler"] = {};
@@ -432,7 +432,7 @@ Role Role::Deserialize(const nlohmann::json &data) {
}
auto permissions = Permissions::Deserialize(data["permissions"]);
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
if (!data["fine_grained_access_handler"].is_object()) {
throw AuthException("Couldn't load user data!");
}
@@ -445,7 +445,7 @@ Role Role::Deserialize(const nlohmann::json &data) {
bool operator==(const Role &first, const Role &second) {
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return first.rolename_ == second.rolename_ && first.permissions_ == second.permissions_ &&
first.fine_grained_access_handler_ == second.fine_grained_access_handler_;
}
@@ -483,13 +483,13 @@ void User::UpdatePassword(const std::optional<std::string> &password) {
}
if (FLAGS_auth_password_strength_regex != default_password_regex) {
if (const auto license_check_result = utils::license::global_license_checker.IsValidLicense(utils::global_settings);
if (const auto license_check_result = license::global_license_checker.IsEnterpriseValid(utils::global_settings);
license_check_result.HasError()) {
throw AuthException(
"Custom password regex is a Memgraph Enterprise feature. Please set the config "
"(\"--auth-password-strength-regex\") to its default value (\"{}\") or remove the flag.\n{}",
default_password_regex,
utils::license::LicenseCheckErrorToString(license_check_result.GetError(), "password regex"));
license::LicenseCheckErrorToString(license_check_result.GetError(), "password regex"));
}
}
std::regex re(FLAGS_auth_password_strength_regex);
@@ -517,7 +517,7 @@ Permissions User::GetPermissions() const {
#ifdef MG_ENTERPRISE
FineGrainedAccessPermissions User::GetFineGrainedAccessLabelPermissions() const {
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return FineGrainedAccessPermissions{};
}
@@ -530,7 +530,7 @@ FineGrainedAccessPermissions User::GetFineGrainedAccessLabelPermissions() const
}
FineGrainedAccessPermissions User::GetFineGrainedAccessEdgeTypePermissions() const {
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return FineGrainedAccessPermissions{};
}
if (role_) {
@@ -563,7 +563,7 @@ nlohmann::json User::Serialize() const {
data["password_hash"] = password_hash_;
data["permissions"] = permissions_.Serialize();
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
data["fine_grained_access_handler"] = fine_grained_access_handler_.Serialize();
} else {
data["fine_grained_access_handler"] = {};
@@ -582,7 +582,7 @@ User User::Deserialize(const nlohmann::json &data) {
}
auto permissions = Permissions::Deserialize(data["permissions"]);
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
if (!data["fine_grained_access_handler"].is_object()) {
throw AuthException("Couldn't load user data!");
}
@@ -595,7 +595,7 @@ User User::Deserialize(const nlohmann::json &data) {
bool operator==(const User &first, const User &second) {
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return first.username_ == second.username_ && first.password_hash_ == second.password_hash_ &&
first.permissions_ == second.permissions_ && first.role_ == second.role_ &&
first.fine_grained_access_handler_ == second.fine_grained_access_handler_;

View File

@@ -90,7 +90,7 @@ QueryData Client::Execute(const std::string &query, const std::map<std::string,
// It is super critical from performance point of view to send the pull message right after the run message. Otherwise
// the performance will degrade multiple magnitudes.
encoder_.MessageRun(query, parameters, {});
encoder_.MessagePull({});
encoder_.MessagePull({{"n", Value(-1)}});
spdlog::debug("Reading run message response");
Signature signature{};

View File

@@ -14,8 +14,8 @@
#include "auth/auth.hpp"
#include "auth/models.hpp"
#include "glue/auth.hpp"
#include "license/license.hpp"
#include "query/frontend/ast/ast.hpp"
#include "utils/license.hpp"
#include "utils/synchronized.hpp"
#ifdef MG_ENTERPRISE
@@ -23,7 +23,7 @@ namespace {
bool IsUserAuthorizedLabels(const memgraph::auth::User &user, const memgraph::query::DbAccessor *dba,
const std::vector<memgraph::storage::LabelId> &labels,
const memgraph::query::AuthQuery::FineGrainedPrivilege fine_grained_privilege) {
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return true;
}
return std::all_of(labels.begin(), labels.end(), [dba, &user, fine_grained_privilege](const auto &label) {
@@ -35,7 +35,7 @@ bool IsUserAuthorizedLabels(const memgraph::auth::User &user, const memgraph::qu
bool IsUserAuthorizedGloballyLabels(const memgraph::auth::User &user,
const memgraph::auth::FineGrainedPermission fine_grained_permission) {
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return true;
}
return user.GetFineGrainedAccessLabelPermissions().Has(memgraph::auth::kAsterisk, fine_grained_permission) ==
@@ -44,7 +44,7 @@ bool IsUserAuthorizedGloballyLabels(const memgraph::auth::User &user,
bool IsUserAuthorizedGloballyEdges(const memgraph::auth::User &user,
const memgraph::auth::FineGrainedPermission fine_grained_permission) {
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return true;
}
return user.GetFineGrainedAccessEdgeTypePermissions().Has(memgraph::auth::kAsterisk, fine_grained_permission) ==
@@ -54,7 +54,7 @@ bool IsUserAuthorizedGloballyEdges(const memgraph::auth::User &user,
bool IsUserAuthorizedEdgeType(const memgraph::auth::User &user, const memgraph::query::DbAccessor *dba,
const memgraph::storage::EdgeTypeId &edgeType,
const memgraph::query::AuthQuery::FineGrainedPrivilege fine_grained_privilege) {
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return true;
}
return user.GetFineGrainedAccessEdgeTypePermissions().Has(
@@ -87,7 +87,7 @@ bool AuthChecker::IsUserAuthorized(const std::optional<std::string> &username,
#ifdef MG_ENTERPRISE
std::unique_ptr<memgraph::query::FineGrainedAuthChecker> AuthChecker::GetFineGrainedAuthChecker(
const std::string &username, const memgraph::query::DbAccessor *dba) const {
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return {};
}
try {
@@ -154,7 +154,7 @@ bool FineGrainedAuthChecker::Has(const memgraph::storage::EdgeTypeId &edge_type,
bool FineGrainedAuthChecker::HasGlobalPrivilegeOnVertices(
const memgraph::query::AuthQuery::FineGrainedPrivilege fine_grained_privilege) const {
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return true;
}
return IsUserAuthorizedGloballyLabels(user_, FineGrainedPrivilegeToFineGrainedPermission(fine_grained_privilege));
@@ -162,7 +162,7 @@ bool FineGrainedAuthChecker::HasGlobalPrivilegeOnVertices(
bool FineGrainedAuthChecker::HasGlobalPrivilegeOnEdges(
const memgraph::query::AuthQuery::FineGrainedPrivilege fine_grained_privilege) const {
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return true;
}
return IsUserAuthorizedGloballyEdges(user_, FineGrainedPrivilegeToFineGrainedPermission(fine_grained_privilege));

View File

@@ -17,7 +17,7 @@
#include "auth/models.hpp"
#include "glue/auth.hpp"
#include "utils/license.hpp"
#include "license/license.hpp"
namespace {
@@ -125,7 +125,7 @@ std::vector<FineGrainedPermissionForPrivilegeResult> GetFineGrainedPermissionFor
const memgraph::auth::FineGrainedAccessPermissions &permissions, const std::string &permission_type,
const std::string &user_or_role) {
std::vector<FineGrainedPermissionForPrivilegeResult> fine_grained_permissions;
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return fine_grained_permissions;
}
const auto global_permission = permissions.GetGlobalPermission();
@@ -166,7 +166,7 @@ std::vector<FineGrainedPermissionForPrivilegeResult> GetFineGrainedPermissionFor
std::vector<std::vector<memgraph::query::TypedValue>> ConstructFineGrainedPrivilegesResult(
const std::vector<FineGrainedPermissionForPrivilegeResult> &privileges) {
std::vector<std::vector<memgraph::query::TypedValue>> grants;
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return {};
}
grants.reserve(privileges.size());
@@ -182,7 +182,7 @@ std::vector<std::vector<memgraph::query::TypedValue>> ConstructFineGrainedPrivil
std::vector<std::vector<memgraph::query::TypedValue>> ShowFineGrainedUserPrivileges(
const std::optional<memgraph::auth::User> &user) {
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return {};
}
const auto &label_permissions = user->GetFineGrainedAccessLabelPermissions();
@@ -201,7 +201,7 @@ std::vector<std::vector<memgraph::query::TypedValue>> ShowFineGrainedUserPrivile
std::vector<std::vector<memgraph::query::TypedValue>> ShowFineGrainedRolePrivileges(
const std::optional<memgraph::auth::Role> &role) {
if (!memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (!memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
return {};
}
const auto &label_permissions = role->GetFineGrainedAccessLabelPermissions();
@@ -231,13 +231,13 @@ AuthQueryHandler::AuthQueryHandler(
bool AuthQueryHandler::CreateUser(const std::string &username, const std::optional<std::string> &password) {
if (name_regex_string_ != kDefaultUserRoleRegex) {
if (const auto license_check_result =
memgraph::utils::license::global_license_checker.IsValidLicense(memgraph::utils::global_settings);
memgraph::license::global_license_checker.IsEnterpriseValid(memgraph::utils::global_settings);
license_check_result.HasError()) {
throw memgraph::auth::AuthException(
"Custom user/role regex is a Memgraph Enterprise feature. Please set the config "
"(\"--auth-user-or-role-name-regex\") to its default value (\"{}\") or remove the flag.\n{}",
kDefaultUserRoleRegex,
memgraph::utils::license::LicenseCheckErrorToString(license_check_result.GetError(), "user/role regex"));
memgraph::license::LicenseCheckErrorToString(license_check_result.GetError(), "user/role regex"));
}
}
if (!std::regex_match(username, name_regex_)) {
@@ -473,20 +473,20 @@ std::vector<std::vector<memgraph::query::TypedValue>> AuthQueryHandler::GetPrivi
if (user) {
grants = ShowUserPrivileges(user);
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
fine_grained_grants = ShowFineGrainedUserPrivileges(user);
}
#endif
} else {
grants = ShowRolePrivileges(role);
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
fine_grained_grants = ShowFineGrainedRolePrivileges(role);
}
#endif
}
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
grants.insert(grants.end(), fine_grained_grants.begin(), fine_grained_grants.end());
}
#endif
@@ -627,7 +627,7 @@ void AuthQueryHandler::EditPermissions(
edit_permissions_fun(user->permissions(), permission);
}
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
for (const auto &label_privilege_collection : label_privileges) {
edit_fine_grained_permissions_fun(user->fine_grained_access_handler().label_permissions(),
label_privilege_collection);
@@ -644,9 +644,9 @@ void AuthQueryHandler::EditPermissions(
edit_permissions_fun(role->permissions(), permission);
}
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
for (const auto &label_privilege : label_privileges) {
edit_fine_grained_permissions_fun(user->fine_grained_access_handler().label_permissions(), label_privilege);
edit_fine_grained_permissions_fun(role->fine_grained_access_handler().label_permissions(), label_privilege);
}
for (const auto &edge_type_privilege : edge_type_privileges) {
edit_fine_grained_permissions_fun(role->fine_grained_access_handler().edge_type_permissions(),

View File

@@ -15,8 +15,8 @@
#include "auth/auth.hpp"
#include "glue/auth.hpp"
#include "license/license.hpp"
#include "query/interpreter.hpp"
#include "utils/license.hpp"
#include "utils/string.hpp"
namespace memgraph::glue {

View File

@@ -1,4 +1,4 @@
// Copyright 2021 Memgraph Ltd.
// Copyright 2022 Memgraph Ltd.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
@@ -12,6 +12,7 @@
#pragma once
#include <filesystem>
#include <fstream>
#include <string>
#include <vector>
@@ -61,3 +62,42 @@ inline void LoadConfig(const std::string &product_name) {
for (int i = 0; i < custom_argc; ++i) free(custom_argv[i]);
delete[] custom_argv;
}
std::pair<std::string, std::string> LoadUsernameAndPassword(const std::string &pass_file) {
std::ifstream file(pass_file);
if (file.fail()) {
spdlog::warn("Problem with opening MG_PASSFILE, memgraph server will start without user");
return {};
}
std::vector<std::string> result;
std::string line;
std::getline(file, line);
size_t pos = 0;
std::string token;
static constexpr std::string_view delimiter{":"};
while ((pos = line.find(delimiter)) != std::string::npos) {
if (line[pos - 1] == '\\') {
line.erase(pos - 1, 1);
token += line.substr(0, pos);
line.erase(0, pos);
} else {
token += line.substr(0, pos);
result.push_back(token);
line.erase(0, pos + delimiter.length());
token = "";
}
}
result.push_back(line);
file.close();
if (result.size() != 2) {
spdlog::warn(
"Wrong data format. Data should be store in format: username:password, memgraph server will start without "
"user");
return {};
}
return {result[0], result[1]};
}

View File

@@ -0,0 +1,6 @@
set(license_src_files
license_sender.cpp
license.cpp)
add_library(mg-license STATIC ${license_src_files})
target_link_libraries(mg-license mg-settings mg-utils mg-requests spdlog::spdlog)

View File

@@ -9,17 +9,19 @@
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#include "utils/license.hpp"
#include "license/license.hpp"
#include <atomic>
#include <charconv>
#include <chrono>
#include <cstdint>
#include <functional>
#include <optional>
#include <unordered_map>
#include "slk/serialization.hpp"
#include "utils/base64.hpp"
#include "utils/cast.hpp"
#include "utils/exceptions.hpp"
#include "utils/logging.hpp"
#include "utils/memory_tracker.hpp"
@@ -27,7 +29,7 @@
#include "utils/spin_lock.hpp"
#include "utils/synchronized.hpp"
namespace memgraph::utils::license {
namespace memgraph::license {
namespace {
inline constexpr std::string_view license_key_prefix = "mglk-";
@@ -69,6 +71,17 @@ LicenseCheckResult IsValidLicenseInternal(const License &license, const std::str
}
} // namespace
std::string LicenseTypeToString(const LicenseType license_type) {
switch (license_type) {
case LicenseType::ENTERPRISE: {
return "enterprise";
}
case LicenseType::OEM: {
return "oem";
}
}
}
void RegisterLicenseSettings(LicenseChecker &license_checker, utils::Settings &settings) {
settings.RegisterSetting(std::string{kEnterpriseLicenseSettingKey}, "",
[&] { license_checker.RevalidateLicense(settings); });
@@ -81,7 +94,7 @@ LicenseChecker global_license_checker;
LicenseChecker::~LicenseChecker() { scheduler_.Stop(); }
std::pair<std::string, std::string> LicenseChecker::GetLicenseInfo(const utils::Settings &settings) const {
std::pair<std::string, std::string> LicenseChecker::ExtractLicenseInfo(const utils::Settings &settings) const {
if (license_info_override_) {
spdlog::warn("Ignoring license info stored in the settings because a different source was specified.");
return *license_info_override_;
@@ -96,7 +109,7 @@ std::pair<std::string, std::string> LicenseChecker::GetLicenseInfo(const utils::
}
void LicenseChecker::RevalidateLicense(const utils::Settings &settings) {
const auto license_info = GetLicenseInfo(settings);
const auto license_info = ExtractLicenseInfo(settings);
RevalidateLicense(license_info.first, license_info.second);
}
@@ -117,18 +130,7 @@ void LicenseChecker::RevalidateLicense(const std::string &license_key, const std
return;
}
struct PreviousLicenseInfo {
PreviousLicenseInfo(std::string license_key, std::string organization_name)
: license_key(std::move(license_key)), organization_name(std::move(organization_name)) {}
std::string license_key;
std::string organization_name;
bool is_valid{false};
};
static utils::Synchronized<std::optional<PreviousLicenseInfo>, utils::SpinLock> previous_license_info;
auto locked_previous_license_info_ptr = previous_license_info.Lock();
auto locked_previous_license_info_ptr = previous_license_info_.Lock();
auto &locked_previous_license_info = *locked_previous_license_info_ptr;
const bool same_license_info = locked_previous_license_info &&
locked_previous_license_info->license_key == license_key &&
@@ -140,7 +142,7 @@ void LicenseChecker::RevalidateLicense(const std::string &license_key, const std
locked_previous_license_info.emplace(license_key, organization_name);
const auto maybe_license = GetLicense(locked_previous_license_info->license_key);
auto maybe_license = GetLicense(locked_previous_license_info->license_key);
if (!maybe_license) {
spdlog::warn(LicenseCheckErrorToString(LicenseCheckError::INVALID_LICENSE_KEY_STRING, "Enterprise features"));
is_valid_.store(false, std::memory_order_relaxed);
@@ -156,22 +158,30 @@ void LicenseChecker::RevalidateLicense(const std::string &license_key, const std
spdlog::warn(LicenseCheckErrorToString(license_check_result.GetError(), "Enterprise features"));
is_valid_.store(false, std::memory_order_relaxed);
locked_previous_license_info->is_valid = false;
license_type_ = maybe_license->type;
set_memory_limit(0);
return;
}
if (!same_license_info) {
spdlog::info("All Enterprise features are active.");
license_type_ = maybe_license->type;
if (license_type_ == LicenseType::ENTERPRISE) {
spdlog::info("Enterprise license is active.");
} else {
spdlog::info("OEM license is active.");
}
is_valid_.store(true, std::memory_order_relaxed);
locked_previous_license_info->is_valid = true;
set_memory_limit(maybe_license->memory_limit);
locked_previous_license_info->license = std::move(*maybe_license);
}
}
void LicenseChecker::EnableTesting() {
void LicenseChecker::EnableTesting(const LicenseType license_type) {
enterprise_enabled_ = true;
is_valid_.store(true, std::memory_order_relaxed);
spdlog::info("All Enterprise features are activated for testing.");
license_type_ = license_type;
spdlog::info("The license type {} is set for testing.", LicenseTypeToString(license_type));
}
void LicenseChecker::CheckEnvLicense() {
@@ -216,20 +226,26 @@ std::string LicenseCheckErrorToString(LicenseCheckError error, const std::string
"following query:\n"
"SET DATABASE SETTING \"enterprise.license\" TO \"your-license-key\"",
feature);
case LicenseCheckError::NOT_ENTERPRISE_LICENSE:
return fmt::format("Your license has an invalid type. To use {} you need to have an enterprise license. \n",
feature);
}
}
LicenseCheckResult LicenseChecker::IsValidLicense(const utils::Settings &settings) const {
LicenseCheckResult LicenseChecker::IsEnterpriseValid(const utils::Settings &settings) const {
if (enterprise_enabled_) [[unlikely]] {
return {};
}
const auto license_info = GetLicenseInfo(settings);
const auto license_info = ExtractLicenseInfo(settings);
const auto maybe_license = GetLicense(license_info.first);
if (!maybe_license) {
return LicenseCheckError::INVALID_LICENSE_KEY_STRING;
}
if (maybe_license->type != LicenseType::ENTERPRISE) {
return LicenseCheckError::NOT_ENTERPRISE_LICENSE;
}
return IsValidLicenseInternal(*maybe_license, license_info.second);
}
@@ -239,7 +255,13 @@ void LicenseChecker::StartBackgroundLicenseChecker(const utils::Settings &settin
scheduler_.Run("licensechecker", std::chrono::minutes{5}, [&, this] { RevalidateLicense(settings); });
}
bool LicenseChecker::IsValidLicenseFast() const { return is_valid_.load(std::memory_order_relaxed); }
utils::Synchronized<std::optional<LicenseInfo>, utils::SpinLock> &LicenseChecker::GetLicenseInfo() {
return previous_license_info_;
}
bool LicenseChecker::IsEnterpriseValidFast() const {
return license_type_ == LicenseType::ENTERPRISE && is_valid_.load(std::memory_order_relaxed);
}
std::string Encode(const License &license) {
std::vector<uint8_t> buffer;
@@ -252,9 +274,10 @@ std::string Encode(const License &license) {
slk::Save(license.organization_name, &builder);
slk::Save(license.valid_until, &builder);
slk::Save(license.memory_limit, &builder);
slk::Save(utils::UnderlyingCast(license.type), &builder);
builder.Finalize();
return std::string{license_key_prefix} + base64_encode(buffer.data(), buffer.size());
return std::string{license_key_prefix} + utils::base64_encode(buffer.data(), buffer.size());
}
std::optional<License> Decode(std::string_view license_key) {
@@ -266,7 +289,7 @@ std::optional<License> Decode(std::string_view license_key) {
const auto decoded = std::invoke([license_key]() -> std::optional<std::string> {
try {
return base64_decode(license_key);
return utils::base64_decode(license_key);
} catch (const std::runtime_error & /*exception*/) {
return std::nullopt;
}
@@ -284,10 +307,12 @@ std::optional<License> Decode(std::string_view license_key) {
slk::Load(&valid_until, &reader);
int64_t memory_limit{0};
slk::Load(&memory_limit, &reader);
return License{.organization_name = organization_name, .valid_until = valid_until, .memory_limit = memory_limit};
std::underlying_type_t<LicenseType> license_type{0};
slk::Load(&license_type, &reader);
return {License{organization_name, valid_until, memory_limit, LicenseType(license_type)}};
} catch (const slk::SlkReaderException &e) {
return std::nullopt;
}
}
} // namespace memgraph::utils::license
} // namespace memgraph::license

View File

@@ -12,26 +12,57 @@
#pragma once
#include <cstdint>
#include <optional>
#include <string>
#include "utils/result.hpp"
#include "utils/scheduler.hpp"
#include "utils/settings.hpp"
#include "utils/spin_lock.hpp"
#include "utils/synchronized.hpp"
namespace memgraph::utils::license {
namespace memgraph::license {
enum class LicenseType : uint8_t { ENTERPRISE, OEM };
std::string LicenseTypeToString(LicenseType license_type);
struct License {
License() = default;
License(std::string organization_name, int64_t valid_until, int64_t memory_limit, LicenseType license_type)
: organization_name{std::move(organization_name)},
valid_until{valid_until},
memory_limit{memory_limit},
type{license_type} {}
std::string organization_name;
int64_t valid_until;
int64_t memory_limit;
LicenseType type;
bool operator==(const License &) const = default;
};
struct LicenseInfo {
LicenseInfo(std::string license_key, std::string organization_name)
: license_key(std::move(license_key)), organization_name{std::move(organization_name)} {}
std::string license_key;
std::string organization_name;
bool is_valid{false};
License license;
};
inline constexpr std::string_view kEnterpriseLicenseSettingKey = "enterprise.license";
inline constexpr std::string_view kOrganizationNameSettingKey = "organization.name";
enum class LicenseCheckError : uint8_t { INVALID_LICENSE_KEY_STRING, INVALID_ORGANIZATION_NAME, EXPIRED_LICENSE };
enum class LicenseCheckError : uint8_t {
INVALID_LICENSE_KEY_STRING,
INVALID_ORGANIZATION_NAME,
EXPIRED_LICENSE,
NOT_ENTERPRISE_LICENSE
};
std::string LicenseCheckErrorToString(LicenseCheckError error, std::string_view feature);
@@ -49,19 +80,25 @@ struct LicenseChecker {
void CheckEnvLicense();
void SetLicenseInfoOverride(std::string license_key, std::string organization_name);
void EnableTesting();
LicenseCheckResult IsValidLicense(const utils::Settings &settings) const;
bool IsValidLicenseFast() const;
void EnableTesting(LicenseType license_type = LicenseType::ENTERPRISE);
// Checks if license is valid and if enterprise is enabled
LicenseCheckResult IsEnterpriseValid(const utils::Settings &settings) const;
bool IsEnterpriseValidFast() const;
void StartBackgroundLicenseChecker(const utils::Settings &settings);
utils::Synchronized<std::optional<LicenseInfo>, utils::SpinLock> &GetLicenseInfo();
private:
std::pair<std::string, std::string> GetLicenseInfo(const utils::Settings &settings) const;
std::pair<std::string, std::string> ExtractLicenseInfo(const utils::Settings &settings) const;
void RevalidateLicense(const utils::Settings &settings);
void RevalidateLicense(const std::string &license_key, const std::string &organization_name);
std::optional<std::pair<std::string, std::string>> license_info_override_;
utils::Synchronized<std::optional<LicenseInfo>, utils::SpinLock> previous_license_info_{std::nullopt};
bool enterprise_enabled_{false};
std::atomic<bool> is_valid_{false};
LicenseType license_type_;
utils::Scheduler scheduler_;
friend void RegisterLicenseSettings(LicenseChecker &license_checker, utils::Settings &settings);
@@ -74,4 +111,4 @@ std::string Encode(const License &license);
// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)
extern LicenseChecker global_license_checker;
} // namespace memgraph::utils::license
} // namespace memgraph::license

View File

@@ -0,0 +1,71 @@
// Copyright 2022 Memgraph Ltd.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
// License, and you may not use this file except in compliance with the Business Source License.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#include "license/license_sender.hpp"
#include <spdlog/spdlog.h>
#include <cstdint>
#include "requests/requests.hpp"
#include "utils/memory_tracker.hpp"
#include "utils/stat.hpp"
#include "utils/synchronized.hpp"
#include "utils/system_info.hpp"
#include "utils/timestamp.hpp"
namespace memgraph::license {
LicenseInfoSender::LicenseInfoSender(std::string url, std::string uuid, std::string machine_id, int64_t memory_limit,
utils::Synchronized<std::optional<LicenseInfo>, utils::SpinLock> &license_info,
std::chrono::seconds request_frequency)
: url_{std::move(url)},
uuid_{std::move(uuid)},
machine_id_{std::move(machine_id)},
memory_limit_{memory_limit},
license_info_{license_info} {
scheduler_.Run("LicenseCheck", request_frequency, [&] { SendData(); });
}
LicenseInfoSender::~LicenseInfoSender() { scheduler_.Stop(); }
void LicenseInfoSender::SendData() {
nlohmann::json data = nlohmann::json::object();
license_info_.WithLock([&data, this](const auto &license_info) mutable {
if (license_info && !license_info->organization_name.empty()) {
const auto memory_info = utils::GetMemoryInfo();
const auto memory_usage = utils::GetMemoryUsage();
data = {{"run_id", uuid_},
{"machine_id", machine_id_},
{"type", "license-check"},
{"license_type", LicenseTypeToString(license_info->license.type)},
{"license_key", license_info->license_key},
{"organization", license_info->organization_name},
{"valid", license_info->is_valid},
{"physical_memory_size", memory_info.memory},
{"swap_memory_size", memory_info.swap},
{"memory_used", memory_usage},
{"runtime_memory_limit", memory_limit_},
{"license_memory_limit", license_info->license.memory_limit},
{"timestamp", utils::Timestamp::Now().SecWithNsecSinceTheEpoch()}};
}
});
if (data.empty()) {
return;
}
if (!requests::RequestPostJson(url_, data,
/* timeout_in_seconds = */ 2 * 60)) {
spdlog::trace("Cannot send license information, enable {} availability!", url_);
}
}
} // namespace memgraph::license

View File

@@ -0,0 +1,50 @@
// Copyright 2022 Memgraph Ltd.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
// License, and you may not use this file except in compliance with the Business Source License.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#pragma once
#include <chrono>
#include <cstdint>
#include <string>
#include <json/json.hpp>
#include "license/license.hpp"
#include "utils/scheduler.hpp"
#include "utils/timer.hpp"
namespace memgraph::license {
class LicenseInfoSender final {
public:
LicenseInfoSender(std::string url, std::string uuid, std::string machine_id, int64_t memory_limit,
utils::Synchronized<std::optional<LicenseInfo>, utils::SpinLock> &license_info,
std::chrono::seconds request_frequency = std::chrono::seconds(8 * 60 * 60));
LicenseInfoSender(const LicenseInfoSender &) = delete;
LicenseInfoSender(LicenseInfoSender &&) noexcept = delete;
LicenseInfoSender &operator=(const LicenseInfoSender &) = delete;
LicenseInfoSender &operator=(LicenseInfoSender &&) noexcept = delete;
~LicenseInfoSender();
private:
void SendData();
const std::string url_;
const std::string uuid_;
const std::string machine_id_;
const int64_t memory_limit_;
utils::Synchronized<std::optional<LicenseInfo>, utils::SpinLock> &license_info_;
utils::Scheduler scheduler_;
};
} // namespace memgraph::license

View File

@@ -26,6 +26,7 @@
#include <string_view>
#include <thread>
#include <fmt/core.h>
#include <fmt/format.h>
#include <gflags/gflags.h>
#include <spdlog/common.h>
@@ -40,6 +41,8 @@
#include "glue/auth_checker.hpp"
#include "glue/auth_handler.hpp"
#include "helpers.hpp"
#include "license/license.hpp"
#include "license/license_sender.hpp"
#include "py/py.hpp"
#include "query/auth_checker.hpp"
#include "query/discard_value_stream.hpp"
@@ -57,7 +60,6 @@
#include "utils/event_counter.hpp"
#include "utils/file.hpp"
#include "utils/flag_validation.hpp"
#include "utils/license.hpp"
#include "utils/logging.hpp"
#include "utils/memory_tracker.hpp"
#include "utils/message.hpp"
@@ -68,6 +70,7 @@
#include "utils/string.hpp"
#include "utils/synchronized.hpp"
#include "utils/sysinfo/memory.hpp"
#include "utils/system_info.hpp"
#include "utils/terminate_handler.hpp"
#include "version.hpp"
@@ -96,6 +99,10 @@
#include "audit/log.hpp"
#endif
constexpr const char *kMgUser = "MEMGRAPH_USER";
constexpr const char *kMgPassword = "MEMGRAPH_PASSWORD";
constexpr const char *kMgPassfile = "MEMGRAPH_PASSFILE";
namespace {
std::string GetAllowedEnumValuesString(const auto &mappings) {
std::vector<std::string> allowed_values;
@@ -132,6 +139,10 @@ std::optional<Enum> StringToEnum(const auto &value, const auto &mappings) {
}
} // namespace
// Short help flag.
// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)
DEFINE_HIDDEN_bool(h, false, "Print usage and exit.");
// Bolt server flags.
DEFINE_string(bolt_address, "0.0.0.0", "IP address on which the Bolt server should listen.");
// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)
@@ -161,6 +172,11 @@ DEFINE_string(bolt_key_file, "", "Key file which should be used for the Bolt ser
DEFINE_string(bolt_server_name_for_init, "",
"Server name which the database should send to the client in the "
"Bolt INIT message.");
// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)
DEFINE_string(init_file, "",
"Path to cypherl file that is used for configuring users and database schema before server starts.");
// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)
DEFINE_string(init_data_file, "", "Path to cypherl file that is used for creating data after server starts.");
// General purpose flags.
// NOTE: The `data_directory` flag must be the same here and in
@@ -470,6 +486,33 @@ struct SessionData {
DEFINE_string(auth_user_or_role_name_regex, memgraph::glue::kDefaultUserRoleRegex.data(),
"Set to the regular expression that each user or role name must fulfill.");
void InitFromCypherlFile(memgraph::query::InterpreterContext &ctx, std::string cypherl_file_path
#ifdef MG_ENTERPRISE
,
memgraph::audit::Log *audit_log
#endif
) {
memgraph::query::Interpreter interpreter(&ctx);
std::ifstream file(cypherl_file_path);
if (file.is_open()) {
std::string line;
while (std::getline(file, line)) {
if (!line.empty()) {
auto results = interpreter.Prepare(line, {}, {});
memgraph::query::DiscardValueResultStream stream;
interpreter.Pull(&stream, {}, results.qid);
#ifdef MG_ENTERPRISE
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
audit_log->Record("", "", line, {});
}
#endif
}
}
file.close();
}
}
class BoltSession final : public memgraph::communication::bolt::Session<memgraph::communication::v2::InputStream,
memgraph::communication::v2::OutputStream> {
public:
@@ -506,7 +549,7 @@ class BoltSession final : public memgraph::communication::bolt::Session<memgraph
username = &user_->username();
}
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
audit_log_->Record(endpoint_.address().to_string(), user_ ? *username : "", query,
memgraph::storage::PropertyValue(params_pv));
}
@@ -685,6 +728,11 @@ int main(int argc, char **argv) {
LoadConfig("memgraph");
gflags::ParseCommandLineFlags(&argc, &argv, true);
if (FLAGS_h) {
gflags::ShowUsageWithFlags(argv[0]);
exit(1);
}
InitializeLogger();
// Unhandled exception handler init.
@@ -779,15 +827,15 @@ int main(int argc, char **argv) {
memgraph::utils::OnScopeExit settings_finalizer([&] { memgraph::utils::global_settings.Finalize(); });
// register all runtime settings
memgraph::utils::license::RegisterLicenseSettings(memgraph::utils::license::global_license_checker,
memgraph::utils::global_settings);
memgraph::license::RegisterLicenseSettings(memgraph::license::global_license_checker,
memgraph::utils::global_settings);
memgraph::utils::license::global_license_checker.CheckEnvLicense();
memgraph::license::global_license_checker.CheckEnvLicense();
if (!FLAGS_organization_name.empty() && !FLAGS_license_key.empty()) {
memgraph::utils::license::global_license_checker.SetLicenseInfoOverride(FLAGS_license_key, FLAGS_organization_name);
memgraph::license::global_license_checker.SetLicenseInfoOverride(FLAGS_license_key, FLAGS_organization_name);
}
memgraph::utils::license::global_license_checker.StartBackgroundLicenseChecker(memgraph::utils::global_settings);
memgraph::license::global_license_checker.StartBackgroundLicenseChecker(memgraph::utils::global_settings);
// All enterprise features should be constructed before the main database
// storage. This will cause them to be destructed *after* the main database
@@ -878,6 +926,29 @@ int main(int argc, char **argv) {
interpreter_context.auth = &auth_handler;
interpreter_context.auth_checker = &auth_checker;
if (!FLAGS_init_file.empty()) {
spdlog::info("Running init file.");
#ifdef MG_ENTERPRISE
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
InitFromCypherlFile(interpreter_context, FLAGS_init_file, &audit_log);
}
#else
InitFromCypherlFile(interpreter_context, FLAGS_init_file);
#endif
}
auto *maybe_username = std::getenv(kMgUser);
auto *maybe_password = std::getenv(kMgPassword);
auto *maybe_pass_file = std::getenv(kMgPassfile);
if (maybe_username && maybe_password) {
auth_handler.CreateUser(maybe_username, maybe_password);
} else if (maybe_pass_file) {
const auto [username, password] = LoadUsernameAndPassword(maybe_pass_file);
if (!username.empty() && !password.empty()) {
auth_handler.CreateUser(username, password);
}
}
{
// Triggers can execute query procedures, so we need to reload the modules first and then
// the triggers
@@ -906,12 +977,15 @@ int main(int argc, char **argv) {
ServerT server(server_endpoint, &session_data, &context, FLAGS_bolt_session_inactivity_timeout, service_name,
FLAGS_bolt_num_workers);
const auto run_id = memgraph::utils::GenerateUUID();
const auto machine_id = memgraph::utils::GetMachineId();
session_data.run_id = run_id;
// Setup telemetry
static constexpr auto telemetry_server{"https://telemetry.memgraph.com/88b5e7e8-746a-11e8-9f85-538a9e9690cc/"};
std::optional<memgraph::telemetry::Telemetry> telemetry;
if (FLAGS_telemetry_enabled) {
telemetry.emplace("https://telemetry.memgraph.com/88b5e7e8-746a-11e8-9f85-538a9e9690cc/",
data_directory / "telemetry", std::chrono::minutes(10));
session_data.run_id = telemetry->GetRunId();
telemetry.emplace(telemetry_server, data_directory / "telemetry", run_id, machine_id, std::chrono::minutes(10));
telemetry->AddCollector("storage", [&db]() -> nlohmann::json {
auto info = db.GetInfo();
return {{"vertices", info.vertex_count}, {"edges", info.edge_count}};
@@ -927,6 +1001,8 @@ int main(int argc, char **argv) {
return memgraph::query::plan::CallProcedure::GetAndResetCounters();
});
}
memgraph::license::LicenseInfoSender license_info_sender(telemetry_server, run_id, machine_id, memory_limit,
memgraph::license::global_license_checker.GetLicenseInfo());
memgraph::communication::websocket::SafeAuth websocket_auth{&auth};
memgraph::communication::websocket::Server websocket_server{
@@ -950,6 +1026,17 @@ int main(int argc, char **argv) {
MG_ASSERT(server.Start(), "Couldn't start the Bolt server!");
websocket_server.Start();
if (!FLAGS_init_data_file.empty()) {
spdlog::info("Running init data file.");
#ifdef MG_ENTERPRISE
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
InitFromCypherlFile(interpreter_context, FLAGS_init_data_file, &audit_log);
}
#else
InitFromCypherlFile(interpreter_context, FLAGS_init_data_file);
#endif
}
server.AwaitShutdown();
websocket_server.AwaitShutdown();

View File

@@ -115,7 +115,7 @@ auto SubgraphVertexAccessor::OutEdges(storage::View view) const -> decltype(impl
auto maybe_edges = impl_.impl_.OutEdges(view, {});
if (maybe_edges.HasError()) return maybe_edges.GetError();
auto edges = std::move(*maybe_edges);
auto graph_edges = graph_->edges();
const auto &graph_edges = graph_->edges();
std::vector<storage::EdgeAccessor> filteredOutEdges;
for (auto &edge : edges) {
@@ -132,7 +132,7 @@ auto SubgraphVertexAccessor::InEdges(storage::View view) const -> decltype(impl_
auto maybe_edges = impl_.impl_.InEdges(view, {});
if (maybe_edges.HasError()) return maybe_edges.GetError();
auto edges = std::move(*maybe_edges);
auto graph_edges = graph_->edges();
const auto &graph_edges = graph_->edges();
std::vector<storage::EdgeAccessor> filteredOutEdges;
for (auto &edge : edges) {

View File

@@ -200,7 +200,7 @@ class SubgraphVertexAccessor final {
return impl_ == v.impl_;
}
auto InEdges(storage::View view) const -> decltype(impl_.OutEdges(view));
auto InEdges(storage::View view) const -> decltype(impl_.InEdges(view));
auto OutEdges(storage::View view) const -> decltype(impl_.OutEdges(view));

View File

@@ -461,7 +461,8 @@ cpp<#
(lcp:define-class aggregation (binary-operator)
((op "Op" :scope :public)
(symbol-pos :int32_t :initval -1 :scope :public
:documentation "Symbol table position of the symbol this Aggregation is mapped to."))
:documentation "Symbol table position of the symbol this Aggregation is mapped to.")
(distinct :bool :initval "false" :scope :public))
(:public
(lcp:define-enum op
(count min max sum avg collect-list collect-map project)
@@ -505,8 +506,8 @@ cpp<#
/// Aggregation's first expression is the value being aggregated. The second
/// expression is the key used only in COLLECT_MAP.
Aggregation(Expression *expression1, Expression *expression2, Op op)
: BinaryOperator(expression1, expression2), op_(op) {
Aggregation(Expression *expression1, Expression *expression2, Op op, bool distinct)
: BinaryOperator(expression1, expression2), op_(op), distinct_(distinct) {
// COUNT without expression denotes COUNT(*) in cypher.
DMG_ASSERT(expression1 || op == Aggregation::Op::COUNT,
"All aggregations, except COUNT require expression");

View File

@@ -2106,7 +2106,7 @@ antlrcpp::Any CypherMainVisitor::visitAtom(MemgraphCypher::AtomContext *ctx) {
// Here we handle COUNT(*). COUNT(expression) is handled in
// visitFunctionInvocation with other aggregations. This is visible in
// functionInvocation and atom producions in opencypher grammar.
return static_cast<Expression *>(storage_->Create<Aggregation>(nullptr, nullptr, Aggregation::Op::COUNT));
return static_cast<Expression *>(storage_->Create<Aggregation>(nullptr, nullptr, Aggregation::Op::COUNT, false));
} else if (ctx->ALL()) {
auto *ident = storage_->Create<Identifier>(
std::any_cast<std::string>(ctx->filterExpression()->idInColl()->variable()->accept(this)));
@@ -2222,9 +2222,7 @@ antlrcpp::Any CypherMainVisitor::visitNumberLiteral(MemgraphCypher::NumberLitera
}
antlrcpp::Any CypherMainVisitor::visitFunctionInvocation(MemgraphCypher::FunctionInvocationContext *ctx) {
if (ctx->DISTINCT()) {
throw utils::NotYetImplemented("DISTINCT function call");
}
const auto is_distinct = ctx->DISTINCT() != nullptr;
auto function_name = std::any_cast<std::string>(ctx->functionName()->accept(this));
std::vector<Expression *> expressions;
for (auto *expression : ctx->expression()) {
@@ -2232,33 +2230,38 @@ antlrcpp::Any CypherMainVisitor::visitFunctionInvocation(MemgraphCypher::Functio
}
if (expressions.size() == 1U) {
if (function_name == Aggregation::kCount) {
return static_cast<Expression *>(storage_->Create<Aggregation>(expressions[0], nullptr, Aggregation::Op::COUNT));
return static_cast<Expression *>(
storage_->Create<Aggregation>(expressions[0], nullptr, Aggregation::Op::COUNT, is_distinct));
}
if (function_name == Aggregation::kMin) {
return static_cast<Expression *>(storage_->Create<Aggregation>(expressions[0], nullptr, Aggregation::Op::MIN));
return static_cast<Expression *>(
storage_->Create<Aggregation>(expressions[0], nullptr, Aggregation::Op::MIN, is_distinct));
}
if (function_name == Aggregation::kMax) {
return static_cast<Expression *>(storage_->Create<Aggregation>(expressions[0], nullptr, Aggregation::Op::MAX));
return static_cast<Expression *>(
storage_->Create<Aggregation>(expressions[0], nullptr, Aggregation::Op::MAX, is_distinct));
}
if (function_name == Aggregation::kSum) {
return static_cast<Expression *>(storage_->Create<Aggregation>(expressions[0], nullptr, Aggregation::Op::SUM));
return static_cast<Expression *>(
storage_->Create<Aggregation>(expressions[0], nullptr, Aggregation::Op::SUM, is_distinct));
}
if (function_name == Aggregation::kAvg) {
return static_cast<Expression *>(storage_->Create<Aggregation>(expressions[0], nullptr, Aggregation::Op::AVG));
return static_cast<Expression *>(
storage_->Create<Aggregation>(expressions[0], nullptr, Aggregation::Op::AVG, is_distinct));
}
if (function_name == Aggregation::kCollect) {
return static_cast<Expression *>(
storage_->Create<Aggregation>(expressions[0], nullptr, Aggregation::Op::COLLECT_LIST));
storage_->Create<Aggregation>(expressions[0], nullptr, Aggregation::Op::COLLECT_LIST, is_distinct));
}
if (function_name == Aggregation::kProject) {
return static_cast<Expression *>(
storage_->Create<Aggregation>(expressions[0], nullptr, Aggregation::Op::PROJECT));
storage_->Create<Aggregation>(expressions[0], nullptr, Aggregation::Op::PROJECT, is_distinct));
}
}
if (expressions.size() == 2U && function_name == Aggregation::kCollect) {
return static_cast<Expression *>(
storage_->Create<Aggregation>(expressions[1], expressions[0], Aggregation::Op::COLLECT_MAP));
storage_->Create<Aggregation>(expressions[1], expressions[0], Aggregation::Op::COLLECT_MAP, is_distinct));
}
auto is_user_defined_function = [](const std::string &function_name) {

View File

@@ -25,6 +25,7 @@
#include "auth/models.hpp"
#include "glue/communication.hpp"
#include "license/license.hpp"
#include "memory/memory_control.hpp"
#include "query/constants.hpp"
#include "query/context.hpp"
@@ -54,7 +55,6 @@
#include "utils/event_counter.hpp"
#include "utils/exceptions.hpp"
#include "utils/flag_validation.hpp"
#include "utils/license.hpp"
#include "utils/likely.hpp"
#include "utils/logging.hpp"
#include "utils/memory.hpp"
@@ -299,7 +299,7 @@ Callback HandleAuthQuery(AuthQuery *auth_query, AuthQueryHandler *auth, const Pa
Callback callback;
const auto license_check_result = utils::license::global_license_checker.IsValidLicense(utils::global_settings);
const auto license_check_result = license::global_license_checker.IsEnterpriseValid(utils::global_settings);
static const std::unordered_set enterprise_only_methods{
AuthQuery::Action::CREATE_ROLE, AuthQuery::Action::DROP_ROLE, AuthQuery::Action::SET_ROLE,
@@ -309,7 +309,7 @@ Callback HandleAuthQuery(AuthQuery *auth_query, AuthQueryHandler *auth, const Pa
if (license_check_result.HasError() && enterprise_only_methods.contains(auth_query->action_)) {
throw utils::BasicException(
utils::license::LicenseCheckErrorToString(license_check_result.GetError(), "advanced authentication features"));
license::LicenseCheckErrorToString(license_check_result.GetError(), "advanced authentication features"));
}
switch (auth_query->action_) {
@@ -1017,7 +1017,7 @@ PullPlan::PullPlan(const std::shared_ptr<CachedPlan> plan, const Parameters &par
ctx_.evaluation_context.properties = NamesToProperties(plan->ast_storage().properties_, dba);
ctx_.evaluation_context.labels = NamesToLabels(plan->ast_storage().labels_, dba);
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && username.has_value() && dba) {
if (license::global_license_checker.IsEnterpriseValidFast() && username.has_value() && dba) {
ctx_.auth_checker = interpreter_context->auth_checker->GetFineGrainedAuthChecker(*username, dba);
}
#endif

View File

@@ -27,6 +27,7 @@
#include <cppitertools/imap.hpp>
#include "spdlog/spdlog.h"
#include "license/license.hpp"
#include "query/auth_checker.hpp"
#include "query/context.hpp"
#include "query/db_accessor.hpp"
@@ -40,6 +41,7 @@
#include "query/procedure/cypher_types.hpp"
#include "query/procedure/mg_procedure_impl.hpp"
#include "query/procedure/module.hpp"
#include "query/typed_value.hpp"
#include "storage/v2/property_value.hpp"
#include "storage/v2/view.hpp"
#include "utils/algorithm.hpp"
@@ -47,7 +49,6 @@
#include "utils/event_counter.hpp"
#include "utils/exceptions.hpp"
#include "utils/fnv.hpp"
#include "utils/license.hpp"
#include "utils/likely.hpp"
#include "utils/logging.hpp"
#include "utils/memory.hpp"
@@ -113,6 +114,7 @@ extern const Event UnionOperator;
extern const Event CartesianOperator;
extern const Event CallProcedureOperator;
extern const Event ForeachOperator;
extern const Event EmptyResultOperator;
} // namespace EventCounter
namespace memgraph::query::plan {
@@ -242,7 +244,7 @@ CreateNode::CreateNodeCursor::CreateNodeCursor(const CreateNode &self, utils::Me
bool CreateNode::CreateNodeCursor::Pull(Frame &frame, ExecutionContext &context) {
SCOPED_PROFILE_OP("CreateNode");
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!context.auth_checker->Has(self_.node_info_.labels,
memgraph::query::AuthQuery::FineGrainedPrivilege::CREATE_DELETE)) {
throw QueryRuntimeException("Vertex not created due to not having enough permission!");
@@ -334,7 +336,7 @@ bool CreateExpand::CreateExpandCursor::Pull(Frame &frame, ExecutionContext &cont
if (!input_cursor_->Pull(frame, context)) return false;
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast()) {
if (license::global_license_checker.IsEnterpriseValidFast()) {
const auto fine_grained_permission = self_.existing_node_
? memgraph::query::AuthQuery::FineGrainedPrivilege::UPDATE
@@ -433,8 +435,7 @@ class ScanAllCursor : public Cursor {
vertices_it_.emplace(vertices_.value().begin());
}
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
!FindNextVertex(context)) {
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker && !FindNextVertex(context)) {
return false;
}
#endif
@@ -731,7 +732,7 @@ bool Expand::ExpandCursor::Pull(Frame &frame, ExecutionContext &context) {
if (in_edges_ && *in_edges_it_ != in_edges_->end()) {
auto edge = *(*in_edges_it_)++;
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!(context.auth_checker->Has(edge, memgraph::query::AuthQuery::FineGrainedPrivilege::READ) &&
context.auth_checker->Has(edge.From(), self_.view_,
memgraph::query::AuthQuery::FineGrainedPrivilege::READ))) {
@@ -752,7 +753,7 @@ bool Expand::ExpandCursor::Pull(Frame &frame, ExecutionContext &context) {
// already done in the block above
if (self_.common_.direction == EdgeAtom::Direction::BOTH && edge.IsCycle()) continue;
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!(context.auth_checker->Has(edge, memgraph::query::AuthQuery::FineGrainedPrivilege::READ) &&
context.auth_checker->Has(edge.To(), self_.view_,
memgraph::query::AuthQuery::FineGrainedPrivilege::READ))) {
@@ -1089,7 +1090,7 @@ class ExpandVariableCursor : public Cursor {
VertexAccessor current_vertex =
current_edge.second == EdgeAtom::Direction::IN ? current_edge.first.From() : current_edge.first.To();
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!(context.auth_checker->Has(current_edge.first, memgraph::query::AuthQuery::FineGrainedPrivilege::READ) &&
context.auth_checker->Has(current_vertex, storage::View::OLD,
memgraph::query::AuthQuery::FineGrainedPrivilege::READ))) {
@@ -1258,7 +1259,7 @@ class STShortestPathCursor : public query::plan::Cursor {
auto out_edges = UnwrapEdgesResult(vertex.OutEdges(storage::View::OLD, self_.common_.edge_types));
for (const auto &edge : out_edges) {
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!(context.auth_checker->Has(edge, memgraph::query::AuthQuery::FineGrainedPrivilege::READ) &&
context.auth_checker->Has(edge.To(), storage::View::OLD,
memgraph::query::AuthQuery::FineGrainedPrivilege::READ))) {
@@ -1284,7 +1285,7 @@ class STShortestPathCursor : public query::plan::Cursor {
auto in_edges = UnwrapEdgesResult(vertex.InEdges(storage::View::OLD, self_.common_.edge_types));
for (const auto &edge : in_edges) {
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!(context.auth_checker->Has(edge, memgraph::query::AuthQuery::FineGrainedPrivilege::READ) &&
context.auth_checker->Has(edge.From(), storage::View::OLD,
memgraph::query::AuthQuery::FineGrainedPrivilege::READ))) {
@@ -1324,7 +1325,7 @@ class STShortestPathCursor : public query::plan::Cursor {
auto out_edges = UnwrapEdgesResult(vertex.OutEdges(storage::View::OLD, self_.common_.edge_types));
for (const auto &edge : out_edges) {
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!(context.auth_checker->Has(edge, memgraph::query::AuthQuery::FineGrainedPrivilege::READ) &&
context.auth_checker->Has(edge.To(), storage::View::OLD,
memgraph::query::AuthQuery::FineGrainedPrivilege::READ))) {
@@ -1349,7 +1350,7 @@ class STShortestPathCursor : public query::plan::Cursor {
auto in_edges = UnwrapEdgesResult(vertex.InEdges(storage::View::OLD, self_.common_.edge_types));
for (const auto &edge : in_edges) {
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!(context.auth_checker->Has(edge, memgraph::query::AuthQuery::FineGrainedPrivilege::READ) &&
context.auth_checker->Has(edge.From(), storage::View::OLD,
memgraph::query::AuthQuery::FineGrainedPrivilege::READ))) {
@@ -1406,7 +1407,7 @@ class SingleSourceShortestPathCursor : public query::plan::Cursor {
// if we already processed the given vertex it doesn't get expanded
if (processed_.find(vertex) != processed_.end()) return;
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!(context.auth_checker->Has(vertex, storage::View::OLD,
memgraph::query::AuthQuery::FineGrainedPrivilege::READ) &&
context.auth_checker->Has(edge, memgraph::query::AuthQuery::FineGrainedPrivilege::READ))) {
@@ -1592,7 +1593,7 @@ class ExpandWeightedShortestPathCursor : public query::plan::Cursor {
int64_t depth) {
auto *memory = evaluator.GetMemoryResource();
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!(context.auth_checker->Has(vertex, storage::View::OLD,
memgraph::query::AuthQuery::FineGrainedPrivilege::READ) &&
context.auth_checker->Has(edge, memgraph::query::AuthQuery::FineGrainedPrivilege::READ))) {
@@ -1902,7 +1903,7 @@ class ExpandAllShortestPathsCursor : public query::plan::Cursor {
auto out_edges = UnwrapEdgesResult(vertex.OutEdges(storage::View::OLD, self_.common_.edge_types));
for (const auto &edge : out_edges) {
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!(context.auth_checker->Has(edge.To(), storage::View::OLD,
memgraph::query::AuthQuery::FineGrainedPrivilege::READ) &&
context.auth_checker->Has(edge, memgraph::query::AuthQuery::FineGrainedPrivilege::READ))) {
@@ -1916,7 +1917,7 @@ class ExpandAllShortestPathsCursor : public query::plan::Cursor {
auto in_edges = UnwrapEdgesResult(vertex.InEdges(storage::View::OLD, self_.common_.edge_types));
for (const auto &edge : in_edges) {
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!(context.auth_checker->Has(edge.From(), storage::View::OLD,
memgraph::query::AuthQuery::FineGrainedPrivilege::READ) &&
context.auth_checker->Has(edge, memgraph::query::AuthQuery::FineGrainedPrivilege::READ))) {
@@ -1973,7 +1974,6 @@ class ExpandAllShortestPathsCursor : public query::plan::Cursor {
edges_on_frame.emplace(edges_on_frame.begin(), current_edge);
auto next_vertex = current_edge_direction == EdgeAtom::Direction::IN ? current_edge.From() : current_edge.To();
frame[self_.common_.node_symbol] = next_vertex;
frame[self_.total_weight_.value()] = current_weight;
if (next_edges_.find({next_vertex, traversal_stack_.size()}) != next_edges_.end()) {
@@ -1986,6 +1986,15 @@ class ExpandAllShortestPathsCursor : public query::plan::Cursor {
}
if ((current_weight > visited_cost_.at(next_vertex)).ValueBool()) continue;
// Place destination node on the frame, handle existence flag
if (self_.common_.existing_node) {
const auto &node = frame[self_.common_.node_symbol];
ExpectType(self_.common_.node_symbol, node, TypedValue::Type::Vertex);
if (node.ValueVertex() != next_vertex) continue;
} else {
frame[self_.common_.node_symbol] = next_vertex;
}
return true;
}
@@ -2366,7 +2375,7 @@ bool Delete::DeleteCursor::Pull(Frame &frame, ExecutionContext &context) {
if (expression_result.type() == TypedValue::Type::Edge) {
auto &ea = expression_result.ValueEdge();
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!(context.auth_checker->Has(ea, query::AuthQuery::FineGrainedPrivilege::CREATE_DELETE) &&
context.auth_checker->Has(ea.To(), storage::View::NEW, query::AuthQuery::FineGrainedPrivilege::UPDATE) &&
context.auth_checker->Has(ea.From(), storage::View::NEW, query::AuthQuery::FineGrainedPrivilege::UPDATE))) {
@@ -2399,7 +2408,7 @@ bool Delete::DeleteCursor::Pull(Frame &frame, ExecutionContext &context) {
case TypedValue::Type::Vertex: {
auto &va = expression_result.ValueVertex();
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!context.auth_checker->Has(va, storage::View::NEW, query::AuthQuery::FineGrainedPrivilege::CREATE_DELETE)) {
throw QueryRuntimeException("Vertex not deleted due to not having enough permission!");
}
@@ -2508,7 +2517,7 @@ bool SetProperty::SetPropertyCursor::Pull(Frame &frame, ExecutionContext &contex
switch (lhs.type()) {
case TypedValue::Type::Vertex: {
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!context.auth_checker->Has(lhs.ValueVertex(), storage::View::NEW,
memgraph::query::AuthQuery::FineGrainedPrivilege::UPDATE)) {
throw QueryRuntimeException("Vertex property not set due to not having enough permission!");
@@ -2525,7 +2534,7 @@ bool SetProperty::SetPropertyCursor::Pull(Frame &frame, ExecutionContext &contex
}
case TypedValue::Type::Edge: {
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!context.auth_checker->Has(lhs.ValueEdge(), memgraph::query::AuthQuery::FineGrainedPrivilege::UPDATE)) {
throw QueryRuntimeException("Edge property not set due to not having enough permission!");
}
@@ -2724,7 +2733,7 @@ bool SetProperties::SetPropertiesCursor::Pull(Frame &frame, ExecutionContext &co
switch (lhs.type()) {
case TypedValue::Type::Vertex:
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!context.auth_checker->Has(lhs.ValueVertex(), storage::View::NEW,
memgraph::query::AuthQuery::FineGrainedPrivilege::UPDATE)) {
throw QueryRuntimeException("Vertex properties not set due to not having enough permission!");
@@ -2735,7 +2744,7 @@ bool SetProperties::SetPropertiesCursor::Pull(Frame &frame, ExecutionContext &co
break;
case TypedValue::Type::Edge:
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!context.auth_checker->Has(lhs.ValueEdge(), memgraph::query::AuthQuery::FineGrainedPrivilege::UPDATE)) {
throw QueryRuntimeException("Edge properties not set due to not having enough permission!");
}
@@ -2778,7 +2787,7 @@ bool SetLabels::SetLabelsCursor::Pull(Frame &frame, ExecutionContext &context) {
SCOPED_PROFILE_OP("SetLabels");
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!context.auth_checker->Has(self_.labels_, memgraph::query::AuthQuery::FineGrainedPrivilege::CREATE_DELETE)) {
throw QueryRuntimeException("Couldn't set label due to not having enough permission!");
}
@@ -2793,7 +2802,7 @@ bool SetLabels::SetLabelsCursor::Pull(Frame &frame, ExecutionContext &context) {
auto &vertex = vertex_value.ValueVertex();
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!context.auth_checker->Has(vertex, storage::View::OLD,
memgraph::query::AuthQuery::FineGrainedPrivilege::UPDATE)) {
throw QueryRuntimeException("Couldn't set label due to not having enough permission!");
@@ -2883,7 +2892,7 @@ bool RemoveProperty::RemovePropertyCursor::Pull(Frame &frame, ExecutionContext &
switch (lhs.type()) {
case TypedValue::Type::Vertex:
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!context.auth_checker->Has(lhs.ValueVertex(), storage::View::NEW,
memgraph::query::AuthQuery::FineGrainedPrivilege::UPDATE)) {
throw QueryRuntimeException("Vertex property not removed due to not having enough permission!");
@@ -2893,7 +2902,7 @@ bool RemoveProperty::RemovePropertyCursor::Pull(Frame &frame, ExecutionContext &
break;
case TypedValue::Type::Edge:
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!context.auth_checker->Has(lhs.ValueEdge(), memgraph::query::AuthQuery::FineGrainedPrivilege::UPDATE)) {
throw QueryRuntimeException("Edge property not removed due to not having enough permission!");
}
@@ -2936,7 +2945,7 @@ bool RemoveLabels::RemoveLabelsCursor::Pull(Frame &frame, ExecutionContext &cont
SCOPED_PROFILE_OP("RemoveLabels");
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!context.auth_checker->Has(self_.labels_, memgraph::query::AuthQuery::FineGrainedPrivilege::CREATE_DELETE)) {
throw QueryRuntimeException("Couldn't remove label due to not having enough permission!");
}
@@ -2951,7 +2960,7 @@ bool RemoveLabels::RemoveLabelsCursor::Pull(Frame &frame, ExecutionContext &cont
auto &vertex = vertex_value.ValueVertex();
#ifdef MG_ENTERPRISE
if (utils::license::global_license_checker.IsValidLicenseFast() && context.auth_checker &&
if (license::global_license_checker.IsEnterpriseValidFast() && context.auth_checker &&
!context.auth_checker->Has(vertex, storage::View::OLD,
memgraph::query::AuthQuery::FineGrainedPrivilege::UPDATE)) {
throw QueryRuntimeException("Couldn't remove label due to not having enough permission!");
@@ -3050,6 +3059,56 @@ void EdgeUniquenessFilter::EdgeUniquenessFilterCursor::Shutdown() { input_cursor
void EdgeUniquenessFilter::EdgeUniquenessFilterCursor::Reset() { input_cursor_->Reset(); }
EmptyResult::EmptyResult(const std::shared_ptr<LogicalOperator> &input)
: input_(input ? input : std::make_shared<Once>()) {}
ACCEPT_WITH_INPUT(EmptyResult)
std::vector<Symbol> EmptyResult::OutputSymbols(const SymbolTable &) const { // NOLINT(hicpp-named-parameter)
return {};
}
std::vector<Symbol> EmptyResult::ModifiedSymbols(const SymbolTable &) const { // NOLINT(hicpp-named-parameter)
return {};
}
class EmptyResultCursor : public Cursor {
public:
EmptyResultCursor(const EmptyResult &self, utils::MemoryResource *mem)
: input_cursor_(self.input_->MakeCursor(mem)) {}
bool Pull(Frame &frame, ExecutionContext &context) override {
SCOPED_PROFILE_OP("EmptyResult");
if (!pulled_all_input_) {
while (input_cursor_->Pull(frame, context)) {
if (MustAbort(context)) {
throw HintedAbortError();
}
}
pulled_all_input_ = true;
}
return false;
}
void Shutdown() override { input_cursor_->Shutdown(); }
void Reset() override {
input_cursor_->Reset();
pulled_all_input_ = false;
}
private:
const UniqueCursorPtr input_cursor_;
bool pulled_all_input_{false};
};
UniqueCursorPtr EmptyResult::MakeCursor(utils::MemoryResource *mem) const {
EventCounter::IncrementCounter(EventCounter::EmptyResultOperator);
return MakeUniqueCursorPtr<EmptyResultCursor>(mem, *this, mem);
}
Accumulate::Accumulate(const std::shared_ptr<LogicalOperator> &input, const std::vector<Symbol> &symbols,
bool advance_command)
: input_(input), symbols_(symbols), advance_command_(advance_command) {}
@@ -3204,7 +3263,8 @@ class AggregateCursor : public Cursor {
// aggregation map. The vectors in an AggregationValue contain one element for
// each aggregation in this LogicalOp.
struct AggregationValue {
explicit AggregationValue(utils::MemoryResource *mem) : counts_(mem), values_(mem), remember_(mem) {}
explicit AggregationValue(utils::MemoryResource *mem)
: counts_(mem), values_(mem), remember_(mem), unique_values_(mem) {}
// how many input rows have been aggregated in respective values_ element so
// far
@@ -3217,6 +3277,10 @@ class AggregateCursor : public Cursor {
utils::pmr::vector<TypedValue> values_;
// remember values.
utils::pmr::vector<TypedValue> remember_;
using TSet = utils::pmr::unordered_set<TypedValue, TypedValue::Hash, TypedValue::BoolEqual>;
utils::pmr::vector<TSet> unique_values_;
};
const Aggregate &self_;
@@ -3292,6 +3356,7 @@ class AggregateCursor : public Cursor {
for (const auto &agg_elem : self_.aggregations_) {
auto *mem = agg_value->values_.get_allocator().GetMemoryResource();
agg_value->values_.emplace_back(DefaultAggregationOpValue(agg_elem, mem));
agg_value->unique_values_.emplace_back(AggregationValue::TSet(mem));
}
agg_value->counts_.resize(self_.aggregations_.size(), 0);
@@ -3310,8 +3375,9 @@ class AggregateCursor : public Cursor {
auto count_it = agg_value->counts_.begin();
auto value_it = agg_value->values_.begin();
auto unique_values_it = agg_value->unique_values_.begin();
auto agg_elem_it = self_.aggregations_.begin();
for (; count_it < agg_value->counts_.end(); count_it++, value_it++, agg_elem_it++) {
for (; count_it < agg_value->counts_.end(); count_it++, value_it++, unique_values_it++, agg_elem_it++) {
// COUNT(*) is the only case where input expression is optional
// handle it here
auto input_expr_ptr = agg_elem_it->value;
@@ -3326,6 +3392,12 @@ class AggregateCursor : public Cursor {
// Aggregations skip Null input values.
if (input_value.IsNull()) continue;
const auto &agg_op = agg_elem_it->op;
if (agg_elem_it->distinct) {
auto insert_result = unique_values_it->insert(input_value);
if (!insert_result.second) {
break;
}
}
*count_it += 1;
if (*count_it == 1) {
// first value, nothing to aggregate. check type, set and continue.

View File

@@ -132,6 +132,7 @@ class Cartesian;
class CallProcedure;
class LoadCsv;
class Foreach;
class EmptyResult;
using LogicalOperatorCompositeVisitor = utils::CompositeVisitor<
Once, CreateNode, CreateExpand, ScanAll, ScanAllByLabel,
@@ -140,7 +141,7 @@ using LogicalOperatorCompositeVisitor = utils::CompositeVisitor<
Expand, ExpandVariable, ConstructNamedPath, Filter, Produce, Delete,
SetProperty, SetProperties, SetLabels, RemoveProperty, RemoveLabels,
EdgeUniquenessFilter, Accumulate, Aggregate, Skip, Limit, OrderBy, Merge,
Optional, Unwind, Distinct, Union, Cartesian, CallProcedure, LoadCsv, Foreach>;
Optional, Unwind, Distinct, Union, Cartesian, CallProcedure, LoadCsv, Foreach, EmptyResult>;
using LogicalOperatorLeafVisitor = utils::LeafVisitor<Once>;
@@ -1554,6 +1555,41 @@ edge lists).")
(:serialize (:slk))
(:clone))
(lcp:define-class empty-result (logical-operator)
((input "std::shared_ptr<LogicalOperator>" :scope :public
:slk-save #'slk-save-operator-pointer
:slk-load #'slk-load-operator-pointer))
(:documentation
"Pulls everything from the input and discards it.
On the first Pull from this operator's Cursor the input Cursor will be Pulled
until it is empty. The results won't be accumulated in the temporary cache.
This technique is used for ensuring that the cursor has been exhausted after
a WriteHandleClause. A typical use case is a `MATCH--SET` query with RETURN statement
missing.
@param input Input @c LogicalOperator. ")
(:public
#>cpp
EmptyResult() {}
EmptyResult(const std::shared_ptr<LogicalOperator> &input);
bool Accept(HierarchicalLogicalOperatorVisitor &visitor) override;
UniqueCursorPtr MakeCursor(utils::MemoryResource *) const override;
std::vector<Symbol> OutputSymbols(const SymbolTable &) const override;
std::vector<Symbol> ModifiedSymbols(const SymbolTable &) const override;
bool HasSingleInput() const override { return true; }
std::shared_ptr<LogicalOperator> input() const override { return input_; }
void set_input(std::shared_ptr<LogicalOperator> input) override {
input_ = input;
}
cpp<#)
(:serialize (:slk))
(:clone))
(lcp:define-class accumulate (logical-operator)
((input "std::shared_ptr<LogicalOperator>" :scope :public
:slk-save #'slk-save-operator-pointer
@@ -1657,7 +1693,8 @@ elements are in an undefined state after aggregation.")
:slk-save #'slk-save-ast-pointer
:slk-load (slk-load-ast-pointer "Expression"))
(op "::Aggregation::Op")
(output-sym "Symbol"))
(output-sym "Symbol")
(distinct bool :initval "false" ))
(:documentation
"An aggregation element, contains:
(input data expression, key expression - only used in COLLECT_MAP, type of
@@ -2282,9 +2319,9 @@ clauses.
(:public
#>cpp
Foreach() = default;
Foreach(std::shared_ptr<LogicalOperator> input,
Foreach(std::shared_ptr<LogicalOperator> input,
std::shared_ptr<LogicalOperator> updates,
Expression *named_expr,
Expression *named_expr,
Symbol loop_variable_symbol);
bool Accept(HierarchicalLogicalOperatorVisitor &visitor) override;

View File

@@ -156,6 +156,7 @@ PRE_VISIT(RemoveProperty);
PRE_VISIT(RemoveLabels);
PRE_VISIT(EdgeUniquenessFilter);
PRE_VISIT(Accumulate);
PRE_VISIT(EmptyResult);
bool PlanPrinter::PreVisit(query::plan::Aggregate &op) {
WithPrintLn([&](auto &out) {
@@ -401,6 +402,8 @@ json ToJson(const Aggregate::Element &elem) {
}
json["op"] = utils::ToLowerCase(Aggregation::OpToString(elem.op));
json["output_symbol"] = ToJson(elem.output_sym);
json["distinct"] = elem.distinct;
return json;
}
////////////////////////// END HELPER FUNCTIONS ////////////////////////////////
@@ -703,6 +706,17 @@ bool PlanToJsonVisitor::PreVisit(EdgeUniquenessFilter &op) {
return false;
}
bool PlanToJsonVisitor::PreVisit(EmptyResult &op) {
json self;
self["name"] = "EmptyResult";
op.input_->Accept(*this);
self["input"] = PopOutput();
output_ = std::move(self);
return false;
}
bool PlanToJsonVisitor::PreVisit(Accumulate &op) {
json self;
self["name"] = "Accumulate";

View File

@@ -80,6 +80,7 @@ class PlanPrinter : public virtual HierarchicalLogicalOperatorVisitor {
bool PreVisit(Optional &) override;
bool PreVisit(Cartesian &) override;
bool PreVisit(EmptyResult &) override;
bool PreVisit(Produce &) override;
bool PreVisit(Accumulate &) override;
bool PreVisit(Aggregate &) override;
@@ -195,6 +196,7 @@ class PlanToJsonVisitor : public virtual HierarchicalLogicalOperatorVisitor {
bool PreVisit(ScanAllByLabelProperty &) override;
bool PreVisit(ScanAllById &) override;
bool PreVisit(EmptyResult &) override;
bool PreVisit(Produce &) override;
bool PreVisit(Accumulate &) override;
bool PreVisit(Aggregate &) override;

View File

@@ -11,10 +11,11 @@
#include "query/plan/read_write_type_checker.hpp"
#define PRE_VISIT(TOp, RWType, continue_visiting) \
bool ReadWriteTypeChecker::PreVisit(TOp &op) { \
UpdateType(RWType); \
return continue_visiting; \
// NOLINTNEXTLINE(cppcoreguidelines-macro-usage)
#define PRE_VISIT(TOp, RWType, continue_visiting) \
bool ReadWriteTypeChecker::PreVisit(TOp &) { /*NOLINT(bugprone-macro-parentheses)*/ \
UpdateType(RWType); \
return continue_visiting; \
}
namespace memgraph::query::plan {
@@ -54,6 +55,7 @@ bool ReadWriteTypeChecker::PreVisit(Cartesian &op) {
return false;
}
PRE_VISIT(EmptyResult, RWType::NONE, true)
PRE_VISIT(Produce, RWType::NONE, true)
PRE_VISIT(Accumulate, RWType::NONE, true)
PRE_VISIT(Aggregate, RWType::NONE, true)
@@ -86,7 +88,7 @@ bool ReadWriteTypeChecker::PreVisit([[maybe_unused]] Foreach &op) {
#undef PRE_VISIT
bool ReadWriteTypeChecker::Visit(Once &op) { return false; }
bool ReadWriteTypeChecker::Visit(Once &) { return false; } // NOLINT(hicpp-named-parameter)
void ReadWriteTypeChecker::UpdateType(RWType op_type) {
// Update type only if it's not the NONE type and the current operator's type

View File

@@ -73,6 +73,7 @@ class ReadWriteTypeChecker : public virtual HierarchicalLogicalOperatorVisitor {
bool PreVisit(Optional &) override;
bool PreVisit(Cartesian &) override;
bool PreVisit(EmptyResult &) override;
bool PreVisit(Produce &) override;
bool PreVisit(Accumulate &) override;
bool PreVisit(Aggregate &) override;

View File

@@ -298,6 +298,15 @@ class IndexLookupRewriter final : public HierarchicalLogicalOperatorVisitor {
return true;
}
bool PreVisit(EmptyResult &op) override {
prev_ops_.push_back(&op);
return true;
}
bool PostVisit(EmptyResult &) override {
prev_ops_.pop_back();
return true;
}
bool PreVisit(Delete &op) override {
prev_ops_.push_back(&op);
return true;

View File

@@ -344,7 +344,8 @@ class ReturnBodyContext : public HierarchicalTreeVisitor {
bool PostVisit(Aggregation &aggr) override {
// Aggregation contains a virtual symbol, where the result will be stored.
const auto &symbol = symbol_table_.at(aggr);
aggregations_.emplace_back(Aggregate::Element{aggr.expression1_, aggr.expression2_, aggr.op_, symbol});
aggregations_.emplace_back(
Aggregate::Element{aggr.expression1_, aggr.expression2_, aggr.op_, symbol, aggr.distinct_});
// Aggregation expression1_ is optional in COUNT(*), and COLLECT_MAP uses
// two expressions, so we can have 0, 1 or 2 elements on the
// has_aggregation_stack for this Aggregation expression.

View File

@@ -180,7 +180,7 @@ class RuleBasedPlanner {
}
}
uint64_t merge_id = 0;
for (auto *clause : query_part.remaining_clauses) {
for (const auto &clause : query_part.remaining_clauses) {
MG_ASSERT(!utils::IsSubtype(*clause, Match::kType), "Unexpected Match in remaining clauses");
if (auto *ret = utils::Downcast<Return>(clause)) {
input_op = impl::GenReturn(*ret, std::move(input_op), *context.symbol_table, is_write, context.bound_symbols,
@@ -203,6 +203,7 @@ class RuleBasedPlanner {
context.bound_symbols.insert(symbol);
input_op =
std::make_unique<plan::Unwind>(std::move(input_op), unwind->named_expression_->expression_, symbol);
} else if (auto *call_proc = utils::Downcast<query::CallProcedure>(clause)) {
std::vector<Symbol> result_symbols;
result_symbols.reserve(call_proc->result_identifiers_.size());
@@ -224,6 +225,7 @@ class RuleBasedPlanner {
input_op =
std::make_unique<plan::LoadCsv>(std::move(input_op), load_csv->file_, load_csv->with_header_,
load_csv->ignore_bad_, load_csv->delimiter_, load_csv->quote_, row_sym);
} else if (auto *foreach = utils::Downcast<query::Foreach>(clause)) {
is_write = true;
input_op = HandleForeachClause(foreach, std::move(input_op), *context.symbol_table, context.bound_symbols,
@@ -233,6 +235,10 @@ class RuleBasedPlanner {
}
}
}
// Is this the only situation that should be covered
if (input_op->OutputSymbols(*context.symbol_table).empty()) {
input_op = std::make_unique<EmptyResult>(std::move(input_op));
}
return input_op;
}
@@ -418,7 +424,8 @@ class RuleBasedPlanner {
std::optional<ExpansionLambda> weight_lambda;
std::optional<Symbol> total_weight;
if (edge->type_ == EdgeAtom::Type::WEIGHTED_SHORTEST_PATH || edge->type_ == EdgeAtom::Type::ALL_SHORTEST_PATHS) {
if (edge->type_ == EdgeAtom::Type::WEIGHTED_SHORTEST_PATH ||
edge->type_ == EdgeAtom::Type::ALL_SHORTEST_PATHS) {
weight_lambda.emplace(ExpansionLambda{symbol_table.at(*edge->weight_lambda_.inner_edge),
symbol_table.at(*edge->weight_lambda_.inner_node),
edge->weight_lambda_.expression});

View File

@@ -22,6 +22,7 @@
#include <type_traits>
#include <utility>
#include "license/license.hpp"
#include "mg_procedure.h"
#include "module.hpp"
#include "query/frontend/ast/ast.hpp"
@@ -32,7 +33,6 @@
#include "storage/v2/view.hpp"
#include "utils/algorithm.hpp"
#include "utils/concepts.hpp"
#include "utils/license.hpp"
#include "utils/logging.hpp"
#include "utils/math.hpp"
#include "utils/memory.hpp"
@@ -1653,7 +1653,7 @@ mgp_error mgp_vertex_set_property(struct mgp_vertex *v, const char *property_nam
auto *ctx = v->graph->ctx;
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast() && ctx && ctx->auth_checker &&
if (memgraph::license::global_license_checker.IsEnterpriseValidFast() && ctx && ctx->auth_checker &&
!ctx->auth_checker->Has(v->getImpl(), v->graph->view,
memgraph::query::AuthQuery::FineGrainedPrivilege::UPDATE)) {
throw AuthorizationException{"Insufficient permissions for setting a property on vertex!"};
@@ -1706,7 +1706,7 @@ mgp_error mgp_vertex_add_label(struct mgp_vertex *v, mgp_label label) {
const auto label_id = std::visit([label](auto *impl) { return impl->NameToLabel(label.name); }, v->graph->impl);
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast() && ctx && ctx->auth_checker &&
if (memgraph::license::global_license_checker.IsEnterpriseValidFast() && ctx && ctx->auth_checker &&
!(ctx->auth_checker->Has(v->getImpl(), v->graph->view,
memgraph::query::AuthQuery::FineGrainedPrivilege::UPDATE) &&
ctx->auth_checker->Has({label_id}, memgraph::query::AuthQuery::FineGrainedPrivilege::CREATE_DELETE))) {
@@ -1750,7 +1750,7 @@ mgp_error mgp_vertex_remove_label(struct mgp_vertex *v, mgp_label label) {
const auto label_id = std::visit([&label](auto *impl) { return impl->NameToLabel(label.name); }, v->graph->impl);
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast() && ctx && ctx->auth_checker &&
if (memgraph::license::global_license_checker.IsEnterpriseValidFast() && ctx && ctx->auth_checker &&
!(ctx->auth_checker->Has(v->getImpl(), v->graph->view,
memgraph::query::AuthQuery::FineGrainedPrivilege::UPDATE) &&
ctx->auth_checker->Has({label_id}, memgraph::query::AuthQuery::FineGrainedPrivilege::CREATE_DELETE))) {
@@ -1846,7 +1846,8 @@ mgp_error mgp_vertex_label_at(mgp_vertex *v, size_t i, mgp_label *result) {
"Expected LabelToName to return a pointer or reference, so we "
"don't have to take a copy and manage memory.");
const auto &name = std::visit([label](const auto *impl) { return impl->LabelToName(label); }, v->graph->impl);
const auto &name = std::visit(
[label](const auto *impl) -> const std::string & { return impl->LabelToName(label); }, v->graph->impl);
return name.c_str();
},
&result->name);
@@ -1988,7 +1989,7 @@ mgp_error mgp_vertex_iter_in_edges(mgp_vertex *v, mgp_memory *memory, mgp_edges_
it->in.emplace(std::move(*maybe_edges));
it->in_it.emplace(it->in->begin());
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
NextPermittedEdge(*it, true);
}
#endif
@@ -2040,7 +2041,7 @@ mgp_error mgp_vertex_iter_out_edges(mgp_vertex *v, mgp_memory *memory, mgp_edges
it->out_it.emplace(it->out->begin());
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
NextPermittedEdge(*it, false);
}
#endif
@@ -2098,7 +2099,7 @@ mgp_error mgp_edges_iterator_next(mgp_edges_iterator *it, mgp_edge **result) {
++*impl_it;
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
NextPermittedEdge(*it, for_in);
}
#endif
@@ -2123,10 +2124,7 @@ mgp_error mgp_edges_iterator_next(mgp_edges_iterator *it, mgp_edge **result) {
return &*it->current_e;
};
if (it->in_it) {
auto *result = next(true);
if (result != nullptr) {
return result;
}
return next(true);
}
return next(false);
},
@@ -2157,8 +2155,9 @@ mgp_error mgp_edge_equal(mgp_edge *e1, mgp_edge *e2, int *result) {
mgp_error mgp_edge_get_type(mgp_edge *e, mgp_edge_type *result) {
return WrapExceptions(
[e] {
const auto &name =
std::visit([e](const auto *impl) { return impl->EdgeTypeToName(e->impl.EdgeType()); }, e->from.graph->impl);
const auto &name = std::visit(
[e](const auto *impl) -> const std::string & { return impl->EdgeTypeToName(e->impl.EdgeType()); },
e->from.graph->impl);
return name.c_str();
},
&result->name);
@@ -2203,7 +2202,7 @@ mgp_error mgp_edge_set_property(struct mgp_edge *e, const char *property_name, m
auto *ctx = e->from.graph->ctx;
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast() && ctx && ctx->auth_checker &&
if (memgraph::license::global_license_checker.IsEnterpriseValidFast() && ctx && ctx->auth_checker &&
!ctx->auth_checker->Has(e->impl, memgraph::query::AuthQuery::FineGrainedPrivilege::UPDATE)) {
throw AuthorizationException{"Insufficient permissions for setting a property on edge!"};
}
@@ -2311,7 +2310,7 @@ mgp_error mgp_graph_create_vertex(struct mgp_graph *graph, mgp_memory *memory, m
[=]() -> mgp_vertex * {
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast() && graph->ctx &&
if (memgraph::license::global_license_checker.IsEnterpriseValidFast() && graph->ctx &&
graph->ctx->auth_checker &&
!graph->ctx->auth_checker->HasGlobalPrivilegeOnVertices(
memgraph::query::AuthQuery::FineGrainedPrivilege::CREATE_DELETE)) {
@@ -2341,7 +2340,7 @@ mgp_error mgp_graph_delete_vertex(struct mgp_graph *graph, mgp_vertex *vertex) {
auto *ctx = graph->ctx;
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast() && ctx && ctx->auth_checker &&
if (memgraph::license::global_license_checker.IsEnterpriseValidFast() && ctx && ctx->auth_checker &&
!ctx->auth_checker->Has(vertex->getImpl(), graph->view,
memgraph::query::AuthQuery::FineGrainedPrivilege::CREATE_DELETE)) {
throw AuthorizationException{"Insufficient permissions for deleting a vertex!"};
@@ -2392,7 +2391,7 @@ mgp_error mgp_graph_detach_delete_vertex(struct mgp_graph *graph, mgp_vertex *ve
return WrapExceptions([=] {
auto *ctx = graph->ctx;
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast() && ctx && ctx->auth_checker &&
if (memgraph::license::global_license_checker.IsEnterpriseValidFast() && ctx && ctx->auth_checker &&
!ctx->auth_checker->Has(vertex->getImpl(), graph->view,
memgraph::query::AuthQuery::FineGrainedPrivilege::CREATE_DELETE)) {
throw AuthorizationException{"Insufficient permissions for deleting a vertex!"};
@@ -2456,7 +2455,7 @@ mgp_error mgp_graph_create_edge(mgp_graph *graph, mgp_vertex *from, mgp_vertex *
#ifdef MG_ENTERPRISE
const auto edge_id =
std::visit([type](auto *impl) { return impl->NameToEdgeType(type.name); }, from->graph->impl);
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast() && ctx && ctx->auth_checker &&
if (memgraph::license::global_license_checker.IsEnterpriseValidFast() && ctx && ctx->auth_checker &&
!ctx->auth_checker->Has(edge_id, memgraph::query::AuthQuery::FineGrainedPrivilege::CREATE_DELETE)) {
throw AuthorizationException{"Insufficient permissions for creating edges!"};
}
@@ -2517,7 +2516,7 @@ mgp_error mgp_graph_delete_edge(struct mgp_graph *graph, mgp_edge *edge) {
return WrapExceptions([=] {
auto *ctx = graph->ctx;
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast() && ctx && ctx->auth_checker &&
if (memgraph::license::global_license_checker.IsEnterpriseValidFast() && ctx && ctx->auth_checker &&
!ctx->auth_checker->Has(edge->impl, memgraph::query::AuthQuery::FineGrainedPrivilege::CREATE_DELETE)) {
throw AuthorizationException{"Insufficient permissions for deleting an edge!"};
}
@@ -2579,7 +2578,7 @@ mgp_vertices_iterator::mgp_vertices_iterator(mgp_graph *graph, memgraph::utils::
vertices(std::visit([graph](auto *impl) { return impl->Vertices(graph->view); }, graph->impl)),
current_it(vertices.begin()) {
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
NextPermitted(*this);
}
#endif
@@ -2628,7 +2627,7 @@ mgp_error mgp_vertices_iterator_next(mgp_vertices_iterator *it, mgp_vertex **res
++it->current_it;
#ifdef MG_ENTERPRISE
if (memgraph::utils::license::global_license_checker.IsValidLicenseFast()) {
if (memgraph::license::global_license_checker.IsEnterpriseValidFast()) {
NextPermitted(*it);
}
#endif

View File

@@ -33,6 +33,28 @@ extern "C" {
namespace memgraph::query::procedure {
constexpr const char *func_code =
"import ast\n\n"
"no_removals = ['collections', 'abc', 'sys']\n"
"modules = set()\n\n"
"def visit_Import(node):\n"
" for name in node.names:\n"
" mod_name = name.name.split('.')[0]\n"
" if mod_name not in no_removals:\n"
" modules.add(mod_name)\n\n"
"def visit_ImportFrom(node):\n"
" if node.module is not None and node.level == 0:\n"
" mod_name = node.module.split('.')[0]\n"
" if mod_name not in no_removals:\n"
" modules.add(mod_name)\n"
"node_iter = ast.NodeVisitor()\n"
"node_iter.visit_Import = visit_Import\n"
"node_iter.visit_ImportFrom = visit_ImportFrom\n"
"node_iter.visit(ast.parse(code))\n";
void ProcessFileDependencies(std::filesystem::path file_path_, const char *module_path, const char *func_code,
PyObject *sys_mod_ref);
ModuleRegistry gModuleRegistry;
Module::~Module() {}
@@ -995,11 +1017,45 @@ bool PythonModule::Close() {
procedures_.clear();
transformations_.clear();
functions_.clear();
// Delete the module from the `sys.modules` directory so that the module will
// be properly imported if imported again.
// Get the reference to sys.modules dictionary
py::Object sys(PyImport_ImportModule("sys"));
if (PyDict_DelItemString(sys.GetAttr("modules").Ptr(), file_path_.stem().c_str()) != 0) {
spdlog::warn("Failed to remove the module from sys.modules");
PyObject *sys_mod_ref = sys.GetAttr("modules").Ptr();
std::string stem = file_path_.stem().string();
ProcessFileDependencies(file_path_, file_path_.stem().c_str(), func_code, sys_mod_ref);
std::vector<std::filesystem::path> submodules;
for (auto it = std::filesystem::recursive_directory_iterator(file_path_.parent_path());
it != std::filesystem::recursive_directory_iterator(); ++it) {
std::string dir_entry_stem = it->path().stem().string();
if (it->is_regular_file() || dir_entry_stem == "__pycache__") continue;
if (dir_entry_stem.find(stem) != std::string_view::npos) {
it.disable_recursion_pending();
submodules.emplace_back(it->path());
}
}
for (const auto &submodule : submodules) {
if (std::filesystem::exists(submodule)) {
std::filesystem::remove_all(submodule / "__pycache__");
for (auto const &rec_dir_entry : std::filesystem::recursive_directory_iterator(submodule)) {
std::string rec_dir_entry_stem = rec_dir_entry.path().stem().string();
if (rec_dir_entry.is_directory() && rec_dir_entry_stem != "__pycache__") {
std::filesystem::remove_all(rec_dir_entry.path() / "__pycache__");
}
std::string rec_dir_entry_ext = rec_dir_entry.path().extension().string();
if (!rec_dir_entry.is_regular_file() || rec_dir_entry_ext != ".py") continue;
ProcessFileDependencies(rec_dir_entry.path().c_str(), file_path_.stem().c_str(), func_code, sys_mod_ref);
}
}
}
// first throw out of cache file
if (PyDict_DelItemString(sys_mod_ref, file_path_.stem().c_str()) != 0) {
spdlog::warn("Failed to remove the module {} from sys.modules", file_path_.stem().c_str());
py_module_ = py::Object(nullptr);
return false;
}
@@ -1011,6 +1067,51 @@ bool PythonModule::Close() {
return true;
}
void ProcessFileDependencies(std::filesystem::path file_path_, const char *module_path, const char *func_code,
PyObject *sys_mod_ref) {
const auto maybe_content =
ReadFile(file_path_); // this is already done at Load so it can somehow be optimized but not sure how yet
if (maybe_content) {
const char *content_value = maybe_content->c_str();
if (content_value) {
PyObject *py_main = PyImport_ImportModule("__main__");
PyObject *py_global_dict = PyModule_GetDict(py_main);
PyDict_SetItemString(py_global_dict, "code", PyUnicode_FromString(content_value));
PyRun_String(func_code, Py_file_input, py_global_dict, py_global_dict);
PyObject *py_res = PyDict_GetItemString(py_global_dict, "modules");
PyObject *iterator = PyObject_GetIter(py_res);
PyObject *module = nullptr;
if (iterator != nullptr) {
while ((module = PyIter_Next(iterator))) {
const char *module_name = PyUnicode_AsUTF8(module);
auto module_name_str = std::string(module_name);
PyObject *sys_iterator = PyObject_GetIter(PyDict_Keys(sys_mod_ref));
if (sys_iterator == nullptr) {
spdlog::warn("Cannot get reference to the sys.modules.keys()");
break;
}
PyObject *sys_mod_key = nullptr;
while ((sys_mod_key = PyIter_Next(sys_iterator))) {
const char *sys_mod_key_name = PyUnicode_AsUTF8(sys_mod_key);
auto sys_mod_key_name_str = std::string(sys_mod_key_name);
if (sys_mod_key_name_str.rfind(module_name_str, 0) == 0 && sys_mod_key_name_str.compare(module_path) != 0) {
PyDict_DelItemString(sys_mod_ref, sys_mod_key_name); // don't test output
}
Py_DECREF(sys_mod_key);
}
Py_DECREF(sys_iterator);
Py_DECREF(module);
}
Py_DECREF(iterator);
}
}
}
}
const std::map<std::string, mgp_proc, std::less<>> *PythonModule::Procedures() const {
MG_ASSERT(py_module_,
"Attempting to access procedures of a module that has "

View File

@@ -1,12 +1,12 @@
set(telemetry_src_files
collectors.cpp
telemetry.cpp
system_info.cpp)
collectors.cpp
telemetry.cpp)
add_library(telemetry_lib STATIC ${telemetry_src_files})
target_link_libraries(telemetry_lib mg-requests mg-kvstore mg-utils)
add_library(mg-telemetry STATIC ${telemetry_src_files})
target_link_libraries(mg-telemetry mg-requests mg-kvstore mg-utils)
option(MG_TELEMETRY_ID_OVERRIDE "Override for the telemetry ID" STRING)
if (MG_TELEMETRY_ID_OVERRIDE)
if(MG_TELEMETRY_ID_OVERRIDE)
message(WARNING "Using telemetry ID override: ${MG_TELEMETRY_ID_OVERRIDE}")
target_compile_definitions(telemetry_lib PRIVATE MG_TELEMETRY_ID_OVERRIDE="${MG_TELEMETRY_ID_OVERRIDE}")
target_compile_definitions(mg-telemetry PRIVATE MG_TELEMETRY_ID_OVERRIDE="${MG_TELEMETRY_ID_OVERRIDE}")
endif()

View File

@@ -17,38 +17,24 @@
#include "requests/requests.hpp"
#include "telemetry/collectors.hpp"
#include "telemetry/system_info.hpp"
#include "utils/file.hpp"
#include "utils/logging.hpp"
#include "utils/system_info.hpp"
#include "utils/timestamp.hpp"
#include "utils/uuid.hpp"
namespace memgraph::telemetry {
namespace {
std::string GetMachineId() {
#ifdef MG_TELEMETRY_ID_OVERRIDE
return MG_TELEMETRY_ID_OVERRIDE;
#else
// We assume we're on linux and we need to read the machine id from /etc/machine-id
const auto machine_id_lines = utils::ReadLines("/etc/machine-id");
if (machine_id_lines.size() != 1) {
return "UNKNOWN";
}
return machine_id_lines[0];
#endif
}
} // namespace
const int kMaxBatchSize = 100;
constexpr auto kMaxBatchSize{100};
Telemetry::Telemetry(std::string url, std::filesystem::path storage_directory,
Telemetry::Telemetry(std::string url, std::filesystem::path storage_directory, std::string uuid, std::string machine_id,
std::chrono::duration<int64_t> refresh_interval, const uint64_t send_every_n)
: url_(std::move(url)),
uuid_(utils::GenerateUUID()),
machine_id_(GetMachineId()),
uuid_(uuid),
machine_id_(machine_id),
send_every_n_(send_every_n),
storage_(std::move(storage_directory)) {
StoreData("startup", GetSystemInfo());
StoreData("startup", utils::GetSystemInfo());
AddCollector("resources", GetResourceUsage);
AddCollector("uptime", [&]() -> nlohmann::json { return GetUptime(); });
scheduler_.Run("Telemetry", refresh_interval, [&] { CollectData(); });
@@ -59,19 +45,15 @@ void Telemetry::AddCollector(const std::string &name, const std::function<const
collectors_.emplace_back(name, func);
}
std::string Telemetry::GetRunId() const { return uuid_; }
Telemetry::~Telemetry() {
scheduler_.Stop();
CollectData("shutdown");
}
void Telemetry::StoreData(const nlohmann::json &event, const nlohmann::json &data) {
nlohmann::json payload = {{"run_id", uuid_},
{"machine_id", machine_id_},
{"event", event},
{"data", data},
{"timestamp", utils::Timestamp::Now().SecWithNsecSinceTheEpoch()}};
nlohmann::json payload = {
{"run_id", uuid_}, {"type", "telemetry"}, {"machine_id", machine_id_},
{"event", event}, {"data", data}, {"timestamp", utils::Timestamp::Now().SecWithNsecSinceTheEpoch()}};
storage_.Put(fmt::format("{}:{}", uuid_, event.dump()), payload.dump());
}

View File

@@ -26,7 +26,7 @@ namespace memgraph::telemetry {
* This class implements the telemetry collector service. It periodically scapes
* all registered collectors and stores their data. With periodically scraping
* the collectors the service collects machine information in the constructor
* and stores it. Also, it calles all collectors once more in the destructor so
* and stores it. Also, it calls all collectors once more in the destructor so
* that final stats can be collected. All data is stored persistently. If there
* is no internet connection the data will be sent when the internet connection
* is reestablished. If there is an issue with the internet connection that
@@ -34,14 +34,11 @@ namespace memgraph::telemetry {
*/
class Telemetry final {
public:
Telemetry(std::string url, std::filesystem::path storage_directory,
Telemetry(std::string url, std::filesystem::path storage_directory, std::string uuid, std::string machine_id,
std::chrono::duration<int64_t> refresh_interval = std::chrono::minutes(10), uint64_t send_every_n = 10);
void AddCollector(const std::string &name, const std::function<const nlohmann::json(void)> &func);
/// Required to expose run_id to Bolt server.
std::string GetRunId() const;
~Telemetry();
Telemetry(const Telemetry &) = delete;

View File

@@ -14,6 +14,7 @@ set(utils_src_files
thread.cpp
thread_pool.cpp
tsc.cpp
system_info.cpp
uuid.cpp)
find_package(Boost REQUIRED)
@@ -23,16 +24,10 @@ find_package(Threads REQUIRED)
add_library(mg-utils STATIC ${utils_src_files})
target_link_libraries(mg-utils PUBLIC Boost::headers fmt::fmt spdlog::spdlog)
target_link_libraries(mg-utils PRIVATE librdtsc stdc++fs Threads::Threads gflags uuid rt)
target_link_libraries(mg-utils PRIVATE librdtsc stdc++fs Threads::Threads gflags json uuid rt)
set(settings_src_files
settings.cpp)
add_library(mg-settings STATIC ${settings_src_files})
target_link_libraries(mg-settings mg-kvstore mg-slk mg-utils)
set(license_src_files
license.cpp)
add_library(mg-license STATIC ${license_src_files})
target_link_libraries(mg-license mg-settings mg-utils)

View File

@@ -37,6 +37,7 @@
M(RemovePropertyOperator, "Number of times RemoveProperty operator was used.") \
M(RemoveLabelsOperator, "Number of times RemoveLabels operator was used.") \
M(EdgeUniquenessFilterOperator, "Number of times EdgeUniquenessFilter operator was used.") \
M(EmptyResultOperator, "Number of times EmptyResult operator was used.") \
M(AccumulateOperator, "Number of times Accumulate operator was used.") \
M(AggregateOperator, "Number of times Aggregate operator was used.") \
M(SkipOperator, "Number of times Skip operator was used.") \

View File

@@ -43,7 +43,7 @@ inline uint64_t GetMemoryUsage() {
pid_t pid = getpid();
uint64_t memory = 0;
auto statm_data = utils::ReadLines(fmt::format("/proc/{}/statm", pid));
if (statm_data.size() >= 1) {
if (!statm_data.empty()) {
auto split = utils::Split(statm_data[0]);
if (split.size() >= 2) {
memory = std::stoull(split[1]) * sysconf(_SC_PAGESIZE);

View File

@@ -9,56 +9,35 @@
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#include "telemetry/system_info.hpp"
#include "utils/system_info.hpp"
#include <string>
#include <sys/utsname.h>
#include <gflags/gflags.h>
#include <sys/utsname.h>
#include "utils/file.hpp"
#include "utils/string.hpp"
namespace memgraph::telemetry {
namespace memgraph::utils {
const nlohmann::json GetSystemInfo() {
// Get `uname`.
struct utsname info;
if (uname(&info) != 0) return {};
// Parse `/etc/os-release`.
std::string os_name, os_version, os_full;
auto os_data = utils::ReadLines("/etc/os-release");
for (auto &row : os_data) {
auto split = utils::Split(row, "=");
if (split.size() < 2) continue;
if (split[0] == "NAME") {
os_name = utils::Trim(split[1], "\"");
} else if (split[0] == "VERSION") {
os_version = utils::Trim(split[1], "\"");
}
os_full = fmt::format("{} {}", os_name, os_version);
}
// Parse `/proc/cpuinfo`.
std::string cpu_model;
uint64_t cpu_count = 0;
auto cpu_data = utils::ReadLines("/proc/cpuinfo");
for (auto &row : cpu_data) {
auto tmp = utils::Trim(row);
if (tmp == "") {
++cpu_count;
} else if (utils::StartsWith(tmp, "model name")) {
auto split = utils::Split(tmp, ":");
if (split.size() != 2) continue;
cpu_model = utils::Trim(split[1]);
}
std::string GetMachineId() {
#ifdef MG_TELEMETRY_ID_OVERRIDE
return MG_TELEMETRY_ID_OVERRIDE;
#else
// We assume we're on linux and we need to read the machine id from /etc/machine-id
const auto machine_id_lines = memgraph::utils::ReadLines("/etc/machine-id");
if (machine_id_lines.size() != 1) {
return "UNKNOWN";
}
return machine_id_lines[0];
#endif
}
MemoryInfo GetMemoryInfo() {
// Parse `/proc/meminfo`.
nlohmann::json ret;
uint64_t memory = 0, swap = 0;
uint64_t memory{0};
uint64_t swap{0};
auto mem_data = utils::ReadLines("/proc/meminfo");
for (auto &row : mem_data) {
auto tmp = utils::Trim(row);
@@ -74,15 +53,55 @@ const nlohmann::json GetSystemInfo() {
}
memory *= 1024;
swap *= 1024;
return {{"architecture", info.machine},
{"cpu_count", cpu_count},
{"cpu_model", cpu_model},
{"kernel", fmt::format("{} {}", info.release, info.version)},
{"memory", memory},
{"os", os_full},
{"swap", swap},
{"version", gflags::VersionString()}};
return {memory, swap};
}
} // namespace memgraph::telemetry
CPUInfo GetCPUInfo() {
// Parse `/proc/cpuinfo`.
std::string cpu_model;
uint64_t cpu_count{0};
auto cpu_data = utils::ReadLines("/proc/cpuinfo");
for (auto &row : cpu_data) {
auto tmp = utils::Trim(row);
if (tmp.empty()) {
++cpu_count;
} else if (utils::StartsWith(tmp, "model name")) {
auto split = utils::Split(tmp, ":");
if (split.size() != 2) continue;
cpu_model = utils::Trim(split[1]);
}
}
return {cpu_model, cpu_count};
}
nlohmann::json GetSystemInfo() {
// Get `uname`.
struct utsname info;
if (uname(&info) != 0) return {};
// Parse `/etc/os-release`.
std::string os_name;
std::string os_version;
std::string os_full;
auto os_data = utils::ReadLines("/etc/os-release");
for (auto &row : os_data) {
auto split = utils::Split(row, "=");
if (split.size() < 2) continue;
if (split[0] == "NAME") {
os_name = utils::Trim(split[1], "\"");
} else if (split[0] == "VERSION") {
os_version = utils::Trim(split[1], "\"");
}
os_full = fmt::format("{} {}", os_name, os_version);
}
const auto cpu_info = GetCPUInfo();
const auto mem_info = GetMemoryInfo();
return {{"architecture", info.machine}, {"cpu_count", cpu_info.cpu_count},
{"cpu_model", cpu_info.cpu_model}, {"kernel", fmt::format("{} {}", info.release, info.version)},
{"memory", mem_info.memory}, {"os", os_full},
{"swap", mem_info.swap}, {"version", gflags::VersionString()}};
}
} // namespace memgraph::utils

View File

@@ -11,16 +11,33 @@
#pragma once
#include <cstdint>
#include <string>
#include <json/json.hpp>
namespace memgraph::telemetry {
namespace memgraph::utils {
// TODO (mferencevic): merge with `utils/sysinfo`
struct MemoryInfo {
uint64_t memory;
uint64_t swap;
};
struct CPUInfo {
std::string cpu_model;
uint64_t cpu_count;
};
std::string GetMachineId();
MemoryInfo GetMemoryInfo();
CPUInfo GetCPUInfo();
/**
* This function returs a dictionary containing some basic system information
* This function return a dictionary containing some basic system information
* (eg. operating system name, cpu information, memory information, etc.).
*/
const nlohmann::json GetSystemInfo();
nlohmann::json GetSystemInfo();
} // namespace memgraph::telemetry
} // namespace memgraph::utils

View File

@@ -43,6 +43,7 @@ add_subdirectory(magic_functions)
add_subdirectory(module_file_manager)
add_subdirectory(monitoring_server)
add_subdirectory(lba_procedures)
add_subdirectory(python_query_modules_reloading)
copy_e2e_python_files(pytest_runner pytest_runner.sh "")
file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/memgraph-selfsigned.crt DESTINATION ${CMAKE_CURRENT_BINARY_DIR})

View File

@@ -10,10 +10,10 @@
# licenses/APL.txt.
import sys
import mgclient
import pytest
import default_config
import mgclient
import pytest
def test_does_default_config_match():
@@ -24,7 +24,15 @@ def test_does_default_config_match():
cursor.execute("SHOW CONFIG")
config = cursor.fetchall()
assert len(config) == len(default_config.startup_config_dict)
define_msg = """
If this test fails after adding a new DEFINE_* flag,
you should decide whether your new flag needs to be
returned in the SHOW CONFIG command. If not, please
use the DEFINE_HIDDEN_* macro instead of DEFINE_* to
prevent SHOW CONFIG from returning it.
"""
assert len(config) == len(default_config.startup_config_dict), define_msg
for flag in config:
flag_name = flag[0]

View File

@@ -163,4 +163,10 @@ startup_config_dict = {
),
"query_max_plans": ("1000", "1000", "Maximum number of generated plans for a query."),
"flag_file": ("", "", "load flags from file"),
"init_file": (
"",
"",
"Path to cypherl file that is used for configuring users and database schema before server starts.",
),
"init_data_file": ("", "", "Path to cypherl file that is used for creating data after server starts."),
}

View File

@@ -1,7 +1,14 @@
template_cluster: &template_cluster
cluster:
main:
args: ["--log-level=TRACE", "--storage-properties-on-edges=True", "--storage-snapshot-interval-sec", "300", "--storage-wal-enabled=True"]
args:
[
"--log-level=TRACE",
"--storage-properties-on-edges=True",
"--storage-snapshot-interval-sec",
"300",
"--storage-wal-enabled=True",
]
log_file: "configuration-check-e2e.log"
setup_queries: []
validation_queries: []

View File

@@ -0,0 +1,8 @@
function(copy_query_modules_reloading_procedures_e2e_python_files FILE_NAME)
copy_e2e_python_files(python_query_modules_reloading ${FILE_NAME})
endfunction()
copy_query_modules_reloading_procedures_e2e_python_files(common.py)
copy_query_modules_reloading_procedures_e2e_python_files(test_reload_query_module.py)
add_subdirectory(procedures)

View File

@@ -0,0 +1,25 @@
# Copyright 2022 Memgraph Ltd.
#
# Use of this software is governed by the Business Source License
# included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
# License, and you may not use this file except in compliance with the Business Source License.
#
# As of the Change Date specified in that file, in accordance with
# the Business Source License, use of this software will be governed
# by the Apache License, Version 2.0, included in the file
# licenses/APL.txt.
import typing
import mgclient
def execute_and_fetch_all(cursor: mgclient.Cursor, query: str, params: dict = {}) -> typing.List[tuple]:
cursor.execute(query, params)
return cursor.fetchall()
def connect(**kwargs) -> mgclient.Connection:
connection = mgclient.connect(host="localhost", port=7687, **kwargs)
connection.autocommit = True
return connection

View File

@@ -0,0 +1,6 @@
copy_query_modules_reloading_procedures_e2e_python_files(test_module.py)
copy_query_modules_reloading_procedures_e2e_python_files(new_test_module.py)
add_subdirectory(mage)
add_subdirectory(new_test_module_utils)

View File

@@ -0,0 +1 @@
add_subdirectory(test_module)

View File

@@ -0,0 +1,3 @@
copy_query_modules_reloading_procedures_e2e_python_files(test_functions.py)
add_subdirectory(test_functions_dir)

View File

@@ -0,0 +1,2 @@
def test_function(a: int, b: int) -> int:
return a + b

View File

@@ -0,0 +1 @@
copy_query_modules_reloading_procedures_e2e_python_files(test_subfunctions.py)

View File

@@ -0,0 +1,2 @@
def test_subfunction(a: int, b: int) -> int:
return a * b

View File

@@ -0,0 +1,14 @@
import mgp
# isort: off
# fmt: off
from new_test_module_utils.new_test_functions import \
test_function as test_function1
from new_test_module_utils.new_test_functions_dir.new_test_subfunctions import \
test_subfunction as test_function2
# fmt: on
@mgp.read_proc
def test(ctx: mgp.ProcCtx, a: mgp.Number, b: mgp.Number) -> mgp.Record(result1=mgp.Number, result2=mgp.Number):
return mgp.Record(result1=test_function1(a, b), result2=test_function2(a, b))

View File

@@ -0,0 +1,3 @@
copy_query_modules_reloading_procedures_e2e_python_files(new_test_functions.py)
add_subdirectory(new_test_functions_dir)

View File

@@ -0,0 +1,2 @@
def test_function(a: int, b: int) -> int:
return a + b

View File

@@ -0,0 +1 @@
copy_query_modules_reloading_procedures_e2e_python_files(new_test_subfunctions.py)

View File

@@ -0,0 +1,2 @@
def test_subfunction(a: int, b: int) -> int:
return a * b

View File

@@ -0,0 +1,13 @@
import mgp
from mage.test_module.test_functions import test_function as test_function1
# isort: off
# fmt: off
from mage.test_module.test_functions_dir.test_subfunctions import \
test_subfunction as test_function2
# fmt: on
@mgp.read_proc
def test(ctx: mgp.ProcCtx, a: mgp.Number, b: mgp.Number) -> mgp.Record(result1=mgp.Number, result2=mgp.Number):
return mgp.Record(result1=test_function1(a, b), result2=test_function2(a, b))

View File

@@ -0,0 +1,180 @@
# Copyright 2022 Memgraph Ltd.
#
# Use of this software is governed by the Business Source License
# included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
# License, and you may not use this file except in compliance with the Business Source License.
#
# As of the Change Date specified in that file, in accordance with
# the Business Source License, use of this software will be governed
# by the Apache License, Version 2.0, included in the file
# licenses/APL.txt.
import os # To be removed
import sys
import pytest
from common import connect, execute_and_fetch_all
COMMON_PATH_PREFIX_TEST1 = "procedures/mage/test_module"
COMMON_PATH_PREFIX_TEST2 = "procedures/new_test_module_utils"
FUNC1_PATH = os.path.join(
os.path.dirname(__file__),
COMMON_PATH_PREFIX_TEST1,
"test_functions.py",
)
FUNC2_PATH = os.path.join(
os.path.dirname(__file__),
COMMON_PATH_PREFIX_TEST1,
"test_functions_dir/test_subfunctions.py",
)
FUNC3_PATH = os.path.join(
os.path.dirname(__file__),
COMMON_PATH_PREFIX_TEST2,
"new_test_functions.py",
)
FUNC4_PATH = os.path.join(
os.path.dirname(__file__),
COMMON_PATH_PREFIX_TEST2,
"new_test_functions_dir/new_test_subfunctions.py",
)
def preprocess_functions(path1: str, path2: str):
with open(path1, "w") as func1_file:
func1_file.write(
"""def test_function(a: int, b: int) -> int:
return a - b
"""
)
with open(path2, "w") as func2_file:
func2_file.write(
"""def test_subfunction(a: int, b: int) -> int:
return a / b
"""
)
def postprocess_functions(path1: str, path2: str):
with open(path1, "w") as func1_file:
func1_file.write(
"""def test_function(a: int, b: int) -> int:
return a + b
"""
)
with open(path2, "w") as func2_file:
func2_file.write(
"""def test_subfunction(a: int, b: int) -> int:
return a * b
"""
)
def test_mg_load_reload_submodule_root_utils():
"""Tests whether mg.load reloads content of some submodule code."""
cursor = connect().cursor()
# First do a simple experiment
test_module_res = execute_and_fetch_all(cursor, "CALL new_test_module.test(10, 2) YIELD * RETURN *;")
try:
assert test_module_res[0][0] == 12 # + operator
assert test_module_res[0][1] == 20 # * operator
# Now modify content of test function
preprocess_functions(FUNC3_PATH, FUNC4_PATH)
# Test that it doesn't work without calling reload
test_module_res = execute_and_fetch_all(cursor, "CALL new_test_module.test(10, 2) YIELD * RETURN *;")
assert test_module_res[0][0] == 12 # + operator
assert test_module_res[0][1] == 20 # * operator
# Reload module
execute_and_fetch_all(cursor, "CALL mg.load('new_test_module');")
test_module_res = execute_and_fetch_all(cursor, "CALL new_test_module.test(10, 2) YIELD * RETURN *;")
assert test_module_res[0][0] == 8 # - operator
assert test_module_res[0][1] == 5 # / operator
finally:
# Revert to the original state for the consistency
postprocess_functions(FUNC3_PATH, FUNC4_PATH)
execute_and_fetch_all(cursor, "CALL mg.load('new_test_module');")
def test_mg_load_all_reload_submodule_root_utils():
"""Tests whether mg.load_all reloads content of some submodule code"""
cursor = connect().cursor()
# First do a simple experiment
test_module_res = execute_and_fetch_all(cursor, "CALL new_test_module.test(10, 2) YIELD * RETURN *;")
try:
assert test_module_res[0][0] == 12 # + operator
assert test_module_res[0][1] == 20 # * operator
# Now modify content of test function
preprocess_functions(FUNC3_PATH, FUNC4_PATH)
# Test that it doesn't work without calling reload
test_module_res = execute_and_fetch_all(cursor, "CALL new_test_module.test(10, 2) YIELD * RETURN *;")
assert test_module_res[0][0] == 12 # + operator
assert test_module_res[0][1] == 20 # * operator
# Reload module
execute_and_fetch_all(cursor, "CALL mg.load_all();")
test_module_res = execute_and_fetch_all(cursor, "CALL new_test_module.test(10, 2) YIELD * RETURN *;")
assert test_module_res[0][0] == 8 # - operator
assert test_module_res[0][1] == 5 # / operator
finally:
# Revert to the original state for the consistency
postprocess_functions(FUNC3_PATH, FUNC4_PATH)
execute_and_fetch_all(cursor, "CALL mg.load_all();")
def test_mg_load_reload_submodule():
"""Tests whether mg.load reloads content of some submodule code."""
cursor = connect().cursor()
# First do a simple experiment
test_module_res = execute_and_fetch_all(cursor, "CALL test_module.test(10, 2) YIELD * RETURN *;")
try:
assert test_module_res[0][0] == 12 # + operator
assert test_module_res[0][1] == 20 # * operator
# Now modify content of test function
preprocess_functions(FUNC1_PATH, FUNC2_PATH)
# Test that it doesn't work without calling reload
test_module_res = execute_and_fetch_all(cursor, "CALL test_module.test(10, 2) YIELD * RETURN *;")
assert test_module_res[0][0] == 12 # + operator
assert test_module_res[0][1] == 20 # * operator
# Reload module
execute_and_fetch_all(cursor, "CALL mg.load('test_module');")
test_module_res = execute_and_fetch_all(cursor, "CALL test_module.test(10, 2) YIELD * RETURN *;")
assert test_module_res[0][0] == 8 # - operator
assert test_module_res[0][1] == 5 # / operator
finally:
# Revert to the original state for the consistency
postprocess_functions(FUNC1_PATH, FUNC2_PATH)
execute_and_fetch_all(cursor, "CALL mg.load('test_module');")
def test_mg_load_all_reload_submodule():
"""Tests whether mg.load_all reloads content of some submodule code"""
cursor = connect().cursor()
# First do a simple experiment
test_module_res = execute_and_fetch_all(cursor, "CALL test_module.test(10, 2) YIELD * RETURN *;")
try:
assert test_module_res[0][0] == 12 # + operator
assert test_module_res[0][1] == 20 # * operator
# Now modify content of test function
preprocess_functions(FUNC1_PATH, FUNC2_PATH)
# Test that it doesn't work without calling reload
test_module_res = execute_and_fetch_all(cursor, "CALL test_module.test(10, 2) YIELD * RETURN *;")
assert test_module_res[0][0] == 12 # + operator
assert test_module_res[0][1] == 20 # * operator
# Reload module
execute_and_fetch_all(cursor, "CALL mg.load_all();")
test_module_res = execute_and_fetch_all(cursor, "CALL test_module.test(10, 2) YIELD * RETURN *;")
assert test_module_res[0][0] == 8 # - operator
assert test_module_res[0][1] == 5 # / operator
finally:
# Revert to the original state for the consistency
postprocess_functions(FUNC1_PATH, FUNC2_PATH)
execute_and_fetch_all(cursor, "CALL mg.load_all();")
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-rA"]))

View File

@@ -0,0 +1,14 @@
test_reload_query_module: &test_reload_query_module
cluster:
main:
args: ["--bolt-port", "7687", "--log-level=TRACE", "--also-log-to-stderr"]
log_file: "py-query-modules-reloading-e2e.log"
setup_queries: []
validation_queries: []
workloads:
- name: "test-reload-query-module" # should be the same as the python file
binary: "tests/e2e/pytest_runner.sh"
proc: "tests/e2e/python_query_modules_reloading/procedures/"
args: ["python_query_modules_reloading/test_reload_query_module.py"]
<<: *test_reload_query_module

View File

@@ -97,6 +97,22 @@ Feature: Update clauses
| a | b | c |
| (:q{x: 'y'}) | [:X{x: 'y'}] | ({y: 't'}) |
Scenario: Match node set properties without return
Given an empty graph
And having executed
"""
CREATE (n1:Node {test: 1})
CREATE (n2:Node {test: 2})
CREATE (n3:Node {test: 3})
"""
When executing query:
"""
MATCH (n:Node)
SET n.test = 4
"""
Then the result should be empty
Scenario: Match, set properties from relationship to relationship, return test
Given an empty graph
When executing query:

View File

@@ -13,24 +13,6 @@ add_subdirectory(auth)
# lba test binaries
add_subdirectory(fine_grained_access)
## distributed ha/basic binaries
#add_subdirectory(ha/basic)
#
## distributed ha/constraints binaries
#add_subdirectory(ha/constraints)
#
## distributed ha/index binaries
#add_subdirectory(ha/index)
#
## distributed ha/large_log_entries binaries
#add_subdirectory(ha/large_log_entries)
#
## distributed ha/leader_election binaries
#add_subdirectory(ha/leader_election)
#
## distributed ha/term_updates binaries
#add_subdirectory(ha/term_updates)
# audit test binaries
add_subdirectory(audit)
@@ -39,3 +21,12 @@ add_subdirectory(ldap)
# mg_import_csv test binaries
add_subdirectory(mg_import_csv)
# license_check test binaries
add_subdirectory(license_info)
#environment variable check binaries
add_subdirectory(env_variable_check)
#flag check binaries
add_subdirectory(flag_check)

View File

@@ -0,0 +1,7 @@
set(target_name memgraph__integration__env_variable_check)
set(tester_target_name ${target_name}__tester)
set(env_check_target_name ${target_name}__check)
add_executable(${tester_target_name} tester.cpp)
set_target_properties(${tester_target_name} PROPERTIES OUTPUT_NAME tester)
target_link_libraries(${tester_target_name} mg-communication)

View File

@@ -0,0 +1,151 @@
#!/usr/bin/python3 -u
# Copyright 2022 Memgraph Ltd.
#
# Use of this software is governed by the Business Source License
# included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
# License, and you may not use this file except in compliance with the Business Source License.
#
# As of the Change Date specified in that file, in accordance with
# the Business Source License, use of this software will be governed
# by the Apache License, Version 2.0, included in the file
# licenses/APL.txt.
import argparse
import os
import subprocess
import sys
import tempfile
import time
from pathlib import Path
from typing import List
SCRIPT_DIR = Path(__file__).absolute()
PROJECT_DIR = SCRIPT_DIR.parents[3]
def wait_for_server(port, delay=0.1):
cmd = ["nc", "-z", "-w", "1", "127.0.0.1", str(port)]
while subprocess.call(cmd) != 0:
time.sleep(0.01)
time.sleep(delay)
def execute_tester(
binary: str,
queries: List[str],
should_fail: bool = False,
failure_message: str = "",
username: str = "",
password: str = "",
check_failure: bool = True,
) -> None:
args = [binary, "--username", username, "--password", password]
if should_fail:
args.append("--should-fail")
if failure_message:
args.extend(["--failure-message", failure_message])
if check_failure:
args.append("--check-failure")
args.extend(queries)
subprocess.run(args).check_returncode()
def start_memgraph(memgraph_args: List[any]) -> subprocess:
memgraph = subprocess.Popen(list(map(str, memgraph_args)))
time.sleep(0.1)
assert memgraph.poll() is None, "Memgraph process died prematurely!"
wait_for_server(7687)
return memgraph
def execute_with_user(queries):
return execute_tester(
tester_binary, queries, should_fail=False, check_failure=True, username="admin", password="admin"
)
def cleanup(memgraph):
if memgraph.poll() is None:
memgraph.terminate()
assert memgraph.wait() == 0, "Memgraph process didn't exit cleanly!"
def execute_without_user(queries, should_fail=False, failure_message="", check_failure=True):
return execute_tester(tester_binary, queries, should_fail, failure_message, "", "", check_failure)
def test_without_env_variables(memgraph_args: List[any]) -> None:
memgraph = start_memgraph(memgraph_args)
execute_without_user(["MATCH (n) RETURN n"], False)
cleanup(memgraph)
def test_with_user_password_env_variables(memgraph_args: List[any]) -> None:
os.environ["MEMGRAPH_USER"] = "admin"
os.environ["MEMGRAPH_PASSWORD"] = "admin"
memgraph = start_memgraph(memgraph_args)
execute_with_user(["MATCH (n) RETURN n"])
execute_without_user(["MATCH (n) RETURN n"], True, "Handshake with the server failed!", True)
cleanup(memgraph)
del os.environ["MEMGRAPH_USER"]
del os.environ["MEMGRAPH_PASSWORD"]
def test_with_passfile_env_variable(storage_directory: tempfile.TemporaryDirectory, memgraph_args: List[any]) -> None:
with open(os.path.join(storage_directory.name, "passfile.txt"), "w") as temp_file:
temp_file.write("admin:admin")
os.environ["MEMGRAPH_PASSFILE"] = storage_directory.name + "/passfile.txt"
memgraph = start_memgraph(memgraph_args)
execute_with_user(["MATCH (n) RETURN n"])
execute_without_user(["MATCH (n) RETURN n"], True, "Handshake with the server failed!", True)
del os.environ["MEMGRAPH_PASSFILE"]
cleanup(memgraph)
def execute_test(memgraph_binary: str, tester_binary: str) -> None:
storage_directory = tempfile.TemporaryDirectory()
memgraph_args = [memgraph_binary, "--data-directory", storage_directory.name]
return_to_prev_state = {}
if "MEMGRAPH_USER" in os.environ:
return_to_prev_state["MEMGRAPH_USER"] = os.environ["MEMGRAPH_USER"]
del os.environ["MG_USER"]
if "MEMGRAPH_PASSWORD" in os.environ:
return_to_prev_state["MEMGRAPH_PASSWORD"] = os.environ["MEMGRAPH_PASSWORD"]
del os.environ["MEMGRAPH_PASSWORD"]
if "MEMGRAPH_PASSFILE" in os.environ:
return_to_prev_state["MEMGRAPH_PASSFILE"] = os.environ["MEMGRAPH_PASSFILE"]
del os.environ["MEMGRAPH_PASSFILE"]
# Start the memgraph binary
# Run the test with all combinations of permissions
print("\033[1;36m~~ Starting env variable check test ~~\033[0m")
test_without_env_variables(memgraph_args)
test_with_user_password_env_variables(memgraph_args)
test_with_passfile_env_variable(storage_directory, memgraph_args)
print("\033[1;36m~~ Ended env variable check test ~~\033[0m")
if "MEMGRAPH_USER" in return_to_prev_state:
os.environ["MEMGRAPH_USER"] = return_to_prev_state["MEMGRAPH_USER"]
if "MEMGRAPH_PASSWORD" in return_to_prev_state:
os.environ["MEMGRAPH_PASSWORD"] = return_to_prev_state["MEMGRAPH_PASSWORD"]
if "MEMGRAPH_PASSFILE" in return_to_prev_state:
os.environ["MEMGRAPH_PASSFILE"] = return_to_prev_state["MEMGRAPH_PASSFILE"]
if __name__ == "__main__":
memgraph_binary = os.path.join(PROJECT_DIR, "build", "memgraph")
tester_binary = os.path.join(PROJECT_DIR, "build", "tests", "integration", "env_variable_check", "tester")
parser = argparse.ArgumentParser()
parser.add_argument("--memgraph", default=memgraph_binary)
parser.add_argument("--tester", default=tester_binary)
args = parser.parse_args()
execute_test(args.memgraph, args.tester)
sys.exit(0)

View File

@@ -0,0 +1,94 @@
// Copyright 2022 Memgraph Ltd.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
// License, and you may not use this file except in compliance with the Business Source License.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#include <gflags/gflags.h>
#include "communication/bolt/client.hpp"
#include "io/network/endpoint.hpp"
#include "io/network/utils.hpp"
DEFINE_string(address, "127.0.0.1", "Server address");
DEFINE_int32(port, 7687, "Server port");
DEFINE_string(username, "", "Username for the database");
DEFINE_string(password, "", "Password for the database");
DEFINE_bool(use_ssl, false, "Set to true to connect with SSL to the server.");
DEFINE_bool(check_failure, false, "Set to true to enable failure checking.");
DEFINE_bool(should_fail, false, "Set to true to expect a failure.");
DEFINE_string(failure_message, "", "Set to the expected failure message.");
int ProcessException(const std::string &exception_message) {
if (FLAGS_should_fail) {
if (!FLAGS_failure_message.empty() && exception_message != FLAGS_failure_message) {
LOG_FATAL(
"The query should have failed with an error message of '{}'' but "
"instead it failed with '{}'",
FLAGS_failure_message, exception_message);
}
return 0;
} else {
LOG_FATAL(
"The query shoudn't have failed but it failed with an "
"error message '{}'",
exception_message);
return 1;
}
}
/**
* Executes queries passed as positional arguments and verifies whether they
* succeeded, failed, failed with a specific error message or executed without a
* specific error occurring.
*/
int main(int argc, char **argv) {
gflags::ParseCommandLineFlags(&argc, &argv, true);
memgraph::communication::SSLInit sslInit;
memgraph::io::network::Endpoint endpoint(memgraph::io::network::ResolveHostname(FLAGS_address), FLAGS_port);
memgraph::communication::ClientContext context(FLAGS_use_ssl);
memgraph::communication::bolt::Client client(context);
try {
client.Connect(endpoint, FLAGS_username, FLAGS_password);
} catch (const memgraph::utils::BasicException &e) {
return ProcessException(e.what());
}
for (int i = 1; i < argc; ++i) {
std::string query(argv[i]);
try {
client.Execute(query, {});
} catch (const memgraph::communication::bolt::ClientQueryException &e) {
if (!FLAGS_check_failure) {
if (!FLAGS_failure_message.empty() && e.what() == FLAGS_failure_message) {
LOG_FATAL(
"The query should have succeeded or failed with an error "
"message that isn't equal to '{}' but it failed with that error "
"message",
FLAGS_failure_message);
}
continue;
}
if (!ProcessException(e.what())) {
return 0;
}
}
if (!FLAGS_check_failure) continue;
if (FLAGS_should_fail) {
LOG_FATAL(
"The query should have failed but instead it executed "
"successfully!");
}
}
return 0;
}

View File

@@ -0,0 +1,11 @@
set(target_name memgraph__integration__flag_check)
set(tester_target_name ${target_name}__tester)
set(flag_check_target_name ${target_name}__flag_check)
add_executable(${tester_target_name} tester.cpp)
set_target_properties(${tester_target_name} PROPERTIES OUTPUT_NAME tester)
target_link_libraries(${tester_target_name} mg-communication)
add_executable(${flag_check_target_name} flag_check.cpp)
set_target_properties(${flag_check_target_name} PROPERTIES OUTPUT_NAME flag_check)
target_link_libraries(${flag_check_target_name} mg-communication)

View File

@@ -0,0 +1,59 @@
// Copyright 2022 Memgraph Ltd.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
// License, and you may not use this file except in compliance with the Business Source License.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#include <gflags/gflags.h>
#include <cstdlib>
#include "communication/bolt/client.hpp"
#include "io/network/endpoint.hpp"
#include "io/network/utils.hpp"
#include "utils/logging.hpp"
DEFINE_string(address, "127.0.0.1", "Server address");
DEFINE_int32(port, 7687, "Server port");
DEFINE_string(username, "admin", "Username for the database");
DEFINE_string(password, "admin", "Password for the database");
DEFINE_bool(use_ssl, false, "Set to true to connect with SSL to the server.");
/**
* Verifies that user 'user' has privileges that are given as positional
* arguments.
*/
int main(int argc, char **argv) {
gflags::ParseCommandLineFlags(&argc, &argv, true);
memgraph::communication::SSLInit sslInit;
memgraph::io::network::Endpoint endpoint(memgraph::io::network::ResolveHostname(FLAGS_address), FLAGS_port);
memgraph::communication::ClientContext context(FLAGS_use_ssl);
memgraph::communication::bolt::Client client(context);
client.Connect(endpoint, FLAGS_username, FLAGS_password);
try {
std::string query(argv[1]);
auto ret = client.Execute(query, {});
uint64_t count_got = ret.records.size();
if (count_got != std::atoi(argv[2])) {
LOG_FATAL("Expected the record to have {} entries but they had {} entries!", argv[2], count_got);
}
} catch (const memgraph::communication::bolt::ClientQueryException &e) {
LOG_FATAL(
"The query shoudn't have failed but it failed with an "
"error message '{}', {}",
e.what(), argv[0]);
}
return 0;
}

View File

@@ -0,0 +1,173 @@
#!/usr/bin/python3 -u
# Copyright 2022 Memgraph Ltd.
#
# Use of this software is governed by the Business Source License
# included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
# License, and you may not use this file except in compliance with the Business Source License.
#
# As of the Change Date specified in that file, in accordance with
# the Business Source License, use of this software will be governed
# by the Apache License, Version 2.0, included in the file
# licenses/APL.txt.
import argparse
import os
import subprocess
import sys
import tempfile
import time
from typing import List
SCRIPT_DIR = os.path.dirname(os.path.realpath(__file__))
PROJECT_DIR = os.path.normpath(os.path.join(SCRIPT_DIR, "..", "..", ".."))
def wait_for_server(port: int, delay: float = 0.1) -> float:
cmd = ["nc", "-z", "-w", "1", "127.0.0.1", str(port)]
while subprocess.call(cmd) != 0:
time.sleep(0.01)
time.sleep(delay)
def execute_tester(
binary: str,
queries: List[str],
should_fail: bool = False,
failure_message: str = "",
username: str = "",
password: str = "",
check_failure: bool = True,
) -> None:
args = [binary, "--username", username, "--password", password]
if should_fail:
args.append("--should-fail")
if failure_message:
args.extend(["--failure-message", failure_message])
if check_failure:
args.append("--check-failure")
args.extend(queries)
subprocess.run(args).check_returncode()
def execute_flag_check(binary: str, queries: List[str], expected: int, username: str = "", password: str = "") -> None:
args = [binary, "--username", username, "--password", password]
args.extend(queries)
args.append(str(expected))
subprocess.run(args).check_returncode()
def start_memgraph(memgraph_args: List[any]) -> subprocess:
memgraph = subprocess.Popen(list(map(str, memgraph_args)))
time.sleep(0.1)
assert memgraph.poll() is None, "Memgraph process died prematurely!"
wait_for_server(7687)
return memgraph
def execute_with_user(tester_binary: str, queries: List[str]) -> None:
return execute_tester(
tester_binary, queries, should_fail=False, check_failure=True, username="admin", password="admin"
)
def execute_without_user(
tester_binary: str,
queries: List[str],
should_fail: bool = False,
failure_message: str = "",
check_failure: bool = True,
) -> None:
return execute_tester(tester_binary, queries, should_fail, failure_message, "", "", check_failure)
def cleanup(memgraph: subprocess):
if memgraph.poll() is None:
memgraph.terminate()
assert memgraph.wait() == 0, "Memgraph process didn't exit cleanly!"
def test_without_any_files(tester_binary: str, memgraph_args: List[str]):
memgraph = start_memgraph(memgraph_args)
execute_without_user(tester_binary, ["MATCH (n) RETURN n"], False)
cleanup(memgraph)
def test_init_file(tester_binary: str, memgraph_args: List[str]):
memgraph = start_memgraph(memgraph_args)
execute_with_user(tester_binary, ["MATCH (n) RETURN n"])
execute_without_user(tester_binary, ["MATCH (n) RETURN n"], True, "Handshake with the server failed!", True)
cleanup(memgraph)
def test_init_data_file(flag_checker_binary: str, memgraph_args: List[str]):
memgraph = start_memgraph(memgraph_args)
execute_flag_check(flag_checker_binary, ["MATCH (n) RETURN n"], 2, "user", "user")
cleanup(memgraph)
def test_init_and_init_data_file(flag_checker_binary: str, tester_binary: str, memgraph_args: List[str]):
memgraph = start_memgraph(memgraph_args)
execute_with_user(tester_binary, ["MATCH (n) RETURN n"])
execute_without_user(tester_binary, ["MATCH (n) RETURN n"], True, "Handshake with the server failed!", True)
execute_flag_check(flag_checker_binary, ["MATCH (n) RETURN n"], 2, "user", "user")
cleanup(memgraph)
def execute_test(memgraph_binary: str, tester_binary: str, flag_checker_binary: str) -> None:
storage_directory = tempfile.TemporaryDirectory()
memgraph_args = [memgraph_binary, "--data-directory", storage_directory.name]
# Start the memgraph binary
with open(os.path.join(os.getcwd(), "dummy_init_file.cypherl"), "w") as temp_file:
temp_file.write("CREATE USER admin IDENTIFIED BY 'admin';\n")
temp_file.write("CREATE USER user IDENTIFIED BY 'user';\n")
with open(os.path.join(os.getcwd(), "dummy_init_data_file.cypherl"), "w") as temp_file:
temp_file.write("CREATE (n:RANDOM) RETURN n;\n")
temp_file.write("CREATE (n:RANDOM {name:'1'}) RETURN n;\n")
# Run the test with all combinations of permissions
print("\033[1;36m~~ Starting env variable check test ~~\033[0m")
test_without_any_files(tester_binary, memgraph_args)
memgraph_args_with_init_file = memgraph_args + [
"--init-file",
os.path.join(os.getcwd(), "dummy_init_file.cypherl"),
]
test_init_file(tester_binary, memgraph_args_with_init_file)
memgraph_args_with_init_data_file = memgraph_args + [
"--init-data-file",
os.path.join(os.getcwd(), "dummy_init_data_file.cypherl"),
]
test_init_data_file(flag_checker_binary, memgraph_args_with_init_data_file)
memgraph_args_with_init_file_and_init_data_file = memgraph_args + [
"--init-file",
os.path.join(os.getcwd(), "dummy_init_file.cypherl"),
"--init-data-file",
os.path.join(os.getcwd(), "dummy_init_data_file.cypherl"),
]
test_init_and_init_data_file(flag_checker_binary, tester_binary, memgraph_args_with_init_file_and_init_data_file)
print("\033[1;36m~~ Ended env variable check test ~~\033[0m")
os.remove(os.path.join(os.getcwd(), "dummy_init_data_file.cypherl"))
os.remove(os.path.join(os.getcwd(), "dummy_init_file.cypherl"))
if __name__ == "__main__":
memgraph_binary = os.path.join(PROJECT_DIR, "build", "memgraph")
tester_binary = os.path.join(PROJECT_DIR, "build", "tests", "integration", "flag_check", "tester")
flag_checker_binary = os.path.join(PROJECT_DIR, "build", "tests", "integration", "flag_check", "flag_check")
parser = argparse.ArgumentParser()
parser.add_argument("--memgraph", default=memgraph_binary)
parser.add_argument("--tester", default=tester_binary)
parser.add_argument("--flag_checker", default=flag_checker_binary)
args = parser.parse_args()
execute_test(args.memgraph, args.tester, args.flag_checker)
sys.exit(0)

View File

@@ -0,0 +1,94 @@
// Copyright 2022 Memgraph Ltd.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
// License, and you may not use this file except in compliance with the Business Source License.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#include <gflags/gflags.h>
#include "communication/bolt/client.hpp"
#include "io/network/endpoint.hpp"
#include "io/network/utils.hpp"
DEFINE_string(address, "127.0.0.1", "Server address");
DEFINE_int32(port, 7687, "Server port");
DEFINE_string(username, "", "Username for the database");
DEFINE_string(password, "", "Password for the database");
DEFINE_bool(use_ssl, false, "Set to true to connect with SSL to the server.");
DEFINE_bool(check_failure, false, "Set to true to enable failure checking.");
DEFINE_bool(should_fail, false, "Set to true to expect a failure.");
DEFINE_string(failure_message, "", "Set to the expected failure message.");
int ProcessException(const std::string &exception_message) {
if (FLAGS_should_fail) {
if (!FLAGS_failure_message.empty() && exception_message != FLAGS_failure_message) {
LOG_FATAL(
"The query should have failed with an error message of '{}'' but "
"instead it failed with '{}'",
FLAGS_failure_message, exception_message);
}
return 0;
} else {
LOG_FATAL(
"The query shoudn't have failed but it failed with an "
"error message '{}'",
exception_message);
return 1;
}
}
/**
* Executes queries passed as positional arguments and verifies whether they
* succeeded, failed, failed with a specific error message or executed without a
* specific error occurring.
*/
int main(int argc, char **argv) {
gflags::ParseCommandLineFlags(&argc, &argv, true);
memgraph::communication::SSLInit sslInit;
memgraph::io::network::Endpoint endpoint(memgraph::io::network::ResolveHostname(FLAGS_address), FLAGS_port);
memgraph::communication::ClientContext context(FLAGS_use_ssl);
memgraph::communication::bolt::Client client(context);
try {
client.Connect(endpoint, FLAGS_username, FLAGS_password);
} catch (const memgraph::utils::BasicException &e) {
return ProcessException(e.what());
}
for (int i = 1; i < argc; ++i) {
std::string query(argv[i]);
try {
client.Execute(query, {});
} catch (const memgraph::communication::bolt::ClientQueryException &e) {
if (!FLAGS_check_failure) {
if (!FLAGS_failure_message.empty() && e.what() == FLAGS_failure_message) {
LOG_FATAL(
"The query should have succeeded or failed with an error "
"message that isn't equal to '{}' but it failed with that error "
"message",
FLAGS_failure_message);
}
continue;
}
if (!ProcessException(e.what())) {
return 0;
}
}
if (!FLAGS_check_failure) continue;
if (FLAGS_should_fail) {
LOG_FATAL(
"The query should have failed but instead it executed "
"successfully!");
}
}
return 0;
}

View File

@@ -0,0 +1,6 @@
set(target_name memgraph__integration__license_info)
set(client_target_name ${target_name}__client)
add_executable(${client_target_name} client.cpp)
set_target_properties(${client_target_name} PROPERTIES OUTPUT_NAME client)
target_link_libraries(${client_target_name} mg-requests mg-license mg-utils)

View File

@@ -0,0 +1,62 @@
// Copyright 2022 Memgraph Ltd.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
// License, and you may not use this file except in compliance with the Business Source License.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.
#include <exception>
#include <string>
#include <gflags/gflags.h>
#include "license/license.hpp"
#include "license/license_sender.hpp"
#include "requests/requests.hpp"
#include "spdlog/spdlog.h"
#include "utils/logging.hpp"
#include "utils/synchronized.hpp"
#include "utils/system_info.hpp"
#include "utils/uuid.hpp"
DEFINE_string(endpoint, "http://127.0.0.1:5500/", "Endpoint that should be used for the test.");
DEFINE_string(license_type, "enterprise", "License type; can be oem or enterprise.");
DEFINE_int64(interval, 1, "Interval used for reporting telemetry in seconds.");
DEFINE_int64(duration, 10, "Duration of the test in seconds.");
memgraph::license::LicenseType StringToLicenseType(const std::string_view license_type) {
if (license_type == "enterprise") {
return memgraph::license::LicenseType::ENTERPRISE;
}
if (license_type == "oem") {
return memgraph::license::LicenseType::OEM;
}
spdlog::critical("Invalid license type!");
std::terminate();
}
int main(int argc, char **argv) {
gflags::SetVersionString("license-info");
gflags::ParseCommandLineFlags(&argc, &argv, true);
memgraph::requests::Init();
memgraph::license::License license{"Memgraph", 0, 0, StringToLicenseType(FLAGS_license_type)};
memgraph::utils::Synchronized<std::optional<memgraph::license::LicenseInfo>, memgraph::utils::SpinLock> license_info{
memgraph::license::LicenseInfo{"mg-testkey", "Memgraph"}};
license_info.WithLock([license = std::move(license)](auto &license_info) {
license_info->license = license;
license_info->is_valid = true;
});
memgraph::license::LicenseInfoSender license_sender(FLAGS_endpoint, memgraph::utils::GenerateUUID(),
memgraph::utils::GetMachineId(), 10000000, license_info,
std::chrono::seconds(FLAGS_interval));
std::this_thread::sleep_for(std::chrono::seconds(FLAGS_duration));
return 0;
}

View File

@@ -0,0 +1,96 @@
#!/usr/bin/python3 -u
# Copyright 2021 Memgraph Ltd.
#
# Use of this software is governed by the Business Source License
# included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
# License, and you may not use this file except in compliance with the Business Source License.
#
# As of the Change Date specified in that file, in accordance with
# the Business Source License, use of this software will be governed
# by the Apache License, Version 2.0, included in the file
# licenses/APL.txt.
import argparse
import json
import os
import subprocess
import sys
import time
SCRIPT_DIR = os.path.dirname(os.path.realpath(__file__))
PROJECT_DIR = os.path.normpath(os.path.join(SCRIPT_DIR, "..", "..", ".."))
def execute_test(**kwargs):
client_binary = kwargs.pop("client")
server_binary = kwargs.pop("server")
start_server = kwargs.pop("start_server", True)
interval = kwargs.pop("interval", 1)
duration = kwargs.pop("duration", 5)
license_type = kwargs.pop("license-type", "enterprise")
timeout = duration * 2 if "hang" not in kwargs else duration * 2 + 60
success = False
client_args = [client_binary, "--interval", interval, "--duration", duration, "--license-type", license_type]
server = None
if start_server:
server = subprocess.Popen(server_binary)
time.sleep(0.4)
assert server.poll() is None, "Server process died prematurely!"
try:
subprocess.run(list(map(str, client_args)), timeout=timeout, check=True)
finally:
if server is None:
success = True
else:
server.terminate()
try:
success = server.wait(timeout=5) == 0
success = True
except subprocess.TimeoutExpired:
server.kill()
return success
def main():
server_binary = os.path.join(SCRIPT_DIR, "server.py")
client_binary = os.path.join(PROJECT_DIR, "build", "tests", "integration", "license_info", "client")
parser = argparse.ArgumentParser()
parser.add_argument("--client", default=client_binary)
parser.add_argument("--server", default=server_binary)
parser.add_argument("--server-url", default="127.0.0.1")
parser.add_argument("--server-port", default="5500")
args = parser.parse_args()
tests = [
{"interval": 2},
{"duration": 10},
{"interval": 2, "duration": 10},
{"license-type": "oem"},
{"license-type": "enterprise"},
]
for test in tests:
print("\033[1;36m~~ Executing test with arguments:", json.dumps(test, sort_keys=True), "~~\033[0m")
try:
success = execute_test(client=args.client, server=args.server, **test)
except Exception as e:
print("\033[1;33m", e, "\033[0m", sep="")
success = False
if not success:
print("\033[1;31m~~", "Test failed!", "~~\033[0m")
sys.exit(1)
else:
print("\033[1;32m~~", "Test ok!", "~~\033[0m")
sys.exit(0)
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,69 @@
#!/usr/bin/python3 -u
# Copyright 2021 Memgraph Ltd.
#
# Use of this software is governed by the Business Source License
# included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
# License, and you may not use this file except in compliance with the Business Source License.
#
# As of the Change Date specified in that file, in accordance with
# the Business Source License, use of this software will be governed
# by the Apache License, Version 2.0, included in the file
# licenses/APL.txt.
import argparse
import json
from http.server import HTTPServer, SimpleHTTPRequestHandler
EXPECTED_LICENSE_INFO_FIELDS = {
"run_id": str,
"machine_id": str,
"type": str,
"license_type": str,
"license_key": str,
"organization": str,
"valid": bool,
"physical_memory_size": int,
"swap_memory_size": int,
"memory_used": int,
"runtime_memory_limit": int,
"license_memory_limit": int,
"timestamp": float,
}
class ServerHandler(SimpleHTTPRequestHandler):
def do_POST(self):
assert self.headers["user-agent"] == "memgraph/license-info", f"The header is {self.headers['user-agent']}"
assert self.headers["accept"] == "application/json", f"The header is {self.headers['accept']}"
assert self.headers["content-type"] == "application/json", f"The header is {self.headers['content-type']}"
content_len = int(self.headers.get("content-length", 0))
data = json.loads(self.rfile.read(content_len).decode("utf-8"))
assert isinstance(data, dict)
for expected_field, expected_type in EXPECTED_LICENSE_INFO_FIELDS.items():
assert expected_field in data, f"Field {expected_field} not found in received data"
assert isinstance(
data[expected_field], expected_type
), f"Field {expected_field} is not correct type: expected {expected_type} got {type(data[expected_field])}"
assert len(EXPECTED_LICENSE_INFO_FIELDS) == len(data), "Expected data size does not match received"
self.send_response(200)
self.end_headers()
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--address", type=str, default="127.0.0.1")
parser.add_argument("--port", type=int, default=5500)
args = parser.parse_args()
with HTTPServer((args.address, args.port), ServerHandler) as srv:
print(f"Serving HTTP server at {args.address}:{args.port}")
srv.serve_forever()
if __name__ == "__main__":
main()

View File

@@ -3,4 +3,4 @@ set(client_target_name ${target_name}__client)
add_executable(${client_target_name} client.cpp)
set_target_properties(${client_target_name} PROPERTIES OUTPUT_NAME client)
target_link_libraries(${client_target_name} mg-requests telemetry_lib)
target_link_libraries(${client_target_name} mg-requests mg-telemetry)

View File

@@ -13,6 +13,8 @@
#include "requests/requests.hpp"
#include "telemetry/telemetry.hpp"
#include "utils/system_info.hpp"
#include "utils/uuid.hpp"
DEFINE_string(endpoint, "http://127.0.0.1:9000/", "Endpoint that should be used for the test.");
DEFINE_int64(interval, 1, "Interval used for reporting telemetry in seconds.");
@@ -24,8 +26,8 @@ int main(int argc, char **argv) {
gflags::ParseCommandLineFlags(&argc, &argv, true);
memgraph::requests::Init();
memgraph::telemetry::Telemetry telemetry(FLAGS_endpoint, FLAGS_storage_directory,
std::chrono::seconds(FLAGS_interval), 1);
memgraph::telemetry::Telemetry telemetry(FLAGS_endpoint, FLAGS_storage_directory, memgraph::utils::GenerateUUID(),
memgraph::utils::GetMachineId(), std::chrono::seconds(FLAGS_interval), 1);
uint64_t counter = 0;
telemetry.AddCollector("db", [&counter]() -> nlohmann::json {

View File

@@ -36,8 +36,7 @@ def execute_test(**kwargs):
timeout = duration * 2 if "hang" not in kwargs else duration * 2 + 60
success = False
server_args = [server_binary, "--interval", interval,
"--duration", duration]
server_args = [server_binary, "--interval", interval, "--duration", duration]
for flag, value in kwargs.items():
flag = "--" + flag.replace("_", "-")
# We handle boolean flags here. The type of value must be `bool`, and
@@ -48,9 +47,15 @@ def execute_test(**kwargs):
else:
server_args.extend([flag, value])
client_args = [client_binary, "--interval", interval,
"--duration", duration,
"--storage-directory", storage_directory]
client_args = [
client_binary,
"--interval",
interval,
"--duration",
duration,
"--storage-directory",
storage_directory,
]
if endpoint:
client_args.extend(["--endpoint", endpoint])
@@ -61,8 +66,7 @@ def execute_test(**kwargs):
assert server.poll() is None, "Server process died prematurely!"
try:
subprocess.run(list(map(str, client_args)), timeout=timeout,
check=True)
subprocess.run(list(map(str, client_args)), timeout=timeout, check=True)
finally:
if server is None:
success = True
@@ -88,16 +92,14 @@ TESTS = [
{"endpoint": "http://127.0.0.1:9000/nonexistant/", "no_check": True},
{"start_server": False},
{"startups": 4, "no_check_duration": True}, # the last 3 tests failed
# to send any data + this test
{"add_garbage": True}
# to send any data + this test
{"add_garbage": True},
]
if __name__ == "__main__":
server_binary = os.path.join(SCRIPT_DIR, "server.py")
client_binary = os.path.join(PROJECT_DIR, "build", "tests",
"integration", "telemetry", "client")
kvstore_console_binary = os.path.join(PROJECT_DIR, "build", "tests",
"manual", "kvstore_console")
client_binary = os.path.join(PROJECT_DIR, "build", "tests", "integration", "telemetry", "client")
kvstore_console_binary = os.path.join(PROJECT_DIR, "build", "tests", "manual", "kvstore_console")
parser = argparse.ArgumentParser()
parser.add_argument("--client", default=client_binary)
@@ -108,19 +110,17 @@ if __name__ == "__main__":
storage = tempfile.TemporaryDirectory()
for test in TESTS:
print("\033[1;36m~~ Executing test with arguments:",
json.dumps(test, sort_keys=True), "~~\033[0m")
print("\033[1;36m~~ Executing test with arguments:", json.dumps(test, sort_keys=True), "~~\033[0m")
if test.pop("add_garbage", False):
proc = subprocess.Popen([args.kvstore_console, "--path",
storage.name], stdin=subprocess.PIPE,
stdout=subprocess.DEVNULL)
proc = subprocess.Popen(
[args.kvstore_console, "--path", storage.name], stdin=subprocess.PIPE, stdout=subprocess.DEVNULL
)
proc.communicate("put garbage garbage".encode("utf-8"))
assert proc.wait() == 0
try:
success = execute_test(client=args.client, server=args.server,
storage=storage.name, **test)
success = execute_test(client=args.client, server=args.server, storage=storage.name, **test)
except Exception as e:
print("\033[1;33m", e, "\033[0m", sep="")
success = False

Some files were not shown because too many files have changed in this diff Show More