Compare commits

...

322 Commits

Author SHA1 Message Date
Tyler Neely
662fd6cedc Remove legacy functionality 2023-03-06 08:41:26 +00:00
Jure Bajic
362dc95e27 Add support for Ubuntu 22.04 ARM (#810) 2023-03-01 18:44:56 +01:00
Kruno Golubic
eead24a562 Add Lima badge to README (#802) 2023-02-24 17:08:07 +01:00
Antonio Filipovic
d79dd69607 Improve performance with props init on node|edge creation (#788) 2023-02-24 15:40:35 +01:00
Katarina Supe
024bf0c578 Merge pull request #806 from memgraph/vpavicic-patch-1
Update README.md
2023-02-23 15:06:40 +01:00
Vlasta
d7443a5558 Update README.md 2023-02-23 15:05:03 +01:00
Kruno Golubic
2df357b012 Add RedHat badge to README (#796) 2023-02-20 18:18:29 +01:00
Kruno Golubic
b2b5a6e2a0 Add Fedora badge to README (#795) 2023-02-20 17:59:57 +01:00
Ante Javor
5e2ee6c817 Improve mgbench C++ client (#760) 2023-02-17 17:54:05 +01:00
Antonio Filipovic
862a1afdf1 Improve Visit performance (#774) 2023-02-17 13:09:25 +01:00
Antonio Filipovic
bbce21e78f Update pull request template (#775) 2023-02-17 11:50:17 +01:00
Jure Bajic
15c8662023 Add support for Fedora 36 (#787) 2023-02-17 10:47:36 +01:00
Katarina Supe
beaba0fc16 Update README with Cloud, Lab and import info (#768) 2023-02-07 15:43:41 +01:00
Josipmrden
8f70c5f2a5 Fix label-based auth using OLD view instead of NEW when merging nodes (#755) 2023-02-01 13:20:26 +01:00
Bruno Sačarić
14c651d3ba edit workflows (#756) 2023-01-31 23:20:49 +01:00
Andi
04efc7a4a6 Remove torch and igraph from sys cache (#720) 2023-01-31 13:11:59 +01:00
Bruno Sačarić
19832b5838 Merge pull request #749 from memgraph/update-license-year-2.5.2
[master < MG] Update license year
2023-01-26 13:08:47 +01:00
Bruno Sačarić
e63fae2d5b Update license year 2023-01-26 13:02:33 +01:00
Bruno Sačarić
34dd47ef07 Fix nested FOREACH shadowing bug (#725) 2023-01-25 20:06:05 +01:00
Jure Bajic
d3f275b231 Exclude license dir from CI trigger (#740) 2023-01-25 18:49:02 +01:00
Ante Pušić
aad4bcb7a0 Fix C++ API memory leak on Relationships() (#743) 2023-01-25 17:23:46 +01:00
Bruno Sačarić
034b54cb72 Fix bug on all shortest paths with an upper bound (#737) 2023-01-25 15:32:00 +01:00
Josipmrden
8cf51d9f68 Fix bug in query plan to use indexes on optional match and foreach (#736)
* Add fix in query plan to use indexes on optional match and foreach
2023-01-25 12:53:33 +01:00
Antonio Filipovic
1cd1da84fd Fix bug on (vertex|edge) properties in C++ API (#732) 2023-01-23 12:57:17 +01:00
Jure Bajic
128a6cd522 Update license year (#739) 2023-01-19 12:31:59 +01:00
niko4299
d9eeedb9ee Adding qid in bolt (#721) 2023-01-18 16:33:03 +01:00
Andi
156e2cd095 On delete triggers invalid edge reference (#717)
* Added check if there is invalid reference to the underlying edge

* Added fix and e2e tests

* Isolation levels tracking based on from_vertex_

* Added explicit transaction test + edge accessor changes based on the vertex_edge

* Autocommit on tests, initialize deleted by checking out_edges

Co-authored-by: Marko Budiselić <marko.budiselic@memgraph.com>
2023-01-18 15:05:10 +01:00
Ante Javor
8b834c702c Update mgbench to run Diff workflow under 30mins (#730) 2023-01-14 16:11:49 +01:00
Katarina Supe
eda5213d95 Release pypi mgp 1.1.1 (#727) 2022-12-24 09:33:53 +01:00
Bruno Sačarić
1f2a15e7c8 Fix MATCH not allowed on replica (#709) 2022-12-23 14:47:12 +01:00
Ante Javor
d72e7fa38d Fix mgp.py create edge type hint and comment (#724) 2022-12-23 10:08:52 +01:00
Antonio Filipovic
e5e37bc14a Fix LOAD CSV large memory usage (#712) 2022-12-22 19:38:48 +01:00
Katarina Supe
3ee068bbf9 Update build badge (#722) 2022-12-20 21:56:05 +01:00
Jure Bajic
68e846b182 Update license year (#711) 2022-12-13 13:54:09 +01:00
Andi
310e305cfb Fix python module reloading (#706) 2022-12-12 21:11:13 +01:00
Marko Budiselić
9d6a23b6bd Add init-file and init-data-file capabilities (#696) 2022-12-09 18:50:33 +01:00
Andi
f2d5ab61c4 Fix Python submodules reloading (#653) 2022-12-09 14:30:41 +01:00
Andi
0f77c85824 Fix cursor exhaustion by adding EmptyResult operator (#667) 2022-12-09 11:44:07 +01:00
niko4299
d6d4153fb7 Fix graph projection bug (#697) 2022-12-08 13:45:20 +01:00
Tyler Neely
7d6a5e5b9c Add support for -h to show help in addition to --help (#682) 2022-12-07 16:51:32 +01:00
Vlasta
c529d52664 Add community links to the README (#693) 2022-12-07 12:42:48 +01:00
Ante Pušić
45451bae3b Fix C++ query modules API bugs (#688) 2022-12-06 16:57:50 +01:00
niko4299
3e11f38548 Add aggregation distinct (#654) (#665) 2022-12-03 13:48:44 +02:00
Jure Bajic
6e4047a847 Bump mgconsole version (#660) 2022-12-01 13:10:08 +01:00
Ante Javor
8febdc12fb Update tests/mgbench README (#679) 2022-11-30 12:43:57 +01:00
Kruno Golubic
3f23a10f44 Update README with one new paragraph and link (#675) 2022-11-29 14:24:02 +01:00
Ante Javor
11300960de Add mixed workload and Neo4j client to mgbench (#566)
* Fix bolt bug inside the C++ client
* Add tail latency stats
* Add hot run option
* Add query caching
* Add jcmd memory tracking
2022-11-28 08:47:22 +01:00
Jure Bajic
1d5f387ddd Add python check (#643) 2022-11-09 11:48:34 +02:00
Marko Budiselić
c4c3a254bf Remove tools/check-build-system (#642) 2022-11-07 18:54:22 +01:00
Jure Bajic
b4beb0fc86 Update license for release 2.4.2 (#641) 2022-11-07 13:07:20 +01:00
Bruno Sačarić
58e6097664 Fix ALLSHORTEST combined with id function (#636) 2022-11-04 19:36:03 +01:00
Jure Bajic
ff21c0705c Add multiple license support (#618)
Make license info available through LicenseChecker
Add LicenseInfoSender
Move license library from utils
Rename telemetry_lib to mg-telemetry
2022-11-04 15:23:43 +01:00
Katarina Supe
2a2b99b02a Release mgp 1.1.0 (#620)
- Updated its README, authors, and published it with the added
   AuthorizationError that was a part of Memgraph 2.4.0 release.
 - Added missing SOURCE_TYPE_KAFKA and SOURCE_TYPE_PULSAR variables in _mgp.
2022-11-02 14:14:12 +01:00
Antonio Filipovic
3daab6ce97 Fix bug in the C API in-edges iterator (#613) 2022-10-25 19:18:44 +02:00
Marko Budiselić
fbd7274c95 Add Fedora 36 as an OS (#599) 2022-10-21 15:22:50 +02:00
Marko Budiselić
6efc84f022 Add libc++ option to the toolchain (#567) 2022-10-20 07:52:59 +02:00
Jure Bajic
287c2e94d1 Update license date (#586) 2022-10-07 14:42:52 +02:00
Antonio Filipovic
417cf4b30b Fix bug related to EdgeType and Label getters in query modules (#582)
Co-authored-by: Kostas Kyrimis <kostaskyrim@gmail.com>
2022-10-06 21:21:11 +02:00
Jure Bajic
68e7fd3d36 Fix architecture check (#583) 2022-10-06 15:55:23 +02:00
Bruno Sačarić
5261d82063 Fix passing user's fine_grained_access_handler instead of role's (#579)
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
2022-09-30 18:27:47 +02:00
Jure Bajic
9eb87bcf3e Update release process (#564)
* Fix centos 7 virtualenv issue
* Fix ubuntu release issue
* Fix architecture check
2022-09-20 18:42:15 +02:00
Jure Bajic
a9491d3e68 Update license date (#561) 2022-09-20 14:23:24 +02:00
Marko Budiselić
b42e47b0be Reduce the size of TypedValue (#560)
- Reduce the size of TypedValue
- Fix double allocation
- Add `Graph` to `TypedValue` unit tests
- Fix allocator usage in `TypedValue`
- Add graph projection to `long_running.cpp` stress test
2022-09-20 14:21:34 +02:00
Marko Budiselić
898c894a48 Merge pull request #484 from memgraph/E129-MG-label-based-authorization
* Add label-based authorization

Co-authored-by: Boris Taševski <boris.tasevski@memgraph.com>
Co-authored-by: Josip Mrden <josip.mrden@memgraph.com>
Co-authored-by: Niko Krvavica <niko.krvavica@memgraph.com>
Co-authored-by: Bruno Sacaric <bruno.sacaric@memgraph.com>
2022-09-16 15:12:54 +02:00
niko4299
a3c2492672 Add fine grained access control to mgbench (#522) 2022-09-15 21:33:15 +02:00
Boris Taševski
a0b8871b36 Fix cland tidy errors and other warning (#555) 2022-09-15 15:51:35 +02:00
Marko Budiselic
bb6cf35441 Merge master cpp module API 2022-09-15 11:29:52 +02:00
Ante Pušić
5bc301d21d Add C++ query modules API (#546)
Co-authored-by: Ante Pusic <ante.pusic@memgraph.com>
Co-authored-by: Josip Mrden <josip.mrden@memgraph.com>
2022-09-15 11:26:26 +02:00
Marko Budiselic
1b89e679df Merge master 2022-09-15 08:32:41 +02:00
Boris Taševski
43e0520bc8 Merge master (#554) 2022-09-15 07:25:36 +02:00
Bruno Sačarić
2c8e45e889 Add run_id to the query summary (#548) 2022-09-14 20:21:06 +02:00
Boris Taševski
0876a8848d Merge master to epic and fix differences (#552) 2022-09-14 18:36:21 +02:00
Boris Taševski
fb4641a6be Fix logic in fine grained permissions (#551) 2022-09-14 12:39:23 +02:00
niko4299
201f75e809 Add MG_ENTERPRISE and license checks (#547) 2022-09-14 01:10:28 +02:00
niko4299
dc8dad9794 Add authorization in SetLabels, RemoveLabels, Allshortestpath cursor (#537) 2022-09-13 17:14:23 +02:00
Boris Taševski
aa02745915 [E129-MG < T1030-MG] Tech debts (#540)
* renamed parameters (#539)

* added variable declarations in ifs; minor code improvements; (#541)

* dba parameter removed (#543)

* Accept -> Has rename; HasGlobalPermissionOnVertices/Edges -> HasGlobalPrivilegeOnVertices/Edges (#545)

* replaced passing dba from reference to pointer
2022-09-13 11:37:17 +02:00
Josipmrden
b2d5a8eeca [E129-MG < T1040-MG] Add exceptions in LBA cursors (#536)
Exceptions added in update and create delete operators instead of logging
2022-09-12 14:04:40 +02:00
Boris Taševski
c09b175c76 [E129-MG < T1006-MG] Expand C API with LBA checks (#527)
* [T1006-MG < T1017-MG] Add LBA checks to all read procedures in C API (#515)

* Initial Impl

* NextPermittedEdge introduced

* revert moving constructor to cpp

* edge from and edge to methods expanded with lba check

* minor fix

* added check to path expand procedure

* Added integration tests for read query procedures

* additional check

* changed iterator type to reference

* comments from pr

Co-authored-by: Josip Mrden <josip.mrden@memgraph.io>

* [T1006-MG < T1018-MG] Add LBA checks to all update procedures in C API (#516)

* Initial Impl

* NextPermittedEdge introduced

* revert moving constructor to cpp

* edge from and edge to methods expanded with lba check

* minor fix

* extended update methods

* added check to path expand procedure

* Added integration tests for read query procedures

* Added integration tests for update query modules

* additional check

* changed iterator type to reference

* fixed bug in Update property for node; fixed 2 e2e tests

* replaced enum

Co-authored-by: Josip Mrden <josip.mrden@memgraph.io>

* [T1006-MG < T1019-MG] Add LBA checks to all Create and Delete procedures in C API (#517)

* Initial Impl

* NextPermittedEdge introduced

* revert moving constructor to cpp

* edge from and edge to methods expanded with lba check

* minor fix

* extended update methods

* initial implementation

* added check to path expand procedure

* Added integration tests for read query procedures

* Added integration tests for update query modules

* Added unit tests for creation of vertex, adding and removing vertex label

* additional check

* changed iterator type to reference

* Added unit tests for create edge

* Corrected query module in create edge

* fixed bug in Update property for node; fixed 2 e2e tests

* fixed merge errors

* Expanded FineGrainedAuthChecker with HasGlobalPermissionOnVertices and HasGlobalPermissionOnEdges

* Removed two wrong checks; Added two global checks

* return null added

* introduced new mgp_error value

* fixed endless loop

* replaced enum

* intermediate

* tests updated

* PermissionDeniedError -> AuthorizationError rename

* rename in enum permission_denied error -> authorization error

* mgp_vertex_remove_label check improved

* quotes changed; order of imports fixed

* string constant introduced

* import fixed

* yaml format

Co-authored-by: Josip Mrden <josip.mrden@memgraph.io>

Co-authored-by: Josip Mrden <josip.mrden@memgraph.io>
2022-09-08 17:48:34 +02:00
Kostas Kyrimis
f1fe77adfb Graph project feature implementation (#508) (#535) 2022-09-07 16:00:49 +03:00
Josip Mrden
35f8978560 Merge branch 'master' into E129-MG-label-based-authorization 2022-09-07 09:28:32 +02:00
Josip Matak
9e8fb2516b Add all shortest path algorithm (#409) 2022-09-06 16:21:32 +02:00
Josip Mrden
0a66feccff Merge branch 'master' into E129-MG-label-based-authorization 2022-09-06 11:14:27 +02:00
Boris Taševski
d008a2ad8d [E129-MG < T1007-MG] Expand Cursors with LBA checks (#524)
* [T1007-MG < T0997-MG] Authorization on paths (#501)

* Added read authorization in paths operators

* [T1007-MG < T1016-MG] Added authorization in create and delete operators (#513)

* Added authorization in RemoveNodeCursor, RemoveExpandCursor, CreateNodeCursor, CreateExpandCursor,MergeCursor

* [T1007-MG < T1014-MG] Add authorization to read operators (#520)

Added label based access control to read operators (ScanAll).

* [T1007-MG < T1015-MG] Add authorization to update operators (SetProperty, SetProperties, RemoveProperty) (#521)

Added label based authorization to update operators

Co-authored-by: niko4299 <51059248+niko4299@users.noreply.github.com>
Co-authored-by: Josip Mrden <josip.mrden@memgraph.io>
2022-09-02 17:12:07 +02:00
Josipmrden
7478300762 [E129-MG < T997-MG] Show label privileges (#506)
Added showing of label privileges functionality to fine grained access control.
2022-08-31 12:14:16 +02:00
János Benjamin Antal
0bc298c3ad Fix handling of the ROUTE Bolt message (#475)
The fields of ROUTE message were not read from the input buffer, thus the
input buffer got corrupted. Sending a new message to the server would result
reading the remaining fields from the buffer, which means reading some values
instead of message signature. Because of this unmet expectation, Memgraph closed
the connection. With this fix, the fields of the ROUTE message are properly
read and ignored.
2022-08-26 13:19:27 +02:00
Boris Taševski
05f120b7d4 [E129-MG < T1004-MG] Expand cypher with more granular label permissions (#500)
* Added enum for more granular access control; Expanded functionality of fine grained access checker; Propagated changes to Edit, Deny and Revoke permissions methods in interpreter

* Introduced Merge method for merging two colle with permissions

* e2e tests implementation started

* Expanded cypher to support fine grained permissions

* ast.lcp::AuthQuery removed labels, added support for label permissions

* promoted label permissions to vector

* removed unnecesary enum value

* expanded glue/auth with LabelPrivilegeToLabelPermission

* added const

* extended Grant Deny and Revoke Privileges with new label privileges

* extended Edit Grant Deny and Revoke Privileges to properly use new model

* Fixed unit tests

* FineGrainedAccessChecker Grant and Deny methods reworked

* Revoke cypher slightly reworked; Revoke for labels works without label permissions

* EditPermission's label_permission lambda now takes two parameters

* constants naming enforced; replaced asterisks with string constant

* removed faulty test addition

* Naming fixes; FineGrainedAccessChecker unit tests introduced

* unnecessary includes removed; minor code improvements

* minor fix

* Access checker reworked; denies and grant merged into single permission object; Created global_permission that applies to all non-created permissions. Grant, Deny and Revoke reworked; Merge method reworked

* Fixed wrong check;

* Fix after merge; renamed constants; removed unused constant

* Fix after merge; workloads.yaml for lbaprocedures e2e tests updated with new grammar

* Fixes after merge

* Fixes after merge

* fixed Revoke that was not fixed after the merge

* updated cypher main visitor tests

* PR review changes; Naming and const fixed, replaced double tertiary with lambda

* unwrapping the iterator fix

* merge 1003 minor fix

* minor spelling fixes

* Introduced visitPrivilegesList because of the doubled code

* const added

* string const to enum

* redundant braces

* added const

* minor code improvement

* e2e tests expanded

* if -> switch

* enum class inherits uint8_t now

* LabelPrililege::EDIT -> LabelPrivilege::UPDATE

* LabelPermission -> EntityPermission; LabelPrivilege -> EntityPrivilege

* EntityPrivilege -> FineGrainedPrivilege; EntityPermission -> FineGrainedPermission
2022-08-22 14:11:43 +02:00
antoniofilipovic
d73d153978 Add logging API (#417) 2022-08-22 14:47:52 +03:00
Boris Taševski
b489ac7cff [E129-MG < T1003-MG] Expand fine grained access checker with more granular permissions (#496)
* Added enum for more granular access control; Expanded functionality of fine grained access checker; Propagated changes to Edit, Deny and Revoke permissions methods in interpreter

* Introduced Merge method for merging two colle with permissions

* e2e tests implementation started

* FineGrainedAccessChecker Grant and Deny methods reworked

* removed faulty test addition

* Naming fixes; FineGrainedAccessChecker unit tests introduced

* unnecessary includes removed; minor code improvements

* Access checker reworked; denies and grant merged into single permission object; Created global_permission that applies to all non-created permissions. Grant, Deny and Revoke reworked; Merge method reworked

* Fixed wrong check;

* PR review changes; Naming and const fixed, replaced double tertiary with lambda

* unwrapping the iterator fix

* minor spelling fixes
2022-08-18 16:59:38 +02:00
niko4299
e15576f56c [E129-MG <-T0982-MG] implement edge type filtering (#489)
* GRANT, REVOKE, DENY and access_checker DONE

* Added AccessChecker to ExecutionContext

* grammar expanded; (#462)

* current

* T0954 mg expand user and role to hold permissions on labels (#465)

* added FineGrainedAccessPermissions class to model

* expanded user and role with fine grained access permissions

* fixed grammar

* [E129 < T0953-MG] GRANT, DENY, REVOKE added in interpreter and mainVisitor (#464)

* GRANT, DENY, REVOKE added in interpreter and mainVisitor

* Commented labelPermissons

* remove labelsPermission adding

* Fixed

* Removed extra lambda

* fixed

* [E129<-T0955-MG] Expand ExecutionContext with label related information (#467)

* added

* Added FineGrainedAccessChecker to Context

* fixed

* Added filtering

* testing

* Added edge filtering to storage, need to add filtering in simple Expand in operator.cpp

* Removed storage changes

* MATCH filtering working

* EdgeTypeFiltering working, just need to test everything again

* Removed FineGrainedAccessChecker

* Removed Expand Path

* Fix

* Tested FineGrainedAccessHandler, need to test AuthChecker

* Added integration test for lba

* Fixed merge conflicts

* PR fix

* fixed

* PR fix

* Fix test

* removed .vscode, .cache, .githooks

* githooks

* added tests

* fixed build

* Changed ast.lcp and User pointer to value in context.hpp

* Fixed test

* Remove denies on grant all

* AuthChecker

* Pr fix, auth_checker still not fixed

* Create mg-glue and extract UserBasedAuthChecker from AuthChecker

* Build fixed, need to fix test

* e2e tests

* e2e test working

* Added unit test, e2e and FineGrainedChecker

* Mege E129, auth_checker tests

* Fixed test

* e2e fix

Co-authored-by: Boris Taševski <36607228+BorisTasevski@users.noreply.github.com>
Co-authored-by: josipmrden <josip.mrden@external-basf.com>
Co-authored-by: János Benjamin Antal <benjamin.antal@memgraph.io>
2022-08-16 15:57:23 +02:00
Boris Taševski
a98463b0bd [E129 < T0996] C-API: Implement using Fine Grained Access Checker in iterator over vertices (#494)
* implemented skipping vertices in Constructor and mgp_vertices_iterator_next

* Added utility function for moving iterator to next permitted vertex

* removed ifdef directive

* NextPermitted parameter type changed from mgp_vertices_iterator* to mgp_vertices_iterator&

* created support for lba-procedures e2e testing; Added test for vertex iterator skipping unauthorized vertices

* removed fixture from tests; converted generator to regular function;
2022-08-12 19:34:47 +02:00
Kruno Golubic
705631a35d Create README file for CSV Import Tools (#493)
Co-authored-by: Marko Budiselić <marko.budiselic@memgraph.com>
2022-08-11 16:10:36 +02:00
Jeremy B
d4f0bb0e38 Correct inconsistencies w.r.t. sync replication (#435)
Add a report for the case where a sync replica does not confirm within a timeout:
-Add a new exception: ReplicationException to be returned when one sync replica does not confirm the reception of messages (new data, new constraint/index, or for triggers)
-Update the logic to throw the ReplicationException when needed for insertion of new data, triggers, or creation of new constraint/index
-Add end-to-end tests to cover the loss of connection with sync/async replicas when adding new data, adding new constraint/indexes, and triggers

Add end-to-end tests to cover the creation and drop of indexes, existence constraints, and uniqueness constraints

Improved tooling function mg_sleep_and_assert to also show the last result when duration is exceeded
2022-08-09 11:29:55 +02:00
Jure Bajic
531db2d47c Fix WebSocket test (#485)
* Fix websocket unit tests hanging
* Rename websocket to monitoring unit test
2022-08-08 14:49:48 +02:00
Boris Taševski
116262d9a0 [E129 < T0956] Filtering nodes in ScanAll cursor [Niko] (#492)
* implemented scanall filtering

* minor code refactor

* FindNextNode -> FindNextVertex
2022-08-04 19:20:17 +02:00
gvolfing
bbfef45b37 Add command to return startup config (#459)
Add a new command that is able to return the set of configurations that that the
given instance of memgraph was started up with. The returned information
currently consists of the name, the default and the current value of each flag.
The hidden property of three flags were removed, namely --query-cost-planner,
--query-vertex-count-to-expand-existing and --query-max-plans. The flag
--log-link-basename was completely removed since it is not used.
2022-08-03 18:08:44 +02:00
János Benjamin Antal
05b00edfd4 Declare mgp_func_context outside the callback function (#481) 2022-08-03 15:15:53 +02:00
Boris Taševski
480df4ed69 Merge old Label Based Auth Epic branch into new one because of commits with bad checks on the old epic branch (#478)
* grammar expanded; (#462)

* T0954 mg expand user and role to hold permissions on labels (#465)

* added FineGrainedAccessPermissions class to model

* expanded user and role with fine grained access permissions

* fixed grammar

* [E129 < T0953-MG] GRANT, DENY, REVOKE added in interpreter and mainVisitor (#464)

* GRANT, DENY, REVOKE added in interpreter and mainVisitor

* Commented labelPermissons

* remove labelsPermission adding

* Removed extra lambda

* [E129<-T0955-MG] Expand ExecutionContext with label related information (#467)

* Added FineGrainedAccessChecker to Context

* fixed failing tests for label based authorization (#480)

* Marked FineGrainedAccessChecker ctor explicit; Introduced change to clang-tidy; (#483)

Co-authored-by: niko4299 <51059248+niko4299@users.noreply.github.com>
2022-08-02 12:51:22 +02:00
Marko Budiselić
80e0e439b7 Add parser stress test (#463) 2022-07-26 20:54:56 +02:00
Jure Bajic
351258ace8 Update clang-tidy job (#476)
Update clang tidy job to only check the diff against the base branch
2022-07-26 16:53:56 +02:00
Marko Budiselić
74d3663821 Upgrade Antrl to 4.10.1 and remove antlr_lock (#441) 2022-07-26 08:31:38 +02:00
gvolfing
eb0b3141d5 Fix aggregation functions on null and group-by inputs (#448)
The `sum()` and `count()` functions were giving results different from the openCypher specification on null `input.` The aggregation functions also had a problem when they were used in a group-by context and were giving results that were not compliant with the openCypher specification.
2022-07-19 12:16:32 +02:00
Marko Budiselić
ff2f8031a9 Extend the PR template with the git message guide (#451) 2022-07-15 14:08:10 +02:00
Marko Budiselić
094d4f282d Improve error-prone OS setup (#428) 2022-07-13 17:48:01 +02:00
Jure Bajic
3dd2657320 Create mgp python package (#433) 2022-07-12 10:54:23 +02:00
gvolfing
6fe474282a Modify logaical operators to conform openCyper regarding checking against NULL in CASE expressions (#432)
* Make `IfOperator` return the `else_expression_` in case of `NULL`

* Add gql_behave tests

* Add gql_behave test to specifically check for the case when the test expression itself is null
2022-07-11 15:00:29 +02:00
gvolfing
7fc0fb6520 Implement ToString function for temporal datatypes (#429)
* Modify `toString` to be able to handle `Date`, `LocalTime`, `LocalDateTime` and `Duration`

* Add unit tests

* Make `operator<<` use the `ToString()` implementations

* Add tests to verify the correctness of negative durations

* Add more tests to look for cases when the individual duration entities overflow.
2022-07-11 13:44:27 +02:00
Jeremy B
063e297e1e Avoid usage of time.sleep (#434)
e2e python: added tooling function around `time.sleep()` that stops as soon as condition is fulfilled and will raise assert if timeout is reached
2022-07-08 10:47:18 +02:00
Ante Javor
86b1688192 Rewrite Python API comments and snippets (#420)
* Update comments
2022-07-07 15:05:56 +02:00
Jeremy B
f629de7e60 Save replication settings (#415)
* Storage takes care of the saving of setting when a new replica is added

* Restore replicas at startup

* Modify interactive_mg_runner + memgraph to support that data-directory can be configured in CONTEXT

* Extend e2e test

* Correct typo

* Add flag to config to specify when replication should be stored (true by default when starting Memgraph)

* Remove un-necessary "--" in yaml file

* Make sure Memgraph stops if a replica can't be restored.

* Add UT covering the parsing  of ReplicaStatus to/from json

* Add assert in e2e script to check that a port is free before using it

* Add test covering crash on Jepsen

* Make sure applciaiton crashes if it starts on corrupted replications' info

Starting with a non-reponsive replica is allowed.

* Add temporary startup flag: this is needed so jepsen do not automatically restore replica on startup of main. This will be removed in T0835
2022-07-07 13:30:28 +02:00
Jeremy B
b737e53456 Remove sync with timeout (#423)
* Remove timout when registering a sync replica

* Simplify jepsen configuration file

* Remove timeout from jepsen configuration

* Add unit test

* Remove TimeoutDispatcher
2022-07-05 09:40:50 +02:00
János Benjamin Antal
10ca68bb2a Remove CODEOWNERS (#427)
Co-authored-by: Marko Budiselić <marko.budiselic@memgraph.com>
2022-07-04 16:42:36 +02:00
Jure Bajic
bfbd8538d4 Update docker release process (#421)
* Fix release directory

* Update release process

* Fix debian arm path
2022-07-04 16:10:33 +02:00
Jeremy B
3e0e17d469 Adding assert in e2e script to check that a port is free before using it (#425)
* Adding assert in e2e script to check that a port is free before using it

* Adding extra logging
2022-07-04 10:14:02 +02:00
Jeremy B
b57f91fcfc Correct concurrency group (#426)
* Correct concurrency group
2022-07-01 16:53:41 +02:00
Jeremy B
066a96c0ae Adding concurrency to cancel current workflow if one commits on the same branch. (#424)
* Adding concurrency to cancel current workflow if one commits on the same branch.
2022-07-01 15:47:50 +02:00
Jeremy B
1ae6b71c5f Registering a replica with timeout 0 should not be allowed (#414) 2022-06-29 11:14:23 +03:00
Jure Bajic
cbe15e7f44 Support Ubuntu 22.04 (#411)
* Fix toolchain url

* Add 22.04 os script

* Update centos-9 stream

* Update zlib

* Upgrade dotnet version

* Add libipt

* Update toolchain extension

* Add release container

* Remove CentOS 8

* Enable arm packaging

* Add centos 9 container

* Ignore missing python virtualenv

* Add virtualenvwrapper

* Rearrange order

* Install virtualenv without user

* Run virtualenv via python

* Add logrotate dependency

* Bump up checkout action version
2022-06-23 14:02:21 +02:00
Jeremy B
65a7ba01da Add information on show replicas to express how up-to-date a replica is (#412)
* Add test

* Add implementation and adapted test

* Update workloads.yaml to have a timeout > 0

* Update tests (failing due to merging of "add replica state")
2022-06-23 10:22:57 +02:00
Kostas Kyrimis
7a2bbd4bb3 Remove ref from std::string_view (#215) 2022-06-21 17:18:43 +03:00
Jeremy B
589e0e098b Forbid two replicas to point to the same ip port (#406) 2022-06-20 17:10:20 +03:00
Jeremy B
41d4185156 Add limit batches option to start stream query (#392) 2022-06-20 15:09:45 +03:00
Marko Budiselić
599c0a641f Add replica state to SHOW REPLICAS (#379) 2022-06-20 13:28:42 +03:00
Tyler Neely
1fb49c4865 Add an uncatchable kill method to the python MemgraphInstanceRunner for testing abrupt failures (#405) 2022-06-15 12:16:11 +03:00
János Benjamin Antal
df1485aeec Run clang-tidy also on our header files (#402) 2022-06-10 11:26:24 +02:00
Jure Bajic
b2e1056389 Fix websocket connection (#403)
* Fix websocket connection with other clients
2022-06-09 22:19:43 +02:00
Jeremy B
e4c9411e63 Correcting USE_JEMALLOC flag (#401)
* Correcting USE_JEMALLOC flag

* Removing references to jemalloc local copy in /libs
2022-06-09 18:55:28 +02:00
Jeremy B
a0bc1371dd Adding "raw message" column to the result returned by CHECK STREAM query (#394)
* Adding "raw message" column to the result return by CHECK STREAM query

* Update way results of CHECK STREAM are built

* Adapting CHECK STREAM integration tests (Pulsar/Kafka) to run with new result structure

* Adding new tests covering the check stream functionality

* Uppercase constants in stream tests

* Reformat f-strings
2022-06-08 23:17:44 +02:00
Marko Budiselić
21ad5d4328 Fix SHOW REPLICATION ROLE and SHOW REPLICAS (#376) 2022-05-20 20:17:59 -07:00
Marko Budiselić
8e3ab1ad0f Add frequent replica ping (#380) 2022-05-20 19:29:17 -07:00
Marko Budiselić
cccf32e79d Add default commented values for Bolt SSL flags (#398) 2022-05-18 08:34:07 +02:00
Jure Bajic
22bd60c613 Fix shutdown call (#395)
* Fix shutdown not called

* Add ssl server tests
2022-05-18 07:50:06 +02:00
János Benjamin Antal
8059a3e653 Improve magic functions docs (#391)
* Improve documentation of magic functions

* Improve wording for decorators
2022-05-17 15:33:28 +02:00
Antonio Andelic
a7f4c98bea Fix module symbol loading (#335)
* Use DEEPBIND

* Add dependency on libstdc++

Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
2022-05-13 11:43:10 +02:00
Jure Bajic
483f4d04bd Update precommit hooks (#393)
* Add pre-commit

* Add clang pre-commit

* Install pre-commit in init
2022-05-11 21:07:04 +02:00
János Benjamin Antal
3e7aef432f Improve docker-compose setup for running streams e2e tests locally (#388)
* Separate pulsar and kafka docker-compose files

* Add docker-compose for redpanda

* Add small docs how to use docker-compose files
2022-05-10 13:31:45 +02:00
Jeremy B
10ea9c773e Making mgp_error enum class when compiling c++. Remains enum when com… (#389)
* Making mgp_error enum class when compiling c++. Remains enum when compiling c.
2022-05-05 17:48:17 +02:00
Jeremy B
b782271be8 Fix shared module loading
* Moving function add_query_module from CMakeLists from tests/e2e/magic_functions to tests/e2e

* Adding failing test copying behavior when loading c module (.so) into memgraph.

* Fixing issue where NO_ERROR status returned  MgpTransAddFixedResult was converted to false

* Removing unnecessary transformation

* removing incorrect parameterization of test

* re-adding parametrized transformation
2022-04-28 20:28:44 +02:00
Jure Bajic
a8ffcfa046 Update license year
* Update license year

Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>

Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
2022-04-27 13:31:37 +02:00
Jure Bajic
7b78665cd8 Implement Bolt over WebSocket with asio
* Replace server implementation with asio

* Add support for bolt over WebSocket
2022-04-27 10:13:16 +02:00
Josip Matak
4abaf27765 Memgraph magic functions (#345)
* Extend mgp_module with include adding functions

* Add return type to the function API

* Change Cypher grammar

* Add Python support for functions

* Implement error handling

* E2e tests for functions

* Write cpp e2e functions

* Create mg.functions() procedure

* Implement case insensitivity for user-defined Magic Functions.
2022-04-21 15:45:31 +02:00
Kostas Kyrimis
ea2806bd57 Implement foreach clause (#351) 2022-04-11 13:55:34 +03:00
Siniša Šušnjar
c8dbaf5979 Small io network socket fixes (#360)
* Modernize AddrInfo

* Modernize Socket
2022-04-08 14:38:13 +02:00
Jure Bajic
17049ada09 Resolve python dependency issues (#372) 2022-04-07 17:56:18 +02:00
Jure Bajic
1b619f51b2 Add docker release action (#356)
* Add docker release action

* Add debian-11 arm script

* Remove test prefix

* Update package_docker script

* Update release script

* Unify architecture extension
2022-04-07 15:23:18 +02:00
János Benjamin Antal
537855a0b2 Fix usages of constexpr (#367)
* Fix usages of constexpr
2022-03-31 13:52:43 +02:00
Jure Bajic
5822b44b15 Fix CentOS 7 release process (#369) 2022-03-31 07:20:53 +02:00
g-despot
bf01c58ed9 Update README.md 2022-03-22 04:50:01 +01:00
g-despot
89a5566f3f Update README.md 2022-03-22 04:49:03 +01:00
g-despot
29452f8774 Update README.md 2022-03-21 16:49:02 +01:00
jbajic
60ad05acff Address review comments 2022-03-14 15:47:41 +01:00
jbajic
4f593c7fca Adapt precommit to run through all files 2022-03-14 15:47:41 +01:00
jbajic
770ea1189a Fix SSL errors 2022-03-14 15:47:41 +01:00
jbajic
695bb343f1 Resolve clang-tidy error 2022-03-14 15:47:41 +01:00
jbajic
12b4ec1589 Add memgraph namespace 2022-03-14 15:47:41 +01:00
Antonio Andelic
b33d2c3940 use boost ssl 2022-03-14 15:47:41 +01:00
Marko Budiselić
477acad1f6 Update CODEOWNERS file (#362) 2022-02-25 13:42:27 +01:00
János Benjamin Antal
ddca2b40f5 Remove unimplemented function declaration (#357) 2022-02-24 11:49:22 +01:00
Antonio Andelic
4817be0add Polishing for release 2022-02-17 14:47:11 +01:00
Jure Bajic
3fb7e5378d Add websocket port argument (#355) 2022-02-17 10:36:15 +01:00
Antonio Andelic
1d88893715 Add Websocket e2e tests (#353) 2022-02-17 10:36:10 +01:00
Jure Bajic
bd2c30fddc Websocket unit tests (#334) 2022-02-17 10:35:56 +01:00
Antonio Andelic
06e6ead4d2 WSS support (#327) 2022-02-17 10:35:56 +01:00
Jure Bajic
728b37080d Fix authorization check (#332) 2022-02-17 10:35:56 +01:00
Antonio Andelic
48a531aac1 Add websocket authentication (#322) 2022-02-17 10:35:48 +01:00
Antonio Andelic
914fc1a656 Send logs over websocket (#316) 2022-02-17 10:31:26 +01:00
Antonio Andelic
1d1c182c2d Add websocket server and log stream (#314) 2022-02-17 10:31:26 +01:00
János Benjamin Antal
c6e19ec09f Fix folly in toolchain-v4 (#346) 2022-02-14 14:31:27 +01:00
János Benjamin Antal
693dab78d2 Remove unused URLs from libs/setup.sh (#352) 2022-02-11 13:56:43 +01:00
Antonio Andelic
69eca9b043 Procedures for handling modules (#330) 2022-02-11 11:29:41 +01:00
Antonio Andelic
5aeaad198b Define SHOW VERSION query (#265) 2022-02-10 10:30:14 +01:00
Antonio Andelic
b23f88c607 Use gzip for packaging docker image (#348) 2022-02-07 15:50:43 +01:00
Kostas Kyrimis
4fd8bdce4c Make temporal types now functions use the query context timestamp (#333) 2022-02-04 12:57:38 +01:00
Antonio Andelic
265b203b00 Build for ARM64 (#340) 2022-02-03 13:03:35 +01:00
János Benjamin Antal
661e5185d8 Fix nested transaction error (#339) 2022-02-03 09:41:23 +01:00
Marko Budiselić
7348ad6800 Add init quicklisp proxy flag (#342) 2022-02-02 09:07:57 +01:00
János Benjamin Antal
6c00d146f2 Add configs for kafka streams (#328) 2022-01-31 17:26:53 +01:00
Antonio Andelic
ced84e17b6 Update nlohmann and remove guards (#336) 2022-01-26 08:12:09 +01:00
Jure Bajic
834ce5f51a Remove support for deb9 (#329) 2022-01-25 12:23:50 +01:00
Antonio Andelic
bb1308acc7 Use libs from toolchain (#326) 2022-01-21 10:22:36 +01:00
János Benjamin Antal
e1f31d3d02 Toolchain v4 with fbthrift (#318) 2022-01-18 12:51:27 +01:00
Josip Matak
339cd9b84e Change Python root directory (#289) 2022-01-18 08:23:17 +01:00
Antonio Andelic
7deac4ac8b Extra procedures transformation info (#310) 2022-01-13 13:46:32 +01:00
Jure Bajic
fec887a67c Adapt header-checker (#323) 2022-01-13 09:18:27 +01:00
Antonio Andelic
f3a3a4ed87 Unsafe git protocol fix (#324) 2022-01-12 11:36:06 +01:00
János Benjamin Antal
ec65187442 Use correct error in log message (#321) 2022-01-10 13:50:46 +01:00
János Benjamin Antal
079c0495c3 Increase reference count when returning Py_None (#320) 2022-01-07 12:56:52 +01:00
Kostas Kyrimis
90cfed2ada Remove unnecessary use of string in server start method (#317) 2021-12-14 16:13:24 +01:00
Antonio Andelic
8716b8e992 Update to newest spdlog and fmt (#309) 2021-12-07 11:05:33 +01:00
Antonio Andelic
a277541354 Update debian base image (#311) 2021-12-07 10:10:04 +01:00
Antonio Andelic
b01564f179 Update license date for 2.1.1 (#312) 2021-12-06 12:35:30 +01:00
Antonio Andelic
d9bb4e2e46 Use default values instantly for batch size/interval (#306) 2021-11-30 16:29:51 +01:00
János Benjamin Antal
05d0aee494 Avoid crash in case of loading old stream (#302) 2021-11-30 15:27:11 +01:00
Antonio Andelic
aabec99a8e Procedure for stream configs (#301) 2021-11-29 08:56:10 +01:00
Antonio Andelic
d277dd49a3 Update stats with query modules (#304) 2021-11-26 15:31:25 +01:00
Antonio Andelic
18f9d19b18 Update pyyaml version (#305) 2021-11-25 10:12:33 +01:00
Antonio Andelic
530eed5c6d Update change date (#303) 2021-11-24 16:26:35 +01:00
Antonio Andelic
12f4e0068a Small polishing and fixes 2021-11-16 17:23:20 +01:00
Antonio Andelic
e8976e0f1c Update mgp API for Pulsar (#298) 2021-11-16 16:00:25 +01:00
Kostas Kyrimis
6eb52581eb Retry transaction on consumer (#294) 2021-11-16 15:58:36 +01:00
Antonio Andelic
8606e69fd6 Add support for type in show streams (#300) 2021-11-16 15:58:36 +01:00
Antonio Andelic
c7b045bffc Pulsar e2e tests (#296) 2021-11-16 15:58:35 +01:00
Antonio Andelic
b66cc66503 Add support for Pulsar queries (#292) 2021-11-16 15:52:58 +01:00
Antonio Andelic
0e4719018a Introduce pulsar dependency (#286) 2021-11-16 15:47:40 +01:00
Antonio Andelic
0ebd52aac3 Support for different consumers (#280) 2021-11-16 15:40:27 +01:00
Antonio Andelic
6c971b856e Fix ORDER BY and temporal types (#299) 2021-11-16 10:59:25 +01:00
Kostas Kyrimis
6cb293688d Fix check stream not reseting back to the correct offset (#295) 2021-11-15 14:44:30 +01:00
Jure Bajic
16709dff6c Add stats and notifications in results metadata (#285) 2021-11-15 13:51:13 +01:00
Kostas Kyrimis
95dd3481c0 Add more information to the MG_ASSERT in release mode (#291) 2021-11-12 09:09:15 +01:00
Kostas Kyrimis
47c0c629c7 Add procedure for setting a kafka stream offset (#282) 2021-11-11 12:07:58 +01:00
Kostas Kyrimis
636c551047 Remove linter concurrency-mt (#287) 2021-11-02 09:19:41 +01:00
Marko Budiselić
72384b2b71 Add internals pages link (#288) 2021-11-01 09:49:22 +01:00
Jure Bajic
a9b1ff9bea Support Bolt v4.3 protocol (#226) 2021-10-28 13:05:09 +02:00
Jure Bajic
e40bc32624 Fix daily benchmark workflow (#284) 2021-10-27 09:05:46 +02:00
Antonio Andelic
bd21bc82b7 Add license to cpp/hpp/py test files (#283) 2021-10-26 08:53:56 +02:00
Jure Bajic
9c1680e82c Change how to get branch name (#281) 2021-10-25 09:01:26 +02:00
Antonio Andelic
1a78c3695d Support multiple types for weighted shortest path (#278) 2021-10-19 14:39:23 +02:00
Kostas Kyrimis
10196f3d7d Add bootstrap servers to create stream (#274) 2021-10-18 10:49:00 +02:00
Jure Bajic
2c86fefbb5 Run diff on master (#275) 2021-10-18 08:40:07 +02:00
Antonio Andelic
df689dccd5 Make Memgraph work with fetch content (#256) 2021-10-14 18:58:42 +02:00
Jure Bajic
6033fbe9bd Clean up topics before starting test (#267) 2021-10-14 16:57:46 +02:00
Jure Bajic
519a204424 Add retry to get_version script (#268) 2021-10-14 16:07:40 +02:00
Antonio Andelic
3fc5e57fef Update dependency source when package all is called (#272) 2021-10-14 15:16:32 +02:00
János Benjamin Antal
906933d2b3 Fix typo in procedure C-API (#273) 2021-10-14 08:50:16 +02:00
Antonio Andelic
e9a937ad6d Update license headers and change date (#270) 2021-10-13 10:06:07 +02:00
Antonio Andelic
6dc2cdfae4 Fix trigger with detach delete (#266) 2021-10-12 16:21:28 +02:00
g-despot
69ea410406 Add CONTRIBUTING and CODE_OF_CONDUCT (#269) 2021-10-12 14:11:07 +02:00
Kostas Kyrimis
24a576c8e9 Make write procedures trigger aware (#262) 2021-10-11 13:39:57 +02:00
Antonio Andelic
d417ffee6e Update community build step (#261) 2021-10-08 14:36:17 +02:00
Antonio Andelic
f868b51483 Update mgconsole version (#260) 2021-10-08 11:19:21 +02:00
g-despot
26dc73c809 Update installation in README (#263) 2021-10-08 10:28:09 +02:00
g-despot
9119c89eba Add updated startup message (#259) 2021-10-07 15:56:06 +02:00
Antonio Andelic
4e7ea34ae9 Add message with link utility (#243) 2021-10-07 14:51:30 +02:00
János Benjamin Antal
d7d291217b Remove debug print (#251) 2021-10-07 09:14:05 +02:00
Marko Budiselić
42facfacbc Add support for Debian 11 (#212) 2021-10-06 16:17:41 +02:00
János Benjamin Antal
bd1e848363 Add issue template (#258) 2021-10-06 11:52:15 +02:00
antonio2368
0ffb58b764 Fix docker packaging and add for platform build (#257) 2021-10-06 10:20:37 +02:00
g-despot
9551b6973a Fix URL in build badge 2021-10-05 14:30:09 +02:00
Jure Bajic
f7ef5f50a4 Update workflow (#255) 2021-10-05 09:03:48 +02:00
Mislav Vuletić
046cd80054 Make readme awesome (#228)
Co-authored-by: antonio2368 <antonio2368@users.noreply.github.com>
Co-authored-by: Ivan Despot <ivan.g.despot@outlook.com>
2021-10-04 16:15:59 +02:00
Jure Bajic
9e678f8cbe Add new memgraph licenses and move third party licenses (#248) 2021-10-03 12:26:48 +02:00
antonio2368
69e4f1784b Small polishing for Release workflow (#250) 2021-10-03 11:09:57 +02:00
Kostas Kyrimis
d04e23805d Address temporal type epic comments (#249) 2021-10-02 18:48:50 +02:00
Kostas Kyrimis
7e82cc6550 Add uniform naming for temporal type properties (#246) 2021-10-02 16:14:27 +02:00
Kostas Kyrimis
18b801a722 Add e2e tests for temporal types (#241) 2021-10-02 15:31:00 +02:00
Kostas Kyrimis
7e81a95b81 Fix parsing negative durations (#237) 2021-10-02 15:30:35 +02:00
antonio2368
0a23eb11ae Define procedure API for temporal types (#229) 2021-10-02 15:30:34 +02:00
Kostas Kyrimis
8ea281649a Remove support for iso duration and fix bolt (#235) 2021-10-02 14:54:48 +02:00
Kostas Kyrimis
da68f86fc9 Add support for printing out in the ast temporal data values (#227) 2021-10-02 14:54:48 +02:00
Kostas Kyrimis
738b5fb8d8 Add duration support for timestamp function (#236) 2021-10-02 14:54:48 +02:00
Kostas Kyrimis
b45ae403e6 Define dump command for all temporal types (#225) 2021-10-02 14:54:48 +02:00
Kostas Kyrimis
d744711e5e Define functions that return a specific field of each temporal type (#221) 2021-10-02 14:54:48 +02:00
Kostas Kyrimis
8d87e38c64 Add temporal type arithmetic for queries (#214) 2021-10-02 14:54:48 +02:00
Kostas Kyrimis
086fc47769 Define subtraction and addition for temporal types (#209) 2021-10-02 14:54:48 +02:00
Kostas Kyrimis
5abcb5081d Define timestamp function that return temporal types in microseconds since epoch representation (#205) 2021-10-02 14:54:48 +02:00
Kostas Kyrimis
9534dfb6ed Change the timestamp function to return microseconds instead of milliseconds (#204) 2021-10-02 14:54:48 +02:00
antonio2368
e2112faff0 Functions for defining temporals (#197) 2021-10-02 14:54:47 +02:00
Kostas Kyrimis
6913feacc7 Add support for temporal data types in the communication Bolt layer (#198) 2021-10-02 14:54:47 +02:00
antonio2368
4e604de9d7 Add support for parsing temporal types (#187) 2021-10-02 14:54:45 +02:00
antonio2368
e628b5ba6e Add temporal types to TypedValue (#176) 2021-10-02 14:53:53 +02:00
antonio2368
5da32c1bff Add TemporalData to PropertyValue (#174) 2021-10-02 14:53:53 +02:00
János Benjamin Antal
ccca46370d Fix RWTypeChecker and some small improvements (#247)
* Fix doc of mgp_graph_vertices

* Make write_proc example meaningful write procedure example

* Improve wrap_exceptions

* Add check for write procedures for ReadWriteTypeChecker

* Change error code in case of invalid default value for optional arguments
2021-10-02 13:18:37 +02:00
János Benjamin Antal
e33fa5f9c0 Add example write procedure (#244) 2021-10-02 13:18:37 +02:00
János Benjamin Antal
721eefe263 Handle write procedures in queries and update docs of properties iterator (#238) 2021-10-02 13:18:36 +02:00
János Benjamin Antal
be9ed7e879 Python wrapper for write procedures (#234)
* Rename mgp_graph_remove to mgp_graph_delete

* Add mgp_graph_detach_delete

* Add PyGraph functions

* Add _mgp exceptions

* Use unified error handling in python wrapper

* Ignore clang-tidy warnings

* Add mgp.Graph, mgp.Vertex and mgp.Edge mutable functions

* Add python write procedure registration

* Add `is_write` result field to mg.procedures

* Use storage::View::NEW for write procedures

* Add simple tests for write procedures

* Remove false information about IDs
2021-10-02 13:17:41 +02:00
Benjamin Antal
482798295e Add write procedures 2021-10-02 13:17:41 +02:00
antonio2368
d58a1cbb58 License key introduction and removing community edition package (#232) 2021-09-29 19:14:39 +02:00
antonio2368
8dc3153fde Don't strip queries after execute for CREATE TRIGGER only (#245) 2021-09-24 13:14:05 +02:00
antonio2368
b94e50bf1c Use from_timestamp for WAL retention (#240) 2021-09-24 12:14:54 +02:00
antonio2368
9f855676cb Add jemalloc purge (#239) 2021-09-21 14:43:27 +02:00
Jure Bajic
f560293657 Add Property Map support in CREATE clause (#220) 2021-09-09 12:39:13 +02:00
János Benjamin Antal
2afc1b99f6 New MGP API (#219) 2021-09-09 10:44:47 +02:00
antonio2368
5a8bda2531 Improve MERGE performance by using indices (#213) 2021-08-30 11:39:06 +02:00
Jure Bajic
0138d277d4 Improve serialization error message (#208) 2021-08-20 10:47:07 +02:00
János Benjamin Antal
09cfca35f8 Add privilege check in triggers and streams (#200) 2021-07-22 16:22:08 +02:00
Marko Budiselić
09c58501f1 Migrate release package to toolchain-v3 (#191) 2021-07-20 15:25:12 +02:00
Jure Bajic
945bbfdc49 Clean up containers (#201) 2021-07-19 08:40:28 +02:00
antonio2368
45ef069372 Create pull request template (#199) 2021-07-15 14:19:02 +02:00
antonio2368
e51954fc94 Update toolchain to v3 (#189)
* Make memgraph buildable with new toolchain

* Use toolchain v3 in workflows
2021-07-08 14:20:48 +02:00
János Benjamin Antal
7776160836 Add third party licenses (#195)
* Add third party licences

* Package the third-party licences
2021-07-08 08:55:09 +02:00
antonio2368
ae280fd8db Polishing and fixes (#196)
* Remove SHOW from Memgraph lexer

* Change get_payload to payload
2021-07-07 18:04:17 +02:00
János Benjamin Antal
fb5a2ed4b6 Small improvements for streams (#193)
* Add timeout to CHECK STREAM

* Handle RdKafka::ERR__MAX_POLL_EXCEEDED in consumer

Co-authored-by: Jure Bajic <jbajic@users.noreply.github.com>

Co-authored-by: Jure Bajic <jbajic@users.noreply.github.com>
2021-07-07 16:19:59 +02:00
antonio2368
6cfec787dc Allow dash in topic name (#192)
* Support dash in the topic names

* Add more topic names and consumer groups in tests
2021-07-07 16:19:59 +02:00
János Benjamin Antal
13c9bf76af Add e2e tests for streams (#190)
* Add base of e2e tests

* Add python dependencies

* Explicitly close customer in destructor

* Parametrize tests and add test for CHECK STREAM

* Add tests for SHOW STREAMS

* Add test for concurrent start/stop during check

* Add test for calling check with an already started stream

* Run streams e2e tests on CI servers

Co-authored-by: antonio2368 <antonio2368@users.noreply.github.com>
Co-authored-by: Jure Bajic <jbajic@users.noreply.github.com>
2021-07-07 16:19:59 +02:00
Kostas Kyrimis
2e1a717dcb Add telemetry for streams and triggers (#188)
* Add event counters to streams and triggers
2021-07-07 16:19:59 +02:00
János Benjamin Antal
ad32db5168 Use transformations in streams and CHECK STREAM (#185)
* Use the correct transformation result type

* Execute the result queries in streams

* Change the result type of parameters to nullable map

* Serialize transformation name

* Fix order of transformation parameters

* Use actual transformation in Streams

* Clear the Python transformations under GIL

* Add CHECK STREAM query

* Handle missing record fields properly
2021-07-07 16:19:59 +02:00
Kostas Kyrimis
a37755ce43 Add remaining query functionality SHOW STREAMS (#184)
* Add query functionality SHOW STREAMS
2021-07-07 16:13:26 +02:00
Kostas Kyrimis
a928c158da Add python API for messages transformations (#181)
* Add python messages/transformations implementation

* Added fixed result return type to transformations

* Added is_deprecated to mgp_trans
2021-07-07 16:08:23 +02:00
János Benjamin Antal
ac230d0c2d Add basic queries for managing streams (#179)
* Add CREATE, START, STOP and DROP queries

* Fix definition of port in replica query

* Explicitly stop the consumer before removing

* Fix offset committing in Consumer

* Add tests for basic stream queries

* Remove unnecessary WITH keywords from CREATE query

* Add tests

* Add STREAM privilege

* Disable not working test

The functionality is tested manually, but I couldn't make it work with
the mock kafka cluster.

* Add support for multiple topic names

* Replace skiplist by synchronized map

* Make Consumer::Test const and improve error handling

The improvement in the error handling is mostly done regarding to the
Test function. Instead of trying to revert the assignments, Test just
stores the last commited assignment. When Start or Test is called, they
check for the last commited assignments, and if it is saved, then they
try to restore it. This way:
1. All the failures are returned to the user (failed to save/restore)
2. Failed assignment cannot terminate Memgraph

* Make Test do not block creating/droping other streams
2021-07-07 16:08:22 +02:00
Kostas Kyrimis
d80ff745eb Introduce mgp_trans api (#175)
* Added public interface for registering mgp_trans and extended modules accordingly

* Added test for mgp_trans

* Added mg.transformations() to the module registry

Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
2021-07-07 15:57:36 +02:00
János Benjamin Antal
d6a6d280dd Add Streams on top of Kafka Consumer (#172)
* Stop the Consumer grafefully when it is destroyed

* Add Streams

* Add Streams to InterpreterContext

* Remove options to limit processed batches in Consumer

* Add Streams unit tests

* Stop waiting for a full batch if the Consumer stopped

* Add ReadLock functionality to Synchronized

* Use per Consumer-based locking

* Replace shared_mutex with RWLock
2021-07-07 15:57:36 +02:00
Kostas Kyrimis
4004e94ca1 Introduce messages C-API (#168) 2021-07-07 15:38:51 +02:00
János Benjamin Antal
36afc6c5f3 Add Kafka consumer (#167)
* Add CMake config for Kafka integration

* Add Consumer

* Add simple unit test for consumer

* Add explicit offset handling and test for it
2021-07-07 15:38:51 +02:00
antonio2368
3c9a46f823 Add Jure to CODEOWNERS (#194) 2021-07-07 12:35:28 +02:00
antonio2368
f994b68ad5 Increase default timeout to 10 minutes (#186) 2021-07-02 09:02:11 +02:00
antonio2368
3b336e3e0b Add CREATE SNAPSHOT query (#182) 2021-06-30 12:31:30 +02:00
antonio2368
715162e205 Disable LOAD CSV with config (#180) 2021-06-30 11:19:13 +02:00
antonio2368
e016c74e4b Trigger restore after query modules load (#183) 2021-06-30 10:01:51 +02:00
antonio2368
15911b64dc Use timers for query timeout thread (#163)
Co-authored-by: Benjamin Antal <benjamin.antal@memgraph.io>
2021-06-16 16:03:58 +02:00
antonio2368
cbf826e0c3 Load WAL on replica using transactions (#95) 2021-06-16 13:22:48 +02:00
antonio2368
644a3a0b2a Codeowners update (#173)
* CODEOWNERS update

* clang-tidy changes trigger the diff
2021-06-15 13:21:05 +02:00
antonio2368
8cd9f696cf Multiple isolation levels (#171)
This PR introduces READ COMMITTED and READ UNCOMMITTED isolation levels.
The isolation level can be set with a config or with a query for different scopes.
2021-06-14 15:47:57 +02:00
antonio2368
90a093bd95 Send py::Object as a reference to callback call (#169)
Sending the py::Object by value caused UB because multiple threads could copy the same object at the same time without the GIL.
By sending the object by reference we eliminate UB and avoid UB.
2021-06-10 10:40:18 +02:00
Marko Budiselić
542a324c96 Fix typing to cypher type impl inside mgp.py (#159) 2021-06-09 19:09:43 +02:00
Josip Seljan
cd03e13443 Upgrade Antlr to v4.9.2 (#161) 2021-06-09 12:50:27 +02:00
antonio2368
be91126134 Run the benchmarks daily (#164) 2021-06-08 08:44:12 +02:00
antonio2368
03cb007339 CHANGELOG redirects to the docs page (#165) 2021-06-07 16:26:15 +02:00
Kostas Kyrimis
524acb17a1 Add global version allocators for C in query modules (#162) 2021-06-07 14:45:05 +02:00
antonio2368
50f6e348dc Disable google-runtime-references clang-tidy check (#166) 2021-06-07 10:40:56 +02:00
Josip Seljan
839d45b3f8 Remove louvain and connectivity modules (#160) 2021-06-02 11:10:16 +02:00
antonio2368
560eb04f67 Small trigger fixes (#158)
* Fix warning message

* Update version

* Run query callbacks only on pull

* Use warn level for failure of loading a trigger
2021-05-26 19:57:08 +02:00
János Benjamin Antal
a3ecc52429 Use all available predefined variables (#157) 2021-05-26 14:37:11 +02:00
János Benjamin Antal
e8a1d15a55 Add filtering based on registered event types (#155)
* Add filtering to TriggerContextCollector

* Add all predefined variable to ANY triggers

* Make variable names consistent with event types
2021-05-26 14:37:11 +02:00
János Benjamin Antal
1abee1ed3a Use single thread for after triggers (#154) 2021-05-26 14:36:41 +02:00
János Benjamin Antal
5af3d0ff68 Add e2e tests for triggers (#152) 2021-05-26 14:36:41 +02:00
antonio2368
62a628c51f Add trigger queries and trigger store (#145)
Co-authored-by: Benjamin Antal <benjamin.antal@memgraph.io>
2021-05-26 14:36:41 +02:00
antonio2368
883f9c7ed3 Trigger event types (#144)
Co-authored-by: Benjamin Antal <benjamin.antal@memgraph.io>
2021-05-26 14:13:14 +02:00
antonio2368
b459639968 Other predefined variables (#143) 2021-05-26 14:13:14 +02:00
antonio2368
11c0dde11c Support removed vertices in the triggers (#140) 2021-05-26 14:13:14 +02:00
antonio2368
2f3fa656d9 Predefined variables (#138)
* Add support for injecting predefined variables in any scope
2021-05-26 14:13:14 +02:00
antonio2368
7bf40eb5d2 Add support for after commit triggers (#136) 2021-05-26 14:13:14 +02:00
antonio2368
7e44434cdf Initial trigger definition (#133)
* Pull out cypher query parsing logic

* Define trigger structure

* Run triggers before commit

* Use skip list for saving triggers
2021-05-26 14:13:14 +02:00
Josip Seljan
2a0b0d969f Replace mg_client with mgconsole (#153) 2021-05-26 11:59:36 +02:00
1196 changed files with 1758 additions and 175449 deletions

View File

@@ -1,5 +1,9 @@
---
Checks: '*,
-abseil-string-find-str-contains,
-altera-id-dependent-backward-branch,
-altera-struct-pack-align,
-altera-unroll-loops,
-android-*,
-cert-err58-cpp,
-cppcoreguidelines-avoid-c-arrays,
@@ -26,6 +30,7 @@ Checks: '*,
-fuchsia-virtual-inheritance,
-google-explicit-constructor,
-google-readability-*,
-google-runtime-references,
-hicpp-avoid-c-arrays,
-hicpp-avoid-goto,
-hicpp-braces-around-statements,
@@ -34,13 +39,14 @@ Checks: '*,
-hicpp-no-assembler,
-hicpp-no-malloc,
-hicpp-use-equals-default,
-hicpp-use-nullptr,
-hicpp-vararg,
-llvm-header-guard,
-llvm-include-order,
-llvmlibc-callee-namespace,
-llvmlibc-implementation-in-namespace,
-llvmlibc-restrict-system-libc-headers,
-misc-non-private-member-variables-in-classes,
-misc-unused-parameters,
-modernize-avoid-c-arrays,
-modernize-concat-nested-namespaces,
-modernize-pass-by-value,
@@ -50,9 +56,14 @@ Checks: '*,
-performance-unnecessary-value-param,
-readability-braces-around-statements,
-readability-else-after-return,
-readability-function-cognitive-complexity,
-readability-implicit-bool-conversion,
-readability-magic-numbers,
-readability-named-parameter'
-readability-named-parameter,
-misc-no-recursion,
-concurrency-mt-unsafe,
-bugprone-easily-swappable-parameters'
WarningsAsErrors: ''
HeaderFilterRegex: 'src/.*'
AnalyzeTemporaryDtors: false
@@ -79,4 +90,3 @@ CheckOptions:
- key: modernize-use-nullptr.NullMacros
value: 'NULL'
...

View File

@@ -16,6 +16,7 @@ tmpdir=$(mktemp -d repo-XXXXXXXX)
trap "rm -rf $tmpdir" EXIT INT
modified_files=$(git diff --cached --name-only --diff-filter=AM $against | sed -nE "/.*\.(cpp|cc|cxx|c|h|hpp)$/p")
FAIL=0
for file in $modified_files; do
echo "Checking $file..."
@@ -23,13 +24,13 @@ for file in $modified_files; do
git checkout-index --prefix="$tmpdir/" -- $file
echo "Running clang-format..."
$project_folder/tools/git-clang-format $tmpdir/$file
code=$?
if [ $code -ne 0 ]; then
break
# Do not break header checker
echo "Running header checker..."
$project_folder/tools/header-checker.py $tmpdir/$file $file --amend-year
CODE=$?
if [ $CODE -ne 0 ]; then
FAIL=1
fi
done;
return $code
return ${FAIL}

34
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,34 @@
---
name: Bug report
about: Create a report to help us improve
title: "[BUG] "
labels: bug
assignees: gitbuda, antonio2368
---
**Memgraph version**
Which version did you use?
**Environment**
Some information about the environment you are using Memgraph on: operating
system, how do you connect, with or without docker, which driver etc.
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Run the following query '...'
2. Click on '....'
**Expected behavior**
A clear and concise description of what you expected to happen.
**Logs**
If applicable, add logs of Memgraph, CLI output or screenshots to help explain
your problem.
**Additional context**
Add any other context about the problem here.

14
.github/pull_request_template.md vendored Normal file
View File

@@ -0,0 +1,14 @@
[master < Epic] PR
- [ ] Check, and update documentation if necessary
- [ ] Write E2E tests
- [ ] Compare the [benchmarking results](https://bench-graph.memgraph.com/) between the master branch and the Epic branch
- [ ] Provide the full content or a guide for the final git message
[master < Task] PR
- [ ] Check, and update documentation if necessary
- [ ] Provide the full content or a guide for the final git message
To keep docs changelog up to date, one more thing to do:
- [ ] Write a release note here
- [ ] Tag someone from docs team in the comments

82
.github/workflows/daily_benchmark.yaml vendored Normal file
View File

@@ -0,0 +1,82 @@
name: Daily Benchmark
on:
workflow_dispatch:
schedule:
- cron: "0 22 * * *"
jobs:
release_benchmarks:
name: "Release benchmarks"
runs-on: [self-hosted, Linux, X64, Diff, Gen7]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build only memgraph release binaries.
cd build
cmake -DCMAKE_BUILD_TYPE=release ..
make -j$THREADS
- name: Run macro benchmarks
run: |
cd tests/macro_benchmark
./harness QuerySuite MemgraphRunner \
--groups aggregation 1000_create unwind_create dense_expand match \
--no-strict
- name: Get branch name (merge)
if: github.event_name != 'pull_request'
shell: bash
run: echo "BRANCH_NAME=$(echo ${GITHUB_REF#refs/heads/} | tr / -)" >> $GITHUB_ENV
- name: Get branch name (pull request)
if: github.event_name == 'pull_request'
shell: bash
run: echo "BRANCH_NAME=$(echo ${GITHUB_HEAD_REF} | tr / -)" >> $GITHUB_ENV
- name: Upload macro benchmark results
run: |
cd tools/bench-graph-client
virtualenv -p python3 ve3
source ve3/bin/activate
pip install -r requirements.txt
./main.py --benchmark-name "macro_benchmark" \
--benchmark-results-path "../../tests/macro_benchmark/.harness_summary" \
--github-run-id "${{ github.run_id }}" \
--github-run-number "${{ github.run_number }}" \
--head-branch-name "${{ env.BRANCH_NAME }}"
- name: Run mgbench
run: |
cd tests/mgbench
./benchmark.py --num-workers-for-benchmark 12 --export-results benchmark_result.json pokec/medium/*/*
- name: Upload mgbench results
run: |
cd tools/bench-graph-client
virtualenv -p python3 ve3
source ve3/bin/activate
pip install -r requirements.txt
./main.py --benchmark-name "mgbench" \
--benchmark-results-path "../../tests/mgbench/benchmark_result.json" \
--github-run-id "${{ github.run_id }}" \
--github-run-number "${{ github.run_number }}" \
--head-branch-name "${{ env.BRANCH_NAME }}"

View File

@@ -1,11 +1,20 @@
name: Diff
concurrency:
group: ${{ github.head_ref || github.sha }}
cancel-in-progress: true
on:
push:
branches:
- master
workflow_dispatch:
pull_request:
paths-ignore:
- 'docs/**'
- '**/*.md'
- '.clang-*'
- "docs/**"
- "**/*.md"
- ".clang-format"
- "CODEOWNERS"
- "licenses/*"
jobs:
community_build:
@@ -13,10 +22,12 @@ jobs:
runs-on: [self-hosted, Linux, X64, Diff]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -25,7 +36,7 @@ jobs:
- name: Build community binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -38,55 +49,67 @@ jobs:
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
ctest -R memgraph__unit --output-on-failure -j$THREADS
- name: Run stress test
run: |
cd tests/stress
./continuous_integration
- name: Create community DEB package
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
# Create community DEB package.
cd build
mkdir output && cd output
cpack -G DEB --config ../CPackConfig.cmake
- name: Save community DEB package
uses: actions/upload-artifact@v2
with:
name: "Community DEB package"
path: build/output/memgraph*.deb
code_analysis:
name: "Code analysis"
runs-on: [self-hosted, Linux, X64, Diff]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
fetch-depth: 0
- name: Build combined ASAN, UBSAN and coverage binaries
# This is also needed if we want do to comparison against other branches
# See https://github.community/t/checkout-code-fails-when-it-runs-lerna-run-test-since-master/17920
- name: Fetch all history for all tags and branches
run: git fetch
- name: Initialize deps
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
- name: Set base branch
if: ${{ github.event_name == 'pull_request' }}
run: |
echo "BASE_BRANCH=origin/${{ github.base_ref }}" >> $GITHUB_ENV
- name: Set base branch # if we manually dispatch or push to master
if: ${{ github.event_name != 'pull_request' }}
run: |
echo "BASE_BRANCH=origin/master" >> $GITHUB_ENV
- name: Python code analysis
run: |
CHANGED_FILES=$(git diff -U0 ${{ env.BASE_BRANCH }}... --name-only)
for file in ${CHANGED_FILES}; do
echo ${file}
if [[ ${file} == *.py ]]; then
python3 -m black --check --diff ${file}
python3 -m isort --check-only --diff ${file}
fi
done
- name: Build combined ASAN, UBSAN and coverage binaries
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
cd build
cmake -DTEST_COVERAGE=ON -DASAN=ON -DUBSAN=ON ..
make -j$THREADS memgraph__unit
@@ -94,7 +117,7 @@ jobs:
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run unit tests. It is restricted to 2 threads intentionally, because higher concurrency makes the timing related tests unstable.
cd build
@@ -103,7 +126,7 @@ jobs:
- name: Compute code coverage
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Compute code coverage.
cd tools/github
@@ -114,17 +137,17 @@ jobs:
tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu
- name: Save code coverage
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/generated/code_coverage.tar.gz
- name: Run clang-tidy
run: |
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Restrict clang-tidy results only to the modified parts
git diff -U0 master... -- src ':!*.hpp' | ./tools/github/clang-tidy/clang-tidy-diff.py -p 1 -j $THREADS -path build | tee ./build/clang_tidy_output.txt
git diff -U0 ${{ env.BASE_BRANCH }}... -- src | ./tools/github/clang-tidy/clang-tidy-diff.py -p 1 -j $THREADS -path build -regex ".+\.cpp" | tee ./build/clang_tidy_output.txt
# Fail if any warning is reported
! cat ./build/clang_tidy_output.txt | ./tools/github/clang-tidy/grep_error_lines.sh > /dev/null
@@ -134,10 +157,12 @@ jobs:
runs-on: [self-hosted, Linux, X64, Diff]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -146,7 +171,7 @@ jobs:
- name: Build debug binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -159,7 +184,7 @@ jobs:
- name: Run leftover CTest tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run leftover CTest tests (all except unit and benchmark tests).
cd build
@@ -191,14 +216,14 @@ jobs:
- name: Run cppcheck and clang-format
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run cppcheck and clang-format.
cd tools/github
./cppcheck_and_clang_format diff
- name: Save cppcheck and clang-format errors
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/cppcheck_and_clang_format.txt
@@ -208,10 +233,12 @@ jobs:
runs-on: [self-hosted, Linux, X64, Diff]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -220,7 +247,7 @@ jobs:
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -236,29 +263,30 @@ jobs:
./continuous_integration
- name: Save quality assurance status
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "GQL Behave Status"
path: |
tests/gql_behave/gql_behave_status.csv
tests/gql_behave/gql_behave_status.html
- name: Run e2e replication tests
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
ctest -R memgraph__unit --output-on-failure -j$THREADS
- name: Run e2e tests
run: |
# TODO(gitbuda): Setup mgclient and pymgclient properly.
cd tests
./setup.sh
source ve3/bin/activate
cd e2e
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../../libs/mgclient/lib python runner.py --workloads-path replication/workloads.yaml
- name: Run e2e memory control tests
run: |
cd tests
./setup.sh
source ve3/bin/activate
cd e2e
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../../libs/mgclient/lib python runner.py --workloads-path memory/workloads.yaml
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../../libs/mgclient/lib python runner.py --workloads-root-directory .
- name: Run stress test (plain)
run: |
@@ -279,21 +307,26 @@ jobs:
- name: Create enterprise DEB package
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
cd build
# create mgconsole
# we use the -B to force the build
make -j$THREADS -B mgconsole
# Create enterprise DEB package.
cd build
mkdir output && cd output
cpack -G DEB --config ../CPackConfig.cmake
- name: Save enterprise DEB package
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Enterprise DEB package"
path: build/output/memgraph*.deb
- name: Save test data
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
if: always()
with:
name: "Test data"
@@ -307,10 +340,12 @@ jobs:
#continue-on-error: true
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -319,7 +354,7 @@ jobs:
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -335,7 +370,7 @@ jobs:
./run.sh test --binary ../../build/memgraph --run-args "test-all --node-configs resources/node-config.edn" --ignore-run-stdout-logs --ignore-run-stderr-logs
- name: Save Jepsen report
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
if: ${{ always() }}
with:
name: "Jepsen Report"
@@ -346,10 +381,12 @@ jobs:
runs-on: [self-hosted, Linux, X64, Diff, Gen7]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -358,12 +395,12 @@ jobs:
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
# Build only memgraph release binarie.
# Build only memgraph release binaries.
cd build
cmake -DCMAKE_BUILD_TYPE=release ..
make -j$THREADS
@@ -375,6 +412,16 @@ jobs:
--groups aggregation 1000_create unwind_create dense_expand match \
--no-strict
- name: Get branch name (merge)
if: github.event_name != 'pull_request'
shell: bash
run: echo "BRANCH_NAME=$(echo ${GITHUB_REF#refs/heads/} | tr / -)" >> $GITHUB_ENV
- name: Get branch name (pull request)
if: github.event_name == 'pull_request'
shell: bash
run: echo "BRANCH_NAME=$(echo ${GITHUB_HEAD_REF} | tr / -)" >> $GITHUB_ENV
- name: Upload macro benchmark results
run: |
cd tools/bench-graph-client
@@ -384,7 +431,8 @@ jobs:
./main.py --benchmark-name "macro_benchmark" \
--benchmark-results-path "../../tests/macro_benchmark/.harness_summary" \
--github-run-id "${{ github.run_id }}" \
--github-run-number "${{ github.run_number }}"
--github-run-number "${{ github.run_number }}" \
--head-branch-name "${{ env.BRANCH_NAME }}"
- name: Run mgbench
run: |
@@ -400,4 +448,5 @@ jobs:
./main.py --benchmark-name "mgbench" \
--benchmark-results-path "../../tests/mgbench/benchmark_result.json" \
--github-run-id "${{ github.run_id }}" \
--github-run-number "${{ github.run_number }}"
--github-run-number "${{ github.run_number }}" \
--head-branch-name "${{ env.BRANCH_NAME }}"

View File

@@ -9,10 +9,12 @@ jobs:
runs-on: [self-hosted, Linux, X64, Ubuntu20.04]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -21,7 +23,7 @@ jobs:
- name: Build debug binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -34,10 +36,10 @@ jobs:
- name: Run clang-tidy
run: |
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# The results are also written to standard output in order to retain them in the logs
./tools/github/clang-tidy/run-clang-tidy.py -p build -j $THREADS -clang-tidy-binary=/opt/toolchain-v2/bin/clang-tidy "$PWD/src/*" |
./tools/github/clang-tidy/run-clang-tidy.py -p build -j $THREADS -clang-tidy-binary=/opt/toolchain-v4/bin/clang-tidy "$PWD/src/*" |
tee ./build/full_clang_tidy_output.txt
- name: Summarize clang-tidy results

View File

@@ -5,244 +5,208 @@ name: Package All
on: workflow_dispatch
jobs:
centos-7_community:
runs-on: [self-hosted, DockerMgBuild]
centos-7:
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package community centos-7
./release/package/run.sh package centos-7
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: centos-7_community
name: centos-7
path: build/output/centos-7/memgraph*.rpm
centos-8_community:
runs-on: [self-hosted, DockerMgBuild]
centos-9:
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package community centos-8
./release/package/run.sh package centos-9
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: centos-8_community
path: build/output/centos-8/memgraph*.rpm
name: centos-9
path: build/output/centos-9/memgraph*.rpm
debian-9_community:
runs-on: [self-hosted, DockerMgBuild]
debian-10:
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package community debian-9
./release/package/run.sh package debian-10
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: debian-9_community
path: build/output/debian-9/memgraph*.deb
debian-10_community:
runs-on: [self-hosted, DockerMgBuild]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v2
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package community debian-10
- name: "Upload package"
uses: actions/upload-artifact@v2
with:
name: debian-10_community
name: debian-10
path: build/output/debian-10/memgraph*.deb
docker_community:
runs-on: [self-hosted, DockerMgBuild]
debian-11:
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package debian-11
- name: "Upload package"
uses: actions/upload-artifact@v3
with:
name: debian-11
path: build/output/debian-11/memgraph*.deb
docker:
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
cd release/package
./run.sh package community debian-10 --for-docker
./run.sh package debian-11 --for-docker
./run.sh docker
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: docker_community
name: docker
path: build/output/docker/memgraph*.tar.gz
ubuntu-1804_community:
runs-on: [self-hosted, DockerMgBuild]
ubuntu-1804:
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package community ubuntu-18.04
./release/package/run.sh package ubuntu-18.04
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: ubuntu-1804_community
name: ubuntu-1804
path: build/output/ubuntu-18.04/memgraph*.deb
ubuntu-2004_community:
runs-on: [self-hosted, DockerMgBuild]
ubuntu-2004:
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package community ubuntu-20.04
./release/package/run.sh package ubuntu-20.04
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: ubuntu-2004_community
name: ubuntu-2004
path: build/output/ubuntu-20.04/memgraph*.deb
centos-7_enterprise:
runs-on: [self-hosted, DockerMgBuild]
ubuntu-2204:
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package enterprise centos-7
./release/package/run.sh package ubuntu-22.04
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: centos-7_enterprise
path: build/output/centos-7/memgraph*.rpm
name: ubuntu-2204
path: build/output/ubuntu-22.04/memgraph*.deb
centos-8_enterprise:
runs-on: [self-hosted, DockerMgBuild]
debian-11-platform:
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package enterprise centos-8
./release/package/run.sh package debian-11 --for-platform
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: centos-8_enterprise
path: build/output/centos-8/memgraph*.rpm
name: debian-11-platform
path: build/output/debian-11/memgraph*.deb
debian-9_enterprise:
runs-on: [self-hosted, DockerMgBuild]
fedora-36:
runs-on: [self-hosted, DockerMgBuild, X64]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package enterprise debian-9
./release/package/run.sh package fedora-36
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: debian-9_enterprise
path: build/output/debian-9/memgraph*.deb
name: fedora-36
path: build/output/fedora-36/memgraph*.rpm
debian-10_enterprise:
runs-on: [self-hosted, DockerMgBuild]
debian-11-arm:
runs-on: [self-hosted, DockerMgBuild, ARM64, strange]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package enterprise debian-10
./release/package/run.sh package debian-11-arm
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: debian-10_enterprise
path: build/output/debian-10/memgraph*.deb
name: debian-11-arm
path: build/output/debian-11-arm/memgraph*.deb
docker_enterprise:
runs-on: [self-hosted, DockerMgBuild]
ubuntu-2204-arm:
runs-on: [self-hosted, DockerMgBuild, ARM64, strange]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
cd release/package
./run.sh package enterprise debian-10 --for-docker
./run.sh docker
./release/package/run.sh package ubuntu-22.04-arm
- name: "Upload package"
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: docker_enterprise
path: build/output/docker/memgraph*.tar.gz
ubuntu-1804_enterprise:
runs-on: [self-hosted, DockerMgBuild]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v2
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package enterprise ubuntu-18.04
- name: "Upload package"
uses: actions/upload-artifact@v2
with:
name: ubuntu-1804_enterprise
path: build/output/ubuntu-18.04/memgraph*.deb
ubuntu-2004_enterprise:
runs-on: [self-hosted, DockerMgBuild]
timeout-minutes: 60
steps:
- name: "Set up repository"
uses: actions/checkout@v2
with:
fetch-depth: 0 # Required because of release/get_version.py
- name: "Build package"
run: |
./release/package/run.sh package enterprise ubuntu-20.04
- name: "Upload package"
uses: actions/upload-artifact@v2
with:
name: ubuntu-2004_enterprise
path: build/output/ubuntu-20.04/memgraph*.deb
name: ubuntu-22.04-arm
path: build/output/ubuntu-22.04-arm/memgraph*.deb

View File

@@ -3,7 +3,7 @@ name: Release CentOS 8
on:
workflow_dispatch:
schedule:
- cron: "0 1 * * *"
- cron: "0 22 * * *"
jobs:
community_build:
@@ -11,11 +11,13 @@ jobs:
runs-on: [self-hosted, Linux, X64, CentOS8]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 960
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -24,7 +26,7 @@ jobs:
- name: Build community binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -34,51 +36,26 @@ jobs:
cmake -DCMAKE_BUILD_TYPE=release -DMG_ENTERPRISE=OFF ..
make -j$THREADS
- name: Create community RPM package
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
# Create community RPM package.
cd build
mkdir output && cd output
cpack -G RPM --config ../CPackConfig.cmake
rpmlint memgraph*.rpm
- name: Save community RPM package
uses: actions/upload-artifact@v2
with:
name: "Community RPM package"
path: build/output/memgraph*.rpm
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
ctest -R memgraph__unit --output-on-failure
- name: Run stress test (plain)
run: |
cd tests/stress
./continuous_integration
- name: Run stress test (large)
run: |
cd tests/stress
./continuous_integration --large-dataset
coverage_build:
name: "Coverage build"
runs-on: [self-hosted, Linux, X64, CentOS8]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -87,7 +64,7 @@ jobs:
- name: Build coverage binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -100,7 +77,7 @@ jobs:
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
@@ -109,7 +86,7 @@ jobs:
- name: Compute code coverage
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Compute code coverage.
cd tools/github
@@ -120,7 +97,7 @@ jobs:
tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu
- name: Save code coverage
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/generated/code_coverage.tar.gz
@@ -130,10 +107,12 @@ jobs:
runs-on: [self-hosted, Linux, X64, CentOS8]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -142,7 +121,7 @@ jobs:
- name: Build debug binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -155,7 +134,7 @@ jobs:
- name: Run leftover CTest tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run leftover CTest tests (all except unit and benchmark tests).
cd build
@@ -187,14 +166,14 @@ jobs:
- name: Run cppcheck and clang-format
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run cppcheck and clang-format.
cd tools/github
./cppcheck_and_clang_format diff
- name: Save cppcheck and clang-format errors
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/cppcheck_and_clang_format.txt
@@ -204,11 +183,13 @@ jobs:
runs-on: [self-hosted, Linux, X64, CentOS8]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 960
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -217,7 +198,7 @@ jobs:
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -230,16 +211,21 @@ jobs:
- name: Create enterprise RPM package
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
cd build
# create mgconsole
# we use the -B to force the build
make -j$THREADS -B mgconsole
# Create enterprise RPM package.
cd build
mkdir output && cd output
cpack -G RPM --config ../CPackConfig.cmake
rpmlint memgraph*.rpm
- name: Save enterprise RPM package
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Enterprise RPM package"
path: build/output/memgraph*.rpm
@@ -247,7 +233,7 @@ jobs:
- name: Run micro benchmark tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run micro benchmark tests.
cd build
@@ -276,29 +262,30 @@ jobs:
./continuous_integration
- name: Save quality assurance status
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "GQL Behave Status"
path: |
tests/gql_behave/gql_behave_status.csv
tests/gql_behave/gql_behave_status.html
- name: Run e2e replication tests
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
ctest -R memgraph__unit --output-on-failure
- name: Run e2e tests
run: |
# TODO(gitbuda): Setup mgclient and pymgclient properly.
cd tests
./setup.sh
source ve3/bin/activate
cd e2e
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../../libs/mgclient/lib python runner.py --workloads-path replication/workloads.yaml
- name: Run e2e memory control tests
run: |
cd tests
./setup.sh
source ve3/bin/activate
cd e2e
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../../libs/mgclient/lib python runner.py --workloads-path memory/workloads.yaml
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../../libs/mgclient/lib python runner.py --workloads-root-directory .
- name: Run stress test (plain)
run: |

View File

@@ -3,7 +3,7 @@ name: Release Debian 10
on:
workflow_dispatch:
schedule:
- cron: "0 1 * * *"
- cron: "0 22 * * *"
jobs:
community_build:
@@ -11,11 +11,13 @@ jobs:
runs-on: [self-hosted, Linux, X64, Debian10]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 960
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -24,7 +26,7 @@ jobs:
- name: Build community binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -34,50 +36,26 @@ jobs:
cmake -DCMAKE_BUILD_TYPE=release -DMG_ENTERPRISE=OFF ..
make -j$THREADS
- name: Create community DEB package
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
# Create community DEB package.
cd build
mkdir output && cd output
cpack -G DEB --config ../CPackConfig.cmake
- name: Save community DEB package
uses: actions/upload-artifact@v2
with:
name: "Community DEB package"
path: build/output/memgraph*.deb
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
ctest -R memgraph__unit --output-on-failure
- name: Run stress test (plain)
run: |
cd tests/stress
./continuous_integration
- name: Run stress test (large)
run: |
cd tests/stress
./continuous_integration --large-dataset
coverage_build:
name: "Coverage build"
runs-on: [self-hosted, Linux, X64, Debian10]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -86,7 +64,7 @@ jobs:
- name: Build coverage binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -99,7 +77,7 @@ jobs:
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
@@ -108,7 +86,7 @@ jobs:
- name: Compute code coverage
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Compute code coverage.
cd tools/github
@@ -119,7 +97,7 @@ jobs:
tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu
- name: Save code coverage
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/generated/code_coverage.tar.gz
@@ -129,10 +107,12 @@ jobs:
runs-on: [self-hosted, Linux, X64, Debian10]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -141,7 +121,7 @@ jobs:
- name: Build debug binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -154,7 +134,7 @@ jobs:
- name: Run leftover CTest tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run leftover CTest tests (all except unit and benchmark tests).
cd build
@@ -186,14 +166,14 @@ jobs:
- name: Run cppcheck and clang-format
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run cppcheck and clang-format.
cd tools/github
./cppcheck_and_clang_format diff
- name: Save cppcheck and clang-format errors
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/cppcheck_and_clang_format.txt
@@ -203,11 +183,13 @@ jobs:
runs-on: [self-hosted, Linux, X64, Debian10]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 960
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -216,7 +198,7 @@ jobs:
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -229,15 +211,20 @@ jobs:
- name: Create enterprise DEB package
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
cd build
# create mgconsole
# we use the -B to force the build
make -j$THREADS -B mgconsole
# Create enterprise DEB package.
cd build
mkdir output && cd output
cpack -G DEB --config ../CPackConfig.cmake
- name: Save enterprise DEB package
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Enterprise DEB package"
path: build/output/memgraph*.deb
@@ -245,7 +232,7 @@ jobs:
- name: Run micro benchmark tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run micro benchmark tests.
cd build
@@ -274,29 +261,30 @@ jobs:
./continuous_integration
- name: Save quality assurance status
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "GQL Behave Status"
path: |
tests/gql_behave/gql_behave_status.csv
tests/gql_behave/gql_behave_status.html
- name: Run e2e replication tests
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
ctest -R memgraph__unit --output-on-failure
- name: Run e2e tests
run: |
# TODO(gitbuda): Setup mgclient and pymgclient properly.
cd tests
./setup.sh
source ve3/bin/activate
cd e2e
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../../libs/mgclient/lib python runner.py --workloads-path replication/workloads.yaml
- name: Run e2e memory control tests
run: |
cd tests
./setup.sh
source ve3/bin/activate
cd e2e
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../../libs/mgclient/lib python runner.py --workloads-path memory/workloads.yaml
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../../libs/mgclient/lib python runner.py --workloads-root-directory .
- name: Run stress test (plain)
run: |
@@ -330,11 +318,13 @@ jobs:
runs-on: [self-hosted, Linux, X64, Debian10, JepsenControl]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 60
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -343,7 +333,7 @@ jobs:
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -359,7 +349,7 @@ jobs:
./run.sh test --binary ../../build/memgraph --run-args "test-all --node-configs resources/node-config.edn" --ignore-run-stdout-logs --ignore-run-stderr-logs
- name: Save Jepsen report
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
if: ${{ always() }}
with:
name: "Jepsen Report"

69
.github/workflows/release_docker.yaml vendored Normal file
View File

@@ -0,0 +1,69 @@
name: Publish Docker images
on:
workflow_dispatch:
inputs:
version:
description: "Memgraph binary version to publish on DockerHub."
required: true
force_release:
type: boolean
required: false
default: false
jobs:
docker_publish:
runs-on: ubuntu-latest
env:
DOCKER_ORGANIZATION_NAME: memgraph
DOCKER_REPOSITORY_NAME: memgraph
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v2
- name: Log in to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Download memgraph binary
run: |
cd release/docker
curl -L https://download.memgraph.com/memgraph/v${{ github.event.inputs.version }}/debian-11/memgraph_${{ github.event.inputs.version }}-1_amd64.deb > memgraph-amd64.deb
curl -L https://download.memgraph.com/memgraph/v${{ github.event.inputs.version }}/debian-11-aarch64/memgraph_${{ github.event.inputs.version }}-1_arm64.deb > memgraph-arm64.deb
- name: Check if specified version is already pushed
run: |
EXISTS=$(docker manifest inspect $DOCKER_ORGANIZATION_NAME/$DOCKER_REPOSITORY_NAME:${{ github.event.inputs.version }} > /dev/null; echo $?)
echo $EXISTS
if [[ ${EXISTS} -eq 0 ]]; then
echo 'The specified version has been already released to DockerHub.'
if [[ ${{ github.event.inputs.force_release }} = true ]]; then
echo 'Forcing the release!'
else
echo 'Stopping the release!'
exit 1
fi
else
echo 'All good the specified version has not been release to DockerHub.'
fi
- name: Build & push docker images
run: |
cd release/docker
docker buildx build \
--build-arg BINARY_NAME="memgraph-" \
--build-arg EXTENSION="deb" \
--platform linux/amd64,linux/arm64 \
--tag $DOCKER_ORGANIZATION_NAME/$DOCKER_REPOSITORY_NAME:${{ github.event.inputs.version }} \
--tag $DOCKER_ORGANIZATION_NAME/$DOCKER_REPOSITORY_NAME:latest \
--file memgraph_deb.dockerfile \
--push .

View File

@@ -3,7 +3,7 @@ name: Release Ubuntu 20.04
on:
workflow_dispatch:
schedule:
- cron: "0 1 * * *"
- cron: "0 22 * * *"
jobs:
community_build:
@@ -11,11 +11,13 @@ jobs:
runs-on: [self-hosted, Linux, X64, Ubuntu20.04]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 960
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -24,7 +26,7 @@ jobs:
- name: Build community binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -34,50 +36,26 @@ jobs:
cmake -DCMAKE_BUILD_TYPE=release -DMG_ENTERPRISE=OFF ..
make -j$THREADS
- name: Create community DEB package
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
# Create community DEB package.
cd build
mkdir output && cd output
cpack -G DEB --config ../CPackConfig.cmake
- name: Save community DEB package
uses: actions/upload-artifact@v2
with:
name: "Community DEB package"
path: build/output/memgraph*.deb
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
ctest -R memgraph__unit --output-on-failure
- name: Run stress test (plain)
run: |
cd tests/stress
./continuous_integration
- name: Run stress test (large)
run: |
cd tests/stress
./continuous_integration --large-dataset
coverage_build:
name: "Coverage build"
runs-on: [self-hosted, Linux, X64, Ubuntu20.04]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -86,7 +64,7 @@ jobs:
- name: Build coverage binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -99,7 +77,7 @@ jobs:
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
@@ -108,7 +86,7 @@ jobs:
- name: Compute code coverage
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Compute code coverage.
cd tools/github
@@ -119,7 +97,7 @@ jobs:
tar -czf code_coverage.tar.gz coverage.json html report.json summary.rmu
- name: Save code coverage
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/generated/code_coverage.tar.gz
@@ -129,10 +107,12 @@ jobs:
runs-on: [self-hosted, Linux, X64, Ubuntu20.04]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -141,7 +121,7 @@ jobs:
- name: Build debug binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -154,7 +134,7 @@ jobs:
- name: Run leftover CTest tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run leftover CTest tests (all except unit and benchmark tests).
cd build
@@ -186,14 +166,14 @@ jobs:
- name: Run cppcheck and clang-format
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run cppcheck and clang-format.
cd tools/github
./cppcheck_and_clang_format diff
- name: Save cppcheck and clang-format errors
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Code coverage"
path: tools/github/cppcheck_and_clang_format.txt
@@ -203,11 +183,13 @@ jobs:
runs-on: [self-hosted, Linux, X64, Ubuntu20.04]
env:
THREADS: 24
MEMGRAPH_ENTERPRISE_LICENSE: ${{ secrets.MEMGRAPH_ENTERPRISE_LICENSE }}
MEMGRAPH_ORGANIZATION_NAME: ${{ secrets.MEMGRAPH_ORGANIZATION_NAME }}
timeout-minutes: 960
steps:
- name: Set up repository
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
# Number of commits to fetch. `0` indicates all history for all
# branches and tags. (default: 1)
@@ -216,7 +198,7 @@ jobs:
- name: Build release binaries
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Initialize dependencies.
./init
@@ -229,15 +211,20 @@ jobs:
- name: Create enterprise DEB package
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
cd build
# create mgconsole
# we use the -B to force the build
make -j$THREADS -B mgconsole
# Create enterprise DEB package.
cd build
mkdir output && cd output
cpack -G DEB --config ../CPackConfig.cmake
- name: Save enterprise DEB package
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "Enterprise DEB package"
path: build/output/memgraph*.deb
@@ -245,7 +232,7 @@ jobs:
- name: Run micro benchmark tests
run: |
# Activate toolchain.
source /opt/toolchain-v2/activate
source /opt/toolchain-v4/activate
# Run micro benchmark tests.
cd build
@@ -274,29 +261,30 @@ jobs:
./continuous_integration
- name: Save quality assurance status
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: "GQL Behave Status"
path: |
tests/gql_behave/gql_behave_status.csv
tests/gql_behave/gql_behave_status.html
- name: Run e2e replication tests
- name: Run unit tests
run: |
# Activate toolchain.
source /opt/toolchain-v4/activate
# Run unit tests.
cd build
ctest -R memgraph__unit --output-on-failure
- name: Run e2e tests
run: |
# TODO(gitbuda): Setup mgclient and pymgclient properly.
cd tests
./setup.sh
source ve3/bin/activate
cd e2e
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../../libs/mgclient/lib python runner.py --workloads-path replication/workloads.yaml
- name: Run e2e memory control tests
run: |
cd tests
./setup.sh
source ve3/bin/activate
cd e2e
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../../libs/mgclient/lib python runner.py --workloads-path memory/workloads.yaml
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../../libs/mgclient/lib python runner.py --workloads-root-directory .
- name: Run stress test (plain)
run: |

20
.pre-commit-config.yaml Normal file
View File

@@ -0,0 +1,20 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.3.0
hooks:
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/psf/black
rev: 22.8.0
hooks:
- id: black
- repo: https://github.com/pycqa/isort
rev: 5.10.1
hooks:
- id: isort
name: isort (python)
- repo: https://github.com/pre-commit/mirrors-clang-format
rev: v13.0.0
hooks:
- id: clang-format

View File

@@ -1,491 +1,5 @@
# Change Log
Change Log for all versions of Memgraph can be found on-line at
https://docs.memgraph.com/memgraph/changelog
## Future
### Bug Fixes
* Fixed parsing of types for Python procedures for types nested in `mgp.List`.
For example, parsing of `mgp.List[mgp.Map]` works now.
* Fixed memory tracking issues. Some of the allocation and deallocation weren't
tracked during the query execution.
* Fixed reading CSV files that are using CRLF as the newline symbol.
* Fixed permission issues for `LOAD CSV`, `FREE MEMORY`, `LOCK DATA DIRECTORY`,
and replication queries.
## v1.4.0
### Breaking Changes
* Changed `MEMORY LIMIT num (KB|MB)` clause in the procedure calls to `PROCEDURE MEMORY LIMIT num (KB|MB)`.
The functionality is still the same.
### Major Feature and Improvements
* Added replication to community version.
* Added support for multiple query modules directories at the same time.
You can now define multiple, comma-separated paths to directories from
which the modules will be loaded using the `--query-modules-directory` flag.
* Added support for programatically reading in data from CSV files through the
`LOAD CSV` clause. We support CSV files with and without a header, the
supported dialect being Excel.
* Added a new flag `--memory-limit` which enables the user to set the maximum total amount of memory
memgraph can allocate during its runtime.
* Added `FREE MEMORY` query which tries to free unusued memory chunks in different parts of storage.
* Added the memory limit and amount of currently allocated bytes in the result of `SHOW STORAGE INFO` query.
* Added `QUERY MEMORY LIMIT num (KB|MB)` to Cypher queries which allows you to limit memory allocation for
the entire query. It can be added only at the end of the entire Cypher query.
* Added logs for the different parts of the recovery process. `INFO`, `DEBUG` and `TRACE` level all contain
additional information that is printed out while the recovery is in progress.
### Bug Fixes
* Fixed garbage collector by correctly marking the oldest current timestamp
after the database was recovered using the durability files.
* Fixed reloading of the modules with changed result names.
* Fixed profile query to show the correct name of the ScanAll operator variant.
## v1.3.0
### Breaking Changes
* Added extra information in durability files to support replication, making it
incompatible with the durability files generated by older versions of
Memgraph. Even though the replication is an Enterprise feature, the files are
compatible with the Community version.
### Major Features and Improvements
* Added support for data replication across a cluster of Memgraph instances.
Supported instance types are MAIN and REPLICA. Supported replication modes
are SYNC (all SYNC REPLICAS have to receive data before the MAIN can commit
the transaction), ASYNC (MAIN doesn't care if data is replicated), SYNC WITH
TIMEOUT (MAIN will wait for REPLICAS within the given timeout period, after
timout, replication isn't aborted but the replication demotes the REPLICA to
the ASYNC mode).
* Added support for query type deduction. Possible query types are `r` (read),
`w` (write), `rw` (read-write). The query type is returned as a part of the
summary.
* Improved logging capabilities by introducing granular logging levels. Added
new flag, `--log-level`, which specifies the minimum log level that will be
printed. E.g., it's possible to print incoming queries or Bolt server states.
* Added ability to lock the storage data directory by executing the `LOCK DATA
DIRECTORY;` query which delays the deletion of the files contained in the
data directory. The data directory can be unlocked again by executing the
`UNLOCK DATA DIRECTORY;` query.
### Bug Fixes and Other Changes
* Added cleanup of query executions if not in an explicit transaction.
* Fix RPC dangling reference.
## v1.2.0
### Breaking Changes
* SSL is disabled by default (`--bolt-cert-file` and `--bolt-key-file` are
empty). This change might only affect the client connection configuration.
### Major Features and Improvements
* Added support for Bolt v4.0 and v4.1.
* Added `mgp_networkx.py` as an alternative implementation of NetworkX graph
objects, which is useful to use Memgraph data from NetworkX algorithms
optimally.
* Added `nxalg.py` query module as a proxy to NetworkX algorithms.
* Added plan optimization to use a label-property index where the property is
not null. As a result, the query engine, instead of scanning all elements and
applying the filter, performs a label-property index lookup when possible.
### Bug Fixes and Other Changes
* Fixed Cypher `ID` function `Null` handling. When the `ID` function receives
`Null`, it will also return `Null`.
* Fixed bug that caused random crashes in SSL communication on platforms
that use older versions of OpenSSL (< 1.1) by adding proper multi-threading
handling.
* Fix `DISCARD` message handling. The query is now executed before discarding
the results.
## v1.1.0
### Major Features and Improvements
* Properties in nodes and edges are now stored encoded and compressed. This
change significantly reduces memory usage. Depending on the specific dataset,
total memory usage can be reduced up to 50%.
* Added support for rescanning query modules. Previously, the query modules
directory was scanned only upon startup. Now it is scanned each time the user
requests to load a query module. The functions used to load the query modules
were renamed to `mg.load()` and `mg.load_all()` (from `mg.reload()` and
`mg.reload_all()`).
* Improved execution performance of queries that have an IN list filter by
using label+property indices.
Example: `MATCH (n:Label) WHERE n.property IN [] ...`
* Added support for `ANY` and `NONE` openCypher functions. Previously, only
`ALL` and `SINGLE` functions were implemented.
### Bug Fixes and Other Changes
* Fixed invalid paths returned by variable expansion when the starting node and
destination node used the same symbol.
Example: `MATCH path = (n:Person {name: "John"})-[:KNOWS*]->(n) RETURN path`
* Improved semantics of `ALL` and `SINGLE` functions to be consistent with
openCypher when handling lists with `Null`s.
* `SHOW CONSTRAINT INFO` now returns property names as a list for unique
constraints.
* Escaped label/property/edgetype names in `DUMP DATABASE` to support names
with spaces in them.
* Fixed handling of `DUMP DATABASE` queries in multi-command transactions
(`BEGIN`, ..., `COMMIT`).
* Fixed handling of various query types in explicit transactions. For example,
constraints were allowed to be created in multi-command transactions
(`BEGIN`, ..., `COMMIT`) but that isn't a transactional operation and as such
can't be allowed in multi-command transactions.
* Fixed integer overflow bugs in `COUNT`, `LIMIT` and `SKIP`.
* Fixed integer overflow bugs in weighted shortest path expansions.
* Fixed various other integer overflow bugs in query execution.
* Added Marvel Comic Universe tutorial.
* Added FootballTransfers tutorial.
## v1.0.0
### Major Features and Improvements
* [Enterprise Ed.] Exposed authentication username/rolename regex as a flag
(`--auth-user-or-role-name-regex`).
* [Enterprise Ed.] Improved auth module error handling and added support for
relative paths.
* Added support for Python query modules. This release of Memgraph supports
query modules written using the already existing C API and the new Python
API.
* Added support for unique constraints. The unique constraint is created with a
label and one or more properties.
* Implemented support for importing CSV files (`mg_import_csv`). The importer
is compatible with the Neo4j batch CSV importer.
* Snapshot and write-ahead log format changed (backward compatible with v0.50).
* Vertices looked up by their openCypher ID (`MATCH (n) WHERE ID(n) = ...`)
will now find the node in O(logn) instead of O(n).
* Improved planning of BFS expansion, a faster, specific approach is now
favored instead of a ScanAll+Filter operation.
* Added syntax for limiting memory of `CALL`.
* Exposed server name that should be used for Bolt handshake as flag
(`--bolt-server-name-for-init`).
* Added several more functions to the query module C API.
* Implemented a storage locking mechanism that prevents the user from
concurrently starting two Memgraph instances with the same data directory.
### Bug Fixes and Other Changes
* [Enterprise Ed.] Fixed a bug that crashed the database when granting
privileges to a user.
* [Enterprise Ed.] Improved Louvain algorithm for community detection.
* Type of variable expansion is now printed in `EXPLAIN` (e.g. ExpandVariable,
STShortestPath, BFSExpand, WeightedShortestPath).
* Correctly display `CALL` in `EXPLAIN` output.
* Correctly delimit arguments when printing the signature of a query module.
* Fixed a planning issue when `CALL` preceded filtering.
* Fixed spelling mistakes in the storage durability module.
* Fixed storage GC indices/constraints subtle race condition.
* Reduced memory allocations in storage API and indices.
* Memgraph version is now outputted to `stdout` when Memgraph is started.
* Improved RPM packaging.
* Reduced number of errors reported in production log when loading query
modules.
* Removed `early access` wording from the Community Offering license.
## v0.50.0
### Breaking Changes
* [Enterprise Ed.] Remove support for Kafka streams.
* Snapshot and write-ahead log format changed (not backward compatible).
* Removed support for unique constraints.
* Label indices aren't created automatically, create them explicitly instead.
* Renamed several database flags. Please see the configuration file for a list of current flags.
### Major Features and Improvements
* [Enterprise Ed.] Add support for auth module.
* [Enterprise Ed.] LDAP support migrated to auth module.
* Implemented new graph storage engine.
* Add support for disabling properties on edges.
* Add support for existence constraints.
* Add support for custom openCypher procedures using a C API.
* Support loading query modules implementing read-only procedures.
* Add `CALL <procedure> YIELD <result>` syntax for invoking loaded procedures.
* Add `CREATE INDEX ON :Label` for creating label indices.
* Add `DROP INDEX ON :Label` for dropping label indices.
* Add `DUMP DATABASE` clause to openCypher.
* Add functions for treating character strings as byte strings.
### Bug Fixes and Other Changes
* Fix several memory management bugs.
* Reduce memory usage in query execution.
* Fix bug that crashes the database when `EXPLAIN` is used.
## v0.15.0
### Breaking Changes
* Snapshot and write-ahead log format changed (not backward compatible).
* `indexInfo()` function replaced with `SHOW INDEX INFO` syntax.
* Removed support for unique index. Use unique constraints instead.
* `CREATE UNIQUE INDEX ON :label (property)` replaced with `CREATE CONSTRAINT ON (n:label) ASSERT n.property IS UNIQUE`.
* Changed semantics for `COUNTER` openCypher function.
### Major Features and Improvements
* [Enterprise Ed.] Add new privilege, `STATS` for accessing storage info.
* [Enterprise Ed.] LDAP authentication and authorization support.
* [Enterprise Ed.] Add audit logging feature.
* Add multiple properties unique constraint which replace unique indices.
* Add `SHOW STORAGE INFO` feature.
* Add `PROFILE` clause to openCypher.
* Add `CREATE CONSTRAINT` clause to openCypher.
* Add `DROP CONSTRAINT` clause to openCypher.
* Add `SHOW CONSTRAINT INFO` feature.
* Add `uniformSample` function to openCypher.
* Add regex matching to openCypher.
### Bug Fixes and Other Changes
* Fix bug in explicit transaction handling.
* Fix bug in edge filtering by edge type and destination.
* Fix bug in query comment parsing.
* Fix bug in query symbol table.
* Fix OpenSSL memory leaks.
* Make authentication case insensitive.
* Remove `COALESCE` function.
* Add movie tutorial.
* Add backpacking tutorial.
## v0.14.0
### Breaking Changes
* Write-ahead log format changed (not backward compatible).
### Major Features and Improvements
* [Enterprise Ed.] Reduce memory usage in distributed usage.
* Add `DROP INDEX` feature.
* Improve SSL error messages.
### Bug Fixes and Other Changes
* [Enterprise Ed.] Fix issues with reading and writing in a distributed query.
* Correctly handle an edge case with unique constraint checks.
* Fix a minor issue with `mg_import_csv`.
* Fix an issue with `EXPLAIN`.
## v0.13.0
### Breaking Changes
* Write-ahead log format changed (not backward compatible).
* Snapshot format changed (not backward compatible).
### Major Features and Improvements
* [Enterprise Ed.] Authentication and authorization support.
* [Enterprise Ed.] Kafka integration.
* [Enterprise Ed.] Support dynamic worker addition in distributed.
* Reduce memory usage and improve overall performance.
* Add `CREATE UNIQUE INDEX` clause to openCypher.
* Add `EXPLAIN` clause to openCypher.
* Add `inDegree` and `outDegree` functions to openCypher.
* Improve BFS performance when both endpoints are known.
* Add new `node-label`, `relationship-type` and `quote` options to
`mg_import_csv` tool.
* Reduce memory usage of `mg_import_csv`.
### Bug Fixes and Other Changes
* [Enterprise Ed.] Fix an edge case in distributed index creation.
* [Enterprise Ed.] Fix issues with Cartesian in distributed queries.
* Correctly handle large messages in Bolt protocol.
* Fix issues when handling explicitly started transactions in queries.
* Allow openCypher keywords to be used as variable names.
* Revise and make user visible error messages consistent.
* Improve aborting time consuming execution.
## v0.12.0
### Breaking Changes
* Snapshot format changed (not backward compatible).
### Major Features and Improvements
* Improved Id Cypher function.
* Added string functions to openCypher (`lTrim`, `left`, `rTrim`, `replace`,
`reverse`, `right`, `split`, `substring`, `toLower`, `toUpper`, `trim`).
* Added `timestamp` function to openCypher.
* Added support for dynamic property access with `[]` operator.
## v0.11.0
### Major Features and Improvements
* [Enterprise Ed.] Improve Cartesian support in distributed queries.
* [Enterprise Ed.] Improve distributed execution of BFS.
* [Enterprise Ed.] Dynamic graph partitioner added.
* Static nodes/edges id generators exposed through the Id Cypher function.
* Properties on disk added.
* Telemetry added.
* SSL support added.
* `toString` function added.
### Bug Fixes and Other Changes
* Document issues with Docker on OS X.
* Add BFS and Dijkstra's algorithm examples to documentation.
## v0.10.0
### Breaking Changes
* Snapshot format changed (not backward compatible).
### Major Features and Improvements
* [Enterprise Ed.] Distributed storage and execution.
* `reduce` and `single` functions added to openCypher.
* `wShortest` edge expansion added to openCypher.
* Support packaging RPM on CentOS 7.
### Bug Fixes and Other Changes
* Report an error if updating a deleted element.
* Log an error if reading info on available memory fails.
* Fix a bug when `MATCH` would stop matching if a result was empty, but later
results still contain data to be matched. The simplest case of this was the
query: `UNWIND [1,2,3] AS x MATCH (n :Label {prop: x}) RETURN n`. If there
was no node `(:Label {prop: 1})`, then the `MATCH` wouldn't even try to find
for `x` being 2 or 3.
* Report an error if trying to compare a property value with something that
cannot be stored in a property.
* Fix crashes in some obscure cases.
* Commit log automatically garbage collected.
* Add minor performance improvements.
## v0.9.0
### Breaking Changes
* Snapshot format changed (not backward compatible).
* Snapshot configuration flags changed, general durability flags added.
### Major Features and Improvements
* Write-ahead log added.
* `nodes` and `relationships` functions added.
* `UNION` and `UNION ALL` is implemented.
* Concurrent index creation is now enabled.
### Bug Fixes and Other Changes
## v0.8.0
### Major Features and Improvements
* CASE construct (without aggregations).
* Named path support added.
* Maps can now be stored as node/edge properties.
* Map indexing supported.
* `rand` function added.
* `assert` function added.
* `counter` and `counterSet` functions added.
* `indexInfo` function added.
* `collect` aggregation now supports Map collection.
* Changed the BFS syntax.
### Bug Fixes and Other Changes
* Use \u to specify 4 digit codepoint and \U for 8 digit
* Keywords appearing in header (named expressions) keep original case.
* Our Bolt protocol implementation is now completely compatible with the protocol version 1 specification. (https://boltprotocol.org/v1/)
* Added a log warning when running out of memory and the `memory_warning_threshold` flag
* Edges are no longer additionally filtered after expansion.
## v0.7.0
### Major Features and Improvements
* Variable length path `MATCH`.
* Explicitly started transactions (multi-query transactions).
* Map literal.
* Query parameters (except for parameters in place of property maps).
* `all` function in openCypher.
* `degree` function in openCypher.
* User specified transaction execution timeout.
### Bug Fixes and Other Changes
* Concurrent `BUILD INDEX` deadlock now returns an error to the client.
* A `MATCH` preceeded by `OPTIONAL MATCH` expansion inconsistencies.
* High concurrency Antlr parsing bug.
* Indexing improvements.
* Query stripping and caching speedups.
## v0.6.0
### Major Features and Improvements
* AST caching.
* Label + property index support.
* Different logging setup & format.
## v0.5.0
### Major Features and Improvements
* Use label indexes to speed up querying.
* Generate multiple query plans and use the cost estimator to select the best.
* Snapshots & Recovery.
* Abandon old yaml configuration and migrate to gflags.
* Query stripping & AST caching support.
### Bug Fixes and Other Changes
* Fixed race condition in MVCC. Hints exp+aborted race condition prevented.
* Fixed conceptual bug in MVCC GC. Evaluate old records w.r.t. the oldest.
transaction's id AND snapshot.
* User friendly error messages thrown from the query engine.
## Build 837
### Bug Fixes and Other Changes
* List indexing supported with preceeding IN (for example in query `RETURN 1 IN [[1,2]][0]`).
## Build 825
### Major Features and Improvements
* RETURN *, count(*), OPTIONAL MATCH, UNWIND, DISTINCT (except DISTINCT in aggregate functions), list indexing and slicing, escaped labels, IN LIST operator, range function.
### Bug Fixes and Other Changes
* TCP_NODELAY -> import should be faster.
* Clear hint bits.
## Build 783
### Major Features and Improvements
* SKIP, LIMIT, ORDER BY.
* Math functions.
* Initial support for MERGE clause.
### Bug Fixes and Other Changes
* Unhandled Lock Timeout Exception.
## Build 755
### Major Features and Improvements
* MATCH, CREATE, WHERE, SET, REMOVE, DELETE.
All the updates to the Change Log can be made in the following repository:
https://github.com/memgraph/docs

View File

@@ -1,338 +0,0 @@
# MemGraph CMake configuration
cmake_minimum_required(VERSION 3.8)
# !! IMPORTANT !! run ./project_root/init.sh before cmake command
# to download dependencies
if(NOT UNIX)
message(FATAL_ERROR "Unsupported operating system.")
endif()
# Set `make clean` to ignore outputs of add_custom_command. If generated files
# need to be cleaned, set ADDITIONAL_MAKE_CLEAN_FILES property.
set_directory_properties(PROPERTIES CLEAN_NO_CUSTOM TRUE)
# ccache setup
# ccache isn't enabled all the time because it makes some problem
# during the code coverage process
find_program(CCACHE_FOUND ccache)
option(USE_CCACHE "ccache:" ON)
message(STATUS "CCache: ${USE_CCACHE}")
if(CCACHE_FOUND AND USE_CCACHE)
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE ccache)
set_property(GLOBAL PROPERTY RULE_LAUNCH_LINK ccache)
endif(CCACHE_FOUND AND USE_CCACHE)
# choose a compiler
# NOTE: must be choosen before use of project() or enable_language()
find_program(CLANG_FOUND clang)
find_program(CLANGXX_FOUND clang++)
if (CLANG_FOUND AND CLANGXX_FOUND)
set(CMAKE_C_COMPILER ${CLANG_FOUND})
set(CMAKE_CXX_COMPILER ${CLANGXX_FOUND})
else()
message(FATAL_ERROR "Couldn't find clang and/or clang++!")
endif()
option(BUILD_FOR_DOCKER "Build Memgraph binary for docker." OFF)
message(STATUS "BUILD_FOR_DOCKER: ${BUILD_FOR_DOCKER}")
# -----------------------------------------------------------------------------
project(memgraph)
# For more information about how to release a new version of Memgraph, see
# `release/README.md`.
# Option that is used to specify which version of Memgraph should be built. The
# default is `ON` which causes the build system to build Memgraph Enterprise.
# Memgraph Community is built if explicitly set to `OFF`.
option(MG_ENTERPRISE "Build Memgraph Enterprise Edition" ON)
# Set the current version here to override the automatic version detection. The
# version must be specified as `X.Y.Z`. Primarily used when building new patch
# versions.
set(MEMGRAPH_OVERRIDE_VERSION "")
# Custom suffix that this version should have. The suffix can be any arbitrary
# string. Primarily used when building a version for a specific customer.
set(MEMGRAPH_OVERRIDE_VERSION_SUFFIX "")
# Variables used to generate the versions.
if (MG_ENTERPRISE)
set(get_version_enterprise "--enterprise")
else()
set(get_version_enterprise "")
endif()
set(get_version_script "${CMAKE_SOURCE_DIR}/release/get_version.py")
# Get version that should be used in the binary.
execute_process(
OUTPUT_VARIABLE MEMGRAPH_VERSION
RESULT_VARIABLE MEMGRAPH_VERSION_RESULT
COMMAND "${get_version_script}" ${get_version_enterprise}
"${MEMGRAPH_OVERRIDE_VERSION}"
"${MEMGRAPH_OVERRIDE_VERSION_SUFFIX}"
"--memgraph-root-dir"
"${CMAKE_SOURCE_DIR}"
)
if(MEMGRAPH_VERSION_RESULT AND NOT MEMGRAPH_VERSION_RESULT EQUAL 0)
message(FATAL_ERROR "Unable to get Memgraph version.")
else()
MESSAGE(STATUS "Memgraph version: ${MEMGRAPH_VERSION}")
endif()
# Get version that should be used in the DEB package.
execute_process(
OUTPUT_VARIABLE MEMGRAPH_VERSION_DEB
RESULT_VARIABLE MEMGRAPH_VERSION_DEB_RESULT
COMMAND "${get_version_script}" ${get_version_enterprise}
--variant deb
"${MEMGRAPH_OVERRIDE_VERSION}"
"${MEMGRAPH_OVERRIDE_VERSION_SUFFIX}"
"--memgraph-root-dir"
"${CMAKE_SOURCE_DIR}"
)
if(MEMGRAPH_VERSION_DEB_RESULT AND NOT MEMGRAPH_VERSION_DEB_RESULT EQUAL 0)
message(FATAL_ERROR "Unable to get Memgraph DEB version.")
else()
MESSAGE(STATUS "Memgraph DEB version: ${MEMGRAPH_VERSION_DEB}")
endif()
# Get version that should be used in the RPM package.
execute_process(
OUTPUT_VARIABLE MEMGRAPH_VERSION_RPM
RESULT_VARIABLE MEMGRAPH_VERSION_RPM_RESULT
COMMAND "${get_version_script}" ${get_version_enterprise}
--variant rpm
"${MEMGRAPH_OVERRIDE_VERSION}"
"${MEMGRAPH_OVERRIDE_VERSION_SUFFIX}"
"--memgraph-root-dir"
"${CMAKE_SOURCE_DIR}"
)
if(MEMGRAPH_VERSION_RPM_RESULT AND NOT MEMGRAPH_VERSION_RPM_RESULT EQUAL 0)
message(FATAL_ERROR "Unable to get Memgraph RPM version.")
else()
MESSAGE(STATUS "Memgraph RPM version: ${MEMGRAPH_VERSION_RPM}")
endif()
# We want the above variables to be updated each time something is committed to
# the repository. That is why we include a dependency on the current git HEAD
# to trigger a new CMake run when the git repository state changes. This is a
# hack, as CMake doesn't have a mechanism to regenerate variables when
# something changes (only files can be regenerated).
# https://cmake.org/pipermail/cmake/2018-October/068389.html
#
# The hack in the above link is nearly correct but it has a fatal flaw. The
# `CMAKE_CONFIGURE_DEPENDS` isn't a `GLOBAL` property, it is instead a
# `DIRECTORY` property and as such must be set in the `DIRECTORY` scope.
# https://cmake.org/cmake/help/v3.14/manual/cmake-properties.7.html
#
# Unlike the above mentioned hack, we don't use the `.git/index` file. That
# file changes on every `git add` (even on `git status`) so it triggers
# unnecessary recalculations of the release version. The release version only
# changes on every `git commit` or `git checkout`. That is why we watch the
# following files for changes:
# - `.git/HEAD` -> changes each time a `git checkout` is issued
# - `.git/refs/heads/...` -> the value in `.git/HEAD` is a branch name (when
# you are on a branch) and you have to monitor the file of the specific
# branch to detect when a `git commit` was issued
# More details about the contents of the `.git` directory and the specific
# files used can be seen here:
# https://git-scm.com/book/en/v2/Git-Internals-Git-References
set(git_directory "${CMAKE_SOURCE_DIR}/.git")
if (EXISTS "${git_directory}")
set_property(DIRECTORY APPEND PROPERTY
CMAKE_CONFIGURE_DEPENDS "${git_directory}/HEAD")
file(STRINGS "${git_directory}/HEAD" git_head_data)
if (git_head_data MATCHES "^ref: ")
string(SUBSTRING "${git_head_data}" 5 -1 git_head_ref)
set_property(DIRECTORY APPEND PROPERTY
CMAKE_CONFIGURE_DEPENDS "${git_directory}/${git_head_ref}")
endif()
endif()
# -----------------------------------------------------------------------------
# setup CMake module path, defines path for include() and find_package()
# https://cmake.org/cmake/help/latest/variable/CMAKE_MODULE_PATH.html
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${PROJECT_SOURCE_DIR}/cmake)
# custom function definitions
include(functions)
# -----------------------------------------------------------------------------
# We want out of source builds, so that cmake generated files don't get mixed
# with source files. This allows for easier clean up.
disallow_in_source_build()
add_custom_target(clean_all
COMMAND ${CMAKE_COMMAND} -P ${PROJECT_SOURCE_DIR}/cmake/clean_all.cmake
COMMENT "Removing all files in ${CMAKE_BINARY_DIR}")
# -----------------------------------------------------------------------------
# build flags -----------------------------------------------------------------
# Export the compile commands so that we can use clang-tidy. Additional benefit
# is easier debugging of compilation and linker flags.
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
# c99-designator is disabled because of required mixture of designated and
# non-designated initializers in Python Query Module code (`py_module.cpp`).
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall \
-Werror=switch -Werror=switch-bool -Werror=return-type \
-Werror=return-stack-address \
-Wno-c99-designator")
# Don't omit frame pointer in RelWithDebInfo, for additional callchain debug.
set(CMAKE_CXX_FLAGS_RELWITHDEBINFO
"${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
# Statically link libgcc and libstdc++, the GCC allows this according to:
# https://gcc.gnu.org/onlinedocs/gcc-10.2.0/libstdc++/manual/manual/license.html
# https://www.gnu.org/licenses/gcc-exception-faq.html
# Last checked for gcc-10.2 which we are using on the build machines.
# ** If we change versions, recheck this! **
# ** Static linking is allowed only for executables! **
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -static-libgcc -static-libstdc++")
# Use gold linker to speedup build
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fuse-ld=gold")
# release flags
set(CMAKE_CXX_FLAGS_RELEASE "-O2 -DNDEBUG")
#debug flags
set(PREFERRED_DEBUGGER "gdb" CACHE STRING
"Tunes the debug output for your preferred debugger (gdb or lldb).")
if ("${PREFERRED_DEBUGGER}" STREQUAL "gdb" AND
"${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang|GNU")
set(CMAKE_CXX_FLAGS_DEBUG "-ggdb")
elseif ("${PREFERRED_DEBUGGER}" STREQUAL "lldb" AND
"${CMAKE_CXX_COMPILER_ID}" STREQUAL "Clang")
set(CMAKE_CXX_FLAGS_DEBUG "-glldb")
else()
message(WARNING "Unable to tune for PREFERRED_DEBUGGER: "
"'${PREFERRED_DEBUGGER}' with compiler: '${CMAKE_CXX_COMPILER_ID}'")
set(CMAKE_CXX_FLAGS_DEBUG "-g")
endif()
# -----------------------------------------------------------------------------
# default build type is debug
if (NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE "Debug")
endif()
message(STATUS "CMake build type: ${CMAKE_BUILD_TYPE}")
# -----------------------------------------------------------------------------
# setup external dependencies -------------------------------------------------
# threading
find_package(Threads REQUIRED)
# optional readline
option(USE_READLINE "Use GNU Readline library if available (default ON). \
Set this to OFF to prevent linking with Readline even if it is available." ON)
if (USE_READLINE)
find_package(Readline)
if (READLINE_FOUND)
add_definitions(-DHAS_READLINE)
endif()
endif()
set(libs_dir ${CMAKE_SOURCE_DIR}/libs)
add_subdirectory(libs EXCLUDE_FROM_ALL)
# Optional subproject configuration -------------------------------------------
option(TEST_COVERAGE "Generate coverage reports from running memgraph" OFF)
option(TOOLS "Build tools binaries" ON)
option(QUERY_MODULES "Build query modules containing custom procedures" ON)
option(ASAN "Build with Address Sanitizer. To get a reasonable performance option should be used only in Release or RelWithDebInfo build " OFF)
option(TSAN "Build with Thread Sanitizer. To get a reasonable performance option should be used only in Release or RelWithDebInfo build " OFF)
option(UBSAN "Build with Undefined Behaviour Sanitizer" OFF)
if (TEST_COVERAGE)
string(TOLOWER ${CMAKE_BUILD_TYPE} lower_build_type)
if (NOT lower_build_type STREQUAL "debug")
message(FATAL_ERROR "Generating test coverage unsupported in non Debug builds. Current build type is '${CMAKE_BUILD_TYPE}'")
endif()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fprofile-instr-generate -fcoverage-mapping")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fprofile-instr-generate -fcoverage-mapping")
endif()
if (MG_ENTERPRISE)
add_definitions(-DMG_ENTERPRISE)
endif()
if (ASAN)
# Enable Addres sanitizer and get nicer stack traces in error messages.
# NOTE: AddressSanitizer uses llvm-symbolizer binary from the Clang
# distribution to symbolize the stack traces (note that ideally the
# llvm-symbolizer version must match the version of ASan runtime library).
# Just make sure llvm-symbolizer is in PATH before running the binary or
# provide it in separate ASAN_SYMBOLIZER_PATH environment variable.
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=address -fno-omit-frame-pointer")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fsanitize=address")
# To detect Stack-use-after-return bugs set run-time flag:
# ASAN_OPTIONS=detect_stack_use_after_return=1
# To check initialization order bugs set run-time flag:
# ASAN_OPTIONS=check_initialization_order=true
# This mode reports an error if initializer for a global variable accesses
# dynamically initialized global from another translation unit, which is
# not yet initialized
# ASAN_OPTIONS=strict_init_order=true
# This mode reports an error if initializer for a global variable accesses
# any dynamically initialized global from another translation unit.
endif()
if (TSAN)
# ThreadSanitizer generally requires all code to be compiled with -fsanitize=thread.
# If some code (e.g. dynamic libraries) is not compiled with the flag, it can
# lead to false positive race reports, false negative race reports and/or
# missed stack frames in reports depending on the nature of non-instrumented
# code. To not produce false positive reports ThreadSanitizer has to see all
# synchronization in the program, some synchronization operations (namely,
# atomic operations and thread-safe static initialization) are intercepted
# during compilation (and can only be intercepted during compilation).
# ThreadSanitizer stack trace collection also relies on compiler instrumentation
# (unwinding stack on each memory access is too expensive).
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=thread")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fsanitize=thread")
# By default ThreadSanitizer uses addr2line utility to symbolize reports.
# llvm-symbolizer is faster, consumes less memory and produces much better
# reports. To use it set runtime flag:
# TSAN_OPTIONS="extern-symbolizer-path=~/llvm-symbolizer"
# For more runtime flags see: https://github.com/google/sanitizers/wiki/ThreadSanitizerFlags
endif()
if (UBSAN)
# Compile with UBSAN but disable vptr check. This is disabled because it
# requires linking with clang++ to make sure C++ specific parts of the
# runtime library and c++ standard libraries are present.
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=undefined -fno-omit-frame-pointer -fno-sanitize=vptr")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fsanitize=undefined -fno-sanitize=vptr")
# Run program with environment variable UBSAN_OPTIONS=print_stacktrace=1.
# Make sure llvm-symbolizer binary is in path.
# To make the program abort on undefined behavior, use UBSAN_OPTIONS=halt_on_error=1.
endif()
set(MG_PYTHON_VERSION "" CACHE STRING "Specify the exact python version used by the query modules")
# Add subprojects
include_directories(src)
add_subdirectory(src)
# Release configuration
add_subdirectory(release)
enable_testing()
add_subdirectory(tests)
if(TOOLS)
add_subdirectory(tools)
endif()
if(QUERY_MODULES)
add_subdirectory(query_modules)
endif()

View File

@@ -1,4 +0,0 @@
/docs/ @gitbuda
/src/communication/ @antonio2368
/src/query/ @the-joksim
/src/storage/ @antonio2368

127
CODE_OF_CONDUCT.md Normal file
View File

@@ -0,0 +1,127 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, caste, color, religion, or sexual
identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
- Demonstrating empathy and kindness toward other people
- Being respectful of differing opinions, viewpoints, and experiences
- Giving and gracefully accepting constructive feedback
- Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
- Focusing on what is best not just for us as individuals, but for the overall
community
Examples of unacceptable behavior include:
- The use of sexualized language or imagery, and sexual attention or advances of
any kind
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing other's private information, such as a physical or email address,
without their explicit permission
- Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
[contact@memgraph.com](contact@memgraph.com). All complaints will be reviewed
and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series of
actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or permanent
ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within the
community.
## Attribution
This Code of Conduct is adapted from the Contributor Covenant, version 2.1,
available at
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html](https://www.contributor-covenant.org/version/2/1/code_of_conduct.html).
Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder][mozilla coc].
For answers to common questions about this code of conduct, see the FAQ at
[https://www.contributor-covenant.org/faq](https://www.contributor-covenant.org/faq).
Translations are available at
[https://www.contributor-covenant.org/translations](https://www.contributor-covenant.org/translations).

121
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,121 @@
# How to contribute?
This is a general purpose guide for contributing to Memgraph. We're still
working out the kinks to make contributing to this project as easy and
transparent as possible, but we're not quite there yet. Hopefully, this document
makes the process for contributing clear and answers some questions that you may
have.
- [How to contribute?](#how-to-contribute)
- [Open development](#open-development)
- [Branch organization](#branch-organization)
- [Bugs & changes](#bugs--changes)
- [Where to find known issues?](#where-to-find-known-issues)
- [Proposing a change](#proposing-a-change)
- [Your first pull request](#your-first-pull-request)
- [Sending a pull request](#sending-a-pull-request)
- [Style guide](#style-guide)
- [How to get in touch?](#how-to-get-in-touch)
- [Code of Conduct](#code-of-conduct)
- [License](#license)
- [Attribution](#attribution)
## Open development
All work on Memgraph is done via [GitHub](https://github.com/memgraph/memgraph).
Both core team members and external contributors send pull requests which go
through the same review process.
## Branch organization
Most pull requests should target the [`master
branch`](https://github.com/memgraph/memgraph/tree/master). We only use separate
branches for developing new features and fixing bugs before they are merged with
`master`. We do our best to keep `master` in good shape, with all tests passing.
Code that lands in `master` must be compatible with the latest stable release.
It may contain additional features but no breaking changes if it's not
absolutely necessary. We should be able to release a new minor version from the
tip of `master` at any time.
## Bugs & changes
### Where to find known issues?
We are using [GitHub Issues](https://github.com/memgraph/memgraph/issues) for
our public bugs. We keep a close eye on this and try to make it clear when we
have an internal fix in progress. Before filing a new task, try to make sure
your problem doesn't already exist.
### Proposing a change
If you intend to change the public API, or make any non-trivial changes to the
implementation, we recommend [filing an
issue](https://github.com/memgraph/memgraph/issues/new). This lets us reach an
agreement on your proposal before you put significant effort into it.
If you're only fixing a bug, it's fine to submit a pull request right away but
we still recommend to file an issue detailing what you're fixing. This is
helpful in case we don't accept that specific fix but want to keep track of the
issue.
### Your first pull request
Working on your first Pull Request? You can learn how from this free video
series:
**[How to Contribute to an Open Source Project on
GitHub](https://app.egghead.io/courses/how-to-contribute-to-an-open-source-project-on-github)**
If you decide to fix an issue, please be sure to check the comment thread in
case somebody is already working on a fix. If nobody is working on it at the
moment, please leave a comment stating that you intend to work on it so other
people don't accidentally duplicate your effort.
If somebody claims an issue but doesn't follow up for more than two weeks, it's
fine to take it over but you should still leave a comment.
### Sending a pull request
The core team is monitoring for pull requests. We will review your pull request
and either merge it, request changes to it, or close it with an explanation.
**Before submitting a pull request,** please make sure the following is done:
1. Fork [the repository](https://github.com/memgraph/memgraph) and create your
branch from `master`.
2. If you've fixed a bug or added code that should be tested, add tests!
3. Use the formatter `clang-format` for C/C++ code and `flake8` for Python code.
`clang-format` will automatically detect the `.clang-format` file in the root
directory while `flake8` can be used with the default configuration.
### Style guide
Memgraph uses the [Google Style
Guide](https://google.github.io/styleguide/cppguide.html) for C++ in most of its
code. You should follow them whenever writing new code.
## How to get in touch?
Aside from communicating directly via Pull Requests and Issues, the Memgraph
Community [Discord Server](https://discord.gg/memgraph) is the best place for
conversing with project maintainers and other community members.
## [Code of Conduct](https://github.com/memgraph/memgraph/blob/master/CODE_OF_CONDUCT.md)
Memgraph has adopted the [Contributor
Covenant](https://www.contributor-covenant.org/) as its Code of Conduct, and we
expect project participants to adhere to it. Please read [the full
text](https://github.com/memgraph/memgraph/blob/master/CODE_OF_CONDUCT.md) so
that you can understand what actions will and will not be tolerated.
## License
By contributing to Memgraph, you agree that your contributions will be licensed
under the [Memgraph licensing
scheme](https://github.com/memgraph/memgraph/blob/master/LICENSE).
## Attribution
This Contributing guide is adapted from the **React.js Contributing guide**
available at
[https://reactjs.org/docs/how-to-contribute.html](https://reactjs.org/docs/how-to-contribute.html).

2438
Doxyfile

File diff suppressed because it is too large Load Diff

5
LICENSE Normal file
View File

@@ -0,0 +1,5 @@
Source code in this repository is variously licensed under the Business Source
License 1.1 (BSL), the Memgraph Enterprise License (MEL). A copy of each licence
can be found in the licences directory. Source code in a given file is licensed
under the BSL and the copyright belongs to The Memgraph Authors unless
otherwise noted at th beginning of the file.

194
README.md
View File

@@ -1,21 +1,187 @@
# Memgraph
<p align="center">
<img width="400px" src="https://uploads-ssl.webflow.com/5e7ceb09657a69bdab054b3a/5e7ceb09657a6937ab054bba_Black_Original%20_Logo.png">
</p>
Memgraph is an ACID compliant high performance transactional distributed
in-memory graph database featuring runtime native query compiling, lock free
data structures, multi-version concurrency control and asynchronous IO.
---
## Development Documentation
<p align="center">
<a href="https://github.com/memgraph/memgraph/blob/master/licenses/APL.txt">
<img src="https://img.shields.io/badge/license-APL-green" alt="license" title="license"/>
</a>
<a href="https://github.com/memgraph/memgraph/blob/master/licenses/BSL.txt">
<img src="https://img.shields.io/badge/license-BSL-yellowgreen" alt="license" title="license"/>
</a>
<a href="https://github.com/memgraph/memgraph/blob/master/licenses/MEL.txt" alt="Documentation">
<img src="https://img.shields.io/badge/license-MEL-yellow" alt="license" title="license"/>
</a>
</p>
Please continue in
[Notion](https://www.notion.so/memgraph/memgraph-0428591638604c8385550e214ea9f3e6).
<p align="center">
<a href="https://github.com/memgraph/memgraph">
<img src="https://img.shields.io/github/actions/workflow/status/memgraph/memgraph/release_debian10.yaml?branch=master&label=build%20and%20test&logo=github"/>
</a>
<a href="https://memgraph.com/docs/" alt="Documentation">
<img src="https://img.shields.io/badge/documentation-Memgraph-orange" />
</a>
</p>
## User Documentation
<p align="center">
<a href="https://memgr.ph/join-discord">
<img src="https://img.shields.io/badge/Discord-7289DA?style=for-the-badge&logo=discord&logoColor=white" alt="Discord"/>
</a>
</p>
Memgraph user documentation is maintained within
[docs](https://github.com/memgraph/docs) repository. The documentation is also
available on [GitBook](https://docs.memgraph.com).
## :clipboard: Description
## Licences
Memgraph is an open source graph database built for real-time streaming and
compatible with Neo4j. Whether you're a developer or a data scientist with
interconnected data, Memgraph will get you the immediate actionable insights
fast.
* [Community](release/LICENSE_COMMUNITY.md)
* [Enterprise](release/LICENSE_ENTERPRISE.md)
Memgraph directly connects to your streaming infrastructure. You can ingest data
from sources like Kafka, SQL, or plain CSV files. Memgraph provides a standard
interface to query your data with Cypher, a widely-used and declarative query
language that is easy to write, understand and optimize for performance. This is
achieved by using the property graph data model, which stores data in terms of
objects, their attributes, and the relationships that connect them. This is a
natural and effective way to model many real-world problems without relying on
complex SQL schemas.
Memgraph is implemented in C/C++ and leverages an in-memory first architecture
to ensure that youre getting the [best possible
performance](http://memgraph.com/benchgraph) consistently and without surprises.
Its also ACID-compliant and highly available.
## :zap: Features
- Run Python, Rust, and C/C++ code natively, check out the
[MAGE](https://github.com/memgraph/mage) graph algorithm library
- Native support for machine learning
- Streaming support
- Replication
- Authentication and authorization
- ACID compliance
## :video_game: Memgraph Playground
You don't need to install anything to try out Memgraph. Check out
our **[Memgraph Playground](https://playground.memgraph.com/)** sandboxes in
your browser.
<p align="left">
<a href="https://playground.memgraph.com/">
<img width="450px" alt="Memgraph Playground" src="https://download.memgraph.com/asset/github/memgraph/memgraph-playground.png">
</a>
</p>
## :floppy_disk: Download & Install
### Windows
[![Windows](https://img.shields.io/badge/Windows-Docker-0078D6?style=for-the-badge&logo=windows&logoColor=white)](https://memgraph.com/docs/memgraph/install-memgraph-on-windows-docker)
[![Windows](https://img.shields.io/badge/Windows-WSL-0078D6?style=for-the-badge&logo=windows&logoColor=white)](https://memgraph.com/docs/memgraph/install-memgraph-on-windows-wsl)
### macOS
[![macOS](https://img.shields.io/badge/macOS-Docker-000000?style=for-the-badge&logo=macos&logoColor=F0F0F0)](https://memgraph.com/docs/memgraph/install-memgraph-on-macos-docker)
[![macOS](https://img.shields.io/badge/lima-AACF41?style=for-the-badge&logo=macos&logoColor=F0F0F0)](https://memgraph.com/docs/memgraph/install-memgraph-on-ubuntu)
### Linux
[![Linux](https://img.shields.io/badge/Linux-Docker-FCC624?style=for-the-badge&logo=linux&logoColor=black)](https://memgraph.com/docs/memgraph/install-memgraph-on-linux-docker)
[![Debian](https://img.shields.io/badge/Debian-D70A53?style=for-the-badge&logo=debian&logoColor=white)](https://memgraph.com/docs/memgraph/install-memgraph-on-debian)
[![Ubuntu](https://img.shields.io/badge/Ubuntu-E95420?style=for-the-badge&logo=ubuntu&logoColor=white)](https://memgraph.com/docs/memgraph/install-memgraph-on-ubuntu)
[![Cent OS](https://img.shields.io/badge/cent%20os-002260?style=for-the-badge&logo=centos&logoColor=F0F0F0)](https://memgraph.com/docs/memgraph/install-memgraph-from-rpm)
[![Fedora](https://img.shields.io/badge/fedora-0B57A4?style=for-the-badge&logo=fedora&logoColor=F0F0F0)](https://memgraph.com/docs/memgraph/install-memgraph-from-rpm)
[![RedHat](https://img.shields.io/badge/redhat-EE0000?style=for-the-badge&logo=redhat&logoColor=F0F0F0)](https://memgraph.com/docs/memgraph/install-memgraph-from-rpm)
You can find the binaries and Docker images on the [Download
Hub](https://memgraph.com/download) and the installation instructions in the
[official documentation](https://memgraph.com/docs/memgraph/installation).
## :cloud: Memgraph Cloud
Check out [Memgraph Cloud](https://memgraph.com/docs/memgraph-cloud) - a cloud service fully managed on AWS and available in 6 geographic regions around the world. Memgraph Cloud allows you to create projects with Enterprise instances of MemgraphDB from your browser.
<p align="left">
<a href="https://memgraph.com/docs/memgraph-cloud">
<img width="450px" alt="Memgraph Cloud" src="https://public-assets.memgraph.com/memgraph-gifs%2Fcloud.gif">
</a>
</p>
## :link: Connect to Memgraph
[Connect to the database](https://memgraph.com/docs/memgraph/connect-to-memgraph) using Memgraph Lab, mgconsole, various drivers (Python, C/C++ and others) and WebSocket.
### :microscope: Memgraph Lab
Visualize graphs and play with queries to understand your data. [Memgraph Lab](https://memgraph.com/docs/memgraph-lab) is a user interface that helps you explore and manipulate the data stored in Memgraph. Visualize graphs, execute ad hoc queries, and optimize their performance.
<p align="left">
<a href="https://memgraph.com/docs/memgraph-lab">
<img width="450px" alt="Memgraph Cloud" src="https://public-assets.memgraph.com/memgraph-gifs%2Flab.gif">
</a>
</p>
## :file_folder: Import data
[Import data](https://memgraph.com/docs/memgraph/import-data) into Memgraph using Kafka, RedPanda or Pulsar streams, CSV and JSON files, or Cypher commands.
## :bookmark_tabs: Documentation
The Memgraph documentation is available at
[memgraph.com/docs](https://memgraph.com/docs).
## :question: Configuration
Command line options that Memgraph accepts are available in the [reference
guide](https://memgraph.com/docs/memgraph/reference-guide/configuration).
## :trophy: Contributing
The main purpose of this repository is to continue evolving Memgraph, making it
faster and easier to use. Development of Memgraph happens in the open on GitHub,
and we are grateful to the community for contributing bug fixes and
improvements. Read below to learn how you can take part in improving Memgraph.
### Code of Conduct
Memgraph has adopted a Code of Conduct that we expect project participants to
adhere to. Please read [the full text](CODE_OF_CONDUCT.md) so that you can
understand what actions will and will not be tolerated.
### Contributing Guide
Read our [contributing guide](CONTRIBUTING.md) to learn about our development
process and how to propose bug fixes and improvements.
### Internals
Read our
[internal](https://memgraph.notion.site/Memgraph-Internals-12b69132d67a417898972927d6870bd2)
docs to learn more about Memgraph's architecture, how to build the project from
source and how to start contributing. All information related to the database,
can be found in the aforementioned docs.
### :scroll: License
Memgraph Community is available under the [BSL
license](./licenses/BSL.txt).</br> Memgraph Enterprise is available under the
[MEL license](./licenses/MEL.txt).
## :busts_in_silhouette: Community
- :purple_heart: [**Discord**](https://discord.gg/memgraph)
- :ocean: [**Stack Overflow**](https://stackoverflow.com/questions/tagged/memgraphdb)
- :busts_in_silhouette: [**Discourse forum**](https://discourse.memgraph.com/)
- :bird: [**Twitter**](https://twitter.com/memgraphdb)
- :movie_camera:
[**YouTube**](https://www.youtube.com/channel/UCZ3HOJvHGxtQ_JHxOselBYg)
<p align="center">
<a href="#">
<img src="https://img.shields.io/badge/⬆️ back_to_top_⬆-white" alt="Back to top" title="Back to top"/>
</a>
</p>

View File

@@ -1,35 +0,0 @@
# Find the GNU Readline library.
# This module plugs into CMake's `find_package` so the example usage is:
# `find_package(Readline REQUIRED)`
# Options to `find_package` are as documented in CMake documentation.
# READLINE_LIBRARY will be a path to the library.
# READLINE_INCLUDE_DIR will be a path to the include directory.
# READLINE_FOUND will be TRUE if the library is found.
#
# If the library is found, an imported target `readline` will be provided. This
# can be used for linking via `target_link_libraries`, without the need to
# explicitly include READLINE_INCLUDE_DIR and link with READLINE_LIBRARY. For
# example: `target_link_libraries(my_executable readline)`.
if (READLINE_LIBRARY AND READLINE_INCLUDE_DIR)
set(READLINE_FOUND TRUE)
else()
find_library(READLINE_LIBRARY readline)
find_path(READLINE_INCLUDE_DIR readline/readline.h)
if (READLINE_LIBRARY AND READLINE_INCLUDE_DIR)
set(READLINE_FOUND TRUE)
if (NOT READLINE_FIND_QUIETLY)
message(STATUS "Found Readline: ${READLINE_LIBRARY} ${READLINE_INCLUDE_DIR}")
endif()
else()
set(READLINE_FOUND FALSE)
if (READLINE_FIND_REQUIRED)
message(FATAL_ERROR "Could not find Readline")
elseif (NOT READLINE_FIND_QUIETLY)
message(STATUS "Could not find Readline")
endif()
endif()
mark_as_advanced(READLINE_LIBRARY READLINE_INCLUDE_DIR)
add_library(readline SHARED IMPORTED)
set_property(TARGET readline PROPERTY INTERFACE_INCLUDE_DIRECTORIES ${READLINE_INCLUDE_DIR})
set_property(TARGET readline PROPERTY IMPORTED_LOCATION ${READLINE_LIBRARY})
endif()

View File

@@ -1,90 +0,0 @@
#.rst:
# FindSeccomp
# -----------
#
# Try to locate the libseccomp library.
# If found, this will define the following variables:
#
# ``Seccomp_FOUND``
# True if the seccomp library is available
# ``Seccomp_INCLUDE_DIRS``
# The seccomp include directories
# ``Seccomp_LIBRARIES``
# The seccomp libraries for linking
#
# If ``Seccomp_FOUND`` is TRUE, it will also define the following
# imported target:
#
# ``Seccomp::Seccomp``
# The Seccomp library
#
# Since 5.44.0.
#=============================================================================
# Copyright (c) 2017 Martin Flöser <mgraesslin@kde.org>
# Copyright (c) 2017 David Kahles <david.kahles96@gmail.com>
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# 1. Redistributions of source code must retain the copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# 3. The name of the author may not be used to endorse or promote products
# derived from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
# OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
# NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#=============================================================================
find_package(PkgConfig QUIET)
pkg_check_modules(PKG_Libseccomp QUIET libseccomp)
find_path(Seccomp_INCLUDE_DIRS
NAMES
seccomp.h
HINTS
${PKG_Libseccomp_INCLUDE_DIRS}
)
find_library(Seccomp_LIBRARIES
NAMES
seccomp
HINTS
${PKG_Libseccomp_LIBRARY_DIRS}
)
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(Seccomp
FOUND_VAR
Seccomp_FOUND
REQUIRED_VARS
Seccomp_LIBRARIES
Seccomp_INCLUDE_DIRS
)
if (Seccomp_FOUND AND NOT TARGET Seccomp::Seccomp)
add_library(Seccomp::Seccomp UNKNOWN IMPORTED)
set_target_properties(Seccomp::Seccomp PROPERTIES
IMPORTED_LOCATION "${Seccomp_LIBRARIES}"
INTERFACE_INCLUDE_DIRECTORIES "${Seccomp_INCLUDE_DIRS}"
)
endif()
mark_as_advanced(Seccomp_LIBRARIES Seccomp_INCLUDE_DIRS)
include(FeatureSummary)
set_package_properties(Seccomp PROPERTIES
URL "https://github.com/seccomp/libseccomp"
DESCRIPTION "The enhanced seccomp library."
)

View File

@@ -1,2 +0,0 @@
file(GLOB build_contents "${CMAKE_BINARY_DIR}/*")
file(REMOVE_RECURSE ${build_contents})

View File

@@ -1,87 +0,0 @@
# prints all included directories
function(list_includes)
get_property(dirs DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
PROPERTY INCLUDE_DIRECTORIES)
foreach(dir ${dirs})
message(STATUS "dir='${dir}'")
endforeach()
endfunction(list_includes)
# get file names from list of file paths
function(get_file_names file_paths file_names)
set(file_names "")
foreach(file_path ${file_paths})
get_filename_component (file_name ${file_path} NAME_WE)
list(APPEND file_names ${file_name})
endforeach()
set(file_names "${file_names}" PARENT_SCOPE)
endfunction()
MACRO(SUBDIRLIST result curdir)
FILE(GLOB children RELATIVE ${curdir} ${curdir}/*)
SET(dirlist "")
FOREACH(child ${children})
IF(IS_DIRECTORY ${curdir}/${child})
LIST(APPEND dirlist ${child})
ENDIF()
ENDFOREACH()
SET(${result} ${dirlist})
ENDMACRO()
function(disallow_in_source_build)
get_filename_component(src_dir ${CMAKE_SOURCE_DIR} REALPATH)
get_filename_component(bin_dir ${CMAKE_BINARY_DIR} REALPATH)
message(STATUS "SOURCE_DIR" ${src_dir})
message(STATUS "BINARY_DIR" ${bin_dir})
# Do we maybe want to limit out-of-source builds to be only inside a
# directory which contains 'build' in name?
if("${src_dir}" STREQUAL "${bin_dir}")
# Unfortunately, we cannot remove CMakeCache.txt and CMakeFiles here
# because they are written after cmake is done.
message(FATAL_ERROR "In source build is not supported! "
"Remove CMakeCache.txt and CMakeFiles and then create a separate "
"directory, e.g. 'build' and run cmake there.")
endif()
endfunction()
# Takes a string of ';' separated VALUES and stores a new string in RESULT,
# where ';' is replaced with given SEP.
function(join values sep result)
# Match non escaped ';' and replace it with separator. This doesn't handle
# the case when backslash is escaped, e.g: "a\\\\;b" will produce "a;b".
string(REGEX REPLACE "([^\\]|^);" "\\1${sep}" tmp "${values}")
# Fix-up escapes by matching backslashes and removing them.
string(REGEX REPLACE "[\\](.)" "\\1" tmp "${tmp}")
set(${result} "${tmp}" PARENT_SCOPE)
endfunction()
# Returns a list of compile flags ready for gcc or clang.
function(get_target_cxx_flags target result)
# First set the CMAKE_CXX_FLAGS variables, then append directory and target
# options in that order. Definitions come last, directory then target.
string(TOUPPER ${CMAKE_BUILD_TYPE} build_type)
set(flags "${CMAKE_CXX_FLAGS} ${CMAKE_CXX_FLAGS_${build_type}}")
get_directory_property(dir_opts DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
COMPILE_OPTIONS)
if(dir_opts)
join("${dir_opts}" " " dir_opts)
string(APPEND flags " " ${dir_opts})
endif()
get_target_property(opts ${target} COMPILE_OPTIONS)
if(opts)
join("${opts}" " " opts)
string(APPEND flags " " ${opts})
endif()
get_directory_property(dir_defs DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
COMPILE_DEFINITIONS)
if(dir_defs)
join("${dir_defs}" " -D" dir_defs)
string(APPEND flags " -D" ${dir_defs})
endif()
get_target_property(defs ${target} COMPILE_DEFINITIONS)
if(defs)
join("${defs}" " -D" defs)
string(APPEND flags " -D" ${defs})
endif()
set(${result} ${flags} PARENT_SCOPE)
endfunction()

View File

@@ -1,95 +0,0 @@
header: >-
Memgraph Configuration
This is the main configuration file for Memgraph. You can modify this file to
suit your specific needs. Additional configuration can be specified by
including another configuration file, in a file pointed to by the
'MEMGRAPH_CONFIG' environment variable or by passing arguments on the command
line.
Each configuration setting is in the form: '--setting-name=value'.
footer: >-
Additional Configuration Inclusion
You can include additional configuration files from this file. Additional
files are processed after this file. Settings that are set in the additional
files will override previously set values. Additional configuration files are
specified with the '--flag-file' flag.
Example:
--flag-file=another.conf
modifications:
# Each modification should consist of the following parameters:
# * name: the name of the flag that should be modified (with underscores)
# [string]
# * value: the value that should be set instead of the binary provided
# default value [string]
# * override: set to `true` to uncomment the config option by default
# [boolean]
- name: "data_directory"
value: "/var/lib/memgraph"
override: true
- name: "log_file"
value: "/var/log/memgraph/memgraph.log"
override: true
- name: "log_level"
value: "WARNING"
override: true
- name: "bolt_num_workers"
value: ""
override: false
- name: "storage_properties_on_edges"
value: "true"
override: true
- name: "storage_recover_on_startup"
value: "true"
override: true
- name: "storage_snapshot_interval_sec"
value: "300"
override: true
- name: "storage_snapshot_on_exit"
value: "true"
override: true
- name: "storage_snapshot_retention_count"
value: "3"
override: true
- name: "storage_wal_enabled"
value: "true"
override: true
- name: "telemetry_enabled"
value: "true"
override: true
- name: "query_modules_directory"
value: "/usr/lib/memgraph/query_modules"
override: true
- name: "auth_module_executable"
value: "/usr/lib/memgraph/auth_module/example.py"
override: false
- name: "memory_limit"
value: "0"
override: true
undocumented:
- "flag_file"
- "also_log_to_stderr"
- "help"
- "help_xml"
- "version"

View File

@@ -1,124 +0,0 @@
#!/usr/bin/env python3
import argparse
import copy
import os
import subprocess
import sys
import textwrap
import xml.etree.ElementTree as ET
import yaml
SCRIPT_DIR = os.path.dirname(os.path.realpath(__file__))
CONFIG_FILE = os.path.join(SCRIPT_DIR, "flags.yaml")
WIDTH = 80
def wrap_text(s, initial_indent="# "):
return "\n#\n".join(
map(lambda x: textwrap.fill(x, WIDTH, initial_indent=initial_indent,
subsequent_indent="# "), s.split("\n")))
def extract_flags(binary_path):
ret = {}
data = subprocess.run([binary_path, "--help-xml"],
stdout=subprocess.PIPE).stdout.decode("utf-8")
root = ET.fromstring(data)
for child in root:
if child.tag == "usage" and child.text.lower().count("warning"):
raise Exception("You should set the usage message!")
if child.tag == "flag":
flag = {}
for elem in child:
flag[elem.tag] = elem.text if elem.text is not None else ""
flag["override"] = False
ret[flag["name"]] = flag
return ret
def apply_config_to_flags(config, flags):
flags = copy.deepcopy(flags)
for name in config["undocumented"]:
flags.pop(name)
for modification in config["modifications"]:
name = modification["name"]
if name not in flags:
print("WARNING: Flag '" + name + "' missing from binary!",
file=sys.stderr)
continue
flags[name]["default"] = modification["value"]
flags[name]["override"] = modification["override"]
return flags
def extract_sections(flags):
sections = []
other = []
current_section = ""
current_flags = []
for name in sorted(flags.keys()):
section = name.split("_")[0]
if section == current_section:
current_flags.append(name)
else:
if len(current_flags) < 2:
other.extend(current_flags)
else:
sections.append((current_section, current_flags))
current_section = section
current_flags = [name]
if len(current_flags) < 2:
other.extend(current_flags)
else:
sections.append((current_section, current_flags))
sections.append(("other", other))
assert set(sum(map(lambda x: x[1], sections), [])) == set(flags.keys()), \
"The section extraction algorithm lost some flags!"
return sections
def generate_config_file(sections, flags):
ret = wrap_text(config["header"]) + "\n\n\n"
for section, section_flags in sections:
ret += wrap_text(section.capitalize(), initial_indent="## ") + "\n\n"
for name in section_flags:
flag = flags[name]
helpstr = flag["meaning"] + " [" + flag["type"] + "]"
ret += wrap_text(helpstr) + "\n"
prefix = "# " if not flag["override"] else ""
ret += prefix + "--" + flag["name"].replace("_", "-") + \
"=" + flag["default"] + "\n\n"
ret += "\n"
ret += wrap_text(config["footer"])
return ret.strip() + "\n"
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("memgraph_binary",
help="path to Memgraph binary")
parser.add_argument("output_file",
help="path where to store the generated Memgraph "
"configuration file")
parser.add_argument("--config-file", default=CONFIG_FILE,
help="path to generator configuration file")
args = parser.parse_args()
flags = extract_flags(args.memgraph_binary)
with open(args.config_file) as f:
config = yaml.safe_load(f)
flags = apply_config_to_flags(config, flags)
sections = extract_sections(flags)
data = generate_config_file(sections, flags)
dirname = os.path.dirname(args.output_file)
if dirname and not os.path.exists(dirname):
os.makedirs(dirname)
with open(args.output_file, "w") as f:
f.write(data)

View File

@@ -172,7 +172,7 @@ relationshipDetail : '[' ( name=variable )? ( relationshipTypes )? ( variableExp
relationshipLambda: '(' traversed_edge=variable ',' traversed_node=variable '|' expression ')';
variableExpansion : '*' (BFS | WSHORTEST)? ( expression )? ( '..' ( expression )? )? ;
variableExpansion : '*' (BFS | WSHORTEST | ALLSHORTEST)? ( expression )? ( '..' ( expression )? )? ;
properties : mapLiteral
| parameter
@@ -279,7 +279,7 @@ idInColl : variable IN expression ;
functionInvocation : functionName '(' ( DISTINCT )? ( expression ( ',' expression )* )? ')' ;
functionName : symbolicName ;
functionName : symbolicName ( '.' symbolicName )* ;
listComprehension : '[' filterExpression ( '|' expression )? ']' ;
@@ -381,6 +381,7 @@ cypherKeyword : ALL
| WHERE
| WITH
| WSHORTEST
| ALLSHORTEST
| XOR
| YIELD
;

View File

@@ -139,6 +139,7 @@ WHEN : W H E N ;
WHERE : W H E R E ;
WITH : W I T H ;
WSHORTEST : W S H O R T E S T ;
ALLSHORTEST : A L L S H O R T E S T ;
XOR : X O R ;
YIELD : Y I E L D ;

View File

@@ -0,0 +1,404 @@
/*
* Copyright 2021 Memgraph Ltd.
*
* Use of this software is governed by the Business Source License
* included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
* License, and you may not use this file except in compliance with the Business Source License.
*
* As of the Change Date specified in that file, in accordance with
* the Business Source License, use of this software will be governed
* by the Apache License, Version 2.0, included in the file
* licenses/APL.txt.
*/
/* Memgraph specific part of Cypher grammar with enterprise features. */
parser grammar MemgraphCypher ;
options { tokenVocab=MemgraphCypherLexer; }
import Cypher ;
memgraphCypherKeyword : cypherKeyword
| AFTER
| ALTER
| ASYNC
| AUTH
| BAD
| BATCH_INTERVAL
| BATCH_LIMIT
| BATCH_SIZE
| BEFORE
| BOOTSTRAP_SERVERS
| CHECK
| CLEAR
| COMMIT
| COMMITTED
| CONFIG
| CONFIGS
| CONSUMER_GROUP
| CREATE_DELETE
| CREDENTIALS
| CSV
| DATA
| DELIMITER
| DATABASE
| DENY
| DROP
| DUMP
| EDGE_TYPES
| EXECUTE
| FOR
| FOREACH
| FREE
| FROM
| GLOBAL
| GRANT
| HEADER
| IDENTIFIED
| ISOLATION
| KAFKA
| LABELS
| LEVEL
| LOAD
| LOCK
| MAIN
| MODE
| NEXT
| NO
| NOTHING
| PASSWORD
| PULSAR
| PORT
| PRIVILEGES
| READ
| REGISTER
| REPLICA
| REPLICAS
| REPLICATION
| REVOKE
| ROLE
| ROLES
| QUOTE
| SESSION
| SETTING
| SETTINGS
| SNAPSHOT
| START
| STATS
| STREAM
| STREAMS
| SYNC
| TIMEOUT
| TO
| TOPICS
| TRANSACTION
| TRANSFORM
| TRIGGER
| TRIGGERS
| UNCOMMITTED
| UNLOCK
| UPDATE
| USER
| USERS
| VERSION
;
symbolicName : UnescapedSymbolicName
| EscapedSymbolicName
| memgraphCypherKeyword
;
query : cypherQuery
| indexQuery
| explainQuery
| profileQuery
| infoQuery
| constraintQuery
| authQuery
| dumpQuery
| replicationQuery
| lockPathQuery
| freeMemoryQuery
| triggerQuery
| isolationLevelQuery
| createSnapshotQuery
| streamQuery
| settingQuery
| versionQuery
| showConfigQuery
;
authQuery : createRole
| dropRole
| showRoles
| createUser
| setPassword
| dropUser
| showUsers
| setRole
| clearRole
| grantPrivilege
| denyPrivilege
| revokePrivilege
| showPrivileges
| showRoleForUser
| showUsersForRole
;
replicationQuery : setReplicationRole
| showReplicationRole
| registerReplica
| dropReplica
| showReplicas
;
triggerQuery : createTrigger
| dropTrigger
| showTriggers
;
clause : cypherMatch
| unwind
| merge
| create
| set
| cypherDelete
| remove
| with
| cypherReturn
| callProcedure
| loadCsv
| foreach
;
updateClause : set
| remove
| create
| merge
| cypherDelete
| foreach
;
foreach : FOREACH '(' variable IN expression '|' updateClause+ ')' ;
streamQuery : checkStream
| createStream
| dropStream
| startStream
| startAllStreams
| stopStream
| stopAllStreams
| showStreams
;
settingQuery : setSetting
| showSetting
| showSettings
;
loadCsv : LOAD CSV FROM csvFile ( WITH | NO ) HEADER
( IGNORE BAD ) ?
( DELIMITER delimiter ) ?
( QUOTE quote ) ?
AS rowVar ;
csvFile : literal ;
delimiter : literal ;
quote : literal ;
rowVar : variable ;
userOrRoleName : symbolicName ;
createRole : CREATE ROLE role=userOrRoleName ;
dropRole : DROP ROLE role=userOrRoleName ;
showRoles : SHOW ROLES ;
createUser : CREATE USER user=userOrRoleName
( IDENTIFIED BY password=literal )? ;
setPassword : SET PASSWORD FOR user=userOrRoleName TO password=literal;
dropUser : DROP USER user=userOrRoleName ;
showUsers : SHOW USERS ;
setRole : SET ROLE FOR user=userOrRoleName TO role=userOrRoleName;
clearRole : CLEAR ROLE FOR user=userOrRoleName ;
grantPrivilege : GRANT ( ALL PRIVILEGES | privileges=grantPrivilegesList ) TO userOrRole=userOrRoleName ;
denyPrivilege : DENY ( ALL PRIVILEGES | privileges=privilegesList ) TO userOrRole=userOrRoleName ;
revokePrivilege : REVOKE ( ALL PRIVILEGES | privileges=revokePrivilegesList ) FROM userOrRole=userOrRoleName ;
privilege : CREATE
| DELETE
| MATCH
| MERGE
| SET
| REMOVE
| INDEX
| STATS
| AUTH
| CONSTRAINT
| DUMP
| REPLICATION
| READ_FILE
| FREE_MEMORY
| TRIGGER
| CONFIG
| DURABILITY
| STREAM
| MODULE_READ
| MODULE_WRITE
| WEBSOCKET
;
granularPrivilege : NOTHING | READ | UPDATE | CREATE_DELETE ;
entityType : LABELS | EDGE_TYPES ;
privilegeOrEntityPrivileges : privilege | entityPrivileges=entityPrivilegeList ;
grantPrivilegesList : privilegeOrEntityPrivileges ( ',' privilegeOrEntityPrivileges )* ;
entityPrivilegeList : entityPrivilege ( ',' entityPrivilege )* ;
entityPrivilege : granularPrivilege ON entityType entities=entitiesList ;
privilegeOrEntities : privilege | entityType entities=entitiesList ;
revokePrivilegesList : privilegeOrEntities ( ',' privilegeOrEntities )* ;
privilegesList : privilege ( ',' privilege )* ;
entitiesList : ASTERISK | listOfEntities ;
listOfEntities : entity ( ',' entity )* ;
entity : COLON symbolicName ;
showPrivileges : SHOW PRIVILEGES FOR userOrRole=userOrRoleName ;
showRoleForUser : SHOW ROLE FOR user=userOrRoleName ;
showUsersForRole : SHOW USERS FOR role=userOrRoleName ;
dumpQuery: DUMP DATABASE ;
setReplicationRole : SET REPLICATION ROLE TO ( MAIN | REPLICA )
( WITH PORT port=literal ) ? ;
showReplicationRole : SHOW REPLICATION ROLE ;
replicaName : symbolicName ;
socketAddress : literal ;
registerReplica : REGISTER REPLICA replicaName ( SYNC | ASYNC )
TO socketAddress ;
dropReplica : DROP REPLICA replicaName ;
showReplicas : SHOW REPLICAS ;
lockPathQuery : ( LOCK | UNLOCK ) DATA DIRECTORY ;
freeMemoryQuery : FREE MEMORY ;
triggerName : symbolicName ;
triggerStatement : .*? ;
emptyVertex : '(' ')' ;
emptyEdge : dash dash rightArrowHead ;
createTrigger : CREATE TRIGGER triggerName ( ON ( emptyVertex | emptyEdge ) ? ( CREATE | UPDATE | DELETE ) ) ?
( AFTER | BEFORE ) COMMIT EXECUTE triggerStatement ;
dropTrigger : DROP TRIGGER triggerName ;
showTriggers : SHOW TRIGGERS ;
isolationLevel : SNAPSHOT ISOLATION | READ COMMITTED | READ UNCOMMITTED ;
isolationLevelScope : GLOBAL | SESSION | NEXT ;
isolationLevelQuery : SET isolationLevelScope TRANSACTION ISOLATION LEVEL isolationLevel ;
createSnapshotQuery : CREATE SNAPSHOT ;
streamName : symbolicName ;
symbolicNameWithMinus : symbolicName ( MINUS symbolicName )* ;
symbolicNameWithDotsAndMinus: symbolicNameWithMinus ( DOT symbolicNameWithMinus )* ;
symbolicTopicNames : symbolicNameWithDotsAndMinus ( COMMA symbolicNameWithDotsAndMinus )* ;
topicNames : symbolicTopicNames | literal ;
commonCreateStreamConfig : TRANSFORM transformationName=procedureName
| BATCH_INTERVAL batchInterval=literal
| BATCH_SIZE batchSize=literal
;
createStream : kafkaCreateStream | pulsarCreateStream ;
configKeyValuePair : literal ':' literal ;
configMap : '{' ( configKeyValuePair ( ',' configKeyValuePair )* )? '}' ;
kafkaCreateStreamConfig : TOPICS topicNames
| CONSUMER_GROUP consumerGroup=symbolicNameWithDotsAndMinus
| BOOTSTRAP_SERVERS bootstrapServers=literal
| CONFIGS configsMap=configMap
| CREDENTIALS credentialsMap=configMap
| commonCreateStreamConfig
;
kafkaCreateStream : CREATE KAFKA STREAM streamName ( kafkaCreateStreamConfig ) * ;
pulsarCreateStreamConfig : TOPICS topicNames
| SERVICE_URL serviceUrl=literal
| commonCreateStreamConfig
;
pulsarCreateStream : CREATE PULSAR STREAM streamName ( pulsarCreateStreamConfig ) * ;
dropStream : DROP STREAM streamName ;
startStream : START STREAM streamName ( BATCH_LIMIT batchLimit=literal ) ? ( TIMEOUT timeout=literal ) ? ;
startAllStreams : START ALL STREAMS ;
stopStream : STOP STREAM streamName ;
stopAllStreams : STOP ALL STREAMS ;
showStreams : SHOW STREAMS ;
checkStream : CHECK STREAM streamName ( BATCH_LIMIT batchLimit=literal ) ? ( TIMEOUT timeout=literal ) ? ;
settingName : literal ;
settingValue : literal ;
setSetting : SET DATABASE SETTING settingName TO settingValue ;
showSetting : SHOW DATABASE SETTING settingName ;
showSettings : SHOW DATABASE SETTINGS ;
showConfigQuery : SHOW CONFIG ;
versionQuery : SHOW VERSION ;

View File

@@ -0,0 +1,120 @@
/*
* Copyright 2021 Memgraph Ltd.
*
* Use of this software is governed by the Business Source License
* included in the file licenses/BSL.txt; by using this file, you agree to be bound by the terms of the Business Source
* License, and you may not use this file except in compliance with the Business Source License.
*
* As of the Change Date specified in that file, in accordance with
* the Business Source License, use of this software will be governed
* by the Apache License, Version 2.0, included in the file
* licenses/APL.txt.
*/
/* Memgraph specific Cypher reserved words used for enterprise features. */
/*
* When changing this grammar make sure to update constants in
* src/query/frontend/stripped_lexer_constants.hpp (kKeywords, kSpecialTokens
* and bitsets) if needed.
*/
lexer grammar MemgraphCypherLexer ;
import CypherLexer ;
UNDERSCORE : '_' ;
AFTER : A F T E R ;
ALTER : A L T E R ;
ASYNC : A S Y N C ;
AUTH : A U T H ;
BAD : B A D ;
BATCH_INTERVAL : B A T C H UNDERSCORE I N T E R V A L ;
BATCH_LIMIT : B A T C H UNDERSCORE L I M I T ;
BATCH_SIZE : B A T C H UNDERSCORE S I Z E ;
BEFORE : B E F O R E ;
BOOTSTRAP_SERVERS : B O O T S T R A P UNDERSCORE S E R V E R S ;
CHECK : C H E C K ;
CLEAR : C L E A R ;
COMMIT : C O M M I T ;
COMMITTED : C O M M I T T E D ;
CONFIG : C O N F I G ;
CONFIGS : C O N F I G S;
CONSUMER_GROUP : C O N S U M E R UNDERSCORE G R O U P ;
CREATE_DELETE : C R E A T E UNDERSCORE D E L E T E ;
CREDENTIALS : C R E D E N T I A L S ;
CSV : C S V ;
DATA : D A T A ;
DELIMITER : D E L I M I T E R ;
DATABASE : D A T A B A S E ;
DENY : D E N Y ;
DIRECTORY : D I R E C T O R Y ;
DROP : D R O P ;
DUMP : D U M P ;
DURABILITY : D U R A B I L I T Y ;
EXECUTE : E X E C U T E ;
FOR : F O R ;
FOREACH : F O R E A C H;
FREE : F R E E ;
FREE_MEMORY : F R E E UNDERSCORE M E M O R Y ;
FROM : F R O M ;
GLOBAL : G L O B A L ;
GRANT : G R A N T ;
GRANTS : G R A N T S ;
HEADER : H E A D E R ;
IDENTIFIED : I D E N T I F I E D ;
IGNORE : I G N O R E ;
ISOLATION : I S O L A T I O N ;
KAFKA : K A F K A ;
LABELS : L A B E L S ;
LEVEL : L E V E L ;
LOAD : L O A D ;
LOCK : L O C K ;
MAIN : M A I N ;
MODE : M O D E ;
MODULE_READ : M O D U L E UNDERSCORE R E A D ;
MODULE_WRITE : M O D U L E UNDERSCORE W R I T E ;
NEXT : N E X T ;
NO : N O ;
NOTHING : N O T H I N G ;
PASSWORD : P A S S W O R D ;
PORT : P O R T ;
PRIVILEGES : P R I V I L E G E S ;
PULSAR : P U L S A R ;
READ : R E A D ;
READ_FILE : R E A D UNDERSCORE F I L E ;
REGISTER : R E G I S T E R ;
REPLICA : R E P L I C A ;
REPLICAS : R E P L I C A S ;
REPLICATION : R E P L I C A T I O N ;
REVOKE : R E V O K E ;
ROLE : R O L E ;
ROLES : R O L E S ;
QUOTE : Q U O T E ;
SERVICE_URL : S E R V I C E UNDERSCORE U R L ;
SESSION : S E S S I O N ;
SETTING : S E T T I N G ;
SETTINGS : S E T T I N G S ;
SNAPSHOT : S N A P S H O T ;
START : S T A R T ;
STATS : S T A T S ;
STOP : S T O P ;
STREAM : S T R E A M ;
STREAMS : S T R E A M S ;
SYNC : S Y N C ;
TIMEOUT : T I M E O U T ;
TO : T O ;
TOPICS : T O P I C S;
TRANSACTION : T R A N S A C T I O N ;
TRANSFORM : T R A N S F O R M ;
TRIGGER : T R I G G E R ;
TRIGGERS : T R I G G E R S ;
UNCOMMITTED : U N C O M M I T T E D ;
UNLOCK : U N L O C K ;
UPDATE : U P D A T E ;
USER : U S E R ;
USERS : U S E R S ;
VERSION : V E R S I O N ;
WEBSOCKET : W E B S O C K E T ;
EDGE_TYPES : E D G E UNDERSCORE T Y P E S ;

View File

@@ -0,0 +1,204 @@
# CSV Import Tool Documentation
CSV is a universal and very versatile data format used to store large quantities
of data. Each Memgraph database instance has a CSV import tool installed called
`mg_import_csv`. The CSV import tool should be used for initial bulk ingestion
of data into the database. Upon ingestion, the CSV importer creates a snapshot
that will be used by the database to recover its state on its next startup.
If you are already familiar with the Neo4j bulk import tool, then using the
`mg_import_csv` tool should be easy. The CSV import tool is fully compatible
with the [Neo4j CSV
format](https://neo4j.com/docs/operations-manual/current/tools/import/). If you
already have a pipeline set-up for Neo4j, you should only replace `neo4j-admin
import` with `mg_import_csv`.
## CSV File Format
Each row of a CSV file represents a single entry that should be imported into
the database. Both nodes and relationships can be imported into the database
using CSV files.
Each set of CSV files must have a header that describes the data that is stored
in the CSV files. Each field in the CSV header is in the format
`<name>[:<type>]` which identifies the name that should be used for that column
and the type that should be used for that column. The type is optional and
defaults to `string` (see the following chapter).
Each CSV field must be divided using the delimiter and each CSV field can either
be quoted or unquoted. When the field is quoted, the first and last character in
the field *must* be the quote character. If the field isn't quoted, and a quote
character appears in it, it is treated as a regular character. If a quote
character appears inside a quoted string then the quote character must be
doubled in order to escape it. Line feeds and carriage returns are ignored in
the CSV file, also, the file can't contain a NULL character.
## Properties
Both nodes and relationships can have properties added to them. When importing
properties, the CSV importer uses the name specified in the header of the
corresponding CSV column for the name of the property. A property is designated
by specifying one of the following types in the header:
- `integer`, `int`, `long`, `byte`, `short`: creates an integer property
- `float`, `double`: creates a float property
- `boolean`, `bool`: creates a boolean property
- `string`, `char`: creates a string property
When importing a boolean value, the CSV field should contain exactly the text
`true` to import a `True` boolean value. All other text values are treated as a
boolean value `False`.
If you want to import an array of values, you can do so by appending `[]` to any
of the above types. The values of the array are then determined by splitting
the raw CSV value using the array delimiter character.
Assuming that the array delimiter is `;`, the following example:
```plaintext
first_name,last_name:string,number:integer,aliases:string[]
John,Doe,1,Johnny;Jo;J-man
Melissa,Doe,2,Mel
```
Will yield these results:
```plaintext
CREATE ({first_name: "John", last_name: "Doe", number: 1, aliases: ["Johnny", "Jo", "J-man"]});
CREATE ({first_name: "Melissa", last_name: "Doe", number: 2, aliases: ["Mel"]});
```
### Nodes
When importing nodes, several more types can be specified in the header of the
CSV file (along with all property types):
- `ID`: id of the node that should be used as the node ID when importing
relationships
- `LABEL`: designates that the field contains additional labels for the node
- `IGNORE`: designates that the field should be ignored
The `ID` field type sets the internal ID that will be used for the node when
creating relationships. It is optional and nodes that don't have an ID value
specified will be imported, but can't be connected to any relationships. If you
want to save the ID value as a property in the database, just specify a name for
the ID (`user_id:ID`). If you just want to use the ID during the import, leave
out the name of the field (`:ID`). The `ID` field also supports creating
separate ID spaces. The ID space is specified with the ID space name appended
to the `ID` type in parentheses (`ID(user)`). That allows you to have the same
IDs (by value) for multiple different node files (for example, numbers from 1 to
N). The IDs in each ID space will be treated as an independent set of IDs that
don't interfere with IDs in another ID space.
The `LABEL` field type adds additional labels to the node. The value is treated
as an array type so that multiple additional labels can be specified for each
node. The value is split using the array delimiter (`--array-delimiter` flag).
### Relationships
In order to be able to import relationships, you must import the nodes in the
same invocation of `mg_import_csv` that is used to import the relationships.
When importing relationships, several more types can be specified in the header
of the CSV file (along with all property types):
- `START_ID`: id of the start node that should be connected with the
relationship
- `END_ID`: id of the end node that should be connected with the relationship
- `TYPE`: designates the type of the relationship
- `IGNORE`: designates that the field should be ignored
The `START_ID` field type sets the start node that should be connected with the
relationship to the end node. The field *must* be specified and the node ID
must be one of the node IDs that were specified in the node CSV files. The name
of this field is ignored. If the node ID is in an ID space, you can specify the
ID space for the in the same way as for the node ID (`START_ID(user)`).
The `END_ID` field type sets the end node that should be connected with the
relationship to the start node. The field *must* be specified and the node ID
must be one of the node IDs that were specified in the node CSV files. The name
of this field is ignored. If the node ID is in an ID space, you can specify the
ID space for the in the same way as for the node ID (`END_ID(user)`).
The `TYPE` field type sets the type of the relationship. Each relationship
*must* have a relationship type, but it doesn't necessarily need to be specified
in the CSV file, it can also be set externally for the whole CSV file. The name
of this field is ignored.
## CSV Importer Flags
The importer has many command line options that allow you to customize the way
the importer loads your data.
The two main flags that are used to specify the input CSV files are `--nodes`
and `--relationships`. Basic description of these flags is provided in the table
and more detailed explainion can be found further down bellow.
| Flag | Description |
|-----------------------| -------------- |
|`--nodes` | Used to specify CSV files that contain the nodes to the importer. |
|`--relationships` | Used to specify CSV files that contain the relationships to the importer.|
|`--delimiter` | Sets the delimiter that should be used when splitting the CSV fields (default `,`)|
|`--quote` | Sets the quote character that should be used to quote a CSV field (default `"`)|
|`--array-delimiter` | Sets the delimiter that should be used when splitting array values (default `;`)|
|`--id-type` | Specifies which data type should be used to store the supplied <br /> node IDs when storing them as properties (if the field name is supplied). <br /> The supported values are either `STRING` or `INTEGER`. (default `STRING`)|
|`--ignore-empty-strings` | Instructs the importer to treat all empty strings as `Null` values <br /> instead of an empty string value (default `false`)|
|`--ignore-extra-columns` | Instructs the importer to ignore all columns (instead of raising an error) <br /> that aren't specified after the last specified column in the CSV header. (default `false`) |
| `--skip-bad-relationships`| Instructs the importer to ignore all relationships (instead of raising an error) <br /> that refer to nodes that don't exist in the node files. (default `false`) |
|`--skip-duplicate-nodes` | Instructs the importer to ignore all duplicate nodes (instead of raising an error). <br /> Duplicate nodes are nodes that have an ID that is the same as another node that was already imported. (default `false`) |
| `--trim-strings`| Instructs the importer to trim all of the loaded CSV field values before processing them further. <br /> Trimming the fields removes all leading and trailing whitespace from them. (default `false`) |
The `--nodes` and `--relationships` flags are used to specify CSV files that
contain the nodes and relationships to the importer. Multiple files can be
specified in each supplied `--nodes` or `--relationships` flag. Files that are
supplied in one `--nodes` or `--relationships` flag are treated by the CSV
parser as one big CSV file. Only the first line of the first file is parsed for
the CSV header, all other files (and rows) are treated as data. This is useful
when you have a very large CSV file and don't want to edit its first line just
to add a CSV header. Instead, you can specify the header in a separate file
(e.g. `users_header.csv` or `friendships_header.csv`) and have the data intact
in the large file (e.g. `users.csv` or `friendships.csv`). Also, you can supply
additional labels for each set of node files.
The format of `--nodes` flag is:
`[<label>[:<label>]...=]<file>[,<file>][,<file>]...`. Take note that only the
first `<file>` part is mandatory, all other parts of the flag value are
optional. Multiple `--nodes` flags can be supplied to describe multiple sets of
different node files. For the importer to work, at least one `--nodes` flag
*must* be supplied.
The format of `--relationships` flag is: `[<type>=]<file>[,<file>][,<file>]...`.
Take note that only the first `<file>` part is mandatory, all other parts of the
flag value are optional. Multiple `--relationships` flags can be supplied to
describe multiple sets of different relationship files. The `--relationships`
flag isn't mandatory.
## CSV Parser Logic
The CSV parser uses the same logic as the standard Python CSV parser. The data
is parsed in the same way as the following snippet:
```python
import csv
for row in csv.reader(stream, strict=True):
# process 'row'
```
Python uses 'excel' as the default dialect when parsing CSV files and the
default settings for the CSV parser are:
- delimiter: `','`
- doublequote: `True`
- escapechar: `None`
- lineterminator: `'\r\n'`
- quotechar: `'"'`
- skipinitialspace: `False`
The above snippet can be expanded to:
```python
import csv
for row in csv.reader(stream, delimiter=',', doublequote=True,
escapechar=None, lineterminator='\r\n',
quotechar='"', skipinitialspace=False,
strict=True):
# process 'row'
```
For more information about the meaning of the above values, see:
https://docs.python.org/3/library/csv.html#csv.Dialect

View File

@@ -1,2 +0,0 @@
archives
build

View File

@@ -1,4 +0,0 @@
# Memgraph Build and Run Environments
Please continue in
[Notion](https://www.notion.so/memgraph/Tools-05e0baafb78a49b386e0063b4833d23d).

View File

@@ -1,3 +0,0 @@
*.deb
*.rpm
*.tar.gz

View File

@@ -1,134 +0,0 @@
#!/bin/bash
set -Eeuo pipefail
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
source "$DIR/../util.sh"
TOOLCHAIN_BUILD_DEPS=(
coreutils gcc gcc-c++ make # generic build tools
wget # used for archive download
gnupg2 # used for archive signature verification
tar gzip bzip2 xz unzip # used for archive unpacking
zlib-devel # zlib library used for all builds
expat-devel libipt-devel libbabeltrace-devel xz-devel python3-devel # gdb
texinfo # gdb
libcurl-devel # cmake
readline-devel # cmake and llvm
libffi-devel libxml2-devel perl-Digest-MD5 # llvm
libedit-devel pcre-devel automake bison # swig
)
TOOLCHAIN_RUN_DEPS=(
make # generic build tools
tar gzip bzip2 xz # used for archive unpacking
zlib # zlib library used for all builds
expat libipt libbabeltrace xz-libs python3 # for gdb
readline # for cmake and llvm
libffi libxml2 # for llvm
)
MEMGRAPH_BUILD_DEPS=(
git # source code control
make pkgconfig # build system
curl wget # for downloading libs
libuuid-devel java-11-openjdk # required by antlr
readline-devel # for memgraph console
python3-devel # for query modules
openssl-devel
libseccomp-devel
python3 python-virtualenv python3-pip nmap-ncat # for qa, macro_benchmark and stress tests
#
# IMPORTANT: python3-yaml does NOT exist on CentOS
# Install it using `pip3 install PyYAML`
#
PyYAML # Package name here does not correspond to the yum package!
libcurl-devel # mg-requests
sbcl # for custom Lisp C++ preprocessing
rpm-build rpmlint # for RPM package building
doxygen graphviz # source documentation generators
which mono-complete dotnet-sdk-3.1 golang nodejs zip unzip java-11-openjdk-devel # for driver tests
autoconf # for jemalloc code generation
)
list() {
echo "$1"
}
check() {
local missing=""
for pkg in $1; do
if [ "$pkg" == git ]; then
if ! which "git" >/dev/null; then
missing="git $missing"
fi
continue
fi
if [ "$pkg" == "PyYAML" ]; then
if ! python3 -c "import yaml" >/dev/null 2>/dev/null; then
missing="$pkg $missing"
fi
continue
fi
if ! yum list installed "$pkg" >/dev/null 2>/dev/null; then
missing="$pkg $missing"
fi
done
if [ "$missing" != "" ]; then
echo "MISSING PACKAGES: $missing"
exit 1
fi
}
install() {
cd "$DIR"
if [ "$EUID" -ne 0 ]; then
echo "Please run as root."
exit 1
fi
# If GitHub Actions runner is installed, append LANG to the environment.
# Python related tests doesn't work the LANG export.
if [ -d "/home/gh/actions-runner" ]; then
echo "LANG=en_US.utf8" >> /home/gh/actions-runner/.env
else
echo "NOTE: export LANG=en_US.utf8"
fi
yum install -y epel-release
yum update -y
yum install -y wget git python3 python3-pip
for pkg in $1; do
if [ "$pkg" == libipt ]; then
if ! yum list installed libipt >/dev/null 2>/dev/null; then
yum install -y http://repo.okay.com.mx/centos/8/x86_64/release/libipt-1.6.1-8.el8.x86_64.rpm
fi
continue
fi
if [ "$pkg" == libipt-devel ]; then
if ! yum list installed libipt-devel >/dev/null 2>/dev/null; then
yum install -y http://repo.okay.com.mx/centos/8/x86_64/release/libipt-devel-1.6.1-8.el8.x86_64.rpm
fi
continue
fi
if [ "$pkg" == dotnet-sdk-3.1 ]; then
if ! yum list installed dotnet-sdk-3.1 >/dev/null 2>/dev/null; then
wget -nv https://packages.microsoft.com/config/centos/7/packages-microsoft-prod.rpm -O packages-microsoft-prod.rpm
rpm -Uvh https://packages.microsoft.com/config/centos/7/packages-microsoft-prod.rpm
yum update -y
yum install -y dotnet-sdk-3.1
fi
continue
fi
if [ "$pkg" == PyYAML ]; then
if [ -z ${SUDO_USER+x} ]; then # Running as root (e.g. Docker).
pip3 install --user PyYAML
else # Running using sudo.
sudo -H -u "$SUDO_USER" bash -c "pip3 install --user PyYAML"
fi
continue
fi
yum install -y "$pkg"
done
}
deps=$2"[*]"
"$1" "${!deps}"

View File

@@ -1,152 +0,0 @@
#!/bin/bash
set -Eeuo pipefail
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
source "$DIR/../util.sh"
TOOLCHAIN_BUILD_DEPS=(
coreutils gcc gcc-c++ make # generic build tools
wget # used for archive download
gnupg2 # used for archive signature verification
tar gzip bzip2 xz unzip # used for archive unpacking
zlib-devel # zlib library used for all builds
expat-devel libipt-devel libbabeltrace-devel xz-devel python36-devel texinfo # for gdb
libcurl-devel # for cmake
readline-devel # for cmake and llvm
libffi-devel libxml2-devel # for llvm
libedit-devel pcre-devel automake bison # for swig
)
TOOLCHAIN_RUN_DEPS=(
make # generic build tools
tar gzip bzip2 xz # used for archive unpacking
zlib # zlib library used for all builds
expat libipt libbabeltrace xz-libs python36 # for gdb
readline # for cmake and llvm
libffi libxml2 # for llvm
)
MEMGRAPH_BUILD_DEPS=(
git # source code control
make pkgconf-pkg-config # build system
curl wget # for downloading libs
libuuid-devel java-11-openjdk # required by antlr
readline-devel # for memgraph console
python36-devel # for query modules
openssl-devel
libseccomp-devel
python36 python3-virtualenv python3-pip nmap-ncat # for qa, macro_benchmark and stress tests
#
# IMPORTANT: python3-yaml does NOT exist on CentOS
# Install it manually using `pip3 install PyYAML`
#
PyYAML # Package name here does not correspond to the yum package!
libcurl-devel # mg-requests
rpm-build rpmlint # for RPM package building
doxygen graphviz # source documentation generators
which mono-complete dotnet-sdk-3.1 nodejs golang zip unzip java-11-openjdk-devel # for driver tests
sbcl # for custom Lisp C++ preprocessing
autoconf # for jemalloc code generation
)
list() {
echo "$1"
}
check() {
local missing=""
for pkg in $1; do
if [ "$pkg" == "PyYAML" ]; then
if ! python3 -c "import yaml" >/dev/null 2>/dev/null; then
missing="$pkg $missing"
fi
continue
fi
if ! yum list installed "$pkg" >/dev/null 2>/dev/null; then
missing="$pkg $missing"
fi
done
if [ "$missing" != "" ]; then
echo "MISSING PACKAGES: $missing"
exit 1
fi
}
install() {
cd "$DIR"
if [ "$EUID" -ne 0 ]; then
echo "Please run as root."
exit 1
fi
# If GitHub Actions runner is installed, append LANG to the environment.
# Python related tests doesn't work the LANG export.
if [ -d "/home/gh/actions-runner" ]; then
echo "LANG=en_US.utf8" >> /home/gh/actions-runner/.env
else
echo "NOTE: export LANG=en_US.utf8"
fi
dnf install -y epel-release
dnf install -y 'dnf-command(config-manager)'
dnf config-manager --set-enabled powertools # Required to install texinfo.
dnf update -y
dnf install -y wget git python36 python3-pip
for pkg in $1; do
if [ "$pkg" == libipt ]; then
if ! dnf list installed libipt >/dev/null 2>/dev/null; then
dnf install -y http://repo.okay.com.mx/centos/8/x86_64/release/libipt-1.6.1-8.el8.x86_64.rpm
fi
continue
fi
if [ "$pkg" == libipt-devel ]; then
if ! yum list installed libipt-devel >/dev/null 2>/dev/null; then
dnf install -y http://repo.okay.com.mx/centos/8/x86_64/release/libipt-devel-1.6.1-8.el8.x86_64.rpm
fi
continue
fi
# Install GDB dependencies not present in the standard repos.
# https://bugs.centos.org/view.php?id=17068
# https://centos.pkgs.org
# Since 2020, there is Babeltrace2 (https://babeltrace.org). Not used
# within GDB yet (an assumption).
if [ "$pkg" == libbabeltrace-devel ]; then
if ! dnf list installed libbabeltrace-devel >/dev/null 2>/dev/null; then
dnf install -y http://repo.okay.com.mx/centos/8/x86_64/release/libbabeltrace-devel-1.5.4-2.el8.x86_64.rpm
fi
continue
fi
if [ "$pkg" == sbcl ]; then
if ! dnf list installed cl-asdf >/dev/null 2>/dev/null; then
dnf install -y https://pkgs.dyn.su/el8/base/x86_64/cl-asdf-20101028-18.el8.noarch.rpm
fi
if ! dnf list installed common-lisp-controller >/dev/null 2>/dev/null; then
dnf install -y https://pkgs.dyn.su/el8/base/x86_64/common-lisp-controller-7.4-20.el8.noarch.rpm
fi
if ! dnf list installed sbcl >/dev/null 2>/dev/null; then
dnf install -y https://pkgs.dyn.su/el8/base/x86_64/sbcl-2.0.1-4.el8.x86_64.rpm
fi
continue
fi
if [ "$pkg" == dotnet-sdk-3.1 ]; then
if ! dnf list installed dotnet-sdk-3.1 >/dev/null 2>/dev/null; then
wget -nv https://packages.microsoft.com/config/centos/8/packages-microsoft-prod.rpm -O packages-microsoft-prod.rpm
rpm -Uvh https://packages.microsoft.com/config/centos/8/packages-microsoft-prod.rpm
dnf update -y
dnf install -y dotnet-sdk-3.1
fi
continue
fi
if [ "$pkg" == PyYAML ]; then
if [ -z ${SUDO_USER+x} ]; then # Running as root (e.g. Docker).
pip3 install --user PyYAML
else # Running using sudo.
sudo -H -u "$SUDO_USER" bash -c "pip3 install --user PyYAML"
fi
continue
fi
dnf install -y "$pkg"
done
}
deps=$2"[*]"
"$1" "${!deps}"

View File

@@ -1,93 +0,0 @@
#!/bin/bash
set -Eeuo pipefail
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
source "$DIR/../util.sh"
TOOLCHAIN_BUILD_DEPS=(
coreutils gcc g++ build-essential make # generic build tools
wget # used for archive download
gnupg # used for archive signature verification
tar gzip bzip2 xz-utils unzip # used for archive unpacking
zlib1g-dev # zlib library used for all builds
libexpat1-dev libipt-dev libbabeltrace-dev liblzma-dev python3-dev texinfo # for gdb
libcurl4-openssl-dev # for cmake
libreadline-dev # for cmake and llvm
libffi-dev libxml2-dev # for llvm
libedit-dev libpcre3-dev automake bison # for swig
)
TOOLCHAIN_RUN_DEPS=(
make # generic build tools
tar gzip bzip2 xz-utils # used for archive unpacking
zlib1g # zlib library used for all builds
libexpat1 libipt2 libbabeltrace1 liblzma5 python3 # for gdb
libcurl4 # for cmake
libreadline7 # for cmake and llvm
libffi6 libxml2 # for llvm
)
MEMGRAPH_BUILD_DEPS=(
git # source code control
make pkg-config # build system
curl wget # for downloading libs
uuid-dev default-jre-headless # required by antlr
libreadline-dev # for memgraph console
libpython3-dev python3-dev # for query modules
libssl-dev
libseccomp-dev
netcat # tests are using nc to wait for memgraph
python3 virtualenv python3-virtualenv python3-pip # for qa, macro_benchmark and stress tests
python3-yaml # for the configuration generator
libcurl4-openssl-dev # mg-requests
sbcl # for custom Lisp C++ preprocessing
doxygen graphviz # source documentation generators
mono-runtime mono-mcs zip unzip default-jdk-headless # for driver tests
dotnet-sdk-3.1 golang nodejs npm
autoconf # for jemalloc code generation
)
list() {
echo "$1"
}
check() {
check_all_dpkg "$1"
}
install() {
cat >/etc/apt/sources.list <<EOF
deb http://deb.debian.org/debian/ buster main non-free contrib
deb-src http://deb.debian.org/debian/ buster main non-free contrib
deb http://deb.debian.org/debian/ buster-updates main contrib non-free
deb-src http://deb.debian.org/debian/ buster-updates main contrib non-free
deb http://security.debian.org/debian-security buster/updates main contrib non-free
deb-src http://security.debian.org/debian-security buster/updates main contrib non-free
EOF
cd "$DIR"
apt update
# If GitHub Actions runner is installed, append LANG to the environment.
# Python related tests doesn't work the LANG export.
if [ -d "/home/gh/actions-runner" ]; then
echo "LANG=en_US.utf8" >> /home/gh/actions-runner/.env
else
echo "NOTE: export LANG=en_US.utf8"
fi
apt install -y wget
for pkg in $1; do
if [ "$pkg" == dotnet-sdk-3.1 ]; then
if ! dpkg -s "$pkg" 2>/dev/null >/dev/null; then
wget -nv https://packages.microsoft.com/config/debian/10/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
dpkg -i packages-microsoft-prod.deb
apt-get update
apt-get install -y apt-transport-https dotnet-sdk-3.1
fi
continue
fi
apt install -y "$pkg"
done
}
deps=$2"[*]"
"$1" "${!deps}"

View File

@@ -1,62 +0,0 @@
#!/bin/bash
set -Eeuo pipefail
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
source "$DIR/../util.sh"
TOOLCHAIN_BUILD_DEPS=(
coreutils gcc g++ build-essential make # generic build tools
wget # used for archive download
gnupg # used for archive signature verification
tar gzip bzip2 xz-utils unzip # used for archive unpacking
zlib1g-dev # zlib library used for all builds
libexpat1-dev libipt-dev libbabeltrace-dev libbabeltrace-ctf-dev liblzma-dev python3-dev texinfo # for gdb
libcurl4-openssl-dev # for cmake
libreadline-dev # for cmake and llvm
libffi-dev libxml2-dev # for llvm
libedit-dev libpcre3-dev automake bison # for swig
)
TOOLCHAIN_RUN_DEPS=(
make # generic build tools
tar gzip bzip2 xz-utils # used for archive unpacking
zlib1g # zlib library used for all builds
libexpat1 libipt1 libbabeltrace1 libbabeltrace-ctf1 liblzma5 python3 # for gdb
libcurl3 # for cmake
libreadline7 # for cmake and llvm
libffi6 libxml2 # for llvm
)
MEMGRAPH_BUILD_DEPS=(
git # source code control
make pkg-config # build system
curl wget # for downloading libs
uuid-dev default-jre-headless # required by antlr
libreadline-dev # for memgraph console
libpython3-dev python3-dev # for query modules
libssl-dev
libseccomp-dev
python3 python-virtualenv python3-pip # for qa, macro_benchmark and stress tests
python3-yaml # for the configuration generator
libcurl4-openssl-dev # mg-requests
sbcl # for custom Lisp C++ preprocessing
doxygen graphviz # source documentation generators
mono-runtime mono-mcs nodejs zip unzip default-jdk-headless # for driver tests
autoconf # for jemalloc code generation
)
list() {
echo "$1"
}
check() {
check_all_dpkg "$1"
}
install() {
install_all_apt "$1"
}
deps=$2"[*]"
"$1" "${!deps}"

View File

@@ -1,36 +0,0 @@
#!/bin/bash
set -Eeuo pipefail
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
source "$DIR/../util.sh"
TOOLCHAIN_BUILD_DEPS=(
pkg
)
TOOLCHAIN_RUN_DEPS=(
pkg
)
MEMGRAPH_BUILD_DEPS=(
pkg
)
list() {
echo "$1"
}
check() {
echo "TODO: Implement ${FUNCNAME[0]}."
exit 1
}
install() {
echo "TODO: Implement ${FUNCNAME[0]}."
exit 1
}
# http://ahmed.amayem.com/bash-indirect-expansion-exploration
deps=$2"[*]"
"$1" "${!deps}"

View File

@@ -1,63 +0,0 @@
#!/bin/bash
set -Eeuo pipefail
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
source "$DIR/../util.sh"
TOOLCHAIN_BUILD_DEPS=(
coreutils gcc g++ build-essential make # generic build tools
wget # archive download
gnupg # archive signature verification
tar gzip bzip2 xz-utils unzip # archive unpacking
zlib1g-dev # zlib library used for all builds
libexpat1-dev libipt-dev libbabeltrace-dev liblzma-dev python3-dev # gdb
texinfo # gdb
libcurl4-openssl-dev # cmake
libreadline-dev # cmake and llvm
libffi-dev libxml2-dev # llvm
libedit-dev libpcre3-dev automake bison # swig
)
TOOLCHAIN_RUN_DEPS=(
make # generic build tools
tar gzip bzip2 xz-utils # used for archive unpacking
zlib1g # zlib library used for all builds
libexpat1 libipt1 libbabeltrace1 liblzma5 python3 # for gdb
libcurl4 # for cmake
libreadline7 # for cmake and llvm
libffi6 libxml2 # for llvm
)
MEMGRAPH_BUILD_DEPS=(
git # source code control
make pkg-config # build system
curl wget # downloading libs
uuid-dev default-jre-headless # required by antlr
libreadline-dev # memgraph console
libpython3-dev python3-dev # for query modules
libssl-dev
libseccomp-dev
python3 virtualenv python3-virtualenv python3-pip # qa, macro bench and stress tests
python3-yaml # the configuration generator
libcurl4-openssl-dev # mg-requests
sbcl # custom Lisp C++ preprocessing
doxygen graphviz # source documentation generators
mono-runtime mono-mcs nodejs zip unzip default-jdk-headless # driver tests
autoconf # for jemalloc code generation
)
list() {
echo "$1"
}
check() {
check_all_dpkg "$1"
}
install() {
apt install -y $1
}
deps=$2"[*]"
"$1" "${!deps}"

View File

@@ -1,85 +0,0 @@
#!/bin/bash
set -Eeuo pipefail
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
source "$DIR/../util.sh"
TOOLCHAIN_BUILD_DEPS=(
coreutils gcc g++ build-essential make # generic build tools
wget # used for archive download
gnupg # used for archive signature verification
tar gzip bzip2 xz-utils unzip # used for archive unpacking
zlib1g-dev # zlib library used for all builds
libexpat1-dev libipt-dev libbabeltrace-dev liblzma-dev python3-dev texinfo # for gdb
libcurl4-openssl-dev # for cmake
libreadline-dev # for cmake and llvm
libffi-dev libxml2-dev # for llvm
libedit-dev libpcre3-dev automake bison # for swig
)
TOOLCHAIN_RUN_DEPS=(
make # generic build tools
tar gzip bzip2 xz-utils # used for archive unpacking
zlib1g # zlib library used for all builds
libexpat1 libipt2 libbabeltrace1 liblzma5 python3 # for gdb
libcurl4 # for cmake
libreadline8 # for cmake and llvm
libffi7 libxml2 # for llvm
)
MEMGRAPH_BUILD_DEPS=(
git # source code control
make pkg-config # build system
curl wget # for downloading libs
uuid-dev default-jre-headless # required by antlr
libreadline-dev # for memgraph console
libpython3-dev python3-dev # for query modules
libssl-dev
libseccomp-dev
netcat # tests are using nc to wait for memgraph
python3 python3-virtualenv python3-pip # for qa, macro_benchmark and stress tests
python3-yaml # for the configuration generator
libcurl4-openssl-dev # mg-requests
sbcl # for custom Lisp C++ preprocessing
doxygen graphviz # source documentation generators
mono-runtime mono-mcs zip unzip default-jdk-headless # for driver tests
dotnet-sdk-3.1 golang nodejs npm
autoconf # for jemalloc code generation
)
list() {
echo "$1"
}
check() {
check_all_dpkg "$1"
}
install() {
cd "$DIR"
apt update
# If GitHub Actions runner is installed, append LANG to the environment.
# Python related tests doesn't work the LANG export.
if [ -d "/home/gh/actions-runner" ]; then
echo "LANG=en_US.utf8" >> /home/gh/actions-runner/.env
else
echo "NOTE: export LANG=en_US.utf8"
fi
apt install -y wget
for pkg in $1; do
if [ "$pkg" == dotnet-sdk-3.1 ]; then
if ! dpkg -s dotnet-sdk-3.1 2>/dev/null >/dev/null; then
wget -nv https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
dpkg -i packages-microsoft-prod.deb
apt-get update
apt-get install -y apt-transport-https dotnet-sdk-3.1
fi
continue
fi
apt install -y "$pkg"
done
}
deps=$2"[*]"
"$1" "${!deps}"

View File

@@ -1,651 +0,0 @@
#!/bin/bash -e
# helpers
pushd () { command pushd "$@" > /dev/null; }
popd () { command popd "$@" > /dev/null; }
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
CPUS=$( cat /proc/cpuinfo | grep processor | wc -l )
cd "$DIR"
# toolchain version
TOOLCHAIN_VERSION=1
# package versions used
GCC_VERSION=8.3.0
BINUTILS_VERSION=2.32
GDB_VERSION=8.2.1
CMAKE_VERSION=3.14.2
CPPCHECK_VERSION=1.87
LLVM_VERSION=8.0.0
SWIG_VERSION=3.0.12 # used only for LLVM compilation
# check for installed dependencies
DISTRO="$( egrep '^(VERSION_)?ID=' /etc/os-release | sort | cut -d '=' -f 2- | sed 's/"//g' | paste -s -d '-' )"
case "$DISTRO" in
debian-9)
DEPS_MANAGER=apt-get
DEPS_COMPILE=(
coreutils gcc g++ build-essential make # generic build tools
wget # used for archive download
gnupg # used for archive signature verification
tar gzip bzip2 xz-utils unzip # used for archive unpacking
zlib1g-dev # zlib library used for all builds
libexpat1-dev libipt-dev libbabeltrace-dev libbabeltrace-ctf-dev liblzma-dev python3-dev texinfo # for gdb
libcurl4-openssl-dev # for cmake
libreadline-dev # for cmake and llvm
libffi-dev libxml2-dev # for llvm
libedit-dev libpcre3-dev automake bison # for swig
)
DEPS_RUN=(
make # generic build tools
tar gzip bzip2 xz-utils # used for archive unpacking
zlib1g # zlib library used for all builds
libexpat1 libipt1 libbabeltrace1 libbabeltrace-ctf1 liblzma5 python3 # for gdb
libcurl3 # for cmake
libreadline7 # for cmake and llvm
libffi6 libxml2 # for llvm
)
;;
debian-10)
DEPS_MANAGER=apt-get
DEPS_COMPILE=(
coreutils gcc g++ build-essential make # generic build tools
wget # used for archive download
gnupg # used for archive signature verification
tar gzip bzip2 xz-utils unzip # used for archive unpacking
zlib1g-dev # zlib library used for all builds
libexpat1-dev libipt-dev libbabeltrace-dev liblzma-dev python3-dev texinfo # for gdb
libcurl4-openssl-dev # for cmake
libreadline-dev # for cmake and llvm
libffi-dev libxml2-dev # for llvm
libedit-dev libpcre3-dev automake bison # for swig
)
DEPS_RUN=(
make # generic build tools
tar gzip bzip2 xz-utils # used for archive unpacking
zlib1g # zlib library used for all builds
libexpat1 libipt2 libbabeltrace1 liblzma5 python3 # for gdb
libcurl4 # for cmake
libreadline7 # for cmake and llvm
libffi6 libxml2 # for llvm
)
;;
ubuntu-18.04)
DEPS_MANAGER=apt-get
DEPS_COMPILE=(
coreutils gcc g++ build-essential make # generic build tools
wget # used for archive download
gnupg # used for archive signature verification
tar gzip bzip2 xz-utils unzip # used for archive unpacking
zlib1g-dev # zlib library used for all builds
libexpat1-dev libipt-dev libbabeltrace-dev liblzma-dev python3-dev texinfo # for gdb
libcurl4-openssl-dev # for cmake
libreadline-dev # for cmake and llvm
libffi-dev libxml2-dev # for llvm
libedit-dev libpcre3-dev automake bison # for swig
)
DEPS_RUN=(
make # generic build tools
tar gzip bzip2 xz-utils # used for archive unpacking
zlib1g # zlib library used for all builds
libexpat1 libipt1 libbabeltrace1 liblzma5 python3 # for gdb
libcurl4 # for cmake
libreadline7 # for cmake and llvm
libffi6 libxml2 # for llvm
)
;;
centos-7)
DEPS_MANAGER=yum
DEPS_COMPILE=(
coreutils gcc gcc-c++ make # generic build tools
wget # used for archive download
gnupg2 # used for archive signature verification
tar gzip bzip2 xz unzip # used for archive unpacking
zlib-devel # zlib library used for all builds
expat-devel libipt-devel libbabeltrace-devel xz-devel python3-devel texinfo # for gdb
libcurl-devel # for cmake
readline-devel # for cmake and llvm
libffi-devel libxml2-devel # for llvm
libedit-devel pcre-devel automake bison # for swig
)
DEPS_RUN=(
make # generic build tools
tar gzip bzip2 xz # used for archive unpacking
zlib # zlib library used for all builds
expat libipt libbabeltrace xz-libs python3 # for gdb
readline # for cmake and llvm
libffi libxml2 # for llvm
)
;;
*)
echo "Unknown distribution: $DISTRO!"
exit 1
;;
esac
find_missing_dependencies () {
local message="$1"; shift
local missing=""
while [ "$1" != "" ]; do
if [ "$DEPS_MANAGER" == "apt-get" ]; then
if ! dpkg -s $1 >/dev/null 2>/dev/null; then
missing="$1 $missing"
fi
elif [ "$DEPS_MANAGER" == "yum" ]; then
if ! yum list installed $1 >/dev/null 2>/dev/null; then
missing="$1 $missing"
fi
else
echo "Invalid package manager: $DEPS_MANAGER!"
exit 1
fi
shift
done
if [ "$missing" != "" ]; then
echo "$message: $missing"
exit 1
fi
}
find_missing_dependencies "Missing dependencies" ${DEPS_COMPILE[@]}
find_missing_dependencies "All dependencies are installed, but the following runtime libraries were not found (they are probably invalid)" ${DEPS_RUN[@]}
# check installation directory
NAME=toolchain-v$TOOLCHAIN_VERSION
PREFIX=/opt/$NAME
mkdir -p $PREFIX >/dev/null 2>/dev/null || true
if [ ! -d $PREFIX ] || [ ! -w $PREFIX ]; then
echo "Please make sure that the directory '$PREFIX' exists and is writable by the current user!"
echo
echo "If unsure, execute these commands as root:"
echo " mkdir $PREFIX && chown $USER:$USER $PREFIX"
echo
echo "Press <return> when you have created the directory and granted permissions."
# wait for the directory to be created
while true; do
read
if [ ! -d $PREFIX ] || [ ! -w $PREFIX ]; then
echo
echo "You can't continue before you have created the directory and granted permissions!"
echo
echo "Press <return> when you have created the directory and granted permissions."
else
break
fi
done
fi
# create archives directory
mkdir -p archives
# download all archives
pushd archives
if [ ! -f gcc-$GCC_VERSION.tar.gz ]; then
wget https://ftp.gnu.org/gnu/gcc/gcc-$GCC_VERSION/gcc-$GCC_VERSION.tar.gz
fi
if [ ! -f binutils-$BINUTILS_VERSION.tar.gz ]; then
wget https://ftp.gnu.org/gnu/binutils/binutils-$BINUTILS_VERSION.tar.gz
fi
if [ ! -f gdb-$GDB_VERSION.tar.gz ]; then
wget https://ftp.gnu.org/gnu/gdb/gdb-$GDB_VERSION.tar.gz
fi
if [ ! -f cmake-$CMAKE_VERSION.tar.gz ]; then
wget https://github.com/Kitware/CMake/releases/download/v$CMAKE_VERSION/cmake-$CMAKE_VERSION.tar.gz
fi
if [ ! -f swig-$SWIG_VERSION.tar.gz ]; then
wget https://github.com/swig/swig/archive/rel-$SWIG_VERSION.tar.gz -O swig-$SWIG_VERSION.tar.gz
fi
if [ ! -f cppcheck-$CPPCHECK_VERSION.tar.gz ]; then
wget https://github.com/danmar/cppcheck/archive/$CPPCHECK_VERSION.tar.gz -O cppcheck-$CPPCHECK_VERSION.tar.gz
fi
if [ ! -f llvm-$LLVM_VERSION.src.tar.xz ]; then
wget https://releases.llvm.org/$LLVM_VERSION/llvm-$LLVM_VERSION.src.tar.xz
wget https://releases.llvm.org/$LLVM_VERSION/cfe-$LLVM_VERSION.src.tar.xz
wget https://releases.llvm.org/$LLVM_VERSION/lld-$LLVM_VERSION.src.tar.xz
wget https://releases.llvm.org/$LLVM_VERSION/compiler-rt-$LLVM_VERSION.src.tar.xz
wget https://releases.llvm.org/$LLVM_VERSION/clang-tools-extra-$LLVM_VERSION.src.tar.xz
fi
if [ ! -f pahole-gdb-master.zip ]; then
wget https://github.com/PhilArmstrong/pahole-gdb/archive/master.zip -O pahole-gdb-master.zip
fi
# verify all archives
# NOTE: Verification can fail if the archive is signed by another developer. I
# haven't added commands to download all developer GnuPG keys because the
# download is very slow. If the verification fails for you, figure out who has
# signed the archive and download their public key instead.
GPG="gpg --homedir .gnupg"
KEYSERVER="hkp://keyserver.ubuntu.com"
mkdir -p .gnupg
chmod 700 .gnupg
# verify gcc
if [ ! -f gcc-$GCC_VERSION.tar.gz.sig ]; then
wget https://ftp.gnu.org/gnu/gcc/gcc-$GCC_VERSION/gcc-$GCC_VERSION.tar.gz.sig
fi
# list of valid gcc gnupg keys: https://gcc.gnu.org/mirrors.html
$GPG --keyserver $KEYSERVER --recv-keys 0xA328C3A2C3C45C06
$GPG --verify gcc-$GCC_VERSION.tar.gz.sig gcc-$GCC_VERSION.tar.gz
# verify binutils
if [ ! -f binutils-$BINUTILS_VERSION.tar.gz.sig ]; then
wget https://ftp.gnu.org/gnu/binutils/binutils-$BINUTILS_VERSION.tar.gz.sig
fi
$GPG --keyserver $KEYSERVER --recv-keys 0xDD9E3C4F
$GPG --verify binutils-$BINUTILS_VERSION.tar.gz.sig binutils-$BINUTILS_VERSION.tar.gz
# verify gdb
if [ ! -f gdb-$GDB_VERSION.tar.gz.sig ]; then
wget https://ftp.gnu.org/gnu/gdb/gdb-$GDB_VERSION.tar.gz.sig
fi
$GPG --keyserver $KEYSERVER --recv-keys 0xFF325CF3
$GPG --verify gdb-$GDB_VERSION.tar.gz.sig gdb-$GDB_VERSION.tar.gz
# verify cmake
if [ ! -f cmake-$CMAKE_VERSION-SHA-256.txt ] || [ ! -f cmake-$CMAKE_VERSION-SHA-256.txt.asc ]; then
wget https://github.com/Kitware/CMake/releases/download/v$CMAKE_VERSION/cmake-$CMAKE_VERSION-SHA-256.txt
wget https://github.com/Kitware/CMake/releases/download/v$CMAKE_VERSION/cmake-$CMAKE_VERSION-SHA-256.txt.asc
# Because CentOS 7 doesn't have the `--ignore-missing` flag for `sha256sum`
# we filter out the missing files from the sums here manually.
cat cmake-$CMAKE_VERSION-SHA-256.txt | grep "cmake-$CMAKE_VERSION.tar.gz" > cmake-$CMAKE_VERSION-SHA-256-filtered.txt
fi
$GPG --keyserver $KEYSERVER --recv-keys 0xC6C265324BBEBDC350B513D02D2CEF1034921684
sha256sum -c cmake-$CMAKE_VERSION-SHA-256-filtered.txt
$GPG --verify cmake-$CMAKE_VERSION-SHA-256.txt.asc cmake-$CMAKE_VERSION-SHA-256.txt
# verify llvm, cfe, lld, clang-tools-extra
if [ ! -f llvm-$LLVM_VERSION.src.tar.xz.sig ]; then
wget https://releases.llvm.org/$LLVM_VERSION/llvm-$LLVM_VERSION.src.tar.xz.sig
wget https://releases.llvm.org/$LLVM_VERSION/cfe-$LLVM_VERSION.src.tar.xz.sig
wget https://releases.llvm.org/$LLVM_VERSION/lld-$LLVM_VERSION.src.tar.xz.sig
wget https://releases.llvm.org/$LLVM_VERSION/compiler-rt-$LLVM_VERSION.src.tar.xz.sig
wget https://releases.llvm.org/$LLVM_VERSION/clang-tools-extra-$LLVM_VERSION.src.tar.xz.sig
fi
# list of valid llvm gnupg keys: https://releases.llvm.org/download.html
$GPG --keyserver $KEYSERVER --recv-keys 0x345AD05D
$GPG --verify llvm-$LLVM_VERSION.src.tar.xz.sig llvm-$LLVM_VERSION.src.tar.xz
$GPG --verify cfe-$LLVM_VERSION.src.tar.xz.sig cfe-$LLVM_VERSION.src.tar.xz
$GPG --verify lld-$LLVM_VERSION.src.tar.xz.sig lld-$LLVM_VERSION.src.tar.xz
$GPG --verify compiler-rt-$LLVM_VERSION.src.tar.xz.sig compiler-rt-$LLVM_VERSION.src.tar.xz
$GPG --verify clang-tools-extra-$LLVM_VERSION.src.tar.xz.sig clang-tools-extra-$LLVM_VERSION.src.tar.xz
popd
# create build directory
mkdir -p build
pushd build
# compile gcc
if [ ! -f $PREFIX/bin/gcc ]; then
if [ -d gcc-$GCC_VERSION ]; then
rm -rf gcc-$GCC_VERSION
fi
tar -xvf ../archives/gcc-$GCC_VERSION.tar.gz
pushd gcc-$GCC_VERSION
./contrib/download_prerequisites
mkdir build && pushd build
# influenced by: https://buildd.debian.org/status/fetch.php?pkg=gcc-8&arch=amd64&ver=8.3.0-6&stamp=1554588545
../configure -v \
--build=x86_64-linux-gnu \
--host=x86_64-linux-gnu \
--target=x86_64-linux-gnu \
--prefix=$PREFIX \
--disable-multilib \
--with-system-zlib \
--enable-checking=release \
--enable-languages=c,c++,fortran \
--enable-gold=yes \
--enable-ld=yes \
--enable-lto \
--enable-bootstrap \
--disable-vtable-verify \
--disable-werror \
--without-included-gettext \
--enable-threads=posix \
--enable-nls \
--enable-clocale=gnu \
--enable-libstdcxx-debug \
--enable-libstdcxx-time=yes \
--enable-gnu-unique-object \
--enable-libmpx \
--enable-plugin \
--enable-default-pie \
--with-target-system-zlib \
--with-tune=generic \
--without-cuda-driver
#--program-suffix=$( printf "$GCC_VERSION" | cut -d '.' -f 1,2 ) \
make -j$CPUS
# make -k check # run test suite
make install
popd && popd
fi
# activate toolchain
export PATH=$PREFIX/bin:$PATH
export LD_LIBRARY_PATH=$PREFIX/lib64
# compile binutils
if [ ! -f $PREFIX/bin/ld.gold ]; then
if [ -d binutils-$BINUTILS_VERSION ]; then
rm -rf binutils-$BINUTILS_VERSION
fi
tar -xvf ../archives/binutils-$BINUTILS_VERSION.tar.gz
pushd binutils-$BINUTILS_VERSION
mkdir build && pushd build
# influenced by: https://buildd.debian.org/status/fetch.php?pkg=binutils&arch=amd64&ver=2.32-7&stamp=1553247092
env \
CC=gcc \
CXX=g++ \
CFLAGS="-g -O2" \
CXXFLAGS="-g -O2" \
LDFLAGS="" \
../configure \
--build=x86_64-linux-gnu \
--host=x86_64-linux-gnu \
--prefix=$PREFIX \
--enable-ld=default \
--enable-gold \
--enable-lto \
--enable-plugins \
--enable-shared \
--enable-threads \
--with-system-zlib \
--enable-deterministic-archives \
--disable-compressed-debug-sections \
--enable-new-dtags \
--disable-werror
make -j$CPUS
# make -k check # run test suite
make install
popd && popd
fi
# compile gdb
if [ ! -f $PREFIX/bin/gdb ]; then
if [ -d gdb-$GDB_VERSION ]; then
rm -rf gdb-$GDB_VERSION
fi
tar -xvf ../archives/gdb-$GDB_VERSION.tar.gz
pushd gdb-$GDB_VERSION
mkdir build && pushd build
# https://buildd.debian.org/status/fetch.php?pkg=gdb&arch=amd64&ver=8.2.1-2&stamp=1550831554&raw=0
env \
CC=gcc \
CXX=g++ \
CFLAGS="-g -O2 -fstack-protector-strong -Wformat -Werror=format-security" \
CXXFLAGS="-g -O2 -fstack-protector-strong -Wformat -Werror=format-security" \
CPPFLAGS="-Wdate-time -D_FORTIFY_SOURCE=2 -fPIC" \
LDFLAGS="-Wl,-z,relro" \
PYTHON="" \
../configure \
--build=x86_64-linux-gnu \
--host=x86_64-linux-gnu \
--prefix=$PREFIX \
--disable-maintainer-mode \
--disable-dependency-tracking \
--disable-silent-rules \
--disable-gdbtk \
--disable-shared \
--without-guile \
--with-system-gdbinit=$PREFIX/etc/gdb/gdbinit \
--with-system-readline \
--with-expat \
--with-system-zlib \
--with-lzma \
--with-babeltrace \
--with-intel-pt \
--enable-tui \
--with-python=python3
make -j$CPUS
make install
popd && popd
fi
# install pahole
if [ ! -d $PREFIX/share/pahole-gdb ]; then
unzip ../archives/pahole-gdb-master.zip
mv pahole-gdb-master $PREFIX/share/pahole-gdb
fi
# setup system gdbinit
if [ ! -f $PREFIX/etc/gdb/gdbinit ]; then
mkdir -p $PREFIX/etc/gdb
cat >$PREFIX/etc/gdb/gdbinit <<EOF
# improve formatting
set print pretty on
set print object on
set print static-members on
set print vtbl on
set print demangle on
set demangle-style gnu-v3
set print sevenbit-strings off
# load libstdc++ pretty printers
add-auto-load-scripts-directory $PREFIX/lib64
add-auto-load-safe-path $PREFIX
# load pahole
python
sys.path.insert(0, "$PREFIX/share/pahole-gdb")
import offsets
import pahole
end
EOF
fi
# compile cmake
if [ ! -f $PREFIX/bin/cmake ]; then
if [ -d cmake-$CMAKE_VERSION ]; then
rm -rf cmake-$CMAKE_VERSION
fi
tar -xvf ../archives/cmake-$CMAKE_VERSION.tar.gz
pushd cmake-$CMAKE_VERSION
# influenced by: https://buildd.debian.org/status/fetch.php?pkg=cmake&arch=amd64&ver=3.13.4-1&stamp=1549799837
echo 'set(CMAKE_SKIP_RPATH ON CACHE BOOL "Skip rpath" FORCE)' >> build-flags.cmake
echo 'set(CMAKE_USE_RELATIVE_PATHS ON CACHE BOOL "Use relative paths" FORCE)' >> build-flags.cmake
echo 'set(CMAKE_C_FLAGS "-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2" CACHE STRING "C flags" FORCE)' >> build-flags.cmake
echo 'set(CMAKE_CXX_FLAGS "-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2" CACHE STRING "C++ flags" FORCE)' >> build-flags.cmake
echo 'set(CMAKE_SKIP_BOOTSTRAP_TEST ON CACHE BOOL "Skip BootstrapTest" FORCE)' >> build-flags.cmake
echo 'set(BUILD_CursesDialog ON CACHE BOOL "Build curses GUI" FORCE)' >> build-flags.cmake
mkdir build && pushd build
../bootstrap \
--prefix=$PREFIX \
--init=../build-flags.cmake \
--parallel=$CPUS \
--system-curl
make -j$CPUS
# make test # run test suite
make install
popd && popd
fi
# compile cppcheck
if [ ! -f $PREFIX/bin/cppcheck ]; then
if [ -d cppcheck-$CPPCHECK_VERSION ]; then
rm -rf cppcheck-$CPPCHECK_VERSION
fi
tar -xvf ../archives/cppcheck-$CPPCHECK_VERSION.tar.gz
pushd cppcheck-$CPPCHECK_VERSION
env \
CC=gcc \
CXX=g++ \
PREFIX=$PREFIX \
CFGDIR=$PREFIX/share/cppcheck/cfg \
make -j$CPUS
env \
CC=gcc \
CXX=g++ \
PREFIX=$PREFIX \
CFGDIR=$PREFIX/share/cppcheck/cfg \
make install
popd
fi
# compile swig
if [ ! -d swig-$SWIG_VERSION/install ]; then
if [ -d swig-$SWIG_VERSION ]; then
rm -rf swig-$SWIG_VERSION
fi
tar -xvf ../archives/swig-$SWIG_VERSION.tar.gz
mv swig-rel-$SWIG_VERSION swig-$SWIG_VERSION
pushd swig-$SWIG_VERSION
./autogen.sh
mkdir build && pushd build
../configure --prefix=$DIR/build/swig-$SWIG_VERSION/install
make -j$CPUS
make install
popd && popd
fi
# compile llvm
if [ ! -f $PREFIX/bin/clang ]; then
if [ -d llvm-$LLVM_VERSION ]; then
rm -rf llvm-$LLVM_VERSION
fi
tar -xvf ../archives/llvm-$LLVM_VERSION.src.tar.xz
mv llvm-$LLVM_VERSION.src llvm-$LLVM_VERSION
tar -xvf ../archives/cfe-$LLVM_VERSION.src.tar.xz
mv cfe-$LLVM_VERSION.src llvm-$LLVM_VERSION/tools/clang
tar -xvf ../archives/lld-$LLVM_VERSION.src.tar.xz
mv lld-$LLVM_VERSION.src/ llvm-$LLVM_VERSION/tools/lld
tar -xvf ../archives/compiler-rt-$LLVM_VERSION.src.tar.xz
mv compiler-rt-$LLVM_VERSION.src/ llvm-$LLVM_VERSION/projects/compiler-rt
tar -xvf ../archives/clang-tools-extra-$LLVM_VERSION.src.tar.xz
mv clang-tools-extra-$LLVM_VERSION.src/ llvm-$LLVM_VERSION/tools/clang/tools/extra
pushd llvm-$LLVM_VERSION
mkdir build && pushd build
# activate swig
export PATH=$DIR/build/swig-$SWIG_VERSION/install/bin:$PATH
# influenced by: https://buildd.debian.org/status/fetch.php?pkg=llvm-toolchain-7&arch=amd64&ver=1%3A7.0.1%7E%2Brc2-1%7Eexp1&stamp=1541506173&raw=0
cmake .. \
-DGCC_INSTALL_PREFIX=$PREFIX \
-DCMAKE_C_COMPILER=$PREFIX/bin/gcc \
-DCMAKE_CXX_COMPILER=$PREFIX/bin/g++ \
-DCMAKE_CXX_LINK_FLAGS="-L$PREFIX/lib64 -Wl,-rpath,$PREFIX/lib64" \
-DCMAKE_INSTALL_PREFIX=$PREFIX \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_CXX_FLAGS_RELWITHDEBINFO="-O2 -DNDEBUG" \
-DCMAKE_CXX_FLAGS=' -fuse-ld=gold -fPIC -Wno-unused-command-line-argument -Wno-unknown-warning-option' \
-DCMAKE_C_FLAGS=' -fuse-ld=gold -fPIC -Wno-unused-command-line-argument -Wno-unknown-warning-option' \
-DLLVM_LINK_LLVM_DYLIB=ON \
-DLLVM_INSTALL_UTILS=ON \
-DLLVM_VERSION_SUFFIX= \
-DLLVM_BUILD_LLVM_DYLIB=ON \
-DLLVM_ENABLE_RTTI=ON \
-DLLVM_ENABLE_FFI=ON \
-DLLVM_BINUTILS_INCDIR=$PREFIX/include/ \
-DLLVM_USE_PERF=yes \
-DLIBCLANG_LIBRARY_VERSION=1 \
-DCLANG_ENABLE_BOOTSTRAP=ON
make -j$CPUS
make -j$CPUS check-clang # run clang test suite
make -j$CPUS check-lld # run lld test suite
make install
popd && popd
fi
# create README
if [ ! -f $PREFIX/README.md ]; then
cat >$PREFIX/README.md <<EOF
# Memgraph Toolchain v$TOOLCHAIN_VERSION
## Included tools
- GCC $GCC_VERSION
- Binutils $BINUTILS_VERSION
- GDB $GDB_VERSION
- CMake $CMAKE_VERSION
- Cppcheck $CPPCHECK_VERSION
- LLVM (Clang, LLD, compiler-rt, Clang tools extra) $LLVM_VERSION
## Required libraries
In order to be able to run all of these tools you should install the following
packages:
\`\`\`
$DEPS_MANAGER install ${DEPS_RUN[@]}
\`\`\`
## Usage
In order to use the toolchain you just have to source the activation script:
\`\`\`
source $PREFIX/activate
\`\`\`
EOF
fi
# create activation script
if [ ! -f $PREFIX/activate ]; then
cat >$PREFIX/activate <<EOF
# This file must be used with "source $PREFIX/activate" *from bash*
# You can't run it directly!
# check for active virtual environments
if [ "\$( type -t deactivate )" != "" ]; then
echo "You already have an active virtual environment!"
return 0
fi
# check that we aren't root
if [ "\$USER" == "root" ]; then
echo "You shouldn't use the toolchan as root!"
return 0
fi
# save original environment
export ORIG_PATH=\$PATH
export ORIG_PS1=\$PS1
export ORIG_LD_LIBRARY_PATH=\$LD_LIBRARY_PATH
# activate new environment
export PATH=$PREFIX/bin:\$PATH
export PS1="(TOOLCHAIN) \$PS1"
export LD_LIBRARY_PATH=$PREFIX/lib:$PREFIX/lib64
# disable root
function su () {
echo "You don't want to use root functions while using the toolchain!"
return 1
}
function sudo () {
echo "You don't want to use root functions while using the toolchain!"
return 1
}
# create deactivation function
function deactivate() {
export PATH=\$ORIG_PATH
export PS1=\$ORIG_PS1
export LD_LIBRARY_PATH=\$ORIG_LD_LIBRARY_PATH
unset ORIG_PATH ORIG_PS1 ORIG_LD_LIBRARY_PATH
unset -f su sudo deactivate
}
EOF
fi
# create toolchain archive
if [ ! -f $NAME-binaries-$DISTRO.tar.gz ]; then
tar --owner=root --group=root -cpvzf $NAME-binaries-$DISTRO.tar.gz -C /opt $NAME
fi
# output final instructions
echo -e "\n\n"
echo "All tools have been built. They are installed in '$PREFIX'."
echo "In order to distribute the tools to someone else, an archive with the toolchain was created in the 'build' directory."
echo "If you want to install the packed tools you should execute the following command:"
echo
echo " tar -xvzf build/$NAME-binaries.tar.gz -C /opt"
echo
echo "Because the tools were built on this machine, you should probably change the permissions of the installation directory using:"
echo
echo " OPTIONAL: chown -R root:root $PREFIX"
echo
echo "In order to use all of the newly compiled tools you should use the prepared activation script:"
echo
echo " source $PREFIX/activate"
echo
echo "Or, for more advanced uses, you can add the following lines to your script:"
echo
echo " export PATH=$PREFIX/bin:\$PATH"
echo " export LD_LIBRARY_PATH=$PREFIX/lib:$PREFIX/lib64"
echo
echo "Enjoy!"

View File

@@ -1,533 +0,0 @@
#!/bin/bash -e
# helpers
pushd () { command pushd "$@" > /dev/null; }
popd () { command popd "$@" > /dev/null; }
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
CPUS=$( grep -c processor < /proc/cpuinfo )
cd "$DIR"
source "$DIR/../util.sh"
DISTRO="$(operating_system)"
# toolchain version
TOOLCHAIN_VERSION=2
# package versions used
GCC_VERSION=10.2.0
BINUTILS_VERSION=2.35.1
case "$DISTRO" in
centos-7) # because GDB >= 9 does NOT compile with readline6.
GDB_VERSION=8.3
;;
*)
GDB_VERSION=10.1
;;
esac
CMAKE_VERSION=3.18.4
CPPCHECK_VERSION=2.2
LLVM_VERSION=11.0.0
SWIG_VERSION=4.0.2 # used only for LLVM compilation
# Check for the dependencies.
echo "ALL BUILD PACKAGES: $($DIR/../os/$DISTRO.sh list TOOLCHAIN_BUILD_DEPS)"
$DIR/../os/$DISTRO.sh check TOOLCHAIN_BUILD_DEPS
echo "ALL RUN PACKAGES: $($DIR/../os/$DISTRO.sh list TOOLCHAIN_RUN_DEPS)"
$DIR/../os/$DISTRO.sh check TOOLCHAIN_RUN_DEPS
# check installation directory
NAME=toolchain-v$TOOLCHAIN_VERSION
PREFIX=/opt/$NAME
mkdir -p $PREFIX >/dev/null 2>/dev/null || true
if [ ! -d $PREFIX ] || [ ! -w $PREFIX ]; then
echo "Please make sure that the directory '$PREFIX' exists and is writable by the current user!"
echo
echo "If unsure, execute these commands as root:"
echo " mkdir $PREFIX && chown $USER:$USER $PREFIX"
echo
echo "Press <return> when you have created the directory and granted permissions."
# wait for the directory to be created
while true; do
read
if [ ! -d $PREFIX ] || [ ! -w $PREFIX ]; then
echo
echo "You can't continue before you have created the directory and granted permissions!"
echo
echo "Press <return> when you have created the directory and granted permissions."
else
break
fi
done
fi
# create archives directory
mkdir -p archives
# download all archives
pushd archives
if [ ! -f gcc-$GCC_VERSION.tar.gz ]; then
wget https://ftp.gnu.org/gnu/gcc/gcc-$GCC_VERSION/gcc-$GCC_VERSION.tar.gz
fi
if [ ! -f binutils-$BINUTILS_VERSION.tar.gz ]; then
wget https://ftp.gnu.org/gnu/binutils/binutils-$BINUTILS_VERSION.tar.gz
fi
if [ ! -f gdb-$GDB_VERSION.tar.gz ]; then
wget https://ftp.gnu.org/gnu/gdb/gdb-$GDB_VERSION.tar.gz
fi
if [ ! -f cmake-$CMAKE_VERSION.tar.gz ]; then
wget https://github.com/Kitware/CMake/releases/download/v$CMAKE_VERSION/cmake-$CMAKE_VERSION.tar.gz
fi
if [ ! -f swig-$SWIG_VERSION.tar.gz ]; then
wget https://github.com/swig/swig/archive/rel-$SWIG_VERSION.tar.gz -O swig-$SWIG_VERSION.tar.gz
fi
if [ ! -f cppcheck-$CPPCHECK_VERSION.tar.gz ]; then
wget https://github.com/danmar/cppcheck/archive/$CPPCHECK_VERSION.tar.gz -O cppcheck-$CPPCHECK_VERSION.tar.gz
fi
if [ ! -f llvm-$LLVM_VERSION.src.tar.xz ]; then
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/llvm-$LLVM_VERSION.src.tar.xz
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/clang-$LLVM_VERSION.src.tar.xz
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/lld-$LLVM_VERSION.src.tar.xz
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/clang-tools-extra-$LLVM_VERSION.src.tar.xz
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/compiler-rt-$LLVM_VERSION.src.tar.xz
fi
if [ ! -f pahole-gdb-master.zip ]; then
wget https://github.com/PhilArmstrong/pahole-gdb/archive/master.zip -O pahole-gdb-master.zip
fi
# verify all archives
# NOTE: Verification can fail if the archive is signed by another developer. I
# haven't added commands to download all developer GnuPG keys because the
# download is very slow. If the verification fails for you, figure out who has
# signed the archive and download their public key instead.
GPG="gpg --homedir .gnupg"
KEYSERVER="hkp://keyserver.ubuntu.com"
mkdir -p .gnupg
chmod 700 .gnupg
# verify gcc
if [ ! -f gcc-$GCC_VERSION.tar.gz.sig ]; then
wget https://ftp.gnu.org/gnu/gcc/gcc-$GCC_VERSION/gcc-$GCC_VERSION.tar.gz.sig
fi
# list of valid gcc gnupg keys: https://gcc.gnu.org/mirrors.html
$GPG --keyserver $KEYSERVER --recv-keys 0x3AB00996FC26A641
$GPG --verify gcc-$GCC_VERSION.tar.gz.sig gcc-$GCC_VERSION.tar.gz
# verify binutils
if [ ! -f binutils-$BINUTILS_VERSION.tar.gz.sig ]; then
wget https://ftp.gnu.org/gnu/binutils/binutils-$BINUTILS_VERSION.tar.gz.sig
fi
$GPG --keyserver $KEYSERVER --recv-keys 0xDD9E3C4F
$GPG --verify binutils-$BINUTILS_VERSION.tar.gz.sig binutils-$BINUTILS_VERSION.tar.gz
# verify gdb
if [ ! -f gdb-$GDB_VERSION.tar.gz.sig ]; then
wget https://ftp.gnu.org/gnu/gdb/gdb-$GDB_VERSION.tar.gz.sig
fi
$GPG --keyserver $KEYSERVER --recv-keys 0xFF325CF3
$GPG --verify gdb-$GDB_VERSION.tar.gz.sig gdb-$GDB_VERSION.tar.gz
# verify cmake
if [ ! -f cmake-$CMAKE_VERSION-SHA-256.txt ] || [ ! -f cmake-$CMAKE_VERSION-SHA-256.txt.asc ]; then
wget https://github.com/Kitware/CMake/releases/download/v$CMAKE_VERSION/cmake-$CMAKE_VERSION-SHA-256.txt
wget https://github.com/Kitware/CMake/releases/download/v$CMAKE_VERSION/cmake-$CMAKE_VERSION-SHA-256.txt.asc
# Because CentOS 7 doesn't have the `--ignore-missing` flag for `sha256sum`
# we filter out the missing files from the sums here manually.
cat cmake-$CMAKE_VERSION-SHA-256.txt | grep "cmake-$CMAKE_VERSION.tar.gz" > cmake-$CMAKE_VERSION-SHA-256-filtered.txt
fi
$GPG --keyserver $KEYSERVER --recv-keys 0xC6C265324BBEBDC350B513D02D2CEF1034921684
sha256sum -c cmake-$CMAKE_VERSION-SHA-256-filtered.txt
$GPG --verify cmake-$CMAKE_VERSION-SHA-256.txt.asc cmake-$CMAKE_VERSION-SHA-256.txt
# verify llvm, cfe, lld, clang-tools-extra
if [ ! -f llvm-$LLVM_VERSION.src.tar.xz.sig ]; then
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/llvm-$LLVM_VERSION.src.tar.xz.sig
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/clang-$LLVM_VERSION.src.tar.xz.sig
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/lld-$LLVM_VERSION.src.tar.xz.sig
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/clang-tools-extra-$LLVM_VERSION.src.tar.xz.sig
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/compiler-rt-$LLVM_VERSION.src.tar.xz.sig
fi
# list of valid llvm gnupg keys: https://releases.llvm.org/download.html
$GPG --keyserver $KEYSERVER --recv-keys 0x345AD05D
$GPG --verify llvm-$LLVM_VERSION.src.tar.xz.sig llvm-$LLVM_VERSION.src.tar.xz
$GPG --verify clang-$LLVM_VERSION.src.tar.xz.sig clang-$LLVM_VERSION.src.tar.xz
$GPG --verify lld-$LLVM_VERSION.src.tar.xz.sig lld-$LLVM_VERSION.src.tar.xz
$GPG --verify clang-tools-extra-$LLVM_VERSION.src.tar.xz.sig clang-tools-extra-$LLVM_VERSION.src.tar.xz
$GPG --verify compiler-rt-$LLVM_VERSION.src.tar.xz.sig compiler-rt-$LLVM_VERSION.src.tar.xz
popd
# create build directory
mkdir -p build
pushd build
# compile gcc
if [ ! -f $PREFIX/bin/gcc ]; then
if [ -d gcc-$GCC_VERSION ]; then
rm -rf gcc-$GCC_VERSION
fi
tar -xvf ../archives/gcc-$GCC_VERSION.tar.gz
pushd gcc-$GCC_VERSION
./contrib/download_prerequisites
mkdir build && pushd build
# influenced by: https://buildd.debian.org/status/fetch.php?pkg=gcc-8&arch=amd64&ver=8.3.0-6&stamp=1554588545
../configure -v \
--build=x86_64-linux-gnu \
--host=x86_64-linux-gnu \
--target=x86_64-linux-gnu \
--prefix=$PREFIX \
--disable-multilib \
--with-system-zlib \
--enable-checking=release \
--enable-languages=c,c++,fortran \
--enable-gold=yes \
--enable-ld=yes \
--enable-lto \
--enable-bootstrap \
--disable-vtable-verify \
--disable-werror \
--without-included-gettext \
--enable-threads=posix \
--enable-nls \
--enable-clocale=gnu \
--enable-libstdcxx-debug \
--enable-libstdcxx-time=yes \
--enable-gnu-unique-object \
--enable-libmpx \
--enable-plugin \
--enable-default-pie \
--with-target-system-zlib \
--with-tune=generic \
--without-cuda-driver
#--program-suffix=$( printf "$GCC_VERSION" | cut -d '.' -f 1,2 ) \
make -j$CPUS
# make -k check # run test suite
make install
popd && popd
fi
# activate toolchain
export PATH=$PREFIX/bin:$PATH
export LD_LIBRARY_PATH=$PREFIX/lib64
# compile binutils
if [ ! -f $PREFIX/bin/ld.gold ]; then
if [ -d binutils-$BINUTILS_VERSION ]; then
rm -rf binutils-$BINUTILS_VERSION
fi
tar -xvf ../archives/binutils-$BINUTILS_VERSION.tar.gz
pushd binutils-$BINUTILS_VERSION
mkdir build && pushd build
# influenced by: https://buildd.debian.org/status/fetch.php?pkg=binutils&arch=amd64&ver=2.32-7&stamp=1553247092
env \
CC=gcc \
CXX=g++ \
CFLAGS="-g -O2" \
CXXFLAGS="-g -O2" \
LDFLAGS="" \
../configure \
--build=x86_64-linux-gnu \
--host=x86_64-linux-gnu \
--prefix=$PREFIX \
--enable-ld=default \
--enable-gold \
--enable-lto \
--enable-plugins \
--enable-shared \
--enable-threads \
--with-system-zlib \
--enable-deterministic-archives \
--disable-compressed-debug-sections \
--enable-new-dtags \
--disable-werror
make -j$CPUS
# make -k check # run test suite
make install
popd && popd
fi
# compile gdb
if [ ! -f $PREFIX/bin/gdb ]; then
if [ -d gdb-$GDB_VERSION ]; then
rm -rf gdb-$GDB_VERSION
fi
tar -xvf ../archives/gdb-$GDB_VERSION.tar.gz
pushd gdb-$GDB_VERSION
mkdir build && pushd build
# https://buildd.debian.org/status/fetch.php?pkg=gdb&arch=amd64&ver=8.2.1-2&stamp=1550831554&raw=0
env \
CC=gcc \
CXX=g++ \
CFLAGS="-g -O2 -fstack-protector-strong -Wformat -Werror=format-security" \
CXXFLAGS="-g -O2 -fstack-protector-strong -Wformat -Werror=format-security" \
CPPFLAGS="-Wdate-time -D_FORTIFY_SOURCE=2 -fPIC" \
LDFLAGS="-Wl,-z,relro" \
PYTHON="" \
../configure \
--build=x86_64-linux-gnu \
--host=x86_64-linux-gnu \
--prefix=$PREFIX \
--disable-maintainer-mode \
--disable-dependency-tracking \
--disable-silent-rules \
--disable-gdbtk \
--disable-shared \
--without-guile \
--with-system-gdbinit=$PREFIX/etc/gdb/gdbinit \
--with-system-readline \
--with-expat \
--with-system-zlib \
--with-lzma \
--with-babeltrace \
--with-intel-pt \
--enable-tui \
--with-python=python3
make -j$CPUS
make install
popd && popd
fi
# install pahole
if [ ! -d $PREFIX/share/pahole-gdb ]; then
unzip ../archives/pahole-gdb-master.zip
mv pahole-gdb-master $PREFIX/share/pahole-gdb
fi
# setup system gdbinit
if [ ! -f $PREFIX/etc/gdb/gdbinit ]; then
mkdir -p $PREFIX/etc/gdb
cat >$PREFIX/etc/gdb/gdbinit <<EOF
# improve formatting
set print pretty on
set print object on
set print static-members on
set print vtbl on
set print demangle on
set demangle-style gnu-v3
set print sevenbit-strings off
# load libstdc++ pretty printers
add-auto-load-scripts-directory $PREFIX/lib64
add-auto-load-safe-path $PREFIX
# load pahole
python
sys.path.insert(0, "$PREFIX/share/pahole-gdb")
import offsets
import pahole
end
EOF
fi
# compile cmake
if [ ! -f $PREFIX/bin/cmake ]; then
if [ -d cmake-$CMAKE_VERSION ]; then
rm -rf cmake-$CMAKE_VERSION
fi
tar -xvf ../archives/cmake-$CMAKE_VERSION.tar.gz
pushd cmake-$CMAKE_VERSION
# influenced by: https://buildd.debian.org/status/fetch.php?pkg=cmake&arch=amd64&ver=3.13.4-1&stamp=1549799837
echo 'set(CMAKE_SKIP_RPATH ON CACHE BOOL "Skip rpath" FORCE)' >> build-flags.cmake
echo 'set(CMAKE_USE_RELATIVE_PATHS ON CACHE BOOL "Use relative paths" FORCE)' >> build-flags.cmake
echo 'set(CMAKE_C_FLAGS "-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2" CACHE STRING "C flags" FORCE)' >> build-flags.cmake
echo 'set(CMAKE_CXX_FLAGS "-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2" CACHE STRING "C++ flags" FORCE)' >> build-flags.cmake
echo 'set(CMAKE_SKIP_BOOTSTRAP_TEST ON CACHE BOOL "Skip BootstrapTest" FORCE)' >> build-flags.cmake
echo 'set(BUILD_CursesDialog ON CACHE BOOL "Build curses GUI" FORCE)' >> build-flags.cmake
mkdir build && pushd build
../bootstrap \
--prefix=$PREFIX \
--init=../build-flags.cmake \
--parallel=$CPUS \
--system-curl
make -j$CPUS
# make test # run test suite
make install
popd && popd
fi
# compile cppcheck
if [ ! -f $PREFIX/bin/cppcheck ]; then
if [ -d cppcheck-$CPPCHECK_VERSION ]; then
rm -rf cppcheck-$CPPCHECK_VERSION
fi
tar -xvf ../archives/cppcheck-$CPPCHECK_VERSION.tar.gz
pushd cppcheck-$CPPCHECK_VERSION
env \
CC=gcc \
CXX=g++ \
PREFIX=$PREFIX \
FILESDIR=$PREFIX/share/cppcheck \
CFGDIR=$PREFIX/share/cppcheck/cfg \
make -j$CPUS
env \
CC=gcc \
CXX=g++ \
PREFIX=$PREFIX \
FILESDIR=$PREFIX/share/cppcheck \
CFGDIR=$PREFIX/share/cppcheck/cfg \
make install
popd
fi
# compile swig
if [ ! -d swig-$SWIG_VERSION/install ]; then
if [ -d swig-$SWIG_VERSION ]; then
rm -rf swig-$SWIG_VERSION
fi
tar -xvf ../archives/swig-$SWIG_VERSION.tar.gz
mv swig-rel-$SWIG_VERSION swig-$SWIG_VERSION
pushd swig-$SWIG_VERSION
./autogen.sh
mkdir build && pushd build
../configure --prefix=$DIR/build/swig-$SWIG_VERSION/install
make -j$CPUS
make install
popd && popd
fi
# compile llvm
if [ ! -f $PREFIX/bin/clang ]; then
if [ -d llvm-$LLVM_VERSION ]; then
rm -rf llvm-$LLVM_VERSION
fi
tar -xvf ../archives/llvm-$LLVM_VERSION.src.tar.xz
mv llvm-$LLVM_VERSION.src llvm-$LLVM_VERSION
tar -xvf ../archives/clang-$LLVM_VERSION.src.tar.xz
mv clang-$LLVM_VERSION.src llvm-$LLVM_VERSION/tools/clang
tar -xvf ../archives/lld-$LLVM_VERSION.src.tar.xz
mv lld-$LLVM_VERSION.src/ llvm-$LLVM_VERSION/tools/lld
tar -xvf ../archives/clang-tools-extra-$LLVM_VERSION.src.tar.xz
mv clang-tools-extra-$LLVM_VERSION.src/ llvm-$LLVM_VERSION/tools/clang/tools/extra
tar -xvf ../archives/compiler-rt-$LLVM_VERSION.src.tar.xz
mv compiler-rt-$LLVM_VERSION.src/ llvm-$LLVM_VERSION/projects/compiler-rt
pushd llvm-$LLVM_VERSION
mkdir build && pushd build
# activate swig
export PATH=$DIR/build/swig-$SWIG_VERSION/install/bin:$PATH
# influenced by: https://buildd.debian.org/status/fetch.php?pkg=llvm-toolchain-7&arch=amd64&ver=1%3A7.0.1%7E%2Brc2-1%7Eexp1&stamp=1541506173&raw=0
cmake .. \
-DCMAKE_C_COMPILER=$PREFIX/bin/gcc \
-DCMAKE_CXX_COMPILER=$PREFIX/bin/g++ \
-DCMAKE_CXX_LINK_FLAGS="-L$PREFIX/lib64 -Wl,-rpath,$PREFIX/lib64" \
-DCMAKE_INSTALL_PREFIX=$PREFIX \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_CXX_FLAGS_RELWITHDEBINFO="-O2 -DNDEBUG" \
-DCMAKE_CXX_FLAGS=' -fuse-ld=gold -fPIC -Wno-unused-command-line-argument -Wno-unknown-warning-option' \
-DCMAKE_C_FLAGS=' -fuse-ld=gold -fPIC -Wno-unused-command-line-argument -Wno-unknown-warning-option' \
-DLLVM_LINK_LLVM_DYLIB=ON \
-DLLVM_INSTALL_UTILS=ON \
-DLLVM_VERSION_SUFFIX= \
-DLLVM_BUILD_LLVM_DYLIB=ON \
-DLLVM_ENABLE_RTTI=ON \
-DLLVM_ENABLE_FFI=ON \
-DLLVM_BINUTILS_INCDIR=$PREFIX/include/ \
-DLLVM_USE_PERF=yes
make -j$CPUS
make -j$CPUS check-clang # run clang test suite
make -j$CPUS check-lld # run lld test suite
make install
popd && popd
fi
# create README
if [ ! -f $PREFIX/README.md ]; then
cat >$PREFIX/README.md <<EOF
# Memgraph Toolchain v$TOOLCHAIN_VERSION
## Included tools
- GCC $GCC_VERSION
- Binutils $BINUTILS_VERSION
- GDB $GDB_VERSION
- CMake $CMAKE_VERSION
- Cppcheck $CPPCHECK_VERSION
- LLVM (Clang, LLD, compiler-rt, Clang tools extra) $LLVM_VERSION
## Required libraries
In order to be able to run all of these tools you should install the following
packages:
\`\`\`
$($DIR/../os/$DISTRO.sh list TOOLCHAIN_RUN_DEPS)
\`\`\`
## Usage
In order to use the toolchain you just have to source the activation script:
\`\`\`
source $PREFIX/activate
\`\`\`
EOF
fi
# create activation script
if [ ! -f $PREFIX/activate ]; then
cat >$PREFIX/activate <<EOF
# This file must be used with "source $PREFIX/activate" *from bash*
# You can't run it directly!
# check for active virtual environments
if [ "\$( type -t deactivate )" != "" ]; then
echo "You already have an active virtual environment!"
return 0
fi
# check that we aren't root
if [ "\$USER" == "root" ]; then
echo "You shouldn't use the toolchan as root!"
return 0
fi
# save original environment
export ORIG_PATH=\$PATH
export ORIG_PS1=\$PS1
export ORIG_LD_LIBRARY_PATH=\$LD_LIBRARY_PATH
# activate new environment
export PATH=$PREFIX/bin:\$PATH
export PS1="($NAME) \$PS1"
export LD_LIBRARY_PATH=$PREFIX/lib:$PREFIX/lib64
# disable root
function su () {
echo "You don't want to use root functions while using the toolchain!"
return 1
}
function sudo () {
echo "You don't want to use root functions while using the toolchain!"
return 1
}
# create deactivation function
function deactivate() {
export PATH=\$ORIG_PATH
export PS1=\$ORIG_PS1
export LD_LIBRARY_PATH=\$ORIG_LD_LIBRARY_PATH
unset ORIG_PATH ORIG_PS1 ORIG_LD_LIBRARY_PATH
unset -f su sudo deactivate
}
EOF
fi
# create toolchain archive
if [ ! -f $NAME-binaries-$DISTRO.tar.gz ]; then
tar --owner=root --group=root -cpvzf $NAME-binaries-$DISTRO.tar.gz -C /opt $NAME
fi
# output final instructions
echo -e "\n\n"
echo "All tools have been built. They are installed in '$PREFIX'."
echo "In order to distribute the tools to someone else, an archive with the toolchain was created in the 'build' directory."
echo "If you want to install the packed tools you should execute the following command:"
echo
echo " tar -xvzf build/$NAME-binaries.tar.gz -C /opt"
echo
echo "Because the tools were built on this machine, you should probably change the permissions of the installation directory using:"
echo
echo " OPTIONAL: chown -R root:root $PREFIX"
echo
echo "In order to use all of the newly compiled tools you should use the prepared activation script:"
echo
echo " source $PREFIX/activate"
echo
echo "Or, for more advanced uses, you can add the following lines to your script:"
echo
echo " export PATH=$PREFIX/bin:\$PATH"
echo " export LD_LIBRARY_PATH=$PREFIX/lib:$PREFIX/lib64"
echo
echo "Enjoy!"

View File

@@ -1,546 +0,0 @@
#!/bin/bash -e
# helpers
pushd () { command pushd "$@" > /dev/null; }
popd () { command popd "$@" > /dev/null; }
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
CPUS=$( grep -c processor < /proc/cpuinfo )
cd "$DIR"
source "$DIR/../util.sh"
DISTRO="$(operating_system)"
# toolchain version
TOOLCHAIN_VERSION=3
# package versions used
GCC_VERSION=10.2.0
BINUTILS_VERSION=2.35.1
case "$DISTRO" in
centos-7) # because GDB >= 9 does NOT compile with readline6.
GDB_VERSION=8.3
;;
*)
GDB_VERSION=10.1
;;
esac
CMAKE_VERSION=3.18.4
CPPCHECK_VERSION=2.2
LLVM_VERSION=11.0.0
SWIG_VERSION=4.0.2 # used only for LLVM compilation
# Check for the dependencies.
echo "ALL BUILD PACKAGES: $($DIR/../os/$DISTRO.sh list TOOLCHAIN_BUILD_DEPS)"
$DIR/../os/$DISTRO.sh check TOOLCHAIN_BUILD_DEPS
echo "ALL RUN PACKAGES: $($DIR/../os/$DISTRO.sh list TOOLCHAIN_RUN_DEPS)"
$DIR/../os/$DISTRO.sh check TOOLCHAIN_RUN_DEPS
# check installation directory
NAME=toolchain-v$TOOLCHAIN_VERSION
PREFIX=/opt/$NAME
mkdir -p $PREFIX >/dev/null 2>/dev/null || true
if [ ! -d $PREFIX ] || [ ! -w $PREFIX ]; then
echo "Please make sure that the directory '$PREFIX' exists and is writable by the current user!"
echo
echo "If unsure, execute these commands as root:"
echo " mkdir $PREFIX && chown $USER:$USER $PREFIX"
echo
echo "Press <return> when you have created the directory and granted permissions."
# wait for the directory to be created
while true; do
read
if [ ! -d $PREFIX ] || [ ! -w $PREFIX ]; then
echo
echo "You can't continue before you have created the directory and granted permissions!"
echo
echo "Press <return> when you have created the directory and granted permissions."
else
break
fi
done
fi
# create archives directory
mkdir -p archives
# download all archives
pushd archives
if [ ! -f gcc-$GCC_VERSION.tar.gz ]; then
wget https://ftp.gnu.org/gnu/gcc/gcc-$GCC_VERSION/gcc-$GCC_VERSION.tar.gz
fi
if [ ! -f binutils-$BINUTILS_VERSION.tar.gz ]; then
wget https://ftp.gnu.org/gnu/binutils/binutils-$BINUTILS_VERSION.tar.gz
fi
if [ ! -f gdb-$GDB_VERSION.tar.gz ]; then
wget https://ftp.gnu.org/gnu/gdb/gdb-$GDB_VERSION.tar.gz
fi
if [ ! -f cmake-$CMAKE_VERSION.tar.gz ]; then
wget https://github.com/Kitware/CMake/releases/download/v$CMAKE_VERSION/cmake-$CMAKE_VERSION.tar.gz
fi
if [ ! -f swig-$SWIG_VERSION.tar.gz ]; then
wget https://github.com/swig/swig/archive/rel-$SWIG_VERSION.tar.gz -O swig-$SWIG_VERSION.tar.gz
fi
if [ ! -f cppcheck-$CPPCHECK_VERSION.tar.gz ]; then
wget https://github.com/danmar/cppcheck/archive/$CPPCHECK_VERSION.tar.gz -O cppcheck-$CPPCHECK_VERSION.tar.gz
fi
if [ ! -f llvm-$LLVM_VERSION.src.tar.xz ]; then
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/llvm-$LLVM_VERSION.src.tar.xz
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/clang-$LLVM_VERSION.src.tar.xz
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/lld-$LLVM_VERSION.src.tar.xz
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/clang-tools-extra-$LLVM_VERSION.src.tar.xz
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/compiler-rt-$LLVM_VERSION.src.tar.xz
fi
if [ ! -f pahole-gdb-master.zip ]; then
wget https://github.com/PhilArmstrong/pahole-gdb/archive/master.zip -O pahole-gdb-master.zip
fi
# verify all archives
# NOTE: Verification can fail if the archive is signed by another developer. I
# haven't added commands to download all developer GnuPG keys because the
# download is very slow. If the verification fails for you, figure out who has
# signed the archive and download their public key instead.
GPG="gpg --homedir .gnupg"
KEYSERVER="hkp://keyserver.ubuntu.com"
mkdir -p .gnupg
chmod 700 .gnupg
# verify gcc
if [ ! -f gcc-$GCC_VERSION.tar.gz.sig ]; then
wget https://ftp.gnu.org/gnu/gcc/gcc-$GCC_VERSION/gcc-$GCC_VERSION.tar.gz.sig
fi
# list of valid gcc gnupg keys: https://gcc.gnu.org/mirrors.html
$GPG --keyserver $KEYSERVER --recv-keys 0x3AB00996FC26A641
$GPG --verify gcc-$GCC_VERSION.tar.gz.sig gcc-$GCC_VERSION.tar.gz
# verify binutils
if [ ! -f binutils-$BINUTILS_VERSION.tar.gz.sig ]; then
wget https://ftp.gnu.org/gnu/binutils/binutils-$BINUTILS_VERSION.tar.gz.sig
fi
$GPG --keyserver $KEYSERVER --recv-keys 0xDD9E3C4F
$GPG --verify binutils-$BINUTILS_VERSION.tar.gz.sig binutils-$BINUTILS_VERSION.tar.gz
# verify gdb
if [ ! -f gdb-$GDB_VERSION.tar.gz.sig ]; then
wget https://ftp.gnu.org/gnu/gdb/gdb-$GDB_VERSION.tar.gz.sig
fi
$GPG --keyserver $KEYSERVER --recv-keys 0xFF325CF3
$GPG --verify gdb-$GDB_VERSION.tar.gz.sig gdb-$GDB_VERSION.tar.gz
# verify cmake
if [ ! -f cmake-$CMAKE_VERSION-SHA-256.txt ] || [ ! -f cmake-$CMAKE_VERSION-SHA-256.txt.asc ]; then
wget https://github.com/Kitware/CMake/releases/download/v$CMAKE_VERSION/cmake-$CMAKE_VERSION-SHA-256.txt
wget https://github.com/Kitware/CMake/releases/download/v$CMAKE_VERSION/cmake-$CMAKE_VERSION-SHA-256.txt.asc
# Because CentOS 7 doesn't have the `--ignore-missing` flag for `sha256sum`
# we filter out the missing files from the sums here manually.
cat cmake-$CMAKE_VERSION-SHA-256.txt | grep "cmake-$CMAKE_VERSION.tar.gz" > cmake-$CMAKE_VERSION-SHA-256-filtered.txt
fi
$GPG --keyserver $KEYSERVER --recv-keys 0xC6C265324BBEBDC350B513D02D2CEF1034921684
sha256sum -c cmake-$CMAKE_VERSION-SHA-256-filtered.txt
$GPG --verify cmake-$CMAKE_VERSION-SHA-256.txt.asc cmake-$CMAKE_VERSION-SHA-256.txt
# verify llvm, cfe, lld, clang-tools-extra
if [ ! -f llvm-$LLVM_VERSION.src.tar.xz.sig ]; then
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/llvm-$LLVM_VERSION.src.tar.xz.sig
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/clang-$LLVM_VERSION.src.tar.xz.sig
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/lld-$LLVM_VERSION.src.tar.xz.sig
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/clang-tools-extra-$LLVM_VERSION.src.tar.xz.sig
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/compiler-rt-$LLVM_VERSION.src.tar.xz.sig
fi
# list of valid llvm gnupg keys: https://releases.llvm.org/download.html
$GPG --keyserver $KEYSERVER --recv-keys 0x345AD05D
$GPG --verify llvm-$LLVM_VERSION.src.tar.xz.sig llvm-$LLVM_VERSION.src.tar.xz
$GPG --verify clang-$LLVM_VERSION.src.tar.xz.sig clang-$LLVM_VERSION.src.tar.xz
$GPG --verify lld-$LLVM_VERSION.src.tar.xz.sig lld-$LLVM_VERSION.src.tar.xz
$GPG --verify clang-tools-extra-$LLVM_VERSION.src.tar.xz.sig clang-tools-extra-$LLVM_VERSION.src.tar.xz
$GPG --verify compiler-rt-$LLVM_VERSION.src.tar.xz.sig compiler-rt-$LLVM_VERSION.src.tar.xz
popd
# create build directory
mkdir -p build
pushd build
# compile gcc
if [ ! -f $PREFIX/bin/gcc ]; then
if [ -d gcc-$GCC_VERSION ]; then
rm -rf gcc-$GCC_VERSION
fi
tar -xvf ../archives/gcc-$GCC_VERSION.tar.gz
pushd gcc-$GCC_VERSION
./contrib/download_prerequisites
mkdir build && pushd build
# influenced by: https://buildd.debian.org/status/fetch.php?pkg=gcc-8&arch=amd64&ver=8.3.0-6&stamp=1554588545
../configure -v \
--build=x86_64-linux-gnu \
--host=x86_64-linux-gnu \
--target=x86_64-linux-gnu \
--prefix=$PREFIX \
--disable-multilib \
--with-system-zlib \
--enable-checking=release \
--enable-languages=c,c++,fortran \
--enable-gold=yes \
--enable-ld=yes \
--enable-lto \
--enable-bootstrap \
--disable-vtable-verify \
--disable-werror \
--without-included-gettext \
--enable-threads=posix \
--enable-nls \
--enable-clocale=gnu \
--enable-libstdcxx-debug \
--enable-libstdcxx-time=yes \
--enable-gnu-unique-object \
--enable-libmpx \
--enable-plugin \
--enable-default-pie \
--with-target-system-zlib \
--with-tune=generic \
--without-cuda-driver
#--program-suffix=$( printf "$GCC_VERSION" | cut -d '.' -f 1,2 ) \
make -j$CPUS
# make -k check # run test suite
make install
popd && popd
fi
# activate toolchain
export PATH=$PREFIX/bin:$PATH
export LD_LIBRARY_PATH=$PREFIX/lib64
# compile binutils
if [ ! -f $PREFIX/bin/ld.gold ]; then
if [ -d binutils-$BINUTILS_VERSION ]; then
rm -rf binutils-$BINUTILS_VERSION
fi
tar -xvf ../archives/binutils-$BINUTILS_VERSION.tar.gz
pushd binutils-$BINUTILS_VERSION
mkdir build && pushd build
# influenced by: https://buildd.debian.org/status/fetch.php?pkg=binutils&arch=amd64&ver=2.32-7&stamp=1553247092
env \
CC=gcc \
CXX=g++ \
CFLAGS="-g -O2" \
CXXFLAGS="-g -O2" \
LDFLAGS="" \
../configure \
--build=x86_64-linux-gnu \
--host=x86_64-linux-gnu \
--prefix=$PREFIX \
--enable-ld=default \
--enable-gold \
--enable-lto \
--enable-plugins \
--enable-shared \
--enable-threads \
--with-system-zlib \
--enable-deterministic-archives \
--disable-compressed-debug-sections \
--enable-new-dtags \
--disable-werror
make -j$CPUS
# make -k check # run test suite
make install
popd && popd
fi
# compile gdb
if [ ! -f $PREFIX/bin/gdb ]; then
if [ -d gdb-$GDB_VERSION ]; then
rm -rf gdb-$GDB_VERSION
fi
tar -xvf ../archives/gdb-$GDB_VERSION.tar.gz
pushd gdb-$GDB_VERSION
mkdir build && pushd build
# https://buildd.debian.org/status/fetch.php?pkg=gdb&arch=amd64&ver=8.2.1-2&stamp=1550831554&raw=0
env \
CC=gcc \
CXX=g++ \
CFLAGS="-g -O2 -fstack-protector-strong -Wformat -Werror=format-security" \
CXXFLAGS="-g -O2 -fstack-protector-strong -Wformat -Werror=format-security" \
CPPFLAGS="-Wdate-time -D_FORTIFY_SOURCE=2 -fPIC" \
LDFLAGS="-Wl,-z,relro" \
PYTHON="" \
../configure \
--build=x86_64-linux-gnu \
--host=x86_64-linux-gnu \
--prefix=$PREFIX \
--disable-maintainer-mode \
--disable-dependency-tracking \
--disable-silent-rules \
--disable-gdbtk \
--disable-shared \
--without-guile \
--with-system-gdbinit=$PREFIX/etc/gdb/gdbinit \
--with-system-readline \
--with-expat \
--with-system-zlib \
--with-lzma \
--with-babeltrace \
--with-intel-pt \
--enable-tui \
--with-python=python3
make -j$CPUS
make install
popd && popd
fi
# install pahole
if [ ! -d $PREFIX/share/pahole-gdb ]; then
unzip ../archives/pahole-gdb-master.zip
mv pahole-gdb-master $PREFIX/share/pahole-gdb
fi
# setup system gdbinit
if [ ! -f $PREFIX/etc/gdb/gdbinit ]; then
mkdir -p $PREFIX/etc/gdb
cat >$PREFIX/etc/gdb/gdbinit <<EOF
# improve formatting
set print pretty on
set print object on
set print static-members on
set print vtbl on
set print demangle on
set demangle-style gnu-v3
set print sevenbit-strings off
# load libstdc++ pretty printers
add-auto-load-scripts-directory $PREFIX/lib64
add-auto-load-safe-path $PREFIX
# load pahole
python
sys.path.insert(0, "$PREFIX/share/pahole-gdb")
import offsets
import pahole
end
EOF
fi
# compile cmake
if [ ! -f $PREFIX/bin/cmake ]; then
if [ -d cmake-$CMAKE_VERSION ]; then
rm -rf cmake-$CMAKE_VERSION
fi
tar -xvf ../archives/cmake-$CMAKE_VERSION.tar.gz
pushd cmake-$CMAKE_VERSION
# influenced by: https://buildd.debian.org/status/fetch.php?pkg=cmake&arch=amd64&ver=3.13.4-1&stamp=1549799837
echo 'set(CMAKE_SKIP_RPATH ON CACHE BOOL "Skip rpath" FORCE)' >> build-flags.cmake
echo 'set(CMAKE_USE_RELATIVE_PATHS ON CACHE BOOL "Use relative paths" FORCE)' >> build-flags.cmake
echo 'set(CMAKE_C_FLAGS "-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2" CACHE STRING "C flags" FORCE)' >> build-flags.cmake
echo 'set(CMAKE_CXX_FLAGS "-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2" CACHE STRING "C++ flags" FORCE)' >> build-flags.cmake
echo 'set(CMAKE_SKIP_BOOTSTRAP_TEST ON CACHE BOOL "Skip BootstrapTest" FORCE)' >> build-flags.cmake
echo 'set(BUILD_CursesDialog ON CACHE BOOL "Build curses GUI" FORCE)' >> build-flags.cmake
mkdir build && pushd build
../bootstrap \
--prefix=$PREFIX \
--init=../build-flags.cmake \
--parallel=$CPUS \
--system-curl
make -j$CPUS
# make test # run test suite
make install
popd && popd
fi
# compile cppcheck
if [ ! -f $PREFIX/bin/cppcheck ]; then
if [ -d cppcheck-$CPPCHECK_VERSION ]; then
rm -rf cppcheck-$CPPCHECK_VERSION
fi
tar -xvf ../archives/cppcheck-$CPPCHECK_VERSION.tar.gz
pushd cppcheck-$CPPCHECK_VERSION
env \
CC=gcc \
CXX=g++ \
PREFIX=$PREFIX \
FILESDIR=$PREFIX/share/cppcheck \
CFGDIR=$PREFIX/share/cppcheck/cfg \
make -j$CPUS
env \
CC=gcc \
CXX=g++ \
PREFIX=$PREFIX \
FILESDIR=$PREFIX/share/cppcheck \
CFGDIR=$PREFIX/share/cppcheck/cfg \
make install
popd
fi
# compile swig
if [ ! -d swig-$SWIG_VERSION/install ]; then
if [ -d swig-$SWIG_VERSION ]; then
rm -rf swig-$SWIG_VERSION
fi
tar -xvf ../archives/swig-$SWIG_VERSION.tar.gz
mv swig-rel-$SWIG_VERSION swig-$SWIG_VERSION
pushd swig-$SWIG_VERSION
./autogen.sh
mkdir build && pushd build
../configure --prefix=$DIR/build/swig-$SWIG_VERSION/install
make -j$CPUS
make install
popd && popd
fi
# compile llvm
if [ ! -f $PREFIX/bin/clang ]; then
if [ -d llvm-$LLVM_VERSION ]; then
rm -rf llvm-$LLVM_VERSION
fi
tar -xvf ../archives/llvm-$LLVM_VERSION.src.tar.xz
mv llvm-$LLVM_VERSION.src llvm-$LLVM_VERSION
tar -xvf ../archives/clang-$LLVM_VERSION.src.tar.xz
mv clang-$LLVM_VERSION.src llvm-$LLVM_VERSION/tools/clang
tar -xvf ../archives/lld-$LLVM_VERSION.src.tar.xz
mv lld-$LLVM_VERSION.src/ llvm-$LLVM_VERSION/tools/lld
tar -xvf ../archives/clang-tools-extra-$LLVM_VERSION.src.tar.xz
mv clang-tools-extra-$LLVM_VERSION.src/ llvm-$LLVM_VERSION/tools/clang/tools/extra
tar -xvf ../archives/compiler-rt-$LLVM_VERSION.src.tar.xz
mv compiler-rt-$LLVM_VERSION.src/ llvm-$LLVM_VERSION/projects/compiler-rt
pushd llvm-$LLVM_VERSION
mkdir build && pushd build
# activate swig
export PATH=$DIR/build/swig-$SWIG_VERSION/install/bin:$PATH
# influenced by: https://buildd.debian.org/status/fetch.php?pkg=llvm-toolchain-7&arch=amd64&ver=1%3A7.0.1%7E%2Brc2-1%7Eexp1&stamp=1541506173&raw=0
cmake .. \
-DCMAKE_C_COMPILER=$PREFIX/bin/gcc \
-DCMAKE_CXX_COMPILER=$PREFIX/bin/g++ \
-DCMAKE_CXX_LINK_FLAGS="-L$PREFIX/lib64 -Wl,-rpath,$PREFIX/lib64" \
-DCMAKE_INSTALL_PREFIX=$PREFIX \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_CXX_FLAGS_RELWITHDEBINFO="-O2 -DNDEBUG" \
-DCMAKE_CXX_FLAGS=' -fuse-ld=gold -fPIC -Wno-unused-command-line-argument -Wno-unknown-warning-option' \
-DCMAKE_C_FLAGS=' -fuse-ld=gold -fPIC -Wno-unused-command-line-argument -Wno-unknown-warning-option' \
-DLLVM_LINK_LLVM_DYLIB=ON \
-DLLVM_INSTALL_UTILS=ON \
-DLLVM_VERSION_SUFFIX= \
-DLLVM_BUILD_LLVM_DYLIB=ON \
-DLLVM_ENABLE_RTTI=ON \
-DLLVM_ENABLE_FFI=ON \
-DLLVM_BINUTILS_INCDIR=$PREFIX/include/ \
-DLLVM_USE_PERF=yes
make -j$CPUS
make -j$CPUS check-clang # run clang test suite
make -j$CPUS check-lld # run lld test suite
make install
popd && popd
fi
# create README
if [ ! -f $PREFIX/README.md ]; then
cat >$PREFIX/README.md <<EOF
# Memgraph Toolchain v$TOOLCHAIN_VERSION
## Included tools
- GCC $GCC_VERSION
- Binutils $BINUTILS_VERSION
- GDB $GDB_VERSION
- CMake $CMAKE_VERSION
- Cppcheck $CPPCHECK_VERSION
- LLVM (Clang, LLD, compiler-rt, Clang tools extra) $LLVM_VERSION
## Required libraries
In order to be able to run all of these tools you should install the following
packages:
\`\`\`
$($DIR/../os/$DISTRO.sh list TOOLCHAIN_RUN_DEPS)
\`\`\`
## Usage
In order to use the toolchain you just have to source the activation script:
\`\`\`
source $PREFIX/activate
\`\`\`
EOF
fi
# create activation script
if [ ! -f $PREFIX/activate ]; then
cat >$PREFIX/activate <<EOF
# This file must be used with "source $PREFIX/activate" *from bash*
# You can't run it directly!
env_error="You already have an active virtual environment!"
# zsh does not recognize the option -t of the command type
# therefore we use the alternative whence -w
if [[ "\$ZSH_NAME" == "zsh" ]]; then
# check for active virtual environments
if [ "\$( whence -w deactivate )" != "deactivate: none" ]; then
echo \$env_error
return 0;
fi
# any other shell
else
# check for active virtual environments
if [ "\$( type -t deactivate )" != "" ]; then
echo \$env_error
return 0
fi
fi
# check that we aren't root
if [[ "\$USER" == "root" ]]; then
echo "You shouldn't use the toolchain as root!"
return 0
fi
# save original environment
export ORIG_PATH=\$PATH
export ORIG_PS1=\$PS1
export ORIG_LD_LIBRARY_PATH=\$LD_LIBRARY_PATH
# activate new environment
export PATH=$PREFIX/bin:\$PATH
export PS1="($NAME) \$PS1"
export LD_LIBRARY_PATH=$PREFIX/lib:$PREFIX/lib64
# disable root
function su () {
echo "You don't want to use root functions while using the toolchain!"
return 1
}
function sudo () {
echo "You don't want to use root functions while using the toolchain!"
return 1
}
# create deactivation function
function deactivate() {
export PATH=\$ORIG_PATH
export PS1=\$ORIG_PS1
export LD_LIBRARY_PATH=\$ORIG_LD_LIBRARY_PATH
unset ORIG_PATH ORIG_PS1 ORIG_LD_LIBRARY_PATH
unset -f su sudo deactivate
}
EOF
fi
# create toolchain archive
if [ ! -f $NAME-binaries-$DISTRO.tar.gz ]; then
tar --owner=root --group=root -cpvzf $NAME-binaries-$DISTRO.tar.gz -C /opt $NAME
fi
# output final instructions
echo -e "\n\n"
echo "All tools have been built. They are installed in '$PREFIX'."
echo "In order to distribute the tools to someone else, an archive with the toolchain was created in the 'build' directory."
echo "If you want to install the packed tools you should execute the following command:"
echo
echo " tar -xvzf build/$NAME-binaries.tar.gz -C /opt"
echo
echo "Because the tools were built on this machine, you should probably change the permissions of the installation directory using:"
echo
echo " OPTIONAL: chown -R root:root $PREFIX"
echo
echo "In order to use all of the newly compiled tools you should use the prepared activation script:"
echo
echo " source $PREFIX/activate"
echo
echo "Or, for more advanced uses, you can add the following lines to your script:"
echo
echo " export PATH=$PREFIX/bin:\$PATH"
echo " export LD_LIBRARY_PATH=$PREFIX/lib:$PREFIX/lib64"
echo
echo "Enjoy!"

View File

@@ -1,50 +0,0 @@
#!/bin/bash
operating_system() {
grep -E '^(VERSION_)?ID=' /etc/os-release | \
sort | cut -d '=' -f 2- | sed 's/"//g' | paste -s -d '-'
}
check_all_yum() {
local missing=""
for pkg in $1; do
if ! yum list installed "$pkg" >/dev/null 2>/dev/null; then
missing="$pkg $missing"
fi
done
if [ "$missing" != "" ]; then
echo "MISSING PACKAGES: $missing"
exit 1
fi
}
check_all_dpkg() {
local missing=""
for pkg in $1; do
if ! dpkg -s "$pkg" >/dev/null 2>/dev/null; then
missing="$pkg $missing"
fi
done
if [ "$missing" != "" ]; then
echo "MISSING PACKAGES: $missing"
exit 1
fi
}
check_all_dnf() {
local missing=""
for pkg in $1; do
if ! dnf list installed "$pkg" >/dev/null 2>/dev/null; then
missing="$pkg $missing"
fi
done
if [ "$missing" != "" ]; then
echo "MISSING PACKAGES: $missing"
exit 1
fi
}
install_all_apt() {
for pkg in $1; do
apt install -y "$pkg"
done
}

View File

@@ -1,824 +0,0 @@
/// @file
/// Provides API for usage in custom openCypher procedures
#ifndef MG_PROCEDURE_H
#define MG_PROCEDURE_H
#ifdef __cplusplus
extern "C" {
#endif
#include <stddef.h>
#include <stdint.h>
/// @name Memory Allocation
///
/// These should be preferred compared to plain malloc calls as Memgraph's
/// execution will handle allocation and deallocation more efficiently. In
/// addition to efficiency, Memgraph can set the limit on allowed allocations
/// thus providing some safety with regards to memory usage. The allocated
/// memory is only valid during the execution of mgp_main. You must not allocate
/// global resources with these functions. None of the functions are
/// thread-safe, because we provide a single thread of execution when invoking a
/// custom procedure. This allows Memgraph to be more efficient as stated
/// before.
///@{
/// Provides memory managament access and state.
struct mgp_memory;
/// Allocate a block of memory with given size in bytes.
/// Unlike malloc, this function is not thread-safe.
/// `size_in_bytes` must be greater than 0.
/// The returned pointer must be freed with mgp_free.
/// NULL is returned if unable to serve the requested allocation.
void *mgp_alloc(struct mgp_memory *memory, size_t size_in_bytes);
/// Allocate an aligned block of memory with given size in bytes.
/// Unlike malloc and aligned_alloc, this function is not thread-safe.
/// `size_in_bytes` must be greater than 0.
/// `alignment` must be a power of 2 value.
/// The returned pointer must be freed with mgp_free.
/// NULL is returned if unable to serve the requested allocation.
void *mgp_aligned_alloc(struct mgp_memory *memory, size_t size_in_bytes,
size_t alignment);
/// Deallocate an allocation from mgp_alloc or mgp_aligned_alloc.
/// Unlike free, this function is not thread-safe.
/// If `ptr` is NULL, this function does nothing.
/// The behavior is undefined if `ptr` is not a value returned from a prior
/// mgp_alloc or mgp_aligned_alloc call with the corresponding `memory`.
void mgp_free(struct mgp_memory *memory, void *ptr);
///@}
/// @name Operations on mgp_value
///
/// struct mgp_value is an immutable container of various values that may appear
/// as arguments and results of a custom procedure. The following functions and
/// types are used to work with mgp_value. Each function returning a non-const
/// mgp_value has allocated a new instance of the result, therefore such
/// mgp_value instances need to be deallocated using mgp_value_destroy.
///@{
/// Immutable container of various values that appear in the query language.
struct mgp_value;
/// List of mgp_value instances
struct mgp_list;
/// Map of character strings to mgp_value instances.
struct mgp_map;
/// Vertex in the graph database.
struct mgp_vertex;
/// Edge in the graph database.
struct mgp_edge;
/// Path containing mgp_vertex and mgp_edge instances.
struct mgp_path;
/// All available types that can be stored in a mgp_value
enum mgp_value_type {
// NOTE: New types need to be appended, so as not to break ABI.
MGP_VALUE_TYPE_NULL,
MGP_VALUE_TYPE_BOOL,
MGP_VALUE_TYPE_INT,
MGP_VALUE_TYPE_DOUBLE,
MGP_VALUE_TYPE_STRING,
MGP_VALUE_TYPE_LIST,
MGP_VALUE_TYPE_MAP,
MGP_VALUE_TYPE_VERTEX,
MGP_VALUE_TYPE_EDGE,
MGP_VALUE_TYPE_PATH,
};
/// Free the memory used by the given mgp_value instance.
void mgp_value_destroy(struct mgp_value *val);
/// Construct a value representing `null` in openCypher.
/// You need to free the instance through mgp_value_destroy.
/// NULL is returned if unable to allocate a mgp_value.
struct mgp_value *mgp_value_make_null(struct mgp_memory *memory);
/// Construct a boolean value.
/// Non-zero values represent `true`, while zero represents `false`.
/// You need to free the instance through mgp_value_destroy.
/// NULL is returned if unable to allocate a mgp_value.
struct mgp_value *mgp_value_make_bool(int val, struct mgp_memory *memory);
/// Construct an integer value.
/// You need to free the instance through mgp_value_destroy.
/// NULL is returned if unable to allocate a mgp_value.
struct mgp_value *mgp_value_make_int(int64_t val, struct mgp_memory *memory);
/// Construct a double floating point value.
/// You need to free the instance through mgp_value_destroy.
/// NULL is returned if unable to allocate a mgp_value.
struct mgp_value *mgp_value_make_double(double val, struct mgp_memory *memory);
/// Construct a character string value from a NULL terminated string.
/// You need to free the instance through mgp_value_destroy.
/// NULL is returned if unable to allocate a mgp_value.
struct mgp_value *mgp_value_make_string(const char *val,
struct mgp_memory *memory);
/// Create a mgp_value storing a mgp_list.
/// You need to free the instance through mgp_value_destroy. The ownership of
/// the list is given to the created mgp_value and destroying the mgp_value will
/// destroy the mgp_list. Therefore, if a mgp_value is successfully created
/// you must not call mgp_list_destroy on the given list.
/// NULL is returned if unable to allocate a mgp_value.
struct mgp_value *mgp_value_make_list(struct mgp_list *val);
/// Create a mgp_value storing a mgp_map.
/// You need to free the instance through mgp_value_destroy. The ownership of
/// the map is given to the created mgp_value and destroying the mgp_value will
/// destroy the mgp_map. Therefore, if a mgp_value is successfully created
/// you must not call mgp_map_destroy on the given map.
/// NULL is returned if unable to allocate a mgp_value.
struct mgp_value *mgp_value_make_map(struct mgp_map *val);
/// Create a mgp_value storing a mgp_vertex.
/// You need to free the instance through mgp_value_destroy. The ownership of
/// the vertex is given to the created mgp_value and destroying the mgp_value
/// will destroy the mgp_vertex. Therefore, if a mgp_value is successfully
/// created you must not call mgp_vertex_destroy on the given vertex.
/// NULL is returned if unable to allocate a mgp_value.
struct mgp_value *mgp_value_make_vertex(struct mgp_vertex *val);
/// Create a mgp_value storing a mgp_edge.
/// You need to free the instance through mgp_value_destroy. The ownership of
/// the edge is given to the created mgp_value and destroying the mgp_value will
/// destroy the mgp_edge. Therefore, if a mgp_value is successfully created you
/// must not call mgp_edge_destroy on the given edge.
/// NULL is returned if unable to allocate a mgp_value.
struct mgp_value *mgp_value_make_edge(struct mgp_edge *val);
/// Create a mgp_value storing a mgp_path.
/// You need to free the instance through mgp_value_destroy. The ownership of
/// the path is given to the created mgp_value and destroying the mgp_value will
/// destroy the mgp_path. Therefore, if a mgp_value is successfully created you
/// must not call mgp_path_destroy on the given path.
/// NULL is returned if unable to allocate a mgp_value.
struct mgp_value *mgp_value_make_path(struct mgp_path *val);
/// Return the type of the value contained in mgp_value.
enum mgp_value_type mgp_value_get_type(const struct mgp_value *val);
/// Return non-zero if the given mgp_value represents `null`.
int mgp_value_is_null(const struct mgp_value *val);
/// Return non-zero if the given mgp_value stores a boolean.
int mgp_value_is_bool(const struct mgp_value *val);
/// Return non-zero if the given mgp_value stores an integer.
int mgp_value_is_int(const struct mgp_value *val);
/// Return non-zero if the given mgp_value stores a double floating-point.
int mgp_value_is_double(const struct mgp_value *val);
/// Return non-zero if the given mgp_value stores a character string.
int mgp_value_is_string(const struct mgp_value *val);
/// Return non-zero if the given mgp_value stores a list of values.
int mgp_value_is_list(const struct mgp_value *val);
/// Return non-zero if the given mgp_value stores a map of values.
int mgp_value_is_map(const struct mgp_value *val);
/// Return non-zero if the given mgp_value stores a vertex.
int mgp_value_is_vertex(const struct mgp_value *val);
/// Return non-zero if the given mgp_value stores an edge.
int mgp_value_is_edge(const struct mgp_value *val);
/// Return non-zero if the given mgp_value stores a path.
int mgp_value_is_path(const struct mgp_value *val);
/// Return the contained boolean value.
/// Non-zero values represent `true`, while zero represents `false`.
/// The result is undefined if mgp_value does not contain the expected type.
int mgp_value_get_bool(const struct mgp_value *val);
/// Return the contained integer.
/// The result is undefined if mgp_value does not contain the expected type.
int64_t mgp_value_get_int(const struct mgp_value *val);
/// Return the contained double floating-point.
/// The result is undefined if mgp_value does not contain the expected type.
double mgp_value_get_double(const struct mgp_value *val);
/// Return the contained character string.
/// The result is undefined if mgp_value does not contain the expected type.
const char *mgp_value_get_string(const struct mgp_value *val);
/// Return the contained list of values.
/// The result is undefined if mgp_value does not contain the expected type.
const struct mgp_list *mgp_value_get_list(const struct mgp_value *val);
/// Return the contained map of values.
/// The result is undefined if mgp_value does not contain the expected type.
const struct mgp_map *mgp_value_get_map(const struct mgp_value *val);
/// Return the contained vertex.
/// The result is undefined if mgp_value does not contain the expected type.
const struct mgp_vertex *mgp_value_get_vertex(const struct mgp_value *val);
/// Return the contained edge.
/// The result is undefined if mgp_value does not contain the expected type.
const struct mgp_edge *mgp_value_get_edge(const struct mgp_value *val);
/// Return the contained path.
/// The result is undefined if mgp_value does not contain the expected type.
const struct mgp_path *mgp_value_get_path(const struct mgp_value *val);
/// Create an empty list with given capacity.
/// You need to free the created instance with mgp_list_destroy.
/// The created list will have allocated enough memory for `capacity` elements
/// of mgp_value, but it will not contain any elements. Therefore,
/// mgp_list_size will return 0.
/// NULL is returned if unable to allocate a new list.
struct mgp_list *mgp_list_make_empty(size_t capacity,
struct mgp_memory *memory);
/// Free the memory used by the given mgp_list and contained elements.
void mgp_list_destroy(struct mgp_list *list);
/// Append a copy of mgp_value to mgp_list if capacity allows.
/// The list copies the given value and therefore does not take ownership of the
/// original value. You still need to call mgp_value_destroy to free the
/// original value.
/// Return non-zero on success, or 0 if there's no capacity or memory to append
/// the mgp_value to mgp_list.
int mgp_list_append(struct mgp_list *list, const struct mgp_value *val);
/// Append a copy of mgp_value to mgp_list increasing capacity if needed.
/// The list copies the given value and therefore does not take ownership of the
/// original value. You still need to call mgp_value_destroy to free the
/// original value.
/// In case of a capacity change, the previously contained elements will move in
/// memory and any references to them will be invalid.
/// Return non-zero on success, or 0 if there's no memory to append the
/// mgp_value to mgp_list.
int mgp_list_append_extend(struct mgp_list *list, const struct mgp_value *val);
/// Return the number of elements stored in mgp_list.
size_t mgp_list_size(const struct mgp_list *list);
/// Return the total number of elements for which there's already allocated
/// memory in mgp_list.
size_t mgp_list_capacity(const struct mgp_list *list);
/// Return the element in mgp_list at given position.
/// NULL is returned if the index is not within mgp_list_size.
const struct mgp_value *mgp_list_at(const struct mgp_list *list, size_t index);
/// Create an empty map of character strings to mgp_value instances.
/// You need to free the created instance with mgp_map_destroy.
/// NULL is returned if unable to allocate a new map.
struct mgp_map *mgp_map_make_empty(struct mgp_memory *memory);
/// Free the memory used by the given mgp_map and contained items.
void mgp_map_destroy(struct mgp_map *map);
/// Insert a new mapping from a NULL terminated character string to a value.
/// If a mapping with the same key already exists, it is *not* replaced.
/// In case of insertion, both the string and the value are copied into the map.
/// Therefore, the map does not take ownership of the original key nor value, so
/// you still need to free their memory explicitly.
/// Return non-zero on success, or 0 if there's no memory to insert a new
/// mapping or a previous mapping already exists.
int mgp_map_insert(struct mgp_map *map, const char *key,
const struct mgp_value *value);
/// Return the number of items stored in mgp_map.
size_t mgp_map_size(const struct mgp_map *map);
/// Return the mapped mgp_value to the given character string.
/// NULL is returned if no mapping exists.
const struct mgp_value *mgp_map_at(const struct mgp_map *map, const char *key);
/// An item in the mgp_map.
struct mgp_map_item;
/// Get the key of the mapped item.
const char *mgp_map_item_key(const struct mgp_map_item *item);
/// Get the value of the mapped item.
const struct mgp_value *mgp_map_item_value(const struct mgp_map_item *item);
/// An iterator over the items in mgp_map.
struct mgp_map_items_iterator;
/// Start iterating over items contained in the given map.
/// The returned mgp_map_items_iterator needs to be deallocated with
/// mgp_map_items_iterator_destroy.
/// NULL is returned if unable to allocate a new iterator.
struct mgp_map_items_iterator *mgp_map_iter_items(const struct mgp_map *map,
struct mgp_memory *memory);
/// Deallocate memory used by mgp_map_items_iterator.
void mgp_map_items_iterator_destroy(struct mgp_map_items_iterator *it);
/// Get the current item pointed to by the iterator.
/// When the mgp_map_items_iterator_next is invoked, the returned pointer
/// to mgp_map_item becomes invalid. On the other hand, pointers obtained
/// with mgp_map_item_key and mgp_map_item_value remain valid
/// throughout the lifetime of a map. Therefore, you can store the key as well
/// as the value before, and use them after invoking
/// mgp_map_items_iterator_next.
/// NULL is returned if the end of the iteration has been reached.
const struct mgp_map_item *mgp_map_items_iterator_get(
const struct mgp_map_items_iterator *it);
/// Advance the iterator to the next item stored in map and return it.
/// The previous pointer obtained through mgp_map_items_iterator_get will
/// be invalidated, but the pointers to key and value will remain valid.
/// NULL is returned if the end of the iteration has been reached.
const struct mgp_map_item *mgp_map_items_iterator_next(
struct mgp_map_items_iterator *it);
/// Create a path with the copy of the given starting vertex.
/// You need to free the created instance with mgp_path_destroy.
/// NULL is returned if unable to allocate a path.
struct mgp_path *mgp_path_make_with_start(const struct mgp_vertex *vertex,
struct mgp_memory *memory);
/// Copy a mgp_path.
/// Returned pointer must be freed with mgp_path_destroy.
/// NULL is returned if unable to allocate a mgp_path.
struct mgp_path *mgp_path_copy(const struct mgp_path *path,
struct mgp_memory *memory);
/// Free the memory used by the given mgp_path and contained vertices and edges.
void mgp_path_destroy(struct mgp_path *path);
/// Append an edge continuing from the last vertex on the path.
/// The edge is copied into the path. Therefore, the path does not take
/// ownership of the original edge, so you still need to free the edge memory
/// explicitly.
/// The last vertex on the path will become the other endpoint of the given
/// edge, as continued from the current last vertex.
/// Return non-zero on success, or 0 if the current last vertex in the path is
/// not part of the given edge. 0 is also returned if unable to allocate memory
/// for path extension.
int mgp_path_expand(struct mgp_path *path, const struct mgp_edge *edge);
/// Return the number of edges in a mgp_path.
size_t mgp_path_size(const struct mgp_path *path);
/// Return the vertex from a path at given index.
/// The valid index range is [0, mgp_path_size].
/// NULL is returned if index is out of range.
const struct mgp_vertex *mgp_path_vertex_at(const struct mgp_path *path,
size_t index);
/// Return the edge from a path at given index.
/// The valid index range is [0, mgp_path_size - 1].
/// NULL is returned if index is out of range.
const struct mgp_edge *mgp_path_edge_at(const struct mgp_path *path,
size_t index);
/// Return non-zero if given paths are equal, otherwise 0.
int mgp_path_equal(const struct mgp_path *p1, const struct mgp_path *p2);
///@}
/// @name Procedure Result
/// These functions and types are used to set the result of your custom
/// procedure.
///@{
/// Stores either an error result or a list of mgp_result_record instances.
struct mgp_result;
/// Represents a record of resulting field values.
struct mgp_result_record;
/// Set the error as the result of the procedure.
/// If there's no memory for copying the error message, 0 is returned.
int mgp_result_set_error_msg(struct mgp_result *res, const char *error_msg);
/// Create a new record for results.
/// The previously returned pointer to mgp_result_record is no longer valid, and
/// you must not use it.
/// Return NULL if unable to allocate a mgp_result_record.
struct mgp_result_record *mgp_result_new_record(struct mgp_result *res);
/// Assign a value to a field in the given record.
/// Return 0 if there's no memory to copy the mgp_value to mgp_result_record or
/// if the combination of `field_name` and `val` does not satisfy the
/// procedure's result signature.
int mgp_result_record_insert(struct mgp_result_record *record,
const char *field_name,
const struct mgp_value *val);
///@}
/// @name Graph Constructs
///@{
/// Label of a vertex.
struct mgp_label {
/// Name of the label as a NULL terminated character string.
const char *name;
};
/// Type of an edge.
struct mgp_edge_type {
/// Name of the type as a NULL terminated character string.
const char *name;
};
/// Iterator over property values of a vertex or an edge.
struct mgp_properties_iterator;
/// Free the memory used by a mgp_properties_iterator.
void mgp_properties_iterator_destroy(struct mgp_properties_iterator *it);
/// Reference to a named property value.
struct mgp_property {
/// Name (key) of a property as a NULL terminated string.
const char *name;
/// Value of the referenced property.
const struct mgp_value *value;
};
/// Get the current property pointed to by the iterator.
/// When the mgp_properties_iterator_next is invoked, the previous
/// mgp_property is invalidated and its value must not be used.
/// NULL is returned if the end of the iteration has been reached.
const struct mgp_property *mgp_properties_iterator_get(
const struct mgp_properties_iterator *it);
/// Advance the iterator to the next property and return it.
/// The previous mgp_property obtained through mgp_properties_iterator_get
/// will be invalidated, and you must not use its value.
/// NULL is returned if the end of the iteration has been reached.
const struct mgp_property *mgp_properties_iterator_next(
struct mgp_properties_iterator *it);
/// Iterator over edges of a vertex.
struct mgp_edges_iterator;
/// Free the memory used by a mgp_edges_iterator.
void mgp_edges_iterator_destroy(struct mgp_edges_iterator *it);
/// ID of a vertex; valid during a single query execution.
struct mgp_vertex_id {
int64_t as_int;
};
/// Get the ID of given vertex.
/// The ID is only valid for a single query execution, you should never store it
/// globally in a query module.
struct mgp_vertex_id mgp_vertex_get_id(const struct mgp_vertex *v);
/// Copy a mgp_vertex.
/// Returned pointer must be freed with mgp_vertex_destroy.
/// NULL is returned if unable to allocate a mgp_vertex.
struct mgp_vertex *mgp_vertex_copy(const struct mgp_vertex *v,
struct mgp_memory *memory);
/// Free the memory used by a mgp_vertex.
void mgp_vertex_destroy(struct mgp_vertex *v);
/// Return non-zero if given vertices are equal, otherwise 0.
int mgp_vertex_equal(const struct mgp_vertex *v1, const struct mgp_vertex *v2);
/// Return the number of labels a given vertex has.
size_t mgp_vertex_labels_count(const struct mgp_vertex *v);
/// Return mgp_label in mgp_vertex at given index.
/// If the index is out of bounds, mgp_label.name is set to NULL.
struct mgp_label mgp_vertex_label_at(const struct mgp_vertex *v, size_t index);
/// Return non-zero if the given vertex has the given label.
int mgp_vertex_has_label(const struct mgp_vertex *v, struct mgp_label label);
/// Return non-zero if the given vertex has a label with given name.
int mgp_vertex_has_label_named(const struct mgp_vertex *v,
const char *label_name);
/// Get a copy of a vertex property mapped to a given name.
/// Returned value must be freed with mgp_value_destroy.
/// NULL is returned if unable to allocate a mgp_value.
struct mgp_value *mgp_vertex_get_property(const struct mgp_vertex *v,
const char *property_name,
struct mgp_memory *memory);
/// Start iterating over properties stored in the given vertex.
/// The returned mgp_properties_iterator needs to be deallocated with
/// mgp_properties_iterator_destroy.
/// NULL is returned if unable to allocate a new iterator.
struct mgp_properties_iterator *mgp_vertex_iter_properties(
const struct mgp_vertex *v, struct mgp_memory *memory);
/// Start iterating over inbound edges of the given vertex.
/// The returned mgp_edges_iterator needs to be deallocated with
/// mgp_edges_iterator_destroy.
/// NULL is returned if unable to allocate a new iterator.
struct mgp_edges_iterator *mgp_vertex_iter_in_edges(const struct mgp_vertex *v,
struct mgp_memory *memory);
/// Start iterating over outbound edges of the given vertex.
/// The returned mgp_edges_iterator needs to be deallocated with
/// mgp_edges_iterator_destroy.
/// NULL is returned if unable to allocate a new iterator.
struct mgp_edges_iterator *mgp_vertex_iter_out_edges(const struct mgp_vertex *v,
struct mgp_memory *memory);
/// Get the current edge pointed to by the iterator.
/// When the mgp_edges_iterator_next is invoked, the previous
/// mgp_edge is invalidated and its value must not be used.
/// NULL is returned if the end of the iteration has been reached.
const struct mgp_edge *mgp_edges_iterator_get(
const struct mgp_edges_iterator *it);
/// Advance the iterator to the next edge and return it.
/// The previous mgp_edge obtained through mgp_edges_iterator_get
/// will be invalidated, and you must not use its value.
/// NULL is returned if the end of the iteration has been reached.
const struct mgp_edge *mgp_edges_iterator_next(struct mgp_edges_iterator *it);
/// ID of an edge; valid during a single query execution.
struct mgp_edge_id {
int64_t as_int;
};
/// Get the ID of given edge.
/// The ID is only valid for a single query execution, you should never store it
/// globally in a query module.
struct mgp_edge_id mgp_edge_get_id(const struct mgp_edge *e);
/// Copy a mgp_edge.
/// Returned pointer must be freed with mgp_edge_destroy.
/// NULL is returned if unable to allocate a mgp_edge.
struct mgp_edge *mgp_edge_copy(const struct mgp_edge *e,
struct mgp_memory *memory);
/// Free the memory used by a mgp_edge.
void mgp_edge_destroy(struct mgp_edge *e);
/// Return non-zero if given edges are equal, otherwise 0.
int mgp_edge_equal(const struct mgp_edge *e1, const struct mgp_edge *e2);
/// Return the type of the given edge.
struct mgp_edge_type mgp_edge_get_type(const struct mgp_edge *e);
/// Return the source vertex of the given edge.
const struct mgp_vertex *mgp_edge_get_from(const struct mgp_edge *e);
/// Return the destination vertex of the given edge.
const struct mgp_vertex *mgp_edge_get_to(const struct mgp_edge *e);
/// Get a copy of a edge property mapped to a given name.
/// Returned value must be freed with mgp_value_destroy.
/// NULL is returned if unable to allocate a mgp_value.
struct mgp_value *mgp_edge_get_property(const struct mgp_edge *e,
const char *property_name,
struct mgp_memory *memory);
/// Start iterating over properties stored in the given edge.
/// The returned mgp_properties_iterator needs to be deallocated with
/// mgp_properties_iterator_destroy.
/// NULL is returned if unable to allocate a new iterator.
struct mgp_properties_iterator *mgp_edge_iter_properties(
const struct mgp_edge *e, struct mgp_memory *memory);
/// State of the graph database.
struct mgp_graph;
/// Return the vertex corresponding to given ID.
/// The returned vertex must be freed using mgp_vertex_destroy.
/// NULL is returned if unable to allocate the vertex or if ID is not valid.
struct mgp_vertex *mgp_graph_get_vertex_by_id(const struct mgp_graph *g,
struct mgp_vertex_id id,
struct mgp_memory *memory);
/// Iterator over vertices.
struct mgp_vertices_iterator;
/// Free the memory used by a mgp_vertices_iterator.
void mgp_vertices_iterator_destroy(struct mgp_vertices_iterator *it);
/// Start iterating over vertices of the given graph.
/// The returned mgp_vertices_iterator needs to be deallocated with
/// mgp_vertices_iterator_destroy.
/// NULL is returned if unable to allocate a new iterator.
struct mgp_vertices_iterator *mgp_graph_iter_vertices(
const struct mgp_graph *g, struct mgp_memory *memory);
/// Get the current vertex pointed to by the iterator.
/// When the mgp_vertices_iterator_next is invoked, the previous
/// mgp_vertex is invalidated and its value must not be used.
/// NULL is returned if the end of the iteration has been reached.
const struct mgp_vertex *mgp_vertices_iterator_get(
const struct mgp_vertices_iterator *it);
/// Advance the iterator to the next vertex and return it.
/// The previous mgp_vertex obtained through mgp_vertices_iterator_get
/// will be invalidated, and you must not use its value.
/// NULL is returned if the end of the iteration has been reached.
const struct mgp_vertex *mgp_vertices_iterator_next(
struct mgp_vertices_iterator *it);
///@}
/// @name Type System
///
/// The following structures and functions are used to build a type
/// representation used in openCypher. The primary purpose is to create a
/// procedure signature for use with openCypher. Memgraph will use the built
/// procedure signature to perform various static and dynamic checks when the
/// custom procedure is invoked.
///@{
/// A type for values in openCypher.
struct mgp_type;
/// Get the type representing any value that isn't `null`.
///
/// The ANY type is the parent type of all types.
const struct mgp_type *mgp_type_any();
/// Get the type representing boolean values.
const struct mgp_type *mgp_type_bool();
/// Get the type representing character string values.
const struct mgp_type *mgp_type_string();
/// Get the type representing integer values.
const struct mgp_type *mgp_type_int();
/// Get the type representing floating-point values.
const struct mgp_type *mgp_type_float();
/// Get the type representing any number value.
///
/// This is the parent type for numeric types, i.e. INTEGER and FLOAT.
const struct mgp_type *mgp_type_number();
/// Get the type representing map values.
///
/// Map values are those which map string keys to values of any type. For
/// example `{ database: "Memgraph", version: 1.42 }`. Note that graph nodes
/// contain property maps, so a node value will also satisfy the MAP type. The
/// same applies for graph relationship values.
///
/// @sa mgp_type_node
/// @sa mgp_type_relationship
const struct mgp_type *mgp_type_map();
/// Get the type representing graph node values.
///
/// Since a node contains a map of properties, the node itself is also of MAP
/// type.
const struct mgp_type *mgp_type_node();
/// Get the type representing graph relationship values.
///
/// Since a relationship contains a map of properties, the relationship itself
/// is also of MAP type.
const struct mgp_type *mgp_type_relationship();
/// Get the type representing a graph path (walk) from one node to another.
const struct mgp_type *mgp_type_path();
/// Build a type representing a list of values of given `element_type`.
///
/// NULL is returned if unable to allocate the new type.
const struct mgp_type *mgp_type_list(const struct mgp_type *element_type);
/// Build a type representing either a `null` value or a value of given `type`.
///
/// NULL is returned if unable to allocate the new type.
const struct mgp_type *mgp_type_nullable(const struct mgp_type *type);
///@}
/// @name Query Module & Procedures
///
/// The following structures and functions are used to build a query module. You
/// will receive an empty instance of mgp_module through your
/// `int mgp_init_module(struct mgp_module *module, struct mgp_memory *memory)`
/// function. Each shared library that wishes to provide a query module needs to
/// have the said function. Inside you can fill the module with procedures,
/// which can then be called through openCypher.
///
/// Arguments to `mgp_init_module` will not live longer than the function's
/// execution, so you must not store them globally. Additionally, you must not
/// use the passed in mgp_memory to allocate global resources.
///@{
/// Stores information on your query module.
struct mgp_module;
/// Describes a procedure of a query module.
struct mgp_proc;
/// Entry-point for a query module procedure, invoked through openCypher.
///
/// Passed in arguments will not live longer than the callback's execution.
/// Therefore, you must not store them globally or use the passed in mgp_memory
/// to allocate global resources.
typedef void (*mgp_proc_cb)(const struct mgp_list *, const struct mgp_graph *,
struct mgp_result *, struct mgp_memory *);
/// Register a read-only procedure with a module.
///
/// The `name` must be a sequence of digits, underscores, lowercase and
/// uppercase Latin letters. The name must begin with a non-digit character.
/// Note that Unicode characters are not allowed. Additionally, names are
/// case-sensitive.
///
/// NULL is returned if unable to allocate memory for mgp_proc; if `name` is
/// not valid or a procedure with the same name was already registered.
struct mgp_proc *mgp_module_add_read_procedure(struct mgp_module *module,
const char *name,
mgp_proc_cb cb);
/// Add a required argument to a procedure.
///
/// The order of adding arguments will correspond to the order the procedure
/// must receive them through openCypher. Required arguments will be followed by
/// optional arguments.
///
/// The `name` must be a valid identifier, following the same rules as the
/// procedure`name` in mgp_module_add_read_procedure.
///
/// Passed in `type` describes what kind of values can be used as the argument.
///
/// 0 is returned if unable to allocate memory for an argument; if invoking this
/// function after setting an optional argument or if `name` is not valid.
/// Non-zero is returned on success.
int mgp_proc_add_arg(struct mgp_proc *proc, const char *name,
const struct mgp_type *type);
/// Add an optional argument with a default value to a procedure.
///
/// The order of adding arguments will correspond to the order the procedure
/// must receive them through openCypher. Optional arguments must follow the
/// required arguments.
///
/// The `name` must be a valid identifier, following the same rules as the
/// procedure `name` in mgp_module_add_read_procedure.
///
/// Passed in `type` describes what kind of values can be used as the argument.
///
/// `default_value` is copied and set as the default value for the argument.
/// Don't forget to call mgp_value_destroy when you are done using
/// `default_value`. When the procedure is called, if this argument is not
/// provided, `default_value` will be used instead. `default_value` must not be
/// a graph element (node, relationship, path) and it must satisfy the given
/// `type`.
///
/// 0 is returned if unable to allocate memory for an argument; if `name` is
/// not valid or `default_value` does not satisfy `type`. Non-zero is returned
/// on success.
int mgp_proc_add_opt_arg(struct mgp_proc *proc, const char *name,
const struct mgp_type *type,
const struct mgp_value *default_value);
/// Add a result field to a procedure.
///
/// The `name` must be a valid identifier, following the same rules as the
/// procedure `name` in mgp_module_add_read_procedure.
///
/// Passed in `type` describes what kind of values can be returned through the
/// result field.
///
/// 0 is returned if unable to allocate memory for a result field; if
/// `name` is not valid or if a result field with the same name was already
/// added. Non-zero is returned on success.
int mgp_proc_add_result(struct mgp_proc *proc, const char *name,
const struct mgp_type *type);
/// Add a result field to a procedure and mark it as deprecated.
///
/// This is the same as mgp_proc_add_result, but the result field will be marked
/// as deprecated.
int mgp_proc_add_deprecated_result(struct mgp_proc *proc, const char *name,
const struct mgp_type *type);
///@}
/// @name Execution
///
/// The following functions are used to control the execution of the procedure.
///
/// @{
/// Return non-zero if the currently executing procedure should abort as soon as
/// possible.
///
/// Procedures which perform heavyweight processing run the risk of running too
/// long and going over the query execution time limit. To prevent this, such
/// procedures should periodically call this function at critical points in
/// their code in order to determine whether they should abort or not. Note that
/// this mechanism is purely cooperative and depends on the procedure doing the
/// checking and aborting on its own.
int mgp_must_abort(const struct mgp_graph *graph);
/// @}
#ifdef __cplusplus
} // extern "C"
#endif
#endif // MG_PROCEDURE_H

View File

@@ -1,3 +0,0 @@
{
mgp_*;
};

View File

@@ -1,798 +0,0 @@
'''
This module provides the API for usage in custom openCypher procedures.
'''
# C API using `mgp_memory` is not exposed in Python, instead the usage of such
# API is hidden behind Python API. Any function requiring an instance of
# `mgp_memory` should go through a `ProcCtx` instance.
#
# `mgp_value` does not exist as such in Python, instead all `mgp_value`
# instances are marshalled to an appropriate Python object. This implies that
# `mgp_list` and `mgp_map` are mapped to `list` and `dict` respectively.
#
# Only the public API is stubbed out here. Any private details are left for the
# actual implementation. Functions have type annotations as supported by Python
# 3.5, but variable type annotations are only available with Python 3.6+
from collections import namedtuple
import functools
import inspect
import sys
import typing
import _mgp
class InvalidContextError(Exception):
'''Signals using a graph element instance outside of the registered procedure.'''
pass
class Label:
'''Label of a Vertex.'''
__slots__ = ('_name',)
def __init__(self, name):
self._name = name
@property
def name(self) -> str:
return self._name
def __eq__(self, other) -> bool:
if isinstance(other, Label):
return self._name == other.name
if isinstance(other, str):
return self._name == other
return NotImplemented
# Named property value of a Vertex or an Edge.
# It would be better to use typing.NamedTuple with typed fields, but that is
# not available in Python 3.5.
Property = namedtuple('Property', ('name', 'value'))
class Properties:
'''A collection of properties either on a Vertex or an Edge.'''
__slots__ = ('_vertex_or_edge', '_len',)
def __init__(self, vertex_or_edge):
if not isinstance(vertex_or_edge, (_mgp.Vertex, _mgp.Edge)):
raise TypeError("Expected '_mgp.Vertex' or '_mgp.Edge', \
got {}".format(type(vertex_or_edge)))
self._len = None
self._vertex_or_edge = vertex_or_edge
def __deepcopy__(self, memo):
# This is the same as the shallow copy, as the underlying C API should
# not support deepcopy. Besides, it doesn't make much sense to actually
# copy _mgp.Edge and _mgp.Vertex types as they are actually references
# to graph elements and not proper values.
return Properties(self._vertex_or_edge)
def get(self, property_name: str, default=None) -> object:
'''Get the value of a property with the given name or return default.
Raise InvalidContextError.
'''
if not self._vertex_or_edge.is_valid():
raise InvalidContextError()
try:
return self[property_name]
except KeyError:
return default
def items(self) -> typing.Iterable[Property]:
'''Raise InvalidContextError.'''
if not self._vertex_or_edge.is_valid():
raise InvalidContextError()
properties_it = self._vertex_or_edge.iter_properties()
prop = properties_it.get()
while prop is not None:
yield Property(*prop)
if not self._vertex_or_edge.is_valid():
raise InvalidContextError()
prop = properties_it.next()
def keys(self) -> typing.Iterable[str]:
'''Iterate over property names.
Raise InvalidContextError.
'''
if not self._vertex_or_edge.is_valid():
raise InvalidContextError()
for item in self.items():
yield item.name
def values(self) -> typing.Iterable[object]:
'''Iterate over property values.
Raise InvalidContextError.
'''
if not self._vertex_or_edge.is_valid():
raise InvalidContextError()
for item in self.items():
yield item.value
def __len__(self) -> int:
'''Raise InvalidContextError.'''
if not self._vertex_or_edge.is_valid():
raise InvalidContextError()
if self._len is None:
self._len = sum(1 for item in self.items())
return self._len
def __iter__(self) -> typing.Iterable[str]:
'''Iterate over property names.
Raise InvalidContextError.
'''
if not self._vertex_or_edge.is_valid():
raise InvalidContextError()
for item in self.items():
yield item.name
def __getitem__(self, property_name: str) -> object:
'''Get the value of a property with the given name or raise KeyError.
Raise InvalidContextError.'''
if not self._vertex_or_edge.is_valid():
raise InvalidContextError()
prop = self._vertex_or_edge.get_property(property_name)
if prop is None:
raise KeyError()
return prop
def __contains__(self, property_name: str) -> bool:
if not self._vertex_or_edge.is_valid():
raise InvalidContextError()
try:
_ = self[property_name]
return True
except KeyError:
return False
class EdgeType:
'''Type of an Edge.'''
__slots__ = ('_name',)
def __init__(self, name):
self._name = name
@property
def name(self) -> str:
return self._name
def __eq__(self, other) -> bool:
if isinstance(other, EdgeType):
return self.name == other.name
if isinstance(other, str):
return self.name == other
return NotImplemented
if sys.version_info >= (3, 5, 2):
EdgeId = typing.NewType('EdgeId', int)
else:
EdgeId = int
class Edge:
'''Edge in the graph database.
Access to an Edge is only valid during a single execution of a procedure in
a query. You should not globally store an instance of an Edge. Using an
invalid Edge instance will raise InvalidContextError.
'''
__slots__ = ('_edge',)
def __init__(self, edge):
if not isinstance(edge, _mgp.Edge):
raise TypeError("Expected '_mgp.Edge', got '{}'".format(type(edge)))
self._edge = edge
def __deepcopy__(self, memo):
# This is the same as the shallow copy, because we want to share the
# underlying C struct. Besides, it doesn't make much sense to actually
# copy _mgp.Edge as that is actually a reference to a graph element
# and not a proper value.
return Edge(self._edge)
def is_valid(self) -> bool:
'''Return True if `self` is in valid context and may be used.'''
return self._edge.is_valid()
@property
def id(self) -> EdgeId:
'''Raise InvalidContextError.'''
if not self.is_valid():
raise InvalidContextError()
return self._edge.get_id()
@property
def type(self) -> EdgeType:
'''Raise InvalidContextError.'''
if not self.is_valid():
raise InvalidContextError()
return EdgeType(self._edge.get_type_name())
@property
def from_vertex(self): # -> Vertex:
'''Raise InvalidContextError.'''
if not self.is_valid():
raise InvalidContextError()
return Vertex(self._edge.from_vertex())
@property
def to_vertex(self): # -> Vertex:
'''Raise InvalidContextError.'''
if not self.is_valid():
raise InvalidContextError()
return Vertex(self._edge.to_vertex())
@property
def properties(self) -> Properties:
'''Raise InvalidContextError.'''
if not self.is_valid():
raise InvalidContextError()
return Properties(self._edge)
def __eq__(self, other) -> bool:
'''Raise InvalidContextError.'''
if not self.is_valid():
raise InvalidContextError()
if not isinstance(other, Edge):
return NotImplemented
return self._edge == other._edge
def __hash__(self) -> int:
return hash(self.id)
if sys.version_info >= (3, 5, 2):
VertexId = typing.NewType('VertexId', int)
else:
VertexId = int
class Vertex:
'''Vertex in the graph database.
Access to a Vertex is only valid during a single execution of a procedure
in a query. You should not globally store an instance of a Vertex. Using an
invalid Vertex instance will raise InvalidContextError.
'''
__slots__ = ('_vertex',)
def __init__(self, vertex):
if not isinstance(vertex, _mgp.Vertex):
raise TypeError("Expected '_mgp.Vertex', got '{}'".format(type(vertex)))
self._vertex = vertex
def __deepcopy__(self, memo):
# This is the same as the shallow copy, because we want to share the
# underlying C struct. Besides, it doesn't make much sense to actually
# copy _mgp.Vertex as that is actually a reference to a graph element
# and not a proper value.
return Vertex(self._vertex)
def is_valid(self) -> bool:
'''Return True if `self` is in valid context and may be used'''
return self._vertex.is_valid()
@property
def id(self) -> VertexId:
'''Raise InvalidContextError.'''
if not self.is_valid():
raise InvalidContextError()
return self._vertex.get_id()
@property
def labels(self) -> typing.List[Label]:
'''Raise InvalidContextError.'''
if not self.is_valid():
raise InvalidContextError()
return tuple(Label(self._vertex.label_at(i))
for i in range(self._vertex.labels_count()))
@property
def properties(self) -> Properties:
'''Raise InvalidContextError.'''
if not self.is_valid():
raise InvalidContextError()
return Properties(self._vertex)
@property
def in_edges(self) -> typing.Iterable[Edge]:
'''Raise InvalidContextError.'''
if not self.is_valid():
raise InvalidContextError()
edges_it = self._vertex.iter_in_edges()
edge = edges_it.get()
while edge is not None:
yield Edge(edge)
if not self.is_valid():
raise InvalidContextError()
edge = edges_it.next()
@property
def out_edges(self) -> typing.Iterable[Edge]:
'''Raise InvalidContextError.'''
if not self.is_valid():
raise InvalidContextError()
edges_it = self._vertex.iter_out_edges()
edge = edges_it.get()
while edge is not None:
yield Edge(edge)
if not self.is_valid():
raise InvalidContextError()
edge = edges_it.next()
def __eq__(self, other) -> bool:
'''Raise InvalidContextError'''
if not self.is_valid():
raise InvalidContextError()
if not isinstance(other, Vertex):
return NotImplemented
return self._vertex == other._vertex
def __hash__(self) -> int:
return hash(self.id)
class Path:
'''Path containing Vertex and Edge instances.'''
__slots__ = ('_path', '_vertices', '_edges')
def __init__(self, starting_vertex_or_path: typing.Union[_mgp.Path, Vertex]):
'''Initialize with a starting Vertex.
Raise InvalidContextError if passed in Vertex is invalid.
'''
# We cache calls to `vertices` and `edges`, so as to avoid needless
# allocations at the C level.
self._vertices = None
self._edges = None
# Accepting _mgp.Path is just for internal usage.
if isinstance(starting_vertex_or_path, _mgp.Path):
self._path = starting_vertex_or_path
elif isinstance(starting_vertex_or_path, Vertex):
vertex = starting_vertex_or_path._vertex
if not vertex.is_valid():
raise InvalidContextError()
self._path = _mgp.Path.make_with_start(vertex)
else:
raise TypeError("Expected '_mgp.Vertex' or '_mgp.Path', got '{}'"
.format(type(starting_vertex_or_path)))
def __copy__(self):
if not self.is_valid():
raise InvalidContextError()
assert len(self.vertices) >= 1
path = Path(self.vertices[0])
for e in self.edges:
path.expand(e)
return path
def __deepcopy__(self, memo):
try:
return Path(memo[id(self._path)])
except KeyError:
pass
# This is the same as the shallow copy, as the underlying C API should
# not support deepcopy. Besides, it doesn't make much sense to actually
# copy _mgp.Edge and _mgp.Vertex types as they are actually references
# to graph elements and not proper values.
path = self.__copy__()
memo[id(self._path)] = path._path
return path
def is_valid(self) -> bool:
return self._path.is_valid()
def expand(self, edge: Edge):
'''Append an edge continuing from the last vertex on the path.
The last vertex on the path will become the other endpoint of the given
edge, as continued from the current last vertex.
Raise ValueError if the current last vertex in the path is not part of
the given edge.
Raise InvalidContextError if using an invalid Path instance or if
passed in edge is invalid.
'''
if not isinstance(edge, Edge):
raise TypeError("Expected '_mgp.Edge', got '{}'".format(type(edge)))
if not self.is_valid() or not edge.is_valid():
raise InvalidContextError()
self._path.expand(edge._edge)
# Invalidate our cached tuples
self._vertices = None
self._edges = None
@property
def vertices(self) -> typing.Tuple[Vertex, ...]:
'''Vertices ordered from the start to the end of the path.
Raise InvalidContextError if using an invalid Path instance.'''
if not self.is_valid():
raise InvalidContextError()
if self._vertices is None:
num_vertices = self._path.size() + 1
self._vertices = tuple(Vertex(self._path.vertex_at(i))
for i in range(num_vertices))
return self._vertices
@property
def edges(self) -> typing.Tuple[Edge, ...]:
'''Edges ordered from the start to the end of the path.
Raise InvalidContextError if using an invalid Path instance.'''
if not self.is_valid():
raise InvalidContextError()
if self._edges is None:
num_edges = self._path.size()
self._edges = tuple(Edge(self._path.edge_at(i))
for i in range(num_edges))
return self._edges
class Record:
'''Represents a record of resulting field values.'''
__slots__ = ('fields',)
def __init__(self, **kwargs):
'''Initialize with name=value fields in kwargs.'''
self.fields = kwargs
class Vertices:
'''Iterable over vertices in a graph.'''
__slots__ = ('_graph', '_len')
def __init__(self, graph):
if not isinstance(graph, _mgp.Graph):
raise TypeError("Expected '_mgp.Graph', got '{}'".format(type(graph)))
self._graph = graph
self._len = None
def __deepcopy__(self, memo):
# This is the same as the shallow copy, because we want to share the
# underlying C struct. Besides, it doesn't make much sense to actually
# copy _mgp.Graph as that always references the whole graph state.
return Vertices(self._graph)
def is_valid(self) -> bool:
'''Return True if `self` is in valid context and may be used.'''
return self._graph.is_valid()
def __iter__(self) -> typing.Iterable[Vertex]:
'''Raise InvalidContextError if context is invalid.'''
if not self.is_valid():
raise InvalidContextError()
vertices_it = self._graph.iter_vertices()
vertex = vertices_it.get()
while vertex is not None:
yield Vertex(vertex)
if not self.is_valid():
raise InvalidContextError()
vertex = vertices_it.next()
def __contains__(self, vertex):
try:
_ = self._graph.get_vertex_by_id(vertex.id)
return True
except IndexError:
return False
def __len__(self):
if not self._len:
self._len = sum(1 for _ in self)
return self._len
class Graph:
'''State of the graph database in current ProcCtx.'''
__slots__ = ('_graph',)
def __init__(self, graph):
if not isinstance(graph, _mgp.Graph):
raise TypeError("Expected '_mgp.Graph', got '{}'".format(type(graph)))
self._graph = graph
def __deepcopy__(self, memo):
# This is the same as the shallow copy, because we want to share the
# underlying C struct. Besides, it doesn't make much sense to actually
# copy _mgp.Graph as that always references the whole graph state.
return Graph(self._graph)
def is_valid(self) -> bool:
'''Return True if `self` is in valid context and may be used.'''
return self._graph.is_valid()
def get_vertex_by_id(self, vertex_id: VertexId) -> Vertex:
'''Return the Vertex corresponding to given vertex_id from the graph.
Access to a Vertex is only valid during a single execution of a
procedure in a query. You should not globally store the returned
Vertex.
Raise IndexError if unable to find the given vertex_id.
Raise InvalidContextError if context is invalid.
'''
if not self.is_valid():
raise InvalidContextError()
vertex = self._graph.get_vertex_by_id(vertex_id)
return Vertex(vertex)
@property
def vertices(self) -> Vertices:
'''All vertices in the graph.
Access to a Vertex is only valid during a single execution of a
procedure in a query. You should not globally store the returned Vertex
instances.
Raise InvalidContextError if context is invalid.
'''
if not self.is_valid():
raise InvalidContextError()
return Vertices(self._graph)
class AbortError(Exception):
'''Signals that the procedure was asked to abort its execution.'''
pass
class ProcCtx:
'''Context of a procedure being executed.
Access to a ProcCtx is only valid during a single execution of a procedure
in a query. You should not globally store a ProcCtx instance.
'''
__slots__ = ('_graph',)
def __init__(self, graph):
if not isinstance(graph, _mgp.Graph):
raise TypeError("Expected '_mgp.Graph', got '{}'".format(type(graph)))
self._graph = Graph(graph)
def is_valid(self) -> bool:
return self._graph.is_valid()
@property
def graph(self) -> Graph:
'''Raise InvalidContextError if context is invalid.'''
if not self.is_valid():
raise InvalidContextError()
return self._graph
def must_abort(self) -> bool:
if not self.is_valid():
raise InvalidContextError()
return self._graph._graph.must_abort()
def check_must_abort(self):
if self.must_abort():
raise AbortError
# Additional typing support
Number = typing.Union[int, float]
Map = typing.Union[dict, Edge, Vertex]
Any = typing.Union[bool, str, Number, Map, Path, list]
List = typing.List
Nullable = typing.Optional
class UnsupportedTypingError(Exception):
'''Signals a typing annotation is not supported as a _mgp.CypherType.'''
def __init__(self, type_):
super().__init__("Unsupported typing annotation '{}'".format(type_))
def _typing_to_cypher_type(type_):
'''Convert typing annotation to a _mgp.CypherType instance.'''
simple_types = {
typing.Any: _mgp.type_nullable(_mgp.type_any()),
object: _mgp.type_nullable(_mgp.type_any()),
list: _mgp.type_list(_mgp.type_nullable(_mgp.type_any())),
Any: _mgp.type_any(),
bool: _mgp.type_bool(),
str: _mgp.type_string(),
int: _mgp.type_int(),
float: _mgp.type_float(),
Number: _mgp.type_number(),
Map: _mgp.type_map(),
Vertex: _mgp.type_node(),
Edge: _mgp.type_relationship(),
Path: _mgp.type_path()
}
try:
return simple_types[type_]
except KeyError:
pass
if sys.version_info >= (3, 8):
complex_type = typing.get_origin(type_)
type_args = typing.get_args(type_)
if complex_type == typing.Union:
# If we have a Union with NoneType inside, it means we are building
# a nullable type.
if isinstance(None, type_args):
types = tuple(t for t in type_args if not isinstance(None, t))
if len(types) == 1:
type_arg, = types
else:
# We cannot do typing.Union[*types], so do the equivalent
# with __getitem__ which does not even need arg unpacking.
type_arg = typing.Union.__getitem__(types)
return _mgp.type_nullable(_typing_to_cypher_type(type_arg))
elif complex_type == list:
type_arg, = type_args
return _mgp.type_list(_typing_to_cypher_type(type_arg))
raise UnsupportedTypingError(type_)
else:
# We cannot get to type args in any reliable way prior to 3.8, but we
# still want to support typing.Optional and typing.List, so just parse
# their string representations. Hopefully, that is always pretty
# printed the same way. `typing.List[type]` is printed as such, while
# `typing.Optional[type]` is printed as 'typing.Union[type, NoneType]'
def parse_type_args(type_as_str):
return tuple(map(str.strip,
type_as_str[type_as_str.index('[') + 1: -1].split(',')))
def fully_qualified_name(cls):
if cls.__module__ is None or cls.__module__ == 'builtins':
return cls.__name__
return cls.__module__ + '.' + cls.__name__
def get_simple_type(type_as_str):
for simple_type, cypher_type in simple_types.items():
if type_as_str == str(simple_type):
return cypher_type
# Fallback to comparing to __name__ if it exits. This handles
# the cases like when we have 'object' which is
# `object.__name__`, but `str(object)` is "<class 'object'>"
try:
if type_as_str == fully_qualified_name(simple_type):
return cypher_type
except AttributeError:
pass
def parse_typing(type_as_str):
if type_as_str.startswith('typing.Union'):
type_args_as_str = parse_type_args(type_as_str)
none_type_as_str = type(None).__name__
if none_type_as_str in type_args_as_str:
types = tuple(t for t in type_args_as_str if t != none_type_as_str)
if len(types) == 1:
type_arg_as_str, = types
else:
type_arg_as_str = 'typing.Union[' + ', '.join(types) + ']'
simple_type = get_simple_type(type_arg_as_str)
if simple_type is not None:
return _mgp.type_nullable(simple_type)
return _mgp.type_nullable(parse_typing(type_arg_as_str))
elif type_as_str.startswith('typing.List'):
type_arg_as_str = parse_type_args(type_as_str)
if len(type_arg_as_str) > 1:
# Nested object could be a type consisting of a list of types (e.g. mgp.Map)
# so we need to join the parts.
type_arg_as_str = ', '.join(type_arg_as_str)
else:
type_arg_as_str = type_arg_as_str[0]
simple_type = get_simple_type(type_arg_as_str)
if simple_type is not None:
return _mgp.type_list(simple_type)
return _mgp.type_list(parse_typing(type_arg_as_str))
raise UnsupportedTypingError(type_)
return parse_typing(str(type_))
# Procedure registration
class Deprecated:
'''Annotate a resulting Record's field as deprecated.'''
__slots__ = ('field_type',)
def __init__(self, type_):
self.field_type = type_
def read_proc(func: typing.Callable[..., Record]):
'''
Register `func` as a a read-only procedure of the current module.
`read_proc` is meant to be used as a decorator function to register module
procedures. The registered `func` needs to be a callable which optionally
takes `ProcCtx` as the first argument. Other arguments of `func` will be
bound to values passed in the cypherQuery. The full signature of `func`
needs to be annotated with types. The return type must be
`Record(field_name=type, ...)` and the procedure must produce either a
complete Record or None. To mark a field as deprecated, use
`Record(field_name=Deprecated(type), ...)`. Multiple records can be
produced by returning an iterable of them. Registering generator functions
is currently not supported.
Example usage.
```
import mgp
@mgp.read_proc
def procedure(context: mgp.ProcCtx,
required_arg: mgp.Nullable[mgp.Any],
optional_arg: mgp.Nullable[mgp.Any] = None
) -> mgp.Record(result=str, args=list):
args = [required_arg, optional_arg]
# Multiple rows can be produced by returning an iterable of mgp.Record
return mgp.Record(args=args, result='Hello World!')
```
The example procedure above returns 2 fields: `args` and `result`.
* `args` is a copy of arguments passed to the procedure.
* `result` is the result of this procedure, a "Hello World!" string.
Any errors can be reported by raising an Exception.
The procedure can be invoked in openCypher using the following calls:
CALL example.procedure(1, 2) YIELD args, result;
CALL example.procedure(1) YIELD args, result;
Naturally, you may pass in different arguments or yield less fields.
'''
if not callable(func):
raise TypeError("Expected a callable object, got an instance of '{}'"
.format(type(func)))
if inspect.iscoroutinefunction(func):
raise TypeError("Callable must not be 'async def' function")
if sys.version_info >= (3, 6):
if inspect.isasyncgenfunction(func):
raise TypeError("Callable must not be 'async def' function")
if inspect.isgeneratorfunction(func):
raise NotImplementedError("Generator functions are not supported")
sig = inspect.signature(func)
params = tuple(sig.parameters.values())
if params and params[0].annotation is ProcCtx:
@functools.wraps(func)
def wrapper(graph, args):
return func(ProcCtx(graph), *args)
params = params[1:]
mgp_proc = _mgp._MODULE.add_read_procedure(wrapper)
else:
@functools.wraps(func)
def wrapper(graph, args):
return func(*args)
mgp_proc = _mgp._MODULE.add_read_procedure(wrapper)
for param in params:
name = param.name
type_ = param.annotation
if type_ is param.empty:
type_ = object
cypher_type = _typing_to_cypher_type(type_)
if param.default is param.empty:
mgp_proc.add_arg(name, cypher_type)
else:
mgp_proc.add_opt_arg(name, cypher_type, param.default)
if sig.return_annotation is not sig.empty:
record = sig.return_annotation
if not isinstance(record, Record):
raise TypeError("Expected '{}' to return 'mgp.Record', got '{}'"
.format(func.__name__, type(record)))
for name, type_ in record.fields.items():
if isinstance(type_, Deprecated):
cypher_type = _typing_to_cypher_type(type_.field_type)
mgp_proc.add_deprecated_result(name, cypher_type)
else:
mgp_proc.add_result(name, _typing_to_cypher_type(type_))
return func

112
init
View File

@@ -1,112 +0,0 @@
#!/bin/bash -e
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
cd "$DIR"
source "$DIR/environment/util.sh"
function print_help () {
echo "Usage: $0 [OPTION]"
echo -e "Check for missing packages and setup the project.\n"
echo "Optional arguments:"
echo -e " -h\tdisplay this help and exit"
}
function setup_virtualenv () {
pushd $1 > /dev/null
echo "Setting up virtualenv for: $1"
# remove old virtualenv
if [ -d ve3 ]; then
rm -rf ve3
fi
# create new virtualenv
virtualenv -p python3 ve3 || exit 1
source ve3/bin/activate
pip --timeout 1000 install -r requirements.txt || exit 1
deactivate
popd > /dev/null
}
if [[ $# -gt 1 ]]; then
print_help
exit 1
elif [[ $# -eq 1 ]]; then
case "$1" in
-h)
print_help
exit 0
;;
*)
# unknown option
print_help
exit 1
;;
esac
fi
DISTRO=$(operating_system)
echo "ALL BUILD PACKAGES: $($DIR/environment/os/$DISTRO.sh list MEMGRAPH_BUILD_DEPS)"
$DIR/environment/os/$DISTRO.sh check MEMGRAPH_BUILD_DEPS
echo "All packages are in-place..."
# create a default build directory
mkdir -p ./build
# quicklisp package manager for Common Lisp
quicklisp_install_dir="$HOME/quicklisp"
if [[ -v QUICKLISP_HOME ]]; then
quicklisp_install_dir="${QUICKLISP_HOME}"
fi
if [[ ! -f "${quicklisp_install_dir}/setup.lisp" ]]; then
wget -nv https://beta.quicklisp.org/quicklisp.lisp -O quicklisp.lisp || exit 1
echo \
"
(load \"${DIR}/quicklisp.lisp\")
(quicklisp-quickstart:install :path \"${quicklisp_install_dir}\")
" | sbcl --script || exit 1
rm -rf quicklisp.lisp || exit 1
fi
ln -Tfs "$DIR/src/lisp" "${quicklisp_install_dir}/local-projects/lcp"
# Install LCP dependencies
# TODO: We should at some point cache or have a mirror of packages we use.
# TODO: move the installation of LCP's dependencies into ./setup.sh
echo \
"
(load \"${quicklisp_install_dir}/setup.lisp\")
(ql:quickload '(:lcp :lcp/test) :silent t)
" | sbcl --script
# Setup libs (download).
cd libs
./cleanup.sh
./setup.sh
cd ..
# setup gql_behave dependencies
setup_virtualenv tests/gql_behave
# setup stress dependencies
setup_virtualenv tests/stress
# setup integration/ldap dependencies
setup_virtualenv tests/integration/ldap
# Setup tests dependencies.
# cd tests
# ./setup.sh
# cd ..
# TODO(gitbuda): Remove setup_virtualenv, replace it with tests/ve3. Take care
# of the build order because tests/setup.py builds pymgclient which depends on
# mgclient which is build after this script by calling make.
echo "Done installing dependencies for Memgraph"
echo "Linking git hooks"
for hook in $(find $DIR/.githooks -type f -printf "%f\n"); do
ln -s -f "$DIR/.githooks/$hook" "$DIR/.git/hooks/$hook"
echo "Added $hook hook"
done;

7
libs/.gitignore vendored
View File

@@ -1,7 +0,0 @@
*
!.gitignore
!setup.sh
!cleanup.sh
!CMakeLists.txt
!__main.cpp
!jemalloc.cmake

View File

@@ -1,218 +0,0 @@
include(ExternalProject)
include(GNUInstallDirs)
include(ProcessorCount)
ProcessorCount(NPROC)
if (NPROC EQUAL 0)
set(NPROC 1)
endif()
set(LIB_DIR ${CMAKE_CURRENT_SOURCE_DIR})
# convenience functions
function(import_header_library name include_dir)
add_library(${name} INTERFACE IMPORTED GLOBAL)
set_property(TARGET ${name} PROPERTY
INTERFACE_INCLUDE_DIRECTORIES ${include_dir})
string(TOUPPER ${name} _upper_name)
set(${_upper_name}_INCLUDE_DIR ${include_dir} CACHE FILEPATH
"Path to ${name} include directory" FORCE)
mark_as_advanced(${_upper_name}_INCLUDE_DIR)
endfunction(import_header_library)
function(import_library name type location include_dir)
add_library(${name} ${type} IMPORTED GLOBAL)
if (${ARGN})
# Optional argument is the name of the external project that we need to
# depend on.
add_dependencies(${name} ${ARGN0})
else()
add_dependencies(${name} ${name}-proj)
endif()
set_property(TARGET ${name} PROPERTY IMPORTED_LOCATION ${location})
# We need to create the include directory first in order to be able to add it
# as an include directory. The header files in the include directory will be
# generated later during the build process.
file(MAKE_DIRECTORY ${include_dir})
target_include_directories(${name} INTERFACE ${include_dir})
endfunction(import_library)
# Calls `ExternalProject_Add(${name}-proj` with default arguments for cmake
# configuration. CMAKE_BUILD_TYPE is set to Release, CMAKE_C_COMPILER and
# CMAKE_CXX_COMPILER are forwarded as used in this project. You can pass
# NO_C_COMPILER option to avoid forwarding CMAKE_C_COMPILER. Installation is
# done in SOURCE_DIR, which defaults to ${CMAKE_CURRENT_SOURCE_DIR}/${name}.
# You can pass additional arguments via CMAKE_ARGS. Dependencies and
# installation can be set as in regular ExternalProject_Add, via DEPENDS and
# INSTALL_COMMAND arguments.
function(add_external_project name)
set(options NO_C_COMPILER)
set(one_value_kwargs SOURCE_DIR)
set(multi_value_kwargs CMAKE_ARGS DEPENDS INSTALL_COMMAND BUILD_COMMAND
CONFIGURE_COMMAND)
cmake_parse_arguments(KW "${options}" "${one_value_kwargs}" "${multi_value_kwargs}" ${ARGN})
set(source_dir ${CMAKE_CURRENT_SOURCE_DIR}/${name})
if (KW_SOURCE_DIR)
set(source_dir ${KW_SOURCE_DIR})
endif()
if (NOT KW_NO_C_COMPILER)
set(KW_CMAKE_ARGS -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER} ${KW_CMAKE_ARGS})
endif()
ExternalProject_Add(${name}-proj DEPENDS ${KW_DEPENDS}
PREFIX ${source_dir} SOURCE_DIR ${source_dir}
CONFIGURE_COMMAND ${KW_CONFIGURE_COMMAND}
CMAKE_ARGS -DCMAKE_BUILD_TYPE=Release
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_INSTALL_PREFIX=${source_dir}
${KW_CMAKE_ARGS}
INSTALL_COMMAND ${KW_INSTALL_COMMAND}
BUILD_COMMAND ${KW_BUILD_COMMAND})
endfunction(add_external_project)
# Calls `add_external_project`, sets NAME_LIBRARY, NAME_INCLUDE_DIR variables
# and adds the library via `import_library`.
macro(import_external_library name type library_location include_dir)
add_external_project(${name} ${ARGN})
string(TOUPPER ${name} _upper_name)
set(${_upper_name}_LIBRARY ${library_location} CACHE FILEPATH
"Path to ${name} library" FORCE)
set(${_upper_name}_INCLUDE_DIR ${include_dir} CACHE FILEPATH
"Path to ${name} include directory" FORCE)
mark_as_advanced(${_upper_name}_LIBRARY ${_upper_name}_INCLUDE_DIR)
import_library(${name} ${type} ${${_upper_name}_LIBRARY} ${${_upper_name}_INCLUDE_DIR})
endmacro(import_external_library)
# setup antlr
import_external_library(antlr4 STATIC
${CMAKE_CURRENT_SOURCE_DIR}/antlr4/runtime/Cpp/lib/libantlr4-runtime.a
${CMAKE_CURRENT_SOURCE_DIR}/antlr4/runtime/Cpp/include/antlr4-runtime
SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/antlr4/runtime/Cpp
CMAKE_ARGS # http://stackoverflow.com/questions/37096062/get-a-basic-c-program-to-compile-using-clang-on-ubuntu-16/38385967#38385967
-DWITH_LIBCXX=OFF # because of debian bug
-DCMAKE_SKIP_INSTALL_ALL_DEPENDENCY=true
BUILD_COMMAND $(MAKE) antlr4_static
# Make a License.txt out of thin air, so that antlr4.6 knows how to build.
# When we upgrade antlr, this will no longer be needed.
INSTALL_COMMAND touch ${CMAKE_CURRENT_SOURCE_DIR}/antlr4/runtime/Cpp/License.txt
COMMAND $(MAKE) install)
# Setup google benchmark.
import_external_library(benchmark STATIC
${CMAKE_CURRENT_SOURCE_DIR}/benchmark/lib/libbenchmark.a
${CMAKE_CURRENT_SOURCE_DIR}/benchmark/include
# Skip testing. The tests don't compile with Clang 8.
CMAKE_ARGS -DBENCHMARK_ENABLE_TESTING=OFF)
# setup fmt format
import_external_library(fmt STATIC
${CMAKE_CURRENT_SOURCE_DIR}/fmt/${CMAKE_INSTALL_LIBDIR}/libfmt.a
${CMAKE_CURRENT_SOURCE_DIR}/fmt/include
# Skip testing.
CMAKE_ARGS -DFMT_TEST=OFF)
# setup rapidcheck (it cannot be external, since it doesn't have install
# target)
set(RC_ENABLE_GTEST ON CACHE BOOL "Build Google Test integration" FORCE)
set(RC_ENABLE_GMOCK ON CACHE BOOL "Build Google Mock integration" FORCE)
mark_as_advanced(RC_ENABLE_GTEST RC_ENABLE_GMOCK)
add_subdirectory(rapidcheck EXCLUDE_FROM_ALL)
# setup google test
add_external_project(gtest SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/googletest)
set(GTEST_INCLUDE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/googletest/include
CACHE PATH "Path to gtest and gmock include directory" FORCE)
set(GMOCK_LIBRARY ${CMAKE_CURRENT_SOURCE_DIR}/googletest/lib/libgmock.a
CACHE FILEPATH "Path to gmock library" FORCE)
set(GMOCK_MAIN_LIBRARY ${CMAKE_CURRENT_SOURCE_DIR}/googletest/lib/libgmock_main.a
CACHE FILEPATH "Path to gmock_main library" FORCE)
set(GTEST_LIBRARY ${CMAKE_CURRENT_SOURCE_DIR}/googletest/lib/libgtest.a
CACHE FILEPATH "Path to gtest library" FORCE)
set(GTEST_MAIN_LIBRARY ${CMAKE_CURRENT_SOURCE_DIR}/googletest/lib/libgtest_main.a
CACHE FILEPATH "Path to gtest_main library" FORCE)
mark_as_advanced(GTEST_INCLUDE_DIR GMOCK_LIBRARY GMOCK_MAIN_LIBRARY GTEST_LIBRARY GTEST_MAIN_LIBRARY)
import_library(gtest STATIC ${GTEST_LIBRARY} ${GTEST_INCLUDE_DIR} gtest-proj)
import_library(gtest_main STATIC ${GTEST_MAIN_LIBRARY} ${GTEST_INCLUDE_DIR} gtest-proj)
import_library(gmock STATIC ${GMOCK_LIBRARY} ${GTEST_INCLUDE_DIR} gtest-proj)
import_library(gmock_main STATIC ${GMOCK_MAIN_LIBRARY} ${GTEST_INCLUDE_DIR} gtest-proj)
# setup google flags
set(GFLAGS_NO_FILENAMES "0")
if ("${CMAKE_BUILD_TYPE}" MATCHES "^(R|r)(E|e)(L|l).+")
set(GFLAGS_NO_FILENAMES "1")
endif()
# setup google flags
import_external_library(gflags STATIC
${CMAKE_CURRENT_SOURCE_DIR}/gflags/lib/libgflags.a
${CMAKE_CURRENT_SOURCE_DIR}/gflags/include
# Not needed, since gflags is C++ only.
NO_C_COMPILER
# Don't register installation in ~/.cmake
CMAKE_ARGS -DREGISTER_INSTALL_PREFIX=OFF
-DBUILD_gflags_nothreads_LIB=OFF
-DGFLAGS_NO_FILENAMES=${GFLAGS_NO_FILENAMES})
# Setup cppitertools
import_header_library(cppitertools ${CMAKE_CURRENT_SOURCE_DIR})
# Setup json
import_header_library(json ${CMAKE_CURRENT_SOURCE_DIR})
# Setup bzip2
import_external_library(bzip2 STATIC
${CMAKE_CURRENT_SOURCE_DIR}/bzip2/libbz2.a
${CMAKE_CURRENT_SOURCE_DIR}/bzip2
# bzip2's Makefile has -g CFLAG which is redundant
CONFIGURE_COMMAND sed -i "s/-Wall -Winline -O2 -g/-Wall -Winline -O2/g" ${CMAKE_CURRENT_SOURCE_DIR}/bzip2/Makefile
BUILD_COMMAND make -C ${CMAKE_CURRENT_SOURCE_DIR}/bzip2
CC=${CMAKE_C_COMPILER}
CXX=${CMAKE_CXX_COMPILER}
INSTALL_COMMAND true)
# Setup zlib
import_external_library(zlib STATIC
${CMAKE_CURRENT_SOURCE_DIR}/zlib/lib/libz.a
${CMAKE_CURRENT_SOURCE_DIR}/zlib
CMAKE_ARGS -DCMAKE_SKIP_INSTALL_ALL_DEPENDENCY=true
BUILD_COMMAND $(MAKE) zlibstatic)
# Setup RocksDB
import_external_library(rocksdb STATIC
${CMAKE_CURRENT_SOURCE_DIR}/rocksdb/lib/librocksdb.a
${CMAKE_CURRENT_SOURCE_DIR}/rocksdb/include
DEPENDS gflags-proj
CMAKE_ARGS -Dgflags_DIR=${CMAKE_CURRENT_SOURCE_DIR}/gflags/lib/cmake/gflags
-DUSE_RTTI=ON
-DWITH_TESTS=OFF
-DCMAKE_INSTALL_LIBDIR=lib
-DCMAKE_SKIP_INSTALL_ALL_DEPENDENCY=true
BUILD_COMMAND $(MAKE) rocksdb)
# Setup libbcrypt
import_external_library(libbcrypt STATIC
${CMAKE_CURRENT_SOURCE_DIR}/libbcrypt/bcrypt.a
${CMAKE_CURRENT_SOURCE_DIR}/libbcrypt
CONFIGURE_COMMAND sed s/-Wcast-align// -i ${CMAKE_CURRENT_SOURCE_DIR}/libbcrypt/crypt_blowfish/Makefile
BUILD_COMMAND make -C ${CMAKE_CURRENT_SOURCE_DIR}/libbcrypt
CC=${CMAKE_C_COMPILER}
CXX=${CMAKE_CXX_COMPILER}
INSTALL_COMMAND true)
# Setup mgclient
import_external_library(mgclient STATIC
${CMAKE_CURRENT_SOURCE_DIR}/mgclient/lib/libmgclient.a
${CMAKE_CURRENT_SOURCE_DIR}/mgclient/include
CMAKE_ARGS -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DBUILD_TESTING=OFF)
find_package(OpenSSL REQUIRED)
target_link_libraries(mgclient INTERFACE ${OPENSSL_LIBRARIES})
# Setup spdlog
import_external_library(spdlog STATIC
${CMAKE_CURRENT_SOURCE_DIR}/spdlog/${CMAKE_INSTALL_LIBDIR}/libspdlog.a
${CMAKE_CURRENT_SOURCE_DIR}/spdlog/include
BUILD_COMMAND $(MAKE) spdlog)
include(jemalloc.cmake)

View File

@@ -1,15 +0,0 @@
#!/bin/bash
# go to script directory
working_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
cd ${working_dir}
# remove archives
rm *.jar *.tar.gz *.tar 2>/dev/null
# remove lib directories
for folder in * ; do
if [ -d "$folder" ]; then
rm -rf $folder
fi
done

View File

@@ -1,55 +0,0 @@
set(JEMALLOC_DIR "${LIB_DIR}/jemalloc")
set(JEMALLOC_SRCS
${JEMALLOC_DIR}/src/arena.c
${JEMALLOC_DIR}/src/background_thread.c
${JEMALLOC_DIR}/src/base.c
${JEMALLOC_DIR}/src/bin.c
${JEMALLOC_DIR}/src/bitmap.c
${JEMALLOC_DIR}/src/ckh.c
${JEMALLOC_DIR}/src/ctl.c
${JEMALLOC_DIR}/src/div.c
${JEMALLOC_DIR}/src/extent.c
${JEMALLOC_DIR}/src/extent_dss.c
${JEMALLOC_DIR}/src/extent_mmap.c
${JEMALLOC_DIR}/src/hash.c
${JEMALLOC_DIR}/src/hook.c
${JEMALLOC_DIR}/src/jemalloc.c
${JEMALLOC_DIR}/src/large.c
${JEMALLOC_DIR}/src/log.c
${JEMALLOC_DIR}/src/malloc_io.c
${JEMALLOC_DIR}/src/mutex.c
${JEMALLOC_DIR}/src/mutex_pool.c
${JEMALLOC_DIR}/src/nstime.c
${JEMALLOC_DIR}/src/pages.c
${JEMALLOC_DIR}/src/prng.c
${JEMALLOC_DIR}/src/prof.c
${JEMALLOC_DIR}/src/rtree.c
${JEMALLOC_DIR}/src/sc.c
${JEMALLOC_DIR}/src/stats.c
${JEMALLOC_DIR}/src/sz.c
${JEMALLOC_DIR}/src/tcache.c
${JEMALLOC_DIR}/src/test_hooks.c
${JEMALLOC_DIR}/src/ticker.c
${JEMALLOC_DIR}/src/tsd.c
${JEMALLOC_DIR}/src/witness.c
${JEMALLOC_DIR}/src/safety_check.c
)
add_library(jemalloc ${JEMALLOC_SRCS})
target_include_directories(jemalloc PUBLIC "${JEMALLOC_DIR}/include")
find_package(Threads REQUIRED)
target_link_libraries(jemalloc PUBLIC Threads::Threads)
target_compile_definitions(jemalloc PRIVATE -DJEMALLOC_NO_PRIVATE_NAMESPACE)
if (CMAKE_BUILD_TYPE STREQUAL "DEBUG")
target_compile_definitions(jemalloc PRIVATE -DJEMALLOC_DEBUG=1 -DJEMALLOC_PROF=1)
endif()
target_compile_options(jemalloc PRIVATE -Wno-redundant-decls)
# for RTLD_NEXT
target_compile_definitions(jemalloc PRIVATE _GNU_SOURCE)
set_property(TARGET jemalloc APPEND PROPERTY INTERFACE_COMPILE_DEFINITIONS USE_JEMALLOC=1)

View File

@@ -1,234 +0,0 @@
#!/bin/bash -e
# Download external dependencies.
local_cache_host=${MGDEPS_CACHE_HOST_PORT:-mgdeps-cache:8000}
working_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
cd "${working_dir}"
# Clones a git repository and optionally cherry picks additional commits. The
# function will try to preserve any local changes in the repo.
# clone GIT_REPO DIR_NAME CHECKOUT_ID [CHERRY_PICK_ID]...
clone () {
local git_repo=$1
local dir_name=$2
local checkout_id=$3
shift 3
# Clone if there's no repo.
if [[ ! -d "$dir_name" ]]; then
echo "Cloning from $git_repo"
# If the clone fails, it doesn't make sense to continue with the function
# execution but the whole script should continue executing because we might
# clone the same repo from a different source.
git clone "$git_repo" "$dir_name" || return 1
fi
pushd "$dir_name"
# Just fetch new commits from remote repository. Don't merge/pull them in, so
# that we don't clobber local modifications.
git fetch
# Check whether we have any local changes which need to be preserved.
local local_changes=true
if git diff --no-ext-diff --quiet && git diff --no-ext-diff --cached --quiet; then
local_changes=false
fi
# Stash regardless of local_changes, so that a user gets a message on stdout.
git stash
# Checkout the primary commit (there's no need to pull/merge).
# The checkout fail should exit this script immediately because the target
# commit is not there and that will most likely create build-time errors.
git checkout "$checkout_id" || exit 1
# Apply any optional cherry pick fixes.
while [[ $# -ne 0 ]]; do
local cherry_pick_id=$1
shift
# The cherry-pick fail should exit this script immediately because the
# target commit is not there and that will most likely create build-time
# errors.
git cherry-pick -n "$cherry_pick_id" || exit 1
done
# Reapply any local changes.
if [[ $local_changes == true ]]; then
git stash pop
fi
popd
}
file_get_try_double () {
primary_url="$1"
secondary_url="$2"
echo "Download primary from $primary_url secondary from $secondary_url"
if [ -z "$primary_url" ]; then echo "Primary should not be empty." && exit 1; fi
if [ -z "$secondary_url" ]; then echo "Secondary should not be empty." && exit 1; fi
filename="$(basename "$secondary_url")"
wget -nv "$primary_url" -O "$filename" || wget -nv "$secondary_url" -O "$filename" || exit 1
echo ""
}
repo_clone_try_double () {
primary_url="$1"
secondary_url="$2"
folder_name="$3"
ref="$4"
echo "Cloning primary from $primary_url secondary from $secondary_url"
if [ -z "$primary_url" ]; then echo "Primary should not be empty." && exit 1; fi
if [ -z "$secondary_url" ]; then echo "Secondary should not be empty." && exit 1; fi
if [ -z "$folder_name" ]; then echo "Clone folder should not be empty." && exit 1; fi
if [ -z "$ref" ]; then echo "Git clone ref should not be empty." && exit 1; fi
clone "$primary_url" "$folder_name" "$ref" || clone "$secondary_url" "$folder_name" "$ref" || exit 1
echo ""
}
# List all dependencies.
# The reason for introducing primary and secondary urls are:
# * HTTPS is hard to cache
# * Remote development workflow is more flexible if people don't have to connect to VPN
# * Direct download from the "source of truth" is slower and unreliable because of the whole internet in-between
# * When a new dependency has to be added, both urls could be the same, later someone could optimize if required
# The goal of having primary urls is to have links to the "local" cache of
# dependencies where these dependencies could be downloaded as fast as
# possible. The actual cache server could be on your local machine, on a
# dedicated machine inside the build cluster or on the actual build machine.
# Download from primary_urls might fail because the cache is not installed.
declare -A primary_urls=(
["antlr4-code"]="http://$local_cache_host/git/antlr4.git"
["antlr4-generator"]="http://$local_cache_host/file/antlr-4.6-complete.jar"
["cppitertools"]="http://$local_cache_host/git/cppitertools.git"
["fmt"]="http://$local_cache_host/git/fmt.git"
["rapidcheck"]="http://$local_cache_host/git/rapidcheck.git"
["gbenchmark"]="http://$local_cache_host/git/benchmark.git"
["gtest"]="http://$local_cache_host/git/googletest.git"
["gflags"]="http://$local_cache_host/git/gflags.git"
["libbcrypt"]="http://$local_cache_host/git/libbcrypt.git"
["bzip2"]="http://$local_cache_host/git/bzip2.git"
["zlib"]="http://$local_cache_host/git/zlib.git"
["rocksdb"]="http://$local_cache_host/git/rocksdb.git"
["mgclient"]="http://$local_cache_host/git/mgclient.git"
["pymgclient"]="http://$local_cache_host/git/pymgclient.git"
["spdlog"]="http://$local_cache_host/git/spdlog"
["jemalloc"]="http://$local_cache_host/git/jemalloc.git"
["nlohmann"]="http://$local_cache_host/file/nlohmann/json/b3e5cb7f20dcc5c806e418df34324eca60d17d4e/single_include/nlohmann/json.hpp"
["neo4j"]="http://$local_cache_host/file/neo4j-community-3.2.3-unix.tar.gz"
)
# The goal of secondary urls is to have links to the "source of truth" of
# dependencies, e.g., Github or S3. Download from secondary urls, if happens
# at all, should never fail. In other words, if it fails, the whole build
# should fail.
declare -A secondary_urls=(
["antlr4-code"]="https://github.com/antlr/antlr4.git"
["antlr4-generator"]="http://www.antlr.org/download/antlr-4.6-complete.jar"
["cppitertools"]="https://github.com/ryanhaining/cppitertools.git"
["fmt"]="https://github.com/fmtlib/fmt.git"
["rapidcheck"]="https://github.com/emil-e/rapidcheck.git"
["gbenchmark"]="https://github.com/google/benchmark.git"
["gtest"]="https://github.com/google/googletest.git"
["gflags"]="https://github.com/memgraph/gflags.git"
["libbcrypt"]="https://github.com/rg3/libbcrypt"
["bzip2"]="https://github.com/VFR-maniac/bzip2"
["zlib"]="https://github.com/madler/zlib.git"
["rocksdb"]="https://github.com/facebook/rocksdb.git"
["mgclient"]="https://github.com/memgraph/mgclient.git"
["pymgclient"]="https://github.com/memgraph/pymgclient.git"
["spdlog"]="https://github.com/gabime/spdlog"
["jemalloc"]="https://github.com/jemalloc/jemalloc.git"
["nlohmann"]="https://raw.githubusercontent.com/nlohmann/json/b3e5cb7f20dcc5c806e418df34324eca60d17d4e/single_include/nlohmann/json.hpp"
["neo4j"]="https://s3-eu-west-1.amazonaws.com/deps.memgraph.io/neo4j-community-3.2.3-unix.tar.gz"
)
# antlr
file_get_try_double "${primary_urls[antlr4-generator]}" "${secondary_urls[antlr4-generator]}"
antlr4_tag="aacd2a2c95816d8dc1c05814051d631bfec4cf3e" # v4.6
repo_clone_try_double "${primary_urls[antlr4-code]}" "${secondary_urls[antlr4-code]}" "antlr4" "$antlr4_tag"
# fix missing include
sed -i 's/^#pragma once/#pragma once\n#include <functional>/' antlr4/runtime/Cpp/runtime/src/support/CPPUtils.h
# remove shared library from install dependencies
sed -i 's/install(TARGETS antlr4_shared/install(TARGETS antlr4_shared OPTIONAL/' antlr4/runtime/Cpp/runtime/CMakeLists.txt
# cppitertools v2.0 2019-12-23
cppitertools_ref="cb3635456bdb531121b82b4d2e3afc7ae1f56d47"
repo_clone_try_double "${primary_urls[cppitertools]}" "${secondary_urls[cppitertools]}" "cppitertools" "$cppitertools_ref"
# fmt
fmt_tag="7bdf0628b1276379886c7f6dda2cef2b3b374f0b" # (2020-11-25)
repo_clone_try_double "${primary_urls[fmt]}" "${secondary_urls[fmt]}" "fmt" "$fmt_tag"
# rapidcheck
rapidcheck_tag="7bc7d302191a4f3d0bf005692677126136e02f60" # (2020-05-04)
repo_clone_try_double "${primary_urls[rapidcheck]}" "${secondary_urls[rapidcheck]}" "rapidcheck" "$rapidcheck_tag"
# google benchmark
benchmark_tag="4f8bfeae470950ef005327973f15b0044eceaceb" # v1.1.0
repo_clone_try_double "${primary_urls[gbenchmark]}" "${secondary_urls[gbenchmark]}" "benchmark" "$benchmark_tag"
# google test
googletest_tag="ec44c6c1675c25b9827aacd08c02433cccde7780" # v1.8.0
repo_clone_try_double "${primary_urls[gtest]}" "${secondary_urls[gtest]}" "googletest" "$googletest_tag"
# google flags
gflags_tag="b37ceb03a0e56c9f15ce80409438a555f8a67b7c" # custom version (May 6, 2017)
repo_clone_try_double "${primary_urls[gflags]}" "${secondary_urls[gflags]}" "gflags" "$gflags_tag"
# libbcrypt
libbcrypt_tag="8aa32ad94ebe06b76853b0767c910c9fbf7ccef4" # custom version (Dec 16, 2016)
repo_clone_try_double "${primary_urls[libbcrypt]}" "${secondary_urls[libbcrypt]}" "libbcrypt" "$libbcrypt_tag"
# neo4j
file_get_try_double "${primary_urls[neo4j]}" "${secondary_urls[neo4j]}"
tar -xzf neo4j-community-3.2.3-unix.tar.gz
mv neo4j-community-3.2.3 neo4j
rm neo4j-community-3.2.3-unix.tar.gz
# nlohmann json
# We wget header instead of cloning repo since repo is huge (lots of test data).
# We use head on Sep 1, 2017 instead of last release since it was long time ago.
mkdir -p json
cd json
file_get_try_double "${primary_urls[nlohmann]}" "${secondary_urls[nlohmann]}"
cd ..
bzip2_tag="0405487e2b1de738e7f1c8afb50d19cf44e8d580" # v1.0.6 (May 26, 2011)
repo_clone_try_double "${primary_urls[bzip2]}" "${secondary_urls[bzip2]}" "bzip2" "$bzip2_tag"
zlib_tag="cacf7f1d4e3d44d871b605da3b647f07d718623f" # v1.2.11.
repo_clone_try_double "${primary_urls[zlib]}" "${secondary_urls[zlib]}" "zlib" "$zlib_tag"
# remove shared library from install dependencies
sed -i 's/install(TARGETS zlib zlibstatic/install(TARGETS zlibstatic/g' zlib/CMakeLists.txt
rocksdb_tag="f3e33549c151f30ac4eb7c22356c6d0331f37652" # (2020-10-14)
repo_clone_try_double "${primary_urls[rocksdb]}" "${secondary_urls[rocksdb]}" "rocksdb" "$rocksdb_tag"
# remove shared library from install dependencies
sed -i 's/TARGETS ${ROCKSDB_SHARED_LIB}/TARGETS ${ROCKSDB_SHARED_LIB} OPTIONAL/' rocksdb/CMakeLists.txt
# mgclient
mgclient_tag="v1.2.0" # (2021-01-14)
repo_clone_try_double "${primary_urls[mgclient]}" "${secondary_urls[mgclient]}" "mgclient" "$mgclient_tag"
sed -i 's/\${CMAKE_INSTALL_LIBDIR}/lib/' mgclient/src/CMakeLists.txt
# pymgclient
pymgclient_tag="4f85c179e56302d46a1e3e2cf43509db65f062b3" # (2021-01-15)
repo_clone_try_double "${primary_urls[pymgclient]}" "${secondary_urls[pymgclient]}" "pymgclient" "$pymgclient_tag"
spdlog_tag="46d418164dd4cd9822cf8ca62a116a3f71569241" # (2020-12-01)
repo_clone_try_double "${primary_urls[spdlog]}" "${secondary_urls[spdlog]}" "spdlog" "$spdlog_tag"
jemalloc_tag="ea6b3e973b477b8061e0076bb257dbd7f3faa756" # (2021-02-11)
repo_clone_try_double "${primary_urls[jemalloc]}" "${secondary_urls[jemalloc]}" "jemalloc" "$jemalloc_tag"
pushd jemalloc
# ThreadPool select job randomly, and there can be some threads that had been
# performed some memory heavy task before and will be inactive for some time,
# but until it will became active again, the memory will not be freed since by
# default each thread has it's own arena, but there should be not more then
# 4*CPU arenas (see opt.nareans description).
#
# By enabling percpu_arena number of arenas limited to number of CPUs and hence
# this problem should go away.
#
# muzzy_decay_ms -- use MADV_FREE when available on newer Linuxes, to
# avoid spurious latencies and additional work associated with
# MADV_DONTNEED. See
# https://github.com/ClickHouse/ClickHouse/issues/11121 for motivation.
./autogen.sh --with-malloc-conf="percpu_arena:percpu,oversize_threshold:0,muzzy_decay_ms:5000,dirty_decay_ms:5000"
popd

View File

@@ -1,42 +0,0 @@
# Memgraph Query Modules CMake configuration
# You should use the top level CMake configuration with -DQUERY_MODULES=ON
# These modules are meant to be shipped with Memgraph installation.
project(memgraph_query_modules)
disallow_in_source_build()
# Everything that is installed here, should be under the "query_modules" component.
set(CMAKE_INSTALL_DEFAULT_COMPONENT_NAME "query_modules")
add_library(example SHARED example.c)
target_include_directories(example PRIVATE ${CMAKE_SOURCE_DIR}/include)
target_compile_options(example PRIVATE -Wall)
# Strip the library in release build.
string(TOLOWER ${CMAKE_BUILD_TYPE} lower_build_type)
if (lower_build_type STREQUAL "release")
add_custom_command(TARGET example POST_BUILD
COMMAND strip -s $<TARGET_FILE:example>
COMMENT "Stripping symbols and sections from example module")
endif()
install(PROGRAMS $<TARGET_FILE:example>
DESTINATION lib/memgraph/query_modules
RENAME example.so)
# Also install the source of the example, so user can read it.
install(FILES example.c DESTINATION lib/memgraph/query_modules/src)
# Install the Python example
install(FILES example.py DESTINATION lib/memgraph/query_modules RENAME py_example.py)
# Install the Python modules
install(FILES graph_analyzer.py DESTINATION lib/memgraph/query_modules)
install(FILES mgp_networkx.py DESTINATION lib/memgraph/query_modules)
install(FILES nxalg.py DESTINATION lib/memgraph/query_modules)
install(FILES wcc.py DESTINATION lib/memgraph/query_modules)
if (MG_ENTERPRISE)
add_subdirectory(louvain)
add_subdirectory(connectivity)
endif()

View File

@@ -1,18 +0,0 @@
set(MODULE src/connectivity_module.cpp)
include_directories(src)
add_library(connectivity SHARED ${MODULE})
target_include_directories(connectivity PRIVATE ${CMAKE_SOURCE_DIR}/include)
# Strip the library in release build.
string(TOLOWER ${CMAKE_BUILD_TYPE} lower_build_type)
if (lower_build_type STREQUAL "release")
add_custom_command(TARGET connectivity POST_BUILD
COMMAND strip -s $<TARGET_FILE:connectivity>
COMMENT "Stripping symbols and sections from connectivity module")
endif()
install(PROGRAMS $<TARGET_FILE:connectivity>
DESTINATION lib/memgraph/query_modules
RENAME connectivity.so)

View File

@@ -1,131 +0,0 @@
#include "mg_procedure.h"
#include <queue>
#include <unordered_map>
// Finds weakly connected components of a graph.
// Time complexity: O(|V|+|E|)
static void weak(const mgp_list *args, const mgp_graph *graph,
mgp_result *result, mgp_memory *memory) {
std::unordered_map<int64_t, int64_t> vertex_component;
mgp_vertices_iterator *vertices_iterator =
mgp_graph_iter_vertices(graph, memory);
if (vertices_iterator == nullptr) {
mgp_result_set_error_msg(result, "Not enough memory");
return;
}
int64_t curr_component = 0;
for (const mgp_vertex *vertex = mgp_vertices_iterator_get(vertices_iterator);
vertex != nullptr;
vertex = mgp_vertices_iterator_next(vertices_iterator)) {
mgp_vertex_id vertex_id = mgp_vertex_get_id(vertex);
if (vertex_component.find(vertex_id.as_int) != vertex_component.end())
continue;
// run bfs from current vertex
std::queue<int64_t> q;
q.push(vertex_id.as_int);
vertex_component[vertex_id.as_int] = curr_component;
while (!q.empty()) {
mgp_vertex *v = mgp_graph_get_vertex_by_id(graph, {q.front()}, memory);
if (v == nullptr) {
mgp_vertices_iterator_destroy(vertices_iterator);
mgp_result_set_error_msg(result, "Not enough memory");
return;
}
q.pop();
// iterate over inbound edges
mgp_edges_iterator *edges_iterator = mgp_vertex_iter_in_edges(v, memory);
if (edges_iterator == nullptr) {
mgp_vertex_destroy(v);
mgp_vertices_iterator_destroy(vertices_iterator);
mgp_result_set_error_msg(result, "Not enough memory");
return;
}
for (const mgp_edge *edge = mgp_edges_iterator_get(edges_iterator);
edge != nullptr; edge = mgp_edges_iterator_next(edges_iterator)) {
mgp_vertex_id next_id = mgp_vertex_get_id(mgp_edge_get_from(edge));
if (vertex_component.find(next_id.as_int) != vertex_component.end())
continue;
vertex_component[next_id.as_int] = curr_component;
q.push(next_id.as_int);
}
// iterate over outbound edges
mgp_edges_iterator_destroy(edges_iterator);
edges_iterator = mgp_vertex_iter_out_edges(v, memory);
if (edges_iterator == nullptr) {
mgp_vertex_destroy(v);
mgp_vertices_iterator_destroy(vertices_iterator);
mgp_result_set_error_msg(result, "Not enough memory");
return;
}
for (const mgp_edge *edge = mgp_edges_iterator_get(edges_iterator);
edge != nullptr; edge = mgp_edges_iterator_next(edges_iterator)) {
mgp_vertex_id next_id = mgp_vertex_get_id(mgp_edge_get_to(edge));
if (vertex_component.find(next_id.as_int) != vertex_component.end())
continue;
vertex_component[next_id.as_int] = curr_component;
q.push(next_id.as_int);
}
mgp_vertex_destroy(v);
mgp_edges_iterator_destroy(edges_iterator);
}
++curr_component;
}
mgp_vertices_iterator_destroy(vertices_iterator);
for (const auto &p : vertex_component) {
mgp_result_record *record = mgp_result_new_record(result);
if (record == nullptr) {
mgp_result_set_error_msg(result, "Not enough memory");
return;
}
mgp_value *mem_id_value = mgp_value_make_int(p.first, memory);
if (mem_id_value == nullptr) {
mgp_result_set_error_msg(result, "Not enough memory");
return;
}
mgp_value *comp_value = mgp_value_make_int(p.second, memory);
if (comp_value == nullptr) {
mgp_value_destroy(mem_id_value);
mgp_result_set_error_msg(result, "Not enough memory");
return;
}
int mem_id_inserted = mgp_result_record_insert(record, "id", mem_id_value);
int comp_inserted =
mgp_result_record_insert(record, "component", comp_value);
mgp_value_destroy(mem_id_value);
mgp_value_destroy(comp_value);
if (!mem_id_inserted || !comp_inserted) {
mgp_result_set_error_msg(result, "Not enough memory");
return;
}
}
}
extern "C" int mgp_init_module(struct mgp_module *module,
struct mgp_memory *memory) {
struct mgp_proc *wcc_proc =
mgp_module_add_read_procedure(module, "weak", weak);
if (!mgp_proc_add_result(wcc_proc, "id", mgp_type_int())) return 1;
if (!mgp_proc_add_result(wcc_proc, "component", mgp_type_int())) return 1;
return 0;
}
extern "C" int mgp_shutdown_module() {
return 0;
}

View File

@@ -1,83 +0,0 @@
// Compile with clang or gcc:
// clang -Wall -shared -fPIC -I <path-to-memgraph-include> example.c -o example.so
// <path-to-memgraph-include> for installed Memgraph will usually be something
// like `/usr/include/memgraph` or `/usr/local/include/memgraph`.
// To use the compiled module, you need to run Memgraph configured to load
// modules from the directory where the compiled module can be found.
#include "mg_procedure.h"
// This example procedure returns 2 fields: `args` and `result`.
// * `args` is a copy of arguments passed to the procedure.
// * `result` is the result of this procedure, a "Hello World!" string.
// In case of memory errors, this function will report them and finish
// executing.
//
// The procedure can be invoked in openCypher using the following calls:
// CALL example.procedure(1, 2) YIELD args, result;
// CALL example.procedure(1) YIELD args, result;
// Naturally, you may pass in different arguments or yield less fields.
static void procedure(const struct mgp_list *args,
const struct mgp_graph *graph, struct mgp_result *result,
struct mgp_memory *memory) {
struct mgp_list *args_copy = mgp_list_make_empty(mgp_list_size(args), memory);
if (args_copy == NULL) goto error_memory;
for (size_t i = 0; i < mgp_list_size(args); ++i) {
int success = mgp_list_append(args_copy, mgp_list_at(args, i));
if (!success) goto error_free_list;
}
struct mgp_result_record *record = mgp_result_new_record(result);
if (record == NULL) goto error_free_list;
// Transfer ownership of args_copy to mgp_value.
struct mgp_value *args_value = mgp_value_make_list(args_copy);
if (args_value == NULL) goto error_free_list;
int args_inserted = mgp_result_record_insert(record, "args", args_value);
// Release `args_value` and contained `args_copy`.
mgp_value_destroy(args_value);
if (!args_inserted) goto error_memory;
struct mgp_value *hello_world_value =
mgp_value_make_string("Hello World!", memory);
if (hello_world_value == NULL) goto error_memory;
int result_inserted =
mgp_result_record_insert(record, "result", hello_world_value);
mgp_value_destroy(hello_world_value);
if (!result_inserted) goto error_memory;
// We have successfully finished, so return without error reporting.
return;
error_free_list:
mgp_list_destroy(args_copy);
error_memory:
mgp_result_set_error_msg(result, "Not enough memory!");
return;
}
// Each module needs to define mgp_init_module function.
// Here you can register multiple procedures your module supports.
int mgp_init_module(struct mgp_module *module, struct mgp_memory *memory) {
struct mgp_proc *proc =
mgp_module_add_read_procedure(module, "procedure", procedure);
if (!proc) return 1;
if (!mgp_proc_add_arg(proc, "required_arg",
mgp_type_nullable(mgp_type_any())))
return 1;
struct mgp_value *null_value = mgp_value_make_null(memory);
if (!mgp_proc_add_opt_arg(proc, "optional_arg",
mgp_type_nullable(mgp_type_any()), null_value)) {
mgp_value_destroy(null_value);
return 1;
}
mgp_value_destroy(null_value);
if (!mgp_proc_add_result(proc, "result", mgp_type_string())) return 1;
if (!mgp_proc_add_result(proc, "args",
mgp_type_list(mgp_type_nullable(mgp_type_any()))))
return 1;
return 0;
}
// This is an optional function if you need to release any resources before the
// module is unloaded. You will probably need this if you acquired some
// resources in mgp_init_module.
int mgp_shutdown_module() {
// Return 0 to indicate success.
return 0;
}

View File

@@ -1,55 +0,0 @@
# Build Memgraph with Python 3.5+ support and run Memgraph so that it loads
# this file as a Query Module. The procedure implemented in this module is just
# a rewrite of the `example.c` procedure.
import mgp
import copy
@mgp.read_proc
def procedure(context: mgp.ProcCtx,
required_arg: mgp.Nullable[mgp.Any],
optional_arg: mgp.Nullable[mgp.Any] = None
) -> mgp.Record(args=list,
vertex_count=int,
avg_degree=mgp.Number,
props=mgp.Nullable[mgp.Map]):
'''
This example procedure returns 4 fields.
* `args` is a copy of arguments passed to the procedure.
* `vertex_count` is the number of vertices in the database.
* `avg_degree` is the average degree of vertices.
* `props` is the properties map of the passed in `required_arg`, if it is
an Edge or a Vertex. In case of a Path instance, properties of the
starting vertex are returned.
Any errors can be reported by raising an Exception.
The procedure can be invoked in openCypher using the following calls:
CALL example.procedure(1, 2) YIELD args, vertex_count;
MATCH (n) CALL example.procedure(n, 1) YIELD * RETURN *;
Naturally, you may pass in different arguments or yield different fields.
'''
# Create a properties map if we received an Edge, Vertex, or Path instance.
props = None
if isinstance(required_arg, (mgp.Edge, mgp.Vertex)):
props = dict(required_arg.properties.items())
elif isinstance(required_arg, mgp.Path):
start_vertex, = required_arg.vertices
props = dict(start_vertex.properties.items())
# Count the vertices and edges in the database; this may take a while.
vertex_count = 0
edge_count = 0
for v in context.graph.vertices:
vertex_count += 1
edge_count += sum(1 for e in v.in_edges)
edge_count += sum(1 for e in v.out_edges)
# Calculate the average degree, as if edges are not directed.
avg_degree = 0 if vertex_count == 0 else edge_count / vertex_count
# Copy the received arguments to make it equivalent to the C example.
args_copy = [copy.deepcopy(required_arg), copy.deepcopy(optional_arg)]
# Multiple rows can be produced by returning an iterable of mgp.Record.
return mgp.Record(args=args_copy, vertex_count=vertex_count,
avg_degree=avg_degree, props=props)

View File

@@ -1,280 +0,0 @@
import sys
import mgp
from collections import OrderedDict
from itertools import chain, repeat
from inspect import cleandoc
from typing import List, Tuple
try:
import networkx as nx
except ImportError as import_error:
sys.stderr.write((
'\n'
'NOTE: Please install networkx to be able to use graph_analyzer '
'module. Using Python:\n'
+ sys.version +
'\n'))
raise import_error
# Imported last because it also depends on networkx.
from mgp_networkx import MemgraphMultiDiGraph # noqa E402
_MAX_LIST_SIZE = 10
@mgp.read_proc
def help() -> mgp.Record(name=str, value=str):
'''Shows manual page for graph_analyzer.'''
records = []
def make_records(name, doc):
return (mgp.Record(name=n, value=v) for n, v in
zip(chain([name], repeat('')), cleandoc(doc).splitlines()))
for func in (help, analyze, analyze_subgraph):
records.extend(make_records("Procedure '{}'".format(func.__name__),
func.__doc__))
for m, v in _get_analysis_mapping().items():
records.extend(make_records("Analysis '{}'".format(m), v.__doc__))
return records
@mgp.read_proc
def analyze(context: mgp.ProcCtx,
analyses: mgp.Nullable[List[str]] = None
) -> mgp.Record(name=str, value=str):
'''
Shows graph information.
In case of multiple results, only the first 10 will be shown.
The optional parameter is a list of graph analyses to run.
If NULL, all available analyses are run.
Example call (give all information):
CALL graph_analyzer.analyze() YIELD *;
Example call (with parameter):
CALL graph_analyzer.analyze(['nodes', 'edges']) YIELD *;
'''
g = MemgraphMultiDiGraph(ctx=context)
recs = _analyze_graph(context, g, analyses)
return [mgp.Record(name=name, value=value) for name, value in recs]
@mgp.read_proc
def analyze_subgraph(context: mgp.ProcCtx,
vertices: mgp.List[mgp.Vertex],
edges: mgp.List[mgp.Edge],
analyses: mgp.Nullable[List[str]] = None
) -> mgp.Record(name=str, value=str):
'''
Shows subgraph information.
In case of multiple results, only the first 10 will be shown.
The optional parameter is a list of graph analyses to run.
If NULL, all available analyses are run.
Example call (give all information):
MATCH (n)-[e]->(m) WITH
collect(n) AS nodes,
collect(e) AS edges
CALL graph_analyzer.analyze_subgraph(nodes, edges) YIELD *
RETURN name, value;
Example call (with parameter):
MATCH (n)-[e]->(m) WITH
collect(n) AS nodes,
collect(e) AS edges
CALL graph_analyzer.analyze_subgraph(nodes, edges, ['nodes', 'edges'])
YIELD *
RETURN name, value;
'''
vertices, edges = map(set, [vertices, edges])
g = nx.subgraph_view(
MemgraphMultiDiGraph(ctx=context),
lambda n: n in vertices,
lambda n1, n2, e: e in edges)
recs = _analyze_graph(context, g, analyses)
return [mgp.Record(name=name, value=value) for name, value in recs]
def _get_analysis_mapping():
return OrderedDict([
('nodes', _number_of_nodes),
('edges', _number_of_edges),
('bridges', _bridges),
('articulation_points', _articulation_points),
('avg_degree', _avg_degree),
('sorted_nodes_degree', _sorted_nodes_degree),
('self_loops', _self_loops),
('is_bipartite', _is_bipartite),
('is_planar', _is_planar),
('is_biconnected: ', _is_biconnected),
('is_weakly_connected', _is_weakly_connected),
('number_of_weakly_components', _weakly_components),
('is_strongly_connected', _is_strongly_connected),
('strongly_components', _strongly_components),
('is_dag', _is_dag),
('is_eulerian', _is_eulerian),
('is_forest', _is_forest),
('is_tree', _is_tree)])
def _get_analysis_func(name: str):
_name_to_proc = _get_analysis_mapping()
return _name_to_proc.get(name.lower())
def _get_analysis_funcs():
return _get_analysis_mapping().values()
def _analyze_graph(context: mgp.ProcCtx,
g: nx.MultiDiGraph,
analyses: List[str]
) -> List[Tuple[str, str]]:
functions = (_get_analysis_funcs() if analyses is None
else [_get_analysis_func(name) for name in analyses])
records = []
for index, f in enumerate(functions):
context.check_must_abort()
if f is None:
raise KeyError('Graph analysis is not supported: ' +
analyses[index])
name, value = f(g)
if isinstance(value, (list, set, tuple)):
value = list(value)[:_MAX_LIST_SIZE]
records.append((name, str(value)))
return records
def _number_of_nodes(g: nx.MultiDiGraph) -> Tuple[str, int]:
'''Returns number of nodes.'''
return 'Number of nodes', nx.number_of_nodes(g)
def _number_of_edges(g: nx.MultiDiGraph) -> Tuple[str, int]:
'''Returns number of edges.'''
return 'Number of edges', nx.number_of_edges(g)
def _avg_degree(g: nx.MultiDiGraph) -> Tuple[str, float]:
'''Returns average degree.'''
_, number_of_nodes = _number_of_nodes(g)
_, number_of_edges = _number_of_edges(g)
avg_degree = (0 if number_of_nodes == 0
else number_of_edges / number_of_nodes)
return 'Average degree', avg_degree
def _sorted_nodes_degree(g: nx.MultiDiGraph) -> Tuple[str, List[int]]:
'''Returns list of sorted nodes degree. [(node_id, degree), ...]'''
nodes_degree = [(n, g.degree(n)) for n in g.nodes()]
nodes_degree.sort(key=lambda x: x[1], reverse=True)
return 'Sorted nodes degree', nodes_degree
def _self_loops(g: nx.MultiDiGraph) -> Tuple[str, int]:
'''Returns number of self loops.'''
return 'Self loops', sum((1 if e[0] == e[1] else 0 for e in g.edges()))
def _is_bipartite(g: nx.MultiDiGraph) -> Tuple[str, bool]:
'''Checks if graph is bipartite.'''
_, number_of_nodes = _number_of_nodes(g)
ret = (False if number_of_nodes == 0
else nx.algorithms.bipartite.basic.is_bipartite(g))
return 'Is bipartite', ret
def _is_planar(g: nx.MultiDiGraph) -> Tuple[str, bool]:
'''Checks if graph is planar.'''
_, number_of_nodes = _number_of_nodes(g)
ret = (False if number_of_nodes == 0
else nx.algorithms.planarity.check_planarity(g)[0])
return 'Is planar', ret
def _is_biconnected(g: nx.MultiDiGraph) -> Tuple[str, bool]:
'''Check if graph is biconnected.'''
_, number_of_nodes = _number_of_nodes(g)
ret = (False if number_of_nodes == 0
else nx.is_biconnected(nx.MultiDiGraph.to_undirected(g)))
return 'Is biconnected', ret
def _is_weakly_connected(g: nx.MultiDiGraph) -> Tuple[str, bool]:
'''Check if graph is weakly connected.'''
_, number_of_nodes = _number_of_nodes(g)
ret = False if number_of_nodes == 0 else nx.is_weakly_connected(g)
return 'Is weakly connected', ret
def _is_strongly_connected(g: nx.MultiDiGraph) -> Tuple[str, bool]:
'''Checks if graph is strongly connected.'''
_, number_of_nodes = _number_of_nodes(g)
ret = False if number_of_nodes == 0 else nx.is_strongly_connected(g)
return 'Is strongly connected', ret
def _is_dag(g: nx.MultiDiGraph) -> Tuple[str, bool]:
'''Check if graph is directed acyclic graph (DAG)'''
_, number_of_nodes = _number_of_nodes(g)
ret = (False if number_of_nodes == 0
else nx.algorithms.dag.is_directed_acyclic_graph(g))
return 'Is DAG', ret
def _is_eulerian(g: nx.MultiDiGraph) -> Tuple[str, bool]:
'''Checks if graph is Eulerian.'''
_, number_of_nodes = _number_of_nodes(g)
ret = (False if number_of_nodes == 0
else nx.algorithms.euler.is_eulerian(g))
return 'Is eulerian', ret
def _is_forest(g: nx.MultiDiGraph) -> Tuple[str, bool]:
'''Checks if graph is forest, all components must be trees.'''
_, number_of_nodes = _number_of_nodes(g)
ret = (False if number_of_nodes == 0
else nx.algorithms.tree.recognition.is_forest(g))
return 'Is forest', ret
def _is_tree(g: nx.MultiDiGraph) -> Tuple[str, bool]:
'''Checks if graph is tree.'''
_, number_of_nodes = _number_of_nodes(g)
ret = (False if number_of_nodes == 0
else nx.algorithms.tree.recognition.is_tree(g))
return 'Is tree', ret
def _bridges(g: nx.MultiDiGraph) -> Tuple[str, int]:
'''Returns number of bridges, multiple edges between same nodes are
mapped to one edge.'''
return 'Number of bridges', sum(1 for _ in nx.bridges(nx.Graph(g)))
def _articulation_points(g: nx.MultiDiGraph):
'''Returns number of articulation points.'''
undirected = nx.MultiDiGraph.to_undirected(g)
return ('Number of articulation points',
sum(1 for _ in nx.articulation_points(undirected)))
def _weakly_components(g: nx.MultiDiGraph):
'''Returns number of weakly components.'''
comps = nx.algorithms.components.number_weakly_connected_components(g)
return 'Number of weakly connected components', comps
def _strongly_components(g: nx.MultiDiGraph):
'''Returns number of strongly connected components.'''
comps = nx.algorithms.components.number_strongly_connected_components(g)
return 'Number of strongly connected components', comps

View File

@@ -1,33 +0,0 @@
set(MAIN src/main.cpp)
set(MODULE src/louvain_module.cpp)
set(SOURCES src/algorithms/louvain.cpp
src/data_structures/graph.cpp)
include_directories(src)
add_library(louvain-core STATIC ${SOURCES})
set_target_properties(louvain-core PROPERTIES POSITION_INDEPENDENT_CODE ON)
add_executable(louvain-main ${MAIN})
target_link_libraries(louvain-main louvain-core)
enable_testing()
add_subdirectory(test)
add_library(louvain SHARED ${MODULE})
target_link_libraries(louvain louvain-core)
target_include_directories(louvain PRIVATE ${CMAKE_SOURCE_DIR}/include)
# Strip the library in release build.
string(TOLOWER ${CMAKE_BUILD_TYPE} lower_build_type)
if (lower_build_type STREQUAL "release")
add_custom_command(TARGET louvain POST_BUILD
COMMAND strip -s $<TARGET_FILE:louvain>
COMMENT "Stripping symbols and sections from louvain module")
endif()
if (NOT MG_COMMUNITY)
install(PROGRAMS $<TARGET_FILE:louvain>
DESTINATION lib/memgraph/query_modules
RENAME louvain.so)
endif()

View File

@@ -1,18 +0,0 @@
/// @file
///
/// The file contains function declarations of several community-detection
/// graph algorithms.
#pragma once
#include "data_structures/graph.hpp"
namespace algorithms {
/// Detects communities of an unidrected, weighted graph using the Louvain
/// algorithm. The algorithm attempts to maximze the modularity of a weighted
/// graph.
///
/// @param graph pointer to an undirected, weighted graph which may contain
/// self-loops.
void Louvain(comdata::Graph *graph);
} // namespace algorithms

View File

@@ -1,163 +0,0 @@
#include "algorithms/algorithms.hpp"
#include <algorithm>
#include <map>
#include <random>
#include <unordered_map>
namespace {
void OptimizeLocally(comdata::Graph *graph) {
// We will consider local optimizations uniformly at random.
std::random_device rd;
std::mt19937 g(rd());
std::vector<uint32_t> p(graph->Size());
std::iota(p.begin(), p.end(), 0);
std::shuffle(p.begin(), p.end(), g);
// Modularity of a graph can be expressed as:
//
// Q = 1 / (2m) * sum_over_pairs_of_nodes[(Aij - ki * kj / 2m) * delta(ci, cj)]
//
// where m is the sum of all weights in the graph,
// Aij is the weight on edge that connects i and j (i=j for a self-loop)
// ki is the sum of weights incident to node i
// ci is the community of node i
// delta(a, b) is the Kronecker delta function.
//
// With some simple algebraic manipulations, we can transform the formula into:
//
// Q = sum_over_components[M * ((sum_over_pairs(Aij + M * ki * kj)))] =
// = sum_over_components[M * (sum_over_pairs(Aij) + M * sum_over_nodes^2(ki))] =
// = sum_over_components[M * (w_contrib(ci) + M * k_contrib^2(ci))]
//
// where M = 1 / (2m)
//
// Therefore, we could store for each community the following:
// * Weight contribution (w_contrib)
// * Weighted degree contribution (k_contrib)
//
// This allows us to efficiently remove a node from one community and insert
// it into a community of its neighbour without the need to recalculate
// modularity from scratch.
std::unordered_map<uint32_t, double> w_contrib;
std::unordered_map<uint32_t, double> k_contrib;
for (uint32_t node_id = 0; node_id < graph->Size(); ++node_id) {
k_contrib[graph->Community(node_id)] += graph->IncidentWeight(node_id);
for (const auto &neigh : graph->Neighbours(node_id)) {
uint32_t nxt_id = neigh.dest;
double w = neigh.weight;
if (graph->Community(node_id) == graph->Community(nxt_id))
w_contrib[graph->Community(node_id)] += w;
}
}
bool stable = false;
double total_w = graph->TotalWeight();
while (!stable) {
stable = true;
for (uint32_t node_id : p) {
std::unordered_map<uint32_t, double> sum_w;
double self_loop = 0;
sum_w[graph->Community(node_id)] = 0;
for (const auto &neigh : graph->Neighbours(node_id)) {
uint32_t nxt_id = neigh.dest;
double weight = neigh.weight;
if (nxt_id == node_id) {
self_loop += weight;
continue;
}
sum_w[graph->Community(nxt_id)] += weight;
}
uint32_t my_c = graph->Community(node_id);
uint32_t best_c = my_c;
double best_dq = 0;
for (const auto &p : sum_w) {
if (p.first == my_c) continue;
uint32_t nxt_c = p.first;
double dq = 0;
// contributions before swap (dq = d_after - d_before)
for (uint32_t c : {my_c, nxt_c})
dq -= w_contrib[c] - k_contrib[c] * k_contrib[c] / (2.0 * total_w);
// leave the current community
dq += (w_contrib[my_c] - 2.0 * sum_w[my_c] - self_loop) -
(k_contrib[my_c] - graph->IncidentWeight(node_id)) *
(k_contrib[my_c] - graph->IncidentWeight(node_id)) /
(2.0 * total_w);
// join a new community
dq += (w_contrib[nxt_c] + 2.0 * sum_w[nxt_c] + self_loop) -
(k_contrib[nxt_c] + graph->IncidentWeight(node_id)) *
(k_contrib[nxt_c] + graph->IncidentWeight(node_id)) /
(2.0 * total_w);
if (dq > best_dq) {
best_dq = dq;
best_c = nxt_c;
}
}
if (best_c != my_c) {
graph->SetCommunity(node_id, best_c);
w_contrib[my_c] -= 2.0 * sum_w[my_c] + self_loop;
k_contrib[my_c] -= graph->IncidentWeight(node_id);
w_contrib[best_c] += 2.0 * sum_w[best_c] + self_loop;
k_contrib[best_c] += graph->IncidentWeight(node_id);
stable = false;
}
}
}
}
} // anonymous namespace
namespace algorithms {
void Louvain(comdata::Graph *graph) {
OptimizeLocally(graph);
// Collapse the locally optimized graph.
uint32_t collapsed_nodes = graph->NormalizeCommunities();
if (collapsed_nodes == graph->Size()) return;
comdata::Graph collapsed_graph(collapsed_nodes);
std::map<std::pair<uint32_t, uint32_t>, double> collapsed_edges;
for (uint32_t node_id = 0; node_id < graph->Size(); ++node_id) {
std::unordered_map<uint32_t, double> edges;
for (const auto &neigh : graph->Neighbours(node_id)) {
uint32_t nxt_id = neigh.dest;
double weight = neigh.weight;
if (graph->Community(nxt_id) < graph->Community(node_id)) continue;
edges[graph->Community(nxt_id)] += weight;
}
for (const auto &neigh : edges) {
uint32_t a = std::min(graph->Community(node_id), neigh.first);
uint32_t b = std::max(graph->Community(node_id), neigh.first);
collapsed_edges[{a, b}] += neigh.second;
}
}
for (const auto &p : collapsed_edges)
collapsed_graph.AddEdge(p.first.first, p.first.second, p.second);
// Repeat until no local optimizations can be found.
Louvain(&collapsed_graph);
// Propagate results from collapsed graph.
for (uint32_t node_id = 0; node_id < graph->Size(); ++node_id) {
graph->SetCommunity(node_id,
collapsed_graph.Community(graph->Community(node_id)));
}
graph->NormalizeCommunities();
}
} // namespace algorithms

View File

@@ -1,99 +0,0 @@
#include "data_structures/graph.hpp"
#include <exception>
#include <numeric>
#include <stdexcept>
#include <unordered_map>
#include <unordered_set>
#include <vector>
namespace comdata {
Graph::Graph(uint32_t n_nodes) : n_nodes_(n_nodes), total_w_(0) {
adj_list_.resize(n_nodes, {});
inc_w_.resize(n_nodes, 0);
// each node starts as its own separate community.
community_.resize(n_nodes);
std::iota(community_.begin(), community_.end(), 0);
}
uint32_t Graph::Size() const { return n_nodes_; }
uint32_t Graph::Community(uint32_t node) const { return community_.at(node); }
void Graph::SetCommunity(uint32_t node, uint32_t c) { community_.at(node) = c; }
uint32_t Graph::NormalizeCommunities() {
std::set<uint32_t> c_id(community_.begin(), community_.end());
std::unordered_map<uint32_t, uint32_t> cmap;
uint32_t id = 0;
for (uint32_t c : c_id) {
cmap[c] = id;
++id;
}
for (uint32_t node_id = 0; node_id < n_nodes_; ++node_id)
community_[node_id] = cmap[community_[node_id]];
return id;
}
void Graph::AddEdge(uint32_t node1, uint32_t node2, double weight) {
if (node1 >= n_nodes_ || node2 >= n_nodes_)
throw std::out_of_range("Node index out of range");
if (weight <= 0) throw std::out_of_range("Weights must be positive");
if (edges_.find({node1, node2}) != edges_.end())
throw std::invalid_argument("Edge already exists");
edges_.emplace(node1, node2);
edges_.emplace(node2, node1);
total_w_ += weight;
adj_list_[node1].emplace_back(node2, weight);
inc_w_[node1] += weight;
if (node1 != node2) {
adj_list_[node2].emplace_back(node1, weight);
inc_w_[node2] += weight;
}
}
uint32_t Graph::Degree(uint32_t node) const {
return static_cast<uint32_t>(adj_list_.at(node).size());
}
double Graph::IncidentWeight(uint32_t node) const { return inc_w_.at(node); }
double Graph::TotalWeight() const { return total_w_; }
double Graph::Modularity() const {
double ret = 0;
// Since all weights should be positive, this implies that our graph has
// no edges.
if (total_w_ == 0) return 0;
std::unordered_map<uint32_t, double> weight_c;
std::unordered_map<uint32_t, double> degree_c;
for (uint32_t i = 0; i < n_nodes_; ++i) {
degree_c[Community(i)] += IncidentWeight(i);
for (const auto &neigh : adj_list_[i]) {
uint32_t j = neigh.dest;
double w = neigh.weight;
if (Community(i) != Community(j)) continue;
weight_c[Community(i)] += w;
}
}
for (const auto &p : degree_c)
ret += weight_c[p.first] - (p.second * p.second) / (2 * total_w_);
ret /= 2 * total_w_;
return ret;
}
const std::vector<Neighbour> &Graph::Neighbours(uint32_t node) const {
return adj_list_.at(node);
}
} // namespace comdata

View File

@@ -1,125 +0,0 @@
/// @file
#pragma once
#include <cstdint>
#include <set>
#include <vector>
namespace comdata {
struct Neighbour {
uint32_t dest;
double weight;
Neighbour(uint32_t d, double w) : dest(d), weight(w) {}
};
/// Class which models a weighted, undirected graph with necessary
/// functionalities for community detection algorithms.
class Graph {
public:
/// Constructs a new graph with a given number of nodes and no edges between
/// them.
///
/// The implementation assumes (and enforces) that all nodes
/// are indexed from 0 to n_nodes.
///
/// @param n_nodes Number of nodes in the graph.
explicit Graph(uint32_t n_nodes);
/// @return number of nodes in the graph.
uint32_t Size() const;
/// Adds a bidirectional, weighted edge to the graph between the given
/// nodes. If both given nodes are the same, the method inserts a weighted
/// self-loop.
///
/// There should be no edges between the given nodes when before invoking
/// this method.
///
/// @param node1 index of an incident node.
/// @param node2 index of an incident node.
/// @param weight real value which represents the weight of the edge.
///
/// @throw std::out_of_range
/// @throw std::invalid_argument
void AddEdge(uint32_t node1, uint32_t node2, double weight);
/// @param node index of node.
///
/// @return community where the node belongs to.
///
/// @throw std::out_of_range
uint32_t Community(uint32_t node) const;
/// Adds a given node to a given community.
///
/// @param node index of node.
/// @param c community where the given node should go in.
///
/// @throw std::out_of_range
void SetCommunity(uint32_t node, uint32_t c);
/// Normalizes the values of communities. More precisely, after invoking this
/// method communities will be indexed by successive integers starting from 0.
///
/// Note: this method is computationally expensive and takes O(|V|)
/// time, i.e., it traverses all nodes in the graph.
///
/// @return number of communities in the graph
uint32_t NormalizeCommunities();
/// Returns the number of incident edges to a given node. Self-loops
/// contribute a single edge to the degree.
///
/// @param node index of node.
///
/// @return degree of given node.
///
/// @throw std::out_of_range
uint32_t Degree(uint32_t node) const;
/// Returns the total weight of incident edges to a given node. Weight
/// of a self loop contributes once to the total sum.
///
/// @param node index of node.
///
/// @return total incident weight of a given node.
///
/// @throw std::out_of_range
double IncidentWeight(uint32_t node) const;
/// @return total weight of all edges in a graph.
double TotalWeight() const;
/// Calculates the modularity of the graph which is defined as a real value
/// between -1 and 1 that measures the density of links inside communities
/// compared to links between communities.
///
/// Note: this method is computationally expensive and takes O(|V| + |E|)
/// time, i.e., it traverses the entire graph.
///
/// @return modularity of the graph.
double Modularity() const;
/// Returns nodes adjacent to a given node.
///
/// @param node index of node.
///
/// @return list of neighbouring nodes.
///
/// @throw std::out_of_range
const std::vector<Neighbour>& Neighbours(uint32_t node) const;
private:
uint32_t n_nodes_;
double total_w_;
std::vector<std::vector<Neighbour>> adj_list_;
std::set<std::pair<uint32_t, uint32_t>> edges_;
std::vector<double> inc_w_;
std::vector<uint32_t> community_;
};
} // namespace comdata

View File

@@ -1,228 +0,0 @@
#include "mg_procedure.h"
#include <exception>
#include <string>
#include <unordered_map>
#include "algorithms/algorithms.hpp"
#include "data_structures/graph.hpp"
namespace {
std::optional<std::unordered_map<int64_t, uint32_t>> NormalizeVertexIds(
const mgp_graph *graph, mgp_result *result, mgp_memory *memory) {
std::unordered_map<int64_t, uint32_t> mem_to_louv_id;
mgp_vertices_iterator *vertices_iterator =
mgp_graph_iter_vertices(graph, memory);
if (vertices_iterator == nullptr) {
mgp_result_set_error_msg(result, "Not enough memory!");
return std::nullopt;
}
uint32_t louv_id = 0;
for (const mgp_vertex *vertex = mgp_vertices_iterator_get(vertices_iterator);
vertex != nullptr;
vertex = mgp_vertices_iterator_next(vertices_iterator)) {
mgp_vertex_id mem_id = mgp_vertex_get_id(vertex);
mem_to_louv_id[mem_id.as_int] = louv_id;
++louv_id;
}
mgp_vertices_iterator_destroy(vertices_iterator);
return mem_to_louv_id;
}
std::optional<comdata::Graph> RunLouvain(
const mgp_graph *graph, mgp_result *result, mgp_memory *memory,
const std::unordered_map<int64_t, uint32_t> &mem_to_louv_id) {
comdata::Graph louvain_graph(mem_to_louv_id.size());
// Extract the graph structure
// TODO(ipaljak): consider filtering nodes and edges by labels.
for (const auto &p : mem_to_louv_id) {
mgp_vertex *vertex = mgp_graph_get_vertex_by_id(graph, {p.first}, memory);
if (!vertex) {
mgp_result_set_error_msg(result, "Not enough memory!");
return std::nullopt;
}
// iterate over inbound edges. This is enough because we will eventually
// iterate over outbound edges in another direction.
mgp_edges_iterator *edges_iterator =
mgp_vertex_iter_in_edges(vertex, memory);
if (edges_iterator == nullptr) {
mgp_vertex_destroy(vertex);
mgp_result_set_error_msg(result, "Not enough memory!");
return std::nullopt;
}
for (const mgp_edge *edge = mgp_edges_iterator_get(edges_iterator);
edge != nullptr; edge = mgp_edges_iterator_next(edges_iterator)) {
const mgp_vertex *next_vertex = mgp_edge_get_from(edge);
mgp_vertex_id next_mem_id = mgp_vertex_get_id(next_vertex);
uint32_t next_louv_id;
try {
next_louv_id = mem_to_louv_id.at(next_mem_id.as_int);
} catch (const std::exception &e) {
const auto msg = std::string("[Internal error] ") + e.what();
mgp_result_set_error_msg(result, msg.c_str());
return std::nullopt;
}
// retrieve edge weight (default to 1)
mgp_value *weight_prop = mgp_edge_get_property(edge, "weight", memory);
if (!weight_prop) {
mgp_vertex_destroy(vertex);
mgp_edges_iterator_destroy(edges_iterator);
mgp_result_set_error_msg(result, "Not enough memory");
}
double weight = 1;
if (mgp_value_is_double(weight_prop))
weight = mgp_value_get_double(weight_prop);
if (mgp_value_is_int(weight_prop))
weight = static_cast<double>(mgp_value_get_int(weight_prop));
mgp_value_destroy(weight_prop);
try {
louvain_graph.AddEdge(p.second, next_louv_id, weight);
} catch (const std::exception &e) {
mgp_vertex_destroy(vertex);
mgp_edges_iterator_destroy(edges_iterator);
mgp_result_set_error_msg(result, e.what());
return std::nullopt;
}
}
mgp_vertex_destroy(vertex);
mgp_edges_iterator_destroy(edges_iterator);
}
try {
algorithms::Louvain(&louvain_graph);
} catch (const std::exception &e) {
const auto msg = std::string("[Internal error] ") + e.what();
mgp_result_set_error_msg(result, msg.c_str());
return std::nullopt;
}
return louvain_graph;
}
void communities(const mgp_list *args, const mgp_graph *graph,
mgp_result *result, mgp_memory *memory) {
try {
// Normalize vertex ids
auto mem_to_louv_id = NormalizeVertexIds(graph, result, memory);
if (!mem_to_louv_id) return;
// Run louvain
auto louvain_graph = RunLouvain(graph, result, memory, *mem_to_louv_id);
if (!louvain_graph) return;
// Return node ids and their corresponding communities.
for (const auto &p : *mem_to_louv_id) {
mgp_result_record *record = mgp_result_new_record(result);
if (record == nullptr) {
mgp_result_set_error_msg(result, "Not enough memory!");
return;
}
mgp_value *mem_id_value = mgp_value_make_int(p.first, memory);
if (mem_id_value == nullptr) {
mgp_result_set_error_msg(result, "Not enough memory!");
return;
}
mgp_value *com_value =
mgp_value_make_int(louvain_graph->Community(p.second), memory);
if (com_value == nullptr) {
mgp_value_destroy(mem_id_value);
mgp_result_set_error_msg(result, "Not enough memory!");
return;
}
int mem_id_inserted =
mgp_result_record_insert(record, "id", mem_id_value);
int com_inserted =
mgp_result_record_insert(record, "community", com_value);
mgp_value_destroy(mem_id_value);
mgp_value_destroy(com_value);
if (!mem_id_inserted || !com_inserted) {
mgp_result_set_error_msg(result, "Not enough memory!");
return;
}
}
} catch (const std::exception &e) {
mgp_result_set_error_msg(result, e.what());
return;
}
}
void modularity(const mgp_list *args, const mgp_graph *graph,
mgp_result *result, mgp_memory *memory) {
try {
// Normalize vertex ids
auto mem_to_louv_id = NormalizeVertexIds(graph, result, memory);
if (!mem_to_louv_id) return;
// Run louvain
auto louvain_graph = RunLouvain(graph, result, memory, *mem_to_louv_id);
if (!louvain_graph) return;
// Return graph modularity after Louvain
// TODO(ipaljak) - consider allowing the user to specify seed communities
// and
// yield modularity values both before and after running
// louvain.
mgp_result_record *record = mgp_result_new_record(result);
if (record == nullptr) {
mgp_result_set_error_msg(result, "Not enough memory!");
return;
}
mgp_value *modularity_value =
mgp_value_make_double(louvain_graph->Modularity(), memory);
if (modularity_value == nullptr) {
mgp_result_set_error_msg(result, "Not enough memory!");
return;
}
int value_inserted =
mgp_result_record_insert(record, "modularity", modularity_value);
mgp_value_destroy(modularity_value);
if (!value_inserted) {
mgp_result_set_error_msg(result, "Not enough memory!");
return;
}
} catch (const std::exception &e) {
mgp_result_set_error_msg(result, e.what());
return;
}
}
} // namespace
extern "C" int mgp_init_module(struct mgp_module *module,
struct mgp_memory *memory) {
struct mgp_proc *community_proc =
mgp_module_add_read_procedure(module, "communities", communities);
if (!community_proc) return 1;
if (!mgp_proc_add_result(community_proc, "id", mgp_type_int())) return 1;
if (!mgp_proc_add_result(community_proc, "community", mgp_type_int()))
return 1;
struct mgp_proc *modularity_proc =
mgp_module_add_read_procedure(module, "modularity", modularity);
if (!modularity_proc) return 1;
if (!mgp_proc_add_result(modularity_proc, "modularity", mgp_type_float()))
return 1;
return 0;
}
extern "C" int mgp_shutdown_module() { return 0; }

View File

@@ -1,28 +0,0 @@
#include <iostream>
#include "algorithms/algorithms.hpp"
#include "data_structures/graph.hpp"
// A simple program that reads the graph from STDIN and
// outputs the detected communities from louvain along with
// its modularity measure on STDOUT.
int main() {
int n;
int m;
std::cin >> n >> m;
comdata::Graph graph(n);
for (int i = 0; i < m; ++i) {
int a;
int b;
double c;
std::cin >> a >> b >> c;
graph.AddEdge(a, b, c);
}
algorithms::Louvain(&graph);
for (int i = 0; i < n; ++i)
std::cout << i << " " << graph.Community(i) << "\n";
std::cout << graph.Modularity() << "\n";
return 0;
}

View File

@@ -1,80 +0,0 @@
---
Checks: '*,
-android-*,
-cert-err58-cpp,
-cppcoreguidelines-avoid-c-arrays,
-cppcoreguidelines-avoid-goto,
-cppcoreguidelines-avoid-magic-numbers,
-cppcoreguidelines-macro-usage,
-cppcoreguidelines-no-malloc,
-cppcoreguidelines-non-private-member-variables-in-classes,
-cppcoreguidelines-owning-memory,
-cppcoreguidelines-pro-bounds-array-to-pointer-decay,
-cppcoreguidelines-pro-bounds-constant-array-index,
-cppcoreguidelines-pro-bounds-pointer-arithmetic,
-cppcoreguidelines-pro-type-member-init,
-cppcoreguidelines-pro-type-reinterpret-cast,
-cppcoreguidelines-pro-type-static-cast-downcast,
-cppcoreguidelines-pro-type-union-access,
-cppcoreguidelines-pro-type-vararg,
-cppcoreguidelines-special-member-functions,
-fuchsia-default-arguments,
-fuchsia-default-arguments-calls,
-fuchsia-default-arguments-declarations,
-fuchsia-overloaded-operator,
-fuchsia-statically-constructed-objects,
-fuchsia-trailing-return,
-fuchsia-virtual-inheritance,
-google-explicit-constructor,
-google-readability-*,
-hicpp-avoid-c-arrays,
-hicpp-avoid-goto,
-hicpp-braces-around-statements,
-hicpp-member-init,
-hicpp-no-array-decay,
-hicpp-no-assembler,
-hicpp-no-malloc,
-hicpp-special-member-functions,
-hicpp-use-equals-default,
-hicpp-vararg,
-llvm-header-guard,
-misc-non-private-member-variables-in-classes,
-misc-unused-parameters,
-modernize-avoid-c-arrays,
-modernize-concat-nested-namespaces,
-modernize-pass-by-value,
-modernize-use-equals-default,
-modernize-use-nodiscard,
-modernize-use-trailing-return-type,
-performance-unnecessary-value-param,
-readability-braces-around-statements,
-readability-else-after-return,
-readability-implicit-bool-conversion,
-readability-magic-numbers,
-readability-named-parameter'
WarningsAsErrors: ''
HeaderFilterRegex: ''
AnalyzeTemporaryDtors: false
FormatStyle: none
CheckOptions:
- key: google-readability-braces-around-statements.ShortStatementLines
value: '1'
- key: google-readability-function-size.StatementThreshold
value: '800'
- key: google-readability-namespace-comments.ShortNamespaceLines
value: '10'
- key: google-readability-namespace-comments.SpacesBeforeComments
value: '2'
- key: modernize-loop-convert.MaxCopySize
value: '16'
- key: modernize-loop-convert.MinConfidence
value: reasonable
- key: modernize-loop-convert.NamingStyle
value: CamelCase
- key: modernize-pass-by-value.IncludeStyle
value: llvm
- key: modernize-replace-auto-ptr.IncludeStyle
value: llvm
- key: modernize-use-nullptr.NullMacros
value: 'NULL'
...

View File

@@ -1,3 +0,0 @@
include_directories(${GTEST_INCLUDE_DIR})
add_subdirectory(unit)

View File

@@ -1,28 +0,0 @@
set(test_prefix louvain__unit__)
add_custom_target(louvain__unit)
add_library(louvain-test STATIC utils.cpp)
set_target_properties(louvain-test PROPERTIES POSITION_INDEPENDENT_CODE ON)
function(add_unit_test test_cpp)
# get exec name (remove extension from the abs path)
get_filename_component(exec_name ${test_cpp} NAME_WE)
set(target_name ${test_prefix}${exec_name})
add_executable(${target_name} ${test_cpp})
# OUTPUT_NAME sets the real name of a target when it is built and can be
# used to help create two targets of the same name even though CMake
# requires unique logical target names
set_target_properties(${target_name} PROPERTIES OUTPUT_NAME ${exec_name})
# TODO: this is a temporary workaround the test build warnings
target_compile_options(${target_name} PRIVATE -Wno-comment -Wno-sign-compare
-Wno-unused-variable)
target_link_libraries(${target_name} spdlog gflags gtest gtest_main Threads::Threads
louvain-core louvain-test)
# register test
add_test(${target_name} ${exec_name})
# add to unit target
add_dependencies(louvain__unit ${target_name})
endfunction(add_unit_test)
add_unit_test(graph.cpp)

View File

@@ -1,349 +0,0 @@
#include <gtest/gtest.h>
#include "data_structures/graph.hpp"
#include "utils.hpp"
// Checks if commmunities of nodes in graph correspond to a given community
// vector.
bool CommunityCheck(const comdata::Graph &graph,
const std::vector<uint32_t> &c) {
if (graph.Size() != c.size()) return false;
for (uint32_t node_id = 0; node_id < graph.Size(); ++node_id)
if (graph.Community(node_id) != c[node_id]) return false;
return true;
}
// Checks if degrees of nodes in graph correspond to a given degree vector.
bool DegreeCheck(const comdata::Graph &graph,
const std::vector<uint32_t> &deg) {
if (graph.Size() != deg.size()) return false;
for (uint32_t node_id = 0; node_id < graph.Size(); ++node_id)
if (graph.Degree(node_id) != deg[node_id]) return false;
return true;
}
// Checks if incident weights of nodes in graph correspond to a given weight
// vector.
bool IncidentWeightCheck(const comdata::Graph &graph,
const std::vector<double> &inc_w) {
if (graph.Size() != inc_w.size()) return false;
for (uint32_t node_id = 0; node_id < graph.Size(); ++node_id)
if (std::abs(graph.IncidentWeight(node_id) - inc_w[node_id]) > 1e-6)
return false;
return true;
}
// Sets communities of nodes in graph. Returns true on success.
bool SetCommunities(comdata::Graph *graph, const std::vector<uint32_t> &c) {
if (graph->Size() != c.size()) return false;
for (uint32_t node_id = 0; node_id < graph->Size(); ++node_id)
graph->SetCommunity(node_id, c[node_id]);
return true;
}
TEST(Graph, Constructor) {
uint32_t nodes = 100;
comdata::Graph graph(nodes);
ASSERT_EQ(graph.Size(), nodes);
for (uint32_t node_id = 0; node_id < nodes; ++node_id) {
ASSERT_EQ(graph.IncidentWeight(node_id), 0);
ASSERT_EQ(graph.Community(node_id), node_id);
}
}
TEST(Graph, Size) {
comdata::Graph graph1 = GenRandomUnweightedGraph(0, 0);
comdata::Graph graph2 = GenRandomUnweightedGraph(42, 41);
comdata::Graph graph3 = GenRandomUnweightedGraph(100, 250);
ASSERT_EQ(graph1.Size(), 0);
ASSERT_EQ(graph2.Size(), 42);
ASSERT_EQ(graph3.Size(), 100);
}
TEST(Graph, Communities) {
comdata::Graph graph = GenRandomUnweightedGraph(100, 250);
for (int i = 0; i < 100; ++i) graph.SetCommunity(i, i % 5);
for (int i = 0; i < 100; ++i) ASSERT_EQ(graph.Community(i), i % 5);
// Try to set communities on non-existing nodes
EXPECT_THROW({ graph.SetCommunity(100, 2); }, std::out_of_range);
EXPECT_THROW({ graph.SetCommunity(150, 0); }, std::out_of_range);
// Try to get a the community of a non-existing node
EXPECT_THROW({ graph.Community(100); }, std::out_of_range);
EXPECT_THROW({ graph.Community(150); }, std::out_of_range);
}
TEST(Graph, CommunityNormalization) {
// Communities are already normalized.
comdata::Graph graph = GenRandomUnweightedGraph(5, 10);
std::vector<uint32_t> init_c = {0, 2, 1, 3, 4};
std::vector<uint32_t> final_c = {0, 2, 1, 3, 4};
ASSERT_TRUE(SetCommunities(&graph, init_c));
graph.NormalizeCommunities();
ASSERT_TRUE(CommunityCheck(graph, final_c));
// Each node in its own community.
graph = GenRandomUnweightedGraph(5, 10);
init_c = {20, 30, 10, 40, 50};
final_c = {1, 2, 0, 3, 4};
ASSERT_TRUE(SetCommunities(&graph, init_c));
graph.NormalizeCommunities();
ASSERT_TRUE(CommunityCheck(graph, final_c));
// Multiple nodes in the same community
graph = GenRandomUnweightedGraph(7, 10);
init_c = {13, 99, 13, 13, 1, 99, 1};
final_c = {1, 2, 1, 1, 0, 2, 0};
ASSERT_TRUE(SetCommunities(&graph, init_c));
graph.NormalizeCommunities();
ASSERT_TRUE(CommunityCheck(graph, final_c));
}
TEST(Graph, AddEdge) {
comdata::Graph graph = GenRandomUnweightedGraph(5, 0);
// Node out of bounds.
EXPECT_THROW({ graph.AddEdge(1, 5, 7); }, std::out_of_range);
// Repeated edge
graph.AddEdge(1, 2, 1);
EXPECT_THROW({ graph.AddEdge(1, 2, 7); }, std::invalid_argument);
// Non-positive edge weight
EXPECT_THROW({ graph.AddEdge(2, 3, -7); }, std::out_of_range);
EXPECT_THROW({ graph.AddEdge(3, 4, 0); }, std::out_of_range);
}
TEST(Graph, Degrees) {
// Graph without edges
comdata::Graph graph = GenRandomUnweightedGraph(5, 0);
std::vector<uint32_t> deg = {0, 0, 0, 0, 0};
ASSERT_TRUE(DegreeCheck(graph, deg));
// Chain
// (0)--(1)--(2)--(3)--(4)
graph = BuildGraph(5, {{0, 1, 1}, {1, 2, 1}, {2, 3, 1}, {3, 4, 1}});
deg = {1, 2, 2, 2, 1};
ASSERT_TRUE(DegreeCheck(graph, deg));
// Tree
// (0)--(3)
// / \
// (1) (2)
// | / \
// (4) (5) (6)
graph = BuildGraph(
7, {{0, 1, 1}, {0, 2, 1}, {0, 3, 1}, {1, 4, 1}, {2, 5, 1}, {2, 6, 1}});
deg = {3, 2, 3, 1, 1, 1, 1};
ASSERT_TRUE(DegreeCheck(graph, deg));
// Graph without self-loops
// (0)--(1)
// | \ | \
// | \ | \
// (2)--(3)-(4)
graph = BuildGraph(5, {{0, 1, 1},
{0, 2, 1},
{0, 3, 1},
{1, 3, 1},
{1, 4, 1},
{2, 3, 1},
{3, 4, 1}});
deg = {3, 3, 2, 4, 2};
ASSERT_TRUE(DegreeCheck(graph, deg));
// Graph with self loop [*nodes have self loops]
// (0)--(1*)
// | \ | \
// | \ | \
// (2*)--(3)-(4*)
graph = BuildGraph(5, {{0, 1, 1},
{0, 2, 1},
{0, 3, 1},
{1, 3, 1},
{1, 4, 1},
{2, 3, 1},
{3, 4, 1},
{1, 1, 1},
{2, 2, 2},
{4, 4, 4}});
deg = {3, 4, 3, 4, 3};
ASSERT_TRUE(DegreeCheck(graph, deg));
// Try to get degree of non-existing nodes
EXPECT_THROW({ graph.Degree(5); }, std::out_of_range);
EXPECT_THROW({ graph.Degree(100); }, std::out_of_range);
}
TEST(Graph, Weights) {
// Graph without edges
comdata::Graph graph = GenRandomUnweightedGraph(5, 0);
std::vector<double> inc_w = {0, 0, 0, 0, 0};
ASSERT_TRUE(IncidentWeightCheck(graph, inc_w));
ASSERT_EQ(graph.TotalWeight(), 0);
// Chain
// (0)--(1)--(2)--(3)--(4)
graph = BuildGraph(5, {{0, 1, 0.1}, {1, 2, 0.5}, {2, 3, 2.3}, {3, 4, 4.2}});
inc_w = {0.1, 0.6, 2.8, 6.5, 4.2};
ASSERT_TRUE(IncidentWeightCheck(graph, inc_w));
ASSERT_NEAR(graph.TotalWeight(), 7.1, 1e-6);
// Tree
// (0)--(3)
// / \
// (1) (2)
// | / \
// (4) (5) (6)
graph = BuildGraph(7, {{0, 1, 1.3},
{0, 2, 0.2},
{0, 3, 1},
{1, 4, 3.2},
{2, 5, 4.2},
{2, 6, 0.7}});
inc_w = {2.5, 4.5, 5.1, 1, 3.2, 4.2, 0.7};
ASSERT_TRUE(IncidentWeightCheck(graph, inc_w));
EXPECT_NEAR(graph.TotalWeight(), 10.6, 1e-6);
// Graph without self-loops
// (0)--(1)
// | \ | \
// | \ | \
// (2)--(3)-(4)
graph = BuildGraph(5, {{0, 1, 0.1},
{0, 2, 0.2},
{0, 3, 0.3},
{1, 3, 0.4},
{1, 4, 0.5},
{2, 3, 0.6},
{3, 4, 0.7}});
inc_w = {0.6, 1, 0.8, 2, 1.2};
ASSERT_TRUE(IncidentWeightCheck(graph, inc_w));
EXPECT_NEAR(graph.TotalWeight(), 2.8, 1e-6);
// Graph with self loop [*nodes have self loops]
// (0)--(1*)
// | \ | \
// | \ | \
// (2*)--(3)-(4*)
graph = BuildGraph(5, {{0, 1, 0.1},
{0, 2, 0.2},
{0, 3, 0.3},
{1, 3, 0.4},
{1, 4, 0.5},
{2, 3, 0.6},
{3, 4, 0.7},
{1, 1, 0.8},
{2, 2, 0.9},
{4, 4, 1}});
inc_w = {0.6, 1.8, 1.7, 2, 2.2};
ASSERT_TRUE(IncidentWeightCheck(graph, inc_w));
EXPECT_NEAR(graph.TotalWeight(), 5.5, 1e-6);
// Try to get incident weight of non-existing node
EXPECT_THROW({ graph.IncidentWeight(5); }, std::out_of_range);
EXPECT_THROW({ graph.IncidentWeight(100); }, std::out_of_range);
}
TEST(Graph, Modularity) {
// Graph without edges
comdata::Graph graph = GenRandomUnweightedGraph(5, 0);
ASSERT_EQ(graph.Modularity(), 0);
// Chain
// (0)--(1)--(2)--(3)--(4)
graph = BuildGraph(5, {{0, 1, 0.1}, {1, 2, 0.5}, {2, 3, 2.3}, {3, 4, 4.2}});
std::vector<uint32_t> c = {0, 1, 1, 2, 2};
SetCommunities(&graph, c);
EXPECT_NEAR(graph.Modularity(), 0.036798254314620096, 1e-6);
// Tree
// (0)--(3)
// / \
// (1) (2)
// | / \
// (4) (5) (6)
graph = BuildGraph(7, {{0, 1, 1.3},
{0, 2, 0.2},
{0, 3, 1},
{1, 4, 3.2},
{2, 5, 4.2},
{2, 6, 0.7}});
c = {0, 0, 1, 0, 0, 1, 2};
SetCommunities(&graph, c);
EXPECT_NEAR(graph.Modularity(), 0.4424617301530794, 1e-6);
// Graph without self-loops
// (0)--(1)
// | \ | \
// | \ | \
// (2)--(3)-(4)
graph = BuildGraph(5, {{0, 1, 0.1},
{0, 2, 0.2},
{0, 3, 0.3},
{1, 3, 0.4},
{1, 4, 0.5},
{2, 3, 0.6},
{3, 4, 0.7}});
c = {0, 1, 1, 1, 1};
SetCommunities(&graph, c);
EXPECT_NEAR(graph.Modularity(), -0.022959183673469507, 1e-6);
// Graph with self loop [*nodes have self loops]
// (0)--(1*)
// | \ | \
// | \ | \
// (2*)--(3)-(4*)
graph = BuildGraph(5, {{0, 1, 0.1},
{0, 2, 0.2},
{0, 3, 0.3},
{1, 3, 0.4},
{1, 4, 0.5},
{2, 3, 0.6},
{3, 4, 0.7},
{1, 1, 0.8},
{2, 2, 0.9},
{4, 4, 1}});
c = {0, 0, 0, 0, 1};
SetCommunities(&graph, c);
EXPECT_NEAR(graph.Modularity(), 0.188842975206611, 1e-6);
// Neo4j example graph
// (0)--(1)---(3)--(4)
// \ / \ /
// (2) (5)
graph = BuildGraph(6, {{0, 1, 1},
{1, 2, 1},
{0, 2, 1},
{1, 3, 1},
{3, 5, 1},
{5, 4, 1},
{3, 4, 1}});
c = {0, 0, 0, 1, 1, 1};
SetCommunities(&graph, c);
EXPECT_NEAR(graph.Modularity(), 0.3571428571428571, 1e-6);
// Example graph from wikipedia
// (0)--(1)--(3)--(4)--(5)
// \ / | \ /
// (2) (7) (6)
// / \
// (8)--(9)
graph = BuildGraph(10, {{0, 1, 1},
{1, 2, 1},
{0, 2, 1},
{1, 3, 1},
{3, 4, 1},
{4, 5, 1},
{5, 6, 1},
{6, 4, 1},
{3, 7, 1},
{7, 8, 1},
{7, 9, 1},
{8, 9, 1}});
c = {0, 0, 0, 0, 1, 1, 1, 2, 2, 2};
SetCommunities(&graph, c);
EXPECT_NEAR(graph.Modularity(), 0.4896, 1e-4);
}

View File

@@ -1,32 +0,0 @@
#include "utils.hpp"
#include <random>
comdata::Graph BuildGraph(
uint32_t nodes, std::vector<std::tuple<uint32_t, uint32_t, double>> edges) {
comdata::Graph G(nodes);
for (auto &edge : edges)
G.AddEdge(std::get<0>(edge), std::get<1>(edge), std::get<2>(edge));
return G;
}
comdata::Graph GenRandomUnweightedGraph(uint32_t nodes, uint32_t edges) {
auto seed =
std::chrono::high_resolution_clock::now().time_since_epoch().count();
std::mt19937 rng(seed);
std::uniform_int_distribution<uint32_t> dist(0, nodes - 1);
std::set<std::tuple<uint32_t, uint32_t, double>> E;
for (uint32_t i = 0; i < edges; ++i) {
int u;
int v;
do {
u = dist(rng);
v = dist(rng);
if (u > v) std::swap(u, v);
} while (u == v || E.find({u, v, 1}) != E.end());
E.insert({u, v, 1});
}
return BuildGraph(nodes, std::vector<std::tuple<uint32_t, uint32_t, double>>(
E.begin(), E.end()));
}

View File

@@ -1,18 +0,0 @@
#pragma once
#include <chrono>
#include <random>
#include <set>
#include <tuple>
#include "data_structures/graph.hpp"
/// Builds the graph from a given number of nodes and a list of edges.
/// Nodes should be 0-indexed and each edge should be provided only once.
comdata::Graph BuildGraph(
uint32_t nodes, std::vector<std::tuple<uint32_t, uint32_t, double>> edges);
/// Generates random undirected graph with a given number of nodes and edges.
/// The generated graph is not picked out of a uniform distribution. All weights
/// are the same and equal to one.
comdata::Graph GenRandomUnweightedGraph(uint32_t nodes, uint32_t edges);

View File

@@ -1,289 +0,0 @@
import sys
import mgp
import collections
try:
import networkx as nx
except ImportError as import_error:
sys.stderr.write((
'\n'
'NOTE: Please install networkx to be able to use Memgraph NetworkX '
'wrappers. Using Python:\n'
+ sys.version +
'\n'))
raise import_error
class MemgraphAdjlistOuterDict(collections.abc.Mapping):
__slots__ = ('_ctx', '_succ', '_multi')
def __init__(self, ctx, succ=True, multi=True):
self._ctx = ctx
self._succ = succ
self._multi = multi
def __getitem__(self, key):
if key not in self:
raise KeyError
return MemgraphAdjlistInnerDict(key, succ=self._succ,
multi=self._multi)
def __iter__(self):
return iter(self._ctx.graph.vertices)
def __len__(self):
return len(self._ctx.graph.vertices)
def __contains__(self, key):
if not isinstance(key, mgp.Vertex):
raise TypeError
return key in self._ctx.graph.vertices
class MemgraphAdjlistInnerDict(collections.abc.Mapping):
__slots__ = ('_node', '_succ', '_multi', '_neighbors')
def __init__(self, node, succ=True, multi=True):
self._node = node
self._succ = succ
self._multi = multi
self._neighbors = None
def __getitem__(self, key):
# NOTE: NetworkX 2.4, classes/coreviews.py:143. UnionAtlas expects a
# KeyError when indexing with a vertex that is not a neighbor.
if key not in self:
raise KeyError
if not self._multi:
return UnhashableProperties(self._get_edge(key).properties)
return MemgraphEdgeKeyDict(self._node, key, self._succ)
def __iter__(self):
yield from self._get_neighbors()
def __len__(self):
return len(self._get_neighbors())
def __contains__(self, key):
if not isinstance(key, mgp.Vertex):
raise TypeError
return key in self._get_neighbors()
def _get_neighbors(self):
if not self._neighbors:
if self._succ:
self._neighbors = set(
e.to_vertex for e in self._node.out_edges)
else:
self._neighbors = set(
e.from_vertex for e in self._node.in_edges)
return self._neighbors
def _get_edge(self, neighbor):
if self._succ:
edge = list(filter(lambda e: e.to_vertex == neighbor,
self._node.out_edges))
else:
edge = list(filter(lambda e: e.from_vertex == neighbor,
self._node.in_edges))
assert len(edge) >= 1
if len(edge) > 1:
raise RuntimeError('Graph contains multiedges but '
'is of non-multigraph type: {}'.format(edge))
return edge[0]
class MemgraphEdgeKeyDict(collections.abc.Mapping):
__slots__ = ('_node', '_neighbor', '_succ', '_edges')
def __init__(self, node, neighbor, succ=True):
self._node = node
self._neighbor = neighbor
self._succ = succ
self._edges = None
def __getitem__(self, key):
if key not in self:
raise KeyError
return UnhashableProperties(key.properties)
def __iter__(self):
yield from self._get_edges()
def __len__(self):
return len(self._get_edges())
def __contains__(self, key):
if not isinstance(key, mgp.Edge):
raise TypeError
return key in self._get_edges()
def _get_edges(self):
if not self._edges:
if self._succ:
self._edges = list(filter(
lambda e: e.to_vertex == self._neighbor,
self._node.out_edges))
else:
self._edges = list(filter(
lambda e: e.from_vertex == self._neighbor,
self._node.in_edges))
return self._edges
class UnhashableProperties(collections.abc.Mapping):
__slots__ = ('_properties')
def __init__(self, properties):
self._properties = properties
def __getitem__(self, key):
return self._properties[key]
def __iter__(self):
yield from self._properties
def __len__(self):
return len(self._properties)
def __contains__(self, key):
return key in self._properties
# NOTE: Explicitly disable hashing. See the comment in MemgraphNodeDict.
__hash__ = None
class MemgraphNodeDict(collections.abc.Mapping):
__slots__ = ('_ctx',)
def __init__(self, ctx):
self._ctx = ctx
def __getitem__(self, key):
if key not in self:
raise KeyError
# NOTE: NetworkX 2.4, classes/digraph.py:484. NetworkX expects the
# tuples provided to add_nodes_from to be unhashable and cause a
# TypeError when trying to index the node dictionary. This happens
# because the data dictionary element of the tuple is unhashable. We do
# the same thing by returning an unhashable data dictionary.
return UnhashableProperties(key.properties)
def __iter__(self):
return iter(self._ctx.graph.vertices)
def __len__(self):
return len(self._ctx.graph.vertices)
def __contains__(self, key):
# NOTE: NetworkX 2.4, graph.py:425. Graph.__contains__ relies on
# self._node's (i.e. the dictionary produced by node_dict_factory)
# __contains__ to raise a TypeError when indexing with something weird,
# e.g. with sets. This is the behavior of dict.
if not isinstance(key, mgp.Vertex):
raise TypeError
return key in self._ctx.graph.vertices
class MemgraphDiGraphBase:
def __init__(self, incoming_graph_data=None, ctx=None, multi=True,
**kwargs):
# NOTE: We assume that our graph will never be given any initial data
# because we already pull our data from the Memgraph database. This
# assert is triggered by certain NetworkX procedures because they
# create a new instance of our class and try to populate it with their
# own data.
assert incoming_graph_data is None
# NOTE: We allow for ctx to be None in order to allow certain NetworkX
# procedures (such as `subgraph`) to work. Such procedures directly
# modify the graph's internal attributes and don't try to populate it
# with initial data or modify it.
self.node_dict_factory = lambda: MemgraphNodeDict(ctx) \
if ctx else self._error
self.node_attr_dict_factory = self._error
self.adjlist_outer_dict_factory = \
lambda: MemgraphAdjlistOuterDict(ctx, multi=multi) \
if ctx else self._error
self.adjlist_inner_dict_factory = self._error
self.edge_key_dict_factory = self._error
self.edge_attr_dict_factory = self._error
# NOTE: We forbid any mutating operations because our graph is
# immutable and pulls its data from the Memgraph database.
for f in ['add_node', 'add_nodes_from', 'remove_node',
'remove_nodes_from', 'add_edge', 'add_edges_from',
'add_weighted_edges_from', 'new_edge_key', 'remove_edge',
'remove_edges_from', 'update', 'clear']:
setattr(self, f, lambda *args, **kwargs: self._error())
super().__init__(None, **kwargs)
# NOTE: This is a necessary hack because NetworkX assumes that the
# customizable factory functions will only ever return *empty*
# dictionaries. In our case, the factory functions return our custom,
# already populated, dictionaries. Because self._pred and self._end are
# initialized by the same factory function, they end up storing the
# same adjacency lists which is not good. We correct that here.
self._pred = MemgraphAdjlistOuterDict(ctx, succ=False, multi=multi)
def _error(self):
raise RuntimeError('Modification operations are not supported')
class MemgraphMultiDiGraph(MemgraphDiGraphBase, nx.MultiDiGraph):
def __init__(self, incoming_graph_data=None, ctx=None, **kwargs):
super().__init__(incoming_graph_data=incoming_graph_data,
ctx=ctx, multi=True, **kwargs)
def MemgraphMultiGraph(incoming_graph_data=None, ctx=None, **kwargs):
return MemgraphMultiDiGraph(incoming_graph_data=incoming_graph_data,
ctx=ctx, **kwargs).to_undirected(as_view=True)
class MemgraphDiGraph(MemgraphDiGraphBase, nx.DiGraph):
def __init__(self, incoming_graph_data=None, ctx=None, **kwargs):
super().__init__(incoming_graph_data=incoming_graph_data,
ctx=ctx, multi=False, **kwargs)
def MemgraphGraph(incoming_graph_data=None, ctx=None, **kwargs):
return MemgraphDiGraph(incoming_graph_data=incoming_graph_data,
ctx=ctx, **kwargs).to_undirected(as_view=True)
class PropertiesDictionary(collections.abc.Mapping):
__slots__ = ('_ctx', '_prop', '_len')
def __init__(self, ctx, prop):
self._ctx = ctx
self._prop = prop
self._len = None
def __getitem__(self, vertex):
if vertex not in self:
raise KeyError
try:
return vertex.properties[self._prop]
except KeyError:
raise KeyError(("{} doesn\t have the required " +
"property '{}'").format(vertex, self._prop))
def __iter__(self):
for v in self._ctx.graph.vertices:
if self._prop in v.properties:
yield v
def __len__(self):
if not self._len:
self._len = sum(1 for _ in self)
return self._len
def __contains__(self, vertex):
if not isinstance(vertex, mgp.Vertex):
raise TypeError
return self._prop in vertex.properties

View File

@@ -1,825 +0,0 @@
import sys
import mgp
try:
import networkx as nx
import numpy # noqa E401
import scipy # noqa E401
except ImportError as import_error:
sys.stderr.write((
'\n'
'NOTE: Please install networkx, numpy, scipy to be able to '
'use proxied NetworkX algorithms. E.g., CALL nxalg.pagerank(...).\n'
'Using Python:\n'
+ sys.version +
'\n'))
raise import_error
# Imported last because it also depends on networkx.
from mgp_networkx import (MemgraphMultiDiGraph, MemgraphDiGraph, # noqa: E402
MemgraphMultiGraph, MemgraphGraph,
PropertiesDictionary)
# networkx.algorithms.approximation.connectivity.node_connectivity
@mgp.read_proc
def node_connectivity(ctx: mgp.ProcCtx,
source: mgp.Nullable[mgp.Vertex] = None,
target: mgp.Nullable[mgp.Vertex] = None
) -> mgp.Record(connectivity=int):
return mgp.Record(connectivity=nx.node_connectivity(
MemgraphMultiDiGraph(ctx=ctx), source, target))
# networkx.algorithms.assortativity.degree_assortativity_coefficient
@mgp.read_proc
def degree_assortativity_coefficient(
ctx: mgp.ProcCtx,
x: str = 'out',
y: str = 'in',
weight: mgp.Nullable[str] = None,
nodes: mgp.Nullable[mgp.List[mgp.Vertex]] = None
) -> mgp.Record(assortativity=float):
return mgp.Record(assortativity=nx.degree_assortativity_coefficient(
MemgraphMultiDiGraph(ctx=ctx), x, y, weight, nodes))
# networkx.algorithms.asteroidal.is_at_free
@mgp.read_proc
def is_at_free(ctx: mgp.ProcCtx) -> mgp.Record(is_at_free=bool):
return mgp.Record(is_at_free=nx.is_at_free(MemgraphGraph(ctx=ctx)))
# networkx.algorithms.bipartite.basic.is_bipartite
@mgp.read_proc
def is_bipartite(ctx: mgp.ProcCtx) -> mgp.Record(is_bipartite=bool):
return mgp.Record(is_bipartite=nx.is_bipartite(
MemgraphMultiDiGraph(ctx=ctx)))
# networkx.algorithms.boundary.node_boundary
@mgp.read_proc
def node_boundary(ctx: mgp.ProcCtx,
nbunch1: mgp.List[mgp.Vertex],
nbunch2: mgp.Nullable[mgp.List[mgp.Vertex]] = None
) -> mgp.Record(boundary=mgp.List[mgp.Vertex]):
return mgp.Record(boundary=list(nx.node_boundary(
MemgraphMultiDiGraph(ctx=ctx), nbunch1, nbunch2)))
# networkx.algorithms.bridges.bridges
@mgp.read_proc
def bridges(ctx: mgp.ProcCtx,
root: mgp.Nullable[mgp.Vertex] = None
) -> mgp.Record(bridges=mgp.List[mgp.Edge]):
g = MemgraphMultiGraph(ctx=ctx)
return mgp.Record(
bridges=[next(iter(g[u][v]))
for u, v in nx.bridges(MemgraphGraph(ctx=ctx),
root=root)])
# networkx.algorithms.centrality.betweenness_centrality
@mgp.read_proc
def betweenness_centrality(ctx: mgp.ProcCtx,
k: mgp.Nullable[int] = None,
normalized: bool = True,
weight: mgp.Nullable[str] = None,
endpoints: bool = False,
seed: mgp.Nullable[int] = None
) -> mgp.Record(node=mgp.Vertex,
betweenness=mgp.Number):
return [mgp.Record(node=n, betweenness=b)
for n, b in nx.betweenness_centrality(
MemgraphDiGraph(ctx=ctx), k=k, normalized=normalized,
weight=weight, endpoints=endpoints, seed=seed).items()]
# networkx.algorithms.chains.chain_decomposition
@mgp.read_proc
def chain_decomposition(ctx: mgp.ProcCtx,
root: mgp.Nullable[mgp.Vertex] = None
) -> mgp.Record(chains=mgp.List[mgp.List[mgp.Edge]]):
g = MemgraphMultiGraph(ctx=ctx)
return mgp.Record(
chains=[[next(iter(g[u][v])) for u, v in d]
for d in nx.chain_decomposition(MemgraphGraph(ctx=ctx),
root=root)])
# networkx.algorithms.chordal.is_chordal
@mgp.read_proc
def is_chordal(ctx: mgp.ProcCtx) -> mgp.Record(is_chordal=bool):
return mgp.Record(is_chordal=nx.is_chordal(MemgraphGraph(ctx=ctx)))
# networkx.algorithms.clique.find_cliques
@mgp.read_proc
def find_cliques(ctx: mgp.ProcCtx
) -> mgp.Record(cliques=mgp.List[mgp.List[mgp.Vertex]]):
return mgp.Record(cliques=list(nx.find_cliques(
MemgraphMultiGraph(ctx=ctx))))
# networkx.algorithms.cluster.clustering
@mgp.read_proc
def clustering(ctx: mgp.ProcCtx,
nodes: mgp.Nullable[mgp.List[mgp.Vertex]] = None,
weight: mgp.Nullable[str] = None
) -> mgp.Record(node=mgp.Vertex, clustering=mgp.Number):
return [mgp.Record(node=n, clustering=c)
for n, c in nx.clustering(
MemgraphDiGraph(ctx=ctx), nodes=nodes,
weight=weight).items()]
# networkx.algorithms.coloring.greedy_color
@mgp.read_proc
def greedy_color(ctx: mgp.ProcCtx,
strategy: str = 'largest_first',
interchange: bool = False
) -> mgp.Record(node=mgp.Vertex, color=int):
return [mgp.Record(node=n, color=c) for n, c in nx.greedy_color(
MemgraphMultiDiGraph(ctx=ctx), strategy, interchange).items()]
# networkx.algorithms.communicability_alg.communicability
@mgp.read_proc
def communicability(ctx: mgp.ProcCtx
) -> mgp.Record(node1=mgp.Vertex, node2=mgp.Vertex,
communicability=mgp.Number):
return [mgp.Record(node1=n1, node2=n2, communicability=v)
for n1, d in nx.communicability(MemgraphGraph(ctx=ctx)).items()
for n2, v in d.items()]
# networkx.algorithms.community.kclique.k_clique_communities
@mgp.read_proc
def k_clique_communities(
ctx: mgp.ProcCtx,
k: int,
cliques: mgp.Nullable[mgp.List[mgp.List[mgp.Vertex]]] = None
) -> mgp.Record(communities=mgp.List[mgp.List[mgp.Vertex]]):
return mgp.Record(communities=[
list(s) for s in nx.community.k_clique_communities(
MemgraphMultiGraph(ctx=ctx), k, cliques)])
# networkx.algorithms.approximation.kcomponents.k_components
@mgp.read_proc
def k_components(ctx: mgp.ProcCtx,
density: mgp.Number = 0.95
) -> mgp.Record(k=int,
components=mgp.List[mgp.List[mgp.Vertex]]):
kcomps = nx.k_components(MemgraphMultiGraph(ctx=ctx), density)
return [mgp.Record(k=k, components=[list(s) for s in comps])
for k, comps in kcomps.items()]
# networkx.algorithms.components.biconnected_components
@mgp.read_proc
def biconnected_components(
ctx: mgp.ProcCtx
) -> mgp.Record(components=mgp.List[mgp.List[mgp.Vertex]]):
comps = nx.biconnected_components(MemgraphMultiGraph(ctx=ctx))
return mgp.Record(components=[list(s) for s in comps])
# networkx.algorithms.components.strongly_connected_components
@mgp.read_proc
def strongly_connected_components(
ctx: mgp.ProcCtx
) -> mgp.Record(components=mgp.List[mgp.List[mgp.Vertex]]):
comps = nx.strongly_connected_components(MemgraphMultiDiGraph(ctx=ctx))
return mgp.Record(components=[list(s) for s in comps])
# networkx.algorithms.connectivity.edge_kcomponents.k_edge_components
#
# NOTE: NetworkX 2.4, algorithms/connectivity/edge_kcompnents.py:367. We create
# a *copy* of the graph because the algorithm copies the graph using
# __class__() and tries to modify it.
@mgp.read_proc
def k_edge_components(
ctx: mgp.ProcCtx,
k: int
) -> mgp.Record(components=mgp.List[mgp.List[mgp.Vertex]]):
return mgp.Record(components=[list(s) for s in nx.k_edge_components(
nx.DiGraph(MemgraphDiGraph(ctx=ctx)), k)])
# networkx.algorithms.core.core_number
@mgp.read_proc
def core_number(ctx: mgp.ProcCtx
) -> mgp.Record(node=mgp.Vertex, core=mgp.Number):
return [mgp.Record(node=n, core=c)
for n, c in nx.core_number(MemgraphDiGraph(ctx=ctx)).items()]
# networkx.algorithms.covering.is_edge_cover
@mgp.read_proc
def is_edge_cover(ctx: mgp.ProcCtx, cover: mgp.List[mgp.Edge]
) -> mgp.Record(is_edge_cover=bool):
cover = set([(e.from_vertex, e.to_vertex) for e in cover])
return mgp.Record(is_edge_cover=nx.is_edge_cover(
MemgraphMultiGraph(ctx=ctx), cover))
# networkx.algorithms.cycles.find_cycle
@mgp.read_proc
def find_cycle(ctx: mgp.ProcCtx,
source: mgp.Nullable[mgp.List[mgp.Vertex]] = None,
orientation: mgp.Nullable[str] = None
) -> mgp.Record(cycle=mgp.Nullable[mgp.List[mgp.Edge]]):
try:
return mgp.Record(cycle=[e for _, _, e in nx.find_cycle(
MemgraphMultiDiGraph(ctx=ctx), source, orientation)])
except nx.NetworkXNoCycle:
return mgp.Record(cycle=None)
# networkx.algorithms.cycles.simple_cycles
#
# NOTE: NetworkX 2.4, algorithms/cycles.py:183. We create a *copy* of the graph
# because the algorithm copies the graph using type() and tries to pass initial
# data.
@mgp.read_proc
def simple_cycles(ctx: mgp.ProcCtx
) -> mgp.Record(cycles=mgp.List[mgp.List[mgp.Vertex]]):
return mgp.Record(cycles=list(nx.simple_cycles(
nx.MultiDiGraph(MemgraphMultiDiGraph(ctx=ctx)).copy())))
# networkx.algorithms.cuts.node_expansion
@mgp.read_proc
def node_expansion(ctx: mgp.ProcCtx, s: mgp.List[mgp.Vertex]
) -> mgp.Record(node_expansion=mgp.Number):
return mgp.Record(node_expansion=nx.node_expansion(
MemgraphMultiDiGraph(ctx=ctx), set(s)))
# networkx.algorithms.dag.topological_sort
@mgp.read_proc
def topological_sort(ctx: mgp.ProcCtx
) -> mgp.Record(nodes=mgp.Nullable[mgp.List[mgp.Vertex]]):
return mgp.Record(nodes=list(nx.topological_sort(
MemgraphMultiDiGraph(ctx=ctx))))
# networkx.algorithms.dag.ancestors
@mgp.read_proc
def ancestors(ctx: mgp.ProcCtx,
source: mgp.Vertex
) -> mgp.Record(ancestors=mgp.List[mgp.Vertex]):
return mgp.Record(ancestors=list(nx.ancestors(
MemgraphMultiDiGraph(ctx=ctx), source)))
# networkx.algorithms.dag.descendants
@mgp.read_proc
def descendants(ctx: mgp.ProcCtx,
source: mgp.Vertex
) -> mgp.Record(descendants=mgp.List[mgp.Vertex]):
return mgp.Record(descendants=list(nx.descendants(
MemgraphMultiDiGraph(ctx=ctx), source)))
# networkx.algorithms.distance_measures.center
#
# NOTE: Takes more parameters.
@mgp.read_proc
def center(ctx: mgp.ProcCtx) -> mgp.Record(center=mgp.List[mgp.Vertex]):
return mgp.Record(center=list(nx.center(MemgraphMultiDiGraph(ctx=ctx))))
# networkx.algorithms.distance_measures.diameter
#
# NOTE: Takes more parameters.
@mgp.read_proc
def diameter(ctx: mgp.ProcCtx) -> mgp.Record(diameter=int):
return mgp.Record(diameter=nx.diameter(MemgraphMultiDiGraph(ctx=ctx)))
# networkx.algorithms.distance_regular.is_distance_regular
@mgp.read_proc
def is_distance_regular(ctx: mgp.ProcCtx
) -> mgp.Record(is_distance_regular=bool):
return mgp.Record(is_distance_regular=nx.is_distance_regular(
MemgraphMultiGraph(ctx=ctx)))
# networkx.algorithms.strongly_regular.is_strongly_regular
@mgp.read_proc
def is_strongly_regular(ctx: mgp.ProcCtx
) -> mgp.Record(is_strongly_regular=bool):
return mgp.Record(is_strongly_regular=nx.is_strongly_regular(
MemgraphMultiGraph(ctx=ctx)))
# networkx.algorithms.dominance.dominance_frontiers
@mgp.read_proc
def dominance_frontiers(ctx: mgp.ProcCtx, start: mgp.Vertex,
) -> mgp.Record(node=mgp.Vertex,
frontier=mgp.List[mgp.Vertex]):
return [mgp.Record(node=n, frontier=list(f))
for n, f in nx.dominance_frontiers(
MemgraphMultiDiGraph(ctx=ctx), start).items()]
# networkx.algorithms.dominance.immediate_dominatorss
@mgp.read_proc
def immediate_dominators(ctx: mgp.ProcCtx, start: mgp.Vertex,
) -> mgp.Record(node=mgp.Vertex,
dominator=mgp.Vertex):
return [mgp.Record(node=n, dominator=d)
for n, d in nx.immediate_dominators(
MemgraphMultiDiGraph(ctx=ctx), start).items()]
# networkx.algorithms.dominating.dominating_set
@mgp.read_proc
def dominating_set(ctx: mgp.ProcCtx, start: mgp.Vertex,
) -> mgp.Record(dominating_set=mgp.List[mgp.Vertex]):
return mgp.Record(dominating_set=list(nx.dominating_set(
MemgraphMultiDiGraph(ctx=ctx), start)))
# networkx.algorithms.efficiency_measures.local_efficiency
@mgp.read_proc
def local_efficiency(ctx: mgp.ProcCtx) -> mgp.Record(local_efficiency=float):
return mgp.Record(local_efficiency=nx.local_efficiency(
MemgraphMultiGraph(ctx=ctx)))
# networkx.algorithms.efficiency_measures.global_efficiency
@mgp.read_proc
def global_efficiency(ctx: mgp.ProcCtx) -> mgp.Record(global_efficiency=float):
return mgp.Record(global_efficiency=nx.global_efficiency(
MemgraphMultiGraph(ctx=ctx)))
# networkx.algorithms.euler.is_eulerian
@mgp.read_proc
def is_eulerian(ctx: mgp.ProcCtx) -> mgp.Record(is_eulerian=bool):
return mgp.Record(is_eulerian=nx.is_eulerian(
MemgraphMultiDiGraph(ctx=ctx)))
# networkx.algorithms.euler.is_semieulerian
@mgp.read_proc
def is_semieulerian(ctx: mgp.ProcCtx) -> mgp.Record(is_semieulerian=bool):
return mgp.Record(is_semieulerian=nx.is_semieulerian(
MemgraphMultiDiGraph(ctx=ctx)))
# networkx.algorithms.euler.has_eulerian_path
@mgp.read_proc
def has_eulerian_path(ctx: mgp.ProcCtx) -> mgp.Record(has_eulerian_path=bool):
return mgp.Record(has_eulerian_path=nx.has_eulerian_path(
MemgraphMultiDiGraph(ctx=ctx)))
# networkx.algorithms.hierarchy.flow_hierarchy
@mgp.read_proc
def flow_hierarchy(ctx: mgp.ProcCtx,
weight: mgp.Nullable[str] = None
) -> mgp.Record(flow_hierarchy=float):
return mgp.Record(flow_hierarchy=nx.flow_hierarchy(
MemgraphMultiDiGraph(ctx=ctx), weight=weight))
# networkx.algorithms.isolate.isolates
@mgp.read_proc
def isolates(ctx: mgp.ProcCtx) -> mgp.Record(isolates=mgp.List[mgp.Vertex]):
return mgp.Record(isolates=list(nx.isolates(
MemgraphMultiDiGraph(ctx=ctx))))
# networkx.algorithms.isolate.is_isolate
@mgp.read_proc
def is_isolate(ctx: mgp.ProcCtx, n: mgp.Vertex
) -> mgp.Record(is_isolate=bool):
return mgp.Record(is_isolate=nx.is_isolate(
MemgraphMultiDiGraph(ctx=ctx), n))
# networkx.algorithms.isomorphism.is_isomorphic
@mgp.read_proc
def is_isomorphic(ctx: mgp.ProcCtx,
nodes1: mgp.List[mgp.Vertex],
edges1: mgp.List[mgp.Edge],
nodes2: mgp.List[mgp.Vertex],
edges2: mgp.List[mgp.Edge]
) -> mgp.Record(is_isomorphic=bool):
nodes1, edges1, nodes2, edges2 = map(set, [nodes1, edges1, nodes2, edges2])
g = MemgraphMultiDiGraph(ctx=ctx)
g1 = nx.subgraph_view(
g, lambda n: n in nodes1, lambda n1, n2, e: e in edges1)
g2 = nx.subgraph_view(
g, lambda n: n in nodes2, lambda n1, n2, e: e in edges2)
return mgp.Record(is_isomorphic=nx.is_isomorphic(g1, g2))
# networkx.algorithms.link_analysis.pagerank_alg.pagerank
@mgp.read_proc
def pagerank(ctx: mgp.ProcCtx,
alpha: mgp.Number = 0.85,
personalization: mgp.Nullable[str] = None,
max_iter: int = 100,
tol: mgp.Number = 1e-06,
nstart: mgp.Nullable[str] = None,
weight: mgp.Nullable[str] = 'weight',
dangling: mgp.Nullable[str] = None,
) -> mgp.Record(node=mgp.Vertex, rank=float):
def to_properties_dictionary(prop):
return None if prop is None else PropertiesDictionary(ctx, prop)
pg = nx.pagerank(MemgraphDiGraph(ctx=ctx), alpha=alpha,
personalization=to_properties_dictionary(personalization),
max_iter=max_iter, tol=tol,
nstart=to_properties_dictionary(nstart), weight=weight,
dangling=to_properties_dictionary(dangling))
return [mgp.Record(node=k, rank=v) for k, v in pg.items()]
# networkx.algorithms.link_prediction.jaccard_coefficient
@mgp.read_proc
def jaccard_coefficient(
ctx: mgp.ProcCtx,
ebunch: mgp.Nullable[mgp.List[mgp.List[mgp.Vertex]]] = None
) -> mgp.Record(u=mgp.Vertex, v=mgp.Vertex,
coef=float):
return [mgp.Record(u=u, v=v, coef=c) for u, v, c
in nx.jaccard_coefficient(MemgraphGraph(ctx=ctx), ebunch)]
# networkx.algorithms.lowest_common_ancestors.lowest_common_ancestor
@mgp.read_proc
def lowest_common_ancestor(ctx: mgp.ProcCtx, node1: mgp.Vertex,
node2: mgp.Vertex
) -> mgp.Record(ancestor=mgp.Nullable[mgp.Vertex]):
return mgp.Record(ancestor=nx.lowest_common_ancestor(
MemgraphDiGraph(ctx=ctx), node1, node2))
# networkx.algorithms.matching.maximal_matching
@mgp.read_proc
def maximal_matching(ctx: mgp.ProcCtx) -> mgp.Record(edges=mgp.List[mgp.Edge]):
g = MemgraphMultiDiGraph(ctx=ctx)
return mgp.Record(edges=list(
next(iter(g[u][v])) for u, v in nx.maximal_matching(g)))
# networkx.algorithms.planarity.check_planarity
#
# NOTE: Returns a graph.
@mgp.read_proc
def check_planarity(ctx: mgp.ProcCtx) -> mgp.Record(is_planar=bool):
return mgp.Record(is_planar=nx.check_planarity(
MemgraphMultiDiGraph(ctx=ctx))[0])
# networkx.algorithms.non_randomness.non_randomness
@mgp.read_proc
def non_randomness(ctx: mgp.ProcCtx,
k: mgp.Nullable[int] = None
) -> mgp.Record(non_randomness=float,
relative_non_randomness=float):
nn, rnn = nx.non_randomness(
MemgraphGraph(ctx=ctx), k=k)
return mgp.Record(non_randomness=nn, relative_non_randomness=rnn)
# networkx.algorithms.reciprocity.reciprocity
@mgp.read_proc
def reciprocity(ctx: mgp.ProcCtx,
nodes: mgp.Nullable[mgp.List[mgp.Vertex]] = None
) -> mgp.Record(node=mgp.Nullable[mgp.Vertex],
reciprocity=mgp.Nullable[float]):
rp = nx.reciprocity(MemgraphMultiDiGraph(ctx=ctx), nodes=nodes)
if nodes is None:
return mgp.Record(node=None, reciprocity=rp)
else:
return [mgp.Record(node=n, reciprocity=r) for n, r in rp.items()]
# networkx.algorithms.shortest_paths.generic.shortest_path
@mgp.read_proc
def shortest_path(ctx: mgp.ProcCtx,
source: mgp.Nullable[mgp.Vertex] = None,
target: mgp.Nullable[mgp.Vertex] = None,
weight: mgp.Nullable[str] = None,
method: str = 'dijkstra'
) -> mgp.Record(source=mgp.Vertex, target=mgp.Vertex,
path=mgp.List[mgp.Vertex]):
sp = nx.shortest_path(MemgraphMultiDiGraph(ctx=ctx), source=source,
target=target, weight=weight, method=method)
if source and target:
sp = {source: {target: sp}}
elif source and not target:
sp = {source: sp}
elif not source and target:
sp = {source: {target: p} for source, p in sp.items()}
return [mgp.Record(source=s, target=t, path=p)
for s, d in sp.items()
for t, p in d.items()]
# networkx.algorithms.shortest_paths.generic.shortest_path_length
@mgp.read_proc
def shortest_path_length(ctx: mgp.ProcCtx,
source: mgp.Nullable[mgp.Vertex] = None,
target: mgp.Nullable[mgp.Vertex] = None,
weight: mgp.Nullable[str] = None,
method: str = 'dijkstra'
) -> mgp.Record(source=mgp.Vertex, target=mgp.Vertex,
length=mgp.Number):
sp = nx.shortest_path_length(MemgraphMultiDiGraph(ctx=ctx), source=source,
target=target, weight=weight, method=method)
if source and target:
sp = {source: {target: sp}}
elif source and not target:
sp = {source: sp}
elif not source and target:
sp = {source: {target: l} for source, l in sp.items()}
else:
sp = dict(sp)
return [mgp.Record(source=s, target=t, length=l)
for s, d in sp.items()
for t, l in d.items()]
# networkx.algorithms.shortest_paths.generic.all_shortest_paths
@mgp.read_proc
def all_shortest_paths(ctx: mgp.ProcCtx,
source: mgp.Vertex,
target: mgp.Vertex,
weight: mgp.Nullable[str] = None,
method: str = 'dijkstra'
) -> mgp.Record(paths=mgp.List[mgp.List[mgp.Vertex]]):
return mgp.Record(paths=list(nx.all_shortest_paths(
MemgraphMultiDiGraph(ctx=ctx), source=source, target=target,
weight=weight, method=method)))
# networkx.algorithms.shortest_paths.generic.has_path
@mgp.read_proc
def has_path(ctx: mgp.ProcCtx,
source: mgp.Vertex,
target: mgp.Vertex) -> mgp.Record(has_path=bool):
return mgp.Record(has_path=nx.has_path(MemgraphMultiDiGraph(ctx=ctx),
source, target))
# networkx.algorithms.shortest_paths.weighted.multi_source_dijkstra_path
@mgp.read_proc
def multi_source_dijkstra_path(ctx: mgp.ProcCtx,
sources: mgp.List[mgp.Vertex],
cutoff: mgp.Nullable[int] = None,
weight: str = 'weight'
) -> mgp.Record(target=mgp.Vertex,
path=mgp.List[mgp.Vertex]):
return [mgp.Record(target=t, path=p)
for t, p in nx.multi_source_dijkstra_path(
MemgraphMultiDiGraph(ctx=ctx), sources, cutoff=cutoff,
weight=weight).items()]
# networkx.algorithms.shortest_paths.weighted.multi_source_dijkstra_path_length
@mgp.read_proc
def multi_source_dijkstra_path_length(ctx: mgp.ProcCtx,
sources: mgp.List[mgp.Vertex],
cutoff: mgp.Nullable[int] = None,
weight: str = 'weight'
) -> mgp.Record(target=mgp.Vertex,
length=mgp.Number):
return [mgp.Record(target=t, length=l)
for t, l in nx.multi_source_dijkstra_path_length(
MemgraphMultiDiGraph(ctx=ctx), sources, cutoff=cutoff,
weight=weight).items()]
# networkx.algorithms.simple_paths.is_simple_path
@mgp.read_proc
def is_simple_path(ctx: mgp.ProcCtx,
nodes: mgp.List[mgp.Vertex]
) -> mgp.Record(is_simple_path=bool):
return mgp.Record(is_simple_path=nx.is_simple_path(
MemgraphMultiDiGraph(ctx=ctx), nodes))
# networkx.algorithms.simple_paths.all_simple_paths
@mgp.read_proc
def all_simple_paths(ctx: mgp.ProcCtx,
source: mgp.Vertex,
target: mgp.Vertex,
cutoff: mgp.Nullable[int] = None
) -> mgp.Record(paths=mgp.List[mgp.List[mgp.Vertex]]):
return mgp.Record(paths=list(nx.all_simple_paths(
MemgraphMultiDiGraph(ctx=ctx), source, target, cutoff=cutoff)))
# networkx.algorithms.tournament.is_tournament
@mgp.read_proc
def is_tournament(ctx: mgp.ProcCtx) -> mgp.Record(is_tournament=bool):
return mgp.Record(is_tournament=nx.tournament.is_tournament(
MemgraphDiGraph(ctx=ctx)))
# networkx.algorithms.traversal.breadth_first_search.bfs_edges
@mgp.read_proc
def bfs_edges(ctx: mgp.ProcCtx,
source: mgp.Vertex,
reverse: bool = False,
depth_limit: mgp.Nullable[int] = None
) -> mgp.Record(edges=mgp.List[mgp.Edge]):
return mgp.Record(edges=list(nx.bfs_edges(
MemgraphMultiDiGraph(ctx=ctx), source, reverse=reverse,
depth_limit=depth_limit)))
# networkx.algorithms.traversal.breadth_first_search.bfs_tree
@mgp.read_proc
def bfs_tree(ctx: mgp.ProcCtx,
source: mgp.Vertex,
reverse: bool = False,
depth_limit: mgp.Nullable[int] = None
) -> mgp.Record(tree=mgp.List[mgp.Vertex]):
return mgp.Record(tree=list(nx.bfs_tree(
MemgraphMultiDiGraph(ctx=ctx), source, reverse=reverse,
depth_limit=depth_limit)))
# networkx.algorithms.traversal.breadth_first_search.bfs_predecessors
@mgp.read_proc
def bfs_predecessors(ctx: mgp.ProcCtx,
source: mgp.Vertex,
depth_limit: mgp.Nullable[int] = None
) -> mgp.Record(node=mgp.Vertex,
predecessor=mgp.Vertex):
return [mgp.Record(node=n, predecessor=p)
for n, p in nx.bfs_predecessors(
MemgraphMultiDiGraph(ctx=ctx), source,
depth_limit=depth_limit)]
# networkx.algorithms.traversal.breadth_first_search.bfs_successors
@mgp.read_proc
def bfs_successors(ctx: mgp.ProcCtx,
source: mgp.Vertex,
depth_limit: mgp.Nullable[int] = None
) -> mgp.Record(node=mgp.Vertex,
successors=mgp.List[mgp.Vertex]):
return [mgp.Record(node=n, successors=s)
for n, s in nx.bfs_successors(
MemgraphMultiDiGraph(ctx=ctx), source,
depth_limit=depth_limit)]
# networkx.algorithms.traversal.depth_first_search.dfs_tree
@mgp.read_proc
def dfs_tree(ctx: mgp.ProcCtx,
source: mgp.Vertex,
depth_limit: mgp.Nullable[int] = None
) -> mgp.Record(tree=mgp.List[mgp.Vertex]):
return mgp.Record(tree=list(nx.dfs_tree(
MemgraphMultiDiGraph(ctx=ctx), source, depth_limit=depth_limit)))
# networkx.algorithms.traversal.depth_first_search.dfs_predecessors
@mgp.read_proc
def dfs_predecessors(ctx: mgp.ProcCtx,
source: mgp.Vertex,
depth_limit: mgp.Nullable[int] = None
) -> mgp.Record(node=mgp.Vertex,
predecessor=mgp.Vertex):
return [mgp.Record(node=n, predecessor=p)
for n, p in nx.dfs_predecessors(
MemgraphMultiDiGraph(ctx=ctx), source,
depth_limit=depth_limit).items()]
# networkx.algorithms.traversal.depth_first_search.dfs_successors
@mgp.read_proc
def dfs_successors(ctx: mgp.ProcCtx,
source: mgp.Vertex,
depth_limit: mgp.Nullable[int] = None
) -> mgp.Record(node=mgp.Vertex,
successors=mgp.List[mgp.Vertex]):
return [mgp.Record(node=n, successors=s)
for n, s in nx.dfs_successors(
MemgraphMultiDiGraph(ctx=ctx), source,
depth_limit=depth_limit).items()]
# networkx.algorithms.traversal.depth_first_search.dfs_preorder_nodes
@mgp.read_proc
def dfs_preorder_nodes(ctx: mgp.ProcCtx,
source: mgp.Vertex,
depth_limit: mgp.Nullable[int] = None
) -> mgp.Record(nodes=mgp.List[mgp.Vertex]):
return mgp.Record(nodes=list(nx.dfs_preorder_nodes(
MemgraphMultiDiGraph(ctx=ctx), source, depth_limit=depth_limit)))
# networkx.algorithms.traversal.depth_first_search.dfs_postorder_nodes
@mgp.read_proc
def dfs_postorder_nodes(ctx: mgp.ProcCtx,
source: mgp.Vertex,
depth_limit: mgp.Nullable[int] = None
) -> mgp.Record(nodes=mgp.List[mgp.Vertex]):
return mgp.Record(nodes=list(nx.dfs_postorder_nodes(
MemgraphMultiDiGraph(ctx=ctx), source, depth_limit=depth_limit)))
# networkx.algorithms.traversal.edgebfs.edge_bfs
@mgp.read_proc
def edge_bfs(ctx: mgp.ProcCtx,
source: mgp.Nullable[mgp.Vertex] = None,
orientation: mgp.Nullable[str] = None
) -> mgp.Record(edges=mgp.List[mgp.Edge]):
return mgp.Record(edges=list(e for _, _, e in nx.edge_bfs(
MemgraphMultiDiGraph(ctx=ctx), source=source,
orientation=orientation)))
# networkx.algorithms.traversal.edgedfs.edge_dfs
@mgp.read_proc
def edge_dfs(ctx: mgp.ProcCtx,
source: mgp.Nullable[mgp.Vertex] = None,
orientation: mgp.Nullable[str] = None
) -> mgp.Record(edges=mgp.List[mgp.Edge]):
return mgp.Record(edges=list(e for _, _, e in nx.edge_dfs(
MemgraphMultiDiGraph(ctx=ctx), source=source,
orientation=orientation)))
# networkx.algorithms.tree.recognition.is_tree
@mgp.read_proc
def is_tree(ctx: mgp.ProcCtx) -> mgp.Record(is_tree=bool):
return mgp.Record(is_tree=nx.is_tree(MemgraphDiGraph(ctx=ctx)))
# networkx.algorithms.tree.recognition.is_forest
@mgp.read_proc
def is_forest(ctx: mgp.ProcCtx) -> mgp.Record(is_forest=bool):
return mgp.Record(is_forest=nx.is_forest(MemgraphDiGraph(ctx=ctx)))
# networkx.algorithms.tree.recognition.is_arborescence
@mgp.read_proc
def is_arborescence(ctx: mgp.ProcCtx) -> mgp.Record(is_arborescence=bool):
return mgp.Record(is_arborescence=nx.is_arborescence(
MemgraphDiGraph(ctx=ctx)))
# networkx.algorithms.tree.recognition.is_branching
@mgp.read_proc
def is_branching(ctx: mgp.ProcCtx) -> mgp.Record(is_branching=bool):
return mgp.Record(is_branching=nx.is_branching(MemgraphDiGraph(ctx=ctx)))
# networkx.algorithms.tree.mst.minimum_spanning_tree
@mgp.read_proc
def minimum_spanning_tree(ctx: mgp.ProcCtx,
weight: str = 'weight',
algorithm: str = 'kruskal',
ignore_nan: bool = False
) -> mgp.Record(nodes=mgp.List[mgp.Vertex],
edges=mgp.List[mgp.Edge]):
gres = nx.minimum_spanning_tree(MemgraphMultiGraph(ctx=ctx),
weight, algorithm, ignore_nan)
return mgp.Record(nodes=list(gres.nodes()),
edges=[e for _, _, e in gres.edges(keys=True)])
# networkx.algorithms.triads.triadic_census
@mgp.read_proc
def triadic_census(ctx: mgp.ProcCtx) -> mgp.Record(triad=str, count=int):
return [mgp.Record(triad=t, count=c)
for t, c in nx.triadic_census(MemgraphDiGraph(ctx=ctx)).items()]
# networkx.algorithms.voronoi.voronoi_cells
@mgp.read_proc
def voronoi_cells(ctx: mgp.ProcCtx,
center_nodes: mgp.List[mgp.Vertex],
weight: str = 'weight'
) -> mgp.Record(center=mgp.Vertex,
cell=mgp.List[mgp.Vertex]):
return [mgp.Record(center=c1, cell=list(c2))
for c1, c2 in nx.voronoi_cells(
MemgraphMultiDiGraph(ctx=ctx), center_nodes,
weight=weight).items()]
# networkx.algorithms.wiener.wiener_index
@mgp.read_proc
def wiener_index(ctx: mgp.ProcCtx, weight: mgp.Nullable[str] = None
) -> mgp.Record(wiener_index=mgp.Number):
return mgp.Record(wiener_index=nx.wiener_index(
MemgraphMultiDiGraph(ctx=ctx), weight=weight))

View File

@@ -1,51 +0,0 @@
import sys
import mgp
try:
import networkx as nx
except ImportError as import_error:
sys.stderr.write(
'\n'
'NOTE: Please install networkx to be able to use wcc module.\n'
'Using Python:\n'
+ sys.version +
'\n')
raise import_error
@mgp.read_proc
def get_components(vertices: mgp.List[mgp.Vertex],
edges: mgp.List[mgp.Edge]
) -> mgp.Record(n_components=int,
components=mgp.List[mgp.List[mgp.Vertex]]):
'''
This procedure finds weakly connected components of a given subgraph of a
directed graph.
The subgraph is defined by a list of vertices and a list edges which are
passed as arguments of the procedure. More precisely, a set of vertices of
a subgraph contains all vertices provided in a list of vertices along with
all vertices that are endpoints of provided edges. Similarly, a set of
edges of a subgraph contains all edges from the list of provided edges.
The procedure returns 2 fields:
* `n_components` is the number of weakly connected components of the
subgraph.
* `components` is a list of weakly connected components. Each component
is given as a list of `mgp.Vertex` objects from that component.
For example, weakly connected components in a subgraph formed from all
vertices labeled `Person` and edges between such vertices can be obtained
using the following openCypher query:
MATCH (n:Person)-[e]->(m:Person)
WITH collect(n) AS nodes, collect(e) AS edges
CALL wcc.get_components(nodes, edges) YIELD *
RETURN n_components, components;
'''
g = nx.DiGraph()
g.add_nodes_from(vertices)
g.add_edges_from([(edge.from_vertex, edge.to_vertex) for edge in edges])
components = [list(wcc) for wcc in nx.weakly_connected_components(g)]
return mgp.Record(n_components=len(components), components=components)

View File

@@ -1,81 +0,0 @@
# Install the license file.
if (MG_ENTERPRISE)
install(FILES ${CMAKE_CURRENT_SOURCE_DIR}/LICENSE_ENTERPRISE.md
DESTINATION share/doc/memgraph RENAME copyright)
else()
install(FILES ${CMAKE_CURRENT_SOURCE_DIR}/LICENSE_COMMUNITY.md
DESTINATION share/doc/memgraph RENAME copyright)
endif()
# Install systemd service (must use absolute path).
install(FILES ${CMAKE_CURRENT_SOURCE_DIR}/memgraph.service
DESTINATION /lib/systemd/system)
# ---- Setup CPack --------
# General setup
set(CPACK_PACKAGE_NAME memgraph)
set(CPACK_PACKAGE_VENDOR "Memgraph Ltd.")
set(CPACK_PACKAGE_DESCRIPTION_SUMMARY
"High performance, in-memory, transactional graph database")
# DEB specific
# Instead of using "name <email>" format, we use "email (name)" to prevent
# errors due to full stop, '.' at the end of "Ltd". (See: RFC 822)
set(CPACK_DEBIAN_PACKAGE_MAINTAINER "tech@memgraph.com (Memgraph Ltd.)")
set(CPACK_DEBIAN_PACKAGE_SECTION non-free/database)
set(CPACK_DEBIAN_PACKAGE_HOMEPAGE https://memgraph.com)
set(CPACK_DEBIAN_PACKAGE_VERSION "${MEMGRAPH_VERSION_DEB}")
set(CPACK_DEBIAN_FILE_NAME "memgraph_${MEMGRAPH_VERSION_DEB}_amd64.deb")
if (MG_ENTERPRISE)
set(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA
"${CMAKE_CURRENT_SOURCE_DIR}/debian/enterprise/conffiles;"
"${CMAKE_CURRENT_SOURCE_DIR}/debian/enterprise/copyright;"
"${CMAKE_CURRENT_SOURCE_DIR}/debian/preinst;"
"${CMAKE_CURRENT_SOURCE_DIR}/debian/prerm;"
"${CMAKE_CURRENT_SOURCE_DIR}/debian/postrm;"
"${CMAKE_CURRENT_SOURCE_DIR}/debian/postinst;")
else()
set(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA
"${CMAKE_CURRENT_SOURCE_DIR}/debian/community/conffiles;"
"${CMAKE_CURRENT_SOURCE_DIR}/debian/community/copyright;"
"${CMAKE_CURRENT_SOURCE_DIR}/debian/preinst;"
"${CMAKE_CURRENT_SOURCE_DIR}/debian/prerm;"
"${CMAKE_CURRENT_SOURCE_DIR}/debian/postrm;"
"${CMAKE_CURRENT_SOURCE_DIR}/debian/postinst;")
endif()
set(CPACK_DEBIAN_PACKAGE_SHLIBDEPS ON)
# Description formatting is important, summary must be followed with a newline and 1 space.
set(CPACK_DEBIAN_PACKAGE_DESCRIPTION "${CPACK_PACKAGE_DESCRIPTION_SUMMARY}
Contains Memgraph, the graph database. It aims to deliver developers the
speed, simplicity and scale required to build the next generation of
applications driver by real-time connected data.")
# Add `openssl` package to dependencies list. Used to generate SSL certificates.
# We also depend on `python3` because we embed it in Memgraph.
set(CPACK_DEBIAN_PACKAGE_DEPENDS "openssl (>= 1.1.0), python3 (>= 3.5.0)")
# RPM specific
set(CPACK_RPM_PACKAGE_URL https://memgraph.com)
set(CPACK_RPM_PACKAGE_VERSION "${MEMGRAPH_VERSION_RPM}")
set(CPACK_RPM_FILE_NAME "memgraph-${MEMGRAPH_VERSION_RPM}-1.x86_64.rpm")
set(CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION
/var /var/lib /var/log /etc/logrotate.d
/lib /lib/systemd /lib/systemd/system /lib/systemd/system/memgraph.service)
set(CPACK_RPM_PACKAGE_REQUIRES_PRE "shadow-utils")
if (MG_ENTERPRISE)
set(CPACK_RPM_USER_BINARY_SPECFILE "${CMAKE_CURRENT_SOURCE_DIR}/rpm/enterprise/memgraph.spec.in")
set(CPACK_RPM_PACKAGE_LICENSE "Memgraph Enterprise Trial License")
else()
set(CPACK_RPM_USER_BINARY_SPECFILE "${CMAKE_CURRENT_SOURCE_DIR}/rpm/community/memgraph.spec.in")
set(CPACK_RPM_PACKAGE_LICENSE "Memgraph Community License")
endif()
# Description formatting is important, no line must be greater than 80 characters.
set(CPACK_RPM_PACKAGE_DESCRIPTION "Contains Memgraph, the graph database.
It aims to deliver developers the speed, simplicity and scale required to build
the next generation of applications driver by real-time connected data.")
# Add `openssl` package to dependencies list. Used to generate SSL certificates.
# We also depend on `python3` because we embed it in Memgraph.
set(CPACK_RPM_PACKAGE_REQUIRES "openssl >= 1.0.0, curl >= 7.29.0, python3 >= 3.5.0")
# All variables must be set before including.
include(CPack)

View File

@@ -1,98 +0,0 @@
# Memgraph Community User License Agreement
This License Agreement governs your use of the Memgraph Community Release (the
"Software") and documentation ("Documentation").
BY DOWNLOADING AND/OR ACCESSING THIS SOFTWARE, YOU ("LICENSEE") AGREE TO THESE
TERMS.
1. License Grant
The Software and Documentation are provided to Licensee at no charge and are
licensed, not sold to Licensee. No ownership of any part of the Software and
Documentation is hereby transferred to Licensee. Subject to (i) the terms and
conditions of this License Agreement, and (ii) any additional license
restrictions and parameters contained on Licensors quotation, website, or
order form, Licensor hereby grants Licensee a personal, non-assignable,
non-transferable and non-exclusive license to install, access and use the
Software (in object code form only) and Documentation for Licensees internal
business purposes (including for use in a production environment) only. All
rights relating to the Software and Documentation that are not expressly
licensed in this License Agreement, whether now existing or which may hereafter
come into existence are reserved for Licensor. Licensee shall not remove,
obscure, or alter any proprietary rights notices (including without limitation
copyright and trademark notices), which may be affixed to or contained within
the Software or Documentation.
Licensor may terminate this License Agreement with immediate effect upon
written notice to the Licensee. Upon termination Licensee shall delete all
electronic copies of all or any part of the Software and/or the Documentation
resident in its systems or elsewhere.
2. Restrictions
Licensee will not, directly or indirectly, (a) copy the Software or
Documentation in any manner or for any purpose; (b) install, access or use any
component of the Software or Documentation for any purpose not expressly
granted in Section 1 above; (c) resell, distribute, publicly display or
publicly perform the Software or Documentation or any component thereof, by
transfer, lease, loan or any other means, or make it available for use by
others in any time-sharing, service bureau or similar arrangement; (d)
disassemble, decrypt, extract, reverse engineer or reverse compile the
Software, or otherwise attempt to discover the source code, confidential
algorithms or techniques incorporated in the Software; (e) export the Software
or Documentation in violation of any applicable laws or regulations; (f)
modify, translate, adapt, or create derivative works from the Software or
Documentation; (g) circumvent, disable or otherwise interfere with
security-related features of the Software or Documentation; (h) use the
Software or Documentation for any illegal purpose, in any manner that is
inconsistent with the terms of this License Agreement, or to engage in illegal
activity; (i) remove or alter any trademark, logo, copyright or other
proprietary notices, legends, symbols or labels on, or embedded in, the
Software or Documentation; or (j) provide access to the Software or
Documentation to third parties.
3. Warranty Disclaimer
THE SOFTWARE AND DOCUMENTATION ARE PROVIDED "AS IS" AND LICENSOR MAKES NO
WARRANTIES, EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION ANY IMPLIED
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE OR NON
INFRINGEMENT OF THIRD PARTIES INTELLECTUAL PROPERTY RIGHTS OR OTHER
PROPRIETARY RIGHTS. NEITHER THIS LICENSE AGREEMENT NOR ANY DOCUMENTATION
FURNISHED UNDER IT IS INTENDED TO EXPRESS OR IMPLY ANY WARRANTY THAT THE
OPERATION OF THE SOFTWARE WILL BE UNINTERRUPTED, TIMELY, OR ERROR-FREE.
4. Limitation of Liability
Licensor shall not in any circumstances be liable, whether in tort (including
for negligence or breach of statutory duty howsoever arising), contract,
misrepresentation (whether innocent or negligent) or otherwise for: loss of
profits, loss of business, depletion of goodwill or similar losses, loss of
anticipated savings, loss of goods, loss or corruption of data or computer
downtime, or any special, indirect, consequential or pure economic loss, costs,
damages, charges or expenses.
Licensor's total aggregate liability in contract, tort (including without
limitation negligence or breach of statutory duty howsoever arising),
misrepresentation (whether innocent or negligent), restitution or otherwise,
arising in connection with the performance or contemplated performance of this
License Agreement shall in all circumstances be limited to GBP10.00 (ten pounds
sterling).
Nothing in this License Agreement shall limit Licensors liability in the case
of death or personal injury caused by negligence, fraud, or fraudulent
misrepresentation, or where it otherwise cannot be limited by law.
5. Technical Data
Licensor may collect and use technical information (such as usage patterns)
gathered when the Licensee downloads and uses the Software. This is generally
statistical data which does not identify an identified or identifiable
individual. It may also include Licensees IP address which is personal data
and is processed in accordance with our Privacy Policy. We only use this
technical information to improve our products.
6. Law and Jurisdiction
This License Agreement is governed by the laws of England and is subject to the
non-exclusive jurisdiction of the courts of England.

View File

@@ -1,671 +0,0 @@
# Memgraph Enterprise - Software Subscription Agreement
Memgraph Limited is registered in England under registration 10195084 and has
its registered office at Suite 4, Ironstone House, Ironstone Way, Brixworth,
Northampton, NN6 9UD ("Memgraph").
Memgraph agrees to license and/or grant you (the "Customer") access to the
Software, as defined below and provide support and services to you only if you
accept and agree to be bound by the terms and conditions in this Software
Subscription Agreement (the "Agreement"). By signing an Order Document (as
defined below), which is subject to and part of this Agreement, installing and
using the Software or by downloading a trial version of the Software, you agree
to be bound by the terms of this Agreement.
Memgraph Enterprise Trial Users: If you receive free of charge trial access to
the Software, you are deemed a "Customer" for purposes of this Agreement except
that you are subject to the additional restrictions and limitations set forth
in Section 3.2 below as to such Software.
## 1. DEFINITIONS.
1.1. "Applicable Laws" means (i) all applicable laws, statutes and regulations,
and (ii) regulatory policies, guidelines and industry codes (in each case
having the force of law), which apply to the provisions of the Software and
Services and this Agreement.
1.2. "Confidential Information" means all information disclosed
by a party ("Disclosing Party") to the other party ("Receiving Party"), whether
orally or in writing, that is designated as confidential or that reasonably
should be understood to be confidential given the nature of the information and
the circumstances of disclosure. Customer's Confidential Information includes
Customer Data; Memgraph Confidential Information includes the Software and
Services; and Confidential Information of each party includes the terms and
conditions of this Agreement and all Orders (including pricing), as well as
business and marketing plans, technology and technical information, product
plans and designs, and business processes disclosed by such party. However,
Confidential Information does not include any information that (i) is or
becomes generally known to the public without breach of any obligation owed to
the Disclosing Party, (ii) was known to the Receiving Party prior to its
disclosure by the Disclosing Party without breach of any obligation owed to the
Disclosing Party, (iii) is received from a third party without breach of any
obligation owed to the Disclosing Party, or (iv) was independently developed by
the Receiving Party.
1.3. "Customer Data" means business information or other data loaded by or for
Customer and/or processed by the Software.
1.4. "Data Protection Legislation" means all applicable data protection and
privacy legislation in force from time to time in the UK including the General
Data Protection Regulation ((EU) 2016/679); the Data Protection Act 2018; the
Privacy and Electronic Communications Directive 2002/58/EC (as updated by
Directive 2009/136/EC) and the Privacy and Electronic Communications
Regulations 2003 (SI 2003/2426) as amended.
1.5. "Order Document" or "Order" means an order on the Memgraph form of
provisioning document that is submitted by or on behalf of Customer, and
executed by or on behalf of the parties referencing this Agreement and that
specifies the Software and/or Services ordered by Customer, as well as the
specific terms and conditions, for that particular transaction.
1.6. "Services" means those services, including Support Services, which may be
provided to Customer by Memgraph pursuant to the terms of this Agreement and
are expressly limited to those services directly related to Customer's use of
the Software, and expressly exclude any other services.
1.7. "Software" means the Memgraph proprietary downloadable graph database
enterprise software ("Enterprise Software") and the associated technical
documentation located at https://docs.memgraph.com/ ("Documentation"), as well
as software updates, upgrades, bug fixes, or modified versions thereof that
Memgraph licenses or provides to Customer directly or indirectly throughout the
Subscription Term.
1.8. "Subscription Term" means the fixed term, of not less than one (1) year,
designated in an Order Document beginning on the Effective Date and ending at
the end of the period stated therein. If no expiration date is specified in an
Order Document, the Subscription Term shall be a one (1) year period ("Minimum
Subscription Term"). A "Subscription" is the binding, non-cancellable contract
for the use of the Software for the Subscription Term as set forth in an Order
Document.
1.9. "Support" means the support and maintenance services, including any
updates, upgrades, patches, enhancements and bug fixes for the Software that
may be provided to Customer by Memgraph pursuant to the terms of this
Agreement.
1.10. "Users" means employees and Contractors of Customer that Customer has
permitted or authorized to access and use of the Software on Customer's behalf
pursuant to the terms of this Agreement.
## 2. ORDERS, DELIVERY; SUPPORT.
2.1. Delivery. Memgraph will promptly deliver the Software after it accepts an
Order and Customer issues a purchase order referencing such Order. Unless
otherwise stated in an Order, Customer is solely responsible for installing
Software on Customer's own computer equipment. In some instances, Customer's
purchasing relationship exists solely between Customer and an authorised
reseller of Memgraph's Software and Services (a "Reseller"), in which case
Sections 5.1-5.3 (Fees and Payment) will be inapplicable to such Order(s), and
the Reseller shall be responsible for submitting Orders and the appropriate
payment method therewith to Memgraph. An Order is not binding until Memgraph
accepts and countersigns the Order.
2.2 Support. Memgraph will use commercially reasonable efforts to provide
Support to Customer in accordance with Memgraph's then-current terms and
conditions set forth at
https://download.memgraph.com/legal/memgraph-support-terms-and-conditions.pdf
at the support tier stated in the applicable Order. The Support terms and
conditions are subject to change at Memgraph's discretion; however, Memgraph
will not materially reduce the level of Support during a Subscription Term for
which Fees have been paid.
## 3. LICENCE GRANTS; RESTRICTIONS AND PROPRIETARY RIGHTS.
Customer's licence and access rights and benefits and Memgraph's obligations to
Customer will vary depending on the product and the type of licence Memgraph is
granting. If you purchased a licence to Memgraph Software, your licence will be
subject to certain use and/or capacity restrictions, as identified on the
applicable Order Document.
3.1. Enterprise Software Licence. In consideration of the Fees paid hereunder
and subject to the terms of this Agreement and the applicable Order, Memgraph
grants Customer a world-wide, non-exclusive, non-transferable,
non-sublicenseable, and limited licence during the applicable Subscription
Term, to install, and use the Enterprise Software up to the maximum capacity
("Licensed Capacity"), and subject to the usage rules, specified in the
applicable Order Document, and to use Documentation solely for Customer's
internal business purposes in connection with operation of the Enterprise
Software.
3.2. Enterprise Trial Licence. If the Customer downloads, accesses, or uses the
Software under a trial licence ("Trial Licence"), then Customer may use one (1)
copy of the Software in accordance with the terms and conditions of this
Agreement for a thirty (30) day period, or such longer trial period represented
by the applicable licence key issued by or expressly authorised by Memgraph
(the "Trial Period"). Trial Licences are permitted solely for Customer's
evaluation use to determine whether to purchase a Subscription to the Software.
Customer may not use a Trial Licence for any other purpose. At the end of the
Trial Period, the Trial Licence will expire (and this Agreement will terminate
as to such Licence and continue to apply to any subsequent Subscription or use
of the Software; if Customer decides not to obtain a Subscription upon
expiration of the Trial Period, it will promptly cease using and will delete
the Software from its computer systems. Memgraph has the right to terminate a
Trial Licence at any time for any reason.
3.3. NO OBLIGATIONS. NOTWITHSTANDING ANYTHING TO THE CONTRARY IN THIS AGREEMENT
OR IN ANY ORDER DOCUMENT, MEMGRAPH WILL HAVE NO WARRANTY, INDEMNITY, SUPPORT,
OR SERVICE LEVEL, OBLIGATIONS WITH RESPECT TO ANY ENTERPRISE TRIAL, OR OTHER
NO-CHARGE SOFTWARE (INCLUDING TOOLS AND UTILITIES) LICENCES.
3.4. General Restrictions. Customer acknowledges that the Software, and its
structure, organization, and source code, constitute Memgraph's and its
suppliers' valuable trade secrets, and that usage of the Software is subject to
the following restrictions:
3.4.1. Restrictions. Customer agrees not to, and not to authorize any third
party to: (i) allow access or use of the Software by anyone other than its
Users; (ii) distribute, embed, sell, rent, transfer, lease, lend, sublicense,
loan, assign, pledge, grant a security interest in, or otherwise make the
Software accessible or available to any third party; except to the limited
extent expressly provided in Section 3.4.2, use the Software in any
service-bureau, timesharing, outsourcing or similar arrangement; or (iii)
disassemble, decompile, reverse engineer or otherwise attempt to derive the
structure, sequence or organization of source code or APIs; (iv) remove or
alter product identification, copyright, trademark or other proprietary
markings contained in or on the Software; (v) modify, adapt, recast, transform
or otherwise prepare a derivative work of the Software or portion thereof; (vi)
conduct any competitive analysis, publish or share with any third party any
results of any technical evaluation or tests performed on the Software, or
disclose Software features, errors or bugs to a third party without Memgraph's
prior written consent; or (vii) engage in any act designed to circumvent any
restriction set forth in this Agreement, in the Software, or in an Order,
including but not limited to restrictions related to Licensed Capacity.
3.4.2. Internal Use Licences; Users. The Software is licensed for Customer's
internal business use and not for distribution or use by third parties. For
clarity, however, Customer may make available to third parties any
Customer-hosted services or other Customer applications or services that make
use of or incorporate the Software, provided and solely to the extent that (i)
Customer's application or hosted service adds primary and significant
functionality to the Software, (ii) Customer does not embed or otherwise
distribute the Software to third parties, (iii) Customer does not provide third
parties direct access to operate or control the Software itself; and (iv)
Customer at all times remains in compliance with the terms of the applicable
licence grants under this Agreement. Subject to the terms and conditions of
this Agreement, in addition to Customer's employees, Customer may permit its
independent contractors and consultants ("Contractors") to serve as Users.
Customer will remain responsible for compliance by each of its Users (including
but not limited to any Contractor Users) with all of the terms and conditions
of this Agreement, and any use of the Software by any Contractors must be for
the sole benefit of Customer.
3.5. Ownership; Reservation of Rights. This is an agreement for use of Memgraph
Software and not an agreement for sale. Customer acknowledges that it is
obtaining only a limited right to use the Software on a licensed basis, and
that irrespective of any use of the words "purchase", "sale" or like terms
hereunder no ownership rights are being conveyed to Customer. Customer agrees
that Memgraph or its suppliers retain all right, title and interest (including
all patent, copyright, trade secret and other intellectual property rights) in
and to the Memgraph Software. Nothing in this Section shall be deemed as
granting Memgraph ownership of Customer Data or in any way impacting Customer's
ownership of Customer Data.
3.6. Third Party Code. The Software may contain or be provided with components
which are licensed from third parties, including components subject to the
terms and conditions of "open source" software licences ("Open Source
Software"). Open Source Software may be identified in the Software,
Documentation, or in a list of the Open Source Software provided to you upon
your written request. To the extent required by the licence that accompanies
the Open Source Software, the terms of such licence will apply in lieu of the
terms of this Agreement with respect to such Open Source Software, including,
without limitation, any provisions governing access to source code,
modification, or reverse engineering.
3.7. IP Ownership. The Parties agree that neither party shall gain, by virtue
of this Agreement, any rights of ownership or any other interest, right or
title of copyrights, patents, trade secrets, trademarks, or any other
intellectual property rights owned by the other Party. Any and all new works
developed in the course of performing obligations pursuant to this Agreement
and all new inventions, innovations or ideas developed by a Party in the course
of performance of its activities under this Agreement, will belong to that
Party who develops the same. Notwithstanding anything to the contrary in this
Section, the Parties understand and agree that any and all proprietary
materials developed by a Party prior to this Agreement and any modifications,
enhancements, improvements or inventions made to such proprietary materials
shall be owned by that Party, regardless of which Party prepared or developed
such modifications, enhancements, improvements or inventions.
## 4. CUSTOMER DATA; OBLIGATIONS OF CUSTOMER AND MEMGRAPH.
4.1. Customer shall retain all of its rights, title, and interest in and to its
intellectual property rights in Customer Data. Customer grants to Memgraph a
non-exclusive, worldwide, limited-term licence solely to host, copy, transmit
and display Customer Data as reasonably necessary for Memgraph to support
Customer's use of the Software, to ensure the security of and to administrate
the Software, and to deliver Services in accordance with this Agreement or as
otherwise outlined in https://memgraph.com/legal/privacy-policy/.
4.2. Protection of Customer Data. Memgraph will maintain appropriate
administrative, physical, and technical safeguards, consistent with generally
prevailing industry standards, for protection of the security, confidentiality,
and integrity of Customer Data, as described in the Documentation. Those
safeguards will include, but will not be limited to, measures for preventing
access, use, modification, or disclosure of Customer Data by Memgraph
personnel, except as permitted by this Agreement.
4.3. Personal data. Both parties will comply with all applicable requirements
of the Data Protection Legislation. This section 4 is in addition to, and does
not relieve, remove or replace, a party's obligations under the Data Protection
Legislation. Notwithstanding the foregoing, the parties acknowledge that, in
the ordinary course of providing the Services, Memgraph shall not process
personal data (as defined in the Data Protection Legislation) on behalf of the
Customer. In the event that the Customer requires Memgraph to process personal
data on its behalf, it shall notify Memgraph and the parties shall execute such
terms as necessary under applicable law.
## 5. FEES AND PAYMENT.
5.1. Fees. Customer will pay Memgraph the fees for the Licences and Services as
set forth in the applicable Order ("Fees"). Customer acknowledges and agrees
that if Customer's use of the Software exceeds the Licensed Capacity set forth
on the applicable Orders or otherwise requires the payment of additional fees
(per the terms of this Agreement), Customer shall be invoiced for such usage
and Customer agrees to pay the additional fees in accordance with this Section.
Notwithstanding the terms of Section 5.4 below (Reconciliation), Customer
acknowledges and agrees that it is obligated to ensure that its Software usage
does not exceed the Licensed Capacity and to promptly notify Memgraph of any
such excess usage no more than thirty (30) days from the last day of the
calendar month during which such excess usage occurred.
5.2. Payment Terms. Except as otherwise specifically set forth on an Order
Document, all fees are due and payable within thirty days after the date of
invoice. Renewal Fees for any renewal Subscription Term (if purchased by
Customer) will be due and payable within thirty (30) days of expiration of the
then-current term. If Fees are not paid when due, or in the event of other
breach of this Agreement, Customer shall discontinue use of the Software and
Memgraph may suspend its performance, including its delivery of technical
support of the Software or other Services without further notice and without
penalty. All Orders (including multi-year Subscriptions with annual payment
schedules) are non-cancellable and all amounts paid are non-refundable, unless
otherwise expressly set forth herein. Any invoiced amount not received by the
due date will accrue late interest at the rate of 1.5% of the outstanding
balance per month, or the maximum rate permitted by applicable law, whichever
is lower.
5.3. Taxes. Fees are exclusive of taxes. Customer will pay any sales, use,
value added, duties, fees and other governmental assessments or charges arising
out of this Agreement and the transactions contemplated herein. Customer will
make all payments free and clear of, and without reduction for, any withholding
taxes.
5.4. Reconciliation. At Memgraph's request from time to time, not exceeding
once per quarter, Customer will provide Memgraph with a report detailing its
use of the Software, including its non-production and/or production use and
using the self-monitoring capabilities of the Software or other means, and
Memgraph may inspect Customer's records related to such report not more
frequently than annually to ensure payment of Fees. Any on-site review will be
conducted during regular business hours at Customer's offices. The parties will
use reasonable good faith efforts to promptly resolve any discrepancies between
licensed usage and actual usage.
## 6. APPLICABLE LAWS.
6.1. Each Party shall perform this Agreement in accordance with all Applicable
Laws. Without prejudice to the foregoing, each Party shall:
6.1.1. comply with all Applicable Laws relating to anti-bribery,
anti-corruption, anti-slavery and human trafficking, including the Bribery Act
2010 and the Modern Slavery Act 2015 (the "Relevant Requirements");
6.1.2. have and maintain in place throughout the Term its own policies and
procedures, including adequate procedures under the Bribery Act 2010, to ensure
compliance with the Relevant Requirements, and will enforce them where
appropriate;
6.1.3. (if not prohibited by law or regulation from doing so) promptly report
to the other Party any request or demand for any undue financial or other
advantage of any kind received by the reporting Party in connection with the
performance of this Agreement; and
6.1.4. (if not prohibited by law or regulation from doing so) notify the other
Party (and email shall be sufficient for this purpose) as soon as it becomes
aware of any actual or suspected slavery or human trafficking in a supply chain
which has a connection with this Agreement.
## 7. REPRESENTATIONS AND WARRANTIES.
7.1. Mutual Representations and Warranties. Each Party represents and warrants
to the other that: (i) it is a corporation lawfully incorporated and validly
existing pursuant to the laws of its place of incorporation; (ii) it has all
requisite power and authority, corporate or otherwise, to execute, deliver and
perform its obligations under this Agreement; and (iii) this Agreement
constitutes its legal, valid and binding obligations and may be enforced
against it.
7.2. Limited Memgraph Warranty; Remedies. Memgraph warrants that the Software,
when used as permitted hereunder, will operate in all material respects as
described in the applicable Documentation, and that Services will be provided
in a professional manner consistent with industry standards.
7.3. Limitations; Remedy. Memgraph does not warrant that the Software or the
Services will be error-free, uninterrupted or meet Customer's specific
requirements or that performance of the Services will be uninterrupted.
Memgraph will have no warranty obligation under Section 7.2 for Customer's
misuse or failure to use the Software in accordance with its Documentation or
this Agreement. Customer's sole and exclusive remedy, and Memgraph's sole and
exclusive obligation, for breach of warranty will be (i) during the thirty (30)
day period following initial Delivery of the Software under an Order,
Memgraph's correction of the program errors that cause the breach of warranty,
or if Memgraph cannot substantially correct such breach in a commercially
reasonable manner, a refund of the fees paid for the nonconforming Software
(ii) during the remainder of the relevant Subscription Term, Memgraph's
delivery of Support with respect to any such program errors. In the event of a
refund remedy, Customer's licences and right to use the Software or receive
Services will end. In the event of any noticed breach of warranty with respect
to Services, Memgraph's sole and exclusive obligation shall be the
re-performance of the deficient Services.
7.4. Disclaimer. THIS SECTION 7 IS A LIMITED WARRANTY AND, EXCEPT EXPRESSLY AS
SET FORTH IN SECTION 7.2, THE SOFTWARE, INCLUDING WITHOUT LIMITATION THE
THIRD-PARTY CODE, AND ALL SERVICES ARE PROVIDED "AS IS". MEMGRAPH MAKES NO
OTHER WARRANTIES OR REPRESENTATIONS, EXPRESS, IMPLIED, STATUTORY OR OTHERWISE,
AND DISCLAIMS ALL IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE, TITLE, AND NON-INFRINGEMENT.
## 8. INDEMNIFICATION.
8.1. By Memgraph. Memgraph will defend against any action against Customer
brought by a third party to the extent the action is based on a claim that the
Software infringes a third party's patent, copyright or trademark (a "Claim")
and indemnify Customer from the damages, liabilities, costs and expenses
(including reasonable attorneys' fees) awarded against Customer or agreed in
settlement by Customer resulting from such Claim. If your use of the Software
is (or in Memgraph's opinion likely to be) enjoined, then Memgraph may, at its
own expense and at its option: (i) substitute substantially similar
functionality for the Software which renders it non-infringing; (ii) procure
for Customer the right to continue to use the Software; or if (i) and (ii) are
not commercially reasonable, terminate this Agreement and refund Customer any
prepaid, unused (pro-rated) Fees for the duration of the then-current
Subscription Term. The foregoing obligations of Memgraph will not apply: (i) if
the Software is modified by any party other than Memgraph, but solely to the
extent the alleged infringement is caused by such modification; (ii) if the
Software is used in combination with other products or processes not provided
or authorized by Memgraph, but solely to the extent the alleged infringement is
caused by such combination; (iii) use of any version or release of Software
other than the most current version or release made available to Customer by
Memgraph, if its use would have avoided the infringement; (iv) any unauthorized
use of the Software. THIS SECTION 8.1 SETS FORTH MEMGRAPH'S SOLE LIABILITY AND
CUSTOMER'S SOLE AND EXCLUSIVE REMEDY WITH RESPECT TO ANY CLAIM OF INTELLECTUAL
PROPERTY INFRINGEMENT.
8.2. By Customer. Customer will indemnify and hold Memgraph and its suppliers
harmless against any claims, liabilities, costs, and expenses (including
reasonable attorneys' fees) that Memgraph or its suppliers may incur as a
result of a third-party claim arising from or related to Customer Data, or
misuse or unauthorised use of the Software by Customer or any User.
8.3. Conditions. All defence and indemnity obligations under Sections 8.1 and
8.2 are conditioned on the indemnitee (i) giving the indemnitor written notice
of the relevant claim within thirty (30) days after the indemnitee receives
notice of the Claim (or sooner if required by applicable law); (ii) reasonably
cooperating with the indemnitor, at the indemnitor's expense, in the defence of
the claim; and (iii) giving the indemnitor sole control of the defence and any
settlement negotiations. The indemnitee may participate in the defence at its
expense.
## 9. LIMITATION OF LIABILITY.
9.1. TO THE EXTENT PERMITTED BY LAW, NEITHER MEMGRAPH NOR CUSTOMER SHALL BE
LIABLE TO THE OTHER OR ANY THIRD PARTY FOR LOST PROFITS (WHETHER DIRECT OR
INDIRECT) OR LOSS OF USE OR DATA, SUBSTITUTE GOODS OR SERVICES, OR FOR
INCIDENTAL, CONSEQUENTIAL, PUNITIVE, SPECIAL OR EXEMPLARY DAMAGES (INCLUDING
DAMAGE TO BUSINESS, REPUTATION OR GOODWILL), OR INDIRECT DAMAGES OF ANY TYPE
HOWEVER CAUSED, WHETHER BY BREACH OF WARRANTY, BREACH OF CONTRACT, IN TORT
(INCLUDING NEGLIGENCE) OR ANY OTHER LEGAL OR EQUITABLE CAUSE OF ACTION EVEN IF
SUCH PARTY HAS BEEN ADVISED OF SUCH DAMAGES IN ADVANCE OR IF SUCH DAMAGES WERE
FORESEEABLE.
9.2. LIMITATIONS ON DIRECT DAMAGES. EXCEPT FOR ANY EXCLUDED CLAIMS AND ANY
DAMAGES THAT CANNOT BE LIMITED UNDER APPLICABLE LAW, IN NO EVENT WILL
MEMGRAPH'S TOTAL AGGREGATE LIABILITY ARISING FROM OR RELATED TO THIS AGREEMENT,
EXCEED AN AMOUNT EQUAL TO THE TOTAL AMOUNT OF FEES PAID OR PAYABLE BY CUSTOMER
TO MEMGRAPH UNDER THIS AGREEMENT DURING THE TWELVE MONTHS PRECEDING THE CLAIM.
THE FOREGOING LIMITATIONS SHALL NOT APPLY TO: (i) PAYMENTS TO A THIRD PARTY
ARISING FROM A PARTY'S OBLIGATIONS UNDER SECTION 8 (INDEMNIFICATION); (ii)
BREACH BY A PARTY OF SECTION 10 (CONFIDENTIAL INFORMATION), AND (iii)
INFRINGEMENT BY A PARTY OF THE OTHER PARTY'S INTELLECTUAL PROPERTY RIGHTS
(COLLECTIVELY EXCLUSIONS (i)-(iii) ARE REFERRED TO AS THE "EXCLUDED CLAIMS").
WITH RESPECT TO ANY EXCLUDED CLAIMS, MEMGRAPH'S TOTAL AGGREGATE LIABILITY SHALL
IN NO EVENT EXCEED £1,000,000.
9.3. Nothing in this Agreement excludes the liability of Memgraph for death or
personal injury caused by the Supplier's negligence; or for fraud or fraudulent
misrepresentation.
## 10. CONFIDENTIALITY.
10.1. The Receiving Party will use the same degree of care that it uses to
protect the confidentiality of its own confidential information of like kind
(but not less than reasonable care) to (i) not use any confidential information
of the Disclosing Party for any purpose outside the scope of this agreement and
(ii) except as otherwise authorized by the Disclosing Party in writing, limit
access to confidential information of the Disclosing Party to those of its and
its affiliates' employees and contractors who need that access for purposes
consistent with this agreement and who have signed confidentiality agreements
with the receiving party containing protections not materially less protective
of the confidential information than those herein. Neither party will disclose
the terms of this agreement or any Orders to any third-party other than its
affiliates, legal counsel, and accountants without the other party's prior
written consent, provided that a party that makes any such disclosure to its
affiliate, legal counsel or accountants will remain responsible for such
affiliate's, legal counsel's, or accountant's compliance with this
"confidentiality" section.
10.2. Compelled Disclosure. The Receiving Party may disclose confidential
information of the Disclosing Party to the extent compelled by law to do so,
provided the Receiving Party gives the Disclosing Party prior notice of the
compelled disclosure (to the extent legally permitted) and reasonable
assistance, at the Disclosing Party's cost, if the Disclosing Party wishes to
contest the disclosure. If the receiving party is compelled by law to disclose
the Disclosing Party's confidential information as part of a civil proceeding
to which the Disclosing Party is a party, and the Disclosing Party is not
contesting the disclosure, the Disclosing Party will reimburse the Receiving
Party for its reasonable cost of complying and providing secure access to that
confidential information.
## 11. TERMINATION.
11.1. Term. The term ("Term") of this Agreement will commence on the Effective
Date and continue until all Subscriptions, licence terms and Orders expire,
unless earlier terminated in accordance with this Section 11.
11.2. Termination for Cause. In the event of a material breach of this
Agreement (excluding any breaches for which an exclusive remedy is expressly
provided), the non-breaching party may terminate this Agreement if such breach
is not cured within thirty (30) days after written notice thereof (except that
for a breach of Section 3.4 ("General Restrictions"), there will be no cure
period). For clarity, material breach of this Agreement includes, but is not
limited to, failure to timely pay amounts due hereunder, exceeding the scope of
any Licence granted hereunder (including the Licensed Capacity), violating the
Licence restrictions, breach of Section 6.1 and failing to protect the other
party's Confidential Information.
11.3. Without affecting any other right or remedy available to it, either party
may terminate this Agreement with immediate effect by giving written notice to
the other party if the other party:
11.3.1. suspends, or threatens to suspend, payment of its debts or is unable to
pay its debts as they fall due or admits inability to pay its debts or is
deemed unable to pay its debts within the meaning of section 123 of the
Insolvency Act 1986, as if the words "it is proved to the satisfaction of the
court" did not appear in sections 123(1)(e) or 123(2) of the Insolvency Act
1986; or
11.3.2. the other party commences negotiations with all or any class of its
creditors with a view to rescheduling any of its debts, or makes a proposal for
or enters into any compromise or arrangement with its creditors other than for
the sole purpose of a scheme for a solvent amalgamation of that other party
with one or more other companies or the solvent reconstruction of that other
party; or
11.3.3. a petition is filed, a notice is given, a resolution is passed, or an
order is made, for or in connection with the winding up of that other party
other than for the sole purpose of a scheme for a solvent amalgamation of that
other party with one or more other companies or the solvent reconstruction of
that other party; or
11.3.4. an application is made to court, or an order is made, for the
appointment of an administrator, or if a notice of intention to appoint an
administrator is given or if an administrator is appointed, over the other
party; or
11.3.5. the holder of a qualifying floating charge over the assets of that
other party has become entitled to appoint or has appointed an administrative
receiver; or
11.3.6. a person becomes entitled to appoint a receiver over the assets of the
other party or a receiver is appointed over the assets of the other party; or
11.3.7. a creditor or encumbrancer of the other party attaches or takes
possession of, or a distress, execution, sequestration or other such process is
levied or enforced on or sued against, the whole or any part of the other
party's assets and such attachment or process is not discharged within 30 days;
or
11.3.8. any event occurs, or proceeding is taken, with respect to the other
party in any jurisdiction to which it is subject that has an effect equivalent
or similar to any of the events mentioned in clause 11.3.1 to clause 11.3.7
(inclusive); or
11.3.9. the other party suspends or ceases, or threatens to suspend or cease,
carrying on all or a substantial part of its business.
11.4. Effect of Termination. Upon the termination of this Agreement: (i) all
licences will terminate; (ii) Customer will immediately discontinue all use of
the affected Software, erase all full and partial copies of the Software and
(subject to this Section) all other tangible embodiments of Memgraph
Confidential Information in Customer's possession or control, and promptly
certify the same to Memgraph; (iii) Memgraph may immediately cease providing
the Services; (iv) (subject to this Section), Memgraph will return or delete
all tangible embodiments of Customer Confidential Information in Memgraph's
possession or control; and (v) Sections 1 ("Definitions"), 3.4 ("General
Restrictions"), 3.5 ("Ownership; Reservation of Rights"), 5 ("Fees and
Payment"), 7.3 ("Limitations"), 7.4 ("Disclaimer"), 8 ("Indemnification"), 9
("Limitation of Liability"), 10 ("Confidentiality"), 11.4 ("Effect of
Termination"), and 12 ("Miscellaneous") will survive. If a party's file
retention policies or a valid legal order provides for backup or archival
copies of files to be retained, such party will notify the other party of such
policy or order, protect the other party's Confidential Information as required
hereunder, and permanently erase, delete, or destroy such Confidential
Information as soon as permissible under such policy or order.
## 12. MISCELLANEOUS.
12.1. Assignment. This Agreement will bind and inure to the benefit of each
party's permitted successors and assigns. Memgraph may assign this Agreement to
any affiliate or in connection with a merger, reorganization, acquisition, or
other transfer of all or substantially all of Memgraph's assets or voting
securities. Customer may not assign or transfer this Agreement, in whole or in
part, without Memgraph's written consent except that Customer may assign its
rights and obligations under this Agreement, in whole but not in part, without
Memgraph's written consent in connection with any merger, consolidation, sale
of all or substantially all of Customer's assets or voting stock, or any other
similar transaction provided that: (i) the assignee is not a direct competitor
of Memgraph; (ii) Customer provides prompt written notice of such assignment to
Memgraph; (iii) the assignee is capable of fully performing Customer's
obligations under this Agreement; and (iv) the assignee agrees to be bound by
the terms and conditions of this Agreement. Any attempt to transfer or assign
this Agreement without such written consent will be null and void.
12.2. Force Majeure. Memgraph shall have no liability to the Customer under
this Agreement if it is prevented from or delayed in performing its obligations
under this Agreement, or from carrying on its business, by acts, events,
omissions or accidents beyond its reasonable control, including strikes,
lock-outs or other industrial disputes (whether involving the workforce of
Memgraph or any other party), failure of a utility service or transport or
telecommunications network, act of God, war, pandemic, riot, civil commotion,
malicious damage, compliance with any law or governmental order, rule,
regulation or direction, accident, breakdown of plant or machinery, fire,
flood, storm or default of suppliers or subcontractors, provided that the
Customer is notified of such an event and its expected duration.
12.3. Governing Law. This Agreement and any dispute or claim arising out of or
in connection with it or its subject matter or formation (including
non-contractual disputes or claims) shall be governed by and construed in
accordance with the law of England and Wales.
12.4. Jurisdiction. Each party irrevocably agrees that the courts of England
and Wales shall have exclusive jurisdiction to settle any dispute or claim
arising out of or in connection with this Agreement or its subject matter or
formation (including non-contractual disputes or claims).
12.5. Severability; Waiver; Construction. If a court of competent jurisdiction
adjudges any provision of this Agreement to be invalid or unenforceable, the
remaining provisions of this Agreement, if capable of substantial performance,
will continue in full force and effect without being impaired or invalidated in
any way. The parties agree to replace any invalid provision with a valid
provision that most closely approximates the intent and economic effect of the
invalid provision. All waivers must be in writing. A party's consent to, or
waiver of, enforcement of this Agreement on one occasion will not be deemed a
waiver of any other provision or such provision on any other occasion. In this
Agreement, the word "including" means "including but not limited to." No
presumption will operate in favour of or against any party as a result of its
role in drafting this Agreement.
12.6. Subcontractors. Memgraph may use the services of subcontractors in
connection with its performance of this Agreement, provided that Memgraph
remains solely responsible for (i) compliance of any such subcontractor with
the terms of this Agreement and (ii) the overall performance of Memgraph as
required under this Agreement.
12.7. Use of Aggregate Data. Customer agrees that Memgraph may collect, use and
disclose quantitative data and metadata derived from the use of the Software
(i) for its own internal, statistical analysis, (ii) to develop and improve the
Software and (iii) to create and distribute reports and other materials
regarding use of the Software. For clarity, any such data collected, used, and
disclosed will be in anonymized aggregate form only and shall not identify
Customer or its Users, or disclose any Customer Data.
12.8. Independent Contractors. The parties are independent contractors. No
agency, partnership, franchise, joint venture, or employment relationship is
intended or created by this Agreement. Neither party has the power or authority
to create or assume any obligation, or make any representations or warranties,
on behalf of the other party.
12.9. Publicity. Memgraph may, in conformity with Customer's trademark usage
guidelines, use Customer's name and logo in Memgraph's sales and marketing
materials, including in business presentations, Customer lists, and on
websites. Neither party will issue a press release regarding this Agreement
without the other party's prior written consent. Neither party will disclose
the terms of this Agreement to any third party, except as required by law.
12.10. Notice. Any notice, consent, or waiver hereunder must be in writing,
addressed to the attention of "Legal Department" at the address set forth
above, and delivered by personal delivery, reputable rapid courier, or
certified/registered mail, return receipt requested, and will be deemed given
upon personal delivery, one (1) day after deposit with an overnight domestic
courier, two (2) days after deposit with an international courier, or five (5)
days after deposit in the certified or registered mail. A party may specify a
new address by providing notice to the other party in accordance with this
Section.
12.11. Compliance with Law. Each party will comply with all applicable laws,
regulations, and orders of any governmental authority of competent jurisdiction
in its performance under this Agreement, including but not limited to those
applicable to data collection and the privacy and security of personal
information, including trans-border data transfers and data breach notification
requirements as required of each party by law.
12.12. Supremacy; Modification. This Agreement will prevail over any written
instrument submitted by Customer; the terms of any purchase order,
acknowledgement, or similar document submitted by Customer to Memgraph will
have no effect. If the express terms of an Order Document conflict with this
Agreement, the terms on the Order Document will prevail, but only with respect
to that Order Document. This Agreement cannot be varied or supplemented by
course of dealing or by usage of trade. All modifications or amendments to this
Agreement must be in writing and signed by both parties, except that subsequent
renewals and purchases of additional Licensed Capacity can be procured by
payment against an issued invoice as set forth in Section 5 ("Fees and
payment") above.
12.13. No Third Party Beneficiaries. This Agreement is not intended and shall
not be construed to give any third party any interest or rights with respect to
or in connection with any agreement or provision herein, except as expressly
provided for in this Agreement.
12.14. Entire Agreement. This Agreement in its original English text, sets
forth the complete, exclusive, and final agreement of the parties concerning
the subject matter hereof, supersedes, replaces, and merges all prior and
contemporaneous agreements, communications, and understandings, both written
and oral, between them concerning the subject matter hereof. This Agreement may
be executed in counterparts.

View File

@@ -1,4 +0,0 @@
# Memgraph Release
Please continue in
[Notion](https://www.notion.so/memgraph/Release-621733518c82456ebc119d2a025b5dd3).

View File

@@ -1,47 +0,0 @@
# Maintainer: tech@memgraph.com (Memgraph Ltd.)
pkgname=memgraph
pkgrel=1
epoch=
# TODO: Maybe take pkgdesc from CMake?
pkgdesc="High performance, in-memory, transactional graph database"
# TODO: Autogenerate architecture? Though, we only support x86_64...
arch=('x86_64')
url="https://memgraph.com"
license=('custom')
groups=()
depends=('gcc-libs')
makedepends=()
checkdepends=()
optdepends=()
provides=()
conflicts=()
replaces=()
backup=("etc/memgraph/memgraph.conf" "etc/logrotate.d/memgraph")
options=()
install=memgraph.install
changelog=
source=("$pkgname-$pkgver.tar.gz")
noextract=()
validpgpkeys=()
package() {
cd $(find . -maxdepth 1 -type d -name 'memgraph*')
# By default, we install the systemd service in /lib (as expected on Debian),
# so move anything like that in /usr/lib.
if [[ -d "lib" ]]; then
mkdir -p usr/lib
cp -a lib/* usr/lib
rm -rf lib
fi
# In case the binary package is built with /usr/local prefix instead of /usr.
if [[ -d "usr/local" ]]; then
mkdir -p usr
cp -a usr/local/* usr
rm -rf usr/local
fi
# Move the license to Arch specific location.
install -Dm644 usr/share/doc/memgraph/copyright usr/share/licenses/memgraph/LICENSE
# We currently don't have anything in usr/share/doc/memgraph
rm -rf usr/share/doc/memgraph
cp -a . $pkgdir
}

View File

@@ -1,16 +0,0 @@
post_install() {
# Add the 'memgraph' user and group and set permissions on
# 'var/*/memgraph' directories.
getent group memgraph >/dev/null || groupadd -r memgraph || exit 1
getent passwd memgraph >/dev/null || \
useradd -r -g memgraph -d /var/lib/memgraph memgraph || exit 1
chown memgraph:memgraph /var/lib/memgraph || exit 1
chmod 750 /var/lib/memgraph || exit 1
chown memgraph:adm /var/log/memgraph || exit 1
chmod 750 /var/log/memgraph || exit 1
echo "Enable and start 'memgraph.service' to use Memgraph" || exit 1
}
pre_remove() {
systemctl disable memgraph.service
}

View File

@@ -1,48 +0,0 @@
#!/bin/bash -e
function print_help () {
echo "Usage: $0 MEMGRAPH_PACKAGE.tar.gz"
echo "Optional arguments:"
echo -e " -h|--help Print help."
}
if [[ $# -ne 1 || "$1" == "-h" || "$1" == "--help" ]]; then
print_help
exit 1
fi
if [[ ! -f "$1" ]]; then
echo "File '$1' does not exist!"
exit 1
fi
# Extract version from .tar.gz name
tgz_name=`echo $(basename $1) | sed 's/.tar.gz//'`
version=`echo ${tgz_name} | sed 's/.*[-_]\(.*\)-.*/\1/'`
script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
rm -rf ${script_dir}/_pack
mkdir -p ${script_dir}/_pack
# Copy the .tar.gz to packaging directory.
cp "$1" ${script_dir}/_pack/memgraph-${version}.tar.gz
cd ${script_dir}/_pack
# Setup PKGBUILD.
echo "pkgver=${version}" > PKGBUILD
cat ../PKGBUILD.proto >> PKGBUILD
# Copy the installation script.
cp ../memgraph.install ./
# Check PKGBUILD validity
updpkgsums PKGBUILD
namcap PKGBUILD
# TODO: Maybe add a custom makepkg.conf and use that
makepkg PACKAGER="tech@memgraph.com (Memgraph Ltd.)"
# Check the final package archive validity and move it in parent directory.
pkg_name=memgraph-${version}-1-x86_64.pkg.tar.xz
namcap --exclude=emptydir $pkg_name
cp $pkg_name ../
echo "Built Arch Package at '${script_dir}/${pkg_name}'"
rm -rf ${script_dir}/_pack

View File

@@ -1,2 +0,0 @@
/etc/memgraph/memgraph.conf
/etc/logrotate.d/memgraph

View File

@@ -1 +0,0 @@
../../LICENSE_COMMUNITY.md

View File

@@ -1,3 +0,0 @@
/etc/memgraph/memgraph.conf
/etc/memgraph/auth_module/ldap.example.yaml
/etc/logrotate.d/memgraph

View File

@@ -1 +0,0 @@
../../LICENSE_ENTERPRISE.md

View File

@@ -1,78 +0,0 @@
#!/bin/sh
# postinst script for memgraph
#
# see: dh_installdeb(1)
set -e
# summary of how this script can be called:
# * <postinst> `configure' <most-recently-configured-version>
# * <old-postinst> `abort-upgrade' <new version>
# * <conflictor's-postinst> `abort-remove' `in-favour' <package>
# <new-version>
# * <postinst> `abort-remove'
# * <deconfigured's-postinst> `abort-deconfigure' `in-favour'
# <failed-install-package> <version> `removing'
# <conflicting-package> <version>
# for details, see https://www.debian.org/doc/debian-policy/ or
# the debian-policy package
case "$1" in
configure)
# Add the 'memgraph' user and group and set permissions on
# 'var/*/memgraph' directories.
adduser --quiet --system --group --home /var/lib/memgraph --no-create-home --shell /bin/bash memgraph || exit 1
echo "Don't forget to switch to the 'memgraph' user to use Memgraph" || exit 1
chown memgraph:memgraph /var/lib/memgraph || exit 1
chmod 750 /var/lib/memgraph || exit 1
chown memgraph:adm /var/log/memgraph || exit 1
chmod 750 /var/log/memgraph || exit 1
# Generate SSL certificates
if [ ! -d /etc/memgraph/ssl ]; then
mkdir /etc/memgraph/ssl || exit 1
openssl req -x509 -newkey rsa:4096 -days 3650 -nodes \
-keyout /etc/memgraph/ssl/key.pem -out /etc/memgraph/ssl/cert.pem \
-subj "/C=GB/ST=London/L=London/O=Memgraph Ltd./CN=Memgraph DB" || exit 1
chown memgraph:memgraph /etc/memgraph/ssl/* || exit 1
chmod 400 /etc/memgraph/ssl/* || exit 1
fi
;;
abort-upgrade|abort-remove|abort-deconfigure)
;;
*)
echo "postinst called with unknown argument \`$1'" >&2
exit 1
;;
esac
# Always setup the systemd memgraph.service. The following was autogenerated
# by dh_systemd_enable and dh_systemd_start, so it should behave as expected.
# This will only remove masks created by d-s-h on package removal.
deb-systemd-helper unmask memgraph.service >/dev/null || true
# was-enabled defaults to true, so new installations run enable.
if deb-systemd-helper --quiet was-enabled memgraph.service; then
# Enables the unit on first installation, creates new
# symlinks on upgrades if the unit file has changed.
deb-systemd-helper enable memgraph.service >/dev/null || true
else
# Update the statefile to add new symlinks (if any), which need to be
# cleaned up on purge. Also remove old symlinks.
deb-systemd-helper update-state memgraph.service >/dev/null || true
fi
if [ -d /run/systemd/system ]; then
systemctl --system daemon-reload >/dev/null || true
deb-systemd-invoke start memgraph.service >/dev/null || true
fi
# Take a look at the preinst script for the detailed explanation.
if dpkg-maintscript-helper supports rm_conffile 2>/dev/null; then
dpkg-maintscript-helper rm_conffile /etc/logrotate.d/memgraph_audit 1.1.999 -- "$@"
fi
exit 0

View File

@@ -1,73 +0,0 @@
#!/bin/sh
# postrm script for memgraph
#
# see: dh_installdeb(1)
set -e
# summary of how this script can be called:
# * <postrm> `remove'
# * <postrm> `purge'
# * <old-postrm> `upgrade' <new-version>
# * <new-postrm> `failed-upgrade' <old-version>
# * <new-postrm> `abort-install'
# * <new-postrm> `abort-install' <old-version>
# * <new-postrm> `abort-upgrade' <old-version>
# * <disappearer's-postrm> `disappear' <overwriter>
# <overwriter-version>
# for details, see https://www.debian.org/doc/debian-policy/ or
# the debian-policy package
var_files="/var/lib/memgraph /var/log/memgraph"
case "$1" in
purge)
# Remove 'var/*/memgraph' directories, even if they contain something.
for var_file in $var_files; do
rm -rf $var_file
done
# Remove generated SSL certificates
if [ -d /etc/memgraph/ssl ]; then
rm -rf /etc/memgraph/ssl
fi
# Don't remove the 'memgraph' user, since we cannot be sure whether it
# existed before.
;;
remove|upgrade|failed-upgrade|abort-install|abort-upgrade|disappear)
# Default behaviour does what we expect, removes untouched installed
# files but keeps configuration.
;;
*)
echo "postrm called with unknown argument \`$1'" >&2
exit 1
;;
esac
# Remove and purge systemd memgraph.service. The following was autogenerated
# by dh_systemd_enable and dh_systemd_start.
if [ -d /run/systemd/system ]; then
systemctl --system daemon-reload >/dev/null || true
fi
if [ "$1" = "remove" ]; then
if [ -x "/usr/bin/deb-systemd-helper" ]; then
deb-systemd-helper mask memgraph.service >/dev/null
fi
fi
if [ "$1" = "purge" ]; then
if [ -x "/usr/bin/deb-systemd-helper" ]; then
deb-systemd-helper purge memgraph.service >/dev/null
deb-systemd-helper unmask memgraph.service >/dev/null
fi
fi
# Take a look at the preinst script for the detailed explanation.
if dpkg-maintscript-helper supports rm_conffile 2>/dev/null; then
dpkg-maintscript-helper rm_conffile /etc/logrotate.d/memgraph_audit 1.1.999 -- "$@"
fi
exit 0

View File

@@ -1,19 +0,0 @@
#!/bin/sh
# preinst script for memgraph
#
# see: dh_installdeb(1)
set -e
# Manage (remove) /etc/logrotate.d/memgraph_audit file because the whole
# logrotate config is moved to /etc/logrotate.d/memgraph since v1.2.0.
# Note: Only used to manage Memgraph Enterprise config but packaged into the
# Memgraph Community as well.
if dpkg-maintscript-helper supports rm_conffile 2>/dev/null; then
# 1.1.999 is chosen because it's high enough version number. It's highly
# unlikely (impossible) that the patch number in v1.1 reaches 999
# (it's 0 on 2020-10-17).
# Tested with: `dpkg --compare-versions -- "1.2" le-nl "1.1.999"`
# (used inside dpkg-maintscript-helper script).
dpkg-maintscript-helper rm_conffile /etc/logrotate.d/memgraph_audit 1.1.999 -- "$@"
fi

View File

@@ -1,32 +0,0 @@
#!/bin/sh
# prerm script for memgraph
#
# see: dh_installdeb(1)
set -e
# summary of how this script can be called:
# * <prerm> `remove'
# * <old-prerm> `upgrade' <new-version>
# * <new-prerm> `failed-upgrade' <old-version>
# * <conflictor's-prerm> `remove' `in-favour' <package> <new-version>
# * <deconfigured's-prerm> `deconfigure' `in-favour'
# <package-being-installed> <version> `removing'
# <conflicting-package> <version>
# for details, see https://www.debian.org/doc/debian-policy/ or
# the debian-policy package
case "$1" in
remove|upgrade|deconfigure|failed-upgrade)
if [ -d /run/systemd/system ]; then
deb-systemd-invoke stop memgraph.service >/dev/null
fi
;;
*)
echo "prerm called with unknown argument \`$1'" >&2
exit 1
;;
esac
exit 0

View File

@@ -1,30 +0,0 @@
FROM debian:buster
# NOTE: If you change the base distro update release/package as well.
ARG deb_release
RUN apt-get update && apt-get install -y \
openssl libcurl4 libssl1.1 python3 libpython3.7 python3-pip \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN pip3 install networkx==2.4 numpy==1.19.2 scipy==1.5.2
COPY ${deb_release} /
# Install memgraph package
RUN dpkg -i ${deb_release}
# Memgraph listens for Bolt Protocol on this port by default.
EXPOSE 7687
# Snapshots and logging volumes
VOLUME /var/log/memgraph
VOLUME /var/lib/memgraph
# Configuration volume
VOLUME /etc/memgraph
USER memgraph
WORKDIR /usr/lib/memgraph
ENTRYPOINT ["/usr/lib/memgraph/memgraph"]
CMD [""]

View File

@@ -1,30 +0,0 @@
FROM debian:buster
# NOTE: If you change the base distro update release/package as well.
ARG deb_release
RUN apt-get update && apt-get install -y \
openssl libcurl4 libssl1.1 libseccomp2 python3 libpython3.7 python3-pip \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN pip3 install networkx==2.4 numpy==1.19.2 scipy==1.5.2
COPY ${deb_release} /
# Install memgraph package
RUN dpkg -i ${deb_release}
# Memgraph listens for Bolt Protocol on this port by default.
EXPOSE 7687
# Snapshots and logging volumes
VOLUME /var/log/memgraph
VOLUME /var/lib/memgraph
# Configuration volume
VOLUME /etc/memgraph
USER memgraph
WORKDIR /usr/lib/memgraph
ENTRYPOINT ["/usr/lib/memgraph/memgraph"]
CMD [""]

View File

@@ -1,47 +0,0 @@
#!/bin/bash -e
# Build and package Docker image of Memgraph.
function print_help () {
echo "Usage: $0 [--latest] MEMGRAPH_PACKAGE.deb"
echo "Optional arguments:"
echo -e "\t-h|--help\t\tPrint help."
echo -e "\t--latest\t\tTag image as latest version."
}
working_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
latest_image=""
tag_latest=""
if [[ $# -eq 2 && "$1" == "--latest" ]]; then
latest_image="memgraph:latest"
tag_latest="-t memgraph:latest"
shift
elif [[ $# -ne 1 || "$1" == "-h" || "$1" == "--help" ]]; then
print_help
exit 1
fi
deb_path="$1"
if [[ ! -f "$deb_path" ]]; then
echo "File '$deb_path' does not exist!"
exit 1
fi
# Copy the .deb to working directory.
cp "$deb_path" "${working_dir}/"
cd ${working_dir}
# Extract version and offering from deb name.
deb_name=`echo $(basename "$deb_path") | sed 's/.deb$//'`
version=`echo ${deb_name} | cut -d '_' -f 2 | rev | cut -d '-' -f 2- | rev | tr '+~' '__'`
offering=`echo ${version} | cut -d '-' -f 2`
dockerfile_path="${working_dir}/memgraph_${offering}.dockerfile"
image_name="memgraph:${version}"
package_name="memgraph-${version}-docker.tar.gz"
# Build docker image.
docker build -t ${image_name} ${tag_latest} -f ${dockerfile_path} --build-arg deb_release=${deb_name}.deb .
docker save ${image_name} ${latest_image} > ${package_name}
rm "${deb_name}.deb"
echo "Built Docker image at '${working_dir}/${package_name}'"

View File

@@ -1,266 +0,0 @@
#!/usr/bin/env python3
import argparse
import re
import subprocess
import sys
import os
# This script is used to determine the current version of Memgraph. The script
# determines the current version using `git` automatically. The user can also
# manually specify a version override and then the supplied version will be
# used instead of the version determined by `git`. All versions (automatically
# detected and manually specified) can have a custom version suffix added to
# them.
#
# The current version can be one of either:
# - release version
# - development version
#
# The release version is either associated with a `release/X.Y` branch
# (automatic detection) or is manually specified. When the version is
# automatically detected from the branch a `.0` is appended to the version.
# Example 1:
# - branch: release/0.50
# - version: 0.50.0
# Example 2:
# - manually supplied version: 0.50.1
# - version: 0.50.1
#
# The development version is always determined using `git` in the following
# way:
# - release version - nearest (older) `release/X.Y` branch version
# - distance from the release branch - Z commits
# - the current commit short hash
# Example:
# - release version: 0.50.0 (nearest older branch `release/0.50`)
# - distance from the release branch: 42 (commits)
# - current commit short hash: 7e1eef94
#
# The script then uses the collected information to generate the versions that
# will be used in the binary, DEB package and RPM package. All of the versions
# have different naming conventions that have to be met and they differ among
# each other.
#
# The binary version is determined using the following two templates:
# Release version:
# <VERSION>-<OFFERING>[-<SUFFIX>]
# Development version:
# <VERSION>+<DISTANCE>~<SHORTHASH>-<OFFERING>[-<SUFFIX>]
# Examples:
# Release version:
# 0.50.1-community
# 0.50.1-enterprise
# 0.50.1-enterprise-veryimportantcustomer
# Development version (master, 12 commits after release/0.50):
# 0.50.0+12~7e1eef94-community
# 0.50.0+12~7e1eef94-enterprise
# 0.50.0+12~7e1eef94-enterprise-veryimportantcustomer
#
# The DEB package version is determined using the following two templates:
# Release version:
# <VERSION>-<OFFERING>[-<SUFFIX>]-1
# Development version (master, 12 commits after release/0.50):
# <VERSION>+<DISTANCE>~<SHORTHASH>-<OFFERING>[-<SUFFIX>]-1
# Examples:
# Release version:
# 0.50.1-community-1
# 0.50.1-enterprise-1
# 0.50.1-enterprise-veryimportantcustomer-1
# Development version (master, 12 commits after release/0.50):
# 0.50.0+12~7e1eef94-community-1
# 0.50.0+12~7e1eef94-enterprise-1
# 0.50.0+12~7e1eef94-enterprise-veryimportantcustomer-1
# For more documentation about the DEB package naming conventions see:
# https://www.debian.org/doc/debian-policy/ch-controlfields.html#version
#
# The RPM package version is determined using the following two templates:
# Release version:
# <VERSION>_1.<OFFERING>[.<SUFFIX>]
# Development version:
# <VERSION>_0.<DISTANCE>.<SHORTHASH>.<OFFERING>[.<SUFFIX>]
# Examples:
# Release version:
# 0.50.1_1.community
# 0.50.1_1.enterprise
# 0.50.1_1.enterprise.veryimportantcustomer
# Development version:
# 0.50.0_0.12.7e1eef94.community
# 0.50.0_0.12.7e1eef94.enterprise
# 0.50.0_0.12.7e1eef94.enterprise.veryimportantcustomer
# For more documentation about the RPM package naming conventions see:
# https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/
# https://fedoraproject.org/wiki/Package_Versioning_Examples
def get_output(*cmd, multiple=False):
ret = subprocess.run(cmd, stdout=subprocess.PIPE, check=True)
if multiple:
return list(map(lambda x: x.strip(),
ret.stdout.decode("utf-8").strip().split("\n")))
return ret.stdout.decode("utf-8").strip()
def format_version(variant, version, offering, distance=None, shorthash=None,
suffix=None):
if not distance:
# This is a release version.
if variant == "deb":
# <VERSION>-<OFFERING>[-<SUFFIX>]-1
ret = "{}-{}".format(version, offering)
if suffix:
ret += "-" + suffix
ret += "-1"
return ret
elif variant == "rpm":
# <VERSION>_1.<OFFERING>[.<SUFFIX>]
ret = "{}_1.{}".format(version, offering)
if suffix:
ret += "." + suffix
return ret
else:
# <VERSION>-<OFFERING>[-<SUFFIX>]
ret = "{}-{}".format(version, offering)
if suffix:
ret += "-" + suffix
return ret
else:
# This is a development version.
if variant == "deb":
# <VERSION>+<DISTANCE>~<SHORTHASH>-<OFFERING>[-<SUFFIX>]-1
ret = "{}+{}~{}-{}".format(version, distance, shorthash, offering)
if suffix:
ret += "-" + suffix
ret += "-1"
return ret
elif variant == "rpm":
# <VERSION>_0.<DISTANCE>.<SHORTHASH>.<OFFERING>[.<SUFFIX>]
ret = "{}_0.{}.{}.{}".format(
version, distance, shorthash, offering)
if suffix:
ret += "." + suffix
return ret
else:
# <VERSION>+<DISTANCE>~<SHORTHASH>-<OFFERING>[-<SUFFIX>]
ret = "{}+{}~{}-{}".format(version, distance, shorthash, offering)
if suffix:
ret += "-" + suffix
return ret
# Parse arguments.
parser = argparse.ArgumentParser(
description="Get the current version of Memgraph.")
parser.add_argument(
"--enterprise", action="store_true",
help="set the current offering to enterprise (default 'community')")
parser.add_argument(
"version", help="manual version override, if supplied the version isn't "
"determined using git")
parser.add_argument(
"suffix", help="custom suffix for the current version being built")
parser.add_argument(
"--variant", choices=("binary", "deb", "rpm"), default="binary",
help="which variant of the version string should be generated")
parser.add_argument(
"--memgraph-root-dir", help="The root directory of the checked out "
"Memgraph repository.", default=".")
args = parser.parse_args()
if not os.path.isdir(args.memgraph_root_dir):
raise Exception("The root directory ({}) is not a valid directory".format(
args.memgraph_root_dir))
os.chdir(args.memgraph_root_dir)
offering = "enterprise" if args.enterprise else "community"
# Check whether the version was manually supplied.
if args.version:
if not re.match(r"^[0-9]+\.[0-9]+\.[0-9]+$", args.version):
raise Exception("Invalid version supplied '{}'!".format(args.version))
print(format_version(args.variant, args.version, offering,
suffix=args.suffix), end="")
sys.exit(0)
# Within CI, after the regular checkout, master is sometimes (e.g. in the case
# of an epic or task branch) NOT created as a local branch. cpack depends on
# variables generated by calling this script during the cmake phase. This
# script needs master to be the local branch. `git fetch origin master:master`
# is creating the local master branch without checking it out. Does nothing if
# master is already there.
try:
current_branch = get_output("git", "rev-parse", "--abbrev-ref", "HEAD")
if current_branch != "master":
branches = get_output("git", "branch")
if "master" in branches:
# If master is present locally, the fetch is allowed to fail
# because this script will still be able to compare against the
# master branch.
try:
get_output("git", "fetch", "origin", "master:master")
except Exception:
pass
else:
# If master is not present locally, the fetch command has to
# succeed because something else will fail otherwise.
get_output("git", "fetch", "origin", "master:master")
except Exception:
print("Fatal error while ensuring local master branch.")
sys.exit(1)
# Get current commit hashes.
current_hash = get_output("git", "rev-parse", "HEAD")
current_hash_short = get_output("git", "rev-parse", "--short", "HEAD")
# We want to find branches that exist on some remote and that are named
# `release/[0-9]+\.[0-9]+`.
branch_regex = re.compile(r"^remotes/[a-zA-Z0-9]+/release/([0-9]+\.[0-9]+)$")
# Find all existing versions.
versions = []
branches = get_output("git", "branch", "--all", multiple=True)
for branch in branches:
match = branch_regex.match(branch)
if match is not None:
version = tuple(map(int, match.group(1).split(".")))
master_branch_merge = get_output("git", "merge-base", "master", branch)
versions.append((version, branch, master_branch_merge))
versions.sort(reverse=True)
# Check which existing version branch is closest to the current commit. We are
# only interested in branches that were branched out before this commit was
# created.
current_version = None
for version in versions:
version_tuple, branch, master_branch_merge = version
current_branch_merge = get_output(
"git", "merge-base", current_hash, branch)
master_current_merge = get_output(
"git", "merge-base", current_hash, "master")
# The first check checks whether this commit is a child of `master` and
# the version branch was created before us.
# The second check checks whether this commit is a child of the version
# branch.
if master_branch_merge == current_branch_merge or \
master_branch_merge == master_current_merge:
current_version = version
break
# Determine current version.
if current_version is None:
raise Exception("You are attempting to determine the version for a very "
"old version of Memgraph!")
version, branch, master_branch_merge = current_version
distance = int(get_output("git", "rev-list", "--count", "--first-parent",
master_branch_merge + ".." + current_hash))
version_str = ".".join(map(str, version)) + ".0"
if distance == 0:
print(format_version(args.variant, version_str, offering,
suffix=args.suffix),
end="")
else:
print(format_version(args.variant, version_str, offering,
distance=distance, shorthash=current_hash_short,
suffix=args.suffix),
end="")

View File

@@ -1,2 +0,0 @@
# logrotate configuration for Memgraph Community
# see "man logrotate" for details

View File

@@ -1,13 +0,0 @@
# logrotate configuration for Memgraph Enterprise
# see "man logrotate" for details
/var/lib/memgraph/durability/audit/audit.log {
# rotate log files daily
daily
# keep one year worth of audit logs
rotate 365
# send SIGUSR2 to notify memgraph to recreate logfile
postrotate
/usr/bin/killall -s SIGUSR2 memgraph
endscript
}

View File

@@ -1,12 +0,0 @@
[Unit]
Description=Memgraph: High performance, in-memory, transactional graph database
[Service]
User=memgraph
Group=memgraph
ExecStart=/usr/lib/memgraph/memgraph
# Uncomment this if Memgraph needs more time to write the snapshot on exit.
#TimeoutStopSec=5min
[Install]
WantedBy=multi-user.target

Some files were not shown because too many files have changed in this diff Show More