Commit Graph

120 Commits

Author SHA1 Message Date
Deda
96cb02ee96 Minor final changes 2024-03-22 17:33:00 +01:00
Deda
e2353449ab Comment out Package job because run-package-in-mgbuilder is not merged 2024-03-22 16:18:36 +01:00
Deda
2ac396e23f Add stress test large and improve workflows 2024-03-22 16:16:45 +01:00
Deda
5c7e8e543b Remove env variables from main workflow 2024-03-22 15:19:44 +01:00
Deda
67a4bb5618 Fix macro benchmarks parallel step name 2024-03-22 14:51:41 +01:00
Deda
5cb7c1de20 Adapt rest of benchamrk tests 2024-03-22 14:49:28 +01:00
Deda
33af1fde14 Fix jepsen conditon in rusable release tests 2024-03-21 18:47:56 +01:00
Deda
d47436a3a0 Fix Debian10 in release build tests 2024-03-21 18:46:02 +01:00
Deda
8edd8a9590 Fix typo in reusable release tests 2024-03-21 18:44:16 +01:00
Marko Barišić
91302ccb05 Merge branch 'master' into run-release-ci-in-mgbuild 2024-03-21 17:04:57 +01:00
Deda
c4387ce34f Adapt release workflow to mgbuild based CI 2024-03-21 16:59:48 +01:00
Andi
f699c0b37f Support bolt+routing (#1796) 2024-03-21 06:41:26 +00:00
Marko Barišić
2ac649f3b5 Upgrade jepsen (#1594)
* Try with jepsen v0.3.5
* Add a few WIP adjustments
* Add replication restore state on startup flag
* Fix some run.sh scripts issues
* Improve cluster commands
* Run Jepsen on debian-12 with toolchain v5
---------
Co-authored-by: Marko Budiselic <mbudiselicbuda@gmail.com>
2024-03-18 16:38:58 +01:00
Marko Barišić
ec8536e11b Make diff run on push to master again (#1826)
* Add workflow dispatch and run on push to master
2024-03-18 11:58:34 +01:00
Marko Barišić
b0cdcd3483 Run CI in mgbuilder containers (#1749)
* Update deployment files for mgbuilders because of toolchain upgrade
* Fix args parameter in builder yaml files
* Add fedora 38, 39 and rockylinux 9.3 mgbuilder Dockerfiles
* Change format of ARG TOOLCHAIN_VERSION from toolchain-vX to vX
* Add function to check supported arch, build type, os and toolchain
* Add options to init subcommand
* Add image names to mgbuilders
* Add v2 of the run.sh script
* Add testing to run2.sh
* Add option for threads --thread
* Add options for enterprise license and organization name
* Make stop mgbuild container step run always
* Add --ci flag to init script
* Move init conditionals under build-memgraph flags
* Add --community flag to build-memgraph
* Change target dir inside mgbuild container
* Add node fix to debian 11, ubuntu 20.04 and ubuntu 22.04
* rm memgraph repo after installing deps
* Add mg user in Dockerfile
* Add step to install rust on all OSs
* Chown files copied into mgbuild container
* Add e2e tests
* Add jepsen test
* Bugfix: Using reference in a callback
* Bugfix: Broad target for e2e tests
* Up db info test limit
* Disable e2e streams tests
* Fix default THREADS
* Prioretize docker compose over docker-compose
* Improve selection between docker compose and docker-compose
* Install PyYAML as mg user
* Fix doxygen install for rocky linux 9.3
* Fix rocky-9.3 environment script to properly install sbcl
* Rename all rocky-9 mentions to rocky-9.3
* Add mgdeps-cache and benchgraph-api hostnames to mgbuild images
* Add logic to pull mgbuild image if missing
* Fix build errors on toolchain-v5 (#1806)
* Rename run2 script, remove run script, add small features to mgbuild.sh
* Add --no-copy flag to build-memgraph to resolve TODO
* Add timeouts to diff jobs
* Fix asio flaky clone, try mgdeps-cache first

---------

Co-authored-by: Andreja Tonev <andreja.tonev@memgraph.io>
Co-authored-by: Ante Pušić <ante.f.pusic@gmail.com>
Co-authored-by: antoniofilipovic <filipovicantonio1998@gmail.com>
2024-03-14 12:19:59 +01:00
Marko Barišić
e88c7a0aa5 Add jobs for pushing ARM packages (#1765)
* Add jobs for pushing ARM packages
2024-02-27 12:08:53 +01:00
Marko Barišić
86ff96697d Minor update to the rc workflow (#1760)
* Increase ARM build timeout to 120 minutes

* Remove PushToS3 job and make each Package job push to S3 individually

* Expand ARM timeout to 150 minutes for added safety; revert this after release
2024-02-26 22:57:21 +01:00
Marko Barišić
e302be98a2 Push successful RC builds to S3 (#1741)
* Add new workflow which calls release build workflows

* Make the workflow build packages only on RC tags

* Change artifact names to include OS name
2024-02-21 17:08:14 +01:00
Andi
7ec648b4ce Add --experimental-enabled=high-availability (#1720) 2024-02-19 16:28:15 +00:00
Marko Barišić
5f2e3f01d0 Turn e2e tests back on for release build workflows (#1725) 2024-02-15 16:20:04 +01:00
Marko Barišić
2c774ff09b Add rules for rc workflows (#1722) 2024-02-15 15:33:14 +01:00
Gareth Andrew Lloyd
f48151576b System replication experimental flag (#1702)
- Remove the compile time control
- Introduce the runtime control flag

New flag `--experimental-enabled=system-replication`
2024-02-13 12:57:18 +00:00
Antonio Filipovic
4f4a569c72 Revert replication tests (#1707) 2024-02-12 16:42:57 +01:00
Antonio Filipovic
54f78f9217 Revert e2e tests and remove flaky ones (#1703) 2024-02-09 12:55:31 +01:00
Marko Barišić
b443934b68 Update release CI job timeouts (#1683)
* Set timeout for jobs in release CI to 60 minutes

* Set timout for stress test large to 12 hours
2024-01-31 19:19:05 +01:00
Marko Barišić
a9ef28c68e Upgrade deprecated github actions (#1673)
* upgrade actions/checkout from v3 to v4

* upgrade actions/upload-artifact from v3 to v4

* upgrade actions/download-artifact from v2 and v3 to v4

* Fix duplicate artifact names in diff.yaml

* Fix duplicate artifact names in release_debian10.yaml and release_ubuntu2004.yaml
2024-01-30 22:52:56 +01:00
Marko Barišić
79361e9205 Disable e2e tests in all workflows (#1677)
* Disable e2e tests in diff.yaml

* Disable e2e tests in release builds
2024-01-30 19:37:00 +01:00
Andi
78a88737f8 HA: Add automatic failover (#1646)
Co-authored-by: antoniofilipovic <filipovicantonio1998@gmail.com>
2024-01-29 15:34:00 +01:00
Marko Barišić
f1484240a0 Make release CI predictable (#1658)
* Move stress test large to a separate workflow
* Combine Debian10 and Ubuntu20.04 stress tests into one workflow
* Remove BigMemory tag from release_debian10
* Move e2e tests to a separate job
* Move debug interation tests to a separate job
* Move release durability and stress tests to a separate job
* Move release benchamarks to separate job
* Add 90 min timeout restriction to all jobs
* Move env variables to workflow level
* Move BUILD_TYPE env var to workflow level
---------

Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
2024-01-26 17:01:25 +01:00
Marko Barišić
1c95c3dc59 Add step to refresh jepsen cluster before test (#1667)
refresh jepsen during diff workflow
2024-01-26 10:50:03 +00:00
Andi
38ade99652 HA: Add coordinator to replication cluster (#1608) 2024-01-24 13:07:51 +01:00
andrejtonev
071df2f439 Replication refactor part 7 (#1550)
* Split queries into system and data queries
* System queries are sequentially executed and generate separate transaction deltas
* System transaction try locks for 100ms
* last_commited_system_ts saved to DBMS durability
* Replicating CREATE/DROP DATABASE
* Sending a system snapshot if REPLICA behind
* Passing a copy of the gatekeeper::access as std::any to all functions that could call an async execution
* Removed delete_on_drop flag (we now always delete on drop)
* Using UUID as the directory name for databases
* DBMS durability update (added versioning and salient information)
* Automatic migration from previous version
* Interpreter can run some queries without a target database
* SHOW REPLICA returns the status of the currently active DB
* Returning UUID instead of db name in the RPC responses
* Using UUIDs for database specification in RPC (not name)
* FrequentCheck forces update on reconnect
* TimestampRpc will detect if a replica is behind, and will update client's state
* Safer SLK reads
* Split SHOW DATABASES in two SHOW DATABASES (list of current databases) and SHOW DATABASE a single string naming the current database

---------

Co-authored-by: Gareth Lloyd <gareth.lloyd@memgraph.io>
2024-01-23 12:06:10 +01:00
Marko Barišić
5e5f215be4 Add steps to bring kafka and pulsar up and down on daily builds (#1643) 2024-01-17 14:44:18 +01:00
Andi
4788a633a6 Improve e2e Kafka and Pulsar testing (#1604) 2024-01-02 13:29:25 +01:00
Marko Barišić
ce2705d012 Remove centOS8 daily build (#1587) 2024-01-02 12:14:25 +01:00
Marko Barišić
e56e516f94 Add BigMemory label to the release build (#1568) 2023-12-07 11:54:43 +01:00
Marko Barišić
70db2fca56 Change package_all to package_memgraph (#1507)
Add the ability to pick a specific package to build
2023-11-23 12:46:04 +00:00
Marko Barišić
9cc060c4b0 Fix error in upload-to-s3 job (#1504) 2023-11-13 13:01:01 +01:00
Marko Barišić
e671a0737e Fix package specific workflow file (#1503) 2023-11-13 12:54:19 +01:00
Marko Barišić
11be3972c4 Add workflow for packaging memgraph for specific target OS (#1502) 2023-11-13 12:20:49 +01:00
Andi
011caf3bf1 Fix mgbench daily upload (#1410) 2023-10-28 12:52:14 +02:00
Andi
b1bd977f7b Fix GQL release worklflows (#1424) 2023-10-28 12:50:16 +02:00
Marko Barišić
e9f3a5fd1b Fix default value in release workflows (#1418) 2023-10-26 16:17:16 +02:00
Josipmrden
fdf63436ab Add cartesian and hash join mgbench (#1393) 2023-10-24 19:44:11 +02:00
Marko Budiselić
945388fba6 Revert to Release under Diff Release workflow (#1390) 2023-10-22 22:31:06 +02:00
Aidar Samerkhanov
667e7f670e Parametrize CI workflows build type (#1324)
The Release is the default, it's also possible to run package_all with
RelWithDebInfo
2023-10-17 00:04:08 +02:00
Aidar Samerkhanov
7fbf5857f2 Add GQL behave tests for on-disk storage (#1238) 2023-10-10 09:27:11 +03:00
Andi
2fd34489af Add mgbench support for disk storage and analytical mode (#1286)
* Add mgbench support for disk storage and analytical mode
2023-10-06 10:19:29 +02:00
Josipmrden
440838c0e9 Add dependency check for e2e tests (#1240) 2023-09-12 11:34:08 -04:00
Josipmrden
79a3c5af8e Add manual performance benchmark execution (#1239) 2023-09-12 11:33:05 -04:00