v0.12.0-beta - Preliminary Neo4j Support current as of 4.0 MR03 (#107)

* refactor InspectionResult -> Advice with unit tests * 4.0 MR code spike * 4.0 MR code spike 2 - stop/start/drop/create databases * progress on #102 security rules * bare queries * [email protected] * ability to run queries against systemdb * database administration start/stop/drop/create * database create/stop/start/delete lifecycle events working * v4.0 changes default_advertised_address config key * fixed auth provider detection for 4.0 * fixed multidb detection for 4.0 * cleanup * page cache tracking re-enabled for 4.0 * add getDefaultDatabase method * don't allow administration of default db or system db * re-enable transaction monitor for 4.0 * allow queries to be run against particular databases * first commit of privileges table for 4.0 * re-enable transaction monitor for 4.0 * cleanup * add privileges kb entry * shim buttons for grant/deny/revoke * leaky promise cleanup * tabbed layout * multidb systems always have a system graph * branch builds * correct aws cli path * fix paths in cache invalidation * when running diagnostics, tags are supported * disable unimplemented buttons for time being * export the status API * return shim databases for pre-4.0 * disable add database button for < 4.0 * full correct URLs in build output * first working commit to alter privileges * incorporate AlterPrivilegeForm * incorporate privilege operations * lint * create operations from system results * deny/revoke individual privileges * move more things into state * lock down modals for individual rows * build more robust mappings * locked forms don't permit editing dropdowns * deprecate old PageCache component * remove deprecated PageCache component from OSPane * Add popup default status * fixes #105 warn on default tx_log retention policy * slight modifications for 4.0.0-beta1 * fix finding missing parameters * run queries against systemdb * set global systemdb constant * globally use driver's SYSTEM_DB constant * 4.0MR3 fix: users, roles against systemdb * run all manager queries on systemdb * account for difference between graph and database * revoke and deny get red buttons * 4.0 transition comments * disable disk utilization for Neo4j 4 * run queries on systemdb * major package update for security fixes * npx react-codemod rename-unsafe-lifecycles https://reactjs.org/blog/2018/03/27/update-on-async-rendering.html * dependency updates * disable privilege features not yet ready for initial 4.0 support * upgrade driver parameter syntax * make 4.0 driver work with Neo4j < 4.0 * ApocMetaStats component first commit * error handling * hoc APOC only wrapper * error handling and state updates * big scary message before you drop a database * version 0.12.0-beta
moxious · Nov 20, 2019 · 66ce361 · 66ce361
1 parent eafc395
commit 66ce361
Show file tree

Hide file tree

Showing 96 changed files with 2,942 additions and 545 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -20,9 +20,13 @@ jobs:
             curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
             sudo python get-pip.py
             sudo pip install setuptools
+
+            pip --version            
             
             cd s3cmd-2.0.2 && sudo python setup.py install
             cd $HOME && s3cmd --version
+
+            pip install --upgrade --user awscli
       - restore_cache:
           name: Restore Yarn Package Cache
           keys:
@@ -53,7 +57,8 @@ jobs:
             # Take careful note of runInBand https://jestjs.io/docs/en/troubleshooting.html#tests-are-extremely-slow-on-docker-and-or-continuous-integration-ci-server
             yarn test --runInBand
       - run: 
-          name: Deploy to S3 (Build Branch)
+          # This creates a runnable copy on S3 **per build**
+          name: Deploy to S3 (BUILD)
           command: |
             if [ "${CIRCLE_BRANCH}" = "master" ]; then
               echo "Skipping branch build; master will be deployed"
@@ -75,8 +80,30 @@ jobs:
               s3cmd modify "s3://$BUCKET/$SUBDIR/static/css/*.css" --mime-type="text/css"
               # Replace package file with the original
               cp package.orig package.json
-              echo "$FINAL_URL" | tee /tmp/artifacts/s3-deploy-branch.txt
+              echo "$FINAL_URL"index.html | tee /tmp/artifacts/s3-deploy-build.txt
             fi
+      - run:
+          # This creates a runnable copy on S3 **per branch** (gets overwritten)
+          name: Deploy to S3 (BRANCH)
+          command: |
+            export SUBDIR="${CIRCLE_BRANCH}"
+            export BUCKET=halin.graphapp.io
+            export HOMEPAGE=http://halin.graphapp.io/
+            export BASE_URL=http://halin.graphapp.io/
+            export FINAL_URL="$BASE_URL$SUBDIR/"
+            cp package.json package.orig
+            # Have to monkeypatch the "homepage" field so that the react build will
+            # work.
+            cat package.json | sed s,$HOMEPAGE,$FINAL_URL, > package.branch
+            cp package.branch package.json
+            cp package.branch /tmp/artifacts/package.branch.json
+            npm run build
+            s3cmd put --recursive -P dist/* "s3://$BUCKET/$SUBDIR/"
+            # Thinking this is a bug in s3 that has to be overcome: https://stackoverflow.com/questions/53708938/s3cmd-flagging-css-with-wrong-mime-type
+            s3cmd modify "s3://$BUCKET/$SUBDIR/static/css/*.css" --mime-type="text/css"
+            # Replace package file with the original
+            cp package.orig package.json
+            echo "$FINAL_URL"index.html | tee /tmp/artifacts/s3-deploy-branch.txt
       - run: 
           name: Build Halin Docker Image
           command: docker build -t mdavidallen/halin:latest -f Dockerfile .
@@ -167,6 +194,10 @@ jobs:
             else
               echo "Not deploying; not master branch."
             fi
+      - run:
+          name: AWS Cache Invalidation
+          command: |
+             ~/.local/bin/aws cloudfront create-invalidation --distribution-id E1WH29O1XG2QXT --paths '/*'
       - store_artifacts:
             path: coverage
             prefix: coverage      

diff --git a/.gitignore b/.gitignore
@@ -27,3 +27,4 @@ grafana
 src/creds/*
 reports
 code-signing
+neo4j-*
diff --git a/create-empty-db.sh b/create-empty-db.sh
@@ -4,11 +4,14 @@ docker stop neo4j-empty
 
 PASSWORD=admin
 CWD=`pwd`
-NEO4J=neo4j:3.5.5
+#NEO4J=neo4j:3.5.5
+#NEO4J=neo4j:4.0.0-alpha09
+NEO4J=neo4j:4.0.0-alpha09mr02-enterprise
 
 docker run -d --name neo4j-empty --rm \
 	-p 127.0.0.1:7474:7474 \
         -p 127.0.0.1:7687:7687 \
+        --env=NEO4J_ACCEPT_LICENSE_AGREEMENT=yes \
         --env=NEO4J_dbms_memory_pagecache_size=1G \
         --env=NEO4J_dbms_memory_heap_initial__size=2G \
         --env=NEO4J_dbms_memory_heap_max__size=4G \

diff --git a/create-neo4j4.sh b/create-neo4j4.sh
@@ -0,0 +1,21 @@
+#!/bin/bash
+# Quick script to start a community instance for testing.
+docker stop neo4j-empty
+
+PASSWORD=admin
+CWD=`pwd`
+NEO4J=neo4j:4.0.0-alpha09mr02-enterprise
+
+docker run -d --name neo4j-empty --rm \
+	-p 127.0.0.1:7474:7474 \
+        -p 127.0.0.1:7687:7687 \
+        --env=NEO4J_ACCEPT_LICENSE_AGREEMENT=yes \
+        --env=NEO4J_dbms_memory_pagecache_size=1G \
+        --env=NEO4J_dbms_memory_heap_initial__size=2G \
+        --env=NEO4J_dbms_memory_heap_max__size=4G \
+	--env NEO4J_AUTH=neo4j/admin \
+        --env NEO4J_dbms_max__databases=5 \
+	-t $NEO4J
+
+echo "When you're ready to run some cypher, execute this:"
+echo docker exec --interactive --tty neo4j-empty bin/cypher-shell -a localhost -u neo4j -p $PASSWORD
diff --git a/package.json b/package.json
@@ -1,7 +1,7 @@
 {
   "name": "halin",
   "description": "Halin helps you monitor and improve your Neo4j graph",
-  "version": "0.11.3",
+  "version": "0.12.0-beta",
   "neo4jDesktop": {
     "apiVersion": "^1.2.0"
   },
@@ -21,25 +21,25 @@
   "dependencies": {
     "@sentry/browser": "^4.4.1",
     "autobind-decorator": "2.2.1",
-    "bluebird": "^3.5.3",
+    "bluebird": "^3.7.1",
     "generic-pool": "^3.6.1",
     "graph-app-kit": "^1.0.4",
-    "lodash": "^4.17.11",
+    "lodash": "^4.17.15",
     "mathjs": "^5.4.0",
-    "moment": "^2.22.2",
-    "neo4j-driver": "^1.7.2",
-    "pondjs": "^0.8.9",
+    "moment": "^2.24.0",
+    "neo4j-driver": "^4.0.0-beta01",
+    "pondjs": "^0.9.0",
     "prop-types": "^15.6.0",
-    "react": "^16.6.3",
+    "react": "^16.12.0",
     "react-csv": "^1.1.1",
     "react-d3-components": "^0.9.1",
-    "react-dom": "^16.6.3",
+    "react-dom": "^16.12.0",
     "react-graph-vis": "^1.0.2",
     "react-minimal-pie-chart": "^4.0.0",
-    "react-scripts": "^2.1.5",
+    "react-scripts": "^2.1.8",
     "react-sortable-tree": "^2.2.0",
     "react-table": "^6.8.6",
-    "react-timeseries-charts": "^0.15.3",
+    "react-timeseries-charts": "^0.16.1",
     "react-toastify": "^5.1.0",
     "ringjs": "^0.0.1",
     "source-map-explorer": "^1.6.0",

diff --git a/release-notes.md b/release-notes.md
@@ -1,5 +1,13 @@
 # Halin Release Notes
 
+## 0.12.0-beta
+
+- Initial support for Neo4j 4.0 as of MR03; multiple databases now supported!
+- New database pane that allows creating, deleting, stopping and starting databases.
+- Fixes #105, warn on default tx_log retention policy
+- Because Neo4j 4.0 is not yet final, this is a beta release subject to change. Support for Neo4j
+3.4 and 3.5 remains steady, but Neo4j 4.0 support is provisional only.
+
 ## 0.11.3
 
 - Fixes #96, adds labels to GC Pause times

diff --git a/src/api/Database.js b/src/api/Database.js
@@ -0,0 +1,24 @@
+/**
+ * A database is a graph that can be stored within Neo4j.
+ * 
+ * Multi-database starts with Neo4j >= 4.0.  For versions of Neo4j prior to 4,
+ * the HalinContext will fake a single database.
+ */
+
+ export default class Database {
+     constructor(name, status, isDefault=false) {
+         this.name = name;
+         this.status = status;
+         this.isDefault = isDefault;
+     }
+
+     getLabel() { 
+         return this.name;
+     }
+
+     getStatus() {
+         return this.status;
+     }
+
+     isOnline() { return this.status === 'online'; }
+ }
diff --git a/src/api/HalinContext.js b/src/api/HalinContext.js
@@ -33,6 +33,7 @@ export default class HalinContext {
         };
         this.debug = false;
         this.mgr = new ClusterManager(this);
+        this.mgr.addListener(e => this.onClusterEvent(e));
     }
 
     members() {
@@ -179,6 +180,10 @@ export default class HalinContext {
         return this.getWriteMember().supportsSystemGraph();
     }
 
+    supportsMultiDatabase() {
+        return this.getWriteMember().supportsMultiDatabase();
+    }
+
     /**
      * Returns true if the context supports authorization overall.
      */
@@ -293,6 +298,15 @@ export default class HalinContext {
         return this.currentUser;
     }
 
+    /**
+     * Listener that fires in ClusterManager whenever a cluster-wide event happens.  This allows the
+     * context to be aware of things changing around it and adjust.
+     * @param {Object} event with keys date, payload, id, type
+     */
+    onClusterEvent(event) {
+        sentry.info('Cluster Event', event);
+    }
+
     checkUser(driver /*, progressCallback */) {
         const q = 'call dbms.showCurrentUser()';
         const session = driver.session();
@@ -454,6 +468,8 @@ export default class HalinContext {
                         this.checkForCluster(active, progressCallback),
                     ]);
                 })
+                // Checking databases must be after checking for a cluster, since we need to know who leader is
+                .then(() => this.getClusterManager().getDatabases())
                 .then(() => {
                     this.getClusterManager().addEvent({
                         type: 'halin',

diff --git a/src/api/HalinContext.test.js b/src/api/HalinContext.test.js
@@ -76,8 +76,8 @@ describe('Halin Context', function () {
             expect(ctx.isCluster()).toBe(true));
         it('knows that it supports native auth', () =>
             expect(ctx.supportsNativeAuth()).toBe(true));
-        it('does not support system graph', () =>
-            expect(ctx.supportsSystemGraph()).toBe(false));
+        it('supports system graph', () =>
+            expect(ctx.supportsSystemGraph()).toBe(true));
         it('can get its base URI', () =>
             expect(ctx.getBaseURI()).toEqual(`bolt://${fakes.basics.host}:${fakes.basics.port}`));
         it('knows its current user', () => {
-Original file line number
+Diff line change
@@ Expand Up / @@ -27,3 +27,4 @@ grafana @@
     src/creds/*
     reports
     code-signing
+    neo4j-*