Cayley 0.7.0 depth function issue


#22

Hi @dennwc

Thanks a lot for the quick fix.

Can you please help me out with the steps or any link to compile from the sources in windows or linux.

I have downloaded the souce code and done the changes in the files. Now, How to build this code ??

Thanks in advance.


#23

You can check Contributing guide, first part of it shows how to build it on Linux.

Once it works you can change build command to build for Windows as well:

go build ./cmd/cayley
GOOS=windows go build ./cmd/cayley

#24

Hi @dennw & @Jorgen

I have tried to build from the cayley source code.I have downloaded the source code and done the code changes in the required files. I have followed the below steps to build but in the last step I am getting below errors :

Steps Followed :

mkdir -p ~/cayley && cd ~/cayley
export GOPATH=pwd
export PATH=$PATH:~/cayley/bin
mkdir -p bin pkg src/github.com/cayleygraph
cd src/github.com/cayleygraph
git clone https://github.com/cayleygraph/cayley
cd cayley
curl https://glide.sh/get | sh
glide install
go build ./cmd/cayley

Error :

cmd/cayley/command/dedup.go:4:2: cannot find package “context” in any of:
/usr/src/pkg/context (from $GOROOT)
/root/cayley/src/context (from $GOPATH)
query/sexp/parser.go:18:2: cannot find package “github.com/badgerodon/peg” in any of:
/usr/src/pkg/github.com/badgerodon/peg (from $GOROOT)
/root/cayley/src/github.com/badgerodon/peg (from $GOPATH)
graph/kv/bolt/bolt.go:23:2: cannot find package “github.com/boltdb/bolt” in any of:
/usr/src/pkg/github.com/boltdb/bolt (from $GOROOT)
/root/cayley/src/github.com/boltdb/bolt (from $GOPATH)
query/graphql/http.go:8:2: cannot find package “github.com/dennwc/graphql/gqlerrors” in any of:
/usr/src/pkg/github.com/dennwc/graphql/gqlerrors (from $GOROOT)
/root/cayley/src/github.com/dennwc/graphql/gqlerrors (from $GOPATH)
query/graphql/graphql.go:13:2: cannot find package “github.com/dennwc/graphql/language/ast” in any of:
/usr/src/pkg/github.com/dennwc/graphql/language/ast (from $GOROOT)
/root/cayley/src/github.com/dennwc/graphql/language/ast (from $GOPATH)
query/graphql/graphql.go:14:2: cannot find package “github.com/dennwc/graphql/language/lexer” in any of:
/usr/src/pkg/github.com/dennwc/graphql/language/lexer (from $GOROOT)
/root/cayley/src/github.com/dennwc/graphql/language/lexer (from $GOPATH)
query/graphql/graphql.go:15:2: cannot find package “github.com/dennwc/graphql/language/parser” in any of:
/usr/src/pkg/github.com/dennwc/graphql/language/parser (from $GOROOT)
/root/cayley/src/github.com/dennwc/graphql/language/parser (from $GOPATH)
query/gizmo/environ.go:24:2: cannot find package “github.com/dop251/goja” in any of:
/usr/src/pkg/github.com/dop251/goja (from $GOROOT)
/root/cayley/src/github.com/dop251/goja (from $GOPATH)
graph/sql/mysql/mysql.go:13:2: cannot find package “github.com/go-sql-driver/mysql” in any of:
/usr/src/pkg/github.com/go-sql-driver/mysql (from $GOROOT)
/root/cayley/src/github.com/go-sql-driver/mysql (from $GOPATH)
quad/pquads/quads.pb.go:23:8: cannot find package “github.com/gogo/protobuf/gogoproto” in any of:
/usr/src/pkg/github.com/gogo/protobuf/gogoproto (from $GOROOT)
/root/cayley/src/github.com/gogo/protobuf/gogoproto (from $GOPATH)
quad/pquads/pio/io.go:32:2: cannot find package “github.com/gogo/protobuf/proto” in any of:
/usr/src/pkg/github.com/gogo/protobuf/proto (from $GOROOT)
/root/cayley/src/github.com/gogo/protobuf/proto (from $GOPATH)
clog/glog/glog.go:7:2: cannot find package “github.com/golang/glog” in any of:
/usr/src/pkg/github.com/golang/glog (from $GOROOT)
/root/cayley/src/github.com/golang/glog (from $GOPATH)
internal/gephi/stream.go:14:2: cannot find package “github.com/julienschmidt/httprouter” in any of:
/usr/src/pkg/github.com/julienschmidt/httprouter (from $GOROOT)
/root/cayley/src/github.com/julienschmidt/httprouter (from $GOPATH)
graph/sql/postgres/postgres.go:14:2: cannot find package “github.com/lib/pq” in any of:
/usr/src/pkg/github.com/lib/pq (from $GOROOT)
/root/cayley/src/github.com/lib/pq (from $GOPATH)
quad/jsonld/jsonld.go:10:2: cannot find package “github.com/linkeddata/gojsonld” in any of:
/usr/src/pkg/github.com/linkeddata/gojsonld (from $GOROOT)
/root/cayley/src/github.com/linkeddata/gojsonld (from $GOPATH)
graph/nosql/nosql.go:9:2: cannot find package “github.com/pborman/uuid” in any of:
/usr/src/pkg/github.com/pborman/uuid (from $GOROOT)
/root/cayley/src/github.com/pborman/uuid (from $GOPATH)
internal/repl/repl.go:29:2: cannot find package “github.com/peterh/liner” in any of:
/usr/src/pkg/github.com/peterh/liner (from $GOROOT)
/root/cayley/src/github.com/peterh/liner (from $GOPATH)
internal/http/docs.go:25:2: cannot find package “github.com/russross/blackfriday” in any of:
/usr/src/pkg/github.com/russross/blackfriday (from $GOROOT)
/root/cayley/src/github.com/russross/blackfriday (from $GOPATH)
cmd/cayley/command/convert.go:8:2: cannot find package “github.com/spf13/cobra” in any of:
/usr/src/pkg/github.com/spf13/cobra (from $GOROOT)
/root/cayley/src/github.com/spf13/cobra (from $GOPATH)
cmd/cayley/command/database.go:14:2: cannot find package “github.com/spf13/viper” in any of:
/usr/src/pkg/github.com/spf13/viper (from $GOROOT)
/root/cayley/src/github.com/spf13/viper (from $GOPATH)
graph/kv/leveldb/leveldb.go:22:2: cannot find package “github.com/syndtr/goleveldb/leveldb” in any of:
/usr/src/pkg/github.com/syndtr/goleveldb/leveldb (from $GOROOT)
/root/cayley/src/github.com/syndtr/goleveldb/leveldb (from $GOPATH)
graph/kv/leveldb/leveldb.go:23:2: cannot find package “github.com/syndtr/goleveldb/leveldb/iterator” in any of:
/usr/src/pkg/github.com/syndtr/goleveldb/leveldb/iterator (from $GOROOT)
/root/cayley/src/github.com/syndtr/goleveldb/leveldb/iterator (from $GOPATH)
graph/kv/leveldb/leveldb.go:24:2: cannot find package “github.com/syndtr/goleveldb/leveldb/opt” in any of:
/usr/src/pkg/github.com/syndtr/goleveldb/leveldb/opt (from $GOROOT)
/root/cayley/src/github.com/syndtr/goleveldb/leveldb/opt (from $GOPATH)
graph/kv/leveldb/leveldb.go:25:2: cannot find package “github.com/syndtr/goleveldb/leveldb/util” in any of:
/usr/src/pkg/github.com/syndtr/goleveldb/leveldb/util (from $GOROOT)
/root/cayley/src/github.com/syndtr/goleveldb/leveldb/util (from $GOPATH)
graph/kv/indexing.go:32:2: cannot find package “github.com/tylertreat/BoomFilters” in any of:
/usr/src/pkg/github.com/tylertreat/BoomFilters (from $GOROOT)
/root/cayley/src/github.com/tylertreat/BoomFilters (from $GOPATH)
graph/nosql/mongo/mongo.go:10:2: cannot find package “gopkg.in/mgo.v2” in any of:
/usr/src/pkg/gopkg.in/mgo.v2 (from $GOROOT)
/root/cayley/src/gopkg.in/mgo.v2 (from $GOPATH)
graph/nosql/mongo/mongo.go:11:2: cannot find package “gopkg.in/mgo.v2/bson” in any of:
/usr/src/pkg/gopkg.in/mgo.v2/bson (from $GOROOT)
/root/cayley/src/gopkg.in/mgo.v2/bson (from $GOPATH)
graph/nosql/elastic/elastic.go:12:2: cannot find package “gopkg.in/olivere/elastic.v5” in any of:
/usr/src/pkg/gopkg.in/olivere/elastic.v5 (from $GOROOT)
/root/cayley/src/gopkg.in/olivere/elastic.v5 (from $GOPATH)

Note : Not able to do so in windows as well.


#25

Can you please share an output of go version and go env?

It looks like your Go installation is below 1.8, while Cayley requires 1.9. You can check this link https://github.com/golang/go/wiki/Ubuntu to update it. The preffered method is via snap package - it should always be up-to-date.

As an alternative I can build a binary for you to save some time.


#26

Thanks @dennwc

It is showing as below :

root@cri-server7:/usr/local# go version
go version go1.2.1 linux/amd64

But I have downloaded the latest version, Please find the below commands :

root@cri-server7:~/cayley/src/github.com/cayleygraph/cayley# sudo apt-get install golang-1.9-go
Reading package lists… Done
Building dependency tree
Reading state information… Done
golang-1.9-go is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 594 not upgraded.

and go env is showing as below :

root@cri-server7:/usr/local# go env
GOARCH=“amd64"
GOBIN=”"
GOCHAR=“6"
GOEXE=”"
GOHOSTARCH=“amd64"
GOHOSTOS=“linux"
GOOS=“linux"
GOPATH=”/root/go"
GORACE=”“
GOROOT=”/usr/lib/go"
GOTOOLDIR=”/usr/lib/go/pkg/tool/linux_amd64"
TERM="dumb"
CC=“gcc"
GOGCCFLAGS=”-g -O2 -fPIC -m64 -pthread"
CXX="g++"
CGO_ENABLED=“1”


#27

According to that wiki page:

Note that golang-1.9-go puts binaries in /usr/lib/go-1.9/bin. If you want them on your PATH, you need to make that change yourself.

They decided to split Go 1.2 and Go 1.9 into different packages, unfortunately.

But if you override which version should be used, it will work:

GOROOT="/usr/lib/go-1.9" /usr/lib/go-1.9/bin/go build -i github.com/cayleygraph/cayley/cmd/cayley

#28

Thanks a lot @dennwc , After facing so many issues, Now I am able to build.

But for write api , i am getting database is closed issue.

root@cri-server7:~/go/src/github.com/cayleygraph/cayley# ./cayley http --config=cayley_example.yml --host 10.236.220.137:64210
I0123 14:17:01.248176 20962 cayley.go:63] Cayley version: 0.7.1 (dev snapshot)
I0123 14:17:01.248758 20962 cayley.go:76] using config file: cayley_example.yml
I0123 14:17:01.249074 20962 database.go:187] using backend “postgres” (postgres://postgres:postgres@10.236.220.117:5432/newdb?sslmode=disable)
I0123 14:17:01.252865 20962 http.go:186] using assets from "."
I0123 14:17:01.253805 20962 http.go:42] listening on 10.236.220.137:64210, web interface at http://10.236.220.137:64210

I0123 14:17:10.204327 20962 http.go:89] started POST /api/v1/write for 192.168.250.11
E0123 14:17:10.204991 20962 quadstore.go:345] couldn’t begin write transaction: sql: database is closed
I0123 14:17:10.205378 20962 http.go:91] completed 400 Bad Request /api/v1/write in 1.066203ms

cayley_example.yml :

store:

backend to use

backend: postgres

address or path for the database

address: “postgres://postgres:postgres@10.236.220.117:5432/cayleydb?sslmode=disable”

open database in read-only mode

read_only: false

backend-specific options

options:
nosync: false
query:
timeout: 30s
load:
ignore_duplicates: false
ignore_missing: false
batch: 10000

Note : Same configuration was working fine when i was trying from my localhost (windows machine).
Postgres is installed in a linux machine and I am able to ping from one linux machine (Where I have installed cayley) to another linux machine (Where I have installed postgres).


#29

Seems related to this issue. Will fix in a moment.


#30

Done. Please pull latest master and rebuild.


#31

Hi @dennwc

Thanks for the quick fix but now I am getting below error for the write api :

Please find the details :

Url : http://10.236.220.137:64210/api/v1/write
Headers :
Content-Type : applicatiojn/json
Body :
[{
“subject”: “Test1”,
“predicate”: “RunsOn”,
“object”: “Test2”
}}

Output :
{
“error”: “pq: relation “nodes” does not exist”
}


#32

It seems like database is not initialized. Maybe previous init commands failed because of that bug.


#33

Hi @dennwc & @Jorgen

After doing all the hard work, still getting the same result :frowning_face:

It is showing some extra result for any level of depth.


#34

I think the reason why you get extra results is that you do extra queries while iterating your result.

For me it works fine now with the last version of master:

var path = g.M().Tag('hop').Both(null, "relation");
var depth = 3;
g.V("Test 2").Tag("source").FollowRecursive(path,depth).Tag("target").ForEach(function(v){
  var node = {
  source: v.hop,
  start: v.source,
  relation : v.relation,
  target: v.target
  }
  g.Emit(node)
})

Result:

go get -u github.com/cayleygraph/cayley
and run go install in cmd/cayley

run cayley on database:
cayley http -d bolt -a database.dat/

Execute the query above with Test2 (without a space on your dataset)


#35

Hi @Jorgen

Thanks for the quick replay but the response is as below for your query :

We are getting 3 RunsOn relationship as a response but it should show 2 instead of 3 as we don’t have any RunsOn from Test3 to Test2.

Note : We have only two RunsOn relationship as below :

Test1 RunsOn Test2
Test2 RunsOn Test3

{
“result”: [
{
“relation”: “RunsOn”,
“source”: “Test2”,
“start”: “Test2”,
“target”: “Test1”
},
{
“relation”: “RunsOn”,
“source”: “Test2”,
“start”: “Test2”,
“target”: “Test3”
},
{
“relation”: “RunsOn”,
“source”: “Test3”,
“start”: “Test2”,
“target”: “Test2”
},
{
“relation”: “DependsOn”,
“source”: “Test3”,
“start”: “Test2”,
“target”: “Test4”
},
{
“relation”: “DependsOn”,
“source”: “Test4”,
“start”: “Test2”,
“target”: “Test5”
},
{
“relation”: “ConnectedTo”,
“source”: “Test4”,
“start”: “Test2”,
“target”: “Test8”
}
]
}


#36

Yeah, but Cayley works good. Maybe my query is not exactly what you want.
But it works correctly because you ask for .Both() which means, any direction (In and Out) on every node.


#37

yeah @Jorgen.

I am much worried about the query response. I have tried with below one as well but no luck. Looking for getting the same response from the query as well.

var path = g.M().Both();
var depth = 2;
g.V(“Test3”).FollowRecursive(path,depth).ForEach( function(v){
g.V(v.id).OutPredicates().ForEach( function®{
g.V(v.id).Out(r.id).ForEach( function(t){
var node = {
source: v.id,
relation : r.id,
target: t.id
}
g.Emit(node)
})
})
})


#38

@Chaitanya maybe you can explain in text what information you want from the result. And to explain what the result set is that you expect.


#39

Let me explain you the requirement @Jorgen

I have already formed a query earlier for our company to show all the related nodes some months back and it was working fine as below.

Query :

var path = g.M().Both();
g.V(“Test1”).FollowRecursive(path).ForEach( function(v){
g.V(v.id).OutPredicates().ForEach( function®{
g.V(v.id).Out(r.id).ForEach( function(t){
var node = {
source: v.id,
relation : r.id,
target: t.id
}
g.Emit(node)
})
})
})

Output :

{
“result”: [
{
“relation”: “RunsOn”,
“source”: “Test2”,
“target”: “Test3”
},
{
“relation”: “RunsOn”,
“source”: “Test1”,
“target”: “Test2”
},
{
“relation”: “DependsOn”,
“source”: “Test3”,
“target”: “Test4”
},
{
“relation”: “DependsOn”,
“source”: “Test4”,
“target”: “Test5”
},
{
“relation”: “ConnectedTo”,
“source”: “Test4”,
“target”: “Test8”
},
{
“relation”: “BelongsTo”,
“source”: “Test5”,
“target”: “Test6”
},
{
“relation”: “ConnectedTo”,
“source”: “Test5”,
“target”: “Test9”
},
{
“relation”: “Contains”,
“source”: “Test6”,
“target”: “Test7”
}
]
}

But due to complex structure now the requirement is to show only the related nodes to some level of depth in the below format.

Like if you will pass Test3 & depth = 2, the output should be as below only -

Note : It means it should show 2 level up and 2 level down nodes only. So,We have below relationship for this 2 level up means Test1 RunsOn Test2 & Test2 RunsOn Test3 and 2 level down means Test3 DependsOn Test4, Test4 ConnectedTo Test8 & Test4 DependsOn Test5 ie; only 5 nodes.
{
“result”: [
{
“relation”: “RunsOn”,
“source”: “Test1”,
“target”: “Test2”
},
{
“relation”: “RunsOn”,
“source”: “Test2”,
“target”: “Test3”
},
{
“relation”: “DependsOn”,
“source”: “Test3”,
“target”: “Test4”
},
{
“relation”: “DependsOn”,
“source”: “Test4”,
“target”: “Test5”
},
{
“relation”: “ConnectedTo”,
“source”: “Test4”,
“target”: “Test8”
}
]
}


#40

@Chaitanya I understand. But what are you trying to show?
Only that there is a relationship in any way? Ingoing or outgoing? Do you need a sense of direction in the result? Or only that they are connected?

Query when only following Outs in one direction (Test 2 out, depth 3):

var path = g.M().Tag('hop').Out(null, "relation");
var depth =  3;
g.V("Test2").Tag("source").FollowRecursive(path,depth).Tag("target").ForEach(function(v){
  var node = {
  source: v.hop,
  start: v.source,
  relation : v.relation,
  target: v.target
  }
  g.Emit(node)

})


#41

@Jorgen We have required both the things ie; in one part of the project we need to show the graph and in another part need to send the json reponse to the other teams for the further action (This part we are handing via java code using gizmo api).

and we have required in the way of sense of direction. if Test1 RunsOn Test2 , So in the output We want only
{
“relation”: “RunsOn”,
“source”: “Test1”,
“target”: “Test2”
}
not
{
“relation”: “RunsOn”,
“source”: “Test2”,
“target”: “Test1”
}

ie; source -> target

But it should include both up & down part of the depth.