Has anybody here ever played with a graph database (NoSQL lols in this thread)

Message Bookmarked
Bookmark Removed

http://www.slideshare.net/emileifrem/neo4j-the-benefits-of-graph-databases-oscon-2009

So I'm probably going to pitch a couple of dudes a ridiculous idea this week and figured why not ask ilcomputers while I'm at it:

- starting with neo4j 'cause it has a wiki already and every other graph DB seems to be strictly commercial/proprietary

- want to begin with 4 billion nodes (one for every IPv4 address)

- relate those nodes to "label" nodes, to form lists (whitelists, blacklists, country codes, ASNs, etc); millions of label nodes possible

- relate those lists to "source" nodes (users, cron jobs, websites) and the sources into source families

- tricky part: relate multiple nodes to "timestamp" nodes, so that a unix time can be associated with an IP' nodes relationship to a label node, e.g. say we have a label node that is just a domain name, and a certain list of addressed resolved from that domain at x time, but did not as of y time (this can be done with a traverser. I am pretty sure.)

- further along down the road, additional relationships for other attributes describing the various nodes; the idea ultimately being I can do almost all of my homework on a network device using a couple of traversals instead of essentially just grepping billions of lines of binary data

So aside from a few presentations and anecdotes from people I definitely don't know, has anybody here worked with Neo4j, or something like it? What eventually kills a graph DB? What do I need to watch out for?

El Tomboto, Sunday, 15 November 2009 05:41 (nine years ago) Permalink

also is it even worth pursuing hadoop & cascading if I only roll with about 50 gigabytes of binfiles a day

El Tomboto, Sunday, 15 November 2009 05:53 (nine years ago) Permalink

hi

ice cr?m, Sunday, 15 November 2009 05:54 (nine years ago) Permalink

hey how you been

El Tomboto, Sunday, 15 November 2009 05:56 (nine years ago) Permalink

o good you know ate some noodle earlier - yrself

ice cr?m, Sunday, 15 November 2009 05:59 (nine years ago) Permalink

busy

El Tomboto, Sunday, 15 November 2009 06:01 (nine years ago) Permalink

cool - i dont really know anything abt this particular technology but i think its sweet u want to have four billion nodes

ice cr?m, Sunday, 15 November 2009 06:03 (nine years ago) Permalink

the od(bms) couple

Louis Cll (darraghmac), Sunday, 15 November 2009 06:07 (nine years ago) Permalink

keeping this b/w us, if you can figure out a way to turn nodes into dollars....oh man

peed on tree (k3vin k.), Sunday, 15 November 2009 06:07 (nine years ago) Permalink

xp

peed on tree (k3vin k.), Sunday, 15 November 2009 06:08 (nine years ago) Permalink

what up tom

house of flying jaggers (J0rdan S.), Sunday, 15 November 2009 06:36 (nine years ago) Permalink

So I'm probably going to pitch a couple of nodes a ridiculous idea this week and figured why not ask ilcomputers while I'm at it:

- starting with neo4j 'cause it has a wiki already and every other graph DB seems to be strictly commercial/proprietary

- want to begin with 4 billion dudes (one for every IPv4 address)

- relate those dudes to "label" dudes, to form lists (whitelists, blacklists, country codes, ASNs, etc); millions of label dudes possible

- relate those lists to "source" dudes (users, cron jobs, websites) and the sources into source families

- tricky part: relate multiple dudes to "timestamp" dudes, so that a unix time can be associated with an IP' dudes relationship to a label dude, e.g. say we have a label dude that is just a domain name, and a certain list of addresses resolved from that domain at x time, but did not as of y time (this can be done with a traverser. I am pretty sure.)

- further along down the road, additional relationships for other attributes describing the various dudes; the idea ultimately being I can do almost all of my homework on a network device using a couple of traversals instead of essentially just grepping billions of lines of binary data

El Tomboto, Sunday, 15 November 2009 07:32 (nine years ago) Permalink

totally ignorant in this particular matter, sorry dude.

caek, Sunday, 15 November 2009 13:01 (nine years ago) Permalink

lol wats a node

luol deng (am0n), Monday, 16 November 2009 16:19 (nine years ago) Permalink

I have never played with a graph database but what you're describing is making me want to play with one.

a Barbie-like nub where he provates should be (HI DERE), Monday, 16 November 2009 16:22 (nine years ago) Permalink

good luck tom

max, Monday, 16 November 2009 16:23 (nine years ago) Permalink

oh and btw I'm all up in yr HOOD next week

a Barbie-like nub where he provates should be (HI DERE), Monday, 16 November 2009 16:23 (nine years ago) Permalink

marko rodriguez's deck about graph DBs seems like a good overview (to the eye of this particular dangerous amateur) - I'm gonna holler at our grant mofos to see if there's existing research in this area that we can piggyback on

El Tomboto, Tuesday, 17 November 2009 08:04 (nine years ago) Permalink

four months pass...

anybody know anything about netcharts? fucking netcharts, man.

Cary grant angry at tits (gnarly sceptre), Wednesday, 31 March 2010 19:32 (eight years ago) Permalink


You must be logged in to post. Please either login here, or if you are not registered, you may register here.