relang

RethinkDB driver in Erlang

Build status

I used wercker to run test on my docker image relang

Why

I want to learn more about RethinkDB to work on my RethinkDB book Simply RethinkDB

Credit

My implementation is very simple and maybe buggy but I inspied by https://github.com/taybin/lethink especially on the query syntax.

Compile

rebar get-deps
rebar compile
erl -pa ebin -pa deps/protobuffs/ebin deps/jsx/ebin

Using

It's quite hard for me(an amateur Erlang programmer) to represent JSON in Erlang world. Later Erlang version has map feature which also help at some point so I opt-in to use them.

The downside is that relang won't run on old Erlang compiler. So keep that in mind and please submit PR to improve syntax, make it easier to read.

%% Creation conection
Connection = relang:connect("127.0.0.1")

relang:r(Connection, [{db_create, "db1}])
relang:r(Connection, [{db_list}])
relang:r(Connection, [{db, ["test"]}, {table_list}])
relang:r(Connection, [{db, ["test"]}, {table_create, ["t1"]}])
relang:r(Connection, [{db, ["test"]}, {table, ["t4"]}, {insert, ["{\"name\":\"vinh\",\"age\":27}"]}]).

Ideally instead of chaining function like in Ruby, we put the query into a list of tuples. I want and I like chaining style but I don't know how to do that in Erlang.

Each of tuples of query includes one to 3 element:

{operation, argument, options}

operation is the name that try to match official Ruby driver. Such as

{table_create, argument, options}
{insert, argument, options}

Argument list element can be anything, from tuple to function when the anonymous function is passed.

Argument is a list, if you pass a tuple, the driver attempt to convert it to a list. How many items of the list is depends on how many item the operation accept.

Some special function has different syntax such as changefeed and filter because they are a bit different.

Cursor

some queries don't return all data. But only a partial of data. we can iterator over the result set use a concept call cursor

{ok, {cursor, Cursor}, _} = relang:r(Connection, [[{db, ["test"]}, {table, ["t4"]}),

next

next can works with a callback

relang:next(cursor, fun(V) -> io:format("Value ~p", [V] end))

or a process

Pid = spawn(...)
relang:next(cursor, Pid)

Changefeeds

relang:r(Connection, [[{db, ["test"]}, {table, ["t4"]}, {change, fun(Item) -> io:format(Item) end}).

• Limit: Only a change command in the list

Filter and row

On the surface, filter looks like they are code that run on driver side, but actually they are serialized and pass to the server for evaluate.

Depend on driver, the syntax of using row with filter is different. Here is how we do it in Erlang:

With exactly match.

l(relang). l(relang_ast). l(log).
relang:r(relang:connect("127.0.0.1"),
  [ {db, [<<"test">>]},
    {table, [<<"tv_shows">>]},
    {filter, [
      [{<<"age">>, 30},
      {<<"name">>, <<"kurei">>},
      {<<"show">>, 1}]
    ]}
  ]
).

It's not powerful enough so we come up With functional style

relang:r(relang:connect(),
[{db, [<<"test">>]}, {table, [<<"tv_shows">>]}, 
                  {filter, fun(X) ->
                               [
                                  {'and', [
                                           {gt, [{field, [X, <<"age">>]}, 22]},
                                           {lt, [{field, [X, <<"age">>]}, 25]},
                                           {match, [{field, [X, <<"name">>]},  <<"^k">>]}
                                          ]}
                                 ]
                           end}]
).

relang:r(C, [{db, [<<"test">>]}, {table,
[<<"tv_shows">>]}, {filter, fun(X) ->
  [
    {'and', [
      {gt, [{field, [X, <<"age">>]}, 22]},
      {lt, [{field, [X, <<"age">>]}, 25]}
    ]}
  ]
end}]).

# find user 22 -> 25 of age, name starts with `k`, and opt-in to `show`
relang:r(C, [{db, [<<"test">>]}, {table,
[<<"tv_shows">>]}, {filter, fun(X) ->
  [
    {'and', [
      {gt, [{field, [X, <<"age">>]}, 22]},
      {lt, [{field, [X, <<"age">>]}, 25]},
      {match, [{field, [X, <<"name">>]},  <<"^k">>]},
      {field, <<"show">>
    ]}
  ]
end}]).

l(relang). l(relang_ast). l(log).
relang:r(relang:connect("127.0.0.1"), [{db, [<<"test">>]}, {table,
[<<"tv_shows">>]}, {filter, fun(X) ->
  [
    {field, [X, <<"show">>]}
  ]
end}]).

Expr(WIP)

relang:r(Connection, [{expr(2)}]).

API

relang:r(relang:connect("127.0.0.1"), [{now}]).

Connect

C1 = relang:connect("127.0.0.1")

Manipulating tables

table_create

relang:r(relang:connect(), [{table_create, geo}]).
% or with db
relang:r(relang:connect(), [{db, test}, {table_create, geo}]).

table_drop

relang:r(relang:connect(), [{table_drop, geo}]).
% or with db
relang:r(relang:connect(), [{db, test}, {table_drop, geo}]).

Selecting data

db

table

relang:r(C1, [{db, [<<"test">>]},  {table, <<"tv_shows">>}]).

get

relang:r(C1, [{db, [<<"test">>]},  {table, <<"tv_shows">>}, {get, <<"primarykey">>]).

Nested field

A ReQL document is a JSON object: a set of key-value pairs, in which each value might be a single value, a list of values, or another set of key-value pairs. When the value of a field contains more fields, we describe these as nested fields.

We access nested field, or indicate it by using a list, to denote a path.

relang:r(C1, 
  [
    {db, [<<"test">>]},
    {table, <<"user">>},
    {get, [<<"primarykey">>, {field: []}]}
  ]
  ).

filter

Reference filter below because they have a different syntax.

Joins

Inner joins

relang:r(relang:connect(),
  [{db, [<<"foodb">>]},
    {table, <<"tv_shows">>},
    {eq_join,
      [<<"compound_id">>,
        [{db, [<<"foodb">>]}, {table, <<"compounds">>}]
      ]
    }
  ]
).

Equal join

Simple form, join use a column on left table whose value is equal to index on right table. Using primary index by default

relang:r(relang:connect(),
  [{db, [<<"foodb">>]},
    {table, <<"tv_shows">>},
    {eq_join,
      [<<"compound_id">>,
        [{db, [<<"foodb">>]}, {table, <<"compounds">>}]
      ]
    }
  ]
).

Or using a secondary index

relang:r(relang:connect(),
  [{db, [<<"foodb">>]},
    {table, <<"tv_shows">>},
    {eq_join,
      [<<"compound_id">>,
        [{db, [<<"foodb">>]}, {table, <<"compounds">>}]
      ],
      [{<<"index">>, <<"different_index">>}]
    }
  ]
).

@TODO Or using function instead of string

l(relang). l(relang_ast). l(log).
relang:r(relang:connect(),
  [{db, [<<"foodb">>]},
    {table, <<"tv_shows">>},
    {eq_join,
      [
        fun (X) ->
            [
              {field, {field, [X, <<"Parent">>]}, <<"Sub">>}
            ]
        end,
        [{table, <<"compounds">>}]
      ]
      ,
      [{<<"index">>, <<"different_index">>}]
    }
  ]
).

More complex expression

l(relang). l(relang_ast). l(log).
relang:r(relang:connect(),
  [{db, [<<"foodb">>]},
    {table, <<"tv_shows">>},
    {eq_join,
      [
        fun (X) ->
            [
              {field, {field, [X, <<"Parent">>]}, <<"Sub">>},
               {nth, 20}
            ]
        end,
        [{table, <<"compounds">>}]
      ]
      ,
      [{<<"index">>, <<"different_index">>}]
    }
  ]
).

zip

Using with eq_join

l(relang). l(relang_ast). l(log).
relang:r(relang:connect(),
  [{db, [<<"foodb">>]},
    {table, <<"compounds_foods">>},
    {eq_join,
      [<<"compound_id">>,
        [{db, [<<"foodb">>]}, {table, <<"compounds">>}]
      ],
      [{<<"index">>, <<"second_index">>}]
    },
    {zip}
  ]
).

Or without index:

l(relang). l(relang_ast). l(log).
relang:r(relang:connect(),
  [{db, [<<"foodb">>]},
    {table, <<"compounds_foods">>},
    {eq_join,
      [<<"compound_id">>,
        [{db, [<<"foodb">>]}, {table, <<"compounds">>}]
      ]
    },
    {zip}
  ]
).

Transformation

nth

relang:r(relang:connect(),
  [
    {table, <<"tv_shows">>},
     {nth, 120}
  ]
).

Writing data

Insert

C1 = relang:connect("127.0.0.1")
relang:r(C1, [{db, [<<"test">>]},  {table, <<"tv_shows">>}, {insert, [[{<<"name">>, <<"kurei">>}, {<<"age">>, <<28>>}]]}])

With nested field,

C1 = relang:connect("127.0.0.1")

Update

relang:r(C1, [{db, [<<"test">>]},  {table, <<"tv_shows">>}, {get, <<"6b443331-d7c9-4304-867d-251db183446f">>}, {update, [[{<<"name">>, <<"kurei kain">>}, {<<"age">>, <<29>>}]]}])

% Or update with option
relang:r(C1,
  [{db, [<<"test">>]},
  {table, <<"tv_shows">>},
  {get, <<"6b443331-d7c9-4304-867d-251db183446f">>},
  {update,
    [[{<<"name">>, <<"kurei kain">>},
    {<<"age">>, <<29>>}]],
    [{<<"durability">>, soft}, {return_changes, false}]
  }
  ])

% Or update nested field
relang:r(relang:connect(),
  [
    {table, users},
    {get, 10001},
    {update, [
              [
               {contact, [{phone, [{cell, <<"408-555-4242">>}]}]}
              ]
             ]}
  ]
).

% Or update with function 
relang:r(relang:connect(),
  [
    {table, posts},
    {update,
      fun(Post) ->
        [{
          views,
            relang:r([
                {field, [Post, views]},
                {add, 100},
                {default, 20}
              ])
        }]
      end
    }
  ]).

Aggregation

Count

relang:query(C, [ {db, [<<"test">>]}, {table, [<<"tv_shows">>]}, {count}]).

Document manipulation

get_field

relang:r(C1,
  [
    {db, [<<"test">>]},
    {table, <<"wall_posts">>},
    {get, <<"primarykey">>},
    {get_field, <<"field">>},
    {get_field, <<"sub_field">>}
  ]).

Or

relang:r(C1,
  [
    {db, [<<"test">>]},
    {table, <<"wall_posts">>},
    {get_field, <<"id">>}
  ]).

keys

relang:r(relang:connect(),
  [
    {db, [<<"test">>]},
    {table, <<"dummy">>},
    {get, <<"7541a1ed-20ae-42f2-b7ea-73fbeb668d07">>},
    {keys}
  ]).
%%
%% Ok {"t":1,"r":[["f","id"]],"n":[]}atom response{ok,[[<<"f">>,<<"id">>]]}

object

relang:r(relang:connect(),  [{object, [<<"k1">>, 1, <<"k2">>, 2]}]).

Geospatial commands

circle

l(relang). l(relang_ast). l(log).
relang:r(relang:connect(), [{circle, [{-122.423246, 37.779388}, 1000]}]).

distance

relang:r(relang:connect(), 
  [
    {distance, 
      [
        relang:r([{point, [-122.423246,37.779388]}]),
        relang:r([{point, [-117.220406,32.719464]}])
      ],
      [{unit, km}]
    }
  ]).

fill

Convert a Line object into a Polygon object. If the last point does not specify the same coordinates as the first point, polygon will close the polygon by connecting them.

Example: Create a line object and then convert it to a polygon.

relang:r(relang:connect(),
  [
    {table, geo},
    {insert, [[
      {id, 201},
      {rectangle, relang:r([
        {line, [
          [-122.423246,37.779388],
          [-122.423246,37.329898],
          [-121.886420,37.329898],
          [-121.886420,37.779388]
        ]}
      ])}
    ]]}
  ]
).

% Try to select it back, for fun :)
relang:r(relang:connect(),
  [
    {table, geo},
    {get, 201}
  ]
).

% using fill to turn it into plolygon
l(relang). l(relang_ast). l(log).
relang:r(relang:connect(),
  [
    {table, geo},
    {get, 201},
    {update,
      fun(Doc) ->
        [{
          rectangle,
            relang:r([
                {field, [Doc, rectangle]},
                {fill}
              ])
        }]
      end,
      [{non_atomic, true}]
    }
  ]
).

geojson

T = [{type,'Point'},
       {coordinates, [ -122.423246, 37.779388 ]
       }
      ].
relang:r(relang:connect(), [{geojson, T]).

Another complex example:

l(relang). l(relang_ast). l(log).
T = [{type,'Point'},
     {coordinates, [ -122.423246, 37.779388 ]
     }
    ]
    .
Q = [
     {table, geo},
     {insert, [[
      {id, sfo},
      {name, <<"San Francisco">>},
      {location, relang:r([{geojson, T}])}
              ]]}
    ].
relang:r(relang:connect(), Q).

to_geojson

get_intersecting

get_nearest

includes

intersects

line

relang:r([
   {line,
    [
     [-122.423246,37.779388],
     [-121.886420,37.329898]
    ]
   }
  ]).

or using in other expression:

relang:r(relang:connect(),   [
    {table, geo},
    {insert, [[
      {id, 101},
      {route,
        relang:r([
          {line, [
            [-122.423246,37.779388], [-121.886420,37.329898]
          ]}
        ])
      }
    ]]}
  ]
).

point

l(relang). l(relang_ast). l(log).
relang:r([{point, [-122.423246, 37.779388]}]).

polygon

l(relang). l(relang_ast). l(log).
relang:r(relang:connect(),  [{polygon,
        [
          [-122.423246,37.779388],
          [-122.423246,37.329898],
          [-121.886420,37.329898],
          [-121.886420,37.779388]
        ]
       }
      ]
).

Or using in other expression

(relang). l(relang_ast). l(log).
relang:r(relang:connect(),     [
      {table, geo},
      {insert, [[
        {id, 101},
        {rectangle, relang:r([{polygon,
            [
              [-122.423246,37.779388],
              [-122.423246,37.329898],
              [-121.886420,37.329898],
              [-121.886420,37.779388]
            ]
           }
          ])
        }
      ]]}
    ]
).

polygon_sub

Development

Make sure to use tcpdump during development for ReQL inspect

tcpdump -nl -w - -i lo0 -c 500 port 28015|strings

Once compile, we can get into REPl

erl -pa ebin -pa deps/protobuffs/ebin deps/jsx/ebin

Test

rebar eu

Integration test with Docker (pending)