Suggestion for high level usage

[def-]

It would be awesome if you could specify an object type at compile-time and could have the Msg automatically recursively unwrapped into this object.

Something like this:

let x = file.unpack.unwrapInto(seq[string])

type Foo = object
  x: int
  name: string
  numbers: seq[int]

let y = file.unpack.unwrapInto(Foo)

[def-]

Here's how I currently unpack some complex data coming from Python: https://gist.github.com/def-/271dcd1baf5ae24d4a53

[akiradeveloper]

Interesting.

But how to distinguish seq[tuple[key, val: int]] as Map and seq[tuple[key, val: int]] as an array of tuple (int, int) annoys me. The former is [header, k1, v1, k2, v2, ...] but the latter is [header, [header, k1, v1], [header, k2, v2], ...]. They aren't identical although the types are.

The problem is how to make a Unwrapper from a type T. I think this can be done with meta-programming if we ignore the issue I mentioned above.

FYI, msgpack-haskell (https://github.com/msgpack/msgpack-haskell) probably do this by declaring Packable/Unpackable type classes but I am not sure how this solves the issue above and the technique is doable with Nim.

I will keep thinking.

[akiradeveloper]

I wrote the following code for experimental purpose. I think divide-and-conquer is a good approach in this case. This approach is by defining intermediate type called Object.

If we can

1. Generate Seq(Int) from seq[int]
2. make final value of type seq[int] from the Object instance

Then the unwrapInto is available but I am not sure how the implementation would be. No.1 seems to be viable by some macro programing but the No.2 doesn't for me.

import msgpack

type
  ObjectKind = enum
    kInt
    kSeq
  Object = ref ObjectObj
  ObjectObj {.acyclic.} = object
    case kind: ObjectKind
    of kInt: vInt: int
    of kSeq: vSeq: tuple[typ: Object, val: seq[Object]]

proc `$`(o: Object): string =
  $(o[])

type Unwrapper = proc (m: Msg): Object {.closure.}

let Int: Object =
  Object(kind: kInt)
proc Seq(t: Object): Object =
  Object(kind: kSeq, vSeq: (t, nil))

proc GenValue(o: Object): Unwrapper =
  case o.kind:
  of kInt:
    return proc (m: Msg): Object =
      let i = unwrapInt(m)
      return Object(kind: kInt, vInt: i)
  of kSeq:
    let typ = o.vSeq.typ
    return proc (m: Msg): Object =
      let v = m.unwrapArray.map(GenValue(typ))
      return Object(kind: kSeq, vSeq: (typ, v))
  else:
    discard

when isMainModule:
  let gen1 = GenValue(Int)
  let m1 = PFixNum(5)
  let v1 = gen1(m1)
  echo expr(v1)

  let gen2 = GenValue(Seq(Int))
  let m2 = Array16(@[Int8(1), UInt8(1)])
  let v2 = gen2(m2)
  echo expr(v2)

output:

(kind: kInt, vInt: 5)
(kind: kSeq, vSeq: (typ: (kind: kInt, vInt: 0), val: @[(kind: kInt, vInt: 1), (kind: kInt, vInt: 1)]))

[akiradeveloper]

Another experiment: Introduce Unwrappable type class but somehow the compiler fails with SEGV. I am not sure if this is compiler bug.

mport msgpack

type Unwrappable[T] = generic x
  x is Msg
  unwrap(x) is T

proc unwrap(x: Unwrappable[int]): int =
  x.unwrapInt

proc unwrap(x: Unwrappable[string]): string =
  x.unwrapString

proc unwrap[T](x: Unwrappable[seq[T]]): seq[T] =
  x.unwrapArray.map(unwrap)

if isMainModule:
  let m1 = wrap(1)
  echo unwrap(m1)

  let m2 = wrap("a")
  echo unwrap(m2)

  let m3 = wrap(@[1,2])
  echo unwrap(m3)

[def-]

I think typeclasses are too experimental for this. The way to go would be a macro that analyzes the passed type and builds the code to parse it based on that.

[akiradeveloper]

I like to write this with recursion like Wrappable type class and wrap which I introduced yesterday but macro doesn't allow recursion . But yeah, I will try by macro for experiment.

I think if there are overloading by return type, we may come little closer to general unwrapping. However, it's not supported by Nim compiler too (but I think Araq is trying to do?).

[def-]

macro doesn't allow recursion

Make the macro call a compile time proc. compile time procs can recurse.

[def-]

This looks like what I asked for: https://github.com/jangko/msgpack4nim

[akiradeveloper]

Oh, this looks even greater than mine. It directly packs nim objects into Stream without once making Msg objects. This means it's faster if it is the usage. To be honest, I don't have much time to get back to msgpack-nim right now so the functionality you are asking for is going to appear very unlikely in near term. Sorry, but I recommend you to use Jangko's version.

On 2015/05/21 21:07, Dennis Felsing wrote:

This looks like what I asked for: https://github.com/jangko/msgpack4nim

---

Reply to this email directly or view it on GitHub: https://github.com/akiradeveloper/msgpack-nim/issues/4#issuecomment-104248586

[def-]

No problem, just wanted to show you that another msgpack library exists now.

[akiradeveloper]

I see. Just use fields and fieldPairs. No need to write crazy macro.

proc pack*[T: tuple|object](s: Stream, val: T) =
  var len = 0
  for field in fields(val):
    inc(len)

  when defined(msgpack_obj_to_map):
    s.pack_map(len)
    for field, value in fieldPairs(val):
      s.pack field
      s.pack value
  else:
    s.pack_array(len)
    for field in fields(val):
      s.pack field

proc pack*[T: ref](s: Stream, val: T) =
  if isNil(val): s.pack_imp_nil()
  else:
    let vald = val[]
    var len = 0
    for field in fields(vald):
      inc(len)

    when defined(msgpack_obj_to_map):
      s.pack_map(len)
      for field, value in fieldPairs(vald):
        s.pack field
        s.pack value
    else:
      s.pack_array(len)
      for field in fields(vald):
        s.pack field