[Help Wanted] Bringing LookupByKey functionality to Go

[Nickersoft]

Hey FlatBuffer folks!

A little over a year ago I came on here asking about plans to add key searching to the Go generator. I had been meaning to set some time aside to see if I could pick it up, and well... I finally managed to.

I've ported over the core Go methods required in my own branch, and have converted the main lookupByKey method to Go:

func lookupByKey(obj *MyStruct, vector flatbuffers.UOffsetT, k string, buf []byte) bool {
	key := []byte(k)
	span := flatbuffers.GetUOffsetT(buf[vector - 4:])
	start := flatbuffers.UOffsetT(0)
	for ok := true; ok; ok = span != 0 {
		middle := span / 2
		tableOffset := Indirect(vector + 4 * (start + middle), buf)
		comp := Compare(flatbuffers.UOffsetT(Offset(4, flatbuffers.UOffsetT(len(buf)) - tableOffset, buf)), key, buf)
		if comp > 0 {
			span = middle
		} else if comp < 0 {
			middle += 1
			start += middle
			span -= middle
		} else {
			obj.Init(buf, tableOffset)
			return true
		}
	}
	return false
}

And the accessor that uses it:

func (rcv *Dictionary) MyStructByKey(obj *MyStruct, key string) bool {
	o := flatbuffers.UOffsetT(rcv._tab.Offset(8))
	if o != 0 {
		return lookupByKey(obj, rcv._tab.Vector(o), key, rcv._tab.Bytes)
	}
	return false
}

I was able to confirm it working against my own buffers.

Unfortunately, I'm still at a bit of a loss when trying to adapt this to the main generator in idl_gen_go.cpp. Apart from C++ just not being a strong suite of mine, I find the code a bit hard to navigate, and the structure of more recent C++ files (such as the Go and Python) generators are more modularized than older ones which contain code for key lookups (Java, C++, Swift, etc.). So I can't exactly blindly copy and paste between the two haha.

I'm curious if anyone is willing to try to meet me halfway to help get this feature out, or at least offer some guidance regarding how the generators are set up.

Would love to be able to get a PR open for this :)

[dbaileychess]

@Nickersoft I can probably help at some point if there are no other offers.

[Nickersoft]

That'd be fantastic @dbaileychess! Let me know when/how you'd like to coordinate. I ended up having to make some modifications to the code I posted above, as I wasn't aware of the difference between the various OffsetT types and it turned out my lookup code wouldn't work on large buffers, as it would max out the integer types.

My latest code is here. For now I've just manually extended my generated code in my Go repo.

[le-michael]

Sorry @Nickersoft didn't see you were working on this.

I have an implementation for this under review in #7644.

[le-michael]

@dbaileychess can we close this?

[Nickersoft]

I'd say so

[Nickersoft]

Hey @le-michael, I just got around to testing the new LookupByKey you added (thanks again btw!) and am getting this error preventing my code from compiling:

It looks like the generated file isn't importing the bytes package

[le-michael]

Hm I was able to reproduce this when using the --gen-onefile flag. Are you using that by any chance?

[Nickersoft]

I am! So it doesn't happen without the flag? Interesting.. as a workaround right now, I'm just manually adding the bytes import and not regenerating my schema on build.

[le-michael]

Yeah it's a bug I'll send out a fix for this.