[FR] allow JSON input in the same form as JSON parts output

[andrewcrook]

the current output JSON feature

$ trurl "https://fake.host/hello#frag" --set user=::moo:: --json
[
  {
    "url": "https://%3a%3amoo%3a%[email protected]/hello#frag",
    "parts": {
      "scheme": "https",
      "user": "::moo::",
      "host": "fake.host",
      "path": "/hello",
      "fragment": "frag"
    }
  }
]

would be useful to be able to reverse this operation it could be used with HEREDOC, files and pipes For example example (I am sure someone could come up with better args)

# HEREDOC
> trurl -stdin json   << EOL
[
  {
    "parts": {
      "scheme": "https",
      "user": "::moo::",
      "host": "fake.host",
      "path": "/hello",
      "fragment": "frag"
    }
  }
]
EOL

https://%3a%3amoo%3a%[email protected]/hello#frag

with files and pipes and plain text list of urls

# json text file
> turl -stdin json  test.json
# pipe json example
> cat test.json  |  jq |  turl -stdin json  --set user=::moo:: --json

# possible additional support for plain text urls  e.h list of urls
> cat usrlist.txt | grep ’fake.host’ |  turl -stdin text  --set user=::moo:: --json

# of course the output doesn't have to be json
> cat usrlist.txt | grep ’fake.host’ |  turl -stdin text  --set user=::moo::

or may be it makes sense to build a separate url builder tool to accompany turl?

[jacobmealey]

This is a request that has been brought up a couple of times. https://github.com/curl/trurl/issues/176#issuecomment-1527462342 the last one I could find was issue 176 where bagder recommended tabling a request for this feature unless others were interested in it.

I do think it could be a helpful feature, but we would need to add a json parsing library (which might provide an opportunity to clean up the current json code). It would be a good bit of work but would be a nice addition to trurl because I suspect we will keep getting requests for this as time goes on.

[bagder]

Agreed, it seems like a feature people will keep suggesting, meaning that there probably is merit to the idea.

[jacobmealey]

I've started looking into getting this working. it looks like json-c is a pretty popular library for this, so i've been using that. I think it makes sense for the json input to be compatible with the json output, so it right now it is an array of objects with one entry ("parts"), it could optionally have the "url" field as well but it would be ignored, and treat "parts" as the only source of url info.

With this, the most basic json input is [{ "parts" : { "host": "example.com" }].

I'm not sure how it should work from a user standpoint. it would be nice if it could all be wrapped up in -f - so you don't have to worry about adding another flag to the command line, and have trurl atomically detect if its a json file but they may get wierd if it's being used with streams, so it may just have to be --json-in for now.

Edit: it also looks like cJSON may be a good choice too, so if json-c seems to big - or more concerning has some strange compiler flags we have to set (-Wno-gnu), so im not sure which the best option is. I'm going to keep fleshing it out with json-c, switching to cJSON now wouldn't be too much work with what I have so i might just try both and see which feels better.

[bagder]

We have decided to not do JSON input. At least not until there is a common/standard JSON URL object that we would support.