request host, port, and scheme all missing when running localhost

[alanvoss]

when implementing my handle function on localhost, I was hoping to be able to utilize scheme, host, and port. I keep getting undefined. I admittedly am not super slick with erlang, but the rest of the application works correctly (when getting raw_path, for example, everything is as expected).

https://github.com/elli-lib/elli/blob/21e2eeb5c718d42aca4a7014db488b4b47799450/src/elli_request.erl#L62-L67

All are undefined when this runs inside my server (though method properly returns).

 handle(Req, [Secrets, VendorConfigs, AvroEncoders]) ->
+    io:fwrite("~1p is the method~n",  [Req#req.method]),
+    io:fwrite("~1p is the host~n",    [Req#req.host]),
+    io:fwrite("~1p is the scheme~n",  [Req#req.scheme]),
+    io:fwrite("~1p is the port~n",    [Req#req.port]),

result:

'POST' is the method
undefined is the host
undefined is the scheme
undefined is the port

calling using curl:

curl -H -v http://localhost:3000/v2/my/path -d '{"data":"1113"}'

System specs:

MacOS Catalina, 10.15.4

Erlang installed using https://github.com/asdf-vm/asdf :

$ cat plugins/erlang/kerl-home/.kerlrc
KERL_CONFIGURE_OPTIONS="--without-javac --with-ssl=/usr/local/opt/[email protected] --enable-wx --with-wx-config=/usr/local/bin/wx-config"

$ erl --version
Erlang/OTP 22 [erts-10.4.4] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [hipe]

$ uname -a
Darwin alanvoss 19.4.0 Darwin Kernel Version 19.4.0: Wed Mar  4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64 x86_64

[tsloughter]

Thanks, Alan!

I asked him to open this here even though I think the underlying problem is likely in Erlang's packet parser since Elli just does:

parse_path({abs_path, FullPath}) ->
    Parsed = case binary:split(FullPath, [<<"?">>]) of
                 [URL]       -> {FullPath, split_path(URL), []};
                 [URL, Args] -> {FullPath, split_path(URL), split_args(Args)}
             end,
    {ok, {undefined, undefined, undefined}, Parsed};
parse_path({absoluteURI, Scheme, Host, Port, Path}) ->
    setelement(2, parse_path({abs_path, Path}), {Scheme, Host, Port});
parse_path(_) ->
    {error, unsupported_uri}.

So if Erlang packet parser is incorrectly returning {abs_path, Path} instead of absoluteURI this would happen and I think looks to be the only way it would happen.

But will need to verify this and could still be an Elli issue.

[yurrriq]

Thanks! I can have a look sometime this week.

[yurrriq]

Wishful thinking, it seems. Will try for this week. Feel free to beat me to it.

[jeffgrunewald]

For what it's worth, this appears to be related to the contents of the packet being passed back from the prim_inet module not containing the scheme, host, or port information. I've tested this locally on MacOS running Big Sur as well as in a container (either Ubuntu or Debian) and a POST request to a local endpoint "http://localhost:4000/create" in a test app results in a packet starting with <<"POST /create HTTP/1.1...">> which decodes to:

{ok, {http_request, 'POST', {abs_path, "/create"}, {1, 1}...}}

if i manually modify the same packet to pre-pend the missing info to the path: <<"POST http://localhost:4000/create HTTP/1.1...">> it decodes as I would expect to:

{ok, {http_request, 'POST', {absoluteURI, http, "localhost", 4000, "/create"}, {1, 1}...}}

I tried to trace this back to the source but best I can tell, it appears to be coming from the C message responses generated in the erts/emulator/drivers/common/inet_drv.c but that's above my pay grade.

[tsloughter]

Oh weird, thanks @jeffgrunewald

I wonder if we are supposed to just infer "localhost" if nothing else is there.

I was going to say the port could be taken from the socket itself but it should probably be the port in the URL and not the port being listened on, since those could technically be different.

[jeffgrunewald]

Where should we get the URL from if it's not able to be parsed from the packet? The host entry in the headers list? Is that something we could reliably count on to be present in a request from any given client?

[jeffgrunewald]

FWIW, it looks like Cowboy attempts to get around this issue by primarily pulling the host from the clients’ host header value and raising an exception if it’s not set, referring to the value being required by RFC7230 5.4: https://github.com/ninenines/cowboy/blob/2a082504996afccf78185182cd168123800bd4f0/src/cowboy_http.erl#L717

[tsloughter]

Thanks @jeffgrunewald

I opened a PR to have scheme get set. I guess we can go with host header for how to set the port and host.