caddyhttp: Added plaintext capability to `file_server` `browser`

[kylosus]

Sorry about the draft and edits, I accidentally published the PR too early.

So this is kind of a niche usecase of mine, so I understand if the collaborators find it too specific to merge. I can maybe write a module instead if other users are interested in the feature.

I have been using caddy for hosting a few things, and one of them is a static file server with convenience scripts that I use in fresh headless machines. I usually just type out the whole URL with the specific files and do curl whatever.sh | sh, but I have found it a little tedious, especially because I forget what I have available sometimes or mix names up, and fetching the big html in the index is not an option. file_server has an undocomented JSON output feature that makes things easier, but it's still a few extra steps to type out an Accept header and parse the JSON.

This PR is twofold: First it adds a new optional field, return_type, to browser for setting the default format of the returned index (html, json or plaintext). This is used when the Accept header is set to /*.

Second, it adds a preliminary text/plain support to the file_server browser that returns a text representation of the file system, when an Accept: text/plain header is present, with the behavior discussed above. This makes it possible get a textual view of the available files quickly in a terminal environment.

Example Caddyfile config:

scripts.localhost {
    root * /srv/caddyscripts/

    @curl {
        header_regexp User-Agent (?i)(curl)
        path /scripts/*
    }

    # Default to plaintext for curl
    file_server @curl {
        browse {
            return_type plain
        }
    }

    file_server /scripts/* browse
}

I made return_type a configurable option to set the default as it allows setting up matchers to serve the index depending on other parts of the request, for instance checking if curl headers are present vs a request from a web browser.

The commit in this PR is a working proof of concept that just returns a newline delimited list of files. I will write a more useful and aesthetically pleaasing version if the collaborators like the feature.

[CLAassistant]

All committers have signed the CLA.

[mholt]

Forgive me if you explained this already at least implicitly, but you can set your own browse template. Why not just make a plaintext template?

[kylosus]

I actually didn't consider using a template because it didn't sound like it was meant to be used in this way, but testing it now, I can kind of replicate what I want to achieve with no extraneous html tags. The template is still parsed and served as an html resource, however, so it's displayed inconsistently - the content is interpreted as the html body in the browser, which ignores newlines (and possibly other formatting), and the line breaks and spaces around template functions mess up the output in the terminal when curl'ing. Additionally, characters in the template are escaped with html entites, which, again, won't be displayed correctly as plaintext outside of a browser environment.

These issues aside, I think it's useful to have this capability in file_browser anyway as it makes things a lot simpler - you can just set the Accept header to get the representation you want without having to write a whole template.

[francislavoie]

I think this is pretty cool.

I could imagine we'd want to add ?query params to request the content includes the file sizes or modified time, like you'd see with the HTML & JSON. Like ?size=1&time=1 or something :sweat_smile: Could be displayed similarly to ls, with size & time first, with tab delimiters for alignment.

FWIW, you can use | jq in your terminal with the JSON response to produce the same output:

curl -s localhost -H'Accept: application/json' | jq -r .[].name

[kylosus]

I was thinking of including these in the plaintext output - basically like a plaintext version of the default html template - but can do query parameters instead too.

[mholt]

@kylosus

I actually didn't consider using a template because it didn't sound like it was meant to be used in this way, but testing it now, I can kind of replicate what I want to achieve with no extraneous html tags. The template is still parsed and served as an html resource,

Ah, indeed we do set the Content-Type to be text/html, but I wonder if you can override that with your own header Content-Type text/plain directive?

Anyway, the idea of plaintext output is growing on me -- I can see the utility in a terminal -- I just want to ensure it's done as simple as possible (for now) -- as this is the only request we've had for it I am not sure how niche it is. Who knows, maybe it could be awesome though.

[kylosus]

Here's some workflow

https://github.com/caddyserver/caddy/assets/33132401/11c4aa45-e379-4cf4-a036-951a9bc61a2c

Maybe even some icons?

[kylosus]

Sorry it took me a while to get around to this. That is pretty neat. How did you get the icons to show up?

No worries, I was busy myself and couldn't respond for some time.

The icons are unicode icon glyphs and need a supported font to be available on the system to display properly (instead of the unicode replacement character). I am using Nerd Fonts.

[mholt]

@kylosus I went ahead and fixed up the lint problems (I think) and simplified the implementation a bit, hope that's ok.

I noticed the config option to set a default return type? I actually removed that since I think a better way is using matchers, for example:

@curl header User-Agent *curl*
request_header @curl Accept text/plain
file_server browse

This serves plaintext response to curl clients, but still serves HTML for browsers. Whereas previously browsers would also be served plaintext instead of HTML.

I think the line/filetype icons are pretty cool. Unfortunately most terminals won't render them :( If they could be used so that they only appear if supported (and simply disappear if not), then I'd be down for that.

Thanks for the contribution!

[kylosus]

Oh, that's a lot nicer, thanks for simplifying the code. I remember trying to set the Accept header like that but it didn't seem to work - I might've been on an older version.

Infortunately I don't think there's a way to hide the icons completely on unsupported terminals, but I think the black box unicode substitution character is an acceptable compromise. Up to you though.

[mholt]

I'm not too concerned about the icons; we can start with this and maybe add some later if we want :)