Static file helper

[vsergeev]

More of a request than an issue, but it'd be nice to have a route helper for static files, e.g. https://flask.palletsprojects.com/en/stable/quickstart/#static-files . Of course, users could open a file and stream it themselves in a request handler, but a native static file helper would 1) expedite a very common use case, 2) allow everyone to benefit from various optimizations (mmap, caching, etc.), 3) offer more safety compared to a user's implementation.

This might be a rabbit hole, but I'd be happy with a handful of totally static paths (not directories, wildcards, etc.).

For my use case, 99% is a websocket server, and otherwise I need to deliver 2-3 bundled files to serve the frontend.

[karlseguin]

I do find this a bit of a rabbit hole, specifically with respect to caching. It's a spectrum and I'm not sure what would be best to offer out-of-the-box.

Should they be really static, loaded once on startup (into memory) and then served as-is? Should they be cached? Or should they be served off-disk on every request?

I guess I could see something like:

router.get("/main.js", staticFile("path/to/main.js", .{
   .cache = 300,
   .content_type = "application/json",
});

Cache could be nullable, 0 = always serve from disk, null = cache forever (probably eagerly load on startup in this case), N = # of seconds.

Is that the type of thing you were thinking of?

[vsergeev]

That would be totally fine, but for my use case -- very few clients, loaded once in a while -- I don't have any requirements on caching, liveness, etc. I guess I see those all as incremental, behind-the-scenes improvements.

In the meanwhile, I can use @embedFile to include files statically at comptime (but this isn't ideal for development when the frontend will see much more churn), or I could write my own request handler to open and stream it.

I understand that there are higher-level frameworks mentioned in the README that can serve files and do much more, but I really don't need their other features. There's a part of me (and I mean this respectfully, because I really do like the http.zig and websocket.zig APIs) that sees the tagline "An HTTP/1.1 server for Zig" that can't serve files as a surprise. Then again, std.http.Server can't either. Nowadays, it seems that many web frameworks have the facility, while others expect nginx/equivalent to handle it.

[vsergeev]

Should they be really static, loaded once on startup (into memory) and then served as-is? Should they be cached? Or should they be served off-disk on every request?

I would imagine the first incarnation would just be served off-disk (for liveness), with an empty .{} options for future optimizations. content_type wouldn't hurt too. I'd use that over writing my own to benefit from anything down the road.

[karlseguin]

Just to set expectations, I'm still not 100% sure about this. I'll try to play with it either this weekend or next.

[sea-grass]

I could potentially see this as (built-in) middleware. Here's an example of what it might look like, as a middleware that can either serve files from the cwd or an absolute path. When handling the request, if the file isn't found, it will pass control onto the next executor. Otherwise, it will stream the file to the response. This example only handles streaming the file but is intentionally structured such that it could be extended to support other methods of serving the file contents (using the Config.Serve option).

const StaticFileMiddleware = struct {
    dir: Config.Dir,
    serve: Config.Serve,

    pub const Config = struct {
        dir: Dir = .cwd,
        serve: Serve = .stream,

        pub const Dir = union(enum) {
            cwd: void,
            abs_path: []const u8,
        };
        pub const Serve = enum { stream };
    };

    pub fn init(config: Config) !StaticFileMiddleware {
        switch (config.dir) {
            .cwd => {},
            .abs_path => |path| {
                try std.fs.accessAbsolute(path, .{});
            },
        }

        return .{ .dir = config.dir, .serve = config.serve };
    }

    pub fn execute(self: *const StaticFileMiddleware, req: *httpz.Request, res: *httpz.Response, executor: anytype) !void {
        const sub_path = parseSubPath(req);

        self.serveFile(sub_path, res) catch |err| switch (err) {
            error.NotFound => return executor.next(),
            else => {
                res.status = 500;
                res.body = "Internal server error";
            },
        };
    }

    fn serveFile(self: *const StaticFileMiddleware, sub_path: []const u8, res: *httpz.Response) !void {
        switch (self.serve) {
            .stream => try self.streamFile(sub_path, res),
        }
    }

    fn streamFile(self: *const StaticFileMiddleware, sub_path: []const u8, res: *httpz.Response) !void {
        var dir = switch (self.dir) {
            .cwd => std.fs.cwd(),
            .abs_path => |path| std.fs.openDirAbsolute(path, .{}),
        };
        defer switch (self.dir) {
            .cwd => {},
            .abs_path => dir.close(),
        };

        var file = dir.openFile(sub_path, .{}) catch return error.NotFound;
        defer file.close();

        res.status = 200;

        var fifo: std.fifo.LinearFifo(u8, .{ .Static = 1024 }) = .init();
        try fifo.pump(file.reader(), res.writer());
    }

    fn openDir(self: *const StaticFileMiddleware) !std.fs.Dir {
        return switch (self.dir) {
            .cwd => std.fs.cwd(),
        };
    }

    /// TODO map req.url to file path
    fn parseSubPath(req: *httpz.Request) []const u8 {
        _ = req.url;

        return "index.html";
    }
};

[vsergeev]

For those interested in the @embedFile() approach, this is what I'm doing to embed the frontend bundle in the backend server:

// Populated by build.zig, type is []const []const u8
const frontend_assets = @import("build_options").frontend_assets;

pub fn main() !void {
    ...
    var router = try server.router(.{});
    inline for (frontend_assets) |path| {
        router.get(path, staticFileHandler(path, &.{}), .{});
    }
    ...
}

fn staticFileHandler(path: []const u8, headers: []const struct { []const u8, []const u8 }) fn (handler: HttpHandler, req: *httpz.Request, res: *httpz.Response) anyerror!void {
    const gen = struct {
        fn handler(_: HttpHandler, _: *httpz.Request, res: *httpz.Response) !void {
            res.content_type = comptime httpz.ContentType.forFile(path);
            res.body = @embedFile("dist" ++ path);
            inline for (headers) |header| res.headers.add(header[0], header[1]);
        }
    };
    return gen.handler;
}

src/dist is symlinked to ../frontend/dist/, where the production bundle is placed. It's nice that the binary is completely standalone with both the backend and frontend.

[df51d]

fwiw here is how I serve my static files right now:

fn serve_file(path: []u8, req: *httpz.Request, res: *httpz.Response) !void {
    const file = try std.fs.openFileAbsolute(path, .{});
    defer file.close();
    const stat = try file.stat();

    res.keepalive = false;
    res.content_type = .forFile(path);

    res.header("Content-Length", try std.fmt.allocPrint(res.arena, "{d}", .{stat.size}));
    const header_buf = try res.prepareHeader();
    try res.conn.stream.writeAll(header_buf);

    const size = try std.posix.sendfile(req.conn.stream.handle, file.handle, 0, 0, &.{}, &.{}, 0);
}

Note that try res.prepareHeader(); is not public, so I had to modify http.zig code to have access to it.

[karlseguin]

@df51d I added a try res.writeHeader();