parseable icon indicating copy to clipboard operation
parseable copied to clipboard

Support systemd-journal-remote for log ingestion

Open udf2457 opened this issue 1 year ago • 3 comments

Please consider supporting systemd-journal-remote for log ingestion.

Docs: https://systemd.io/JOURNAL_EXPORT_FORMATS/#journal-export-format https://www.freedesktop.org/software/systemd/man/systemd-journal-upload.service.html# https://www.freedesktop.org/software/systemd/man/systemd-journal-remote.service.html#

Example from parseable "competitor": https://sematext.com/docs/logagent/input-plugin-journald-upload/

The core benefit is that systemd-journal-remote is clearly available out-of-the-box on many Linux platforms, and it pushes over http/https to a host. No need for third-party agents.

udf2457 avatar Aug 27 '24 08:08 udf2457

hey @udf2457 thank you for the issue. Since Parseable allows dynamic schema, you should be able to ingest data from HTTP(s) calls already. Not sure if there are server changes needed to achieve this.

Have you tried the ingestion? To quickly test this you can point the ingestion call to our demo server's ingestor: http://ec2-3-136-154-35.us-east-2.compute.amazonaws.com:443/ (admin, admin).

nitisht avatar Aug 27 '24 09:08 nitisht

Hi @nitisht

According to your docs....

The Parseable server detects the schema from first event

So I guess the question is whether you are setup to detect Journal Export Format, which AFAIK is binary ? 😉

udf2457 avatar Aug 27 '24 09:08 udf2457

So I guess the question is whether you are setup to detect Journal Export Format, which AFAIK is binary ? 😉

This is indeed a better question! 😄 Right now we don't support binary formats. Let me look deeper into the format

nitisht avatar Aug 27 '24 14:08 nitisht

@nitisht is this really not planned or is your robot being premature ?

udf2457 avatar Feb 27 '25 17:02 udf2457