Warn user when vector does not have permission to read systemd journalctl

[gwk]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Case

I am a new vector user. My colleague and I just spent a lot of time trying to figure out why our journald source was not emitting any events. The answer was that the user running vector (a dedicated service user also named vector) was not a member of the systemd-journal linux group (I believe adm would also work). The vector logs gave us no hints as to the problem. The logs simply said that all health checks had passed and offered no further clues. The only indicating symptom was that the journald/checkpoint.txt file was empty.

Attempted Solutions

I enabled vector debug logging to see if that would tell me something but it yielded no further clues.

Proposal

Although this is clearly a permissions issue outside of vector itself, I hope the dev team will consider adding some kind of check to indicate whether vector has permission to read from the journal so as to prevent this kind of silent failure.

Version

vector 0.49.0 (x86_64-unknown-linux-gnu dc7e792 2025-08-12 13:47:08.632326804)

[titaneric]

Note that there is a warning in the journald documents

This source requires permissions to run journalctl. When installed from a package manager this should be handled automatically, otherwise ensure the running user is part of the systemd-journal group.

[gwk]

@titaneric thank you. I actually read that in the beginning, added it in our test environment, then forgot about it and found myself trying to figure out why our production server was different than test. The root question is, why is this a silent failure in vector?

I'm guessing the answer is that vector runs journalctl as a subprocess, and journalctl does not exit with a permission failure, but rather yields no logs. I see now that journalctl does issue a hint, but exits cleanly.

Here I am running journalctl as a user that is not a member of systemd-journal, and asking for the records of a unit executed as a different user:

$ sudo -u webapp journalctl -u caddy.service
Hint: You are currently not seeing messages from other users and the system.
      Users in groups 'adm', 'systemd-journal', 'wheel' can see all messages.
      Pass -q to turn off this notice.
-- No entries --

The hint goes to stderr. I suppose that if vector had not silenced/ignored this then I would have seen it in vector's output and immediately solved my problem. So my question becomes, does vector pass the -q, or send journalctl's stderr to /dev/null, or perhaps consume and ignore it?

[titaneric]

So my question becomes, does vector pass the -q, or send journalctl's stderr to /dev/null, or perhaps consume and ignore it?

Interesting, I could dive into the journald source further.

[gwk]

I found this; it does not look like anything is being done with stderr when constructing the command. process::Command documentation says that it defaults to "inherit" or "piped". I don't know much rust so that's as far as I got.

• vector/src/sources/journald.rs

  Line 675 in a7d91b3

  fn make_command(&self, checkpoint: Option<&str>) -> Command {

• https://doc.rust-lang.org/std/process/struct.Command.html#method.stderr

I also looked at the systemd source to see if perhaps the hint is only printed if stderr is a tty, but that does not seem to be the case. It is printed using the log_notice macro. See access_check_var_log_journal, journal_access_check_and_warn and its callers.

• https://github.com/systemd/systemd/blob/main/src/shared/journal-util.c