In my last post, I discussed using Zabbix to monitor varnish.  I said it was easy, and that was mostly true — but it also missed one detail that made the situation a bit more complex.  It has nothing to do with Zabbix or Varnish really, but the way in which I run my production servers.

Specifically, I run SELinux in enforcing mode.

Where the agent could easily get data on my development hosts, it instantly failed in production.  The reason was simple; SELinux was blocking access.  This is the double-edged sword of SELinux:  it forces you to truly consider every little security decision you make.

And this time, after years of getting away with simple file context changes, I was actually forced to learn something about SELinux policies.

The Nature of the Problem

SELinux is a cranky beast.  It assigns contexts all over the place — users, files, etc. — and they all have to match the loaded security policies in order to be allowed to do anything.  This is generally a good thing:  it prevents security breaches by drastically limiting what a given application can do.

Most of the time an administrator is doing something simple, like setting up a web root in another location on the disk.  In these cases, the solution is easy:  the semanage fcontext command can set the new directory such that it should have the right context (httpd_sys_content_t in this example), and then restorecon on the directory in question can apply that context.

This is, for the most part, the most complicated thing that general admins run into with SELinux.

In the case of the Zabbix agent, however, we’re doing something decidedly non-standard.  The agent (running as the zabbix user) shells out to varnishstat, which in turn tries to read the Varnish shared memory log to get at the stats it needs.  By default, this log lives at /var/lib/varnish/<hostname>/_.vsm.  This file is memory-mapped into the various processes that need access to it.

The file has the varnishd_var_lib_t security context assigned.  The Zabbix agent is running in the zabbix_agent_t context.  zabbix_agent_t does not have access to varnishd_var_lib_t.

And that’s a problem.