to use mailinabox as a munin monitoring hub
I've been using Mail-in-a-Box to self-host my own email and DNS for about four-ish years. I got in before VPS providers started cracking down on SMTP traffic, and I've been pretty happy with the experience. Except for the time I procrastinated upgrading so far that I had to get upgrade instructions off the Wayback Machine, but that was my own fault.
Mail-in-a-Box comes configured with a Munin installation. I'd never really touched it beyond browsing the graphs and hmmm-ing sagely, but as I was moving a bunch of other servers around I realized that Munin is intended to aggregate monitoring from a whole network, not just one server. I had five servers scattered around doing things like hosting websites and serving Steam games, so hooking them up to a monitoring hub seemed like a win.
I didn't see any official Mail-in-a-Box documentation or discussion on the forum about extending the baked-in Munin configuration, so I spelunked through the Munin guide. In the Debian/Ubuntu ecosystem, Munin comes in two packages:
munin, which runs the central hub collecting statsmunin-node, which sends stats to the hub
Mail-in-a-Box has both (since it's the hub and a node), plus some plugins:
ello@mailinabox:~$ dpkg -l |grep munin
ii munin
ii munin-common
ii munin-doc
ii munin-node
ii munin-plugins-core
ii munin-plugins-extra
setting up the node
I started with my Signal proxy on Debian 11 and installed packages for the node:
ello@signalproxy:~$ apt install munin-node munin-plugins-extra
Munin matches up nodes and stats by hostname, so you've got to make sure the hostname reported by the node is the same as the one the hub expects. You can either override the hostname in the node configuration:
# /etc/munin/munin-node.conf
host_name signalproxy
or set the right hostname globally. I went with the latter option:
ello@signalproxy:~$ sudo hostnamectl set-hostname signalproxy
I also configured my node to accept queries from Mail-in-a-Box's IP (v6 because why not), on a different port than the default:
# allow connections from localhost and my mailinabox
allow ^127\.0\.0\.1$
allow ^::1$
allow ^mail:in:a:box:ipv6:address::1$
# bind to anything
host *
# on this port
port 54545
I restarted the node to pick up the config changes, and allowed connections through my firewall:
ello@signalproxy:~$ sudo service munin-node restart
ello@signalproxy:~$ sudo ufw allow 54545/tcp
testing the node
At this point, I could connect locally via telnet. cap or list showed me what the node can do from there, and I pulled in raw stats with fetch:
ello@signalproxy:~$ telnet localhost 54545
Trying ::1...
Connected to localhost.
Escape character is '^]'.
# munin node at signalproxy
list
cpu df df_inode entropy [etc.]
fetch cpu
user.value 1840164
nice.value 6889
system.value 992316
idle.value 607417611
iowait.value 242459
irq.value 0
softirq.value 83675
steal.value 816665
guest.value 0
.
quit
Connection closed by foreign host.
The same thing worked from the Mail-in-a-Box hub:
ello@mailinabox:~$ telnet signal:proxy:ipv6::1 54545
Trying signal:proxy:ipv6::1...
Connected to signal:proxy:ipv6::1.
Escape character is '^]'.
# munin node at signalproxy
fetch cpu
user.value 156358562
nice.value 114661
system.value 23173125
idle.value 387236287
iowait.value 50082
irq.value 0
softirq.value 259002
steal.value 0
guest.value 0
.
So far so good.
the hub
The Mail-in-a-Box Munin hub is already configured to talk to its own node:
# /etc/munin/munin.conf
# a simple host tree
[mailinabox]
address 127.0.0.1
Note that the hostname in [ brackets ] there matches the local node configuration:
# /etc/munin/munin-node.conf
host_name mailinabox
I could have added my new node directly to munin.conf, but then my configuration would not be picked up by the Mail-in-a-Box backup process, which only backs up /home/user-data (or whatever you've configured as $STORAGE_ROOT). Instead, I created a dedicated config file just for my signalproxy host in user-data:
# /home/user-data/munin-nodes/signalproxy-node.conf
[signalproxy]
address munin://[signal:proxy:ipv6::1]
port 54545
and symlinked it into Munin's ad-hoc config directory:
ello@mailinabox:~$ cd /etc/munin/munin-conf.d
ello@mailinabox:~$ sudo ln -s /home/user-data/munin-nodes/signalproxy-node.conf
I restarted Munin to pick up the new node:
ello@mailinabox:~$ sudo service munin restart
and ran an update manually to see if it was working:
ello@mailinabox:~$ sudo -u munin /usr/share/munin/munin-update --debug --nofork --host signalproxy --service cpu
That's it! After five minutes or so, the new node showed up in the Munin Monitoring section of my Mail-in-a-Box admin interface. I repeated the process for the rest of my network, and then admired all the pretty graphs.