Puppet is extremely useful, but has a number of annoying deficiencies – among them, the way it handles node configuration. External Node Classifiers Are A Thing, but you have to use something like Foreman to store and supply the data. This gets annoying; most such solutions I’ve seen so far have overly complicated messes for user interfaces.

Then you add in the desire to use terraform to configure puppet, and they’re mostly rendered inappropriate anyway.

In the last round I used HashiCorp’s Consul for puppet ENC data, which works surprisingly well – especially since Consul can also store terraform state, which I also need. As I was putting together the infrastructure for my new business, though, I decided that I didn’t want to use Consul, and that meant that I needed another solution.

And so I decided to go hacktastic, and use the already-extant DNS infrastructure for this purpose.

Yup.

I went there…

The Which

ENC data is represented as a blob of YAML. That blob specifies what puppet classes to apply, various parameters, and so on. The format and usage of this is another topic entirely, so I’m going to ignore that. The important thing is that… it’s a blob of YAML.

Here’s an example from the system I’m building, as assembled by terraform:

"classes":
  "ivlabs": {}
  "ivlabs::role::admin": {}
"environment": "aus"
"parameters": {}

This is typical for my new environment: only 5 or so lines of YAML. The worst case scenario (so far), on the other hand, is 34 lines, or 842 bytes. So far I haven’t seen it as very likely that I’m going to exceed it; usually I just type the name of a role class into a module parameter, and off I go.

Not too much to it. Which brings us to–

The How

DNS has exactly one reasonable way (that I know of) to store abitrary data about a host: the TXT record. I would love it if they would someday add ATTR records for key/value storage for metadata about DNS entries (and the things they point to), but that’s neither here nor there. TXT records are pretty simple, and… well.

They’re not actually all that great for this.

You can store roughly 4K in a TXT record, but it’s not as straightforward as it seems. They end up getting split up into something like 256-byte segments, give or take a byte or two. The tooling (at least on the command line) tends not to compensate for this fact.

Plus, it’s not binary data. It needs to be base64-encoded, and we may as well gzip it while we’re at it to save a bit of space in the larger cases. To that end, we end up with a record that looks like this when you go to retrieve it (with the actual content redacted for privacy and security reasons):

<redacted>-01.<redacted> descriptive text "puppet-enc=####################################################################################################################################################################################################################################################" "###############################################################################################################################################################################################################################################################" "####="

The puppet-enc= prefix is intended to differentiate it from any TXT records added by future applications. Given how long this particular entry was, you can see where the tool (the host utilty from bind-utils on Rocky 9 in this case) split it up into multiple strings.

Given that I’m trying to avoid writing actual code for this outside of terraform, that’s very annoying.

Speaking of which, how did this entry get into DNS? Well that part is shockingly easy. I just used the standard dynamic DNS provider in terraform (see this post for the basics of how to set that up). The terraform code looks like this:

locals {
  #
  # Create the basic ENC data
  #
  puppet_enc_data = yamlencode({
    classes     = merge({"ivlabs" = {}}, local.rolearg, var.puppet_classes)
    environment = local.environment_map[var.domain]
    parameters  = var.puppet_params
  })
}

resource dns_txt_record_set "puppet-enc" {
  zone = "${var.domain}."
  name = var.hostname
  txt = [
    "puppet-enc=${base64gzip(local.puppet_enc_data)}"
  ]
  ttl = 2  # Only a couple seconds TTL in case we need to change it.
}

This is actualy part of my global “vm” module, hence the seemingly-generic resource naming. It’s pretty straightforward: build a terraform data structure matching what you want in your yaml, covert it to yaml, gzip and base64 encode it, and stick it in a TXT record.

The WAT

Of course, now you need to get it back out…

As with the Consul-based version, there’s a puppet ENC shell script that takes an FQDN as an argument and spits out the associated ENC data. It’s no longer a one-liner consul kv get, however. The vagaries of the host utility as it pertains to output are unfortunate.

It probably would have been faster to do this if I hadn’t been lazy enough not to want to actually write a program…

But I digress.

The resulting script looks like this:

#!/bin/bash

INDATA=$(host -t TXT $1 | grep -v "has no TXT")
if [ ! -z "$INDATA" ] ; then
  echo $INDATA | cut -f2- -d\" | grep ^puppet-enc= | cut -f2- -d= | sed -e 's/\" \"//g' -e 's/\"$//' | base64 -d | gzip -d
  exit 0
else
  echo Host not found
  exit 1
fi

Evil, I know. Grab the data, make sure it has “puppet-enc=” in it, chop it up on = signs and take everything after the first one, sed out the extra string breaks and the last quote mark, and then base64 decode and de-gzip it. Simple, right?

WAT.

Yeah, this is very bad hackery, and the lazy kind at that. I have a feeling it’s going to break horribly the first time I add another TXT record for a given hostname with some other content. I’m just too lazy to fix it at the moment.

That, and… well.

The Conclusion

This was… an interesting experiment. It proves that it works, at least; it’s still in place in my new environment, and now that I have the obvious kinks worked out of the ENC script, it actually works extremely reliably. I’m a bit surprised by that.

That said, I’m probably going to rip it out and go back to Consul.

Why, you ask?

There are a number of deficiencies in this thing that make it… unwise… to continue with it:

• The 4K limit is just that: an arbitrary limit. If I wanted to bypass it, I would need to go a lot further than this, and it would get very ugly, very quickly. It’s doable (albeit an abuse of DNS), but it would no longer be a quick hack.

• The instant I put truly sensitive data into an ENC file, this thing becomes a non-start. There’s no encryption (and no, I don’t count gzip and/or base64 as encryption!). There’s no read-side security, full stop. If you’re in my environment, you can get access to any and all ENC data.

  This is not good.

• I can’t store my terraform state in DNS, which currently means I’m running a PostgreSQL server for that purpose. This is… overkill, and will be far more effort than Consul to manage. It needs to go away.

So while I hope you enjoy the hack, I’m almost certainly going to yank it all and go back to using Consul, where I can have security, store my terraform state, and not have arbitrary length limits.

Oh well. It was fun while it lasted!