Squirrel Logo

Managing Home Assistant YAML config files

Johan Vromans
Articles » Managing HA configs


Managing Home Assistant YAML config files

Note: I’m a software developer and I consider Home Assistant YAML config files as a piece of software. If you are not familiar with basic software development, source editing and version control then this article is not for you. Also note that I work on a Linux workstation. Everything I do can alse be done on Windows and Mac if you read the appropriate documentation.

Everything described here is available in my Github repository.

Home Assistant is an open source home automation that puts local control and privacy first. It is powered by a worldwide community of tinkerers and DIY enthusiasts.

Home Asssistant allows most of its configuration via a friendly web-based UI, but it is also possible — and sometines necessary — to manually write substantial parts of the configuration in the form of a collection of YAML files.

While it is possible to store all of the configuration in one big file, configuration.yaml, the Home Assistant documentation advices to split the configuration data in multiple smaller and easier to maintain files, and describes two ways to have the individial files combined into the main config: including a file, and including a directory of files.

This is the relevant part of my configuration.yaml that handles the includes:

group:      !include groups.yaml
automation: !include automations.yaml
script:     !include scripts.yaml
scene:      !include scenes.yaml
sensor:     !include sensors.yaml
template:   !include templates.yaml
schedule:   !include schedules.yaml
recorder:   !include recorder.yaml

homeassistant:
  packages: !include_dir_named packages

lovelace:
  mode: storage
  dashboards: !include dashboards.yaml

It is important to know that !include can only include a complete YAML object, you cannot include arbitrary parts from files.

The files and basic maintenance

Home Assistant is installed on a dedicated thin client. This system runs the Home Assistant Operating System. This is very easy to install and maintain. I have added several add-ons, of which Samba is relevant here: it allows me to share the configs on the HAOS system with my PC.

On the PC I have mounted the HAOS config dir on the folder proc (short for production). I also have a directory work that contains a copy of the HAOS configuration that I use for maintaince and development. The production environment is updated by copying (installing) modified files from the development environment.

Three tools are important now: GNU Emacs, the source editor to modify the files, Git for version control, and GNU Make to perform build steps and coordinate updates. Later some more tools will follow.

The basic workflow is to modify and test configs in the work folder, and them install them in the prod folder using the make program. This program uses a data file Makefile with Make rules like:

${DST}/configuration.yaml :: configuration.yaml
	install --mode=0644 configuration.yaml ${DST}/configuration.yaml

DST is a Make variable that has the actual location of the production folder. The example is for simplicity, since I have many more files to maintain I use generic rules:

CONFIGS   := $(basename $(wildcard *.yaml))
ALL       := $(addprefix ${DST}/,$(addsuffix .yaml,${CONFIGS}))

all :: ${ALL}

${DST}/%.yaml :: %.yaml
	install --mode=0644 $< $@

CONFIGS is a list of all YAML files in the current directory, with their .yaml suffix stripped off. ALL is a list of all targets in the production folder, constructed from CONFIGS by adding the .yaml suffix and the folder as prefix.

CONFIGS: automations configurations ...
ALL:     ${DST}/automations.yaml ${DST}/configurations.yaml ...

When one or more config files are modified this will trigger the Make rule and execute the corresponding install commands.

The subdirectories dashboards and packages have a similar setup.

Leveraging development with generators

Manually writing YAML files can be a bit tedious, sometimes frustrating, and often boring. Consider this part of a dashboard cellar.yaml:

entities:

  - type: custom:multiple-entity-row
    entity: sensor.kelder_temperature
    icon: mdi:thermometer
    name: Kelder
    show_state: false
    entities:
      - entity: sensor.kelder_temperature
        name: false
      - entity: sensor.kelder_humidity
        format: precision0
        name: false

  - type: custom:multiple-entity-row
    entity: sensor.kruipruimte_temperature
    icon: mdi:thermometer
    name: Kruipruimte
    show_state: false
      - entities:
      - entity: sensor.kruipruimte_temperature
        name: false
      - entity: sensor.kruipruimte_humidity
        format: precision0
        name: false

I have a lot of these items in my dashboards. The items are almost identical, only the display names and sensor names are different. From the perspecitve of software development this is not desired. Repetitive tasks are boring, and copy/paste often leads to errors. So instead of writing these fragments over and over again I called in the help of a generator: the tpage program, part of the Template Toolkit.

tpage and the Template Toolkit

As most templating tools, tpage reads an input file, in our case a prepared YAML document, executes any embedded templating instructions, and writes the resultant output to a selected destination. This is not essentially different from the templates that Home Assistant uses. What is different is that tpage operates on the content of the file as a whole, while Home Assistant templates are limited to a very limited set of single YAML items.

From here on we’ll refer to Template Toolkit templates when the term templates is used. By adding template instructions to the cellar example it is no longer a YAML file, but a Toolkit Template. So we will rename it from cellar.yaml to cellar.tt:

entities:

[% name = "Kelder" %]
[% sensor = "kelder" %]

  - type: custom:multiple-entity-row
    entity: sensor.[% sensor %]_temperature
    icon: mdi:thermometer
    name: [% name %]
    show_state: false
    entities:
      - entity: sensor.[% sensor %]_temperature
        name: false
      - entity: sensor.[% sensor %]_humidity
        format: precision0
        name: false

[% name = "Kruipruimte"; sensor = "kruipruimte" %]

  - type: custom:multiple-entity-row
    entity: sensor.[% sensor %]_temperature
    icon: mdi:thermometer
    name: [% name %]
    show_state: false
    entities:
      - entity: sensor.[% sensor %]_temperature
        name: false
      - entity: sensor.[% sensor %]_humidity
        format: precision0
        name: false

As you can see there are instructions to set variables to values, and instructions to substitute these variables. Both items are now identical, making cut/paste less prone to errors.

Using the INCLUDE facility we can move the whole item to a separate template file, and reduce the example to:

entities:
  - [% INCLUDE temphum name = "Kelder", sensor = "kelder" %]
  - [% INCLUDE temphum name = "Kruipruimte", sensor = "kruipruimte" %]

Here temphum is the name of the template file.

But wait! YAML is very critical with respect to indentation. Wouldn’t this fail horribly if the actual indentation is not precisely as anticipated? For example:

entities:
  - type: vertical-stack
    cards:
      - [% INCLUDE temphum name = "Kelder", sensor = "kelder" %]
      - [% INCLUDE temphum name = "Kruipruimte", sensor = "kruipruimte" %]

The answer is yes. But do not despair, we have a secret weapon: YAML flow style. In short, instead of this:

obj:
   attr: value
   list:
     - item1
     - item2

it is also possible to write:

obj: { attr: value, list: [ item1, item2 ] }

This makes it possible to write templates as ‘indentation independent’ YAML.

Here is the complete temphum template. It takes five arguments, four of which have a sensible default (Github).

[%# Multi-entity row for temperature/humidity.		-*- tt -*-

Arguments:

sensor	   e.g. badkamer, hal 
label      defaults to "Temperatuur"
sensor_t   defaults to 'sensor.' _ sensor _ '_temperature'
sensor_h   defaults to 'sensor.' _ sensor _ '_humidity'
icon       defaults to 'mdi:thermometer'

-%][%-

DEFAULT label    = "Temperatuur";
DEFAULT sensor_t = "sensor.${sensor}_temperature";
DEFAULT sensor_h = "sensor.${sensor}_humidity";
DEFAULT icon     = "mdi:thermometer";

-%]{
  entity: [% sensor_t %],
  type: custom:multiple-entity-row,
  name: "[% label.dquote %]",
  icon: [% icon %],
  show_state: false,
  entities: [
    { entity: [% sensor_t %],
      name: false },
    { entity: [% sensor_h %],
      name: false,
      format: precision0 } ] }

Everything is documented in the Template Toolkit documentation.

Adding template processing to make

As described earlier, updating the Home Assistant config is handled by rules in the Makefile. It has a rule for .yaml files and it is fairly straightforward to add one for .tt files:

CONFIGS   := $(basename $(wildcard *.yaml)) $(basename $(wildcard *.tt))
ALL       := $(addprefix ${DST}/,$(addsuffix .yaml,${CONFIGS}))
TTLIB     := $(shell realpath ../lib/tt)

all :: ${ALL}

${DST}/%.yaml :: %.yaml
	install --mode=0644 $< $@

${DST}/%.yaml :: %.tt ${TTLIB}/*
	tpage --include_path=${TTLIB} $< > $(basename $<).yaml \
	&& install --mode=0644 $(basename $<).yaml $@ \
	&& rm $(basename $<).yaml

The main changes are adding the .tt files to CONFIGS, adding TTLIB where we stored the temphum template, and a slightly more complex rule to process our config template with tpage into a temporary .yaml file, installing the yaml file, and remove it.

Generator programs

Sometimes even the power of the Template Toolkit is not sufficient to produce the desired results. For this I use generator programs, programs that are written to produce the config a specific package.

It will carry to far to go into all the details, since most of these programs are written by me just to do what I want them to, and not always of general use.

As an example a small Perl program that produces the config file for a number of MQTT based temperature sensors.

#! perl

use warnings;
use strict;
use utf8;

use HA::MQTT::Device;

my $d = HA::MQTT::Device->new
  ( name	      => "Systems",
    root	      => "tele",
    topic_root	      => "",
    model	      => "Systems",
    manufacturer      => "Misc",
    identifiers       => [ "systems" ],
    sw_version        => "0",
  );


for ( qw( Phoenix NAS1 Srv1 Srv4 ) ) {
    $d->add_sensor( { name => "$_ Temperature (°C)",
		      value => "float",
		      state_topic => "~/".lc($_)."/temperature" } );
}

binmode STDOUT => ':utf8';

print "# MQTT sensors for systems              -*- hass -*-\n\n";
print $d->as_string;

The resultant config file defines a script that, when run, defines a series of MQTT sensors via autodiscovery. It also defines an automation that will trigger this script when Home Assistant starts. This makes sure the sensors are defined after Home Assistant has started. Finally it defines some friendly name customizations for the sensors.

# MQTT sensors for systems              -*- hass -*-

script:

  systems_define_sensors:
    mode: single
    sequence:
      - data:
          payload: |
            {
               "device" : {
                  "identifiers" : [
                     "systems"
                  ],
                  "manufacturer" : "Misc",
                  "model" : "Systems",
                  "name" : "Systems",
                  "sw_version" : "0"
               },
               "name" : "Systems Phoenix Temperature",
               "state_topic" : "~/phoenix/temperature",
               "unique_id" : "systems_phoenix_temperature",
               "unit_of_measurement" : "°C",
               "value_template" : {% raw %}"{{ value|float }}"{% endraw %},
               "~" : "tele"
            }
          retain: 0
          topic: homeassistant/sensor/tele/phoenix_temperature/config
        service: mqtt.publish
      - data:
          payload: |
            {
               "device" : {
                  "identifiers" : [
                     "systems"
                  ],
                  "manufacturer" : "Misc",
                  "model" : "Systems",
                  "name" : "Systems",
                  "sw_version" : "0"
               },
               "name" : "Systems NAS1 Temperature",
               "state_topic" : "~/nas1/temperature",
               "unique_id" : "systems_nas1_temperature",
               "unit_of_measurement" : "°C",
               "value_template" : {% raw %}"{{ value|float }}"{% endraw %},
               "~" : "tele"
            }
          retain: 0
          topic: homeassistant/sensor/tele/nas1_temperature/config
        service: mqtt.publish
      - data:
          payload: |
            {
               "device" : {
                  "identifiers" : [
                     "systems"
                  ],
                  "manufacturer" : "Misc",
                  "model" : "Systems",
                  "name" : "Systems",
                  "sw_version" : "0"
               },
               "name" : "Systems Srv1 Temperature",
               "state_topic" : "~/srv1/temperature",
               "unique_id" : "systems_srv1_temperature",
               "unit_of_measurement" : "°C",
               "value_template" : {% raw %}"{{ value|float }}"{% endraw %},
               "~" : "tele"
            }
          retain: 0
          topic: homeassistant/sensor/tele/srv1_temperature/config
        service: mqtt.publish
      - data:
          payload: |
            {
               "device" : {
                  "identifiers" : [
                     "systems"
                  ],
                  "manufacturer" : "Misc",
                  "model" : "Systems",
                  "name" : "Systems",
                  "sw_version" : "0"
               },
               "name" : "Systems Srv4 Temperature",
               "state_topic" : "~/srv4/temperature",
               "unique_id" : "systems_srv4_temperature",
               "unit_of_measurement" : "°C",
               "value_template" : {% raw %}"{{ value|float }}"{% endraw %},
               "~" : "tele"
            }
          retain: 0
          topic: homeassistant/sensor/tele/srv4_temperature/config
        service: mqtt.publish

automation:

  # Setup autodiscovery for the sensors.
  - alias: Trigger Systems device sensors definitions
    id: Automation__Trigger_Systems_device_sensors_definitions
    trigger:
      - platform: homeassistant
        event: start
    action:
      - service: script.systems_define_sensors
        data: {}

homeassistant:

  customize:

    sensor.systems_phoenix_temperature:
      friendly_name: Phoenix Temperature (°C)

    sensor.systems_nas1_temperature:
      friendly_name: NAS1 Temperature (°C)

    sensor.systems_srv1_temperature:
      friendly_name: Srv1 Temperature (°C)

    sensor.systems_srv4_temperature:
      friendly_name: Srv4 Temperature (°C)

If a package requires more scripts and automations besides the generated parts, the following approach can be used (Github):

#! perl

use warnings;
use strict;
use utf8;

use HA::MQTT::Device;
use Template;

my $d = HA::MQTT::Device->new
  ( name	      => "Systems",
    root	      => "tele",
    topic_root	      => "",
    model	      => "Systems",
    manufacturer      => "Misc",
    identifiers       => [ "systems" ],
    sw_version        => "0",
  );


for ( qw( Phoenix NAS1 Srv1 Srv4 ) ) {
    $d->add_sensor( { name => "$_ Temperature (°C)",
		      value => "float",
		      state_topic => "~/".lc($_)."/temperature" } );
}

binmode STDOUT => ':utf8';

my $res = $d->generate;

my $xp = Template->new;

my $tmp = join( "", map { $d->detab($_) } <DATA> );
$xp->process( \$tmp, $res );

__DATA__
# MQTT sensors for systems              -*- hass -*-

script:

[% script %]

automation:

[% automation %]

homeassistant:

  customize:

[% customize %]

This will produce the same output as the previous approach, give or take a few empty lines. It is trivial to see where custom code should go.

Conclusions

As a old style software developer I find it much easier to deal with YAML and other text files that are under my control. It is reassuring all files are all under version control, so I have detailed insight in the history of my changes and easy ways to rollback changes in case I screw up. Using templates and generators further reduce the dullness of boring repetitions and likely copy/paste errors. Programming is fun!

Everything described here is available in my Github repository.



© Copyright 2003-2023 Johan Vromans. All Rights Reserved.
articles/hass_config/index.html last modified 21:05:11 31-Jan-2023