This post describes how I’m using Incus (formerly LXD) for end-to-end testing of the nixa configuration management tool.

Incus is a really nice api-first, cluster-capable daemon for managing lxc containers and KVM vms. This makes it easy to interact with in python, for programatically generating test machines.

Having repeated such VM crud boilerplate for multiple projects (nixa, nk3, bgp-unnumbered), I finally got around to writing nixos_mock to be imported and used by such projects.

nixos_mock is admittedly quite sparse; it provides a Cluster class of incus Nodes running nixos with properties like name, ip, ssh configuration, etc.

With this i’m able to import Cluster in nixa/e2e, generate a nixa inventory with jinja, and actually run nixa apply and nixa upgrades on the ad-hoc vms:

test: lint
  nix-shell --run 'cd e2e; python3 main.py --cleanup --deploy --persist'
  nix-shell --run 'python3 nixa -i e2e/test-inventory.yaml -p2'
  nix-shell --run 'python3 nixa -i e2e/test-inventory.yaml --upgrade -p2 -a boot'
  nix-shell --run 'cd e2e; python3 main.py --cleanup'

Heres a full run:

nixa > just test
black .
All done! ✨ 🍰 ✨
9 files left unchanged.
flake8 . --ignore=E501,W503
nix-shell --run 'cd e2e; python3 main.py --cleanup --deploy --persist'
Device root overridden for nixa-e2e-a1ad
waiting for lxd agent on nixa-e2e-a1ad
Device root overridden for nixa-e2e-422d
waiting for lxd agent on nixa-e2e-422d
configuring nixa-e2e-a1ad
configuring nixa-e2e-422d
waiting for valid ipv4 address on nixa-e2e-a1ad
found nixa-e2e-a1ad at 10.38.154.193
nixa-e2e-a1ad created
waiting for valid ipv4 address on nixa-e2e-422d
found nixa-e2e-422d at 10.38.154.35
nixa-e2e-422d created
nix-shell --run 'python3 nixa -i e2e/test-inventory.yaml -p2'
10.38.154.35 is reachable
10.38.154.193 is reachable
applying modules ['e2e.nix'] to nixa-e2e: ['10.38.154.193', '10.38.154.35']
10.38.154.193:
10.38.154.35:
---

+++

@@ -1,17 +1,8 @@

-{ modulesPath, ... }: {
+{
   imports = [
-    "${modulesPath}/virtualisation/incus-virtual-machine.nix"
-    ./ssh-keys.nix
-    ./incus.nix
+    ./hardware-configuration.nix
+
+    ./e2e.nix
+
   ];
-
-  system.stateVersion = "24.11";
-
-  services.openssh = {
-    enable = true;
-    startWhenNeeded = false;
-    settings.KbdInteractiveAuthentication = false;
-    settings.PasswordAuthentication = false;
-    settings.PermitRootLogin = "yes";
-  };
 }
---

+++

@@ -1,17 +1,8 @@

-{ modulesPath, ... }: {
+{
   imports = [
-    "${modulesPath}/virtualisation/incus-virtual-machine.nix"
-    ./ssh-keys.nix
-    ./incus.nix
+    ./hardware-configuration.nix
+
+    ./e2e.nix
+
   ];
-
-  system.stateVersion = "24.11";
-
-  services.openssh = {
-    enable = true;
-    startWhenNeeded = false;
-    settings.KbdInteractiveAuthentication = false;
-    settings.PasswordAuthentication = false;
-    settings.PermitRootLogin = "yes";
-  };
 }
e2e.nix created
e2e.nix created
rebuilding NixOS on 10.38.154.35
rebuilding NixOS on 10.38.154.193
nix-shell --run 'python3 nixa -i e2e/test-inventory.yaml --upgrade -p2 -a boot'
10.38.154.35 is reachable
10.38.154.193 is reachable
nixa-e2e:
upgrading 10.38.154.193:
enforcing nixos channel nixos-24.11
upgrading 10.38.154.35:
enforcing nixos channel nixos-24.11
6 derivations built
Rebooting 10.38.154.35
6 derivations built
Rebooting 10.38.154.193
10.38.154.35 is reachable
10.38.154.193 is reachable
nix-shell --run 'cd e2e; python3 main.py --cleanup'
nixa-e2e-422d deleted
nixa-e2e-a1ad deleted

Future work may entail standing up a (secure) web-accessable Incus api that can be reached by github actions, and running just test on every push.