Why systemd-nspawn for Development Containers?
If you've ever wanted to let an AI code agent or experimental script run wild without worrying about it wreaking havoc on your system, you need isolation. While Docker is the go-to solution for many, systemd-nspawn offers a compelling alternative that's built right into systemd-based Linux distributions.
What Makes systemd-nspawn Different?
systemd-nspawn is a lightweight container manager that comes pre-installed with systemd. Think of it as "chroot on steroids" - it provides OS-level virtualization without the overhead of full virtual machines or even Docker's daemon architecture. Here's why it's particularly suited for development sandboxes:
Advantages:
-
Ultra-lightweight: Minimal overhead compared to VMs or Docker
-
Native integration: Uses systemd's built-in features, no extra daemon required
-
Fast startup: Containers boot in seconds
-
Full system containers: Run complete init systems, perfect for testing systemd services
-
No layered filesystem complexity: Direct filesystem access makes debugging easier
-
Resource efficient: Multiple containers share the host kernel with minimal memory overhead
Perfect Use Cases:
-
Testing destructive scripts or untrusted code
-
Giving AI code agents (like Cursor, Opencode, Copilot, or Claude Code) a safe playground
-
Development environments that mirror production
-
Testing system-level changes without risking your host
-
Learning system administration in isolation
-
Building and testing packages in clean environments
This guide demonstrates the setup on Arch Linux, but the concepts apply to any systemd-based distribution (Ubuntu, Debian, Fedora, etc.) with minor command adjustments.
Prerequisites
Before starting, ensure you have:
-
A Linux system running systemd (most modern distributions)
-
Root or sudo access
-
The
arch-install-scripts
package (providespacstrap
) -
Basic understanding of Linux command line
-
At least 1-2 GB free disk space per container
Install prerequisites on Arch Linux:
sudo pacman -S arch-install-scripts
Step-by-Step Setup Guide
Step 1: Create Container Directory Structure
We'll create the container in your home directory for easy access, then symlink it to the standard systemd-nspawn location:
mkdir -p ~/containers/mycontainer
sudo ln -s ~/containers/mycontainer /var/lib/machines/mycontainer
Why this approach?
-
Keeps containers in your home directory (easier backups, user control)
-
The symlink to
/var/lib/machines/
lets systemd'smachinectl
tools discover and manage it -
You can create multiple containers by repeating with different names
Directory Structure Explained:
-
~/containers/
- Your personal container storage -
mycontainer
- The root filesystem of your container (change this name as needed) -
/var/lib/machines/
- Standard location where systemd looks for containers
Step 2: Install Base System with pacstrap
Bootstrap a minimal Arch Linux installation into the container:
sudo pacstrap -K -c ~/containers/mycontainer base openssh
Command breakdown:
-
pacstrap
- Arch's tool for installing packages to a new root -
-K
- Initialize an empty pacman keyring in the container -
-c
- Use the host's package cache (faster, saves bandwidth) -
base
- Minimal Arch Linux base system -
openssh
- SSH server for remote access
What's being installed? This creates a minimal bootable Linux system (~150-200 MB) with:
-
Core utilities (coreutils, bash, systemd)
-
Package manager (pacman)
-
SSH server for remote access
-
Basic networking tools
Time estimate: 2-5 minutes depending on your internet speed.
Step 3: Set Root Password
Enter the container and set a root password for SSH access:
sudo systemd-nspawn -D ~/containers/mycontainer
passwd
logout
What's happening here:
-
systemd-nspawn -D
spawns a shell in the container (likechroot
but better) -
-D
specifies the directory (root filesystem) of the container -
passwd
sets the root password inside the container -
logout
exits back to your host system
Security note: You're setting a password inside the container, not on your host. This isolation is key to the security model.
Step 4: Enable SSH and Networking in Container
Boot the container with a full init system to properly enable services:
sudo systemd-nspawn -bD ~/containers/mycontainer
Command breakdown:
-
-b
(boot) - Starts systemd inside the container as PID 1 (full boot sequence) -
Without
-b
, you just get a shell; with-b
, you get a complete system
After it boots, log in as root with your password, then run:
systemctl enable sshd
systemctl enable systemd-networkd
sed -i 's/^#*PermitRootLogin.*/PermitRootLogin yes/' /etc/ssh/sshd_config
poweroff
What each command does:
-
systemctl enable sshd
- Start SSH server automatically on boot -
systemctl enable systemd-networkd
- Enable networking on boot (for DHCP) -
The
sed
command modifies SSH config to allow root login (needed for initial access) -
poweroff
cleanly shuts down the container
Networking Architecture:
The container will get a virtual network interface (host0
) that connects to a virtual Ethernet pair on the host. This provides network isolation while still allowing internet access via NAT.
Step 5: Configure Static Network on Host
The host needs systemd-networkd to provide network connectivity for containers. First enable systemd-networkd:
sudo systemctl enable --now systemd-networkd
Then create a static network configuration for the container's virtual interface:
sudo vim /etc/systemd/network/50-ve-mycontainer.network
Add the following configuration:
[Match]
Name=ve-mycontainer
Driver=veth
[Network]
Address=192.168.200.1/24
LinkLocalAddressing=yes
DHCPServer=no
IPMasquerade=yes
LLDP=yes
EmitLLDP=customer-bridge
Restart the networking service to apply the configuration:
sudo systemctl restart systemd-networkd
What this configuration does:
-
[Match]
section targets the virtual Ethernet interface for your container (ve-mycontainer
) -
[Network]
section configures the host-side interface with a static IP192.168.200.1/24
-
IPMasquerade=yes
enables NAT so the container can access the internet through the host -
DHCPServer=no
disables DHCP since we'll use static addressing -
LinkLocalAddressing=yes
enables link-local addressing for fallback connectivity
Step 6: Start Container with machinectl
Now use systemd's container management tool to start your container:
sudo machinectl start mycontainer
About machinectl:
machinectl
is systemd's high-level tool for managing containers and VMs. It handles:
- Starting/stopping containers
- Network setup (creates virtual Ethernet pairs)
- Integration with systemd's journal (logging)
- Resource limits and cgroups
What's happening behind the scenes:
1. systemd creates a virtual Ethernet pair (veth)
2. One end goes into the container (host0
), one stays on host (ve-mycontainer
)
3. The container boots with systemd as init
4. The host interface gets configured with static IP 192.168.200.1/24
5. SSH server starts automatically
6. We'll configure the container with static IP 192.168.200.200/24
in the next step
Step 7: Configure Container Static Network
Shell into the container and configure the static network:
sudo machinectl shell mycontainer
Create the static network config inside the container:
cat > /etc/systemd/network/80-container-host0.network << 'EOF'
[Match]
Name=host0
[Network]
DHCP=no
Address=192.168.200.200/24
Gateway=192.168.200.1
DNS=192.168.200.1
[Link]
RequiredForOnline=yes
EOF
Restart networking inside the container:
systemctl restart systemd-networkd
Check the result:
ip -4 addr show host0
Expected output:
2: host0@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 192.168.200.200/24 brd 192.168.200.255 scope host0
Exit the container:
exit
Reboot the container to ensure all services start properly:
sudo machinectl reboot mycontainer
Step 8: SSH into Your Container
Connect to the container via SSH from your host using the static IP:
ssh root@192.168.200.200
What you can do now:
-
Full isolated Linux environment
-
Install packages (
pacman -S ...
) -
Run experiments, destructive scripts, untrusted code
-
Test system configurations
-
Give AI agents unrestricted access without risk
Example use case - AI code agent:
# From your host, give an AI agent access to the container
ssh root@192.168.200.200 "cd /workspace && ai-agent-run --dangerous-mode"
If something breaks, just destroy and recreate the container in seconds!
Container Management Commands
Here's your complete toolkit for managing containers:
# Start a container (if stopped)
sudo machinectl start mycontainer
# Stop a running container (graceful shutdown)
sudo machinectl stop mycontainer
# Restart a container
sudo machinectl restart mycontainer
# Enable container to start at boot
sudo machinectl enable mycontainer
# Disable auto-start
sudo machinectl disable mycontainer
# Check container status and IP address
sudo machinectl status mycontainer
# Get an interactive shell (without SSH)
sudo machinectl shell mycontainer
# Login to console (like physical machine access)
sudo machinectl login mycontainer
# Execute a single command in container
sudo machinectl shell mycontainer /usr/bin/command
# List all containers
sudo machinectl list
# Show container properties
sudo machinectl show mycontainer
# Power off immediately (hard stop)
sudo machinectl poweroff mycontainer
# Remove/unregister a container (doesn't delete files)
sudo machinectl remove mycontainer
# Copy files to container
sudo machinectl copy-to mycontainer /local/file /container/path
# Copy files from container
sudo machinectl copy-from mycontainer /container/file /local/path
Advanced Tips and Troubleshooting
Resource Limits
Limit container resources using systemd slices:
# Limit CPU (50%)
sudo systemctl set-property systemd-nspawn@mycontainer.service CPUQuota=50%
# Limit RAM (1GB)
sudo systemctl set-property systemd-nspawn@mycontainer.service MemoryMax=8G
# Limit I/O
sudo systemctl set-property systemd-nspawn@mycontainer.service IOWeight=100
How it works:
When machinectl start mycontainer runs, systemd creates a systemd-nspawn@mycontainer.service unit
The @ syntax indicates it's an instantiated service template
systemd-nspawn@ is the template, mycontainer is the instance name
You can verify the service name with:
sudo systemctl status systemd-nspawn@mycontainer.service
Persistent Storage
Mount host directories into containers:
# Start with bind mount
sudo systemd-nspawn -bD ~/containers/mycontainer \
--bind=/home/user/projects:/workspace
Or configure in /etc/systemd/nspawn/mycontainer.nspawn
:
[Files]
Bind=/home/user/projects:/workspace
Creating Multiple Containers
Easily spin up multiple isolated environments:
# Create second container
mkdir -p ~/containers/testbox
sudo ln -s ~/containers/testbox /var/lib/machines/testbox
sudo pacstrap -K -c ~/containers/testbox base openssh
# Repeat configuration steps...
Cleanup and Reset
Destroy a container completely:
# Stop container
sudo machinectl stop mycontainer
# Remove from systemd
sudo machinectl disable mycontainer
# Delete filesystem
sudo rm -rf ~/containers/mycontainer
sudo rm /var/lib/machines/mycontainer
Then recreate from Step 1 for a fresh start!
Common Issues
Container won't start:
-
Check logs:
sudo journalctl -u systemd-nspawn@mycontainer.service
-
Verify symlink:
ls -la /var/lib/machines/
No network in container:
-
Verify systemd-networkd:
systemctl status systemd-networkd
-
Check host network config:
cat /etc/systemd/network/50-ve-mycontainer.network
-
Check container network config:
sudo machinectl shell mycontainer cat /etc/systemd/network/80-container-host0.network
-
Restart networkd:
sudo systemctl restart systemd-networkd
-
Verify container IP:
sudo machinectl shell mycontainer ip -4 addr show host0
SSH connection refused:
-
Verify sshd is running in container:
sudo machinectl shell mycontainer systemctl status sshd
-
Check SSH config:
sudo machinectl shell mycontainer cat /etc/ssh/sshd_config | grep PermitRootLogin
-
Try direct shell first:
sudo machinectl login mycontainer
Security Considerations
Isolation Level:
-
systemd-nspawn provides OS-level virtualization, not full VM isolation
-
Containers share the host kernel (like Docker)
-
Not suitable for hostile/untrusted multi-tenant scenarios
-
Perfect for development sandboxing and testing
Hardening Options:
# Start with additional security
sudo systemd-nspawn -bD ~/containers/mycontainer \
--private-network \ # No network access
--read-only \ # Read-only root filesystem
--drop-capability=all # Drop all Linux capabilities
Best Practices:
-
Use non-root users inside containers when possible
-
Keep containers updated:
sudo machinectl shell mycontainer pacman -Syu
-
Use
--ephemeral
flag for truly disposable environments -
Consider SELinux/AppArmor for additional isolation
Choose systemd-nspawn when:
-
You need full OS containers for testing
-
You want minimal dependencies (systemd only)
-
You're testing systemd services specifically
-
You want direct filesystem access for debugging
Choose Docker when:
-
You need portable, reproducible builds
-
You're deploying applications to production
-
You want a rich ecosystem of pre-built images
-
You need orchestration (Kubernetes, Swarm)
Conclusion
systemd-nspawn provides a lightweight, powerful way to create isolated development environments without the complexity of full virtualization or container orchestration platforms. It's ideal for:
-
Safe experimentation with code that might be destructive
-
AI code agent sandboxing where you want to give tools unrestricted access
-
System-level testing of services and configurations
-
Development environments that mirror production systems
-
Learning Linux without fear of breaking your main system
The tight integration with systemd means you get enterprise-grade container management with tools you already have installed. Start simple, experiment freely, and destroy/recreate at will!