After an installation, an automatic one especially, a host needs proper network settings before being provisioned like any other hosts via Ansible common rule.

The goal is to ensure:

  • defining the host’s hostname (most needed after an automatic installation using a generic name)
  • setup of the network, taking advantage of the switches redundancy if possible (bonding using active-backup ; balance-alb was tested but did not work), with VLAN and bridging support

Blade-specific Setup

Blades are in a specific network environment with the two microblade internal switches, so a bonding configuration is needed for redundancy.

The playbook fetches the Blade’s hardware information to get the MAC address of the first interface, search it in the DHCP server’s ARP table to get the dynamic IP, and create a temporary Ansible host to connect to. Then it computes the settings and apply them.

Limitations and Todolist

Interfaces Order

Interface order can be problematic, especially when not using Network Manager (as it renames interfaces to match the configured MAC address).

Automated Generation Method / Red Hat Systems Only

Unfortunately the nmcli Ansible module is plagued by multiple bugs and is unusable. We thought about issuing the command directly but it is fragile. Also when this playbook was started we needed support for Fedora Atomic, and at that time (version 26) there was no Network manager support. So we decided to generate configuration files directly for the needed OSes.

This means the playbook can currently only provision OSes from the RedHat family. Using different templates it should not be difficult to add support for Debian-based OSes in the future. The manual procedure is described below.

Starting from CentOS 8 (to get a more recent version of systemd) we might reconsider using systemd-networkd. As support for Fedora Atomic 26 was dropped, using Network Manager might also be an option.

Host IP definition

A few settings are needed to guide the Ansible playbook to compute the settings.

Here is an example you would store in host_vars/<host>/net.yml (adding to the list of interfaces):

      untagged_vlan: 'OSCI-Public'
      tagged_vlans: ['OSCI-Management', 'OSCI-Internal']
      untagged_vlan: 'OSCI-Public'
      tagged_vlans: ['OSCI-Management', 'OSCI-Internal']
      untagged_vlan: ['OSCI-example']
    0: [2, 3]

The host declared in the inventory is used to define the hostname.

ips defines the host’s IP for each VLAN it is part of. There is no need to define a netmask/gateway/… as these parameters are part of the VLAN global definitions in group_vars/all/network.yml.

bonding defines the list of bonding interfaces. The key defines the interface name (bond<key>) and the value is the interface pair.

Interfaces not part of a bonding will also be configured.

Generating Network Configuration Automatically

On Debian systems please refer to the next chapter.

When settings are ready, just fire the playbook:

 ansible-playbook --diff -l <host> playbooks/network_setup.yml

If the playbook says:

  • Could not find MAC address: either the blade position is wrong and there is no blade inserted, or there is a bug in the playbook
  • Could not find IP address (not in ARP table): most probably the ARP entry expired, so you would need to trigger some network communication with Catton/Speedy (console login + DHCP client restart, reboot, ping dynamic IP from Catton/Speedy…)

Manual setup

IPv6 Suffix Discovery

On the host, list the interface you would like to add IPv6 to:

# ip a s eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:64:ab:5a brd ff:ff:ff:ff:ff:ff
    inet brd scope global noprefixroute eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe64:ab5a/64 scope link noprefixroute
       valid_lft forever preferred_lft forever

If the interface is a bonding one, then you will need to configure it first for IPv4 before looking for the IPv6 suffix.

Look at the link-local IPv6 IP fe80::5054:ff:fe64:ab5a/64. Remove the link-local prefix fe80:: and the CIDR range /64 and you get the <ipv6-suffix> 5054:ff:fe64:ab5a.

It is later combined with the network prefix and CIDR range of our infrastructure.

Network Configuration on Debian Systems

If the host has redundant links, add the following paragraph in /etc/network/interfaces:

auto bond0
iface bond0 inet static
        address <ipv4>/24
        bond-slaves <eth-iface-1> <eth-iface-2>
        bond-mode active-backup
        bond-arp-interval 100
        bond-arp-validate filter_active
iface bond0 inet6 static
        address 2620:52:3:1:<ipv6-suffix>/64
        gateway 2620:52:3:1:ffff:ffff:ffff:fffe

If the host has only one link, add the following paragraph in /etc/network/interfaces:

auto <eth-iface>
iface <eth-iface> inet static
        address <ipv4>/24
iface <eth-iface> inet6 static
        address 2620:52:3:1:<ipv6-suffix>/64
        gateway 2620:52:3:1:ffff:ffff:ffff:fffe