Router interface on provider network replies to ARP requests but nothing else
Hi,
I'm trying to learn OpenStack by following the OpenStack Installation Tutorial for Ubuntu, which describes the Newton release. I've chosen to use network option 2 in that tutorial, which results in a provider network and a self-service network. Everything looked OK, up until the point where the tutorial asks to ping the IP address of the gateway IP address on the provider network: no matter where I ping from, replies never come.
I noticed that ARP resolution for the gateway IP address did work: when pinging from my desktop machine (which is in the provider network), its ARP table contained the same MAC-address that is shown in the output of
neutron router-port-list.
Also, when starting an instance it doesn't seem to receive an IP address via DHCP: if I interpret the log correctly, the instance doesn't seem to receive any replies to the DHCPDISCOVERs it sends.
My guess is that once I solved the ping problem, the DHCP problem will also be solved.
I double checked whether I followed the tutorial correctly, but I wasn't able to find what I did wrong. I would be grateful if somebody could point me in the right direction.
My setup:
I've created a controller node and one compute node. These are Ubuntu 16.04 LTS virtual machines on an ESXi 6.5 host. The ESXi host has just one physical network port. Each VM has two virtual network interfaces: one connected to the default (untagged) VLAN, another connected to a tagged VLAN3. The vSwitch is configured to allow forged transmits and MAC changes, but is not configured for promiscuous mode. This configuration seems to work correctly: when I configure a static address on the virtual interfaces connected to the provider network (the untagged VLAN), I can ping these from my desktop just fine.
Network: 10.0.0.0/24: VLAN3, gateway 10.0.0.253, routed/NATted to the internet 192.168.178.0/24: default VLAN, gateway 192.168.178.253. routed/NATted to the internet Controller node: ens160: 10.0.0.100 ens192: unnumbered Compute node: ens160: 10.0.0.120 ens192: unnumbered
Pinging from the gateway to
192.168.178.155 results in no ping replies, and this tcpdump fragment and ARP cache contents:
09:27:24.728856 arp who-has 192.168.178.155 tell 192.168.178.253 09:27:24.729116 arp reply 192.168.178.155 is-at fa:16:3e:55:6a:e5 192.168.178.155 fa:16:3e:55:6a:e5 re0 11m31s
...which is the MAC address of the gateway as shown in the output of
neutron router-port-list as shown below. Pinging from my Windows desktop (which is also in the untagged VLAN and the 192.168.178/24 subnet) results in the same: an ARP cache entry but no replies to the ping:
192.168.178.155 fa-16-3e-55-6a-e5 dynamic
The OpenStack network configuration is as follows:
(more)(more)
joskam@controller:~$ openstack service list +----------------------------------+----------+----------+ | ID | Name | Type | +----------------------------------+----------+----------+ | 5b8761e7b63e49428254dd21ad546b93 | nova ...
Can you ping the other direction? Ping from the router to the gateway? You can get into the router using network namespaces. Do:
# ip netns exec 30c10e23-5d2f-4c71-a305-ad54ce8d92e6 ping 192.168.178.253
You can also do things like checking the routing table and interfaces inside that namespace.
Thanks. While digging further, I found that pinging the router from within its own namespace did work, and some further tcpdumping of several interfaces later the realization sunk in that I was seeing traffic from the LAN, but only broadcast traffic, and it had to be the vSwitch somehow.