The OpenStack Networking components are deployed on the Controller, Compute, and Network nodes in the following configuration:
In case of Two Node Development Cluster :-
Controller node: hosts the Neutron server service, which provides the networking API and communicates with and tracks the agents.
DHCP agent: spawns and controls dnsmasq processes to provide leases to instances. This agent also spawns neutron-ns-metadata-proxy processes as part of the metadata system.
Metadata agent: Provides a metadata proxy to the nova-api-metadata service. The neutron-ns-metadata-proxy direct traffic that they receive in their namespaces to the proxy.
OVS plugin agent: Controls OVS network bridges and routes between them via patch, tunnel, or tap without requiring an external OpenFlow controller.
L3 agent: performs L3 forwarding and NAT.
In case of Three Node or more ( several Compute Nodes) :-
Separate box hosts Neutron Server and all services mentioned above
Compute node: has an OVS plugin agent and openstack-nova-compute service.
Namespaces (View Identifying and Troubleshooting Neutron Namespaces )
For each network you create, the Network node (or Controller node, if combined) will have a unique network namespace (netns) created by the DHCP and Metadata agents. The netns hosts an interface and IP addresses for dnsmasq and the neutron-ns-metadata-proxy. You can view the namespaces with the `ip netns list` command, and can interact with the namespaces with the `ip netns exec
namespace command` command.Every l2-agent/private network has an associated dhcp namespace and
Every l3-agent/router has an associated router namespace.
Network namespace starts with dhcp- followed by the ID of the network.
Router namespace starts with qrouter- followed by the ID of the router.
Source admin credentials and get network list
[root@dfw02 ~(keystone_admin)]$ neutron net-list
+--------------------------------------+------+-----------------------------------------------------+
| id | name | subnets |
+--------------------------------------+------+-----------------------------------------------------+
| 1eea88bb-4952-4aa4-9148-18b61c22d5b7 | int | fa930cea-3d51-4cbe-a305-579f12aa53c0 10.0.0.0/24 |
| 426bb226-0ab9-440d-ba14-05634a17fb2b | int1 | 9e0d457b-c4c4-45cf-84e2-4ac7550f3b06 40.0.0.0/24 |
| 780ce2f3-2e6e-4881-bbac-857813f9a8e0 | ext | f30e5a16-a055-4388-a6ea-91ee142efc3d 192.168.1.0/24 |
+--------------------------------------+------+-----------------------------------------------------+
Using command `ip netns list` run following commands to get tenants
qdhcp-* names
[root@dfw02 ~(keystone_admin)]$ ip netns list | grep 1eea88bb-4952-4aa4-9148-18b61c22d5b7
qdhcp-1eea88bb-4952-4aa4-9148-18b61c22d5b7
[root@dfw02 ~(keystone_admin)]$ ip netns list | grep 426bb226-0ab9-440d-ba14-05634a17fb2b
qdhcp-426bb226-0ab9-440d-ba14-05634a17fb2b
Check tenants Namespace via getting IP and ping this IP inside namespaces
[root@dfw02 ~(keystone_admin)]$ ip netns exec qdhcp-426bb226-0ab9-440d-ba14-05634a17fb2b ifconfig
lo: flags=73
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 0 (Local Loopback)
RX packets 35 bytes 4416 (4.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 35 bytes 4416 (4.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ns-343b0090-24: flags=4163
inet 40.0.0.3 netmask 255.255.255.0 broadcast 40.0.0.255
inet6 fe80::f816:3eff:fe01:8b55 prefixlen 64 scopeid 0x20
ether fa:16:3e:01:8b:55 txqueuelen 1000 (Ethernet)
RX packets 3251 bytes 386284 (377.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1774 bytes 344082 (336.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
PING 40.0.0.3 (40.0.0.3) 56(84) bytes of data.
64 bytes from 40.0.0.3: icmp_seq=1 ttl=64 time=0.047 ms
64 bytes from 40.0.0.3: icmp_seq=2 ttl=64 time=0.041 ms
64 bytes from 40.0.0.3: icmp_seq=3 ttl=64 time=0.032 ms
--- 40.0.0.3 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.032/0.040/0.047/0.006 ms
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 40.0.0.3:53 0.0.0.0:* LISTEN 4370/dnsmasq
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 10.0.0.3:53 0.0.0.0:* LISTEN 4368/dnsmasq
neutron 2431 0.3 0.3 263948 30700 ? Ss 08:42 0:28 /usr/bin/python /usr/bin/neutron-dhcp-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/dhcp_agent.ini --log-file /var/log/neutron/dhcp-agent.log
nobody 4368 0.0 0.0 15532 832 ? S 08:43 0:00 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=ns-40dd712c-e4 --except-interface=lo --pid-file=/var/lib/neutron/dhcp/1eea88bb-4952-4aa4-9148-18b61c22d5b7/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/1eea88bb-4952-4aa4-9148-18b61c22d5b7/host --dhcp-optsfile=/var/lib/neutron/dhcp/1eea88bb-4952-4aa4-9148-18b61c22d5b7/opts --leasefile-ro --dhcp-range=set:tag0,10.0.0.0,static,120s --dhcp-lease-max=256 --conf-file=/etc/neutron/dnsmasq.conf --domain=openstacklocal
nobody 4370 0.0 0.0 15532 872 ? S 08:43 0:00 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=ns-343b0090-24 --except-interface=lo --pid-file=/var/lib/neutron/dhcp/426bb226-0ab9-440d-ba14-05634a17fb2b/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/426bb226-0ab9-440d-ba14-05634a17fb2b/host --dhcp-optsfile=/var/lib/neutron/dhcp/426bb226-0ab9-440d-ba14-05634a17fb2b/opts --leasefile-ro --dhcp-range=set:tag0,40.0.0.0,static,120s --dhcp-lease-max=256 --conf-file=/etc/neutron/dnsmasq.conf --domain=openstacklocal
[root@dfw02 ~(keystone_admin)]$ ps -f --pid 4370 | fold -s -w 82
UID PID PPID C STIME TTY TIME CMD
nobody 4370 1 0 08:43 ? 00:00:00 dnsmasq --no-hosts --no-resolv
--strict-order --bind-interfaces --interface=ns-343b0090-24 --except-interface=lo
--pid-file=/var/lib/neutron/dhcp/426bb226-0ab9-440d-ba14-05634a17fb2b/pid
--dhcp-hostsfile=/var/lib/neutron/dhcp/426bb226-0ab9-440d-ba14-05634a17fb2b/host
--dhcp-optsfile=/var/lib/neutron/dhcp/426bb226-0ab9-440d-ba14-05634a17fb2b/opts
--leasefile-ro --dhcp-range=set:tag0,40.0.0.0,static,120s --dhcp-lease-max=256
--conf-file=/etc/neutron/dnsmasq.conf --domain=openstacklocal
[root@dfw02 ~(keystone_admin)]$ ps -f --pid 4368 | fold -s -w 82
UID PID PPID C STIME TTY TIME CMD
nobody 4368 1 0 08:43 ? 00:00:00 dnsmasq --no-hosts --no-resolv
--strict-order --bind-interfaces --interface=ns-40dd712c-e4 --except-interface=lo
--pid-file=/var/lib/neutron/dhcp/1eea88bb-4952-4aa4-9148-18b61c22d5b7/pid
--dhcp-hostsfile=/var/lib/neutron/dhcp/1eea88bb-4952-4aa4-9148-18b61c22d5b7/host
--dhcp-optsfile=/var/lib/neutron/dhcp/1eea88bb-4952-4aa4-9148-18b61c22d5b7/opts
--leasefile-ro --dhcp-range=set:tag0,10.0.0.0,static,120s --dhcp-lease-max=256
--conf-file=/etc/neutron/dnsmasq.conf --domain=openstacklocal
List interfaces inside dhcp namespace
[root@dfw02 ~(keystone_admin)]$ ip netns exec qdhcp-426bb226-0ab9-440d-ba14-05634a17fb2b ip a
1: lo:
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ns-343b0090-24:
link/ether fa:16:3e:01:8b:55 brd ff:ff:ff:ff:ff:ff
inet 40.0.0.3/24 brd 40.0.0.255 scope global ns-343b0090-24
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe01:8b55/64 scope link
valid_lft forever preferred_lft forever
[root@dfw02 ~(keystone_admin)]$ ip netns exec qdhcp-426bb226-0ab9-440d-ba14-05634a17fb2b ip r
default via 40.0.0.1 dev ns-343b0090-24
40.0.0.0/24 dev ns-343b0090-24 proto kernel scope link src 40.0.0.3
Check routing inside the router namespace
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-86b3008c-297f-4301-9bdc-766b839785f1 ip r
default via 192.168.1.1 dev qg-9c090153-08
40.0.0.0/24 dev qr-e031db6b-d0 proto kernel scope link src 40.0.0.1
192.168.1.0/24 dev qg-9c090153-08 proto kernel scope link src 192.168.1.114
Get routers list via similar grep and network-id to obtain Routers Namespaces
[root@dfw02 ~(keystone_admin)]$ neutron router-list
+--------------------------------------+---------+-----------------------------------------------------------------------------+
| id | name | external_gateway_info |
+--------------------------------------+---------+-----------------------------------------------------------------------------+
| 86b3008c-297f-4301-9bdc-766b839785f1 | router2 | {"network_id": "780ce2f3-2e6e-4881-bbac-857813f9a8e0", "enable_snat": true} |
| bf360d81-79fb-4636-8241-0a843f228fc8 | router1 | {"network_id": "780ce2f3-2e6e-4881-bbac-857813f9a8e0", "enable_snat": true} |
+--------------------------------------+---------+-----------------------------------------------------------------------------+
Now get qrouter-* namespaces via `ip netns list` command :-
[root@dfw02 ~(keystone_admin)]$ ip netns list | grep 86b3008c-297f-4301-9bdc-766b839785f1
qrouter-86b3008c-297f-4301-9bdc-766b839785f1
[root@dfw02 ~(keystone_admin)]$ ip netns list | grep bf360d81-79fb-4636-8241-0a843f228fc8
qrouter-bf360d81-79fb-4636-8241-0a843f228fc8
Now verify L3 forwarding & NAT via command `iptables -L -t nat` inside router namespace and check routing port 80 for 169.254.169.254 to the RDO Havana Controller's ( in my configuration running Neutron Server Service along with all agents) host at metadata port
8700[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-86b3008c-297f-4301-9bdc-766b839785f1 iptables -L -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
neutron-l3-agent-PREROUTING all -- anywhere anywhere
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
neutron-l3-agent-OUTPUT all -- anywhere anywhere
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
neutron-l3-agent-POSTROUTING all -- anywhere anywhere
neutron-postrouting-bottom all -- anywhere anywhere
Chain neutron-l3-agent-OUTPUT (1 references)
target prot opt source destination
DNAT all -- anywhere dfw02.localdomain to:40.0.0.2
DNAT all -- anywhere dfw02.localdomain to:40.0.0.6
DNAT all -- anywhere dfw02.localdomain to:40.0.0.5
Chain neutron-l3-agent-POSTROUTING (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere ! ctstate DNAT
Chain neutron-l3-agent-PREROUTING (1 references)
target prot opt source destination
REDIRECT tcp -- anywhere 169.254.169.254 tcp dpt:http redir ports 8700
DNAT all -- anywhere dfw02.localdomain to:40.0.0.2
DNAT all -- anywhere dfw02.localdomain to:40.0.0.6
DNAT all -- anywhere dfw02.localdomain to:40.0.0.5
Chain neutron-l3-agent-float-snat (1 references)
target prot opt source destination
SNAT all -- 40.0.0.2 anywhere to:192.168.1.107
SNAT all -- 40.0.0.6 anywhere to:192.168.1.104
SNAT all -- 40.0.0.5 anywhere to:192.168.1.110
Chain neutron-l3-agent-snat (1 references)
target prot opt source destination
neutron-l3-agent-float-snat all -- anywhere anywhere
SNAT all -- 40.0.0.0/24 anywhere to:192.168.1.114
Chain neutron-postrouting-bottom (1 references)
target prot opt source destination
neutron-l3-agent-snat all -- anywhere anywhere
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-bf360d81-79fb-4636-8241-0a843f228fc8 iptables -L -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
neutron-l3-agent-PREROUTING all -- anywhere anywhere
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
neutron-l3-agent-OUTPUT all -- anywhere anywhere
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
neutron-l3-agent-POSTROUTING all -- anywhere anywhere
neutron-postrouting-bottom all -- anywhere anywhere
Chain neutron-l3-agent-OUTPUT (1 references)
target prot opt source destination
DNAT all -- anywhere dfw02.localdomain to:10.0.0.2
Chain neutron-l3-agent-POSTROUTING (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere ! ctstate DNAT
Chain neutron-l3-agent-PREROUTING (1 references)
target prot opt source destination
REDIRECT tcp -- anywhere 169.254.169.254 tcp dpt:http redir ports 8700
DNAT all -- anywhere dfw02.localdomain to:10.0.0.2
Chain neutron-l3-agent-float-snat (1 references)
target prot opt source destination
SNAT all -- 10.0.0.2 anywhere to:192.168.1.103
Chain neutron-l3-agent-snat (1 references)
target prot opt source destination
neutron-l3-agent-float-snat all -- anywhere anywhere
SNAT all -- 10.0.0.0/24 anywhere to:192.168.1.100
Chain neutron-postrouting-bottom (1 references)
target prot opt source destination
neutron-l3-agent-snat all -- anywhere anywhere
(B) ( through a NAT rule in the router namespace)
Check the NAT table
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-86b3008c-297f-4301-9bdc-766b839785f1 iptables -t nat -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N neutron-l3-agent-OUTPUT
-N neutron-l3-agent-POSTROUTING
-N neutron-l3-agent-PREROUTING
-N neutron-l3-agent-float-snat
-N neutron-l3-agent-snat
-N neutron-postrouting-bottom
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A POSTROUTING -j neutron-l3-agent-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A neutron-l3-agent-OUTPUT -d 192.168.1.112/32 -j DNAT --to-destination 40.0.0.2
-A neutron-l3-agent-OUTPUT -d 192.168.1.113/32 -j DNAT --to-destination 40.0.0.4
-A neutron-l3-agent-OUTPUT -d 192.168.1.104/32 -j DNAT --to-destination 40.0.0.6
-A neutron-l3-agent-OUTPUT -d 192.168.1.110/32 -j DNAT --to-destination 40.0.0.5
-A neutron-l3-agent-POSTROUTING ! -i qg-9c090153-08 ! -o qg-9c090153-08 -m conntrack ! --ctstate DNAT -j ACCEPT
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8700
-A neutron-l3-agent-PREROUTING -d 192.168.1.112/32 -j DNAT --to-destination 40.0.0.2
-A neutron-l3-agent-PREROUTING -d 192.168.1.113/32 -j DNAT --to-destination 40.0.0.4
-A neutron-l3-agent-PREROUTING -d 192.168.1.104/32 -j DNAT --to-destination 40.0.0.6
-A neutron-l3-agent-PREROUTING -d 192.168.1.110/32 -j DNAT --to-destination 40.0.0.5
-A neutron-l3-agent-float-snat -s 40.0.0.2/32 -j SNAT --to-source 192.168.1.112
-A neutron-l3-agent-float-snat -s 40.0.0.4/32 -j SNAT --to-source 192.168.1.113
-A neutron-l3-agent-float-snat -s 40.0.0.6/32 -j SNAT --to-source 192.168.1.104
-A neutron-l3-agent-float-snat -s 40.0.0.5/32 -j SNAT --to-source 192.168.1.110
-A neutron-l3-agent-snat -j neutron-l3-agent-float-snat
-A neutron-l3-agent-snat -s 40.0.0.0/24 -j SNAT --to-source 192.168.1.114
-A neutron-postrouting-bottom -j neutron-l3-agent-snat
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-bf360d81-79fb-4636-8241-0a843f228fc8 iptables -t nat -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N neutron-l3-agent-OUTPUT
-N neutron-l3-agent-POSTROUTING
-N neutron-l3-agent-PREROUTING
-N neutron-l3-agent-float-snat
-N neutron-l3-agent-snat
-N neutron-postrouting-bottom
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A POSTROUTING -j neutron-l3-agent-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A neutron-l3-agent-OUTPUT -d 192.168.1.103/32 -j DNAT --to-destination 10.0.0.2
-A neutron-l3-agent-POSTROUTING ! -i qg-54e34740-87 ! -o qg-54e34740-87 -m conntrack ! --ctstate DNAT -j ACCEPT
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8700
-A neutron-l3-agent-PREROUTING -d 192.168.1.103/32 -j DNAT --to-destination 10.0.0.2
-A neutron-l3-agent-float-snat -s 10.0.0.2/32 -j SNAT --to-source 192.168.1.103
-A neutron-l3-agent-snat -j neutron-l3-agent-float-snat
-A neutron-l3-agent-snat -s 10.0.0.0/24 -j SNAT --to-source 192.168.1.100
-A neutron-postrouting-bottom -j neutron-l3-agent-snat
Ping to verify network connections
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-86b3008c-297f-4301-9bdc-766b839785f1 ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=47 time=42.6 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=47 time=40.8 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=47 time=41.6 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=47 time=41.0 ms
Verifying service listening at 8700 port inside routers namespaces
output seems like this :-
(C) (to an instance of the neutron-ns-metadata-proxy)
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-86b3008c-297f-4301-9bdc-766b839785f1 netstat -lntp | grep 8700
tcp 0 0 0.0.0.0:8700 0.0.0.0:* LISTEN 4946/python
Check process with pid 4946
[root@dfw02 ~(keystone_admin)]$ ps -ef | grep 4946
root 4946 1 0 08:58 ? 00:00:00 /usr/bin/python /bin/neutron-ns-metadata-proxy --pid_file=/var/lib/neutron/external/pids/86b3008c-297f-4301-9bdc-766b839785f1.pid --metadata_proxy_socket=/var/lib/neutron/metadata_proxy --router_id=86b3008c-297f-4301-9bdc-766b839785f1 --state_path=/var/lib/neutron --metadata_port=8700 --verbose --log-file=neutron-ns-metadata-proxy-86b3008c-297f-4301-9bdc-766b839785f1.log --log-dir=/var/log/neutron
root 10396 11489 0 16:33 pts/3 00:00:00 grep --color=auto 4946
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-bf360d81-79fb-4636-8241-0a843f228fc8 netstat -lntp | grep 8700
tcp 0 0 0.0.0.0:8700 0.0.0.0:* LISTEN 4746/python
Check process with pid 4746
[root@dfw02 ~(keystone_admin)]$ ps -ef | grep 4746
root 4746 1 0 08:58 ? 00:00:00 /usr/bin/python /bin/neutron-ns-metadata-proxy --pid_file=/var/lib/neutron/external/pids/bf360d81-79fb-4636-8241-0a843f228fc8.pid --metadata_proxy_socket=/var/lib/neutron/metadata_proxy --router_id=bf360d81-79fb-4636-8241-0a843f228fc8 --state_path=/var/lib/neutron --metadata_port=8700 --verbose --log-file=neutron-ns-metadata-proxy-bf360d81-79fb-4636-8241-0a843f228fc8.log --log-dir=/var/log/neutron
Now run following commands inside routers namespaces to check status of neutron-metadata port :-
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-86b3008c-297f-4301-9bdc-766b839785f1 netstat -na
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:8700 0.0.0.0:* LISTEN
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags Type State I-Node Path
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-bf360d81-79fb-4636-8241-0a843f228fc8 netstat -na
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:8700 0.0.0.0:* LISTEN
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags Type State I-Node Path
OR this way
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-86b3008c-297f-4301-9bdc-766b839785f1 netstat -anpt
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:8700 0.0.0.0:* LISTEN 4667/python
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-bf360d81-79fb-4636-8241-0a843f228fc8 netstat -anpt
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:8700 0.0.0.0:* LISTEN 4459/python
[root@dfw02 ~(keystone_admin)]$ ps -aux | grep 4667
boris 3638 0.0 0.0 466780 7652 ? Sl 08:43 0:00 /usr/bin/ibus-daemon --replace --xim --panel disable
root 4667 0.0 0.3 263788 25664 ? S 08:43 0:00 /usr/bin/python /bin/neutron-ns-metadata-proxy --pid_file=/var/lib/neutron/external/pids/86b3008c-297f-4301-9bdc-766b839785f1.pid --metadata_proxy_socket=/var/lib/neutron/metadata_proxy --router_id=86b3008c-297f-4301-9bdc-766b839785f1 --state_path=/var/lib/neutron --metadata_port=8700 --verbose --log-file=neutron-ns-metadata-proxy-86b3008c-297f-4301-9bdc-766b839785f1.log --log-dir=/var/log/neutron
root 21538 0.0 0.0 112676 916 pts/0 R+ 12:26 0:00 grep --color=auto 4667
[root@dfw02 ~(keystone_admin)]$ ps -aux | grep 4459
root 4459 0.0 0.2 260052 24080 ? S 08:43 0:00 /usr/bin/python /bin/neutron-ns-metadata-proxy --pid_file=/var/lib/neutron/external/pids/bf360d81-79fb-4636-8241-0a843f228fc8.pid --metadata_proxy_socket=/var/lib/neutron/metadata_proxy --router_id=bf360d81-79fb-4636-8241-0a843f228fc8 --state_path=/var/lib/neutron --metadata_port=8700 --verbose --log-file=neutron-ns-metadata-proxy-bf360d81-79fb-4636-8241-0a843f228fc8.log --log-dir=/var/log/neutron
root 21606 0.0 0.0 112676 920 pts/0 R+ 12:27 0:00 grep --color=auto 4459
Outside routers namespace it would look like
(D) (to the actual Nova metadata service)
Run on Controller , hosting Neutron Server Service :-
[root@dfw02 ~(keystone_admin)]$ netstat -lntp | grep 8700
tcp 0 0 0.0.0.0:8700 0.0.0.0:* LISTEN 2746/python
Check process with pid 2746
[root@dfw02 ~(keystone_admin)]$ ps -ef | grep 2746
nova 2746 1 0 08:57 ? 00:02:31 /usr/bin/python /usr/bin/nova-api --logfile /var/log/nova/api.log
nova 2830 2746 0 08:57 ? 00:00:00 /usr/bin/python /usr/bin/nova-api --logfile /var/log/nova/api.log
nova 2851 2746 0 08:57 ? 00:00:10 /usr/bin/python /usr/bin/nova-api --logfile /var/log/nova/api.log
nova 2858 2746 0 08:57 ? 00:00:02 /usr/bin/python /usr/bin/nova-api --logfile /var/log/nova/api.log
root 9976 11489 0 16:31 pts/3 00:00:00 grep --color=auto 2746
So , we actually verified statement from Direct access to Nova metadata
in an environment running Neutron, a request from your instance must traverse a number of steps:
1. From the instance to a router,(A)
2. Through a NAT rule in the router namespace, (B)
3. To an instance of the neutron-ns-metadata-proxy, (C)
4. To the actual Nova metadata service (D)
Troubleshooting inside routers namespace :-
[root@dfw02 ~(keystone_admin)]$ neutron router-list
+--------------------------------------+---------+-----------------------------------------------------------------------------+
| id | name | external_gateway_info |
+--------------------------------------+---------+-----------------------------------------------------------------------------+
| 86b3008c-297f-4301-9bdc-766b839785f1 | router2 | {"network_id": "780ce2f3-2e6e-4881-bbac-857813f9a8e0", "enable_snat": true} |
| bf360d81-79fb-4636-8241-0a843f228fc8 | router1 | {"network_id": "780ce2f3-2e6e-4881-bbac-857813f9a8e0", "enable_snat": true} |
+--------------------------------------+---------+-------------------------------------------------------
[root@dfw02 ~(keystone_admin)]$ ip netns list
qrouter-86b3008c-297f-4301-9bdc-766b839785f1
qrouter-bf360d81-79fb-4636-8241-0a843f228fc8
qdhcp-1eea88bb-4952-4aa4-9148-18b61c22d5b7
qdhcp-426bb226-0ab9-440d-ba14-05634a17fb2b
Then select corresponding namespace qrouter-86b3008c-297f-4301-9bdc-766b839785f1
and run following commands
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-86b3008c-297f-4301-9bdc-766b839785f1 ifconfig
lo: flags=73
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 0 (Local Loopback)
RX packets 34 bytes 4008 (3.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 34 bytes 4008 (3.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
qg-9c090153-08: flags=4163
inet 192.168.1.114 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::f816:3eff:feb2:24bc prefixlen 64 scopeid 0x20
ether fa:16:3e:b2:24:bc txqueuelen 1000 (Ethernet)
RX packets 261453 bytes 329837642 (314.5 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 104979 bytes 8514797 (8.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
qr-e031db6b-d0: flags=4163
inet 40.0.0.1 netmask 255.255.255.0 broadcast 40.0.0.255
inet6 fe80::f816:3eff:fe83:9090 prefixlen 64 scopeid 0x20
ether fa:16:3e:83:90:90 txqueuelen 1000 (Ethernet)
RX packets 106216 bytes 8579334 (8.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 236556 bytes 327026227 (311.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-86b3008c-297f-4301-9bdc-766b839785f1 ping -c 3 40.0.0.1
PING 40.0.0.1 (40.0.0.1) 56(84) bytes of data.
64 bytes from 40.0.0.1: icmp_seq=1 ttl=64 time=0.044 ms
64 bytes from 40.0.0.1: icmp_seq=2 ttl=64 time=0.036 ms
64 bytes from 40.0.0.1: icmp_seq=3 ttl=64 time=0.048 ms
--- 40.0.0.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.036/0.042/0.048/0.009 ms
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-86b3008c-297f-4301-9bdc-766b839785f1 ping -c 3 192.168.1.114
PING 192.168.1.114 (192.168.1.114) 56(84) bytes of data.
64 bytes from 192.168.1.114: icmp_seq=1 ttl=64 time=0.036 ms
64 bytes from 192.168.1.114: icmp_seq=2 ttl=64 time=0.048 ms
64 bytes from 192.168.1.114: icmp_seq=3 ttl=64 time=0.050 ms
--- 192.168.1.114 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.036/0.044/0.050/0.009 ms
*********************************************************************************
Ping VM (192.168.1.109,10.0.0.11) from dfw02.localdomain (Controller Node)
*********************************************************************************
[root@dfw02 ~(keystone_andrew)]$ neutron router-list
+--------------------------------------+---------+-----------------------------------------------------------------------------+
| id | name | external_gateway_info |
+--------------------------------------+---------+-----------------------------------------------------------------------------+
| 0686e526-0584-4004-a8b0-7d64b376f0ce | router5 | {"network_id": "780ce2f3-2e6e-4881-bbac-857813f9a8e0", "enable_snat": true} |
+--------------------------------------+---------+-----------------------------------------------------------------------------+
[root@dfw02 ~(keystone_andrew)]$ ip netns | grep 0686e526-0584-4004-a8b0-7d64b376f0ce
qrouter-0686e526-0584-4004-a8b0-7d64b376f0ce
[root@dfw02 ~(keystone_andrew)]$ ip netns exec qrouter-0686e526-0584-4004-a8b0-7d64b376f0ce ip a
1: lo:
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: qr-52247b0b-d9:
link/ether fa:16:3e:c8:4b:56 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.5/24 brd 10.0.0.255 scope global qr-52247b0b-d9
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fec8:4b56/64 scope link
valid_lft forever preferred_lft forever
3: qg-edd58124-43:
link/ether fa:16:3e:c4:14:08 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.107/24 brd 192.168.1.255 scope global qg-edd58124-43
valid_lft forever preferred_lft forever
inet 192.168.1.108/32 brd 192.168.1.108 scope global qg-edd58124-43
valid_lft forever preferred_lft forever
inet 192.168.1.109/32 brd 192.168.1.109 scope global qg-edd58124-43
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fec4:1408/64 scope link
valid_lft forever preferred_lft forever
[root@dfw02 ~(keystone_andrew)]$ ip netns exec qrouter-0686e526-0584-4004-a8b0-7d64b376f0ce iptables -S -t nat
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N neutron-l3-agent-OUTPUT
-N neutron-l3-agent-POSTROUTING
-N neutron-l3-agent-PREROUTING
-N neutron-l3-agent-float-snat
-N neutron-l3-agent-snat
-N neutron-postrouting-bottom
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A POSTROUTING -j neutron-l3-agent-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A neutron-l3-agent-OUTPUT -d 192.168.1.109/32 -j DNAT --to-destination 10.0.0.11
-A neutron-l3-agent-OUTPUT -d 192.168.1.108/32 -j DNAT --to-destination 10.0.0.13
-A neutron-l3-agent-POSTROUTING ! -i qg-edd58124-43 ! -o qg-edd58124-43 -m conntrack ! --ctstate DNAT -j ACCEPT
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8700
-A neutron-l3-agent-PREROUTING -d 192.168.1.109/32 -j DNAT --to-destination 10.0.0.11
-A neutron-l3-agent-PREROUTING -d 192.168.1.108/32 -j DNAT --to-destination 10.0.0.13
-A neutron-l3-agent-float-snat -s 10.0.0.11/32 -j SNAT --to-source 192.168.1.109
-A neutron-l3-agent-float-snat -s 10.0.0.13/32 -j SNAT --to-source 192.168.1.108
-A neutron-l3-agent-snat -j neutron-l3-agent-float-snat
-A neutron-l3-agent-snat -s 10.0.0.0/24 -j SNAT --to-source 192.168.1.107
-A neutron-postrouting-bottom -j neutron-l3-agent-snat
[root@dfw02 ~(keystone_andrew)]$ ovs-vsctl show | grep 52247b0b-d9
Port "tap52247b0b-d9"
Interface "tap52247b0b-d9"
[root@dfw02 ~(keystone_andrew)]$ ovs-vsctl show | grep edd58124-43
Port "tapedd58124-43"
Interface "tapedd58124-43"
[root@dfw02 ~]# tcpdump -i tapedd58124-43 -vv > external.dump
tcpdump: WARNING: tapedd58124-43: no IPv4 address assigned
tcpdump: listening on tapedd58124-43, link-type EN10MB (Ethernet), capture size 65535 bytes
^C97 packets captured
97 packets received by filter
0 packets dropped by kernel
[root@dfw02 ~]# cat external.dump
19:31:37.945501 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.1.2 tell 192.168.1.1, length 46
19:31:38.864392 IP (tos 0x0, ttl 64, id 42451, offset 0, flags [DF], proto ICMP (1), length 84)
dfw02.localdomain > 192.168.1.109: ICMP echo request, id 5970, seq 1, length 64
19:31:38.865648 IP (tos 0x0, ttl 63, id 20170, offset 0, flags [none], proto ICMP (1), length 84)
192.168.1.109 > dfw02.localdomain: ICMP echo reply, id 5970, seq 1, length 64
19:31:38.945376 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.1.2 tell 192.168.1.1, length 46
19:31:39.865793 IP (tos 0x0, ttl 64, id 42452, offset 0, flags [DF], proto ICMP (1), length 84)
dfw02.localdomain > 192.168.1.109: ICMP echo request, id 5970, seq 2, length 64
19:31:39.866715 IP (tos 0x0, ttl 63, id 20171, offset 0, flags [none], proto ICMP (1), length 84)
192.168.1.109 > dfw02.localdomain: ICMP echo reply, id 5970, seq 2, length 64
19:31:39.945585 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.1.2 tell 192.168.1.1, length 46
19:31:40.866540 IP (tos 0x0, ttl 64, id 42453, offset 0, flags [DF], proto ICMP (1), length 84)
dfw02.localdomain > 192.168.1.109: ICMP echo request, id 5970, seq 3, length 64
19:31:40.867564 IP (tos 0x0, ttl 63, id 20172, offset 0, flags [none], proto ICMP (1), length 84)
192.168.1.109 > dfw02.localdomain: ICMP echo reply, id 5970, seq 3, length 64
19:31:41.087427 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.1.2 tell 192.168.1.1, length 46
19:31:41.867575 IP (tos 0x0, ttl 64, id 42454, offset 0, flags [DF], proto ICMP (1), length 84)
dfw02.localdomain > 192.168.1.109: ICMP echo request, id 5970, seq 4, length 64
19:31:41.868586 IP (tos 0x0, ttl 63, id 20173, offset 0, flags [none], proto ICMP (1), length 84)
192.168.1.109 > dfw02.localdomain: ICMP echo reply, id 5970, seq 4, length 64
19:31:42.087457 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.1.2 tell 192.168.1.1, length 46
19:31:42.868633 IP (tos 0x0, ttl 64, id 42455, offset 0, flags [DF], proto ICMP (1), length 84)
dfw02.localdomain > 192.168.1.109: ICMP echo request, id 5970, seq 5, length 64
19:31:42.869507 IP (tos 0x0, ttl 63, id 20174, offset 0, flags [none], proto ICMP (1), length 84)
192.168.1.109 > dfw02.localdomain: ICMP echo reply, id 5970, seq 5, length 64
19:31:43.087446 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.1.2 tell 192.168.1.1, length 46
[root@dfw02 ~]# tcpdump -i tap52247b0b-d9 -vv > internal.dump
[root@dfw02 ~]# cat internal.dump
19:31:38.864428 IP (tos 0x0, ttl 63, id 42451, offset 0, flags [DF], proto ICMP (1), length 84)
dfw02.localdomain > 10.0.0.11: ICMP echo request, id 5970, seq 1, length 64
19:31:38.865630 IP (tos 0x0, ttl 64, id 20170, offset 0, flags [none], proto ICMP (1), length 84)
10.0.0.11 > dfw02.localdomain: ICMP echo reply, id 5970, seq 1, length 64
19:31:39.865819 IP (tos 0x0, ttl 63, id 42452, offset 0, flags [DF], proto ICMP (1), length 84)
dfw02.localdomain > 10.0.0.11: ICMP echo request, id 5970, seq 2, length 64
19:31:39.866519 IP (tos 0x0, ttl 64, id 20171, offset 0, flags [none], proto ICMP (1), length 84)
10.0.0.11 > dfw02.localdomain: ICMP echo reply, id 5970, seq 2, length 64
19:31:40.866569 IP (tos 0x0, ttl 63, id 42453, offset 0, flags [DF], proto ICMP (1), length 84)
dfw02.localdomain > 10.0.0.11: ICMP echo request, id 5970, seq 3, length 64
19:31:40.867313 IP (tos 0x0, ttl 64, id 20172, offset 0, flags [none], proto ICMP (1), length 84)
10.0.0.11 > dfw02.localdomain: ICMP echo reply, id 5970, seq 3, length 64
19:31:41.867606 IP (tos 0x0, ttl 63, id 42454, offset 0, flags [DF], proto ICMP (1), length 84)
dfw02.localdomain > 10.0.0.11: ICMP echo request, id 5970, seq 4, length 64
19:31:41.868348 IP (tos 0x0, ttl 64, id 20173, offset 0, flags [none], proto ICMP (1), length 84)
10.0.0.11 > dfw02.localdomain: ICMP echo reply, id 5970, seq 4, length 64
19:31:42.868655 IP (tos 0x0, ttl 63, id 42455, offset 0, flags [DF], proto ICMP (1), length 84)
dfw02.localdomain > 10.0.0.11: ICMP echo request, id 5970, seq 5, length 64
19:31:42.869267 IP (tos 0x0, ttl 64, id 20174, offset 0, flags [none], proto ICMP (1), length 84)
10.0.0.11 > dfw02.localdomain: ICMP echo reply, id 5970, seq 5, length 64
19:31:43.865499 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.11 tell 10.0.0.5, length 28
19:31:43.866516 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.11 is-at fa:16:3e:b2:50:44 (oui Unknown), length 28
19:31:43.869544 IP (tos 0x0, ttl 63, id 42456, offset 0, flags [DF], proto ICMP (1), length 84)
dfw02.localdomain > 10.0.0.11: ICMP echo request, id 5970, seq 6, length 64
19:31:43.870143 IP (tos 0x0, ttl 64, id 20175, offset 0, flags [none], proto ICMP (1), length 84)
10.0.0.11 > dfw02.localdomain: ICMP echo reply, id 5970, seq 6, length 64
**************************************************************************
Troubleshooting DHCP inside qdhcp-xxxxxxxxxxx namespace
**************************************************************************
[root@dfw02 ~(keystone_admin)]$ neutron net-list+--------------------------------------+------+-----------------------------------------------------+
| id | name | subnets |
+--------------------------------------+------+-----------------------------------------------------+
| 1eea88bb-4952-4aa4-9148-18b61c22d5b7 | int | fa930cea-3d51-4cbe-a305-579f12aa53c0 10.0.0.0/24 |
| 426bb226-0ab9-440d-ba14-05634a17fb2b | int1 | 9e0d457b-c4c4-45cf-84e2-4ac7550f3b06 40.0.0.0/24 |
| 780ce2f3-2e6e-4881-bbac-857813f9a8e0 | ext | f30e5a16-a055-4388-a6ea-91ee142efc3d 192.168.1.0/24 |
| 971fd74b-66e4-4016-b997-433e124a22ad | int2 | caca1f54-3211-4395-ba3c-a0c26d9605c0 10.0.0.0/24 |
+--------------------------------------+------+-----------------------------------------------------+
[root@dfw02 ~(keystone_admin)]$ ip netns | grep 971fd74b-66e4-4016-b997-433e124a22ad
qdhcp-971fd74b-66e4-4016-b997-433e124a22ad
[root@dfw02 ~(keystone_admin)]$ ip netns exec qdhcp-971fd74b-66e4-4016-b997-433e124a22ad ip a
1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ns-a2f724d1-74: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether fa:16:3e:75:95:6c brd ff:ff:ff:ff:ff:ff
inet 10.0.0.12/24 brd 10.0.0.255 scope global ns-a2f724d1-74
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe75:956c/64 scope link
valid_lft forever preferred_lft forever
[root@dfw02 ~(keystone_admin)]$ ovs-vsctl show | grep a2f724d1-74
Port "tapa2f724d1-74"
Interface "tapa2f724d1-74"
[root@dfw02 ~(keystone_admin)]$ tcpdump -i tapa2f724d1-74 -vv
tcpdump: WARNING: tapa2f724d1-74: no IPv4 address assigned
tcpdump: listening on tapa2f724d1-74, link-type EN10MB (Ethernet), capture size 65535 bytes
10:20:01.150267 IP (tos 0x0, ttl 64, id 42847, offset 0, flags [DF], proto UDP (17), length 328)
10.0.0.11.bootpc > 10.0.0.12.bootps: [udp sum ok] BOOTP/DHCP, Request from fa:16:3e:b2:50:44 (oui Unknown), length 300, xid 0x6040a47f, Flags [none] (0x0000)
Client-IP 10.0.0.11
Client-Ethernet-Address fa:16:3e:b2:50:44 (oui Unknown)
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Request
Hostname Option 12, length 13: "ubuntutrs0426"
Parameter-Request Option 55, length 13:
Subnet-Mask, BR, Time-Zone, Default-Gateway
Domain-Name, Domain-Name-Server, Option 119, Hostname
Netbios-Name-Server, Netbios-Scope, MTU, Classless-Static-Route
NTP
10:20:01.150956 IP (tos 0xc0, ttl 64, id 4988, offset 0, flags [none], proto UDP (17), length 356)
10.0.0.12.bootps > 10.0.0.11.bootpc: [bad udp cksum 0x1578 -> 0x98ac!] BOOTP/DHCP, Reply, length 328, xid 0x6040a47f, Flags [none] (0x0000)
Client-IP 10.0.0.11
Your-IP 10.0.0.11
Server-IP 10.0.0.12
Client-Ethernet-Address fa:16:3e:b2:50:44 (oui Unknown)
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: ACK
Server-ID Option 54, length 4: 10.0.0.12
Lease-Time Option 51, length 4: 120
RN Option 58, length 4: 53
RB Option 59, length 4: 98
Subnet-Mask Option 1, length 4: 255.255.255.0
BR Option 28, length 4: 10.0.0.255
Domain-Name Option 15, length 14: "openstacklocal"
Hostname Option 12, length 14: "host-10-0-0-11"
Default-Gateway Option 3, length 4: 10.0.0.5
Domain-Name-Server Option 6, length 4: ns2.donpac.ru
MTU Option 26, length 2: 1454
10:20:06.162099 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.12 tell 10.0.0.11, length 28
10:20:06.162117 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.12 is-at fa:16:3e:75:95:6c (oui Unknown), length 28
10:20:47.948046 IP (tos 0x0, ttl 64, id 42848, offset 0, flags [DF], proto UDP (17), length 328)
10.0.0.11.bootpc > 10.0.0.12.bootps: [udp sum ok] BOOTP/DHCP, Request from fa:16:3e:b2:50:44 (oui Unknown), length 300, xid 0x6040a47f, Flags [none] (0x0000)
Client-IP 10.0.0.11
Client-Ethernet-Address fa:16:3e:b2:50:44 (oui Unknown)
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Request
Hostname Option 12, length 13: "ubuntutrs0426"
Parameter-Request Option 55, length 13:
Subnet-Mask, BR, Time-Zone, Default-Gateway
Domain-Name, Domain-Name-Server, Option 119, Hostname
Netbios-Name-Server, Netbios-Scope, MTU, Classless-Static-Route
NTP
10:20:47.948559 IP (tos 0xc0, ttl 64, id 4989, offset 0, flags [none], proto UDP (17), length 356)
10.0.0.12.bootps > 10.0.0.11.bootpc: [bad udp cksum 0x1578 -> 0x98ac!] BOOTP/DHCP, Reply, length 328, xid 0x6040a47f, Flags [none] (0x0000)
Client-IP 10.0.0.11
Your-IP 10.0.0.11
Server-IP 10.0.0.12
Client-Ethernet-Address fa:16:3e:b2:50:44 (oui Unknown)
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: ACK
Server-ID Option 54, length 4: 10.0.0.12
Lease-Time Option 51, length 4: 120
RN Option 58, length 4: 53
RB Option 59, length 4: 98
Subnet-Mask Option 1, length 4: 255.255.255.0
BR Option 28, length 4: 10.0.0.255
Domain-Name Option 15, length 14: "openstacklocal"
Hostname Option 12, length 14: "host-10-0-0-11"
Default-Gateway Option 3, length 4: 10.0.0.5
Domain-Name-Server Option 6, length 4: ns2.donpac.ru
MTU Option 26, length 2: 1454
10:20:52.962105 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.12 tell 10.0.0.11, length 28
10:20:52.962127 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.12 is-at fa:16:3e:75:95:6c (oui Unknown), length 28
10:21:35.955647 IP (tos 0x0, ttl 64, id 42849, offset 0, flags [DF], proto UDP (17), length 328)
10.0.0.11.bootpc > 10.0.0.12.bootps: [udp sum ok] BOOTP/DHCP, Request from fa:16:3e:b2:50:44 (oui Unknown), length 300, xid 0x6040a47f, Flags [none] (0x0000)
Client-IP 10.0.0.11
Client-Ethernet-Address fa:16:3e:b2:50:44 (oui Unknown)
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Request
Hostname Option 12, length 13: "ubuntutrs0426"
Parameter-Request Option 55, length 13:
Subnet-Mask, BR, Time-Zone, Default-Gateway
Domain-Name, Domain-Name-Server, Option 119, Hostname
Netbios-Name-Server, Netbios-Scope, MTU, Classless-Static-Route
NTP
10:21:35.956161 IP (tos 0xc0, ttl 64, id 4990, offset 0, flags [none], proto UDP (17), length 356)
10.0.0.12.bootps > 10.0.0.11.bootpc: [bad udp cksum 0x1578 -> 0x94ac!] BOOTP/DHCP, Reply, length 328, xid 0x6040a47f, Flags [none] (0x0000)
Client-IP 10.0.0.11
Your-IP 10.0.0.11
Server-IP 10.0.0.12
Client-Ethernet-Address fa:16:3e:b2:50:44 (oui Unknown)
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: ACK
Server-ID Option 54, length 4: 10.0.0.12
Lease-Time Option 51, length 4: 120
RN Option 58, length 4: 55
RB Option 59, length 4: 100
Subnet-Mask Option 1, length 4: 255.255.255.0
BR Option 28, length 4: 10.0.0.255
Domain-Name Option 15, length 14: "openstacklocal"
Hostname Option 12, length 14: "host-10-0-0-11"
Default-Gateway Option 3, length 4: 10.0.0.5
Domain-Name-Server Option 6, length 4: ns2.donpac.ru
MTU Option 26, length 2: 1454
10:21:40.962062 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.12 tell 10.0.0.11, length 28
10:21:40.962082 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.12 is-at fa:16:3e:75:95:6c (oui Unknown), length 28
^C
12 packets captured
*********************************************
/var/log/neutro/dnsmasq.log reports*********************************************
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 client provides name: ubuntutrs0426
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 DHCPREQUEST(ns-a2f724d1-74) 10.0.0.11 fa:16:3e:b2:50:44
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 tags: tag0, known, ns-a2f724d1-74
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 DHCPACK(ns-a2f724d1-74) 10.0.0.11 fa:16:3e:b2:50:44 host-10-0-0-11
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 requested options: 1:netmask, 28:broadcast, 2:time-offset, 3:router,
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 requested options: 15:domain-name, 6:dns-server, 119:domain-search,
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 requested options: 12:hostname, 44:netbios-ns, 47:netbios-scope,
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 requested options: 26:mtu, 121:classless-static-route, 42:ntp-server
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 next server: 10.0.0.12
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 sent size: 1 option: 53 message-type 5
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 54 server-identifier 10.0.0.12
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 51 lease-time 2m
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 58 T1 54s
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 59 T2 1m39s
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 1 netmask 255.255.255.0
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 28 broadcast 10.0.0.255
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 sent size: 14 option: 15 domain-name openstacklocal
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 sent size: 14 option: 12 hostname host-10-0-0-11
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 3 router 10.0.0.5
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 6 dns-server 83.221.202.254
Apr 27 09:38:47 dnsmasq-dhcp[3935]: 1614849151 sent size: 2 option: 26 mtu 1454
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 available DHCP subnet: 10.0.0.0/255.255.255.0
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 client provides name: ubuntutrs0426
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 DHCPREQUEST(ns-a2f724d1-74) 10.0.0.11 fa:16:3e:b2:50:44
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 tags: tag0, known, ns-a2f724d1-74
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 DHCPACK(ns-a2f724d1-74) 10.0.0.11 fa:16:3e:b2:50:44 host-10-0-0-11
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 requested options: 1:netmask, 28:broadcast, 2:time-offset, 3:router,
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 requested options: 15:domain-name, 6:dns-server, 119:domain-search,
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 requested options: 12:hostname, 44:netbios-ns, 47:netbios-scope,
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 requested options: 26:mtu, 121:classless-static-route, 42:ntp-server
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 next server: 10.0.0.12
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 sent size: 1 option: 53 message-type 5
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 54 server-identifier 10.0.0.12
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 51 lease-time 2m
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 58 T1 56s
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 59 T2 1m41s
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 1 netmask 255.255.255.0
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 28 broadcast 10.0.0.255
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 sent size: 14 option: 15 domain-name openstacklocal
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 sent size: 14 option: 12 hostname host-10-0-0-11
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 3 router 10.0.0.5
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 sent size: 4 option: 6 dns-server 83.221.202.254
Apr 27 09:39:34 dnsmasq-dhcp[3935]: 1614849151 sent size: 2 option: 26 mtu 1454**************************************************************************
Troubleshooting via qrouter-xxxxxxxxxxx namespace
**************************************************************************
[root@dfw02 ~(keystone_andrew)]$ neutron router-list
+--------------------------------------+---------+-----------------------------------------------------------------------------+
| id | name | external_gateway_info |
+--------------------------------------+---------+-----------------------------------------------------------------------------+
| 0686e526-0584-4004-a8b0-7d64b376f0ce | router5 | {"network_id": "780ce2f3-2e6e-4881-bbac-857813f9a8e0", "enable_snat": true} |
+--------------------------------------+---------+-----------------------------------------------------------------------------+
[root@dfw02 ~(keystone_andrew)]$ ip netns | grep 0686e526-0584-4004-a8b0-7d64b376f0ce
qrouter-0686e526-0584-4004-a8b0-7d64b376f0ce
[root@dfw02 ~(keystone_andrew)]$ neutron net-list
+--------------------------------------+------+--------------------------------------------------+
| id | name | subnets |
+--------------------------------------+------+--------------------------------------------------+
| 426bb226-0ab9-440d-ba14-05634a17fb2b | int1 | 9e0d457b-c4c4-45cf-84e2-4ac7550f3b06 40.0.0.0/24 |
| 780ce2f3-2e6e-4881-bbac-857813f9a8e0 | ext | f30e5a16-a055-4388-a6ea-91ee142efc3d |
| 971fd74b-66e4-4016-b997-433e124a22ad | int2 | caca1f54-3211-4395-ba3c-a0c26d9605c0 10.0.0.0/24 |
+--------------------------------------+------+--------------------------------------------------+
[root@dfw02 ~(keystone_andrew)]$ ip netns | grep 971fd74b-66e4-4016-b997-433e124a22ad
qdhcp-971fd74b-66e4-4016-b997-433e124a22ad
[root@dfw02 ~(keystone_andrew)]$ nova list
+--------------------------------------+---------------+-----------+------------+-------------+-------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+---------------+-----------+------------+-------------+-------------------------------+
| 29546a14-a731-4eb3-acbc-d118e5b6dba6 | UbuntuTRS0426 | ACTIVE | None | Running | int2=10.0.0.11, 192.168.1.109 |
| 19cb7aeb-e0b5-4e21-9793-f72e250d1524 | VF20RS025 | SUSPENDED | None | Shutdown | int2=10.0.0.13, 192.168.1.108 |
+--------------------------------------+---------------+-----------+------------+-------------+----------
[root@dfw02 ~(keystone_andrew)]$ ip netns exec qrouter-0686e526-0584-4004-a8b0-7d64b376f0ce ping -c 3 192.168.1.109
PING 192.168.1.109 (192.168.1.109) 56(84) bytes of data.
64 bytes from 192.168.1.109: icmp_seq=1 ttl=64 time=6.68 ms
64 bytes from 192.168.1.109: icmp_seq=2 ttl=64 time=0.943 ms
64 bytes from 192.168.1.109: icmp_seq=3 ttl=64 time=0.934 ms
--- 192.168.1.109 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 0.934/2.854/6.687/2.710 ms
[root@dfw02 ~(keystone_andrew)]$ ip netns exec qdhcp-971fd74b-66e4-4016-b997-433e124a22ad ip r
default via 10.0.0.5 dev ns-a2f724d1-74
10.0.0.0/24 dev ns-a2f724d1-74 proto kernel scope link src 10.0.0.12
[root@dfw02 ~(keystone_andrew)]$ ip netns exec qrouter-0686e526-0584-4004-a8b0-7d64b376f0ce ip r
default via 192.168.1.1 dev qg-edd58124-43
10.0.0.0/24 dev qr-52247b0b-d9 proto kernel scope link src 10.0.0.5
192.168.1.0/24 dev qg-edd58124-43 proto kernel scope link src 192.168.1.107
[root@dfw02 ~(keystone_andrew)]$ ip netns exec qrouter-0686e526-0584-4004-a8b0-7d64b376f0ce ip a
1: lo:
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: qr-52247b0b-d9:
link/ether fa:16:3e:c8:4b:56 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.5/24 brd 10.0.0.255 scope global qr-52247b0b-d9
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fec8:4b56/64 scope link
valid_lft forever preferred_lft forever
3: qg-edd58124-43:
link/ether fa:16:3e:c4:14:08 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.107/24 brd 192.168.1.255 scope global qg-edd58124-43
valid_lft forever preferred_lft forever
inet 192.168.1.109/32 brd 192.168.1.109 scope global qg-edd58124-43
valid_lft forever preferred_lft forever
inet 192.168.1.108/32 brd 192.168.1.108 scope global qg-edd58124-43
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fec4:1408/64 scope link
valid_lft forever preferred_lft forever
[root@dfw02 ~(keystone_andrew)]$ ip netns exec qrouter-0686e526-0584-4004-a8b0-7d64b376f0ce ifconfig
lo: flags=73
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 0 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
qg-edd58124-43: flags=4163
inet 192.168.1.107 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::f816:3eff:fec4:1408 prefixlen 64 scopeid 0x20
ether fa:16:3e:c4:14:08 txqueuelen 1000 (Ethernet)
RX packets 43867 bytes 4183329 (3.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 43 bytes 3774 (3.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
qr-52247b0b-d9: flags=4163
inet 10.0.0.5 netmask 255.255.255.0 broadcast 10.0.0.255
inet6 fe80::f816:3eff:fec8:4b56 prefixlen 64 scopeid 0x20
ether fa:16:3e:c8:4b:56 txqueuelen 1000 (Ethernet)
RX packets 79 bytes 9182 (8.9 KiB)
RX errors 0 dropped 20 overruns 0 frame 0
TX packets 62 bytes 3924 (3.8 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@dfw02 ~(keystone_andrew)]$ ip netns exec qrouter-0686e526-0584-4004-a8b0-7d64b376f0ce iptables -S -t nat
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N neutron-l3-agent-OUTPUT
-N neutron-l3-agent-POSTROUTING
-N neutron-l3-agent-PREROUTING
-N neutron-l3-agent-float-snat
-N neutron-l3-agent-snat
-N neutron-postrouting-bottom
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A POSTROUTING -j neutron-l3-agent-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A neutron-l3-agent-OUTPUT -d 192.168.1.109/32 -j DNAT --to-destination 10.0.0.11
-A neutron-l3-agent-OUTPUT -d 192.168.1.108/32 -j DNAT --to-destination 10.0.0.13
-A neutron-l3-agent-POSTROUTING ! -i qg-edd58124-43 ! -o qg-edd58124-43 -m conntrack ! --ctstate DNAT -j ACCEPT
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8700
-A neutron-l3-agent-PREROUTING -d 192.168.1.109/32 -j DNAT --to-destination 10.0.0.11
-A neutron-l3-agent-PREROUTING -d 192.168.1.108/32 -j DNAT --to-destination 10.0.0.13
-A neutron-l3-agent-float-snat -s 10.0.0.11/32 -j SNAT --to-source 192.168.1.109
-A neutron-l3-agent-float-snat -s 10.0.0.13/32 -j SNAT --to-source 192.168.1.108
-A neutron-l3-agent-snat -j neutron-l3-agent-float-snat
-A neutron-l3-agent-snat -s 10.0.0.0/24 -j SNAT --to-source 192.168.1.107
-A neutron-postrouting-bottom -j neutron-l3-agent-snat
Capturing packages on external qrouter-xxxxxxxxx interface :-
[root@dfw02 ~(keystone_andrew)]$ ip netns exec qrouter-0686e526-0584-4004-a8b0-7d64b376f0ce tcpdump -ln -i qg-edd58124-43
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on qg-edd58124-43, link-type EN10MB (Ethernet), capture size 65535 bytes
06:50:44.246371 IP 192.168.1.127 > 192.168.1.109: ICMP echo request, id 7859, seq 861, length 64
06:50:44.247367 IP 192.168.1.109 > 192.168.1.127: ICMP echo reply, id 7859, seq 861, length 64
06:50:44.495209 ARP, Request who-has 192.168.1.2 tell 192.168.1.1, length 46
06:50:45.247341 IP 192.168.1.127 > 192.168.1.109: ICMP echo request, id 7859, seq 862, length 64
06:50:45.248270 IP 192.168.1.109 > 192.168.1.127: ICMP echo reply, id 7859, seq 862, length 64
06:50:45.941235 ARP, Request who-has 192.168.1.2 tell 192.168.1.1, length 46
06:50:46.248322 IP 192.168.1.127 > 192.168.1.109: ICMP echo request, id 7859, seq 863, length 64
06:50:46.249236 IP 192.168.1.109 > 192.168.1.127: ICMP echo reply, id 7859, seq 863, length 64
06:50:46.941218 ARP, Request who-has 192.168.1.2 tell 192.168.1.1, length 46
06:50:47.249346 IP 192.168.1.127 > 192.168.1.109: ICMP echo request, id 7859, seq 864, length 64
06:50:47.250218 IP 192.168.1.109 > 192.168.1.127: ICMP echo reply, id 7859, seq 864, length 64
06:50:47.941395 ARP, Request who-has 192.168.1.2 tell 192.168.1.1, length 46
06:50:48.249331 IP 192.168.1.127 > 192.168.1.109: ICMP echo request, id 7859, seq 865, length 64
06:50:48.250344 IP 192.168.1.109 > 192.168.1.127: ICMP echo reply, id 7859, seq 865, length 64
06:50:49.250347 IP 192.168.1.127 > 192.168.1.109: ICMP echo request, id 7859, seq 866, length 64
06:50:49.251258 IP 192.168.1.109 > 192.168.1.127: ICMP echo reply, id 7859, seq 866, length 64
06:50:49.550297 ARP, Request who-has 192.168.1.2 tell 192.168.1.1, length 46
06:50:50.251222 IP 192.168.1.127 > 192.168.1.109: ICMP echo request, id 7859, seq 867, length 64
06:50:50.252261 IP 192.168.1.109 > 192.168.1.127: ICMP echo reply, id 7859, seq 867, length 64
06:50:50.550328 ARP, Request who-has 192.168.1.2 tell 192.168.1.1, length 46
06:50:51.252351 IP 192.168.1.127 > 192.168.1.109: ICMP echo request, id 7859, seq 868, length 64
06:50:51.253353 IP 192.168.1.109 > 192.168.1.127: ICMP echo reply, id 7859, seq 868, length 64
06:50:51.550324 ARP, Request who-has 192.168.1.2 tell 192.168.1.1, length 46
06:50:52.253552 IP 192.168.1.127 > 192.168.1.109: ICMP echo request, id 7859, seq 869, length 64
06:50:52.254457 IP 192.168.1.109 > 192.168.1.127: ICMP echo reply, id 7859, seq 869, length 64
06:50:52.258296 ARP, Request who-has 192.168.1.127 tell 192.168.1.107, length 28
06:50:52.258479 ARP, Reply 192.168.1.127 is-at 00:22:15:63:e4:e2, length 28
^C
Capturing packages on internal qrouter-xxxxxxxx interface :-
[root@dfw02 ~(keystone_andrew)]$ ip netns exec qrouter-0686e526-0584-4004-a8b0-7d64b376f0ce tcpdump -ln -i qr-52247b0b-d9
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on qr-52247b0b-d9, link-type EN10MB (Ethernet), capture size 65535 bytes
06:51:08.267668 IP 192.168.1.127 > 10.0.0.11: ICMP echo request, id 7859, seq 885, length 64
06:51:08.268374 IP 10.0.0.11 > 192.168.1.127: ICMP echo reply, id 7859, seq 885, length 64
06:51:08.274294 ARP, Request who-has 10.0.0.11 tell 10.0.0.5, length 28
06:51:08.274990 ARP, Reply 10.0.0.11 is-at fa:16:3e:b2:50:44, length 28
06:51:09.268367 IP 192.168.1.127 > 10.0.0.11: ICMP echo request, id 7859, seq 886, length 64
06:51:09.269085 IP 10.0.0.11 > 192.168.1.127: ICMP echo reply, id 7859, seq 886, length 64
06:51:10.269354 IP 192.168.1.127 > 10.0.0.11: ICMP echo request, id 7859, seq 887, length 64
06:51:10.270086 IP 10.0.0.11 > 192.168.1.127: ICMP echo reply, id 7859, seq 887, length 64
06:51:11.270366 IP 192.168.1.127 > 10.0.0.11: ICMP echo request, id 7859, seq 888, length 64
06:51:11.271118 IP 10.0.0.11 > 192.168.1.127: ICMP echo reply, id 7859, seq 888, length 64
06:51:12.271596 IP 192.168.1.127 > 10.0.0.11: ICMP echo request, id 7859, seq 889, length 64
06:51:12.272319 IP 10.0.0.11 > 192.168.1.127: ICMP echo reply, id 7859, seq 889, length 64
06:51:13.272699 IP 192.168.1.127 > 10.0.0.11: ICMP echo request, id 7859, seq 890, length 64
06:51:13.273459 IP 10.0.0.11 > 192.168.1.127: ICMP echo reply, id 7859, seq 890, length 64
06:51:14.271649 ARP, Request who-has 10.0.0.5 tell 10.0.0.11, length 28
06:51:14.271666 ARP, Reply 10.0.0.5 is-at fa:16:3e:c8:4b:56, length 28
06:51:14.273607 IP 192.168.1.127 > 10.0.0.11: ICMP echo request, id 7859, seq 891, length 64
06:51:14.274199 IP 10.0.0.11 > 192.168.1.127: ICMP echo reply, id 7859, seq 891, length 64
06:51:15.275419 IP 192.168.1.127 > 10.0.0.11: ICMP echo request, id 7859, seq 892, length 64
06:51:15.276194 IP 10.0.0.11 > 192.168.1.127: ICMP echo reply, id 7859, seq 892, length 64
06:51:16.276754 IP 192.168.1.127 > 10.0.0.11: ICMP echo request, id 7859, seq 893, length 64
06:51:16.277464 IP 10.0.0.11 > 192.168.1.127: ICMP echo reply, id 7859, seq 893, length 64
06:51:17.277371 IP 192.168.1.127 > 10.0.0.11: ICMP echo request, id 7859, seq 894, length 64
06:51:17.278135 IP 10.0.0.11 > 192.168.1.127: ICMP echo reply, id 7859, seq 894, length 64
06:51:18.278647 IP 192.168.1.127 > 10.0.0.11: ICMP echo request, id 7859, seq 895, length 64
06:51:18.279358 IP 10.0.0.11 > 192.168.1.127: ICMP echo reply, id 7859, seq 895, length 64
**************************************************************************
Troubleshooting via qdhcp-xxxxxxxxxxx namespace
**************************************************************************
[root@dfw02 ~(keystone_andrew)]$ neutron net-list
+--------------------------------------+------+--------------------------------------------------+
| id | name | subnets |
+--------------------------------------+------+--------------------------------------------------+
| 426bb226-0ab9-440d-ba14-05634a17fb2b | int1 | 9e0d457b-c4c4-45cf-84e2-4ac7550f3b06 40.0.0.0/24 |
| 780ce2f3-2e6e-4881-bbac-857813f9a8e0 | ext | f30e5a16-a055-4388-a6ea-91ee142efc3d |
| 971fd74b-66e4-4016-b997-433e124a22ad | int2 | caca1f54-3211-4395-ba3c-a0c26d9605c0 10.0.0.0/24 |
+--------------------------------------+------+--------------------------------------------------+
[root@dfw02 ~(keystone_andrew)]$ ip netns | grep 971fd74b-66e4-4016-b997-433e124a22ad
qdhcp-971fd74b-66e4-4016-b997-433e124a22ad
[root@dfw02 ~(keystone_andrew)]$ ip netns exec qdhcp-971fd74b-66e4-4016-b997-433e124a22ad ifconfig
lo: flags=73
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 0 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ns-a2f724d1-74: flags=4163
inet 10.0.0.12 netmask 255.255.255.0 broadcast 10.0.0.255
inet6 fe80::f816:3eff:fe75:956c prefixlen 64 scopeid 0x20
ether fa:16:3e:75:95:6c txqueuelen 1000 (Ethernet)
RX packets 27 bytes 3464 (3.3 KiB)
RX errors 0 dropped 5 overruns 0 frame 0
TX packets 18 bytes 2992 (2.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@dfw02 ~(keystone_andrew)]$ ip netns exec qdhcp-971fd74b-66e4-4016-b997-433e124a22ad tcpdump -ln -i ns-a2f724d1-74
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ns-a2f724d1-74, link-type EN10MB (Ethernet), capture size 65535 bytes
09:35:05.337881 IP 10.0.0.11.bootpc > 10.0.0.12.bootps: BOOTP/DHCP, Request from fa:16:3e:b2:50:44, length 300
09:35:05.338468 IP 10.0.0.12.bootps > 10.0.0.11.bootpc: BOOTP/DHCP, Reply, length 328
09:35:10.348322 ARP, Request who-has 10.0.0.12 tell 10.0.0.11, length 28
09:35:10.348342 ARP, Reply 10.0.0.12 is-at fa:16:3e:75:95:6c, length 28
09:35:44.517803 IP 10.0.0.11.bootpc > 10.0.0.12.bootps: BOOTP/DHCP, Request from fa:16:3e:b2:50:44, length 300
09:35:44.518373 IP 10.0.0.12.bootps > 10.0.0.11.bootpc: BOOTP/DHCP, Reply, length 328
09:35:49.522353 ARP, Request who-has 10.0.0.11 tell 10.0.0.12, length 28
09:35:49.523370 ARP, Reply 10.0.0.11 is-at fa:16:3e:b2:50:44, length 28
09:36:35.222754 IP 10.0.0.11.bootpc > 10.0.0.12.bootps: BOOTP/DHCP, Request from fa:16:3e:b2:50:44, length 300
09:36:35.223247 IP 10.0.0.12.bootps > 10.0.0.11.bootpc: BOOTP/DHCP, Reply, length 328
09:36:40.226298 ARP, Request who-has 10.0.0.11 tell 10.0.0.12, length 28
09:36:40.227281 ARP, Reply 10.0.0.11 is-at fa:16:3e:b2:50:44, length 28
09:37:22.469672 IP 10.0.0.11.bootpc > 10.0.0.12.bootps: BOOTP/DHCP, Request from fa:16:3e:b2:50:44, length 300
09:37:22.470165 IP 10.0.0.12.bootps > 10.0.0.11.bootpc: BOOTP/DHCP, Reply, length 328
09:37:27.474303 ARP, Request who-has 10.0.0.11 tell 10.0.0.12, length 28
09:37:27.475363 ARP, Reply 10.0.0.11 is-at fa:16:3e:b2:50:44, length 28
************************
GATEWAY TESTING
************************
[root@dfw02 ~(keystone_boris)]$ neutron router-list
+--------------------------------------+---------+-----------------------------------------------------------------------------+
| id | name | external_gateway_info |
+--------------------------------------+---------+-----------------------------------------------------------------------------+
| 86b3008c-297f-4301-9bdc-766b839785f1 | router2 | {"network_id": "780ce2f3-2e6e-4881-bbac-857813f9a8e0", "enable_snat": true} |
+--------------------------------------+---------+-----------------------------------------------------------------------------+
[root@dfw02 ~(keystone_boris)]$ ip netns | grep 86b3008c-297f-4301-9bdc-766b839785f1
qrouter-86b3008c-297f-4301-9bdc-766b839785f1
[root@dfw02 ~(keystone_boris)]$ ip netns exec qrouter-86b3008c-297f-4301-9bdc-766b839785f1 ifconfig
lo: flags=73
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 0 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
qg-9c090153-08: flags=4163
inet 192.168.1.114 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::f816:3eff:feb2:24bc prefixlen 64 scopeid 0x20
ether fa:16:3e:b2:24:bc txqueuelen 1000 (Ethernet)
RX packets 7884 bytes 8246691 (7.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 3230 bytes 276065 (269.5 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
qr-e031db6b-d0: flags=4163
inet 40.0.0.1 netmask 255.255.255.0 broadcast 40.0.0.255
inet6 fe80::f816:3eff:fe83:9090 prefixlen 64 scopeid 0x20
ether fa:16:3e:83:90:90 txqueuelen 1000 (Ethernet)
RX packets 3427 bytes 295222 (288.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 6224 bytes 8112404 (7.7 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@dfw02 ~(keystone_boris)]$ neutron net-list
+--------------------------------------+------+--------------------------------------------------+
| id | name | subnets |
+--------------------------------------+------+--------------------------------------------------+
| 426bb226-0ab9-440d-ba14-05634a17fb2b | int1 | 9e0d457b-c4c4-45cf-84e2-4ac7550f3b06 40.0.0.0/24 |
| 780ce2f3-2e6e-4881-bbac-857813f9a8e0 | ext | f30e5a16-a055-4388-a6ea-91ee142efc3d |
+--------------------------------------+------+--------------------------------------------------+
[root@dfw02 ~(keystone_boris)]$ ip netns | grep 426bb226-0ab9-440d-ba14-05634a17fb2b
qdhcp-426bb226-0ab9-440d-ba14-05634a17fb2b
[root@dfw02 ~(keystone_boris)]$ ip netns exec qdhcp-426bb226-0ab9-440d-ba14-05634a17fb2b route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 40.0.0.1 0.0.0.0 UG 0 0 0 ns-343b0090-24
40.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 ns-343b0090-24
*************************************
Gateway test via qdhcp namespace
************************************
[root@dfw02 ~(keystone_boris)]$ ip netns exec qdhcp-426bb226-0ab9-440d-ba14-05634a17fb2b ping -c3 40.0.0.1
PING 40.0.0.1 (40.0.0.1) 56(84) bytes of data.
64 bytes from 40.0.0.1: icmp_seq=1 ttl=64 time=0.246 ms
64 bytes from 40.0.0.1: icmp_seq=2 ttl=64 time=0.125 ms
64 bytes from 40.0.0.1: icmp_seq=3 ttl=64 time=0.058 ms
--- 40.0.0.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.058/0.143/0.246/0.077 ms
[root@dfw02 ~(keystone_boris)]$ ip netns exec qdhcp-426bb226-0ab9-440d-ba14-05634a17fb2b ping -c3 192.168.1.114
PING 192.168.1.114 (192.168.1.114) 56(84) bytes of data.
64 bytes from 192.168.1.114: icmp_seq=1 ttl=64 time=0.225 ms
64 bytes from 192.168.1.114: icmp_seq=2 ttl=64 time=0.061 ms
64 bytes from 192.168.1.114: icmp_seq=3 ttl=64 time=0.070 ms
--- 192.168.1.114 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.061/0.118/0.225/0.076 ms
*****************************************
Gateway test via qroute namespace
*****************************************
[root@dfw02 ~(keystone_boris)]$ ip netns exec qrouter-86b3008c-297f-4301-9bdc-766b839785f1 ping -c3 40.0.0.1
PING 40.0.0.1 (40.0.0.1) 56(84) bytes of data.
64 bytes from 40.0.0.1: icmp_seq=1 ttl=64 time=0.051 ms
64 bytes from 40.0.0.1: icmp_seq=2 ttl=64 time=0.036 ms
64 bytes from 40.0.0.1: icmp_seq=3 ttl=64 time=0.049 ms
--- 40.0.0.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.036/0.045/0.051/0.008 ms
[root@dfw02 ~(keystone_boris)]$ ip netns exec qrouter-86b3008c-297f-4301-9bdc-766b839785f1 ping -c3 192.168.1.114
PING 192.168.1.114 (192.168.1.114) 56(84) bytes of data.
64 bytes from 192.168.1.114: icmp_seq=1 ttl=64 time=0.056 ms
64 bytes from 192.168.1.114: icmp_seq=2 ttl=64 time=0.047 ms
64 bytes from 192.168.1.114: icmp_seq=3 ttl=64 time=0.035 ms
--- 192.168.1.114 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.035/0.046/0.056/0.008 ms
******************************************************************
Verifying port status of private network at the router
******************************************************************
[root@dfw02 ~(keystone_andrew)]$ neutron router-list
+--------------------------------------+---------+-----------------------------------------------------------------------------+
| id | name | external_gateway_info |
+--------------------------------------+---------+-----------------------------------------------------------------------------+
| 0686e526-0584-4004-a8b0-7d64b376f0ce | router5 | {"network_id": "780ce2f3-2e6e-4881-bbac-857813f9a8e0", "enable_snat": true} |
+--------------------------------------+---------+-----------------------------------------------------------------------------+
[root@dfw02 ~(keystone_andrew)]$ neutron router-port-list router5
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| 52247b0b-d9c5-4602-a134-9e97a4049d62 | | fa:16:3e:c8:4b:56 | {"subnet_id": "caca1f54-3211-4395-ba3c-a0c26d9605c0", "ip_address": "10.0.0.5"} |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
[root@dfw02 ~(keystone_andrew)]$ neutron port-show 52247b0b-d9c5-4602-a134-9e97a4049d62
+-----------------------+---------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+---------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| device_id | 0686e526-0584-4004-a8b0-7d64b376f0ce |
| device_owner | network:router_interface |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "caca1f54-3211-4395-ba3c-a0c26d9605c0", "ip_address": "10.0.0.5"} |
| id | 52247b0b-d9c5-4602-a134-9e97a4049d62 |
| mac_address | fa:16:3e:c8:4b:56 |
| name | |
| network_id | 971fd74b-66e4-4016-b997-433e124a22ad |
| security_groups | |
| status | ACTIVE |
| tenant_id | 225c6392f7704cb685a779ff38d796cd |
+-----------------------+---------------------------------------------------------------------------------+
[root@dfw02 ~(keystone_andrew)]$ neutron net-list
+--------------------------------------+------+--------------------------------------------------+
| id | name | subnets |
+--------------------------------------+------+--------------------------------------------------+
| 426bb226-0ab9-440d-ba14-05634a17fb2b | int1 | 9e0d457b-c4c4-45cf-84e2-4ac7550f3b06 40.0.0.0/24 |
| 780ce2f3-2e6e-4881-bbac-857813f9a8e0 | ext | f30e5a16-a055-4388-a6ea-91ee142efc3d |
| 971fd74b-66e4-4016-b997-433e124a22ad | int2 | caca1f54-3211-4395-ba3c-a0c26d9605c0 10.0.0.0/24 |
+--------------------------------------+------+--------------------------------------------------+
[root@dfw02 ~(keystone_andrew)]$ ip netns | grep 971fd74b-66e4-4016-b997-433e124a22ad
qdhcp-971fd74b-66e4-4016-b997-433e124a22ad
[root@dfw02 ~(keystone_andrew)]$ ip netns exec qdhcp-971fd74b-66e4-4016-b997-433e124a22ad route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.0.5 0.0.0.0 UG 0 0 0 ns-a2f724d1-74
10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 ns-a2f724d1-74
[root@dfw02 ~(keystone_andrew)]$ ip netns exec qdhcp-971fd74b-66e4-4016-b997-433e124a22ad tcpdump -ln -i ns-a2f724d1-74
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ns-a2f724d1-74, link-type EN10MB (Ethernet), capture size 65535 bytes
11:11:04.396683 IP 10.0.0.11.bootpc > 10.0.0.12.bootps: BOOTP/DHCP, Request from fa:16:3e:25:9f:19, length 300
11:11:04.397222 IP 10.0.0.12.bootps > 10.0.0.11.bootpc: BOOTP/DHCP, Reply, length 328
11:11:09.401563 ARP, Request who-has 10.0.0.12 tell 10.0.0.11, length 28
11:11:09.401586 ARP, Reply 10.0.0.12 is-at fa:16:3e:75:95:6c, length 28
11:11:49.832274 IP 10.0.0.11.bootpc > 10.0.0.12.bootps: BOOTP/DHCP, Request from fa:16:3e:25:9f:19, length 300
11:11:49.832908 IP 10.0.0.12.bootps > 10.0.0.11.bootpc: BOOTP/DHCP, Reply, length 328
11:11:54.841355 ARP, Request who-has 10.0.0.12 tell 10.0.0.11, length 28
11:11:54.841374 ARP, Reply 10.0.0.12 is-at fa:16:3e:75:95:6c, length 28
[root@dfw02 ~(keystone_andrew)]$ ovs-vsctl show | grep a2f724d1-74
Port "tapa2f724d1-74"
Interface "tapa2f724d1-74"
[root@dfw02 ~(keystone_andrew)]$ tcpdump -ln -i tapa2f724d1-74
tcpdump: WARNING: tapa2f724d1-74: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tapa2f724d1-74, link-type EN10MB (Ethernet), capture size 65535 bytes
11:14:24.818170 IP 10.0.0.11.bootpc > 10.0.0.12.bootps: BOOTP/DHCP, Request from fa:16:3e:25:9f:19, length 300
11:14:24.818874 IP 10.0.0.12.bootps > 10.0.0.11.bootpc: BOOTP/DHCP, Reply, length 328
11:14:29.822081 ARP, Request who-has 10.0.0.11 tell 10.0.0.12, length 28
11:14:29.823144 ARP, Reply 10.0.0.11 is-at fa:16:3e:25:9f:19, length 28
11:15:17.246387 IP 10.0.0.11.bootpc > 10.0.0.12.bootps: BOOTP/DHCP, Request from fa:16:3e:25:9f:19, length 300
11:15:17.246828 IP 10.0.0.12.bootps > 10.0.0.11.bootpc: BOOTP/DHCP, Reply, length 328
11:15:22.249816 ARP, Request who-has 10.0.0.12 tell 10.0.0.11, length 28
11:15:22.249840 ARP, Reply 10.0.0.12 is-at fa:16:3e:75:95:6c, length 28
11:16:12.991922 IP 10.0.0.11.bootpc > 10.0.0.12.bootps: BOOTP/DHCP, Request from fa:16:3e:25:9f:19, length 300
11:16:12.992476 IP 10.0.0.12.bootps > 10.0.0.11.bootpc: BOOTP/DHCP, Reply, length 328
11:16:17.993603 ARP, Request who-has 10.0.0.12 tell 10.0.0.11, length 28
11:16:17.993624 ARP, Reply 10.0.0.12 is-at fa:16:3e:75:95:6c, length 28
********************************************
For every qrouter namespace
********************************************
[root@dfw02 ~(keystone_admin)]$ ip netns exec qrouter-0686e526-0584-4004-a8b0-7d64b376f0ce ifconfig
lo: flags=73
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 0 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
qg-edd58124-43: flags=4163
inet 192.168.1.107 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::f816:3eff:fec4:1408 prefixlen 64 scopeid 0x20
ether fa:16:3e:c4:14:08 txqueuelen 1000 (Ethernet)
RX packets 182061 bytes 246382615 (234.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 93023 bytes 7595030 (7.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
qr-52247b0b-d9: flags=4163
inet 10.0.0.5 netmask 255.255.255.0 broadcast 10.0.0.255
inet6 fe80::f816:3eff:fec8:4b56 prefixlen 64 scopeid 0x20
ether fa:16:3e:c8:4b:56 txqueuelen 1000 (Ethernet)
RX packets 93343 bytes 7629908 (7.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 179494 bytes 246083635 (234.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@dfw02 ~(keystone_admin)]$ ovs-vsctl list-ports br-ex | grep edd58124-43
tapedd58124-43
[root@dfw02 ~(keystone_admin)]$ ovs-vsctl list-ports br-int | grep 52247b0b-d9
tap52247b0b-d9
********************************************
For every qdhcp namespace
********************************************
[root@dfw02 ~(keystone_admin)]$ ip netns exec qdhcp-426bb226-0ab9-440d-ba14-05634a17fb2b ifconfig
lo: flags=73
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 0 (Local Loopback)
RX packets 1 bytes 576 (576.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1 bytes 576 (576.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ns-343b0090-24: flags=4163
inet 40.0.0.3 netmask 255.255.255.0 broadcast 40.0.0.255
inet6 fe80::f816:3eff:fe01:8b55 prefixlen 64 scopeid 0x20
ether fa:16:3e:01:8b:55 txqueuelen 1000 (Ethernet)
RX packets 15 bytes 1214 (1.1 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 8 bytes 648 (648.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@dfw02 ~(keystone_admin)]$ ovs-vsctl list-ports br-int | grep 343b0090-24
tap343b0090-24
*******************************
For my particular config
*******************************
[root@dfw02 ~(keystone_admin)]$ ovs-vsctl show
7d78d536-3612-416e-bce6-24605088212f
Bridge br-int
Port "tap52247b0b-d9"
tag: 3
Interface "tap52247b0b-d9"
Port br-int
Interface br-int
type: internal
Port "tapf933e768-42"
tag: 1
Interface "tapf933e768-42"
Port "tapa2f724d1-74"
tag: 3
Interface "tapa2f724d1-74"
Port "tap40dd712c-e4"
tag: 1
Interface "tap40dd712c-e4"
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port "tap343b0090-24"
tag: 2
Interface "tap343b0090-24"
Port "tape031db6b-d0"
tag: 2
Interface "tape031db6b-d0"
Bridge br-ex
Port "p37p1"
Interface "p37p1"
Port br-ex
Interface br-ex
type: internal
Port "tap54e34740-87"
Interface "tap54e34740-87"
Port "tapedd58124-43"
Interface "tapedd58124-43"
Port "tap9c090153-08"
Interface "tap9c090153-08"
Bridge br-tun
Port "gre-2"
Interface "gre-2"
type: gre
options: {in_key=flow, local_ip="192.168.1.127", out_key=flow, remote_ip="192.168.1.137"}
Port br-tun
Interface br-tun
type: internal
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
ovs_version: "2.0.1"
[root@dfw02 ~(keystone_admin)]$ ip netns list
qrouter-86b3008c-297f-4301-9bdc-766b839785f1
qrouter-bf360d81-79fb-4636-8241-0a843f228fc8
qrouter-0686e526-0584-4004-a8b0-7d64b376f0ce
qdhcp-971fd74b-66e4-4016-b997-433e124a22ad
qdhcp-426bb226-0ab9-440d-ba14-05634a17fb2b
qdhcp-1eea88bb-4952-4aa4-9148-18b61c22d5b7
References
1. http://techbackground.blogspot.ru/2013/05/the-quantum-l3-router-and-floating-ips.html
2. OpenStack Networking concepts
