Routing basics: build two routers, break them, fix them

Four boxes in a line: pca → r1 → r2 → pcb. Each veth pair is a cable with one end in each box.

# for n in pca r1 r2 pcb; do ip netns add $n; ip -n $n link set lo up; done
# ip link add a-r1 netns pca type veth peer name r1-a netns r1
# ip link add r1-r2 netns r1 type veth peer name r2-r1 netns r2
# ip link add r2-b netns r2 type veth peer name b-r2 netns pcb
# ip -n pca link set a-r1 up; ip -n r1 link set r1-a up; ip -n r1 link set r1-r2 up
# ip -n r2 link set r2-r1 up; ip -n r2 link set r2-b up; ip -n pcb link set b-r2 up

Interface names tell you where each cable goes: r1-r2 is r1’s end of the cable to r2.

Three subnets: 10.1.1.0/24 on the left, 10.2.2.0/24 on the right, and a tiny 10.0.12.0/30 between the routers (a /30 has exactly two usable addresses, one per end). Then let r1 and r2 forward packets, which is the one setting that makes a Linux box a router.

# ip -n pca addr add 10.1.1.10/24 dev a-r1
# ip -n r1 addr add 10.1.1.1/24 dev r1-a
# ip -n r1 addr add 10.0.12.1/30 dev r1-r2
# ip -n r2 addr add 10.0.12.2/30 dev r2-r1
# ip -n r2 addr add 10.2.2.1/24 dev r2-b
# ip -n pcb addr add 10.2.2.10/24 dev b-r2
# ip netns exec r1 sysctl -w net.ipv4.ip_forward=1
# ip netns exec r2 sysctl -w net.ipv4.ip_forward=1

Look at what r1 and pca know right now.

# ip -n r1 route
# ip -n pca route

Our run printed, for r1 then pca:

10.0.12.0/30 dev r1-r2 proto kernel scope link src 10.0.12.1
10.1.1.0/24 dev r1-a proto kernel scope link src 10.1.1.1

10.1.1.0/24 dev a-r1 proto kernel scope link src 10.1.1.10

Reading it. Every line is “to reach this network, use this interface”. proto kernel scope link means a connected route: Linux added it by itself the moment you gave the interface an address. Nobody has mentioned 10.2.2.0/24 to anyone yet.

Ping pcb from pca, add default gateways on both PCs, and ping again.

# ip netns exec pca ping -c 1 -W 1 10.2.2.10
# ip -n pca route add default via 10.1.1.1
# ip -n pcb route add default via 10.2.2.1
# ip netns exec pca ping -c 1 -W 1 10.2.2.10

The first ping fails instantly with connect: Network is unreachable: pca has no route at all for 10.2.2.10, so the packet never leaves. The second gets further and fails differently: From 10.1.1.1 icmp_seq=1 Destination Net Unreachable.

Reading the errors. “Network is unreachable” comes from your own machine (fix: a route or default gateway on the PC). “Destination Net Unreachable” from 10.1.1.1 is r1 replying “I got your packet but I have no route for it”. The address after “From” tells you which device to fix next.

Tell r1 that 10.2.2.0/24 is reached through r2’s end of the link.

# ip -n r1 route add 10.2.2.0/24 via 10.0.12.2
# ip -n r1 route

A new line appears: 10.2.2.0/24 via 10.0.12.2 dev r1-r2. via names the next hop: the neighbour to hand the packet to, not the final destination.

Ping again while capturing on r2’s side of the router link, then ask r2 how it would reach pca.

# ip netns exec r2 tcpdump -n -l -i r2-r1 -c 2 icmp &
# ip netns exec pca ping -c 2 -W 1 10.2.2.10
# ip -n r2 route get 10.1.1.10
# ip netns exec r2 nstat -az TcpExtIPReversePathFilter

The ping still shows 100% packet loss, yet tcpdump on r2 prints both echo requests arriving (IP 10.1.1.10 > 10.2.2.10: ICMP echo request). r2’s answer to the route question is Network is unreachable, and in our run the TcpExtIPReversePathFilter counter read 2: one drop per ping.

What happened. r2 can reach pcb, but it has no route back to 10.1.1.0/24. Ubuntu turns on reverse-path filtering (rp_filter; it was set to 2, “loose”, on our machine), so r2 drops any packet whose sender it couldn’t reply to. Without that filter the request would reach pcb and the reply would die at r2 instead. Either way the cause is the same: routing has to work in both directions.

The request got there. The answer didn’t. I call that a group chat. — Nyx

The defender’s angle. Reverse-path filtering is also an anti-spoofing control: a packet claiming to come from a network that couldn’t possibly be behind that interface gets dropped at the door. Internet providers apply the same idea at their network edge to stop forged source addresses.

# ip -n r2 route add 10.1.1.0/24 via 10.0.12.1
# ip netns exec pca ping -c 2 10.2.2.10
# ip netns exec pca traceroute -n 10.2.2.10

The ping finally works: 2 received, ttl=62. The packet started at 64 and lost one for each router it crossed. Our traceroute:

traceroute to 10.2.2.10 (10.2.2.10), 30 hops max, 60 byte packets
 1  10.1.1.1  0.020 ms
 2  10.0.12.2  0.073 ms
 3  10.2.2.10  0.013 ms

Why this matters. Traceroute lists each router that sends back a “time exceeded” message, so you can see exactly where along the path a broken network stops answering.

# ip -n pca route get 10.2.2.10
# ip -n r1 route get 10.2.2.10

Ours printed 10.2.2.10 via 10.1.1.1 dev a-r1 src 10.1.1.10 for pca and 10.2.2.10 via 10.0.12.2 dev r1-r2 src 10.0.12.1 for r1.

Why this matters. ip route get (on Windows, Find-NetRoute -RemoteIPAddress) answers “which route wins for this exact address?” without you reading the whole table. On a machine with VPNs and several interfaces, that saves a lot of guessing.

Add a /32 route for pcb alone on r1 that throws packets away, test, then remove it.

# ip -n r1 route add blackhole 10.2.2.10/32
# ip -n r1 route
# ip netns exec pca ping -c 1 -W 1 10.2.2.10
# ip -n r1 route del blackhole 10.2.2.10/32

The table now holds both 10.2.2.0/24 via 10.0.12.2 and blackhole 10.2.2.10, and the ping dies silently (100% packet loss, no error). Remove the /32 and it works again.

Why. When several routes match, the router picks the most specific one, the one with the longest prefix. A /32 beats a /24 every time, whatever order they were added in.

The attacker’s view. Announce a more specific route than the real owner, and traffic follows you. That is the basic mechanism behind BGP hijacks on the internet; the best-known case is 2008, when Pakistan Telecom announced a more-specific route for YouTube addresses and briefly pulled much of the world’s YouTube traffic to itself. Inside a network, a rogue or mistaken static route does the same thing quietly.

The defender’s move. Know what your routing tables should contain and check them (ip monitor route prints every change as it happens). On the internet side, networks filter route announcements and validate them with RPKI.

# for n in pca r1 r2 pcb; do ip netns del $n; done
# exit

Deleting the namespaces removes the cables and every route you added. Your machine’s own routing table was never touched.