To have routing tables be adequately selected by routing rules with an interface bound using SO_BINDTODEVICE, the keyword oif should be used:

oif NAME

select the outgoing device to match. The outgoing interface is only available for packets originating from local sockets that are bound to a device.

Else during initial lookup, when no source address is also specified, the lookup is using the default INADDR_ANY (aka 0.0.0.0/0) as source selector before figuring out the correct source IP address later (eg from hinted source or other algorithms). As no added routing rule will match INADDR_ANY all the additional tables are skipped and this will end up evaluating only with the main routing table with an additional filter by the bound device: no default route is defined and thus no adequate gateway is used, except for br0 which has the default route also in the main table.

ip rule add oif br0 lookup 3
ip rule add oif br1 lookup 4
ip rule add oif br2 lookup 5
ip rule add oif br3 lookup 6

All the kernel evaluation can be verified with ip route get:

ip route get

get a single route

this command gets a single route to a destination and prints its contents exactly as the kernel sees it.

Before the additional rules

• br0

  Assuming that for the sake of symmetry there's also a routing table like this (but it doesn't really matter: the main routing table is good enough for br0):

  default via 10.3.0.1 dev br0 table 3 
  10.3.0.0/16 dev br0 table 3 scope link src 10.3.15.22 
  
  
  # ip route get from 10.3.15.22 to 8.8.8.8
  8.8.8.8 from 10.3.15.22 via 10.3.0.1 dev br0 table 3 uid 0 
      cache 
  # ip route get oif br0 to 8.8.8.8
  8.8.8.8 via 10.3.0.1 dev br0 src 10.3.15.22 uid 0 
      cache 
  

  In the 2nd case, when just binding to the interface without selecting a source address, the main table gets used, but that's still correct, because the main table also has the default route for br0: it works

• br1

  # ip route get from 10.4.15.22 to 8.8.8.8
  8.8.8.8 from 10.4.15.22 via 10.4.0.1 dev br1 table 4 uid 0 
      cache 
  # ip route get oif br1 to 8.8.8.8
  8.8.8.8 dev br1 src 10.4.15.22 uid 0 
      cache 
  

  In the 2nd case, as there's nothing defining a gateway in the main table, the route uses no gateway and because the interface is forced, it's assumed that 8.8.8.8 is directly connected (tcpdump would probably show ARP queries emitted about 8.8.8.8 if anything).

  It doesn't work.

After the additional rules for the bound interface case

• br0

  # ip route get oif br0 to 8.8.8.8
  8.8.8.8 via 10.3.0.1 dev br0 table 3 src 10.3.15.22 uid 0 
      cache 
  

  While the overall result is the same: it works, this time the additional table was used.

• br1

  # ip route get oif br1 to 8.8.8.8
  8.8.8.8 via 10.4.0.1 dev br1 table 4 src 10.4.15.22 uid 0 
      cache 
  

  This time, the oif br1 selector selected routing table 4 which does define a proper default route with a proper gateway: it works.

Note: A similar setup using L3 interfaces (eg: VPN interfaces like WireGuard or OpenVPN in tun mode) wouldn't require this, because L3 interfaces don't need any gateway (even if defined it's not used since there's no L2 layer below IP to resolve) and thus no specific route definition is needed for traffic to be emitted when binding to such L3 interface. Anyway it's still a good idea to use it too, especially if the routing table contains routing negations.