can ping but not ssh - eth0 lan

Question

Stumped which direction to troubleshoot......

Server1 can ssh out to any computer but no other computer can ssh to Server1. Even if Server1 ssh to Server2 -> Server2 can not ssh back through same open ssh port to Server1.

Server1 can ssh to Server3 then through ssh tunnel ssh to Server2

Server1=>Server3=>Server2

Why can Server1 tunnel to Server3 through Server2 but not through Server3 back to Server1?

If connected by crossover cable Server1 can directly connect to Server2 but tunnel back to Server1 from Server2 is not possible.

Server1=>Server2=>[X]Server1

if connected to router

# ssh 192.168.4.1
# ssh: connect to host 192.168.4.1 port 22: Connection refused

# nc 192.168.4.1 22   (does not connect)

# nc -l 22  (on 192.168.4.1)
# nc: Address already in use

Server1: 192.168.4.2

Server2: 192.168.4.1

Server3: 192.168.4.3

Firewall disabled for testing. IP table accept all. Suspect mis-configured routing table....

Server1 Kernel IP routing table

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 wlan0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 wlan0
192.168.4.0     0.0.0.0         255.255.254.0   U     0      0        0 eth0
192.168.4.0     0.0.0.0         255.255.252.0   U     0      0        0 eth0

edit: reviewed old working saved " # route -n " outputs, metric for 192.168.4.0 (the subnet of the servers) is 100 . Is it possible the metric setting displayed by route -n prevent ssh tunneling?

edit:

# tcpdump -c 25 -i eth0 -v 

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
19:19:37.121131 IP (tos 0x10, ttl 64, id 19952, offset 0, flags [DF], proto TCP (6), length 176)
    homie.ssh > 192.168.4.6.38360: Flags [P.], cksum 0x89fb (incorrect -> 0xecf8), seq 3089869718:3089869842, ack 4078834853, win 271, options [nop,nop,TS val 11868537 ecr 3677449774], length 124
19:19:37.121627 IP (tos 0x10, ttl 64, id 22362, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.4.6.38360 > homie.ssh: Flags [.], cksum 0x8577 (correct), ack 124, win 501, options [nop,nop,TS val 3677449895 ecr 11868537], length 0
19:19:37.135066 IP (tos 0x10, ttl 64, id 19953, offset 0, flags [DF], proto TCP (6), length 184)
    homie.ssh > 192.168.4.6.38360: Flags [P.], cksum 0x8a03 (incorrect -> 0xe8ff), seq 124:256, ack 1, win 271, options [nop,nop,TS val 11868540 ecr 3677449895], length 132
19:19:37.135254 IP (tos 0x0, ttl 64, id 37912, offset 0, flags [DF], proto UDP (17), length 70)
    homie.57077 > gateway.domain: 17930+ PTR? 6.4.168.192.in-addr.arpa. (42)
19:19:37.135317 IP (tos 0x10, ttl 64, id 22363, offset 0, flags [DF], proto TCP (6), length 52)

Answer 1

routes

192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 wlan0
192.168.4.0     0.0.0.0         255.255.254.0   U     0      0        0 eth0
192.168.4.0     0.0.0.0         255.255.252.0   U     0      0        0 eth0

1. can't have same subnet with two different netmasks

2. If two different subnets both point to same gateway 0.0.0.0 traffic bond for internal gw will get 'lost'

   # sudo route del -net 192.168.4.0 gw 0.0.0.0 netmask 255.255.255.0 dev eth0

try now to use both networks separately. route is auto-generated so it "may" work by itself. if not, add route manually to remote ip (server), 192.168.4.0/24 does not seem to work, maybe someone can explain why....

`# route add -net 192.168.4.0 netmask 255.255.254.0 gw 192.168.4.2 eth0'

took a number of days to figure out the issue so posting here for my reference and maybe it will help someone with a similar problem looking for troubleshooting steps for ping but no ssh.