Split tunneling question and egressing out the correct interface for inbound connections

[Philip Prindeville]

Hi,

I have the following configurations.  I use wireguard but the same would be true with IPsec or GRE, I think.

Network:

...
config interface 'sandbox'
	option device 'eth1.4'
	option proto 'static'
	option ipaddr '192.168.4.1/24'
	option ip6assgin '60'
	# added by philipp
	option igmp_snooping 1

config interface 'wan'
	option device 'eth3'
	option proto 'dhcp'

config interface 'wg0'
	option proto 'wireguard'
	option private_key '...'
	option listen_port '10366'
	## our v4 SNAT'd address
	list addresses '50.20.195.61/31'
	# rely on the default route to our endpoint
	option nohostroute 1

config wireguard_wg0
	option public_key '...'
	option endpoint_host '50.20.195.1'
	option endpoint_port '10366'
	# no, we'll use PBR
	option route_allowed_ips '0'
	option persistent_keepalive '3'
	list allowed_ips '0.0.0.0/0'
	list allowed_ips '::/0'
	option defaultroute '0'

config route
	option target '0.0.0.0/0'
	option interface 'wg0'
	option table 200
	option proto 'static'
	# this is an oddity of Coretransit
	option source '50.20.195.61'

# mail traffic should egress via coretransit (see route immediately above)
config rule
	option name 'send SMTP via coretransit'
	option src '192.168.4.49/32'
	option in 'sandbox'
	option ipproto 'tcp'
	option dport '25'
	option lookup '200'

# without this rule my return traffic tries to egress via wan (my default gateway)
# with the wrong SNAT'd (masqueraded) address.
config rule
	option name 'send SMTP via coretransit back'
	option src '192.168.4.49/32'
	option in 'sandbox'
	option ipproto 'tcp'
	option sport '25'
	option lookup '200'

# traffic sourced from this interface should egress this interface
config rule
	option src '50.20.195.61/32'
	option lookup 200


So the problem is that traffic that came in from outside wants to egress via my 'wan' interface as that's the default route out.  Even doing a ping of 50.20.195.61 (without that last rule).

How do I force traffic that was an inbound connection to egress via the same interface it came in on?  The connection table in iptables should track that, right?

Firewall:

...
# where I keep my servers, only explicit traffic in or out
config zone
	option name		sandbox
	list   network		'sandbox'
	option input		REJECT
	option output		REJECT
	option forward		REJECT
	option log		1
	option log_limit	'30/s'

config zone
	option name		wan
	list   network		'wan'
	list   network		'wan6'
	option input		DROP
	option output		ACCEPT
	option forward		DROP
	option masq		1
	option mtu_fix		0
	option log		1
	option log_limit	'30/s'

config zone
	option name		wan2
	list   network		'wg0'
	option input		DROP
	option output		ACCEPT
	option forward		DROP
	option masq		1
	option mtu_fix		1
	option log		1
	option log_limit	'30/s'

config forwarding
	option src		lan
	option dest		wan

config forwarding
	option src		lan
	option dest		wan2

...
# make sure that mail can egress via Coretransit
config rule
	option name		Allow-SMTP-Sandbox
	option src		sandbox
	option proto		tcp
	option dest_port	25
	option dest		wan2
	option target		ACCEPT
	option family		ipv4

...
config redirect
	option name		'portforward inbound SMTP'
	option target		DNAT
	option src		wan2
	option dest		sandbox
	option src_dport	25
	option dest_port	25
	option proto		tcp
	option dest_ip		192.168.4.49
	##option reflection	0


The problem is that other services (HTTP, HTTPS, IMAP/S, Submission) might be connected to internally (via 'lan'), or externally (via 'wan2') and the traffic needs to do the right thing.

How do I leverage the connection table to do that?  Or what mechanisms exist in firewall4 or pbr to make sure it happens correctly?

Reflection only affects internal traffic trying to reach a redirected port via the external address, right?

Thanks,

-Philip