Bind serial device address to specific usb port

My OS: Raspbian Stretch Lite

tl;dr;

1. I have 2 identical USB device
2. I want to make sure their address didn't get confused
3. My thinking goes to bind the address to specific USB Port. I'm fine to "sacrifice" the port
4. Since I'm new on linux, I don't know how to do it. As far as my knowledge goes, udev differentiate device by it's ID, not port (different manufacturer, models, etc)
5. Is there any other recommended method?

Long version:
So, I have this UART-to-USB converter with the same manufacturer, so it have the same ID in udev and all. I want to make sure that the 2 device didn't get confused. For example, /dev/ttyUSB0 will always point to the specific hardware that I wanted (e.g.: inside. The 2 device will be separated by a door, so it's really important not to mess which is outside and inside).

I don't know how to accomplish this. My thinking goes to bind the specific port so that when the device is plugged to those specific port, it'll always give the address ttyUSB0 to that device. Is it even possible to do that? I'm new to linux and I know there's udev rule. But as far as I can find, it only bind device by it's id, not port. Or is there any other recommended way to do it using udev?

Here's my udevadm for the device:

  looking at device '/devices/pci0000:00/0000:00:14.0/usb3/3-2/3-2:1.0/ttyUSB0/tty/ttyUSB0':
    KERNEL=="ttyUSB0"
    SUBSYSTEM=="tty"
    DRIVER==""

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb3/3-2/3-2:1.0/ttyUSB0':
    KERNELS=="ttyUSB0"
    SUBSYSTEMS=="usb-serial"
    DRIVERS=="cp210x"
    ATTRS{port_number}=="0"

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb3/3-2/3-2:1.0':
    KERNELS=="3-2:1.0"
    SUBSYSTEMS=="usb"
    DRIVERS=="cp210x"
    ATTRS{authorized}=="1"
    ATTRS{bAlternateSetting}==" 0"
    ATTRS{bInterfaceClass}=="ff"
    ATTRS{bInterfaceNumber}=="00"
    ATTRS{bInterfaceProtocol}=="00"
    ATTRS{bInterfaceSubClass}=="00"
    ATTRS{bNumEndpoints}=="02"
    ATTRS{interface}=="CP2102 USB to UART Bridge Controller"
    ATTRS{supports_autosuspend}=="1"

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb3/3-2':
    KERNELS=="3-2"
    SUBSYSTEMS=="usb"
    DRIVERS=="usb"
    ATTRS{authorized}=="1"
    ATTRS{avoid_reset_quirk}=="0"
    ATTRS{bConfigurationValue}=="1"
    ATTRS{bDeviceClass}=="00"
    ATTRS{bDeviceProtocol}=="00"
    ATTRS{bDeviceSubClass}=="00"
    ATTRS{bMaxPacketSize0}=="64"
    ATTRS{bMaxPower}=="100mA"
    ATTRS{bNumConfigurations}=="1"
    ATTRS{bNumInterfaces}==" 1"
    ATTRS{bcdDevice}=="0100"
    ATTRS{bmAttributes}=="80"
    ATTRS{busnum}=="3"
    ATTRS{configuration}==""
    ATTRS{devnum}=="57"
    ATTRS{devpath}=="2"
    ATTRS{idProduct}=="ea60"
    ATTRS{idVendor}=="10c4"
    ATTRS{ltm_capable}=="no"
    ATTRS{manufacturer}=="Silicon Labs"
    ATTRS{maxchild}=="0"
    ATTRS{product}=="CP2102 USB to UART Bridge Controller"
    ATTRS{quirks}=="0x0"
    ATTRS{removable}=="removable"
    ATTRS{serial}=="0001"
    ATTRS{speed}=="12"
    ATTRS{urbnum}=="1909"
    ATTRS{version}==" 1.10"

Note: If binding a port to a specific serial address means I can't use the port for any other device, then it's fine to me