Do you have a reference to the UIImageView? If so, then set its content mode to aspect fit. Like this:
theImageView.contentMode = .scaleAspectFit
The clipsToBounds you set only covers up any parts of child views that are sticking out of the bounds of the parent view, so that's why it doesn't do anything for you.
OR if you're using Interface Builder, set this option:

So, what if you don't have the reference to the UIImageView?...
You could iterate through the subviews of your scroll view, and whenever it finds a UIImageView, you can set the content mode like that. Something like:
//This is off the top of my head, so my filtering may not be right...
//This is also a one and done solution if you've got a lot of images in your scroll view
for anImgVw in imageScrollView.subviews.filter({$0.isKind(of: UIImageView.self)})
{
anImgVw.contentMode = .scaleAspectFit
}
Otherwise, I'm not sure if it's possible without a reference to the UIImageView.