The only downside is that you have more libraries in your computer (i.e. the gtk2 libraries) some consider this somehow "polluting" your setup with unnecessary libraries/dependencies, there are Qt purists and GTK purists out there which absolutely refuse to use anything that relies on anything else than their favorite gui library.
But the way I look at it, if there would be downsides, they would be KDE's fault, not GTK2's (not even Qt's). I've noticed discrepancies in launch times (and assume there could be some discrepancies in performance, equally minor) of programs in various desktop environments. Usually Qt based environments will launch Qt based programs faster than GTK based ones, and vice versa (not absolute, but it is commonly the case). This was a bigger problem in the past. But now... who gives a damn about a couple of milliseconds difference? My favorite environment is Enlightenment, it is based on EFL (neither Qt nor GTK) everything worked fine (Qt and GTK applications) within that.
So no you are not missing anything, and you should only choose your favorite programs as your primary choices, disregard everything else. You customize the system to your needs and your desires, not your environments needs or someone else's ideologies. Never forget that, otherwise what would be the point of customizing things to begin with? If you don't notice a difference, everything is working just fine, that's what matters.
My favorite image editor (and well since I like to paint as well) is Krita, it is based on KDE framework, and with KDE framework it pulls in a mountain of terrible fat KDE-related dependencies (not a problem if I'd already use KDE, but sadly it just doesn't agree with me). This doesn't stop me from using it. Not on Enlightenment, not Openbox, not Xmonad, not XFCE, not anything because that is the program I want, and while the dependencies are many and fat, it's not like they're getting in my way somehow, they're not slowing down my system, they're just making it a couple hundred megabytes larger than it otherwise would be, that's a price I'm simply willing to pay. Are you?
Similarly, Krita might take a second less to launch under KDE than other environments, should I allow that to stop me from using something else? I don't think so, if I don't like KDE but happen to like a KDE associated program, I will just use that program on my favorite environment regardless.