Yes, smart tvs have very limited capability. I've had many customers ask me why they have issues with thier streaming services and when I ask them what they are using to stream they say thier tv. I always tell them to go with android streaming devices. 4k firestick is a good start, a MI box s...
In my opinion I would definitely use a streaming device android based. I use Nvidia shield and they work great. Yes your internet affects things as well but android devices do perform better than using your smart TV.