the app itself has a low res considering all the other applications i run on my mac
You seem to have far more experience with graphics than I do. I have not noticed a problem with resolution playing iStripper on either my MacBook Pro or on a 4K 40" display on my 2013 Mac Pro.
its also very choppy and lags all the time
This is a problem. I've been reporting it for years with no improvement. For me it is a random problem that lasts 10-20 minutes and then goes away. When it is present, the latency for even a mouse click can be more than 2 minutes and the model will freeze every so often. This problem is much worse when I try to play more than one card in small mode on multiple displays on my Mac Pro. Since, for me, this problem only occurs occasionally, I've simply waited it out, but I'd really like Totem to fix it.
if i download 720p then i cant really scale that much
This is true. A 720p show has a nominal height of about 700 pixels but that includes enough space above the model's head for her arms. If you try to play a 720p card at a scale of more than about 40% on your MacBook you will begin to see jaggies since iStripper is trying to scale up the native image. I never play 720p card at more than 25% on my 4K display for this reason.
if i download 3k then i can scale it to a 100+
You probably will see some degradation in quality about 90%.
Try this experiment: Try playing different cards at different scales and turn the "Don't zoom over 100% of the clip's original size." checkbox in Settings on and off. When the scale is high enough you will see that some images are being scaled up.