Yes, it really isn't all that easy, unfortunately. I pretty much handle it the same as you do, although sideloading apks comes with its own share of potential problems and we really need to watch what we're installing that way. Simply grabbing them from the playstore is one of the better options, but of course not always an option (since the DRM stuff the apps are looking for isn't on the phone.)

At the end of the day, what goes for the computer goes for the smartphone x 10: Consider everything done on it at least potentially publicly available or worse.