Every collection starts out small. Whether we're talking books, music, video games, toys, whatever. It all has to start somewhere. Don't start buying things with an end game in mind. It will distract you from the true reason why you wanted to collect in the first place, you enjoy what you're collecting.
With that in mind, here's some things to keep in mind.
1) Buy what you think you'll enjoy. Don't worry about having something on your shelf just to impress others. Your money and attention are precious, so focus it on the things you want.
2) Don't be afraid to spend a little more for quality. You're already asking about binding and different editions and all, and that's great. If you want to take a little more time or spend a little more money for that perfect version, do it. It's worth it in the long run.
3) Don't be afraid to trade up. Did you really enjoy "Smoke From This Alter" by Louis L'Amour? Did you want to find a first edition copy? Buy a modern copy now, to fill in that gap in your collection, and if and when you come across on a good deal on that first edition, that's when you decide whether or not to trade up. Maybe by that time, you'll think "Hey. I don't want to spend hundreds of dollars on a book I already have. That's silly."
4) Let your collection grow organically. This is important for a lot of reasons. Your tastes slowly change over the years, whether you realize it or not. If you buy everything you want today, the you 5 or 10 years from now might those particular titles as much and is miffed at the you from 5 years ago because you blew all your money and now you're afraid to buy more. Also, because your collection grows slowly and over time, it gives you time to get organization under control.
5) Insure your collection. Renters insurance. Homeowners insurance. What have you. Talk to your insurance agent. Let them know what you have and that you want to protect it. You might have to get special coverage, but it's worth it to protect your investment. It's also important to understand the terms of your coverage. Make sure that your $5000 first edition of Mice and Men is actually covered for that amount, and not say $5.
6) Speaking of money, collecting is a hobby. Not an investment. Unless you're SUPER savvy about what you're doing, you'll never make as much out of your collection as to what you spent on it. It's possible to do if you really want, but it requires further investment in time and effort to educate yourself on your hobby.
7) Most importantly, don't be afraid to let things go. You own your things, don't let them own you. It's hard. Collecting is an investment in time, money, and emotions. Sometimes though, you gotta cull your collection for whatever reason. If you aren't willing to let go, you've lost control and you need to find a way to get it back. Collecting is a hobby. Hoarding is a problem.