Keep it tucked in your glove compartment. You never know when you'll be nearly out of gas or desperate for food on the road at a place that won't take your card for whatever reason. Twenty bucks can seem like nothing these days. When it's a meal, a shower and a couple gallons of gas at a truck stop, it's the best double saw buck ever.
Getting an EZPass was one of the best decisions I've made since I do a decent amount of traveling. Having something that allows you to drive on through without waiting for a line at a toll, and having it all done electronically is such a nice thing.
But you are tracked, in a way which is hardly consequential given that you most likely have your phone on your person and definitely have a credit card that you are bound to use at your destination, but is nonetheless a concern.
This is good advice. I make sure to always have at least some cash in my car. Usually just $5 or so for parking meters. My wife always has a $20 in her car though. Good call.