I graduated recently, and believe it or not, I can already notice some of my academic skills starting to go. Not necessarily theory or critical thinking, but simple things. Spelling, being unable to form a sentence the right way, not being able to think of the right word to use which would fit perfectly, etc.
Not to be selfish, but... it's part of the reason I'm here. I get to type out my thoughts and stay sharp. I have a CPA exam to complete before this year is over as well, and the idea of not being sharp mentally is disconcerting.
Now that I'm typing stream of consciousness, I'm now realizing that I might actually be having withdrawal from writing papers. Side note - I just couldn't figure out how to spell withdrawal because of my dumb Baltimorean accent.
Relax. It's like riding a bike. Four years after graduating I had to solve a problem for a client using dimensional analysis; I had to look at my textbooks again but if you truly learned it the first time, you'll relearn anything you need in minutes.
You're fine. If you've ever taken foreign language classes in k-12 or college you're aware that once you stop doing a executing a particular task, you get worse at it. The more you've done in the past, the slower the decline, but you'll degrade all the same. I think you'll find now that you're out of school you're going to start to become slightly worse at 'being a student' because...you're no longer a student doing the things that a student does. You may be reading less, writing fewer papers (spelling less), solving fewer mathematical equations that don't have a specific application other than the textbook asked you to, etc. But while those skills degrade a bit, you'll be spending your time on other things. Right now your skill at studying for the CPA exam and test taking is improving (along with a set of complimentary 'halo' skills), and after that you'll get a job and spend your time learning whatever skills a CPA uses every single day. You'll get a little worse at some things while you're busy getting really good at other things, and you'll always be able to brush up on the old stuff pretty easily. Also, outside of academia, there is no right or wrong way to get an answer you need. You'll find you already have all the tools you need to research and answer, draft a paper, solve a math problem, etc. It's all open-book now ;)