I studied Japanese a lot on my own, but did take courses for 6-7 years in high school and college also! I haven’t become fluent, but what you learn teaches you how to find the rest of the knowledge you need. Usually it’s just vocabulary that you need to look up (the same as English). The grammar is a lot more organized and strict than the English language, so I think it’d be totally worth it to learn!!
I’ve noticed also, that some people are better at reading or speaking, usually because of their hobbies. The ones with better reading/grammar tend to be ones who read novels and news in Japanese. The ones better at speaking usually watch a lot of Japanese TV (not just anime) and use it more often in their daily lives.
My Japanese has gotten rusty, so I hope to start practicing again. There are a few Japanese video games I need to play through, and TV shows to watch. I might start reading manga again… hmmmm laugh Perhaps, I can get the chance to go to Japan again when I get rich and jobless! [Oxymoron?]