I suggest taking Calc II & III as a non-degree seeking student and see if your passion sustains itself.
This, except instead of Calc III I suggest taking a proofs course. Calculus is quite different from most courses in the math major and a proofs course will give you a better flavor for what to expect in an undergrad math major.
I also think what you should do depends on the expense of this path. If you enjoy learning the material and it doesn't cost you much then so what if you remain a middle school math teacher? You have enriched your life for the learning and perhaps have improved your teaching by putting yourself in your students' shoes for a bit. I wouldn't pay the cost of a an entire undergrad BA though unless you expect some kind of return on that investment.