Yes, it's going to be very tough especially in this economy.  However, it's possible, but will likely be a very difficult route to take.

I was in a similar situation 3 years ago when I had to quit IT due to the horrible IT job market and tried going full time with tax.  I wanted to do tax with a CPA firm for a while because CPA firms tend to get the best tax engagements and have work year-round.

To begin with, CPA firms want CPAs or serious CPA candidates, so when you don't have an accounting degree, it's difficult to convince them that you'll soon become a CPA.  I didn't even want to get into accounting, just wanted some full-time tax experience after doing taxes on the side for a few years.  After a few interviews with CPA firms who quickly complained that I'm not a CPA and rejected me, I got a job at my last CPA firm after convincing them that I would go back to school for an accounting degree to become a CPA.

And that's what I've been doing for the last 3 years.  It was really tough, especially when I was in school.  A lot of people know how tough it is to study for the CPA exam while working full time in public accounting, but it's even tougher to be going to school.  Especially when it's tax season.  So often I have felt overwhelmed, depressed, frustrated, and sad.

Now I'm waiting on my last score and pray that it will be a pass so that I will finally reach the end of this tunnel soon.  I really hope it will be worth it.