As a libertarian (small L) and as someone who has a deep interest in logic, philosophy and politics, this issue might perhaps be the most difficult issue I've ever encountered to develop a consistent view on.
My overall guiding framework with all issues is simple. I don't believe in the initiation of force except in cases of self-defense. I believe everyone should be left alone to do as they wish, so long as they are not infringing upon anyone else's natural rights. I'm a firm believer in the non-aggression principle.
Abortion is tricky and I've given it an immense amount of thought...
At this point in my life I am prepared to make the unequivocal statement that abortion is an immoral act. With that being said, it doesn't make those who have had abortions bad people and it doesn't make them immoral. Life isn't black and white. Good people make bad choices all the time.
I've also found no way around drawing the inevitable conclusion: Abortion is murder. At least without sacrificing logic and consistency.
Supporters of abortion will argue that it's the woman's body and it should be her choice. To that I say, you're right. It is your choice. The choice is made when you choose to have sex. Actions have consequences and pregnancy is an obvious consequence of having sexual relations. To which the next argument will be something like, "well not everyone knows about the risks associated with sex." Frankly, I think that argument is quite absurd in and of itself and even so, it doesn't matter. I'll reiterate. Actions have consequences. If you throw a rock over a bridge and that rock hits someone below you on the head, you are still responsible for the harm you've done to that person. You may not have intended for the rock to hit anyone but that does not matter. It's of no consequence. Ignorance has never been considered a valid argument in context of the law. That standard does not hold anywhere else in society, nor does it hold here.
This is typically when the red herring rape scenarios come into play. I don't take this argument very seriously as it accounts for less than 1% of all abortion cases. Even so, morally speaking the baby had nothing to do with the decision of the father to rape the mother so why it should suffer the consequences is beyond me. Two wrongs don't make a right.
As a society we've already established that the fetus has rights. If a pregnant woman were to be murdered, her assailant would be charged with the murder of both the mother and the child. It's not consistent to then say the mother can end the life of that fetus.
Then you've got the entirely separate issue of Planned Parenthood and it's racist origins... Margaret Sanger and her support of eugenics. It's disgusting.
So where do I go with all this? Who knows. Should be have laws restricting abortion? I don't know to what extent. But I do believe that we should be doing our best to encourage people to bring their children to term. Just on a moral basis, we should not be encouraging abortion or presenting it as the easiest way out. We should be making adoption an easier option for parents on both sides of the equation.
Back to the original question. Given my moral argument, I can absolutely tell you that the government and the tax payer should not be footing the bill for abortions in any way, shape, or form. That's an easy answer though because this holds true for any product or service. Governments have no right to force citizens to pay for something they do not want to pay for. This includes abortion, yes, but it also includes all health services in general. If you understood what "rights" are, you would know that health care is not one. Therefor a government has no moral authority to provide such services.