Metaphysical Skepticism --Does it imply egoism?

I have been reading a lot of ethics lately, including, if I may name drop, Sidgwick's renowned The Methods of Ethics.
And its gotten me thinking again about how metaphysical views can influence ethical views in a lot of ways, and I'm not just talking about the metaphysics of ethics either. And this certainly isn't a good thing, because metaphysics is tricky and contentious.
I have two arguments rolling around in my head with different conclusions based on different metaphysical views. The first one, an argument for what we might call "practical egoism," is what I want to go over here. Perhaps someone can point out if I have gone wrong.
First, the metaphysical view that my argument rests on--let's call it "metaphysical skepticism." Metaphysical skepticism doesn't deny that there are metaphysical truths, but that they are unknowable by human beings. Its a very reasonable view, and something roughly like what I have in mind is what Colin McGinn argues for in Problems of Philosophy: The Limits of Inquiry.
Most importantly for my current argument is the view that the problem of other minds is unsolvable. The problem of other minds is the problem of knowing the mental states of other persons besides ourselves, or even if they have mental states at all.
Everyone practically believes that all humans have mental states and feelings just like their own, based on inferences from their behavior--Colin winces in pain when I kick him in the shin, just as I would; therefore he must be feeling the same way I do when it happens to me.
Now we couldn't really get around very well in our day-to-day lives without assuming this--its perhaps an evolutionary necessity that we assume persons move and act intentionally and feel similar qualitative states to our own. But this inference isn't really a valid one--its a generalization from an sample size of 1--yourself. And in no other area of inquiry would that fly. So there is the problem of other minds.
Metaphysical skepticism says that this problem is unsolvable--we can't, even in principle, have any evidence for or against the existence of minds other than our own.
Philosophically reasonable position, right?
Ok, so much for our metaphysical basis.
Nothing ethically harmful follows from this, does it? How could it? Well, actually, for all practical purposes, the rational believer of metaphysical skepticism should be an egoist, as far as I can tell. Say I am a consequentialist--as long as my ultimate ends is something mental or intentional, such as desire-fulfillment or something more like hedonism, I can only safely assume that I am a being with desires and qualitative states like pleasure and pain. Because I have no idea about other beings and objects, I rationally ought not risk trying to increase their pleasures or desire-fulfillment. For maybe these other beings are zombies, without qualitative states. Or maybe they're even worse than zombies--lacking both qualitative and intentional states (if thats possible--some people define desires and intentions behavioristically, so thats one 'out' for the metaphysical skeptic). But even if zombies are metaphysically impossible, (which, it should be noted, the metaphysical skeptic doesn't confirm or deny) there is no way of knowing if Colin McGinn's wincing is actually represented in his mind pleasurably, and he feels great joy and exhileration at the feeling of tissue damage, though his behavior doesn't show this in any way. Its impossible for the metaphysical skeptic to rule this out.
Now, as noted above, its not really feasible for the skeptic about the problem of other minds to live her life as if other people don't have minds--its psychologically impossible to doubt it, I presume. But nevertheless, from a rational standpoint, the metaphysical skeptic should seemingly try to suppress this thought as much as possible. To try to please the people around us would be like trying to please a rock--we have no idea how to do it, so it is fruitless to try. The metaphysical skeptic is in the same position with human beings.

But receding to a more Kantian view will not help here, for a simpler reason--respect for persons goes out the window when we have no safe way of deciding who are persons. We could have a strictly scientific test for personhood, I suppose, but I don't think its feasible in practice. This is a point for another post, but it seems like any strictly scientific test of personhood is going to either be too wide, and bring in things like computers, or too narrow, and rule out infants or intelligent non-human animals. But, I admit, that is another 'out' for the metaphysical skeptic, because that is a contentious point.

Either of these stances, consequentialism or deontology, when coupled with metaphysical skepticism, appear to imply, for all practical purposes, egoism. Though both the utilitarian and Kantian agree in principle to treat all sentient beings/persons with respect, neither can know what beings fall into the proper category, and are thus hopeless to take moral action, outside of their own mental life.
Now I think there are ways to weasel out of these conclusions, but I don't immediately see how. Anyone have any ideas?