There is a good amount of reasons why many people don’t play it.
As Mixah said, at the beginning of its life span the game had a lot of glitches, it had to deal with a lot of bad press due the fiasco of the DLC chars and the DLC gems.
Add that everything that many hated from the SF4 series was on SFxT and worst, the gems didn’t help, since not many people felt that were a good addition to the game, specially because it was obvious that they wanted to have a easy money grab with the dlc gems.
For others, the game is between SF4 and MVC3 in terms of gameplay (how true is that? i don’t know), so those who don’t like one or the other are not pleased with the game.
There are also people who simply dislike the way that is played in general, who don’t like the mechanics present on the game, just look for how many complain about the chain system, or people who hate rolling after a knock down.