do both. i play the sf and kof series on a stick, but i prefer pad for the tekken and mk series.
I have been playing on a pad since the NES came out (1985… FUCK, I"M OLD!) and now because I always push so much psi into my dpads and practically break them in months, I now have problems with my left hand thumb maintaining strength (wait till all you get older). I have made the task of unlearning 27 years of exclusive pad usage to not only save my muscle strength in my thumb, but to be a better player cause FADC is a bitch to pull on a pad.
It really doesn’t seem like most of you read the first post at all. 
Memory is associative.
You learn something on a certain control method and you will associate that technique with that control method.
And your issue sounds like nervousness that is combined with lack of association. The association and comfort with the method is built with practice.
LOL! that shit cracked me up
This really shouldn’t ever be the case. Even if you practiced for a year on the stick, never touching the pad, and then you decide to give up on stick and go back to pad, even if you aren’t very good at pad at first, you should be able to get as good as you were and, with more practice, theoretically better.
While I am new to fighting games, I’m not new to learning things (sounds retarded, but it’s relevant). Everything is part abstraction part application. If you have the physical skills required to pull off moves on your pad, that muscle memory really shouldn’t just completely devolve into the realm of impossible progress. You seem to already have the abstraction aspect down, the next part is application, SF4 and Marvel are two different games, so it makes sense that you wouldn’t be able to fully apply all of your physical knowledge of one game to the next. In fact, you could be so comfortable with certain physical aspects of Marvel, that the difference of SF4 is completely wrecking your execution; however, with time and practice this should correct itself and you should be able to play both comfortably.
It’s hard for me to say this accurately because I am a new player and new to stick, but my suspicion is that I’ve played so many games with a pad that playing on a stick is very awkward; so I think for someone who has grown up with video games, if they have built a considerable amount of muscle memory using a stick for one game, when the mechanics of another game change, it makes sense that they wouldn’t be immediately good with a stick and noticeably better with a pad since that is the paraphernalia they grew up with; they have more experience with the pad on a multitude of platforms, and have probably experienced every kind of mechanic and positioning available on the pad; however, in your case, it seems that you only really know the mechanics and positioning on a stick for one game.
However, with that said, I think an arcade stick is infinitely better than a pad in almost every conceivable aspect for a fighting game. Not only is it aesthetically better for me, but, after practice, execution is more accurate; I feel much more in control when I can use my entire arm (wrist and fingers) instead of having to just use my thumb to input direction.
So, with all that said, I think you should stick it out with stick 