First off, you need to know the following: VGA and Component are different signals. Most “converter cables” aren’t actually converters – they’re adapters for devices that can specifically either output or input component (YPbPr) signals over a VGA D-Sub connector. This is not the case (to my knowledge, someone correct me if I’m wrong) with the XBox 360; it’s usually used for projectors and other specialty things… it’s an off-spec use of the connector that’s not even really standardized to my knowledge.
Secondly, “Component Quality” is actually not limited to EDTV (480p) – “HD” resolutions like 720p/1080i, and even 1080p in some cases, can be pushed through component cables.
You may be thinking “composite” and “component” are the same thing – they’re not. Component video is over three RCA-connector terminated wires (Red, Green, and Blue), while composite is a single Yellow RCA connector. Both of these will usually have an extra pair of RCA connectors for analog stereo audio (white and red). Most cheap USB capture cards are composite, but this is very low resolution (painful to play on, at this point).
Also, you should know, capturing at a different resolution than what you’re playing at is… well, it’s a pain. There are a few devices that do that by down-scaling on the device itself, but at that point you may as well just capture at the original resolution. Unless there’s a really good reason you want to record at a lower resolution, I’d advise against trying to work like this.
So now for actual advice:
First of all, the easiest answer would be to stop playing via VGA. If your TV has a component video input, use that. This would let you opt for a Hauppauge HD-PVR, or any other component capture device (AverTV just released a standalone device that’s supposedly pretty good, but I can’t find it on Amazon :|), and would simplify your set-up immensely. Instead of splitting the signal, you can just use the component pass-through to your display… makes everything a lot easier to set-up and understand. Then you’d be capturing and playing at 720p, and the intensive part of the capture process (the encoding) is happening in a separate box.
If component for your TV isn’t an option directly, you can still make it an option – get a Component -> VGA converter box. Then you can do Game console -> Component capture box -> (passthrough) -> VGA adapter -> Monitor. Still a relatively simple video path, and you get to play and capture at 720p. There’s plenty of info in this thread and this one over here for everything I’ve just described.
Hope this is helpful, good luck!