This is an interesting discussion that I hadn’t really considered previously. I was more concerned with people knowing that the reported response times were relative (to the control).
I think the argument that Nobus3r1 ver. 2.0 makes is sound. No matter how much testing you do, you are still bound to results in quantity of Frames, not milliseconds. The value in numerous samples is the more that you test, the more accurate the percentage becomes (i.e. “this stick has 1F lag 90.87% of the time”). You can maintain your same tier structure, it’s a matter of representing the data in a more direct way (one that doesn’t interpolate data we don’t actually have). We definitely have frame data. We cannot know millisecond data without further diagnostic tools (USB packet sniffing or something like my USB Decoder hardware setup I shared previously).
-ud
The experiment is repeatable, and the ‘fixed stick lag’ theory makes testable predictions - you’re welcome to propose an alternative theory, and run an experiment to test the theories against each other, or to run the experiment yourself and see if you get a different result.
I’m not sure what, precisely, this is supposed to mean, but it’s almost certainly incorrect. People will notice events that take less than 1/30 of a second to occur (like a single swipe of a CRT), and can notice timing differences of less than 1/30 of a second at least some of the time.
The limit here is the minimum time interval between the system sampling stick A and stick B, and not the time interval between reading stick A and stick A again, and that’s much shorter than a frame.
Moreover, it’s perfectly sensible to talk about lag in ‘seconds (equivalent)’ if the behavior of sticks matches up nicely to time deltas.
Hey guys, this is an interesting discussion. I’ve been doing some statistics lately in my day job, so I figured I can comment on these experiments. I took a look at your excel spreadsheet, Teyah, and for the most part I think what you’ve done is not bad.
A couple of points I wanted to make:
The data is collected in terms of frames, but we can actually represent the data in terms of “relative” lag in milliseconds correctly. The fact that the data is discrete (-1, 0, 1, 2 frames of lag), doesn’t invalidate the data, but rather it is really just creating bins for the data to fall in. -1 frames of lag would be when the test pcb reacts faster than the control pcb, which happens very seldomly. So, the bins are not a problem. It would a more accurate test if we could measure more precisely, but that is all of the data we have to work with.
After the data is collected, the frame data is fitted to a normal distribution, which is not a terrible choice. In fact, it’s probably a good choice if one were to assume a distribution. The alternative would be to estimate the parameters of a distribution (using maximum likelihood or a regression analysis). But if we consider the normal distribution to be a good fit to the data, then Teyah did his calculations correctly (I took a look at the calculations in the excel file).
Since the data is binned, that doesn’t invalidate the unit conversion from frames to milliseconds. What we’re actually doing is fitting a gaussian distribution to the binned data and calculating the mean of the distribution, so it’s the “mean relative lag” of the pcb in milliseconds. That all holds up mathematically. The only assumption being made is about the assumed distribution.
Keep up the good work, this is really useful stuff!
Not yet but I own a 4 kai as well and last night I tested it against my cerberus and Kai lost every single punch in a Iori mirror match in kof xiii, i mean evey single time. I dont have figures and numbers but I can definitely say hori is pretty damn slow.
I think all of the new Hori sticks have an issue with input lag.
I have the Hrap4 Kai and Madcatz TE2. I use both equally. I’ve always noticed that the Hrap4 Kai felt slightly laggier than the TE2. I tried a similar test, like the Youtube video above. And the TE2 would win roughly 6-4 to 6.5-3.5 times vs the Hrap4 Kai.
Low and behold, I take a look at this thread today and see the same things I’ve been suspecting and have tested.
Teyah,
Fisrt of all, congratulations… That was a great job!
I’d like to ask you again (like the others before me) to test Razer Atrox 360 and if possible, ONE.
I have an Atrox 360 + Cerberus in dual mode.
So… I’m… huh… we are expeting it…
Thanks bro!
See ya!
Major site update today. There’s been a lot of people asking me for results for the Fighting Commander 4, and with huge thanks to @ MultiTrackDrift (twitter, not srk) sending one over from Japan, I was able to get results on it for PS4 and PS3.
Here’s the full site update from my main page since I’m too exhausted to retype.
07/11/2014: Major Update (PS4, PS3)
Updated results and raw data to include results for several new PS4 and PS3 sticks:
Updated data on Hori Real Arcade Pro v4 on PS3 (PS3 mode) to 1000 trials
Added data for Hori Real Arcade Pro v4 (PS3) for PS4 mode
Added data for Mad Catz TE2 (PS3) for both PS3 and PS4 modes
Added data for Guilty Gear Xrd Custom Fighter’s Pad (PS3) for both PS3 and PS4 modes
Added data for Guilty Gear Xrd Custom Fighter’s Pad (PS4)
Added data for Hori Fighting Commander 4 (PS3) for both PS3 and PS4 modes
Added data for Hori Fighting Commander 4 (PS4)
Added data for Hori RAP4 Premium VLX Kuro (PS3) for both PS3 and PS4 modes
Added data for Hori RAP4 Premium VLX Kuro (PS4)
Added data for Hori RAP V Silent (PS3) for both PS3 and PS4 modes
Added data for Hori RAP V Silent (PS4)
Clarified difference between Qanba Q4RAF Black old and new models on Results page
Also added average delay for the new line of Qanba Q4RAF Black & Eightarc Fusion sticks
I haven’t tested two identical stick models against each other. This is a good idea - I’ll add in it in on my next update if possible.
All of my tests do alternate between 1P/2P sides and USB port 1/port 2 at the halfway point in each test, so any potential bias towards a particular player side or USB port should be removed. (This being said, I haven’t yet seen any indications that a particular side or port is favoured)
I would also like to test the Atrox for 360, but it’s been difficult getting ahold of one of them. I’m no longer buying arcade sticks (for now anyway) and nobody in my area has one unfortunately.
Oh, I should also mention that you should always try to play in PS4 mode when using a PS4 stick on PS3. The PS4 mode has a faster polling rate and results in about 6-7 ms faster response time. For all the effort of flipping a switch, I think it’s worth it.
From my experience PS4 sticks in PS4 mode on a PS3 seems to have problems, sometimes buttons like the PS or even the stick stops working out of nowhere. Tested with V Hayabusa Kai, VLX and TE2.
Please keep in mind that PS4 mode on the Mad Catz and Hori sticks are never intended for the PS3.
I sure if you complain to Hori and to Mad Catz their response would be switch the stick to the appropriate mode.
How much does any of this really matter? Let’s all pretend for a second that we’re not the best players in the world. How much would most people even notice this? And are top players who use some of these sticks rated “C” or worse really having any issues?
I’m not sure since this is the first I’ve heard of such a thing, but I can report on the results from my tests. Of the 6000 trials performed on PS3, I did not see one case of a dropped input on any of the following sticks/pads in PS4 mode:
GGXrd Custom Fighter’s Pad
Hori Fighting Commander 4
Hori Real Arcade Pro v4
Hori Real Arcade Pro V Silent
Hori VLX Kuro
Mad Catz TE2
Input display was on and both sides remained in sync for the entire duration of the tests, other than one stick being behind the other by 1 frame some portion of the time.
The Sony Dual Shock 4 did have some input drops when there was interference from other Bluetooth devices nearby or when there were obstructions between the PCB and console, but that does not relate to the PCB being in “PS4” mode while connected to the PS3.
If people are claiming there are dropped inputs then they would do well to perform tests, detail their methodology and post their results (eg. X # of drops per Y # of inputs)
