I read the stuff, I understand what was trying to be done (and not done). I also read though all the Raw Data. The testing procedure is not accurate enough for the results being displayed. The Chart with all the ms data is calling for more accuracy that the test can provide with. Its all bad science and bad math. The thread is not providing facts, its fueling a witch hunt. And the events of Evo right now is ether disproving your theories or prove that professional players can overcome and adapt to controller lag. I under stand the method fine, I saying your method is bull shit.
@Darksakul , “your 're so boring!”
Some people can feel lag of arcade stick ,Teyah result seem to correspond.
We just talk about it…if you think we are mads or liars or we don’t have right to talk…that you’re problem.
Which is humanly not possible to notice. The Human brain lags more than what Teyah claiming these sticks lag.
The Human reaction time is anywhere from 100ms to 300ms.
if you do claim as such, I have to say its all in your head.
How much the human brain lags is quite irrelevant to how much the sticks lag and whether that can be felt or not. It just means they won’t make or break your performance (which has been shown to be true by tournament results the world over)
The testing method is valid, Darksakul. I’m a computational scientist (PhD in engineering and postdoc in mathematics). Even though the data is binned into discrete frame increments, a distribution is fit to the data. The distribution is continuous and you can estimate the mean of this continuous distribution. At this point the conversion to ms is meaningful and straightforward since it is a fixed 60 fps game.
The question of whether or not you can feel the lag is a good one, but I for one am really glad to see these tests being done.
Human reaction to push button is one thing , human reaction to see something , understand or feel are other.
Off course , you always can adapt to your arcade stick , but more is faster , more you have time to react…one extra frame is one extra frame to tech throw , to block overhead , to anti air or any movement can lead to victory.
If you can properly feel difference between two arcade stick , one laggy…just try to compare your victory percentage on a long run.
Well besides for the impossible to verify claim that you are a scientist. There is a few uncertainties not even the test can account for.
And even if you are a scientist, it does not guarantee that you are right.
My arguments
- We know what frame a input happens but we don’t know how far in that frame in ms does an action takes place.
- As long as 2 inputs are made in the same frame of animation, it does not matter who is first being in the same frame counts as the two inputs being made simultaneously.
- We never established if player 1 lags more or less than player 2 on each system, at no time is a board tested against it’s the very same board (including hardware and firmware versions).
- the definition of 0ms used for testing varies from console to console.
- There is a possibility that due to the testing procedures that one board can effect the other as they are not electrically isolated from each other.
- As we do not know when a input is actually made in microseconds, to assign actual ms valves is not acceptable. Yes we can derive an approximation using statistical data, as two sticks are tested 50 times against each other and the results are recorded as Player 1 Hit, Player 2 hit or Tie. If the test results reflect on the actual data I would not have such a problem. But you are asking for a higher precision of test results that the testing procedure allows for.
- Not testing in multiple games, as different games treats the inputs differently deepening on the game engine.
- Animation delay is never taken in account.
- There no proof that the frame rate is a solid constant 60 fps over all tests.
Well, I could post a pic of my diploma, but I have a feeling that wouldn’t convince you either, lol.
I think you should read more about this kind of stuff because there is nothing wrong with the testing being done.
A bunch of those are unimportant or controlled for by the methodology.I know that UMvC3 has inconsistent timing, so point 9 might have some meat to it. Mostly, though, the goalpost pushing is getting pretty blatant.
Regardless, if you really think that Tehay’s results are misrepresentative, feel free to falsify them: Propose a better theory or do an experiment where his theory of consistent stick lag is shown to be incorrect. Or, if you really think that the variation in delay is insignificant in-game, maybe you can do a test to show that.
I don’t need to, the Bias displayed here is too strong for you to even accept the points I made even a remote possibility.
Any counter data I offer would also be ignored.
The lag probably won’t appreciably affect your win percentage (not in most games, anyway - something like 3rd Strike or KOF13 Ryo where you’re kind of expected to hitconfirm single hits could prove to be different, but most games’ multihit hitconfirms make that stuff just not matter that much) - how well you play is much, much more important. How smooth and enjoyable playing feels is another thing entirely. And even if your win-% would increase, I severely doubt it’d be because of having a bit less lag in the stick - it would be a result of the increased feeling of smoothness making you play in a more brave and decisive manner.
This is Tech Talk and we don’t need a game-winning reason to measure things! Arguments about whether small amounts of lag affect gameplay are irrelevant and belong in another thread. Anyone can look at tournament performance and see players doing well using all types of sticks.
Thanks for detailing your concerns, Darksakul. Here are my honest answers to them:
- This would be a valid concern for a test with a small number of trials (20 or fewer), but with a large number of trials (100+) this issue is effectively removed. Any early bias towards one end or the other of the frame spectrum would be ironed out via averaging through 1000 trials per set.
- I couldn’t find an argument in this point, unless you are stating that all inputs made on the same frame will arrive at the same time, regardless of stick/PCB - which has been proven false by myself and every other stick/PCB lag test video out there. Depending on the PCBs tested, some inputs arrive later than others, even when pressed at the same moment on the same frame - this is what I am measuring in my tests.
- Player 1 vs Player 2 lag/bias is addressed by swapping player sides and USB ports at the mid way point.
- “0 ms” isn’t used in my tests (it was earlier, but I corrected this when Undamned pointed out that this test doesn’t measure any absolute values and I shouldn’t report them as such). “+0 ms” is used instead, which reports lag which is benchmarked against the fastest stick on each console.
- This is a relevant concern, however I’m fairly confident this is a non-issue - previous tests done by toodles have shown no discernible difference between isolating or not isolating the signals, and testing the sticks by hand also show no significant differences from the results under my method.
- The methodology assumes that the game polls for inputs multiple times over one frame, so ms values would be relevant to use. If people do not agree with this, they can instead go to the Raw Data page and view the results as %0F delay, %1F delay, %2F delay, though this would not change the rankings in any way.
- Correct, I have only tested in Guilty Gear XX AC (PS3, 360) and Guilty Gear Xrd (PS4). You are welcome to perform tests in other games if you would like!
- I’m not sure what you mean by “animation delay” - both moves used (throw attempts) are identical so there would be no additional delay on one or the other side.
- Under the conditions used (same character, plain stage, no interactions or special effects ongoing), the game remained in its natural 60 FPS state at substantially all times. Any bias to the results due to frame drops would be incredibly minor and would be ironed out via averaging, as 1000 trials were taken per data set.
Several of these points are explained on the overview page, I would greatly appreciate if everyone would please read and interpret this before posting:
http://teyah.net/sticklag/overview.html
Teyah, could you test the HRAP.V4 Silent and HRAP.4 Kai? Also, I would appreciate if you could indicate which sticks had the same PCB. For example, does the HRAP.VX-SA Kai have the same PCB as the HRAP.VX-SA?
Ah sorry I had meant to update the thread with the VX-SA / Kai issue, but have been really busy with work and travel for the last couple of weeks. Evo was a ton of fun as always, shoutouts to everyone who recognized me for these tests, lol.
Here’s the VX-SA Kai, from another thread here on srk:
And here’s the underside of my VX-SA PCB:
Looks similar, but has a different version number so we can’t be sure without testing. From my experience testing slight variations of PCBs, I would say it’s a safe bet that it has delay that is similar to the original but again, can’t be 100% certain without testing.
For the other Horis, I will likely test those and others when I can get ahold of them. As I mentioned I’m done buying sticks for now (I’m very happy with my current stick: dual modded PS4 Xrd pad PS360+ with Astro P2 layout) so I’m relying on others to provide sticks for me to test.
All right. Thanks, man.
It’s a dangerous question to ask because everyone’s frame of reference is so different. All LCDs lag, and all modern games are played on LCDs. In fact most LCDs lag more than the majority of these sticks, but I suspect no one is pushing for the tourney standard to be CRT.
Each game/community/player will have a different answer. A lot of SF3 players will tell you that all LCDs are garbage, laggy PCBs should be avoided, and that arcade is the only competitive standard that should be used. Meanwhile SF4 players play on laggy monitors, many of them on Madcatz sticks, they played on PS3 for several years, many learned online with input delay netcode, and to them a few ms of lag difference on a stick probably doesn’t sound like that big a deal.
The only answer to the question is how it works for you. If you’re happy with X stick, then good! Keep using it. This thread, it seems to me, is only a resource for people who care about this. As a player who only seriously plays SF3 on arcade and uses Online Edition on CRT as a training tool, I can say I’m glad this resource is available and it matches some of the conceptions about the differences in sticks I already had.
I like when people test things. It lets you get to the truth of a matter rather than have to guess about information. People often confuse their own execution or their own reactions on lag. When we took two superguns with undamned’s USB converters down to Vegas this year, I made sure to test them before we went to make sure additional lag wasn’t an issue. That way it’s easy to say “no, you just suck” when someone complains that they teched that throw and blames it on the setup.
I think too much time/effort has been spent in this thread arguing about whether these results matter or what they tell you. It’s a pointless discussion, because the answer is not obvious, nor is it the same for every game. And let’s be real - most people who regularly post in Tech Talk are not known or relevant players in any game. So very few people who’ve posted in this thread can give you any meaningful answer about what effect it has on high level play in any game. They’re just guessing based on a combination of observation and what they already want to believe. So I think it’s much more worthwhile to challenge the science behind the measurements, rather than whether they have real world/practical uses.
tl;dr version it matters as much as you think it matters. if you care, buy a low lag stick. if you don’t, keep your current stick and be happy. whee!
A bit off topic, but…
That’s gotta be a custom, right? I’ve been looking for a stick with a centered Sega P2 layout (so no Virtual Stick Pro), and I haven’t found a stock stick with that layout other than the Sega Dreamcast Agetec “Green Goblin.” And even then the lever placement is off because of the VMU. Spare some pics?
Yep, it’s a custom panel made by B-boy tekken / AMP Up Customs here on srk. He made a custom panel for Tokido awhile back for the Capcom Pro Tour, you might have heard of him then. Not sure if is he still doing them now, he was taking some time off last I checked.
Recent pic taken at Evo by Japanese player Kuni:
Stick is an inch higher than true Astro P2 since he wasn’t able to move it down due to the panel being too small, but the button layout is what I was really after.
He lost access to his laser sadly. It’s a real shame, I love my Fighting Edge panel he made.