When I did my tests, I simply spliced both ground (common 0) and the attack button (as common 1) then bridged them to complete the circuit. This way, all buttons are pressed and released at exactly the same moment and there is not a chance for fuckery.
My purpose was to simply test common controllers on the PS4 to see which were better, relative to one another. If I were to craft a test to obtain hard numbers for latency, I’d snoop the lines with a scope. Given the results, though, the Universal Brook board was a hair better than the Dualshock 4. Both of which though, shit-beat everything else. The Hori Fighting Commander and MKX pads were by and far the worst, FYI.
F15 is the absolute earliest the image could possibly occur, given that this is the frame after the button is pushed on. Now, I don’t believe the screen is going to update in one frame, however as I said in the initial passage, it’s not easy to work out exactly which frame it is. When I’ve compared to camera methods the results seem consistent. This also eliminates one of my bugbears of that method, which is deciding whether to count when the LED is half lit and deciding whether to count when the screen is half progressed.
I’m actually waiting for the Razer stargazer to release (was meant to be Q2 2016, has flicked into Q3 2016) to add a 60fps webcam to my setup. Hoping to make a capture setup that includes all the accepted modes of assessment in one (LEDs/60fps camera/120 fps camera) as well as the PS4 camera.
Did a lot of testing with methods involving either the share or home button on the PS4, as well as a few different methods. Unfortunately, when pressing the home button, although it takes 68 frames on average, and 68 is the mode, it can vary by as much as 66-70 frames (SD = 0.87, 1x66, 18x67, 33x68 , 20x69, 3x70). Similarly with the share button values range from 105-109. Will test again when firmware update 4.00 comes out, but for now it seems that there is too much variation to really be effective as a testing method.
You still did not answer how did you get to those actual results. Even your methodology as you yet to decide when to start counting frames.
Problem with the slow motion camera setup is that with a 60 fps camera you need your camera in sync with the display, which is nearly impossible to do.
Even with a 120 fps or faster, you have to factor in your display and camera is not in sync.
Also you still need to factor in the latency of the display’s input/output.
Also tried a somewhat different approach, which I think holds some promise for PS3 input lag testing, possibly not so much the PS4. This method will measure the joystick and game lag, but not monitor lag. May well be able to be applied to older consoles as well. I remember reading a similar method on a Japanese site a few years ago, where the video signal was dimmed with a button press, but can’t find the original post anymore.
Basically what this involves in putting an optocoupler in between the joystick button signal and another in between the AV signals so that on captured footage you can see when the button is pressed, either by changing from black and white to colour or by the sound coming on.
The reason why I started looking into using the share button to take screenshots was to try to avoid a lot of the issues that come with capturing off a monitor with a separate camera equipment and button presses. If the time from pushing share is consistent, you should be able to get information without any external capture equipment. However, a programmable stick of equivalent would be required, as the way the PS4 captures is it will take a screenshot after release of the button, not on pressing it (possibly waiting to see if it is held or not).
So, doing a trial with BlazBlue (fastest responding game accross multiple different testing methodologies, I always find it disappointing that everyone is so ready to heap criticism on SFV but nobody praises BB).
Using a programmable stick, connected to a PS360+ (old version, upgraded firmware). I had an input of Share for 1 frame on what I call frame zero for reference. On frame 13 the square button is pressed for one frame. On frame 14 the triangle button is pressed for one frame. On frame 15 the circle button is pressed for one frame. The reason to do it this way is so that the screen shot should occur on frame showing the square and triangle buttons pressed, but not the circle button (trial runs to determine the timing of this were performed.)`
Total of 40 runs. 21 times a screenshot was not taken. 4 times there were typos in the input (2 of which were also not captured). 17 times the correct input was performed and a screenshot was taken.
To address the obvious initial concern - why were there only 19 screen shots in 40 trials? My suspicion is that the window for screenshots is every second frame. Reason being, across multiple trials and different devices, there always seems to be a ~50% failure rate for screenshots. I could hold the share button for two frames instead, but I’m fairly sure this would result in uncertainty as to which frame is actually captured, which I would prefer to avoid. So I have left it like this for now.
The results were, other than the typo each time the screenshot showed square,triangle. Never showed square,triangle,circle. This was what I was hoping for, and suggested that this would be a reasonable testing modality.
Unfortunately, extending the same trial to SFV did not result in similarly good results. The setup was similar except that on frame 0 share was pushed, on frame 10 square was pushed, on frame 11 triangle was pushed and on frame 12 circle was pushed.
Of 20 screenshots (with a further 19 failed screenshots) 12 times square triangle appeared on the screen, 4 times only a appeared on the screen, 2 times abc appeared on the screen and there were two typos (i.e. a fault with the programmable stick inputs).
This makes it less reliable in terms of doing a single test and getting an answer for input lag. With multiple tests, we could be fairly reassured that SFV is slower than BBCPE.
One thing that is worth mentioning, it seems that in BBCPE the command history always appears on the same frame as the animation starts, whereas I believe in SFV sometimes they are not on the same frame. For example, in SFV even though square triangle was displayed on the screen 12/20 of the times, the first frame of animation appeared only 6/20 times, with the neutral stance still present 12/20 times.
I should also mention that I tried a similar test on the UFB (even though lag of controllers should have no bearing on this test). Compared with the PS360+ (which had 2-4 typos in 40 runs) the UFB had almost as many typos as successful inputs. This is something I’ve noticed before in interfacing between the programmable setup and the UFB, but I haven’t wanted to make too much comment on it, as I don’t think my setup is exactly designed to test this.
Now, one of the main issues with this style of testing, assuming it worked well, was that it requires some way of timing the share button press (or probably the release really) to be an exact number of frame prior to the button press in order to provide meaningful information. This isn’t a scenario available to most people, so I looked to see if there was another way to apply it.
I had hoped that by holding the home or share button, and then tying that input to the input of another button, it would allow people to test for themselves. I tested both with El Gato HD 60 Pro and also with my Canon 700D (60fps 720p capture). As you have pointed out, syncing issues with using a camera are a problem, however given that the variance was up to 5 frames, I feel that the utility of this method is not really worthwhile at the moment anyway. As I said, I will check it again when firmware 4.00 comes out, as there has been some mention of revamping the home and share screens, which may improve this.
So far as my counting method goes, I count from frame 0 being when the button is depressed. The newer Hori button seem to have a lot less resistance than previous versions, so hopefully half depressed buttons are less of a problem. One of the main issues I believe is happening is that it seems that the PS4 camera captures from top to bottom each frame, if your finger speed (coming down to push the button) exceed the speed at which the camera is capturing, then you get a weird merging of two images. I’ve tended to do the counting off the El Gato captured image, in the hope of removing the issues with counting off the screen (if even the best screens are 9-10msec of lag, then there will be inconsistencies in terms of when the frame is captured).
There are several other avenues of testing that I’ve explored without much success, such as using sound as a trigger for capture and potentially a way to sync to the PS4. Using a camera remote control to take photos at set times. Programming an LCD attached to an arduino to give milisecond level timing displays. All of these have their own problems.
In terms of synchronising with the display or capture, what I’m really waiting for is for the Razer Stargazer to come out. My hope is that if we can have two different 60fps cameras on the same display, we can get a bit further in terms of working out whether the PS4 camera is in sync with the HDMI output or not. It was meant to come out in Q2, has been pushed into Q3 and with a month to go there is no new news.
Let me know if there’s anything else you want me to clarify.
I’ll do a bit of a write up on my new method (from the most recent posts) once I’ve had more time to play with it.)
Hi Noodals. I bought a used arcade stick that came with silent sanwa buttons. Coming from playing on a pad, I have experienced that the input speed on these silent buttons are quite poor. I tried to research the buttons but could not find any information where the cons are stated. However, I like the silent feature because it lets me use the stick at night without waking up the neighbours. They Hayabusa buttons you are referring to, do they also come as silent versions? Based on your quote above, it sounds like these buttons are much faster than sanwas and definitely faster than silent editions. Is this true?
When testing a button input on pad simultanously with the arcade stick that has the silent buttons, the pad input ALWAYS wins. Are the silent buttons worn out or is this a con you have to live with when choosing to use them?
Very complicated. To explain what’s going on, the arduino is designed to trigger a button press on the controller (using the PS360+ old version as it is the easiest one to attach testing the arduino to for now). This is done through opticouplers to prevent any electical interference between the two systems.
At the same time, a signal is sent to another opticoupler, that is normally blocking the path of the red limb of a component signal. When the button is pressed, the signal is allowed to travel through and a whole full colour signal appears on the screeen.
Now, this doesn’t actually occur for a full frame, instead I just press it for 1 msec. The reason for this is, I want to know exactly where within the frame the input is occuring. If you look carefully (and or download this video and review it frame by frame) you’ll see that about 10 times per second there is a coloured bar that appears on the screen, however only when it is sitting over the energy bars at the bottom of the screen will it trigger a response, three frames later. I will link to a tweet showing this in the comment section (if I can).
This should allow for very precise estimation of either the time that a game takes to respond to an input or the time it takes for a joystick input to be registered by the game.
Incidentally, I have a similar effect occurring with one of the audio outputs, except this time it pauses for a full frame. Unfortunately, I’ve been fairly disappointed in trying to analyse sound via normal video editing software (I use sony vegas). There seems to be flucuation in the video/audio sync that is up to 2 frames, which defeats the puprose of this analysis. I will continue to see if I can make sense of this.
I did some testing on some PS3 games. The way this was done was using the method described above. Counting is with zero on the frame where the coloured bar appears, next frame is one and so on until two things - 1. some marker for command history appears and 2. animation changes.
The reason I’ve looked at both of these is to try to catch any issues with hard to read animations, and also because the command history changing is the first time the game acknowledges the input as being read. An n refers to no command history available to use.
Of interest - there is no way that KOFXIII is 6 frames whereas KOFXII is 2 frames. I will go back and test this, the way I had my setup rigged was just to push square each time, and for KOFXIII this resulted in light kick, which probably has more startup frames (the first few of which are not animated). KOFXII is the fastest game that I’ve ever measured though. Subtle animation changes can be hard to judge at times.
The results of when command history appears are somewhat interesting. For example, Tekken always puts it the frame before the animation, Soul Calibur on the frame of animation starting and SF4 after starting. Interestingly, BBCT had it after the animation starts, whereas now it is on the same frame.
There are a few more games that I’ve captured, and not yet reviewed, but not too many big titles missing. Also, I have a component cable for xbox 360 coming in the post, so hopefully will be able to do some testing on that soon. Also, I’ve ordered an HDMI to component adapter, so if the lag for this is either small or consistent (or both) I may be able to check modern games as well.
Edit - added the newer tests and reorganised into (almost) alphabetical order.
Two quick things to add. Initial tests were all done on 720p capture, I’ve gone back and checked 480p with blazblue and the timing seems the same.
Also, I went back and checked KOFXIII with LP instead of LK. Same result!!! Two frames between colour change on screen and command history appearing, six frames between colour change on screen and animation starting.
Did no one notice this at the time? Anybody want to confirm this independently?
This is what I find interesting, lots of people are jumping on the 8F SFV bandwagon, without really know what the baseline is or how other games compare.
Edit - posted some new results, but have now integrated them into the main list. Will replace that with some of my thoughts.
So, from what I can see there are a few interesting results.
Average input lag seems to be 3 frames. To be precise, counting is with frame 0 on the coloured bar, frame 1 on the next frame (no animation change yet), frame 2 (no animation change) and frame 3 (animation/command history change). It is likely that you can use the height of the coloured bar on the first frame to give further information about lag, however this is not an exactly correlation to time, because there seems to be parts off the screen that still take time).
If we split games into tiers
2F (rare) - KOFXII, aquapazza
3F (mainly 2D games) - Arcana, Battle Fantasia, Blazblue, Dengeki bunko, SF3 3rd, Ultimax, Uniel
4F (most of the rest of the 2d games, some 3D games) - MK, MKvDC, SCIV, Tekken Hybrid, T5DR, T6, UMvC3, VF5
5F - SCV, SFxT, Sony Smash bros, TTTHD, VF5FT
6F - DOA5LR, KOFXIII, SSF2HDR, Tekken Revolution, TTT2.
9F - Pokken.
Tekken Hybrid to Tekken Tag Tournament 2 adds two frames of input lag.
KOFXIII we’ve already mentioned
Pokken is a special case, because people have already noticed there are frames of additional input lag added (six!). What people don’t seem to get is this is worse than SFV, because this is six frames after from the command history updating.
Also, I have another HDMI to Component adapter on the way, online reviews seem to suggest it will work with PS4/Xbox-one. If so, I should be able to apply this test to the modern games as well.
Blazblue is a good game to reference for input lag, for the reason being it is across all platforms (PC, PS3 PS4 XBox360 and XboxOne). Its response for animation is clear (I use Ragna, and pressing A he clearly lifts his fist up as the animation begins). Command history is on the same frame.
Tested using either the home button or share button on PS4 4.0 firmware. Unfortunately again there is too much variation to be reliable as a single test for input lag. Tried both pressing and holding buttons, and once again up to 5 frames of variation.
More happily though, having ordered a HDMI to component converter and been disappointed that it only did some of my HDMI signals (PS3/X360/PC) but not other (X1/PS4) I ordered another converted which arrived this week. This one seems to convert everything! I haven’t tested the X360 yet, but everything else seems to work fine.
I also checked it to see if it introduces input lag, and it seems as though it doesn’t, or at least if it does it is in the millisecond range (far less than a frame). The way I did this was by checking where the marker appears for a game I’ve already tested extensively on component cables (BBCPE) and then rechecked with HDMI cables and converter, with no major difference in results.
So, having had a busy week, tonight I got an hour or two to myself and finally got to test out some modern games. The following is what I was able to put together. Same format as previous, first number refers to the time at which a change in the command history occurs, with an n representing a game that doesn’t have a command history. The second number is when there is a change in the animation. Last Blade 2 is quite hard to pick, as it seems to already blur/?interpolate frames.