3rd Strike Lag/Speed Tests

Classic Fighting Games Street Fighter III: 3rd Strike
1

3S Speed Test Results

3S Lag Results - Spreadsheet
CPS3 - 3.6
PS2 - 4.7
PS3 (filters off) - 3.8
PS3 (filters on) - 3.8
360 (filters off) - 4.1
360 (training mode) - 3.9
Dreamcast - 6.

Emulated 3S Lag Results (Excluding Monitor Lag)
FBA (aero on) - 6.0
FBA (aero off) - 4.6
Shmupmame (aero on) - 5.4
Shmupmame (aero off) - 3.8
Shmupmame (fullscreen aero off) - 3.7

Emulated 3S Lag Results (Including Monitor Lag)
FBA (aero on) - 6.5
FBA (aero off) - 5.1
Shmupmame (aero on) - 6.0
Shmupmame (aero off) - 4.4
Shmupmame (fullscreen aero off) - 4.3
*All these averages should be +/- 0.5 frames of the actual lag

Platforms Tested - CPS3, PS2, PS3 (PS3 with filters on and off), 360 (and 360 in training mode), Dreamcast, FBA, and Shmupmame

Here’s the setup we used:
I wired the hk button and an led to a DPST switch so that the led lights up as soon as hk is pressed.
Video was recorded at 120fps.
We recorded Chun whiffing far hk (since her far hk has an obvious first frame of animation) 50 times for each system.

An MC Cthulhu dual modded with a hori pcb was used for PS2 and PS3.
A non-dual modded hori pcb was used for the 360 and for the emulators
A stock dreamcast controller was used for the dreamcast.

A CRT was used for the consoles and CPS3
We made sure the crt’s “screen tear” was above chun’s head on the console ports.

An LCD monitor (Dell U2312HM) was used for the emulators

I’ll refer to frames output by whatever system is being tested as a “gameplay frame,” and a frame captured by the camera as a “camera frame” since the frame rates are different.

I counted the first camera frame where the led is lit as frame 1, and I counted the camera frame where the move visibly starts up as the last frame. I then divided this number by 2 to get the lag for 60fps with half a frame of precision.

Something I noticed is that CPS3 was not displaying frames in sync with the camera, whereas every other console is in sync with the camera for the entire video. This is because the consoles refresh at 60Hz and CPS3 refreshes at 59.583 Hz, which means that each frame on CPS3 is displayed slightly longer on CPS3 than on console. This isn’t really new information, but it’s for whoever didn’t already know. The difference in length of each frame from CPS3 to console is only about 110μs. There’s some images below that help illustrate this.

Here’s two halfs of a frame from CPS3.

Spoiler

http://i.imgur.com/h3FvfMJ.png

Here’s two halfs of a frame from CPS3 somewhere around 1 second later.

Spoiler

http://i.imgur.com/2Le6zT6.png

Here’s two halfs of a frame from the PS3.

Spoiler

http://i.imgur.com/aFv1q72.png

Here’s two halfs of a frame from PS3 somewhere around 1 second later.

Spoiler

http://i.imgur.com/jvk84g4.png

1 Like
2

Big thanks to ESN for his help on this.

Basically the tl;dr version of this is that OE lags the same as CPS3, and PS2 lags one frame more.

We’ll also run the same test for OE online to make sure the netcode doesn’t add any hidden input delay. First with both players at ggpo delay = 0, the second with one player at delay 0 and the other at max delay and see if there’s any difference from offline in either case.

3

Also of anyone else has a high speed camera and can run the ps2 test to check our results, that would be cool. Wanna make sure it’s actually ps2 version lagging a frame and that the Cthulhu has nothing to do with it.

4

I was going to ask to try and use a ps2
controller.

5

might be able to. gotta see what the PS2 pcb looks like. I’ll post once we know. yeah I agree that’s probably better so we can be sure.

6

I have a feeling the mc cthulhu doesn’t lag on ps2 because of the lack of variation in the ps2 test. Every trial was 5 frames except for 1, which lagged 4 frames. This could just be a coincidence, or perhaps ps2/anniversary collection handles input windows differently than other consoles. I would think that if the cthulhu added lag, ps2 would occasionally lag 6 frames (occurring whenever the button is pressed toward the end of a frame). Either that or the cthulhu lags 16.7ms i.e. 1 frame. I still want to test a ps2 controller to make sure though.

7

Great tests, but I’m wondering about a few things; 1) Could you test OE with an 100% positive lag free HD monitor/tv to see does that make a differance? I’m thinking about some games actually having MORE lag when they’re played on a CRT, can’t remember any examples now but it’s real. 2) OE lags less with filters ON…? Did you test all the filters + scanlines and combos of those?

The odd frames you’re seeing from the CPS3 are because it’s refresh rate is different, something like 59.583. No Capcom Play System outputs a solid 60hz, they’re all like that and that’s why I think all the emulated versions feels off compared to arcade due to emulated always being solid 60hz, or whatever your screen hz is.

8

IMO there is no 100% lag free hd monitor.

9

Yeah it is a bit of an oxymoron, but I meant 0 or next to none input lag.

10

I was really surprised to see that there was MORE lag with the filters off!

11

I’ve been saying for YEARS that the input on ps2 version felt off, I’m glad there’s proof now

12

Very nice!
Funny, it lags more with filter off, but then it’s closer to arcade lol!

I may be wrong (not really good at video editing stuff), but the half frame thing should not matter at all if your image is not interlaced. (how did you record videos? 720p? 720i?)
Image is entirely drawn on each frame, so you’ll never see half of current frame and half of prev one. It only darkens a part of the image, possibly making it unusable but that’s it.

13

since the testing system has a confidence interval of 1/2 frame, the little differences between filter settings are statistical, just due to the test having a finite number of trials, since an 1/2 frame precision tool can’t measure directly an order of magnitude of 1/6th frame (which is more than the discrepance between the cps3 result and the OE filters on result).
Increasing the repetitions (or capturing at a higher framerate) would pull the results even closer in the bell curve.
Bottom line, filters on or of doesn’t matter.
And probably OE is the exact same as cps3 version; if not, it’s still far closer than ps2.

14

It’s easy to tell the input delay from daipan loop. Completely different timing.

15

They exist, but they’re hard to find and expensive. Iiyama Prolite 2210ES (i forget the letters) have been tested to have 0 input delay. Buddy of mine had to import it from Japan and thoroughly tested it for lag, since that’s a pretty nitpicky point for the ST scene.

They’ve been the standards for certain European tournaments for years, and they’re fantastic.

16

@isotopez
@Lance3rd

Have you looked into Groovymame yet? A good/great setup for it requires: XP, crt_emu drivers (so an older HD Radeon graphics card to maximize the emulation), preferably 120hz capable VGA monitor or a 15khz arcade monitor (one for lessened visual lag, the other for the more accurate picture.) a consistent soundcard (onboard works too) and of course the proper settings… XP polling rate change, Groovymame config etc. (newest release accepts audio latency in the decimals… frame_delay decreases emulation lag if your system can handle it…)

I’ve only been able to try out the Linux release, so I’m unfortunately stuck with added input delay compared to XP, simply because of the way Linux handles inputs. I’m going to try XP out on a different HDD though, hopefully soon… Luckily, my onboard graphics is fully compatible with the crt_emu drivers = )

Hope you guys can get that tested, as I said before, the game just “felt” better to me on it, even though it had slightly more input delay because of Linux…

17

I think we can do that. Actually we might hold off on all the emu tests until the cab gets here, then if I understand correctly we can convert the output from my PC to work on one of the cab monitors. That’s possible right? That way every platform we test will be on CRT.

18

I believe the conversion is possible, I don’t know much about it though as I only have access to a 25khz-97khz VGA monitor. From my understanding, using something like a VGA-SCART should allow that, but I don’t know anything about 15khz monitors I’m afraid…

Also, I guess there’s a way to bypass any USB delays by using a JPAC and going through a COM/Printer port or something? Can’t remember the details, but it sounded very interesting. You should be able to find everything you need on their forums [details=Spoiler]http://forum.arcadecontrols.com/index.php/board,52.0.html[/details]

They can certainly point you in the correct direction if you can’t find what you’re looking for… I don’t have enough knowledge on the subject to give any sort of guarantees
= /

19

Can you guys explain this in laymans terms?

so ps3 and cps3 on a CRT is the same?

I could have swore I heard somewhere that arcade 3s doesn’t run at 60fps or 60hz or something.

I don’t know how different the arcade cabinet monitor is, could this be a factor in the difference between console versions and arcade?

Edit:

not sure where you got this info but I knew I’ve heard it somewhere

20

I just realized none of the images I posted really illustrated what I was describing with the “half frame” thing. Here’s a better image of the startup of chun’s hk.

Spoiler

http://i.imgur.com/dNY3TAv.png

The bottom part is darker, but you can still see the previous image before hk starts up. If chun is cut in half, then the half frame thing doesn’t matter, but if only the top part of the screen is lit up (where nothing really changes other than the timer), then I get no information about chun’s state.

Nicely said. To those thinking that there’s more lag with filters off, it’s not statistically true. A much bigger sample size would probably show that they’re either the same, or even perhaps that filters add a little lag. That would require a higher speed camera (as hol horse said) and a larger sample size. Larger sample sizes would make these tests way too tedious though.

I was fairly certain I saw somewhere that cps3 didn’t run at exactly 60fps, but I couldn’t find the actual number anywhere, so I didn’t want to make any assumptions.

Edit: I remembered where I saw the frame rate for cps3. Mame tells you the frame rate, which is 59.583, when you look at “Game Information” inside the in-game menu.

As hol horse pointed out, there isn’t a statistical difference between the lags of cps3 and ps3 (filters on and off). For our test we turned all the filters on. I find it doubtful that any games lag more on a crt unless there’s some kind of image scaling or converting going on, but I could be wrong. Either way, 3s was made for crts.

Cps3 and ps3 do seem to run at different frame rates as yomipower pointed out and as the tests indicate. Since the camera only has 1/2 a frame of precision and the frame rates are so close together, the difference in speed shouldn’t really be noticeable in a lag test like this. The difference in speed may or may not be noticeable in the other tests we plan on doing (holding up and measuring the length of a long animation). Since a frame on cps3 lasts for 16.78ms, and a frame on console lasts for 16.67ms, it’s possible that cps3 does actually lag more than console (by 110μs), but this also depends on how each system implements its input windows i.e. does the input window stay open for the entire frame or just a portion of the frame.