When the Sony FX9 arrived at CineD headquarters earlier this year, we really wanted to walk it through our lab testing procedures. However, when we checked the first results of the laboratory test, we contacted selected industry experts. Read on why …
You may have wondered why we haven't released a lab test for the new Sony FX9 yet, even though my colleague Nino bought one back in January 2020. Well, we did the lab test back in January – but the results were created with the internal codec, we wonder what's going on here.
Was the firmware / image not final yet? Would there be an update soon? So we reached out to Sony and a select few industry experts and shared all of our results and measurements with them. The first answers and subsequent discussions did not lead to further findings. Sony stated that there are well-established methods of measuring sensor performance. If CineD does its own lab tests, that's fine.
Beyond that, there was nothing to suggest that new firmware or any kind of image enhancement would be available anytime soon – and we learned back in April that the 2.0 FW would not be released until later this year in October, after it was originally for “Summer 2020 ”was announced. The camera was released last year.
(Sony recently announced another 3.0 firmware update with no release date, with a few more additional features: S700PTP remote protocol, center scan mode for Super 16mm lenses, and B4 lens support when using the adapter).
After a careful review (and again …) we decided to release what we found earlier this year.
Rolling shutter of the Sony FX9 – full screen UHD 16: 9
With our flashlight at 300 Hz, we get the typical sequence of black and white bars (due to reading from CMOS sensors). Counting the bars gives a result of 22.2 ms For information on the Sony FX9, please refer to the following illustration.
This value is slightly worse than the measured values that we received for the SIGMA fp (20.8 ms), for example, but is in the middle field for a full-frame sensor that reads a 6K image. For example, the Panasonic S1H has 24.2 ms in 4K DCI 17: 9 full screen mode (7% less image height than 16: 9 UHD mode).
However, if you compare this with the new Canon C500 Mark II, which shows a value of 15.8 ms (in DCI 17: 9 mode), the FX9 falls significantly behind – and now the new Canon R5 even showed 15.5 ms in 17: 9 8K full screen display – the best result we have ever measured for a full screen sensor in 17: 9!
Now we know why the FX9 cannot perform 60 fps when reading the UHD full sensor – there is simply not enough time to read the entire sensor. By the way, this explains why there will be 5K cropping modes for 60 fps in the upcoming new firmware (recording in 4K).
Rolling shutter measurement of the Sony FX9 in UHD full screen mode at 25 fps.
Dynamic range of the Sony FX9 in XAVC UHD ISO800
If you don't know how we test dynamic range please have a look here.
The Sony FX9 has a double ISO sensor with 800 and 4000 as two native ISO values. So let's test ISO800 first – here is the waveform diagram of the XYLA21 step diagram – it shows 12 stops. A 13th stop and even a 14th stop are still visible, but within the background noise:
Waveform diagram of the Xyla21 step diagram at ISO800 for the internal XAVC all-I SLOG3.SGamut from Sony FX9 UHD – 12 f-stops can be identified above the background noise.
Running the files through IMATEST will confirm this result:
IMATEST dynamic range analysis of the Sony FX9 at ISO800 using the internal All-I-XAVC codec at UHD 25fps with SLOG3.SGamut. 11.5 Stops are calculated using a signal-to-noise ratio of 2 – the mean in the upper right table.
This dynamic range results from 11.5 stops At our threshold of a signal-to-noise ratio of 2 (SNR = 2) in UHD (3840 × 2160, internal noise reduction “off”) for a full-frame sensor that samples down from 6K, this is okay, but nothing special.
As you can see from previous measurements, it is about a notch better than the full-frame mirrorless cameras from Sony a7S II or a7 III in UHD mode, but less than the full-frame cameras Panasonic S1 and Panasonic S1H 12.7 stops in 6K mode (see our article here). In addition, according to IMATEST, the Canon C500 Mark II comes with 13.1 stops (internal XF-AVC, full screen) (see article here). Note that we haven't yet received a production model version of the Sony a7S III to run our lab tests with (but you can view and read our real-world review here).
What should be considered: Yes, while our noise threshold (at SNR = 2, medium value (pink line)) already starts at 11.5 stops, additional stops are clearly visible. Look at the left side of the middle graph (RMS noise) in the IMATEST result – see the figure below.
Sony FX9 IMATEST result at ISO800 – some additional stops are visible beyond the SNR = 1 value.
Since the codec being used can properly encode these extra (noisy) stops, you can work with those in post production using noise reduction, etc. If you fast forward to our latitude test (below) you will find that the internal codec is incapable of doing this properly. However, future firmware RAW options should definitely take this into account.
Incidentally, setting the internal noise reduction to "high" leads to 11.9 stops for the Sony FX9 at ISO800.
Now let's look at the second native ISO 4000.
Dynamic range Sony FX9 at ISO4000
At ISO 4000 we get exactly the same result as the lower native ISO 800 – this is the first camera we tested that lives up to the promise that the higher native ISO settings won't affect the image. For example, the Panasonic S1H shows a slight drop in dynamic range (0.4 stops) when switching to the higher native ISO setting, as does the BMPCC6K.
The waveform diagram of the step diagram for ISO 4000 confirms this – it looks strikingly similar to the waveform diagram for ISO 800:
Waveform diagram of the Xyla21 step diagram at ISO4000 for the internal XAVC all-I SLOG3.SGamut of the Sony FX9 UHD – again 12 f-stops can be identified above the background noise.
The IMATEST analysis also shows this result at ISO 4000, 11.5 stops are calculated for a signal-to-noise ratio of 2 (mean value in the upper right table):
IMATEST dynamic range analysis of the Sony FX9 at ISO4000 using the internal All-I-XAVC codec at UHD 25fps with SLOG3.SGamut. 11.5 Stops are calculated.
Latitude (underexposure test) of the Sony FX9
Of course we also carried out our standard underexposure test for the FX9: With S-Gamut3 / Slog3, noise reduction “off”, we have Nino's face at max. 60% luma value on the waveform monitor, the white sheet of paper is around 65%.
This results in a basic exposure of 360 ° shutter at f4.0 at ISO 800 (UHD 25fps) – our 0 stops below the baseline. From there we underexpose the studio scene one by one by reducing the shutter angle to 180 °, 90 °, 45 °, 22.5 ° and finally to 11.25 ° (5 stops underexposure).
In post-production (with DaVinci Resolve 16.1 Studio), the underexposed images are reset to the exposure value 0.
This test shows a lot about how well a camera (and the codec it uses) retains colors and details. Most cameras break apart at 3 to 4 f-stops. The most recent exception so far: the Canon C300 Mark III, which maintains a usable image at 5 f-stops, stops underexposure.
For example, the Canon C500 Mark II mentioned above retains a usable image at 5 stops with underexposure (but has faint lines in the image and uses extra noise reduction in the post).
Let's see how far we can take the Sony FX9 …
Sony FX9 with basic exposure (0 f-stops below) with XAVC UHD 25fps ISO 800 S-Gamut3 / Slog3.
The above picture shows the basic exposure. Now let's jump forward and take a look at 3 underexposure stops that have been pushed back to zero:
Sony FX9 3 stops underexposed and is reset to 0 with XAVC All-I UHD 25fps ISO 800 S-Gamut3 / Slog3
Unfortunately, even with three underexposure stops pushed back to zero, we can see very blotchy noise spots, especially in darker areas in the lower right or on the dark side of Nino's face.
Here is a noise suppression test using DaVinci Resolve 16.1 with 3 frames of temporal noise at 5 thresholds and 5 spatial:
Sony FX9 3 stops underexposed and is reset to 0 with XAVC All-I UHD 25fps ISO 800 S-Gamut3 / Slog3. 3 frames temporal noise suppression threshold5, spatial 5 in Resolve have been added.
Okay, with noise reduction, the underexposed image can still be used with 3 f-stops. If you now move on to the 4 stops, the image will be underexposed:
Sony FX9 at 4 stops below, with XAVC all-I ISO 800 UHD 25fps S-Gamut3 / Slog3 pushed back – the point of no return has been reached.
As can be seen, the noise distorts the image to the point where it cannot be fully restored. I tried again in DaVinci Resolve to reduce the noise even more. See the resulting picture below:
Sony FX9 at 4 stops while pushing back with All-I XAVC UHD ISO 800 25fps S-Gamut3 / Slog3 – with noise suppression in Resolve (3 frames in time, threshold value 10 and 10 in space). For example, the blocky paths of chroma noise in the lower right corner cannot be removed.
The type of noise shown in the underexposed 4-stop image is not finely divided, but rather represents spotty patches of chroma noise – these cannot be easily removed without further sacrifice to the image in the mail.
For reference only, here the picture is underexposed at 5 f-stops, pushed back to zero:
Sony FX9 at 5 stops below, with all-I XAVC UHD ISO 800 25fps S-Gamut3 / Slog3 pushed back to 0
It's hard to say why the picture falls apart pretty early even when compared to other cameras we've tested so far.
A possible reason for the poor performance could be the internal codec and the ability to capture noise. Although the highest quality settings were used (and I reconfirmed them in the post with the Sony catalyst software, see the image below), the All-I codec is only 250 Mbps – for an All-I codec in UHD this is a rather low bit rate and can add to the big, nasty patches of chroma noise that we see.
Internal codec for the tests re-confirmed in the post by Sony's Catalyst software.
The results for rolling shutter and dynamic range for a full-frame sensor are more in the middle to lower end of the spectrum – to the extent that we directed Sony and selected industry experts. These discussions went on for a while but didn't yield any further insight – it is what it seems.
Additionally, the latitude test shown above showed that you could use no less than 3 underexposure stops and slide the image back. Otherwise, ugly noise will damage the image to an extent that cannot be recovered – using the internal XAVC codec.
Of course, we are excited to see if future firmware upgrades might improve image quality. The next one is coming pretty soon (in October) as mentioned above, so we'll see – and this should also allow for 16-bit RAW output via the optional XDCA-FX9 unit (though we'll need to see if at this point There will also be a recorder that is powerful enough to record 16-bit RAW. Also, don't forget that Sony has already announced that it will be releasing a smaller sibling for the FX9, the Sony FX6, by the end of 2020. However, it did no technical details given yet.
Already using the Sony FX9? What is your experience with the camera? Please let us know in the comments below.