How does the Output buffer usage in Cinegy Capture influence the ‘Session Closing’ time?
Posted by Oleh Muliarchuk on 18 April 2018 09:40
Cinegy always aspires to make all products reliable and as much as possible resistant to the short- and mid-term hardware/network failures.
In Cinegy Capture we have introduced the file write buffer (“Output buffer”), which helps to keep real-time capturing even if the write speed is not enough for writing all the files with the proper speed:
If the file storage has the short-term issue and can’t push the frames on the disk in real-time, the frames are being collected in the buffer(s) and the Capture session does not fail immediately, but continues capturing, until there is free space in the buffers.
If the writing speed is slower than the real-time, filling of the output buffers may increase during the ingest, as they can’t write all the data in time. Then there are two scenarios possible:
1. If the disk speed is close to the real-time and buffers are being filled rather slowly, the session is stopped by the operator before the buffers are completely full. The ingest session will stop correctly in this case but Capture Control will indicate the ‘Session closing’ status and continue writing the data from output buffers to the disk, to avoid any data loss. How long it is going to take depends on the degree of output buffers filling. So, if the disks are slow, and there is a lot of data collected in the buffers during ingest, it may take the reasonable time until all the data are pushed to the disk – this is exactly the ‘Session closing” time. The default buffer size is 2 Gb per each output, so it may be up to 2Gb data to push on the disk in the worst case. But the big advantage is that you won’t lose any data, everything will be saved on the disk.
2. If the disk is clearly slower than the real-time, the output buffer(s) will overflow quite quickly. In this case the ingest session fails with indication in the log file ‘File write buffer overflow’. The Capture will stop pushing the data on the disk, the ‘Session closing’ status will not last long but the session will fail, as mentioned:
Let’s see it on the example.
If just one single channel with 5 outputs is run, the disk works near to the real-time (or even faster), no fails happen and Ingest closing phase is not too long, because the buffers are not filled heavily.
In case of running two channels, the disk system is unable to write all files with the required speed, and one (or more) buffers are filling fast, which leads to the ingest stop with the ‘Buffer overflow’ indication.
Therefore, it’s very important to benchmark the network/disk environment in the ‘concurrent access’ mode. Try to emulate several streams from the same machine, writing to the target storage at the same time, to see whether all of them exhibit enough speed to push uncompressed bit-rate in the real-time.