Bug: Unhandled exception - Code 3765269347

[pdeloos]

Observed Behavior

Tried to play an mp3 stream and CasparCG gets into a fatal exception and quits

Expected behaviour

The stream would start or, if it cannot, at least keeps running.

Logging:
[2024-06-21 14:57:51.401] [info] Sent more than 512 bytes to 127.0.0.1
[2024-06-21 14:57:52.416] [info] async_event_server[:5250] Client 127.0.0.1 disconnected (0 connections).
[2024-06-21 14:57:54.908] [error] [ffmpeg] [tls @ 0000020114D10700] Error in the pull function.
[2024-06-21 14:57:54.908] [error]
[2024-06-21 14:57:54.908] [error] [ffmpeg] [tls @ 0000020114D10700] IO error: Error number -138 occurred
[2024-06-21 14:57:54.908] [error]
[2024-06-21 14:57:54.908] [warning] [ffmpeg] [https @ 0000020114A28940] Will reconnect at 13130914 in 0 second(s), error=Error number -138 occurred.
[2024-06-21 14:57:54.908] [warning]
[2024-06-21 14:57:55.064] [warning] ffmpeg[https://go_fm.streampartner.nl/live|547.0200/0.0000] Waiting for audio frame...
[2024-06-21 14:57:59.977] [error] [ffmpeg] [tcp @ 0000020114D10F40] Connection to tcp://go_fm.streampartner.nl:443 failed: Error number -138 occurred
[2024-06-21 14:57:59.977] [error]
[2024-06-21 14:57:59.977] [error] [ffmpeg] [tls @ 0000020114D10700] The specified session has been invalidated for some reason.
[2024-06-21 14:57:59.977] [error]
[2024-06-21 14:57:59.977] [warning] [ffmpeg] [https @ 0000020114A28940] Will reconnect at 13130914 in 1 second(s), error=I/O error.
[2024-06-21 14:57:59.977] [warning]
[2024-06-21 14:58:01.366] [info] async_event_server[:5250] Accepted connection from 127.0.0.1 (1 connections).
[2024-06-21 14:58:01.367] [info] Received message from 127.0.0.1: CLS\r\n
[2024-06-21 14:58:01.387] [info] Sent more than 512 bytes to 127.0.0.1
[2024-06-21 14:58:02.398] [info] async_event_server[:5250] Client 127.0.0.1 disconnected (0 connections).
[2024-06-21 14:58:05.413] [warning] ffmpeg[https://go_fm.streampartner.nl/live|547.0200/0.0000] Waiting for audio frame...
[2024-06-21 14:58:06.023] [error] [ffmpeg] [tcp @ 0000020114D11E80] Connection to tcp://go_fm.streampartner.nl:443 failed: Error number -138 occurred
[2024-06-21 14:58:06.023] [error]
[2024-06-21 14:58:06.023] [error] [ffmpeg] [tls @ 0000020114D10700] The specified session has been invalidated for some reason.
[2024-06-21 14:58:06.023] [error]
[2024-06-21 14:58:06.023] [warning] [ffmpeg] [https @ 0000020114A28940] Will reconnect at 13130914 in 3 second(s), error=I/O error.
[2024-06-21 14:58:06.023] [warning]
[2024-06-21 14:58:11.376] [info] async_event_server[:5250] Accepted connection from 127.0.0.1 (1 connections).
[2024-06-21 14:58:11.376] [info] Received message from 127.0.0.1: CLS\r\n
[2024-06-21 14:58:11.400] [info] Sent more than 512 bytes to 127.0.0.1
[2024-06-21 14:58:12.413] [info] async_event_server[:5250] Client 127.0.0.1 disconnected (0 connections).
[2024-06-21 14:58:14.070] [error] [ffmpeg] [tcp @ 0000020114D11000] Connection to tcp://go_fm.streampartner.nl:443 failed: Error number -138 occurred
[2024-06-21 14:58:14.070] [error]
[2024-06-21 14:58:14.070] [error] [ffmpeg] [tls @ 0000020114D10700] The specified session has been invalidated for some reason.
[2024-06-21 14:58:14.070] [error]
[2024-06-21 14:58:14.070] [warning] [ffmpeg] [https @ 0000020114A28940] Will reconnect at 13130914 in 7 second(s), error=I/O error.
[2024-06-21 14:58:14.070] [warning]
[2024-06-21 14:58:15.758] [warning] ffmpeg[https://go_fm.streampartner.nl/live|547.0200/0.0000] Waiting for audio frame...
[2024-06-21 14:58:21.263] [warning] [ffmpeg] [https @ 0000020114A28940] HTTP error 404 File Not Found
[2024-06-21 14:58:21.263] [warning]
[2024-06-21 14:58:21.264] [error] [ffmpeg] [tls @ 0000020114D10700] The specified session has been invalidated for some reason.
[2024-06-21 14:58:21.264] [error]
[2024-06-21 14:58:21.264] [warning] [ffmpeg] [https @ 0000020114A28940] Will reconnect at 13130914 in 15 second(s), error=I/O error.
[2024-06-21 14:58:21.264] [warning]
[2024-06-21 14:58:21.375] [info] async_event_server[:5250] Accepted connection from 127.0.0.1 (1 connections).
[2024-06-21 14:58:21.375] [info] Received message from 127.0.0.1: CLS\r\n
[2024-06-21 14:58:21.408] [info] Sent more than 512 bytes to 127.0.0.1
[2024-06-21 14:58:22.417] [info] async_event_server[:5250] Client 127.0.0.1 disconnected (0 connections).
[2024-06-21 14:58:26.114] [warning] ffmpeg[https://go_fm.streampartner.nl/live|547.0200/0.0000] Waiting for audio frame...
[2024-06-21 14:58:31.367] [info] async_event_server[:5250] Accepted connection from 127.0.0.1 (1 connections).
[2024-06-21 14:58:31.367] [info] Received message from 127.0.0.1: CLS\r\n
[2024-06-21 14:58:31.388] [info] Sent more than 512 bytes to 127.0.0.1
[2024-06-21 14:58:32.399] [info] async_event_server[:5250] Client 127.0.0.1 disconnected (0 connections).
[2024-06-21 14:58:36.465] [warning] ffmpeg[https://go_fm.streampartner.nl/live|547.0200/0.0000] Waiting for audio frame...
[2024-06-21 14:58:36.631] [error] [ffmpeg] [mp3float @ 0000020114C51A80] Header missing
[2024-06-21 14:58:36.631] [error]
[2024-06-21 14:58:36.643] [warning] ffmpeg[https://go_fm.streampartner.nl/live|547.0200/0.0000] Latency: 4733
[2024-06-21 14:58:36.648] [fatal] #######################
[2024-06-21 14:58:36.648] [fatal] UNHANDLED EXCEPTION:
[2024-06-21 14:58:36.648] [fatal] Adress:00007FFF728EBA99
[2024-06-21 14:58:36.648] [fatal] Code:3765269347
[2024-06-21 14:58:36.648] [fatal] Flag:1
[2024-06-21 14:58:36.648] [fatal] Info:00000007D98FF490
[2024-06-21 14:58:36.648] [fatal] Continuing execution.
[2024-06-21 14:58:36.648] [fatal] #######################

Steps to reproduce

The error does not happen each time, but when it is, it is playing an mp3 stream

Environment

• Commit: 1e25c7a Stable
• Server version: 2.4
• Operating system: Windows 10

[leftearsons]

I have also faced multiple cases with Fatal errors causing CasparCG to crash, which seem to be related to unhandled ffmpeg errors.
@Julusian, Do all ffmpeg errors cause CasparCG to exit with a fatal error?

Below are the logs from some related examples i have (also related to ffmpeg audio (aac codec) like the mp3 case above) :

[2024-05-24 14:54:41.093] [info]    [ffmpeg] [https @ 00000228F7B054C0] Opening 'https://lc-live-http-ipv4.akamaized.net/at/cw/30714/4715109/desktop-sydney/basketball_australia_nbl1_north_retail_4715109_1364k_1716545710_0001492.ts?cid=30714&mid=46686537&ecid=4715109&pid=1&dtid=1&sid=375823909825&gc=-OA&gsd=_g&grm=1&hdnts=ip=85.73.218.100~exp=1716637003~acl=%2Fat%2Fcw%2F30714%2F4715109%2Fdesktop-sydney%2F%2A~hmac=508f61d28dc35398b70dbffe7ddbc3c7835a0c884d15efe747ada3a37a93ac9a' for reading
[2024-05-24 14:54:41.093] [info]    
[2024-05-24 14:54:42.332] [warning] [ffmpeg] [aac @ 00000228FF5014C0] Sample rate index in program config element does not match the sample rate index configured by the container.
[2024-05-24 14:54:42.332] [warning] 
[2024-05-24 14:54:42.332] [error]   [ffmpeg] [aac @ 00000228FF5014C0] decode_pce: Input buffer exhausted before END element found
[2024-05-24 14:54:42.332] [error]   
[2024-05-24 14:54:42.333] [fatal]   #######################
[2024-05-24 14:54:42.333] [fatal]    UNHANDLED EXCEPTION: 
[2024-05-24 14:54:42.333] [fatal]   Adress:00007FFCEE3B4F99
[2024-05-24 14:54:42.333] [fatal]   Code:3765269347
[2024-05-24 14:54:42.333] [fatal]   Flag:1
[2024-05-24 14:54:42.333] [fatal]   Info:00000057D072F5B0
[2024-05-24 14:54:42.333] [fatal]   Continuing execution. 
[2024-05-24 14:54:42.333] [fatal]   #######################

[2023-12-19 08:21:09.782] [info]    [ffmpeg] [https @ 000001FC4B75E080] Opening 'https://lc-live-http-ipv4.akamaized.net/at/30714/4071431/desktop/tennis_itf_women_itf_new_zealand_02a_women_singles_retail_4071431_1364k_1702964371_1239.ts?cid=30714&mid=46269047&ecid=4071431&pid=1&dtid=1&sid=181807191443&gc=tU8&gsd=uw&grm=1&hdnts=ip=85.73.218.100~exp=1703052098~acl=%2Fat%2F30714%2F4071431%2Fdesktop%2F%2A~hmac=f82a35cddd0e86fc8acf4e743091780225003f7e29280cfefc0c71639ffceec8' for reading
[2023-12-19 08:21:09.782] [info]    
[2023-12-19 08:21:12.254] [warning] [ffmpeg] [aac @ 000001FC60D02880] Multiple frames in a packet.
[2023-12-19 08:21:12.254] [warning] 
[2023-12-19 08:21:12.333] [error]   [ffmpeg] [aac @ 000001FC60D02880] Error decoding AAC frame header.
[2023-12-19 08:21:12.333] [error]   
[2023-12-19 08:21:12.337] [fatal]   #######################
[2023-12-19 08:21:12.337] [fatal]    UNHANDLED EXCEPTION: 
[2023-12-19 08:21:12.337] [fatal]   Adress:00007FFA6B6ECD29
[2023-12-19 08:21:12.337] [fatal]   Code:3765269347
[2023-12-19 08:21:12.337] [fatal]   Flag:1
[2023-12-19 08:21:12.337] [fatal]   Info:00000047E705F2D0
[2023-12-19 08:21:12.337] [fatal]   Continuing execution. 
[2023-12-19 08:21:12.337] [fatal]   #######################

If this is the case (unhandled ffmpeg errors causing CasparCG to crash), wouldn't it make sense to just kill the related producer/consumer when such an internal ffmpeg error occurs?