Unmanaged crash when incoming video data has no pixel format

[lisardggY]

I'm writing an app that connects to various RTSP video camera feeds, and every once in a while, a corrupt video stream (which also fails to show on raw ffmpeg or VLC) would not only fail, but also crash my UWP app with an uncatchable memory violation error.

When debugging, I noticed that this error occurred when FFmpeg failed to correctly identify the pixel format for the incoming string, leading the avVideoCodecCtx->pix_fmt set to AV_PIX_FMT_NONE (i.e. -1). When this is passed to sws_getContext in UncompressedVideoSampleProvider::AllocateResources, it spawns the "attempt was made to access invalid memory" error I ultimately get in my C# code.

For now, I've forked the library and added an explicit check, before the call to AllocateResources, which throws an exception (that can be handled by my C# code) that the feed is invalid, and that's a good enough solution for me. However, it might not be the correct general-purpose handling for the library.

I can create a PR for my fix, but I'm guessing you would prefer to fix it some other way. In any case, cleaner error handling (whether via exception or HRESULT) is probably required for these scenarios.

[lukasf]

@mcosmin222 We will have to fix this in our fork as well.