[REQ] Image/text overlays

[MarcoRavich]

Hi there,
as already asked here it would be useful to implement stream labeling and if can help - since your software relays on ffmpeg too, we found this nice script by @apple-fritter that may help:
https://github.com/apple-fritter/RTSP.downmix.4-to-1_2x2/blob/main/downmix.sh

And what about overlay images too ?
This would be extremely useful for those who needs, for example, safety areas like this:

Hope that inspires !

[iEvgeny]

Hi!
It is planned to implement HTML subset layers that can be rendered on top of the current stream. This will allow to place any information, which in addition can be received from the remote host.
However, for the starters, the plan is to implement hardware decoding acceleration and some refactoring of the application kernel.

[apple-fritter]

That is an interesting suggestion @forart, Tell me more.

[MarcoRavich]

Thanks for reply @apple-fritter !

Well, our non-profit organization choosed cctv-viewer (which guarantees the best latencies) to preview PTZ cams' shots in live shows that we stream, so it would be useful for operators to have overlayed so-called "Safe Zones"/center reference (check out the top-left Preview screen on Atem switchers):

Since this usage is outside the scope of @iEvgeny's project, we can't ask him to implement it as we'd need, but we believe that a text/image overlay feature would be useful for all users.

Last but not least, we're collecting/doxing/sorting all RTSP-viewers related gits (yours too, of course) we found hoping that could stimulate collaboration between their devs. [ dirty-draft here ]

Hope that inspires !

[apple-fritter]

I appreciate being included in this discussion, @forart . I'll take a look at what solutions I can come up with!

[MarcoRavich]

Update:

Here's an ffplay commandline written by (driven) ChatGPT that draws "resolution-aware" EBU safe areas and a simple viewfinder:

ffplay -vf "drawgrid=x=(iw/2):(ih/2):t=3:[email protected], drawbox=x=(iw-(iw*0.675))/2:y=(ih-ih)/2:w=iw*0.675:h=ih:[email protected]:t=fill, drawbox=x=(iw-iw*0.95)/2:y=(ih-ih*0.95)/2:w=iw*0.95:h=ih*0.95:color=red@1, drawbox=x=(iw-iw*0.965)/2:y=(ih-ih*0.965)/2:w=iw*0.965:h=ih*0.965:color=green@1" filename.ext

Hope that inspires.

[MarcoRavich]

Bump.

Here's what getonboard.dev (a ChatGPT-based git analyzer/modifyer) suggest to add such "ffmpeg viewfinder" in cctv:

To overlay a viewfinder in the ievgeny/cctv-viewer project, you would need to modify the QML files that handle video rendering. The primary file for this is src/Player.qml.

However, the current QML and C++ codebase does not support FFmpeg video filters, which you're trying to use. FFmpeg filters like drawgrid, drawbox etc., are not directly usable in QML or C++ without a supporting backend.

To achieve this, you would need to extend the backend (which is based on Qt Multimedia) to support FFmpeg filters. This would involve writing C++ code to interface with the FFmpeg library, apply the filters to the video frames, and then display the filtered frames.

This is a non-trivial task and would require a good understanding of both the FFmpeg library and the Qt Multimedia framework. If you're not familiar with these, you might want to consider hiring a developer who is.

Here's a rough outline of the steps you would need to take:

1. Write a C++ class that uses the FFmpeg library to apply filters to video frames. This class should take raw video frames as input, apply the desired FFmpeg filters, and output the filtered frames.
2. Modify the Player.qml file to use this new class for video rendering. You would need to expose your new C++ class to QML, probably by registering it as a new QML type.
3. Update the QML code to create an instance of your new class, pass the video frames to it, and display the filtered frames.

Please note that this is a high-level outline and the actual implementation could be quite complex, depending on the specifics of your project and the FFmpeg filters you want to use.

And, asking 'Are you able to modify the code to draw an overlayed cross on the center of each video displayed by cctv ?' the output is:

To overlay a cross on the center of each video displayed by the CCTV viewer, you can modify the Player.qml file. You can add a Canvas item to draw the cross. Here's an example of how you might do this:

VideoOutput {
    id: videoOutput
    source: qmlAvPlayer
    anchors.fill: parent
}

Canvas {
    id: crossCanvas
    anchors.fill: parent
    renderTarget: Canvas.FramebufferObject
    contextType: "2d"

    onPaint: {
        var ctx = getContext("2d");
        ctx.strokeStyle = "red";
        ctx.lineWidth = 2;

        // Draw vertical line
        ctx.beginPath();
        ctx.moveTo(width / 2, 0);
        ctx.lineTo(width / 2, height);
        ctx.stroke();

        // Draw horizontal line
        ctx.beginPath();
        ctx.moveTo(0, height / 2);
        ctx.lineTo(width, height / 2);
        ctx.stroke();
    }
}

In this example, the Canvas item is a child of the Rectangle that also contains the VideoOutput. This means it will be drawn on top of the video. The onPaint function is called whenever the canvas needs to be redrawn, and it draws a red cross on the canvas.

Please note that this is a simple example and might not cover all your needs. Depending on your requirements, you might need to create a more complex overlay, handle interactions with the overlay, or dynamically change the overlay based on the video content. These would require more advanced QML and possibly integrating with C++ code as well.

...does @iEvgeny agree with it ?

[iEvgeny]

Only QML is sufficient to implement an overlay. It is not difficult. It is more difficult to implement a quality UI for this functionality.
You can think about it, if you wish. However, it should not be a viewfinder-specific feature.

[MarcoRavich]

Well, I don't know if it's a good idea to embed a QML drawing tool in cctv-viewer, however I would avoid "reinvent the wheel"...
...you could "grab" inspiration from:

• a bit dated but simple and clean Canvas-Demo by @EmttER;
• cool (measurement-only, but should be moddable) @patrickelectric's qml-rules;
• many other projects here @ GH (but honestly dunno if/how works).

Last but not least, I would suggest you to consider @eralban's Custom QML Video Player approach wich overlay controls on top of video content too.

Hope that inspires !

[MarcoRavich]

Just found this very interesting (PySide/PyQT/QML/Python/Qt) overlay camera controls GUI by @makerinchina-iot:

(https://img-blog.csdnimg.cn/0006f96ec89c4295817c573015b7f4de.png)

Git: https://github.com/makerinchina-iot/raspberry_pyside_notes#v-6-camera-viewer-gui

[iEvgeny]

Refactoring the viewport layout engine and overlays is a priority. I hope for some result by the end of this year.

[MarcoRavich]

Cool.

May more resources (such as the above one) help ?

[iEvgeny]

I basically already have a finished idea of what it should look like. I just need to find the time.

[MarcoRavich]

I basically already have a finished idea of what it should look like. I just need to find the time.

Cool, we'll have to just wait then.

[MarcoRavich]

Only QML is sufficient to implement an overlay. It is not difficult.

Bump.

As suggested, I finally putted my fingers on QML code (Player.qml), so here's the 1st working test of a (basic/rude but proportional) viewfinder + EBU safe areas + dotted diagonals overlayed in all cctv-viewer streams-areas:
https://github.com/MarcoRavich/cctv-viewer/blob/c43510309fdf2ef2902c372e2cb3c4a13260ebee/src/Player.qml

...of course you need to build it.

I'll play more with. Suggestions are welcome.

(screenshots soon)

EDIT
@iEvgeny Should I put those mods into ViewportsLayout.qml to activate/deactivate them on (specific) key pressing ?

[MarcoRavich]

Bump2

Player.qml updated again: viewfinder is empty now.

https://github.com/MarcoRavich/cctv-viewer/blob/f9021fdea965912219a0d8da1ef61570692dcca6/src/Player.qml

[iEvgeny]

Unfortunately, I don't have time to review other people's code right now.
However, I can safely say that in this form, due to its narrow specialization, this code cannot be included in the main branch.

[MarcoRavich]

Unfortunately, I don't have time to review other people's code right now.

Hi there, thanks for reply.

However, I can safely say that in this form, due to its narrow specialization, this code cannot be included in the main branch.

Don't worry, it's a "very-alpha" customization for a pretty specific use (so, as you can notice, I've forked the git but didn't request any pull, actually).

However, I would like to make both safe areas and the viewfinder overlays de/activateable on certain-keys pressing.

So the question is: is better to implement those overlays code inside Player.qml or ViewportsLayout.qml ?

Thanks in advance fo any suggestion you can provide.

[iEvgeny]

Ofcourse, inside Player.qml. But handling possible shortcuts for de/activate viewfinder overlay should be implemented in ViewportsLayout.qml.

See handling "F" shortcut as example:

cctv-viewer/src/ViewportsLayout.qml

Line 254 in 8bd8571

var fullScreenKey = QT_TR_NOOP("F", "Shortcut");

[MarcoRavich]

Ofcourse, inside Player.qml. But handling possible shortcuts for de/activate viewfinder overlay should be implemented in ViewportsLayout.qml.

See handling "F" shortcut as example:

cctv-viewer/src/ViewportsLayout.qml

Line 254 in 8bd8571

var fullScreenKey = QT_TR_NOOP("F", "Shortcut");

Since I've almost finished the viewfinder/guides overlays (my updated Player.qml), I'm going to modify ViewportsLayout.qml to trying make those de/activable by key pressing.

ChatGPT generates this code for, but I honestly don't know how to integrate it into ViewportsLayout.qml...

Item {
    id: viewportsLayout
    focus: true // Capture keyboard events

    // Overlays controls
    property bool showCenteredDottedCross: false
    property bool showPerpendicularGuidelines: false
    property bool showGraphicSafeArea: false

    GridLayout {
        id: layout
        anchors.fill: parent
        columns: 2
        spacing: 10

        // Player component example
        Repeater {
            model: 4 // Viewports number
            Player {
                id: player
                showCenteredDottedCross: viewportsLayout.showCenteredDottedCross
                showPerpendicularGuidelines: viewportsLayout.showPerpendicularGuidelines
                showGraphicSafeArea: viewportsLayout.showGraphicSafeArea
                Layout.fillWidth: true
                Layout.fillHeight: true
            }
        }
    }

    // Keyboard events manager
    Keys.onPressed: {
        switch (event.key) {
        case Qt.Key_P:
            showCenteredDottedCross = !showCenteredDottedCross;
            console.log("Centered Dotted Cross toggled:", showCenteredDottedCross);
            break;
        case Qt.Key_O:
            showPerpendicularGuidelines = !showPerpendicularGuidelines;
            console.log("Perpendicular Guidelines toggled:", showPerpendicularGuidelines);
            break;
        case Qt.Key_I:
            showGraphicSafeArea = !showGraphicSafeArea;
            console.log("Graphic Safe Area toggled:", showGraphicSafeArea);
            break;
        default:
            break;
        }
    }
}

...and of course Player.qml needs to be modded too:

Item {
    id: player

    property bool showCenteredDottedCross: false
    property bool showPerpendicularGuidelines: false
    property bool showGraphicSafeArea: false

    Canvas {
        anchors.fill: parent

        onPaint: {
            var ctx = getContext("2d");
            ctx.clearRect(0, 0, width, height);

            if (player.showCenteredDottedCross) {
                // Center dotted cross
                ctx.strokeStyle = "gray";
                ctx.setLineDash([5, 5]);
                ctx.beginPath();
                ctx.moveTo(width / 2, 0);
                ctx.lineTo(width / 2, height);
                ctx.moveTo(0, height / 2);
                ctx.lineTo(width, height / 2);
                ctx.stroke();
            }

            if (player.showPerpendicularGuidelines) {
                // Perpendicular guidelines
                ctx.setLineDash([]);
                ctx.strokeStyle = "blue";
                ctx.beginPath();
                ctx.moveTo(0, 0);
                ctx.lineTo(width, height);
                ctx.moveTo(width, 0);
                ctx.lineTo(0, height);
                ctx.stroke();
            }

            if (player.showGraphicSafeArea) {
                // EBU graphic-safe area
                ctx.fillStyle = "rgba(255, 0, 0, 0.2)";
                var safeMargin = 0.1; // 10% from border
                ctx.fillRect(width * safeMargin, height * safeMargin,
                             width * (1 - 2 * safeMargin), height * (1 - 2 * safeMargin));
            }
        }
    }
}

I'm pretty confused now...