Taking a picture with the model.

[pecapela10]

Hey team at sceneview.
Thanks for the good work with the library specially in compose it works very nice and super easy to implement.

I want to ask not sure exactly if someone had this issue in the past, but if it is possible to get the current frame image but with the model, I know about the acquire camera image function present in Frame class. But at the current state there is any function possible to do this? If not did someone implement some solution that makes this possible?

Thanks in advance, looking forward to see the future of the library and ready to help at the some point.

Kind regards.
PC

[hlefe]

I don’t know if this might help you, but here’s how I implemented the screenshot capture for my AR view (which properly displays the models) :

private fun handleSnapshot(result: MethodChannel.Result) {
    try {
        mainScope.launch {
            val bitmap =
                withContext(Dispatchers.Main) {
                    val bitmap =
                        Bitmap.createBitmap(
                            sceneView.width,
                            sceneView.height,
                            Bitmap.Config.ARGB_8888,
                        )

                    try {
                        val listener =
                            PixelCopy.OnPixelCopyFinishedListener { copyResult ->
                                if (copyResult == PixelCopy.SUCCESS) {
                                    val byteStream = java.io.ByteArrayOutputStream()
                                    bitmap.compress(Bitmap.CompressFormat.PNG, 100, byteStream)
                                    val byteArray = byteStream.toByteArray()
                                    result.success(byteArray)
                                } else {
                                    result.error("SNAPSHOT_ERROR", "Failed to capture snapshot", null)
                                }
                            }

                        PixelCopy.request(
                            sceneView,
                            bitmap,
                            listener,
                            Handler(Looper.getMainLooper()),
                        )
                    } catch (e: Exception) {
                        result.error("SNAPSHOT_ERROR", e.message, null)
                    }
                }
        }
    } catch (e: Exception) {
        result.error("SNAPSHOT_ERROR", e.message, null)
    }
}

My full use of SceneView can be found here:
https://github.com/hlefe/ar_flutter_plugin_flutterflow/blob/sceneview-android/android/src/main/kotlin/io/carius/lars/ar_flutter_plugin_flutterflow/ArView.kt

[dineshkumarappdeveloper]

Here you go

fun getCurrentFrameAsBitmap(frame: Frame): Bitmap? {
var image: Image? = null
for (i in 0..4) { // Retry up to 5 times
try {
image = frame.acquireCameraImage()
break
} catch (e: NotYetAvailableException) {
Log.d("ARSceneView", "Image is not yet available. Retrying...", e)
Thread.sleep(50) // Wait for 50ms before retrying
}
}

if (image == null) {
    return null // Return null if the image is still not available
}

val planes: Array<Image.Plane> = image.planes
val yBuffer: ByteBuffer = planes[0].buffer
val uBuffer: ByteBuffer = planes[1].buffer
val vBuffer: ByteBuffer = planes[2].buffer

val ySize = yBuffer.remaining()
val uSize = uBuffer.remaining()
val vSize = vBuffer.remaining()

val nv21 = ByteArray(ySize + uSize + vSize)

yBuffer.get(nv21, 0, ySize)
val uBytes = ByteArray(uSize)
val vBytes = ByteArray(vSize)
uBuffer.get(uBytes, 0, uSize)
vBuffer.get(vBytes, 0, vSize)

for (i in 0 until uSize) {
    nv21[ySize + i * 2] = vBytes[i]
    nv21[ySize + i * 2 + 1] = uBytes[i]
}

val yuvImage = YuvImage(nv21, android.graphics.ImageFormat.NV21, image.width, image.height, null)
val out = ByteArrayOutputStream()
yuvImage.compressToJpeg(android.graphics.Rect(0, 0, image.width, image.height), 100, out)
val jpegData = out.toByteArray()

val bitmap = android.graphics.BitmapFactory.decodeByteArray(jpegData, 0, jpegData.size)

image.close()
return bitmap.rotate()

}

fun Bitmap.rotate(value: Float = 90F) : Bitmap {
val matrix = android.graphics.Matrix()
matrix.postRotate(value)
return Bitmap.createBitmap(this, 0, 0, this.width, this.height, matrix, true)
}

[Hanif-MI]

You can get the image using the below code.

import android.graphics.Bitmap
import android.os.Handler
import android.os.HandlerThread
import android.view.PixelCopy
import io.github.sceneview.ar.ARSceneView


const val IMAGE_CAPTURE_HANDLER_NAME = "PixelCopier"

/**
 * Capture image from [ARSceneView] using [PixelCopy].
 *
 * @return Returns the [Bitmap] of view.
 */
fun ARSceneView.captureImage(): Bitmap {
    // Create a bitmap the size of the scene view.
    val bitmap = Bitmap.createBitmap(
        width, height,
        Bitmap.Config.ARGB_8888
    )

    // Create a handler thread to offload the processing of the image.
    val handlerThread = HandlerThread(IMAGE_CAPTURE_HANDLER_NAME)
    handlerThread.start()

    PixelCopy.request(this, bitmap, { copyResult ->
        if (copyResult == PixelCopy.SUCCESS) {
            print("Created bitmap from ARSceneView success.")
        } else {
            print("Failed to create bitmap from ARSceneView.")
        }
        handlerThread.quitSafely()
    }, Handler(handlerThread.looper))

    return bitmap
}