Screen recording takes long time until first bytes arrive on sender side

[paulwyszynski]

Hi, I have a question regarding screen recording via MediaProjection and streaming it to another device. There's a function getGlInterface().setForceRender(true) which should actually force a constant fps. I've tested it at least with ffplay: Starting the steam doesn't start ffplay. It only starts when the content on the screen starts to change. Tested it with UdpDisplay, GenericStream etc. (Audio source is set to NoAudioSource()) This seems to happen if you don't record audio. With the sample ScreenService from the repo it seems to work as long as you have a valid audio source set up. As you mentioned this is due to the MediaProjection service which only emits video bytes on screen content change.

To Reproduce Steps to reproduce the behavior:

1. Sample App: Disable audio (remove mic etc. audio permission from manifest and in MainActivity)
2. Set audio source to NoAudioSource
3. Start ScreenActivity
4. Mirror screen to ffplay: (e.g.: ffplay -fflags nobuffer -flags low_delay -framedrop udp://@:5001
5. Wait and start to go to the home screen for instance

Expected behavior Stream should start immediately like with allowed audio permissons and valid audio source.

Smartphone (please complete the following information):

• Library version 2.6.2
• Device: Samsung S24
• OS: Android 15
• Media server ?
• Class used: UdpStream, GenericStream, UdpDisplay

Additional context Also remove microphone from the service foreground type in AndroidManifest

[pedroSG94]

Hello,

Did you try configure UdpStream or GenericStream with setOnlyVideo method?

genericStream.getStreamClient().setOnlyVideo(true)

[paulwyszynski]

Hi! yeah I tried this now. Unfortunately, it did not help. I also had to setup audio even I don't want to send it:

   override fun onStartCommand(
        intent: Intent?,
        flags: Int,
        startId: Int,
    ): Int {
        if (intent?.action == STOP_ACTION) {
            stopSelf()
            return START_NOT_STICKY
        }

        startForeground(NOTIFICATION_ID, createNotification())

        val resultCode = intent?.getIntExtra(RESULT_CODE, -1) ?: -1
        val resultData = mediaProjectionIntentHolder.getIntent()
        try {
            mediaProjection = mediaProjectionManager.getMediaProjection(resultCode, resultData!!)
            udpStream =
                GenericStream(
                    this,
                    connectChecker,
                    ScreenSource(this, mediaProjection!!),
                    NoAudioSource(),
                ).apply {
                    getGlInterface().setForceRender(true, 15)
                    getStreamClient().setOnlyVideo(true) // <-----
                }
        } catch (e: Exception) {
            Log.e(TAG, "Failed to get MediaProjection", e)
            stopSelf()
            return START_NOT_STICKY
        }
        udpStream?.prepareVideo(1280, 720, 6144000, rotation = 90)
        udpStream?.prepareAudio(32000, true, 128 * 1000, true, true)
        udpStream?.startStream("udp://$ip:$port")
        return START_STICKY
    }

[paulwyszynski]

ok the issue is not present when streaming to Exoplayer. Only the delay is a bit huge when setting up Exoplayer with UdpDataSource

[pedroSG94]

Hello,

Thank you for the report. I will check it. One option could be create a new AudioSource that send empty bytes (silence) to the stream and use it instead of NoAudioSource.

[pedroSG94]

I created a new AudioSource to solve it: https://github.com/pedroSG94/RootEncoder/commit/d69176afc78309a5fe429184150a06d2d4ce3a87

You can copy the SilenceAudioSource if you want use it in your library version and without wait to a new library version

[paulwyszynski]

Thanks! I'm testing more low latency libs at the moment. If i will have any issues with it, I'll replace the audio source with this one 🎉