Camera records video quite bad (drops frames), video isn't smooth

[anonym24]

Next example record video weirdly, it drops a lot of frames, not smooth

What could be the issue?

public class TestMp4MuxerActivity2 extends AppCompatActivity {

    private CameraStreamPublisher streamPublisher;
    private CameraPreviewTextureView cameraPreviewTextureView;
    private InstantVideoCamera instantVideoCamera;
    private Handler handler;
    private HandlerThread handlerThread;
    private TextView outDirTxt;
    private String outputDir;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        outputDir = Environment.getExternalStorageDirectory() + "/test_mp4_encode.mp4";
        setContentView(R.layout.activity_test_mp4_muxer);
        cameraPreviewTextureView = findViewById(R.id.camera_produce_view);
        cameraPreviewTextureView.setOnDrawListener(new H264Encoder.OnDrawListener() {
            @Override
            public void onGLDraw(ICanvasGL canvasGL, List<GLTexture> producedTextures, List<GLTexture> consumedTextures) {
                GLTexture texture = producedTextures.get(0);
                GLTexture mediaTexture = producedTextures.get(1);
                drawVideoFrame(canvasGL, texture.getSurfaceTexture(), texture.getRawTexture(), mediaTexture);
            }

        });
        outDirTxt = findViewById(R.id.output_dir_txt);
        outDirTxt.setText(outputDir);

        instantVideoCamera = new InstantVideoCamera(Camera.CameraInfo.CAMERA_FACING_FRONT, 1920, 1080);

        handlerThread = new HandlerThread("StreamPublisherOpen");
        handlerThread.start();
        handler = new Handler(handlerThread.getLooper()) {
            @Override
            public void handleMessage(Message msg) {
                super.handleMessage(msg);
                StreamPublisher.StreamPublisherParam streamPublisherParam = new StreamPublisher.StreamPublisherParam(1920, 1080, 8000 * 1000, 30, 1, 44100, 19200);
                streamPublisherParam.outputFilePath = outputDir;
                streamPublisherParam.setInitialTextureCount(2);
                streamPublisher.prepareEncoder(streamPublisherParam, new H264Encoder.OnDrawListener() {
                    @Override
                    public void onGLDraw(ICanvasGL canvasGL, List<GLTexture> producedTextures, List<GLTexture> consumedTextures) {
                        GLTexture texture = consumedTextures.get(1);
                        GLTexture mediaTexture = consumedTextures.get(0);
                        drawVideoFrame(canvasGL, texture.getSurfaceTexture(), texture.getRawTexture(), mediaTexture);
                    }

                });
                try {
                    streamPublisher.startPublish();
                } catch (IOException e) {
                    e.printStackTrace();
                    ((TextView)findViewById(R.id.test_camera_button)).setText("START");
                }
            }
        };

        streamPublisher = new CameraStreamPublisher(new MP4Muxer(), cameraPreviewTextureView, instantVideoCamera);
        streamPublisher.setOnSurfacesCreatedListener(new CameraStreamPublisher.OnSurfacesCreatedListener() {
            @Override
            public void onCreated(List<GLTexture> producedTextureList, StreamPublisher streamPublisher) {
                //GLTexture texture = producedTextureList.get(1);
                GLTexture mediaTexture = producedTextureList.get(1);
                streamPublisher.addSharedTexture(new GLTexture(mediaTexture.getRawTexture(),
                        mediaTexture.getSurfaceTexture()));
                //mediaSurface = new Surface(texture.getSurfaceTexture());
            }
        });
    }

    private void drawVideoFrame(ICanvasGL canvasGL, @Nullable SurfaceTexture outsideSurfaceTexture,
                                @Nullable BasicTexture outsideTexture, GLTexture mediaTexture) {
        int width = outsideTexture.getWidth();
        int height = outsideTexture.getHeight();

        canvasGL.drawSurfaceTexture(outsideTexture, outsideSurfaceTexture,
                0, 0, width, height);
    }

    @Override
    protected void onResume() {
        super.onResume();
        streamPublisher.resumeCamera();
    }

    @Override
    protected void onPause() {
        super.onPause();
        streamPublisher.pauseCamera();
        if (streamPublisher.isStart()) {
            streamPublisher.closeAll();
        }
    }

    @Override
    protected void onDestroy() {
        super.onDestroy();
        handlerThread.quitSafely();
    }

    public void clickStartTest(View view) {
        TextView textView = (TextView) view;
        if (streamPublisher.isStart()) {
            streamPublisher.closeAll();
            textView.setText("START");

            String file = Environment.getExternalStorageDirectory() +
                    File.separator + "test_mp4_encode.mp4";
            Intent intent = new Intent(Intent.ACTION_VIEW, Uri.parse(file));
            intent.setDataAndType(Uri.parse(file), "video/mp4");
            startActivity(intent);
        } else {
            streamPublisher.resumeCamera();
            handler.sendEmptyMessage(1);
            textView.setText("STOP");
        }
    }
}

[anonym24]

also video is upside down

[anonym24]

Here's my sample of video recording with text draw filter

AndroidInstantVideoDrawText.zip

I don't why but video recording is very bad here

[ChillingVan]

I try your demo but the video seems good. Have you try to remove the draw text? And are the samples in this project working well for your device? And you may try IAndroidCanvasHelper to draw text

[anonym24]

@ChillingVan it doesn't matter if add text or not, video is still bad

I'm not sure than what looks ok for you, can you send some video recorded using sample?

[ChillingVan]

test_mp4_encode.mp4_20180728_091235.mp4.zip

I compress it from 9MB to 300KB

[anonym24]

just compare next two videos

one was recorded using your library - test_mp4_encode.zip

and the other using default camera app on my phone - VID_20180729_170813.zip

with your library it makes some sharp movements, sharp frame changes

[ChillingVan]

I use PotPlayer and press F many times to analyse frame by frame. But only feels a little bit difference. You may use this:

extends GLMultiTexProducerView

    @Override
    protected int getRenderMode() {
        return GLThread.RENDERMODE_CONTINUOUSLY;
    }

Then this will draw in 60 fps.