pedroSG94
audio tick noise
Hi.
How can I remove tick noise when video and audio(music) are encoded together and send to AWS S3 suing rtmp? Here is test case.
- play music in phone
- I capture the music sound with mic
- also I capture video from camera
Every thing is fine and good except sound. sound have have a tick noise. Your sample app have a same issue.
Hello,
Try this: Go to this line: https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/app/src/main/java/com/pedro/rtpstreamer/openglexample/OpenGlRtmpActivity.java#L114 And add this:
rtmpCamera1.setMicrophoneMode(MicrophoneMode.SYNC);
Check if the problem persists. If your problem is not solved, try with another server to discard a problem with AWS: https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/wiki/Media-servers-tested#node-rtsp-rtmp-server
One more question on tick noise. Now I am using Camera2ApiManager.java. If I use your default app which is use Camera1ApiManager.java, it have much less tick noise than Camera2ApiManager.java. Dose tick noise come from video and audio sync ?
and also in VideoEncoder.java , how can I change KEY_COLOR_FORMAT value, currently it seem to be just work for SURFACE only. Please guide me.. thanks
I haven't audio tick noise with Camera2 or Camera1 in my devices for that reason I told you test using microphone sync mode and check with other server in my last post to try detect the possible reason.
About KEY_COLOR_FORMAT it is only used in Camera1Base with SurfaceView and TextureView, because in this mode, encode frames buffer to buffer mode (you set an Array directly to encoder instead of capture frames from Surface) instead of surface to buffer because this is the only way to support Api 16 and Api 17
I need some guide on audio encoding in MicrophoneManager.java I like to test AudioFormat.ENCODING_PCM_8BIT and also ENCODING_PCM_FLOAT, would guide me ?
What do you want exactly? If you only want encode frames, if you change it in MicrophoneManager all should work fine but you need change few configuration in stream (record should work fine but I never tested it)
Question.
My program is camera2api base, audio from mic make a very crystal sound. Default RTMP of your app, it make more soft sound. In program I can not found any differences in audio configuration . Because tick and popcorn sound is very closely related sound color as I tested results.
Your Default RTMP make tick noise too, but much better than my app which is use camer2api. And also I download Larix broadcast RTMP app from google store, it dose make any tick noise with AWS medialive. But sound color is very soft. that is reason I am trying to make sound soft and I am check video encoding side too. Please guide me .. . thanks a lots
As I told you before. I can't reproduce any audio problem with AWS medialive with my devices so I need few things to know the posible reason and help you:
- Did you try use microphone in sync mode?:
//call it before prepareAudio
rtmpCamera1.setMicrophoneMode(MicrophoneMode.SYNC);
- Can you try stream camera1 and camera2 with OpenGlView and share me audio result?
- Can you record a video with camera1 and camera2 with OpenGlView and check if you have audio problems?
- Can you share me configuration used in AWS medialive (maybe this is realted with that configuration and I can reproduce your problem) and player used?.
About Larix app, Is it making audio ticks? what do you means with sound color very soft?
AWS configuration { "name": "ticknoise", "id": "4592273", "arn": "arn:aws:medialive:ap-northeast-2:015730161442:channel:4592273", "inputAttachments": [ { "inputId": "395987", "inputAttachmentName": "ticknoise", "inputSettings": { "sourceEndBehavior": "CONTINUE", "inputFilter": "FORCED", "filterStrength": 5, "deblockFilter": "ENABLED", "denoiseFilter": "ENABLED", "smpte2038DataPreference": "IGNORE", "audioSelectors": [], "captionSelectors": [] } } ], "state": "RUNNING", "pipelinesRunningCount": 2, "destinations": [ { "id": "stjz1s", "settings": [ { "url": "s3://hlsonly/a" }, { "url": "s3://hlsonly/b" } ], "mediaPackageSettings": [] } ], "egressEndpoints": [ { "sourceIp": "15.164.112.168" }, { "sourceIp": "15.165.65.160" } ], "encoderSettings": { "audioDescriptions": [ { "codecSettings": { "aacSettings": { "inputType": "NORMAL", "bitrate": 192000, "codingMode": "CODING_MODE_2_0", "rawFormat": "NONE", "spec": "MPEG4", "profile": "LC", "rateControlMode": "VBR", "sampleRate": 48000, "vbrQuality": "LOW" } }, "audioTypeControl": "FOLLOW_INPUT", "languageCodeControl": "FOLLOW_INPUT", "name": "audio_242ngc" } ], "captionDescriptions": [], "globalConfiguration": { "supportLowFramerateInputs": "DISABLED", "inputEndAction": "NONE", "outputTimingSource": "INPUT_CLOCK", "outputLockingMode": "PIPELINE_LOCKING" }, "outputGroups": [ { "outputGroupSettings": { "hlsGroupSettings": { "incompleteSegmentBehavior": "AUTO", "discontinuityTags": "INSERT", "adMarkers": [], "captionLanguageSetting": "OMIT", "captionLanguageMappings": [], "hlsCdnSettings": { "hlsS3Settings": { "logUploads": "ENABLED" } }, "inputLossAction": "EMIT_OUTPUT", "manifestCompression": "NONE", "destination": { "destinationRefId": "stjz1s" }, "ivInManifest": "INCLUDE", "ivSource": "FOLLOWS_SEGMENT_NUMBER", "clientCache": "ENABLED", "tsFileMode": "SEGMENTED_FILES", "manifestDurationFormat": "FLOATING_POINT", "segmentationMode": "USE_SEGMENT_DURATION", "redundantManifest": "DISABLED", "outputSelection": "MANIFESTS_AND_SEGMENTS", "streamInfResolution": "INCLUDE", "iFrameOnlyPlaylists": "DISABLED", "indexNSegments": 10, "programDateTime": "EXCLUDE", "programDateTimePeriod": 600, "keepSegments": 10, "segmentLength": 60, "timedMetadataId3Frame": "PRIV", "timedMetadataId3Period": 10, "hlsId3SegmentTagging": "DISABLED", "codecSpecification": "RFC_4281", "directoryStructure": "SINGLE_DIRECTORY", "segmentsPerSubdirectory": 10000, "mode": "LIVE" } }, "outputs": [ { "outputSettings": { "hlsOutputSettings": { "nameModifier": "_1", "hlsSettings": { "standardHlsSettings": { "m3u8Settings": { "audioFramesPerPes": 4, "audioPids": "492-498", "nielsenId3Behavior": "NO_PASSTHROUGH", "patInterval": 0, "pcrControl": "PCR_EVERY_PES_PACKET", "pmtPid": "480", "programNum": 1, "scte35Pid": "500", "scte35Behavior": "NO_PASSTHROUGH", "timedMetadataPid": "502", "timedMetadataBehavior": "NO_PASSTHROUGH", "videoPid": "481" }, "audioRenditionSets": "" } }, "h265PackagingType": "HVC1" } }, "outputName": "7n21qc", "videoDescriptionName": "video_n3k9db", "audioDescriptionNames": [ "audio_242ngc" ], "captionDescriptionNames": [] } ] } ], "timecodeConfig": { "source": "SYSTEMCLOCK" }, "videoDescriptions": [ { "codecSettings": { "h264Settings": { "afdSignaling": "NONE", "colorMetadata": "INSERT", "adaptiveQuantization": "AUTO", "bitrate": 2500000, "bufSize": 3000000, "entropyEncoding": "CABAC", "flickerAq": "ENABLED", "forceFieldPictures": "DISABLED", "framerateControl": "SPECIFIED", "framerateNumerator": 30, "framerateDenominator": 1, "gopBReference": "DISABLED", "gopClosedCadence": 1, "gopSize": 30, "gopSizeUnits": "FRAMES", "subgopLength": "DYNAMIC", "scanType": "PROGRESSIVE", "level": "H264_LEVEL_AUTO", "lookAheadRateControl": "MEDIUM", "maxBitrate": 3000000, "numRefFrames": 1, "parControl": "INITIALIZE_FROM_SOURCE", "profile": "HIGH", "rateControlMode": "VBR", "syntax": "DEFAULT", "sceneChangeDetect": "ENABLED", "spatialAq": "ENABLED", "temporalAq": "ENABLED", "timecodeInsertion": "DISABLED" } }, "height": 720, "name": "video_n3k9db", "respondToAfd": "NONE", "sharpness": 50, "scalingBehavior": "DEFAULT", "width": 1280 } ] }, "roleArn": "arn:aws:iam::015730161442:role/AllowMediaLiveAccessRole", "inputSpecification": { "codec": "AVC", "resolution": "HD", "maximumBitrate": "MAX_10_MBPS" }, "logLevel": "WARNING", "tags": {}, "channelClass": "STANDARD", "pipelineDetails": [ { "pipelineId": "0", "activeInputAttachmentName": "", "activeInputSwitchActionName": "", "activeMotionGraphicsUri": "", "activeMotionGraphicsActionName": "" }, { "pipelineId": "1", "activeInputAttachmentName": "", "activeInputSwitchActionName": "", "activeMotionGraphicsUri": "", "activeMotionGraphicsActionName": "" } ], "maintenanceWindow": "NOT_NEEDED", "maintenanceStatus": "" }
- Did you try use microphone in sync mode?:
//call it before prepareAudio rtmpCamera1.setMicrophoneMode(MicrophoneMode.SYNC);
- Can you try stream camera1 and camera2 with OpenGlView and share me audio result?
- Can you record a video with camera1 and camera2 with OpenGlView and check if you have audio problems?
Can you check this, please?
After check it, Compile my app exmaple in the last commit to test and follow this video to configure AWS exactly as I have to discard problems, it is working perfectly for me: https://www.youtube.com/watch?v=_zet_5rwvZ4&t=590s
Now I am using camera2in my app. To camera1 I have to change many part of my program. anyway I tested rtmpCamera2.setMicrophoneMode(MicrophoneMode.SYNC); results is same. I give you the link can find.. https://drive.google.com/file/d/1IguyjySTho1Q110vjIqm4VIx3ziGueyG/view?usp=sharing https://drive.google.com/file/d/1fWpKu3V996LBLp83lPuN0eYMOJozwHW-/view?usp=sharing
last part of the video you can hear tick and popcorn...
To reduce noise, I change video codec as follow MediaCodecInfo.CodecProfileLevel.MPEG4ProfileAdvancedCoding, MediaCodecInfo.CodecProfileLevel.AACObjectLC thru this I reduced nosise a lot. Is it right way or I have to change any others... or would you give me some proper combination of those ?
can you guide me how to apply OpenGlview ? I just change the following in camera2base,java is it ok ? @Deprecated //public Camera2Base(SurfaceView surfaceView) { public Camera2Base(SurfaceView glInterface) { this.surfaceView = surfaceView; this.context = surfaceView.getContext(); init(context); }
Ok, I can heard ticks in the first video but it is really low, the second is clear in my opinion. We can try this:
- First of all, compile my app example and record a video to compare audio and you can know if the problem is produced streaming or in microphone.
- If you are using noise suppressor or echo cancelation disable it (this filters could produce problem in few devices)
About video codec parameters. How do you set this parameters? VideoEncoder should not be related with audio noise and your second parameter is for audio not video and it is already in use.
About OpenGlview, you don't need to change anything to support openglview. You can check my app example and use OpenGl as in the example.
-
It is from streaming.
-
I off all options, echo and noise cancellation and also set mono.
-
in my main activity.kt, calling prepareVidfeo rtmpCamera2.prepareVideo( if (isVertical) width else height, if (isVertical) height else width, 30, bitrate, 3, CameraHelper.getCameraOrientation(this), MediaCodecInfo.CodecProfileLevel.MPEG4ProfileAdvancedCoding, MediaCodecInfo.CodecProfileLevel.MPEG4Level6
and i tested AudioObjectType.AAC_MAIN and AAC.LC both in aacpacket.kt
From my testing, audio noise is related with video codec and AudioObjectType.
-
OpenGLview have noise also..
Would you give some sample code for MicrophoneManagerManual ?
I just use your code, means that I did not change anything, I just use as you coded, thread share or not.
So I set SYNC then more noise is made.
SYNC mode use MicrophoneManagerManual and ASYNC mode MicrophoneManager.
The only difference is that MicrophoneManager is designed to drop frames in a callback automatically after start and MicrophoneManagerManual need call a method that get a frame for you instead of drop it automatically.
Code example:
//Create microphone
MicrophoneManagerManual microphoneManagerManual = new MicrophoneManagerManual();
//Configure microphone (you can set parameters if needed)
microphoneManagerManual.createMicrophone();
//Start microphone
microphoneManagerManual.start();
//Get pcm frames from microphone. This block the thread so you should use a thread for it.
//Get a frame of default size
microphoneManagerManual.getInputFrame();
//Get a frame with a custom size (fill ByteBuffer)
microphoneManagerManual.read(ByteBuffer, size);
//Stop microphone
microphoneManagerManual.stop();
Sorry I tested again with your OpenGlview and recording 10 minutes. Some part of recorded content have popcorn noises...
Local record using my library with startRecord method, right?
In this case this is related with MicrophoneManager class, maybe with AudioEncoder but it is really weird. I think that your microphone device produce that noise maybe related with a configuration. I recommend you test with other sample rate and using mono channel
Do you have other device to test and discard that you only have problems with one device?
- I think yes, I just tested it your app.
- sorry I have two android phone for test.
Simple question.
- I like to disable auto focus mode in camera2.
- can I change aac audio encoder to opus ? I tired, some part of source modify corresponding to opus. but I could not found all to be modified.
thanks for your helping... thanks
Disable auto focus:
// call it after startpreview
rtmpCamera1.disableAutoFocus();
RTMP doesn't support opus audio codec (page 31): https://www.adobe.com/content/dam/acom/en/devnet/rtmp/pdf/rtmp_specification_1.0.pdf
1 - Did you test multiple sample rate in prepareAudio? Specify prepareAudio configuration (all parameters) tested. 2 - Which devices do you have to test? Let me know both models and API 3 - What do you mean with "I think yes". Did you press start/stop record button in my app example? Check the video in folder: Android/data/com.pedro.rtpstreamer/files/rtmp-rtsp-stream-client-java
- samplerate tested
- 44100
- 22050
- Test Device
- Galaxy A40(SM-A405S) / AOS 11 ver
- Galaxy S10 5G / AOS 11 ver
- Yes, I check as you mentioned folder location. ^^
Question
- When encoding time, does decoder involve in encoding process ?
- Any idea to make sound more soft ... As my test experiences, if sound were sharper than more noise is happen.
I need a help...
I like to modify and add some audio encoder parameters audioFormat.setInteger(MediaFormat.KEY_AAC_SBR_MODE, 0); audioFormat.setInteger(MediaFormat.KEY_COMPLEXITY, 1); audioFormat.setInteger(MediaFormat.KEY_QUALITY, 1); audioFormat.setInteger(MediaFormat.KEY_PRIORITY, 0);
Is it right place ? I modify AudioEncoder.java at 71 line. int channelCount = (isStereo) ? 2 : 1; MediaFormat audioFormat = MediaFormat.createAudioFormat(CodecUtil.AAC_MIME, sampleRate, channelCount); audioFormat.setInteger(MediaFormat.KEY_BIT_RATE, bitRate); audioFormat.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, maxInputSize); audioFormat.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectLC);
I add above parameter in here. codec.configure(audioFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
Yes, that is the right place.
samplerate tested
- 44100
- 22050
Test Device
- Galaxy A40(SM-A405S) / AOS 11 ver
- Galaxy S10 5G / AOS 11 ver
- Yes, I check as you mentioned folder location. ^^
Question
- When encoding time, does decoder involve in encoding process ?
- Any idea to make sound more soft ... As my test experiences, if sound were sharper than more noise is happen.
1 - Decoder is only used in FromFile mode I don't understand you. 2 - I have no more idea. Did you test with my video example to configure stream exactly as I did? It is working for me. https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/issues/925#issuecomment-921588862
Regard on 2 - I have no more idea. Did you test with my video example to configure stream exactly as I did? It is working for me. #925 (comment) Yes, I did it. I did test both media package and S3, but all have same popcorn and tick noise.
Is it just happened to me? I download many rtmp encoder apps from google app store. All of them have same issue.
Now I am testing samsung audio encoder. It generate less noise than google and aac.
Did you test using other rtmp server? https://github.com/ossrs/srs
I had a similar problem time ago, but the noise was really noticeable. This time it is working so much smoother than before (the noise is low compared with the other issue): https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/issues/277 It isn't only you, but I tested in both platform with this problem (YouTube and AWS) and all seems solved because I can't reproduce it like before (for this reason I closed that issue). I have no idea how to solve it.
I did it with WOWZA server, It is great fine. No noise at all. So I dit not with srs... but my current working environment is AWS medialive. it is a problem.... No I am trying to solve this issue with video. I think video is very closely related to audio for example frame rate... Most device of AOS does not support CBR, but audio is just working very periodic manner(sample rate). I will test something on this, I will share when I done this. Does it make sense for audio issues, my test plan ?
questions about buffer mode in audio encoder.
What is difference buffer mode true and false ? From my test, false of buffer mode is better than true of buffer mode. Is it correct test results?
In your case It has no effect. Buffer mode is used only when MediaCodec is used in sync mode (async mode was introduced in API 23+): https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/encoder/src/main/java/com/pedro/encoder/BaseEncoder.java#L68 If you use sync mode in AudioEncoder and set that variable to false the result is no audio data because that variable indicate that you are filling encoder using a buffer instead of render a surface. It is used mainly with VideoEncoder that can use a Surface with API 18+.
It is used only here: https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/encoder/src/main/java/com/pedro/encoder/BaseEncoder.java#L156 This means that you will fill encoder input with a byte[]. It is not necessary if you have a surface, because you will fill that input rendering that surface.
Most device of AOS does not support CBR, but audio is just working very periodic manner(sample rate).
CBR is normally used for Video only and library is configured to use it if you have a hardware encoder with this feature.
rtmpCamera1.setMicrophoneMode(MicrophoneMode.SYNC);
worked for me.
In some devices, library works good, but in some no-name-brand devices the audio has tick noise.
Changing microphone mode to SYNC fixed the problem.
It wasn't a server side problem.
