Genymobile
Using pc microphone to talk during a call
- [x] I have checked that a similar feature request does not already exist.
Is your feature request related to a problem? Please describe. Because of my disability i can only control my smartphone with scrcpy. Sometimes whenever i receive a call, the smartphone is too far away from me and they can't hear me properly using handsfree.
Describe the solution you'd like Redirecting the input of the microphone plugged to my pc to the smartphone mic would be fantastic.
According to its website, AudioRelay forwards audio from the device microphone to the computer speakers, or from the computer microphone to the device speakers.
It does not expose the computer microphone as an audio input on the device (so that you can use it phone calls for example), that's very different.
I have a POC that can inject audio into Android microphone on Android 13. Previous versions are not supported because ADB shell doesn't have the required permission (MODIFY_AUDIO_ROUTING). It doesn't create a new microphone, but adds audio into each input device (needs to be exhaustively listed):
I tested it in Chrome and MIUI Sound Recorder apps, it might not work in other apps if they use a different input device or some completely different audio record API.
import android.annotation.SuppressLint;
import android.content.Context;
import android.media.AudioAttributes;
import android.media.AudioFormat;
import android.media.AudioManager;
import android.media.AudioTrack;
import android.media.MediaRecorder;
import android.os.Looper;
import java.util.Objects;
public class Main {
private static AudioAttributes createAudioAttributes(int capturePreset) throws Exception {
var audioAttributesBuilder = new AudioAttributes.Builder();
var setCapturePresetMethod =
audioAttributesBuilder.getClass().getDeclaredMethod("setCapturePreset", int.class);
setCapturePresetMethod.invoke(audioAttributesBuilder, capturePreset);
return audioAttributesBuilder.build();
}
@SuppressLint("DefaultLocale")
public static void main(String... args) throws Exception {
Thread.setDefaultUncaughtExceptionHandler((t, e) -> {
e.printStackTrace();
});
Looper.prepareMainLooper();
// var systemMain = android.app.ActivityThread.systemMain();
@SuppressLint("PrivateApi")
var activityThreadClass = Class.forName("android.app.ActivityThread");
@SuppressLint("DiscouragedPrivateApi")
var systemMainMethod = activityThreadClass.getDeclaredMethod("systemMain");
var systemMain = systemMainMethod.invoke(null);
Objects.requireNonNull(systemMain);
// var systemContext = systemMain.getSystemContext();
var getSystemContextMethod = systemMain.getClass().getDeclaredMethod("getSystemContext");
var systemContext = (Context) getSystemContextMethod.invoke(systemMain);
Objects.requireNonNull(systemContext);
// var audioMixRuleBuilder = new AudioMixingRule.Builder();
@SuppressLint("PrivateApi")
var audioMixRuleBuilderClass =
Class.forName("android.media.audiopolicy.AudioMixingRule$Builder");
var audioMixRuleBuilder = audioMixRuleBuilderClass.newInstance();
try {
// Added in Android 13, but previous versions don't work because lack of permission.
// audioMixRuleBuilder.setTargetMixRole(MIX_ROLE_INJECTOR);
var setTargetMixRoleMethod =
audioMixRuleBuilder.getClass().getDeclaredMethod("setTargetMixRole", int.class);
var MIX_ROLE_INJECTOR = 1;
setTargetMixRoleMethod.invoke(audioMixRuleBuilder, MIX_ROLE_INJECTOR);
} catch (Exception ignored) {
}
var addMixRuleMethod = audioMixRuleBuilder.getClass()
.getDeclaredMethod("addMixRule", int.class, Object.class);
var RULE_MATCH_ATTRIBUTE_CAPTURE_PRESET = 0x1 << 1;
// audioMixRuleBuilder.addMixRuleMethod(RULE_MATCH_ATTRIBUTE_CAPTURE_PRESET,
// createAudioAttributes(MediaRecorder.AudioSource.DEFAULT));
addMixRuleMethod.invoke(audioMixRuleBuilder, RULE_MATCH_ATTRIBUTE_CAPTURE_PRESET,
createAudioAttributes(MediaRecorder.AudioSource.DEFAULT));
// audioMixRuleBuilder.addMixRuleMethod(RULE_MATCH_ATTRIBUTE_CAPTURE_PRESET,
// createAudioAttributes(MediaRecorder.AudioSource.MIC));
addMixRuleMethod.invoke(audioMixRuleBuilder, RULE_MATCH_ATTRIBUTE_CAPTURE_PRESET,
createAudioAttributes(MediaRecorder.AudioSource.MIC));
// audioMixRuleBuilder.addMixRuleMethod(RULE_MATCH_ATTRIBUTE_CAPTURE_PRESET,
// createAudioAttributes(MediaRecorder.AudioSource.VOICE_COMMUNICATION));
addMixRuleMethod.invoke(audioMixRuleBuilder, RULE_MATCH_ATTRIBUTE_CAPTURE_PRESET,
createAudioAttributes(
MediaRecorder.AudioSource.VOICE_COMMUNICATION));
// audioMixRuleBuilder.addMixRuleMethod(RULE_MATCH_ATTRIBUTE_CAPTURE_PRESET,
// createAudioAttributes(MediaRecorder.AudioSource.UNPROCESSED));
addMixRuleMethod.invoke(audioMixRuleBuilder, RULE_MATCH_ATTRIBUTE_CAPTURE_PRESET,
createAudioAttributes(MediaRecorder.AudioSource.UNPROCESSED));
// var audioMixingRule = audioMixRuleBuilder.build();
var audioMixRuleBuildMethod = audioMixRuleBuilder.getClass().getDeclaredMethod("build");
var audioMixingRule = audioMixRuleBuildMethod.invoke(audioMixRuleBuilder);
Objects.requireNonNull(audioMixingRule);
// var audioMixBuilder = new AudioMix.Builder(audioMixingRule);
@SuppressLint("PrivateApi")
var audioMixBuilderClass = Class.forName("android.media.audiopolicy.AudioMix$Builder");
var audioMixBuilderConstructor =
audioMixBuilderClass.getDeclaredConstructor(audioMixingRule.getClass());
var audioMixBuilder = audioMixBuilderConstructor.newInstance(audioMixingRule);
var audioFormat = new AudioFormat.Builder().setEncoding(AudioFormat.ENCODING_PCM_16BIT)
.setSampleRate(44100)
.setChannelMask(AudioFormat.CHANNEL_IN_STEREO)
.build();
// audioMixBuilder.setFormat(audioFormat);
var setFormatMethod =
audioMixBuilder.getClass().getDeclaredMethod("setFormat", AudioFormat.class);
setFormatMethod.invoke(audioMixBuilder, audioFormat);
// audioMixBuilder.setRouteFlags(ROUTE_FLAG_LOOP_BACK);
var setRouteFlagsMethod =
audioMixBuilder.getClass().getDeclaredMethod("setRouteFlags", int.class);
var ROUTE_FLAG_LOOP_BACK = 0x1 << 1;
setRouteFlagsMethod.invoke(audioMixBuilder, ROUTE_FLAG_LOOP_BACK);
// var audioMix = audioMixBuilder.build();
var audioMixBuildMethod = audioMixBuilder.getClass().getDeclaredMethod("build");
var audioMix = audioMixBuildMethod.invoke(audioMixBuilder);
Objects.requireNonNull(audioMix);
// var audioPolicyBuilder = new AudioPolicy.Builder(systemContext);
@SuppressLint("PrivateApi")
var audioPolicyBuilderClass =
Class.forName("android.media.audiopolicy.AudioPolicy$Builder");
var audioPolicyBuilderConstructor =
audioPolicyBuilderClass.getDeclaredConstructor(Context.class);
var audioPolicyBuilder = audioPolicyBuilderConstructor.newInstance(systemContext);
// audioPolicyBuilder.addMix(audioMix);
var addMixMethod =
audioPolicyBuilder.getClass().getDeclaredMethod("addMix", audioMix.getClass());
addMixMethod.invoke(audioPolicyBuilder, audioMix);
// var audioPolicy = audioPolicyBuilder.build();
var audioPolicyBuildMethod = audioPolicyBuilder.getClass().getDeclaredMethod("build");
var audioPolicy = audioPolicyBuildMethod.invoke(audioPolicyBuilder);
Objects.requireNonNull(audioPolicy);
var audioManager = (AudioManager) systemContext.getSystemService(AudioManager.class);
// audioManager.registerAudioPolicy(audioPolicy);
var registerAudioPolicyMethod = audioManager.getClass()
.getDeclaredMethod("registerAudioPolicy", audioPolicy.getClass());
// noinspection DataFlowIssue
var result = (int) registerAudioPolicyMethod.invoke(audioManager, audioPolicy);
if (result != 0) {
System.out.println("registerAudioPolicy failed");
return;
}
// var audioTrack = audioPolicy.createAudioTrackSource(audioMix);
var createAudioTrackSourceMethod = audioPolicy.getClass()
.getDeclaredMethod("createAudioTrackSource", audioMix.getClass());
var audioTrack = (AudioTrack) createAudioTrackSourceMethod.invoke(audioPolicy, audioMix);
Objects.requireNonNull(audioTrack);
audioTrack.play();
// Generate a square wave at around 440Hz
var samples = new short[440 * 100];
for (var i = 0; i < samples.length; i += 1) {
samples[i] = (i / 100) % 2 == 0 ? Short.MAX_VALUE : Short.MIN_VALUE;
}
new Thread(() -> {
while (true) {
System.out.println("write");
audioTrack.write(samples, 0, samples.length);
}
}).start();
}
}
I couldn't
Il Gio 29 Giu 2023, 10:04 Pardiaca @.***> ha scritto:
Would be a great feature!
@Blastgraphic https://github.com/Blastgraphic , could you find an alternative?
— Reply to this email directly, view it on GitHub https://github.com/Genymobile/scrcpy/issues/3880#issuecomment-1612594695, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEMAYMO7VBWAYUBJKOTXN5TXNUZHNANCNFSM6AAAAAAWNKSANI . You are receiving this because you were mentioned.Message ID: @.***>
What ChatGPT says:
Yes, there are software solutions available that can help you redirect the input of a microphone connected to your PC to your smartphone's microphone during a call. One such software is VB-CABLE Virtual Audio Device.
Unfortunately vb-cable works only on mac or win, yes it redirects various audio inputs into output but you can t redirect from pc to android. (I use vb-cable during karaoke online with my friends to let other listen my desktop audio and mic in sync.) Although If someone finds a way to do this, i ll be more than happy.
Il Gio 29 Giu 2023, 10:12 Pardiaca @.***> ha scritto:
What ChatGPT says:
Yes, there are software solutions available that can help you redirect the input of a microphone connected to your PC to your smartphone's microphone during a call. One such software is VB-CABLE Virtual Audio Device.
— Reply to this email directly, view it on GitHub https://github.com/Genymobile/scrcpy/issues/3880#issuecomment-1612604147, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEMAYMLEJPYLGVEVL3D2ZCLXNU2FPANCNFSM6AAAAAAWNKSANI . You are receiving this because you were mentioned.Message ID: @.***>
I saw this thread a few weeks ago and looked for alternative solutions. I found one and can confirm it is working after testing, however it does depend on pretty specific bluetooth hardware (I had success using a BCM2070) and a cleanroom environment (I used archlinux).
The bluetooth workaround is confirmed working and highly flexible (because the call audio becomes a set of ALSA devices), e.g. this is what I see on my machine:
$ bluealsa-aplay -L
bluealsa:DEV=...,PROFILE=a2dp,SRV=org.bluealsa
Galaxy S10e, trusted phone, capture
A2DP (SBC): S16_LE 2 channels 44100 Hz
bluealsa:DEV=...,PROFILE=sco,SRV=org.bluealsa
Galaxy S10e, trusted phone, capture
SCO (CVSD): S16_LE 1 channel 8000 Hz
bluealsa:DEV=...,PROFILE=sco,SRV=org.bluealsa
Galaxy S10e, trusted phone, playback
SCO (CVSD): S16_LE 1 channel 8000 Hz
Using alsa loopback (see the wiki link below for the specific commands) I am able to route my PC mic into the android phone call. Likewise I can loopback the output audio to play on the PC speakers, thus mixing with the scrcpy regular media output.
I know it's quite the cumbersome project but I wanted to at least chime in for what it's worth and say it's certainly possible and confirmed working if you really need something like this.
https://github.com/Arkq/bluez-alsa/wiki/Using-BlueALSA-with-HFP-and-HSP-Devices
Awesome, thanks for the info!
Il giorno mer 5 lug 2023 alle ore 10:42 Keyvan Fatehi < @.***> ha scritto:
I saw this thread a few weeks ago and looked for alternative solutions. I found one and can confirm it is working after testing, however it does depend on pretty specific bluetooth hardware (I had success using a BCM2070) and a cleanroom environment (I used archlinux).
The bluetooth workaround is confirmed working and highly flexible (because the call audio becomes a set of ALSA devices), e.g. this is what I see on my machine:
$ bluealsa-aplay -L bluealsa:DEV=...,PROFILE=a2dp,SRV=org.bluealsa Galaxy S10e, trusted phone, capture A2DP (SBC): S16_LE 2 channels 44100 Hz bluealsa:DEV=...,PROFILE=sco,SRV=org.bluealsa Galaxy S10e, trusted phone, capture SCO (CVSD): S16_LE 1 channel 8000 Hz bluealsa:DEV=...,PROFILE=sco,SRV=org.bluealsa Galaxy S10e, trusted phone, playback SCO (CVSD): S16_LE 1 channel 8000 Hz
Using alsa loopback (see the wiki link below for the specific commands) I am able to route my PC mic into the android phone call. Likewise I can loopback the output audio to play on the PC speakers, thus mixing with the scrcpy regular media output.
I know it's quite the cumbersome project but I wanted to at least chime in for what it's worth and say it's certainly possible and confirmed working if you really need something like this.
https://github.com/Arkq/bluez-alsa/wiki/Using-BlueALSA-with-HFP-and-HSP-Devices
— Reply to this email directly, view it on GitHub https://github.com/Genymobile/scrcpy/issues/3880#issuecomment-1621303718, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEMAYMME43YPWOQBU27NQTDXOUSF7ANCNFSM6AAAAAAWNKSANI . You are receiving this because you were mentioned.Message ID: @.***>
and a cleanroom environment (I used archlinux).
Thank you for sharing your solution. Is there a way to do this in Windows environment? Would it be possible to explain the steps as a guideline for implementing it in Windows environment?
@Nirvanatin as far as I am aware, there is nothing like bluealsa on Windows. However, if your Windows supports Hyper-V, and you have acquired the compatible bluetooth hardware mentioned earlier, it may be compatible with the USB passthrough software with which you can operate bluealsa within an archlinux VM on your Windows computer.
Would it be possible to explain the steps as a guideline for implementing it in Windows environment?
I can check the aforementioned hardware/software compatibility and let you know if this is possible in a follow-up comment.
@Nirvanatin I tried getting it to work within Windows as described but I ran into a wall. That said, it appears that this person was able to get that particular adapter (BCM2070) working but it's unclear to me what I may be missing. If you make an attempt I'd be curious to know your results. Best of luck!
According to its website, AudioRelay forwards audio from the device microphone to the computer speakers, or from the computer microphone to the device speakers.
It does not expose the computer microphone as an audio input on the device (so that you can use it phone calls for example), that's very different.
I use audio relay between two computers:
- A: has keyboard, mouse and a Rode NT US microphone (which also acts as my headphone input)
- B: has the apps that want a microphone and speakers.
start audio relay on both:
- A sends microphone to B,
- B sends speakers to A.
on Computer B, i can launch slack, teams, etc and people hear me speak from the microphone on A.
This would seem to suggest that since audio relay can do it, why can't scrpy?
