Questions on instrumenting HttpURLConnection (part of Android SDK) api calls using byte buddy android plugin

[surbhiia]

As the Android SDK classes like HttpURLConnection can’t be instrumented, an alternative is to instrument all calls to HttpURLConnection APIs which are in android application code itself. For that, it will be required to visit all application code classes and visit all methods and visit all instructions of a method to identify if any of it is a method call to HttpURLConnection APIs. I plan to use byte buddy android plugin and Advice API to do it. But I want to wrap a particular bytecode instruction ( invokevirtual calls to any of the HttpURLConnection APIs ) with before / after / throw methods. I do not want to wrap a particular method. I did not find a way to do that. Is it possible with any other byte buddy APIs? If I create a custom ClassVisitor, can I wrap it into an AsmVisitorWrapper so it can be supplied to DynamicType.Builder<?> builder.visit() inside apply method of my plugin so I can use this plugin with Byte buddy android plugin library to do compile time instrumentation.

One other concern I had was that as this plugin would apply to all application code classes, it might increase build time. Have you come across any such byte buddy android plugin usage where a plugin applies to all classes and would you know the performance impact of that? Any suggestions to prevent a huge increase in build time while still looking at all classes?

Thanks a lot!

[raphw]

The functionality you are looking for is called MemberSubstitution, where you can replace all interactions with a different method of your choice. Byte Buddy also supports Android builds: https://github.com/raphw/byte-buddy/tree/master/byte-buddy-gradle-plugin/android-plugin

[surbhiia]

Thanks @raphw! Sorry, I did not link to all the things I was referring to in my question. By planning to use byte buddy android plugin - I did mean the same repo that you referred to above. I will look into MemberSubstitution API you linked above and get back to you with more questions if any.

[surbhiia]

Hi @raphw, I’m trying to use MemberSubstitution APIs to replace connection.getResponseCode() instruction with replacementForResponseCode(connection). “replacementForResponseCode(connection)” is not present in android application code which calls “connection.getResponseCode()” but it is present in a separate instrumentation library codebase which android application depends on as shown below. There are a few issues I’m facing, it would be really helpful if you could suggest some things I can try to resolve these. Thanks a lot!

Issues/ Questions

1. Byte buddy plugin is unable to find calls like connection.getResponseCode()

• [it was successfully able to find calls like getResponseCode() - which are not invoked on a connection object instance, when I used .method(named(“getResponseCode”))]

2. It is unable to find replacementForResponseCode method from the instrumentation library codebase.

• [it was successfully able to find replacement methods defined in the same java file which calls getResponseCode(), when I used .replaceWithMethod(named(“ replacementForResponseCode”))]

3. I also want to pass the connection object to the replacement method. so I want to effectively call “replacementForResponseCode(connection)” . Is that possible?

4. I have a list of around 10 methods like “connection.getResponseCode()” which need to be replaced with 10 other methods like “replacementForResponseCode(connection)”. Is it possible to do this in one byte buddy plugin or I will need 10?

Byte buddy Plugin apply code

try {
    return builder.visit(MemberSubstitution.strict()
            .method(is(HttpURLConnection.class.getDeclaredMethod("getResponseCode")))          //Issue 1
            .replaceWith(ReplacementMethods.class.getDeclaredMethod("replacementForResponseCode"))     //Issue 2
            .on(isMethod()));
} catch (NoSuchMethodException e) {
    throw new RuntimeException(e);
}

Another library artifact that contains replacement methods

class ReplacementMethods {

public void replacementForResponseCode(URLConnection connection){
	//Before
	connection.setRequestProperty(“Custom Header”, “DummyValue”);
	
	//Original method call but now it is surrounded by throw
	try {
		String responseCode = connection.getResponseCode();
	} catch(IOException e){
		reportError();
	}

	//After
	reportSuccess();       
}}

Android Application activity code that is instrumented

URL url = new URL("http://httpbin.org/headers");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
String responseCode = connection.getResponseCode();
String responseMessage = connection.getResponseMessage();

Android Application gradle dependencies

bytebuddy(…plugin containing artifact)
implementation(… library artifact containing replacement methods class)

I also tried adding the library artifact with an "api" dependency instead of "implementation". Also, tried including the ReplacementMethods class in the same java package as the byte buddy plugin.

[raphw]

I would assume your replacement method should be static? Is there no exception or anything? Worst case, intercept the visitMethod method in the MemberSubstitution with a debugger to see why the methods are not matched.

[surbhiia]

Thanks @raphw for your suggestions. Changing to static did not work. I debugged to get to the exact error. It is - The replacement class (containing replacement method) is not assignable to the class that contains the method call to be replaced (attaching screenshots). It is coming on MemberSubstituion line 1269 as shown in screenshot below.

But I think the replacement class should be assignable to “java.lang.Object” but that check fails. It checks that in TypeDescription line 7767 (attached screenshot) but that gives false and if condition is not executed.

(Ignore the okhttp3 word in package names in below screenshots. I was using an old setup to test for HttpURLConnection use case.)

Screenshot 1 - Exception string

Screenshot 2 - Exception string continued

Screenshot 3 - MemberSubstitution Line 1269 where error occurs

Screenshot 4 - Debug values for MemberSubstitution Line 1269 when error occurs

Screenshot 5 - Type Description Line 7767 which I think should have matched and no error should have occurred

[raphw]

This does not look right from the error message. Was it not replacementForResponseCode you wanted to substitute with?

[surbhiia]

@raphw Actually what I put in the code snippets here in my previous comment was a representation of what I was doing but the names in my actual code are different. I assure you there's no issues with the method naming. In my actual code the method name is surbhiReplacementMethod() .

The issue here was that the two classes types are not assignable to each other - ReplacementMethods.class and FirstFragment.class (as shown in screenshot 4 above - debug values of the two things compared when error occurs.) The error string is not apt in printing out the method name as what's compared is the two classes.

(it is successfully able to find the method surbhiReplacementMethod() using reflection. And should be able to successfully invoke it if this error comparing classes wouldn't have occurred.)

[raphw]

Can you try with a trivial class:

public class Sample {
  public static int replacementForResponseCode(URLConnection connection) {
    return 0;
  }
}

That is what should do the trick.

[surbhiia]

Thanks a lot @raphw. Super grateful for your help! I tried a class like you suggested and it worked! I was even able to add other parts of my code to this class and test and it worked. Looks like I was missing one of all the requirements earlier - static, same return type, first argument needs to be of same type/supertype as the calling object on which method is invoked.

The only remaining questions I have right now is - I have a list of around 10 methods like “connection.getResponseCode()” which need to be replaced with 10 other methods like “replacementForResponseCode(connection)”. Is it possible to do this in one byte buddy android plugin or I will need 10?

My current plugin code looks like this: return builder.visit(MemberSubstitution.strict() .method(is(HttpURLConnection.class.getDeclaredMethod("getResponseCode"))) //line 1 .replaceWith(Sample.class.getDeclaredMethod("replacementForResponseCode", URLConnection.class)) //line2 .on(isMethod()));

I want to replace a lot of different HttpURLConnection/HttpsURLConnection/URLConnection APIs with my own replacement methods defined in my sample class similar to replacementForResponseCode method. Example:

- HttpURLConnection.class.getDeclaredMethod("getResponseCode") --> Sample.class.getDeclaredMethod("replacementForResponseCode", URLConnection.class)
- URLConnection.class.getDeclaredMethod("getContentType") --> Sample.class.getDeclaredMethod("replacementForURLConnectionContentType", URLConnection.class)

I'm not sure if I can have a list of method definitions to be matched to, in line 1 and how to tell line 2 what method matched, so line 2 replacement function can take it and decide what replacement method to call...

[raphw]

You can stack them. This is the most effective way anyways.

[mmgzentertainment]

can bytebuddy instrument telephonyManager.networkOperator directly within an own app?

[surbhiia]

@raphw I tried to look for ways I can stack multiple byte buddy plugins into one, but didn't find any. Can you please elaborate or point out any available examples to do so? Did you mean using the StackManipulation APIs to define a series of transformations in one DynamicType.Builder.visit method? If so, can you please share a way to stack let's say the below two:

Replacing getResponseCode

`  return builder.visit(MemberSubstitution.relaxed()
                    .method(is(HttpURLConnection.class.getDeclaredMethod("getResponseCode")))
                    .replaceWith(Sample.class.getDeclaredMethod("replacementForResponseCode", URLConnection.class))
                    .on(isMethod()));`

Replacing getResponseMessage

return builder.visit(MemberSubstitution.relaxed()
                    .method(is(HttpURLConnection.class.getDeclaredMethod("getResponseMessage")))
                    .replaceWith(Sample.class.getDeclaredMethod("replacementForResponseMessage", URLConnection.class))
                    .on(isMethod()));

I tried below stacking with "AsmVisitorWrapper.Compound" API and this worked. But was thinking if it can be optimized more.

`    return builder.visit(
                    new AsmVisitorWrapper.Compound(

                        MemberSubstitution.relaxed()
                        .method(is(HttpURLConnection.class.getDeclaredMethod("getResponseCode")))
                        .replaceWith(Sample.class.getDeclaredMethod("replacementForResponseCode", URLConnection.class))
                        .on(isMethod()),

                    MemberSubstitution.relaxed()
                    .method(is(HttpURLConnection.class.getDeclaredMethod("getResponseMessage")))
                    .replaceWith(Sample.class.getDeclaredMethod("replacementForResponseMessage", URLConnection.class))
                    .on(isMethod())

                     )
            );`

It would be great if I can do something like below, but not sure how to get nameofTheMatchedMethodAsString in replaceWith on line 4

`  try {
       return builder.visit(MemberSubstitution.relaxed()
       .method(ElementMatchers.anyOf(HttpURLConnection.class.getDeclaredMethod("getResponseMessage"),
           HttpURLConnection.class.getDeclaredMethod("getResponseCode")))
        .replaceWith(Sample.class.getDeclaredMethod("replacementForAll", URLConnection.class, "**nameofTheMatchedMethodAsString**")).  //line 4
         .on(isMethod()));
   } catch (NoSuchMethodException e) {
            throw new RuntimeException(e);
  }`

[raphw]

Have a looke at the replacement chain where you can add methods with custom arguments, based on annotation.