Friday, August 16, 2013

Java8's Virtual Extension Methods - Fail?

After an interesting discussion at work last night, I wanted to check my mental model about the new Java 8 feature, virtual extension methods. After reading Anton Arhipov's article on the feature, I came to the conclusion that the feature was conceptually inconsistent and more surprisingly unsafe, effectively failing at a significant piece of the feature's intended purpose.

In Brian Goetz's presentation, he clearly states:
The whole point of this feature is being able to compatibly evolve APIs
He continues to describe that there are two types of compatibility involved, source and binary compatibility, that "The key operation we care about is adding new methods with defaults to existing interfaces" and that "We care more about avoiding binary incompatibilities than source incompatibilities". This goal is appropriate considering that a major motivation for the feature is adding support for forEach to existing collections classes without requiring they implement new methods and recompile to enable support under Java 8.
Brian's notion of binary compatibility puzzles me, considering he identifies the problem I realized after reading Anton's article. Brian states: 
Currently several vectors through which an “innocent” change to an interface can break code... Add an extension method which is identical to an extension method in another interface, and classes exist that implement both interfaces
Multiple inheritance was a primary motivator for interfaces in the Java language, and there are natural places where this situation would arise: for instance implementing similar interfaces for different frameworks. After arguing with coworkers about whether this would be an issue, I decided to test my hypothesis.

The experiment:
  • Create two interfaces
  • Create a class implementing both interfaces
  • Create a main class that dynamically loads the class, instantiates an object, casts the object to the respective interface types and calls the methods
  • Update the interfaces to add a default method with the same signature
  • Create a new main class that calls the inherited method
  • Run the main method with the class built against the original interface
 
interface Interface1 {
  void go1();
}
interface Interface2 {
  void go2();
}
class Class1 implements Interface1, Interface2 {
  void go1() {
    System.out.println("go1");
  }
  void go2() {
    System.out.println("go2");
  }
}
class Main {
  public static void main(String [] args) throws Exception {
    Object o1 = Class.forName("Class1").newInstance();
    Interface1 i1 = (Interface1)o1;
    i1.go1();
    Interface2 i2 = (Interface2)o1;
    i2.go2();
  }
}

Compiling and running the code under Java 6, we get the expected output:
go1
go2

I archived the classes to easily simulate what happens with class libraries. I can mix and match Java 6 implementations of Java 8 interfaces:
  • jre6ifaces.jar - contains Interface types (think java.util.List)
  • jre6usercode.jar - contains Class types (think your implementation of java.util.List)
  • jre6client.jar - contains the main class
I then reimplemented the existing interfaces to have default methods: 

interface Interface1 {
  void go1();
  default void stop() { System.out.println("stop1"); }
}
interface Interface2 {
  void go2();
  default void stop() { System.out.println("stop2"); }
}
I tried to recompile the existing class, which should inherit the functionality from the interfaces. I got an error, so the compiler is checking for this. The designers obviously knew of the problem and protect us from creating that situation for new code. However, remember the primary goal was extending existing code compatibly:

$ javac *.java
Class1.java:1: error: class Class1 inherits unrelated defaults for stop() from types Interface1 and Interface2
public class Class1 implements Interface1, Interface2 {
       ^
By specifying the method directly in the subclass, Interface1's stop implementation, we resolve the conflict and the compiler happily compiles our code. I changed the Main class to use the new method:

public class Main {
  public static void main(String [] args) throws Exception {
    Object c1 = Class.forName("Class1").newInstance();

    Interface1 i = (Interface1)c1;
    i.go1();

    i.stop();
    Interface2 j = (Interface2)c1;

    j.go2();
    j.stop();

  }
}
Again the output is what we would expect:
go1
stop1
go2
stop1
I then archived the java8 classes into jars, just like I did with the Java 6 ones:
  • jre8ifaces.jar - contains Interface types (think java.util.List)
  • jre8usercode.jar - contains Class types (think my impl of java.util.List)
  • jre8client.jar - contains the main class
Finally, I ran the following:

$ java -cp "jre8ifaces.jar;jre8client.jar;jre6usercode.jar" Main
This simulates a natural situation: 
  • ifaces represent the new collections interfaces in Java 8
  • client is new client code under Java 8 wanting to use the forEach loop with pre-Java 8 collections implementations
  • usercode are pre-Java 8 collections implementations that should run safely in the new runtime environment 
I received the following result:

$ java -cp "jre8ifaces.jar;jre8client.jar;jre6usercode.jar" Main
go1
Exception in thread "main" java.lang.AbstractMethodError: Conflicting default methods: Interface1.stop Interface2.stop
        at Class1.stop(Class1.java)
        at Main.main(Main.java:9)
So, despite everything passing static analysis by the compiler, previously defect-free code is failing unexpectedly at runtime under Java 8. Binary compatibility is not upheld. Taking this into account, reconsider the goals according to Oracle's Java language architect, Brian Goetz:
  • "The whole point of this feature is being able to compatibly evolve API"
  • "The key operation we care about is adding new methods with defaults to existing interfaces" 
  • "We care more about avoiding binary incompatibilities than source incompatibilities"
Do you think the goal was achieved?