Back2Basics: Demystifying Eta Expansion

In this blog, we will talk about eta expansion in Scala. How they behave under the hood before and after Scala version 2.12.

Scala has both methods and functions in Scala. We refer them interchangeably but there are situations where the difference matters.

When you see the bytecode generated by the Scala compiler for Test,

scala> :javap -c Test
Compiled from ""
public class Test {
public boolean isEven(int);
Code:
0: iload_1
1: iconst_2
2: irem
3: iconst_0
4: if_icmpne 11
7: iconst_1
8: goto 12
11: iconst_0
12: ireturn

public scala.Function1<java.lang.Object, java.lang.Object> isEvn();
Code:
0: aload_0
1: getfield #17 // Field isEvn:Lscala/Function1;
4: areturn

public Test();
Code:
0: aload_0
1: invokespecial #21 // Method java/lang/Object."":()V
4: aload_0
5: new #23 // class Test$$anonfun$1
8: dup
9: aload_0
10: invokespecial #26 // Method Test$$anonfun$1."":(LTest;)V
13: putfield #17 // Field isEvn:Lscala/Function1;
16: return
}

Notice, isEven method created using def, is a regular function but isEvn is an instance of the Function1 trait. Besides, that methods are not values but functions are.

scala> val x = 4
x: Int = 4

scala> x
res1: Int = 4

scala> def isEven(x: Int) = x % 2 == 0
isEven: (x: Int)Boolean

scala> val isEvn = (x: Int) => x % 2 == 0
isEvn: Int => Boolean = <Function1>

scala> isEvn
res2: Int => Boolean = <Function1>

scala> isEven
:10: error: missing arguments for method isEven;
follow this method with `_' if you want to treat it as a partially applied function
isEven
^

Notice, when we print the value of x and isEvn both gave us the information of itself but when we tried to do with isEven method, the compiler throws an error because isEven is not a value.

We saw that when we create a method using val, Scala compiler created an instance of the Function1 trait.

There are series of traits from Function0, Function1 to Function22. You can not have the Function23 trait. And for every function value scala compiler creates an anonymous class.

scala> val isEvn = (x: Int) => x % 2 == 0
isEvn: Int => Boolean = <Function1>

scala> val isEvn = new Function1[Int, Boolean]{
    | def apply(x: Int) = x % 2 == 0
    | }
isEvn: Int => Boolean = <Function1>

Second isEvn implementation is the actual implementation done by Scala Compiler under the hood. Every FunctionN trait has an apply method where our implementation goes. When you compile code Test.scala and see Scala compiler creates two class files for Test i.e Test.class and Test$$anonfun$1.class. Test$$anonfun$1.class is anonymous class created for function in Scala.

Whatever we have talked about so far about FunctionN trait and anonymous class for every function values, this all happens before Scala version 2.12. But after Scala version 2.12, Scala Compiler does not create anonymous classes for every function. Instead, it uses factories to create dynamic classes on the fly. Jar becomes smaller and loads faster.

When you again compile the same file. You will notice there is only one class file i.e Test.class. Let see the bytecode generated,

javap -c Test.class
Compiled from "Test.scala"
public class Test {
public boolean isEven(int);
Code:
0: iload_1
1: iconst_2
2: irem
3: iconst_0
4: if_icmpne 11
7: iconst_1
8: goto 12
11: iconst_0
12: ireturn

public scala.Function1<java.lang.Object, java.lang.Object> isEvn();
Code:
0: aload_0
1: getfield #25 // Field isEvn:Lscala/Function1;
4: areturn

public static final boolean $anonfun$isEvn$1(int);
Code:
0: iload_0
1: iconst_2
2: irem
3: iconst_0
4: if_icmpne 11
7: iconst_1
8: goto 12
11: iconst_0
12: ireturn

public Test();
Code:
0: aload_0
1: invokespecial #31 // Method java/lang/Object."":()V
4: aload_0
5: invokedynamic #50, 0 // InvokeDynamic #0:apply$mcZI$sp:()Lscala/runtime/java8/JFunction1$mcZI$sp;
10: putfield #25 // Field isEvn:Lscala/Function1;
13: return
}

Notice Compiler has smartly defined $anonfun$isEvn$1 as a static method under the hood so that it could be called without an object. And in Test() what is invokedynamic call?

invokedynamic is used to generate code at runtime. Basically, lambdas are not invoked using invokedynamic. Their object representation is created using invokedynamic, the actual invocation is a regular call either invokevirtual or invokeinterface or invokestatic.

Let’s back to our main aim of eta expansion. We have seen how Scala Compiler does automatic eta expansion. There is a situation where the compiler is not expecting a function so it won’t do automatic eta expansion.

For example, creating a method and trying to store it in Map

 scala> def double(x: Int) = x * 2
 double: (x: Int)Int

scala> Map("2x" -> double)
 :13: error: missing argument list for method double
 Unapplied methods are only converted to functions when a function type is expected.
 You can make this conversion explicit by writing `double _` or `double(_)` instead of `double`.
 Map("2x" -> double)
 ^

Here Compiler has thrown an error because we have method and compiler is not expecting a function so there is no automatic eta expansion.

So there are two ways of explicitly doing eta expansion as follows:

1. Explicitly declare the type of value to be a function.
2. Treat the method to be as partially applied function

scala> val doubleFunction: Int => Int = double
doubleFunction: Int => Int = $$Lambda$1353/1147731368@5fc1e4fb

scala> val doublePartial = double _
doublePartial: Int => Int = $$Lambda$1354/1471558227@c0013b8

Notice both ways have converted a method into a function. Now we try to store in Map, it won’t throw an error.

scala> Map("2x" -> doublePartial)
res5: scala.collection.immutable.Map[String,Int => Int] = Map(2x -> $$Lambda$1460/2076486718@504274c1)

That’s all for now. We have seen that methods and functions are not the same things in Scala. How Scala compiler smartly does converts the method into a function using eta expansion. And techniques to do manually eta expansion if Scala compiler won’t do automatically.

Please feel free to suggest and comment.

References:

1. Scala Functions vs Methods
2. Methods as functions
3. Methods are not Functions