FPC JVM/Building

From Lazarus wiki
Revision as of 14:12, 27 September 2012 by Jonas (talk | contribs) (→‎Building the compiler and RTL: fixed typo)
Jump to navigationJump to search

Building the JVM compiler

The instructions below are for Unix-like platforms, but by changing the forward slashes to backslashes (and adding .exe after the compiler binary name when copying it) most if not all commands will work on Windows too.

Every sequence of commands below starts from the assumption that you are in the top-level FPC source directory.

Prerequisites

A complete and working installation of FPC 2.6.0, as well as an installation of the JDK 1.5 or later (with the binaries of both in the path).

Obtaining the FPC sources

The FPC JVM backend is now part of svn trunk. To check it out using svn:

 svn co http://svn.freepascal.org/svn/fpc/trunk fpc

Building the compiler and RTL

Note: all of the commands below require that the latest native FPC release for your platform is in the PATH.

First you have to build a version of the fpcmake program that knows about the JVM target:

 cd fpc/utils/fpcm
 make clean all

Afterwards, move the generated fpcmake(.exe) somewhere outside the fpc sources directory.

Execute the following command in the top level directory of the checked out sources, i.e. in the fpc directory (replace FullPathToInstallLocation with the directory where to install the result, and FullPathToFpcmakeCompiledAbove with the directory where you put the fpcmake(.exe) you compiled above):

make FPCMAKE=FullPathToFpcmakeCompiledAbove/fpcmake(.exe) CROSSOPT="-O2 -g" CPU_TARGET=jvm OS_TARGET=java ALLOW_WARNINGS=1 INSTALL_PREFIX=FullPathToInstallLocation all crossinstall

If you wish to also create a run time library that can be used for Android development, use the following command (replace FullPathToInstallLocation with the directory where to install the result):

make FPCMAKE=FullPathToFpcmakeCompiledAbove/fpcmake(.exe) CROSSOPT="-O2 -g" CPU_TARGET=jvm OS_TARGET=android ALLOW_WARNINGS=1 INSTALL_PREFIX=FullPathToInstallLocation all crossinstall

The generated compiler is called ppcjvm/ppcjvm.exe.

In both cases, a ppcjvm (or ppcjvm.exe) binary will be created in the compiler directory. Copy it somewhere in your PATH. The compiler generated by both build processes is the same, so you only need one of them.

When compiling programs using this compiler, make sure to specify the correct -Fu parameter to

Building the fpcjres utility

This step is optional, and is only required for the raw {$r xxx} support. Run the following command in the top level directory of the checked out sources:

make all

The compiled fpcjres utility will be located in utils/fpcres.

Additional tools

Building javapp

The tool use to generate Pascal import headers from Java classes is called javapp. It is written in Java, because it is based on the javap utility from the JDK (version 6).

The source code for javapp can be found under utils/javapp in the FPC source tree. To build it, create a new Eclipse project and import the sources. Then add jgrapht.jar from http://www.cs.sunysb.edu/~algorith/implement/jgrapht/distrib/ to the project (used for topological sorting in order to determine the dependencies between the classes). The main class file is Main.java.

Creating the Java RTL headers

To create the rtl/java/java_sysh.inc and rtl/java/java_sys.inc files, run javapp as follows:

 java -jar javapp.jar -protected java.lang.System java.lang.Object java.io.Serializable java.io.IOException java.io.IIOException java.lang.Readable java.lang.Float java.lang.Double java.lang.Byte java.lang.Short java.lang.Integer java.lang.Long java.lang.Boolean java.lang.String java.lang.StringBuilder java.lang.StringBuffer java.lang.CharSequence java.lang.AbstractStringBuilder java.lang.Appendable java.lang.Number java.lang.IndexOutOfBoundsException java.lang.RuntimeException java.lang.IllegalArgumentException java.lang.IllegalStateException java.lang.UnsupportedOperationException java.lang.Exception java.lang.Throwable java.lang.NoSuchMethodException java.lang.LinkageError java.lang.Error java.lang.Comparable java.lang.Character java.text.Collator java.lang.Cloneable java.lang.Runtime java.lang.ThreadLocal java.math.BigInteger java.util.Comparator java.lang.reflect.Array java.lang.Class java.lang.reflect.GenericDeclaration java.lang.reflect.Type java.lang.reflect.Method java.lang.reflect.AccessibleObject java.lang.reflect.Member java.lang.reflect.AnnotatedElement java.lang.reflect.InvocationTargetException java.lang.reflect.Field java.util.Arrays java.util.Map java.util.HashMap java.util.Set java.util.Collection java.util.AbstractMap java.util.Calendar java.lang.Iterable java.lang.Enum  java.lang.Math java.util.AbstractCollection java.util.AbstractSet java.util.EnumSet java.util.BitSet java.util.Iterator java.nio.Buffer java.nio.ByteBuffer java.nio.CharBuffer java.nio.charset. -i -o java_sys

To create the rtl/java/jdk15.pas and rtl/java/jdk15.inc files, use

 java -jar javapp.jar -x java.lang.System -x java.lang.Object -x java.io.Serializable -x java.io.IOException -x java.io.IIOException -x java.lang.Readable -x java.lang.Float -x java.lang.Double -x java.lang.Float -x java.lang.Double -x java.lang.Byte -x java.lang.Short -x java.lang.Integer -x java.lang.Long -x java.lang.Boolean -x java.lang.String -x java.lang.StringBuilder -x java.lang.StringBuffer -x java.lang.CharSequence -x java.lang.AbstractStringBuilder -x java.lang.Appendable -x java.lang.Number -x java.lang.IndexOutOfBoundsException -x java.lang.RuntimeException -x java.lang.IllegalArgumentException -x java.lang.IllegalStateException -x java.lang.UnsupportedOperationException -x java.lang.Exception -x java.lang.Throwable -x java.lang.LinkageError -x java.lang.Error -x java.lang.Comparable -x java.lang.Character  -x java.text.Collator -x java.lang.Cloneable -x java.lang.Runtime -x java.lang.ThreadLocal -x java.math.BigInteger -x java.util.Comparator -x java.lang.reflect.Array -x java.lang.Class -x java.lang.reflect.GenericDeclaration -x java.lang.reflect.Type -x java.lang.reflect.Method -x java.lang.reflect.AccessibleObject -x java.lang.reflect.Member -x java.lang.reflect.AnnotatedElement -x java.lang.reflect.Field -x java.lang.reflect.InvocationTargetException -x java.util.Arrays -x java.util.Map -x java.util.HashMap -x java.util.Set -x java.util.Collection -x java.util.AbstractMap -x java.util.Calendar -x java.lang.Iterable -x java.lang.Enum -x java.lang.Math -x java.util.AbstractCollection -x java.util.AbstractSet -x java.util.EnumSet -x java.util.BitSet -x java.util.Iterator -x java.nio.Buffer -x java.nio.ByteBuffer -x java.nio.CharBuffer -x java.nio.charset. -a sun. -a com.sun. -a apple. -a java.awt.Dialog -protected java. javax. org. -o jdk15

Creating the Android RTL headers

To create the rtl/android/jvm/java_sysh_android.inc and rtl/android/jvm/java_sys_android.inc files, use (same command line as for creating the equivalent Java RTL includes, except for the additional bootclasspath and extdirs parameters)

 java -jar javapp.jar -bootclasspath /full/path/to/android-sdk/platforms/android-14/android.jar -extdirs . -protected java.lang.System java.lang.Object java.io.Serializable java.io.IOException java.io.IIOException java.lang.Readable java.lang.Float java.lang.Double java.lang.Byte java.lang.Short java.lang.Integer java.lang.Long java.lang.Boolean java.lang.String java.lang.StringBuilder java.lang.StringBuffer java.lang.CharSequence java.lang.AbstractStringBuilder java.lang.Appendable java.lang.Number java.lang.IndexOutOfBoundsException java.lang.RuntimeException java.lang.IllegalArgumentException java.lang.IllegalStateException java.lang.UnsupportedOperationException java.lang.Exception java.lang.Throwable java.lang.NoSuchMethodException java.lang.LinkageError java.lang.Error java.lang.Comparable java.lang.Character java.text.Collator java.lang.Cloneable java.lang.Runtime java.lang.ThreadLocal java.math.BigInteger java.util.Comparator java.lang.reflect.Array java.lang.Class java.lang.reflect.GenericDeclaration java.lang.reflect.Type java.lang.reflect.Method java.lang.reflect.AccessibleObject java.lang.reflect.Member java.lang.reflect.AnnotatedElement java.lang.reflect.InvocationTargetException java.lang.reflect.Field java.util.Arrays java.util.Map java.util.HashMap java.util.Set java.util.Collection java.util.AbstractMap java.util.Calendar java.lang.Iterable java.lang.Enum  java.lang.Math java.util.AbstractCollection java.util.AbstractSet java.util.EnumSet java.util.BitSet java.util.Iterator java.nio.Buffer java.nio.ByteBuffer java.nio.CharBuffer java.nio.charset. -i -o java_sys

To create the rtl/android/jvm/androidr14.pas and rtl/android/jvm/androidr14.inc files, use

 java -jar javapp.jar -bootclasspath /full/path/to/android-sdk/platforms/android-14/android.jar -extdirs . -protected -x java.lang.System -x java.lang.Object -x java.io.Serializable -x java.io.IOException -x java.io.IIOException -x java.lang.Readable -x java.lang.Float -x java.lang.Double -x java.lang.Float -x java.lang.Double -x java.lang.Byte -x java.lang.Short -x java.lang.Integer -x java.lang.Long -x java.lang.Boolean -x java.lang.String -x java.lang.StringBuilder -x java.lang.StringBuffer -x java.lang.CharSequence -x java.lang.AbstractStringBuilder -x java.lang.Appendable -x java.lang.Number -x java.lang.IndexOutOfBoundsException -x java.lang.RuntimeException -x java.lang.IllegalArgumentException -x java.lang.IllegalStateException -x java.lang.UnsupportedOperationException -x java.lang.Exception -x java.lang.Throwable -x java.lang.LinkageError -x java.lang.Error -x java.lang.Comparable -x java.lang.Character  -x java.text.Collator -x java.lang.Cloneable -x java.lang.Runtime -x java.lang.ThreadLocal -x java.math.BigInteger -x java.util.Comparator -x java.lang.reflect.Array -x java.lang.Class -x java.lang.reflect.GenericDeclaration -x java.lang.reflect.Type -x java.lang.reflect.Method -x java.lang.reflect.AccessibleObject -x java.lang.reflect.Member -x java.lang.reflect.AnnotatedElement -x java.lang.reflect.Field -x java.lang.reflect.InvocationTargetException -x java.util.Arrays -x java.util.Map -x java.util.HashMap -x java.util.Set -x java.util.Collection -x java.util.AbstractMap -x java.util.Calendar -x java.lang.Iterable -x java.lang.Enum -x java.lang.Math -x java.util.AbstractCollection -x java.util.AbstractSet -x java.util.EnumSet -x java.util.BitSet -x java.util.Iterator -x java.nio.Buffer -x java.nio.ByteBuffer -x java.nio.CharBuffer -x java.nio.charset. -a sun. -a com.sun. -a apple. -a java.awt. java. javax. org. android. junit. -o androidr14

You will have to make several manual changes to the result before it compiles though:

  • there is a circular dependency involving the AVViewGroup.LayoutParams nested class, which results in javapp being unable to order all classes. You will have to manually extract that class and redeclare it as AVViewGroup_LayoutParams = class external 'android.view' name 'ViewGroup$LayoutParams' (JLObject), and then replace all references to the original class to this new class
  • there is a class with a constant field called CREATE, which results in javapp generating wrong names for its constructors (<init>_). Rename the constant to CREATE_ and the constructors to Create
  • The declaration for AAActionBar.InnerTab is missing, add it as InnerTab = class abstract external 'android.app' name 'Tab' end; inside AAActionBar

Creating interfaces for other Java classes

 java -jar full/path/to/javapp.jar -classpath full/path/to/bcel-6.0-SNAPSHOT.jar -protected -o bcel org.apache.bcel.

The long parameter lists above are due to the split of the classes between the system unit and the jdk15 unit. The extra -classpath parameter is required when dumping classes that are not part of the standard JDK, and has to point to the jar file(s) and/or directory/ies containing the class hierarchy you want to dump. If they are in the current directory, do not forget to explicitly pass "-classpath ."

Important:

  • If an identifier argument to javapp ends in a ".", it is interpreted as a package name, otherwise as a class name.
  • In case circular dependencies of inner classes are involved, you will have to sacrifice one of the classes involved in the cycle by passing it using the -a option (anonymous). In that case, this class will be translated as a formal class (like java.awt.Dialog above, which is involved in a circular dependency with an inner class of java.awt.Window).
  • The -i option causes javapp to generate include files, otherwise it generates a standalone unit
  • Pascal does not support the concept of Java-style inner classes, but only that of static nested classes. As a result, Java inner classes are not added to the translated headers.

The javapp source code is unfortunately not very clean. The fact that the javap source code was not very clean to start with did not help, but the situation has not exactly improved...

External tools

Building Jasmin

Jasmin is the Java byte code assembler used by the compiler to translate the generated assembler files (*.j) into Java class files (*.class).

Building BCEL

BCEL is a Java byte code verification framework that can be used to look for errors in Java class files. See FPC_JVM/Debugging for usage information.

  • Check out the svn sources using the following command
 svn co http://svn.eu.apache.org/repos/asf/commons/proper/bcel/trunk bcel
  • Before building, apply the patch at ftp://ftp.freepascal.org/pub/fpc/contrib/jvm/bcel-r1157681-returntype.patch to prevent incorrect verification errors about return type mismatches.
  • To build BCEL, you first have to install Apache Maven
  • Once Maven is installed and its mvn binary is in your PATH, run mvn package in the bcel source directory. The first time you do so, Maven will download and configure a massive amount of plugins, which takes quite some time. After building, the new version of BCEL will be available as target/bcel-6.0-SNAPSHOT.jar