Difference between revisions of "Creating bindings for C libraries/fr"
Paulatreides (talk | contribs) |
Paulatreides (talk | contribs) |
||
Line 191: | Line 191: | ||
Fix: Add '''Remove empty type/var/const sections''' to the '''After h2pas''' tools. | Fix: Add '''Remove empty type/var/const sections''' to the '''After h2pas''' tools. | ||
+ | |||
+ | ==h2pas forgets to add IFDEFs for function bodies== | ||
+ | |||
+ | h2pas converts macros to functions. But if the macro was in an IFDEF block, h2pas does not add IFDEFs around the function body. | ||
+ | |||
+ | Fix: Add the tool '''Add missing h2pas IFDEFs for function bodies'''. Add this right in front of the tool '''Reduce compiler directives in pascal file'''. | ||
==Implicit Types== | ==Implicit Types== | ||
Line 310: | Line 316: | ||
==The pascal file contains a lot of unneccessary ifdef== | ==The pascal file contains a lot of unneccessary ifdef== | ||
− | h2pas translates even the C #ifdef directives. Many of them are C specific and not needed under FPC and can even make it impossible many tools on this side to explore the code. That's why there is a tool to clean up and disable or remove many unneeded IFDEFs: '''Reduce compiler directives in pascal file'''. | + | h2pas translates even the C #ifdef directives. Many of them are C specific and not needed under FPC and can even make it impossible many tools on this side to explore the code. That's why there is a tool to clean up and disable or remove many unneeded IFDEFs: '''Reduce compiler directives in pascal file'''. Do not forget to insert the tool '''Add missing h2pas IFDEFs for function bodies''' in front of it. |
+ | |||
+ | ===Hints about unneeded directives ($IFDEFs, $DEFINEs, ...)=== | ||
+ | |||
+ | You can see the directives in the [[IDE Window: Code Explorer|Code Explorer]]. | ||
+ | |||
+ | The tool will only remove the $IFDEFs and $DEFINEs, that are unneeded on all platforms. That means for example a {$IFDEF CPU386} will not be removed - even if you are currently working on an intel 32 bit compatible machine it will not be removed. This allows to create platform independent bindings. Of course C headers often contain a lot of IFDEFs not needed under FPC (Delphi, Kylix, gpc, ...). | ||
+ | |||
+ | For example: Many C headers are enclosed in | ||
+ | {$IFNDEF FOO} | ||
+ | {$DEFINE FOO} | ||
+ | ... | ||
+ | {$ENDIF} | ||
+ | |||
+ | This trick allows to include a C header file multiple times - like the units under pascal. So, pascal does not need this. Therefore the tool ''Reduce compiler directives in pascal file'' has two properties: ''Defines'' and ''Undefines''. Click on the ''...'' button of the property to edit them. Each line can contain a macroname, that should be (un)defined. Add the line ''FOO'' to the ''Undefines''. This way the '''$IFNDEF FOO''' is always true and the tool will remove it. The '''$DEFINE FOO''' is now unneeded too and will be removed too. | ||
+ | |||
+ | ====Example: MPICH2==== | ||
+ | Add the following to the ''Undefines'' property: | ||
+ | MPI_INCLUDED | ||
+ | MPIO_INCLUDE | ||
+ | NEEDS_MPI_FINT | ||
+ | MPI_BUILD_PROFILING | ||
+ | MPICH_SUPPRESS_PROTOTYPES | ||
+ | MPI_Wtime | ||
+ | HAVE_MPICH2G2 | ||
+ | FOO | ||
+ | HAVE_PRAGMA_HP_SEC_DEF | ||
+ | |||
+ | |||
+ | Add the following to the ''Defines'' property: | ||
+ | MPICH2 | ||
+ | MPICH_NEW_MPIIO | ||
==The C header files contain redefinitions== | ==The C header files contain redefinitions== | ||
Line 453: | Line 490: | ||
*a high lvl tool to gather many tools into one. | *a high lvl tool to gather many tools into one. | ||
− | |||
*h2pas converts macros to simple "constant A=B;" statements. A tool is needed to fix the cases where B is a type, var or function. | *h2pas converts macros to simple "constant A=B;" statements. A tool is needed to fix the cases where B is a type, var or function. | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
*A tool to reorder a unit to fix forward definitions. This requires a tool to remove unneeded IFDEFs. | *A tool to reorder a unit to fix forward definitions. This requires a tool to remove unneeded IFDEFs. | ||
*A tool to fix functions without parameter names by searching the .c files. | *A tool to fix functions without parameter names by searching the .c files. | ||
*Find missing identifiers and let the user check which one to comment or replace with base types. | *Find missing identifiers and let the user check which one to comment or replace with base types. | ||
− | *Create a list of | + | *Create a list of macro functions, which were half translated. |
Revision as of 17:06, 14 August 2007
│
English (en) │
español (es) │
français (fr) │
日本語 (ja) │
русский (ru) │
Overview
This page describes how to create pascal bindings for C libraries. Normally Pascal can not use C libraries directly. You have to create a pascal translation for every C function, type and variable. There is the H2Pas tool, which can automatically translate many common C things. And there is a GUI for Lazarus using h2pas and other tools to automate the creation. And the GUI helps to create a rule set, that can be used to update the bindings. So the next version of the c library can be converted far more easier. A nice feature of the h2pas wizard is, that it automatically use temporary files, so the c header files will never be changed automatically.
Work flow
- Fetch the C header files you want to translate.
- Create a working directory and give your bindings a name
- Create a new h2pas project with the h2pas wizard.
- add the .h files to the project
- setup the h2pas options
- run the wizard
- fix errors by adding text tools and run the wizard again
- when h2pas runs without errors, test compilation and add optional tools to beautify the output
- write some test programs to test your bindings
- publish your bindings on lazarus-ccr or the Free Pascal website
Install tools
The h2pas tool comes with every normal fpc installation.
Install the h2paswizard package in the Lazarus IDE. Go to "Components -> Configure installed packages ...", select from the right list the H2PasWizard package and click 'Install selection', then 'Save and rebuild IDE'. Restart the IDE and you get a new menu entry: Tools -> h2pas
Fetch C header files
The C header files .h describes the interface of a C library. They are normally not provided with the library. You have to get the sources or the development package of the library. For example the C header files of the gtk libraries are in the package gtk+-devel.
Example: MPICH2
Download the mpich2-1.0.3.tar.gz from http://www-unix.mcs.anl.gov/mpi/mpich2/ and unpack them. The .h files are under mpich2-1.0.3/src/include.
Create a working directory and name your bindings
Create a directory with a useful name. A name should not contain special characters, like spaces, umlaute, points or commas. Copy the .h files.
Example: MPICH2
The h2p directory will be used for the pascal files. The h2p/c_sources directory will be used for the .h files.
mkdir -p h2p/c_sources cp mpich2-1.0.3/src/include/*.h h2p/c_sources/
Create a new h2pas project with the h2pas wizard
Open the h2pas wizard: "Tools -> h2pas". This will open a window. You can switch between this window and the other IDE windows. Your last h2pas project will be loaded automatically. To create a new project click on "Settings -> New/Clear settings". Then click on the bottom button "Save settings" and choose a filename.
Example: MPICH2
Click on "Settings -> New/Clear settings". Then click on the bottom button "Save settings" and save it as h2p/mpi2.h2p.
Add the .h files to the project
In the "C header files" page you can add/delete the .h files. You can enable/disable .h files to convert only part of the files.
Example: MPICH2
Click on "C header files -> Add .h files ..." and select "mpi.h". It will be enabled automatically.
Setup the h2pas options
Under "h2pas Options" you can set the parameters for the h2pas program.
Example: MPICH2
- Enable -e, -D, -p, -w and disable all others.
- The -l library path is "mpich".
- Output extension is ".pas"
- Output directory is h2pas/, which is the default, so leave it empty
Run the wizard
Click on the bottom button "Run h2pas". This will copy the <example>.h file to a temporary <example>.tmp.h file and run the tools listed under "Before h2pas". Then it runs h2pas to convert the <example>.tmp.h into <example>.inc or <example>.pas or whatever output extension you setup on the h2pas page.
If h2pas finds a syntax error, the IDE will open the example.tmp.h file and jumps to the error line. h2pas often only reports 'syntax error', which is very unspecific. See below for a list of common problems and possible fixes.
Example: MPICH2
The h2pas wizard already contains all tools to convert all the specials of this header file, so h2pas runs without errors. But the created unit is not yet ready. Read further.
h2pas reports an error
Here is a list of common C structures, not recognized by h2pas and how to fix them:
h2pas problem: extern "C"
Some header files contain the C++ namespace block:
#if defined(__cplusplus) extern "C" { #endif ... #if defined(__cplusplus) } #endif
Fix: Add the Remove C++ 'extern "C"' lines to the before h2pas tools.
h2pas problem: Empty macro
Some header files contain empty macros used for further extensions:
#define MPI_FILE_DEFINED
Fix: Add the Remove empty C macros to the before h2pas tools.
h2pas problem: Implicit array types
C allows implicit arrays in parameter types. For example:
int MPI_Group_range_incl(MPI_Group, int, int [][3], MPI_Group *);
The int [][3] is an implicit type, which is not allowed under Pascal. h2pas supports adding pointer types. So, it enough to replace all [] with *.
Fix: Add the Replace [] with * to the before h2pas tools.
h2pas problem: Makros for 0 pointers
Some header files contain typed 0 pointers:
#define MPI_BOTTOM (void *)0
Fix: Add the Replace macro values 0 pointer like (char *)0 with NULL to the before h2pas tools.
Test compilation and add tools to beautify the output
When h2pas runs without errors it has created a pascal file. The -i switch defines if it is a unit or an include file. The next step is test compilation. Setup up a test project that uses the new pascal source. Then use the 'Run h2pas and compile' button in the h2pas wizard.
Example: MPICH2
Create a new project with Project -> New Project -> Program. Save it a mpihelloworld. Change the code to
program MPIHelloWorld; {$mode objfpc}{$H+} {$linklib mpich} {$linklib rt} uses {$IFDEF UNIX}pthreads{$ENDIF}, MPI; begin MPI_Init(@argc, @argv); writeln('Hello, world.'); MPI_Finalize; end.
Add the h2p/mpi.pas to the project. Project -> Add editor file to project.
Common compiler errors on h2pas output
Sometimes h2pas does not create valid pascal code. Here are the common problems and how to fix them.
Unit name contains the file path
h2pas sometimes add the whole path to the unitname.
Fix: Add Replace "unit filename;" with "unit name;" to the After h2pas tools.
Missing include files
h2pas converts #include directives to pascal {$i } directives. If you create a unit for wach header file, then you can remove all include directives with the tool Remove includes.
Forward type not resolved
For example: mpi.pas(26,16) Error: Forward type not resolved "PMPI_Aint" The error line is often a pointer type to a record:
PMPI_Aint = ^MPI_Aint;
h2pas adds PMPI_Aint to give nameless C pointers like *MPI_Aint a name. It adds the pointers to the start of the file. But pascal requires, that the forward definition is in the same type section. Sometimes h2pas adds a pointer type, although it already exists.
Fix: Add the tool Remove redefined pointer types to the After h2pas tools.
Removing system types
h2pas adds some system types, like PLongint, which are nowadays part of the system unit.
Fix: Add Remove type redefinitons like PLongint to the After h2pas tools.
Empty type/var/const sections
After the above tools removed some variables, types, constants some setions become empty, which is not allowed in pascal.
Fix: Add Remove empty type/var/const sections to the After h2pas tools.
h2pas forgets to add IFDEFs for function bodies
h2pas converts macros to functions. But if the macro was in an IFDEF block, h2pas does not add IFDEFs around the function body.
Fix: Add the tool Add missing h2pas IFDEFs for function bodies. Add this right in front of the tool Reduce compiler directives in pascal file.
Implicit Types
C allows implicits types in parameters. For example: int [][3]. Pascal does not allow this, so you must give it a name and declare a type. But h2pas does not do this automatically and adds instead the implicit type in a quasi pascal syntax. For example:
int some_func(int [][3]);
becomes
function some_func(_para1:array[0..2] of Plongint):longint;cdecl;external;
Fix: Luckily there is now a tool to remove these implicit types and add explicit types, it's called Replace implicit types. Add it to the after tool list.
Array of nothing
Sometimes h2pas converts the C ellipsis parameter type '...' wrongly to 'array of )'. It should be 'array of const)'.
Fix: Add the tool Fix open arrays to the after tools.
Identifier not found
There are three cases:
- The identifier is defined in the unit, but after it is used
- The identifier is defined in another unit (another .h file).
- The identifier is not defined anywhere.
The identifier is defined in the unit, but after it is used
In pascal forward defined types are only allowed in a type section. For example:
type TMyRecord = record Next: PMyRecord; // using the forward definition end; PMyRecord = ^TMyRecord;
The below is not allowed, because the forward definition is in another type section:
type TMyRecord = record Next: PMyRecord; // using the forward definition end; type PMyRecord = ^TMyRecord;
Solution: The code must be reordered. There is no tool yet, to do this automatically.
The identifier is defined in another unit (another .h file)
Solution 1: Add the unit to the uses section
If the other unit is already using this unit, then you have a circle. A circle dependency is allowed between .h files, but not between pascal units. In this case you must move code between both units or use IFDEFs like the gtk2 bindings or use the below merge function of the h2pas wizard.
Solution 2: Merge the two include files
The h2pas wizard can merge include files into one. For example first.h includes via the c include directive the file second.h. Select the second.h in the wizard and check the 'Merge file' feature. Then the source of second.h' will be appended to first.h before sending the output to h2pas. As always: This not change the header files.
Proposal
A tool to fix most of the above issues could work this way: Gather all constants of all units of the bindings and put them into one big constant section. Do the same for all types and put them all into one big type section behind the constant section.
Disadvantage: You can no longer see, which identifier belongs to which .h file. At least some comments should be added.
Example: MPICH2
The type MPI_Request is defined in mpi.h and used in mpio.h. Both c header files heavily use each other, so you can not put them into two separated units. Select mpio.h and enable the 'Merge file' function.
The identifier is not defined anywhere
Probably you are missing a header file or you are using one with the wrong version or it is private identifier.
Solution: Search the header file and convert it too. If you can not find it or you don't want to translate this file, you can comment the identifier or replace it.
Example: MPICH2
The mpi.h file defines MPI_File as pointer of ADIOI_FileD, but ADIOI_FileD is not defined public:
typedef struct ADIOI_FileD *MPI_File;
The h2pas tool translated this to:
MPI_File = ^ADIOI_FileD;
Because ADIOI_FileD is not defined public, it is a private structure, so you can simply replace it with a Pointer: Add to the 'After h2pas' tools a new tool of type 'Search and replace' with the following properties:
Property | Value |
---|---|
Caption | Replace ADIOI_FileD with Pointer |
Name | ReplaceADIOI_FileDwithPointer |
SearchFor | ADIOI_FileD |
ReplaceWith | Pointer |
Options | [trtMatchCase,trtWholeWord] |
Illegal expression
Example: MPICH2
Compiler gives Illegal expression on the following statement
const MPI_LONG_LONG = MPI_LONG_LONG_INT;
The reason is, that h2pas translated the MPI_LONG_LONG_INT constant to a function.
Solution: Fix h2pas or use a trick:
Replace the
#define MPI_LONG_LONG MPI_LONG_LONG_INT
with
#define MPI_LONG_LONG ((MPI_Datatype)0x4c000809)
by adding a Search and replace tool before h2pas.
Common other issues
The pascal file contains a lot of unneccessary ifdef
h2pas translates even the C #ifdef directives. Many of them are C specific and not needed under FPC and can even make it impossible many tools on this side to explore the code. That's why there is a tool to clean up and disable or remove many unneeded IFDEFs: Reduce compiler directives in pascal file. Do not forget to insert the tool Add missing h2pas IFDEFs for function bodies in front of it.
Hints about unneeded directives ($IFDEFs, $DEFINEs, ...)
You can see the directives in the Code Explorer.
The tool will only remove the $IFDEFs and $DEFINEs, that are unneeded on all platforms. That means for example a {$IFDEF CPU386} will not be removed - even if you are currently working on an intel 32 bit compatible machine it will not be removed. This allows to create platform independent bindings. Of course C headers often contain a lot of IFDEFs not needed under FPC (Delphi, Kylix, gpc, ...).
For example: Many C headers are enclosed in
{$IFNDEF FOO} {$DEFINE FOO} ... {$ENDIF}
This trick allows to include a C header file multiple times - like the units under pascal. So, pascal does not need this. Therefore the tool Reduce compiler directives in pascal file has two properties: Defines and Undefines. Click on the ... button of the property to edit them. Each line can contain a macroname, that should be (un)defined. Add the line FOO to the Undefines. This way the $IFNDEF FOO is always true and the tool will remove it. The $DEFINE FOO is now unneeded too and will be removed too.
Example: MPICH2
Add the following to the Undefines property:
MPI_INCLUDED MPIO_INCLUDE NEEDS_MPI_FINT MPI_BUILD_PROFILING MPICH_SUPPRESS_PROTOTYPES MPI_Wtime HAVE_MPICH2G2 FOO HAVE_PRAGMA_HP_SEC_DEF
Add the following to the Defines property:
MPICH2 MPICH_NEW_MPIIO
The C header files contain redefinitions
This is ok for C compilers, because a type is the same if its definition is the same. But for Pascal each type is a different type.
Solution: Add the tool Remove redefinitions in pascal unit to the after h2pas tools. As the name implies, it only works on units, not on include files. And it expect at least a valid pascal syntax. That's why it is important to add this tool after the tools that fix the pascal syntax like Replace "unit filename;" with "unit name;", Remove empty type/var/const sections, Remove includes, Replace implicit types and Fix open arrays.
Example: MPICH2
The mpi.h contains several redefinitions like MPIIMPL_HAVE_MPI_TYPE_CREATE_DARRAY. They can all be fixed with this tool.
Alias macros are always translated by h2pas to constants
h2pas converts macros like
#define A B
to
const A = B;
which is almost always correct. But some c header contain aliases for types, variables and functions too.
Example: MPICH2
The mpi.h contains the macro
#define MPI_LONG_LONG MPI_LONG_LONG_INT
h2pas converts this simply to
const MPI_LONG_LONG = MPI_LONG_LONG_INT;
but MPI_LONG_LONG_INT is a function.
ToDo: Write a tool to auto fix aliases.
h2pas creates functions for constants
For example:
function MPI_2COMPLEX : MPI_Datatype; begin MPI_2COMPLEX:=MPI_Datatype(MPI_DATATYPE_NULL); end;
All used identifiers are types and constants, so this function can be replaced with a simple constant. This is checked and done by the tool Replace simple functions with constants.
The C header files do not contain parameter names
For example there is an ugly function without parameter names:
int MPI_Intercomm_create(MPI_Comm, int, MPI_Comm, int, int, MPI_Comm * );
The parameter names are defined in the corresponding .c file:
int MPI_Intercomm_create(MPI_Comm local_comm, int local_leader, MPI_Comm peer_comm, int remote_leader, int tag, MPI_Comm *newintercomm)
Solution: There is no solution yet. Except manual editing or searching for other .h files. Often such .h files were auto generated and there are some nicer header files somewhere on the internet.
Proposal: Write a clever tool, that searches functions with missing parameter names and the corresponding functions in the .c files with parameter names and improve the .h file.
Publish your bindings on lazarus-ccr or Free Pascal
ToDo
Writing your own converter tools
Using the tool "Search and replace"
Many things like renaming a variable can be done by the Search and replace tool. Add the tool via the Add new tool button on either the 'Before h2pas' or the 'After h2pas' page. Then set the SearchFor, ReplaceWith, Options and Caption property.
Example: rename an identifier Tguint to guint
Property | Value |
---|---|
Caption | Rename Tguint to guint |
SearchFor | Tguint |
ReplaceWith | guint |
Options | [trtMatchCase,trtWholeWord] |
Example: rename several identifiers
Rename Tguint to guint, Tgulong to gulong, Tgint to gint:
Property | Value |
---|---|
Caption | Rename Tguint to guint |
SearchFor | gint|gulong) |
ReplaceWith | $1 |
Options | [trtMatchCase,trtWholeWord,trtRegExpr] |
Improving an existing tool
You found a bug and want to fix it or you want to extend one of the above tools. Great!
Most of the above tools are defined in the h2pasconvert unit, which is part of the h2paswizard package. Basically a tool needs a classname, a description and an Execute method. To test/debug a tool outside the IDE and save a lot of compilation time, see the project components/simpleideintf/examples/testh2pastool.lpi. Compile it and start it on a console/terminal with the filename of a .h file as first command parameter. For example:
./testh2pastool files/h2pastest.pas
Writing a custom tool
You can write your own conversion tools and register them in the IDE easily. Start a package and a add a new unit (say unit1) for your new class. See the existing tools as example and read the prior section. When you wrote a new tool and tested it with the above simpleideintf project, then register it in the IDE: Add a register procedure to your unit (unit1), like the following (pseudo code):
uses Classes, ..., IDETextConverter; type TYourToolClass = class(TCustomTextConverterTool) public class function ClassDescription: string; override; function Execute(aText: TIDETextConverter): TModalResult; override; end; procedure Register; implementation procedure Register; begin TextConverterToolClasses.RegisterClass(TYourToolClass); end;
Do not forget to enable the register checkbox of the unit in the package editor, otherwise the Register procedure will not be called by the IDE. Then install your package in the IDE.
Future work / Missing stuff
- a high lvl tool to gather many tools into one.
- h2pas converts macros to simple "constant A=B;" statements. A tool is needed to fix the cases where B is a type, var or function.
- A tool to reorder a unit to fix forward definitions. This requires a tool to remove unneeded IFDEFs.
- A tool to fix functions without parameter names by searching the .c files.
- Find missing identifiers and let the user check which one to comment or replace with base types.
- Create a list of macro functions, which were half translated.