Web Service Toolkit

"Web Service Toolkit” is a web services package for FPC and Lazarus; “Web Service Toolkit” is meant to ease web services consumption and creation by FPC and Lazarus users.

Client Side ( service consumption )

Overview

“Web Service Toolkit” is made of two parts, a command line tool “ws_helper” and a collection of support units. Given an interface definition file( describing à web service ), “ws_helper” will create a FPC unit containing a proxy implementing that interface. At runtime when a call targeting the web service is issued, the proxy's role is to :

• marshall the call paramaters,
• make the call to the target web service,
• receive the call return and unmarshall output parameters to the caller.

Behind the scene, the proxy will take care of the SOAP plumbing details.

Example

We will use the “google web api”, freely available for personal use at this adress “http://www.google.com/apis/”.In order to use this service, we have to translate its exposed WSDL interface to Pascal langage. By now “Web Service Toolkit” does not contain a WSDL to pascal compiler, so we will assume this translation available as below ( this is an incomplete translation, but it's enough for the sample ).

  Unit googlewebapi;
  {$mode objfpc}{$H+}

  interface
  uses SysUtils, Classes;

  Type

    IGoogleSearch = Interface
      function doSpellingSuggestion(
        const key:string;
        const phrase:string
      ):string;
    End;

  Implementation
  End.

Invoking “ws_helper” at the prompt with the file “googlewebapi.pas” as argument will produce a file “googlewebapi_proxy.pas” as below.

  Unit googlewebapiimpunit;
  {$mode objfpc}{$H+}

  Interface
  Uses SysUtils, Classes, base_service_intf, service_intf, googlewebapi;

  Type

    TGoogleSearch_Proxy=class(TBaseProxy,IGoogleSearch)
    Protected
      function doSpellingSuggestion(
        Const key : string;
        Const phrase : string
      ):string;
    End;

  Implementation
  uses TypInfo;

  { TGoogleSearch_Proxy implementation }
  function TGoogleSearch_Proxy.doSpellingSuggestion(
    Const key : string;
    Const phrase : string
  ):string;
  Var
    locSerializer : IFormatterClient;
    strPrmName : string;
  Begin
    locSerializer := GetSerializer();
    Try
      locSerializer.BeginCall('doSpellingSuggestion', GetTarget());
        locSerializer.Put('key', TypeInfo(string), key);
        locSerializer.Put('phrase', TypeInfo(string), phrase);
      locSerializer.EndCall();

      MakeCall();

      locSerializer.BeginCallRead();
        strPrmName := 'return';
        locSerializer.Get(TypeInfo(string), strPrmName, result);

    Finally
      locSerializer.Clear();
    End;   
  End;

  End.

Then we can build a sample program for the “IGoogleSearch” service(see below). In order to compile this program you must have Indy ( tested with Indy10 ) which can be found at this location “http://www.indyproject.org/Sockets/index.en.iwp” or ICS( tested with the latest ICS-V5 Distribution ) at that adress “http://www.overbyte.be/frame_index.html” as it is used for the HTTP protocol.

  program test_google_api;
  {$mode objfpc}{$H+}

  uses
    Classes, SysUtils,
    base_service_intf, service_intf, soap_formatter,
    //indy_http_protocol,
    ics_http_protocol,
    googlewebapi, googlewebapi_proxy;

  Const
    sADRESS = 'http:Adress=http://api.google.com/search/beta2';
    sTARGET = 'urn:GoogleSearch';
    sKEY    = '<your google key here>';
    sSERVICE_PROTOCOL = 'SOAP';

  Var
    tmpObj : IGoogleSearch;
    strBuffer : string;
  begin
    ICS_RegisterHTTP_Transport();
    WriteLn();
    WriteLn('Enter phrase to spell :');
    ReadLn(strBuffer);
    tmpObj := TGoogleSearch_Proxy.Create(sTARGET,sSERVICE_PROTOCOL,sADRESS);
    Try
      strBuffer := tmpObj.doSpellingSuggestion(sKEY,strBuffer);
      WriteLn('google spell >>> ',strBuffer);
    Except
      On E : Exception Do
        WriteLn(E.Message);
    End;
    ReadLn();
  end.

The units base_service_intf, service_intf, soap_formatter, indy_http_protocol, ics_http_protocol as provided with this toolkit; Below is the result of a execution session spelling for “freepscal lzarus” written with a missing letter 'a' beetwen the letter 'p' and the the letter 's' and a missing letter 'a' beetwen the letter 'L' and the the letter 'z' .

 > .\tests\google_api\test_google_api.exe
 Enter phrase to spell :
 freepscal lzarus
 google spell >>> freepascal lazarus

Google spells it correctly : “freepascal lazarus”!

Server Side ( service creation )

Overview.

Web Service Toolkit contains a server side framework for service creation. Key features are:

• Service definition( interface ) is separated from implementation,
• Interface and implementations are not bound to message protocol,
• Support for SOAP 1.1 and a binary protocol,
• The framework is not bound to a transport protocol.
• Easy to add support for application servers

Example

In order to create a service, we have to :

• define its interface,
• provide an implementation and register that one for the service,
• provide a binder that will route calls targeting the service to the implementation and register that one,
• host the service into an application server( TCP server, HTTP Server, ... ).

Defining the service Interface

We will use the interface defined below for our sample. The complete projects of the example is located in the folder “\tests\tcp_server\calculator”.

unit calculator;

{$mode objfpc}{$H+}

interface

uses SysUtils, base_service_intf;

Type

  TBinaryArgsResult = class(TBaseComplexRemotable)
    private
    FArg_A: Integer;
    FArg_B: Integer;
    FArg_OP: string;
    FArg_R: Integer;
  Published
    Property Arg_A : Integer Read FArg_A Write FArg_A;
    Property Arg_B : Integer Read FArg_B Write FArg_B;
    Property Arg_R : Integer Read FArg_R Write FArg_R;
    Property Arg_OP : string Read FArg_OP Write FArg_OP;
  End;

  ICalculator = Interface
    function AddInt(Const A:Integer;Const B:Integer):TBinaryArgsResult;
    function DivInt(Const A:Integer;Const B:Integer):Integer;
  End;


implementation

uses base_soap_formatter;

Initialization
    GetTypeRegistry().Register(sXSD_NS,TypeInfo(TBinaryArgsResult),'TBinaryArgsResult');

end.

Providing an implementation for the service

“ws_helper” has options to generate proxy file, basic implementation skeleton file and a binder file ( see the following listing).

 ws_helper [-p] [-b] [-i] [-oPATH] inputFilename
   -p  Generate service proxy
   -b  Generate service binder
   -i  Generate service minimal implementation
   -o  PATH  Output directory

Executing “ws_helper” with the -i and -b options as below will produce two files : calculator_imp.pas and calculator_binder.pas.

 ws_helper\ws_helper.exe -i -b -osrv tests\tcp_server\calculator\calculator.pas
 ws_helper Copyright (c) 2006 by Inoussa OUEDRAOGO
 File "tests\tcp_server\calculator\calculator.pas" parsed succesfully.

The calculator_imp.pas unit contains a skeleton implementation class for the interface. It defines a procedure named RegisterCalculatorImplementationFactory. The procedure registers the class as the service implementation provider in the implementation registry.

  Unit calculator_imp;
  {$mode objfpc}{$H+}

  Interface
  Uses SysUtils, Classes, 
       base_service_intf, server_service_intf, server_service_imputils, calculator;

  Type

    TCalculator_ServiceImp=class(TSimpleFactoryItem,ICalculator)
    Protected
      function AddInt(Const A : Integer;Const B : Integer):TBinaryArgsResult;
      function DivInt(Const A : Integer;Const B : Integer):Integer;
    End;

    procedure RegisterCalculatorImplementationFactory();

  Implementation

  { TCalculator_ServiceImp implementation }

  function TCalculator_ServiceImp.AddInt( Const A : Integer; Const B : Integer):TBinaryArgsResult;
  Begin // ws_helper will generate empty methods, so we have to provide them
    Result := TBinaryArgsResult.Create();
    Try
      Result.Arg_OP := '+';
      Result.Arg_A := A;
      Result.Arg_B := B;
      Result.Arg_R := A + B;
    Except
      FreeAndNil(Result);
      Raise;
    End;
  End;

  function TCalculator_ServiceImp.DivInt( Const A : Integer; Const B : Integer):Integer;
  Begin // ws_helper will generate empty methods, so we have to provide them
    Result := A div B;
  End;

  procedure RegisterCalculatorImplementationFactory();
  Begin
    GetServiceImplementationRegistry().Register(
      'Calculator',
      TSimpleItemFactory.Create(TCalculator_ServiceImp) as IItemFactory
    );
  End;

  End.

Providing a binder for the service.

The binder's role is to:

• unpack the incoming message,
• set up the call stack,
• make the call against the registered implementation,
• serialize the execution stack to create the return message.

The calculator_binder.pas unit generated by ws_helper, contains :

• TCalculator_ServiceBinder : the actual binder class,
• TCalculator_ServiceBinderFactory a factory class for the binder and
• Server_service_RegisterCalculatorService : the binder factory registration procedure.

The following code extract shows the unit interface part and a method handler of the binder.

  Unit calculator_binder;
  {$mode objfpc}{$H+}

  Interface
  Uses SysUtils, Classes, base_service_intf, server_service_intf, calculator;

  Type

    TCalculator_ServiceBinder=class(TBaseServiceBinder)
    Protected
      procedure AddIntHandler(AFormatter:IFormatterResponse);
      procedure DivIntHandler(AFormatter:IFormatterResponse);
    Public
      constructor Create();
    End;

    TCalculator_ServiceBinderFactory = class(TInterfacedObject,IItemFactory)
    protected
      function CreateInstance():IInterface;
    End;

    procedure Server_service_RegisterCalculatorService();

  Implementation
  uses TypInfo;

  procedure TCalculator_ServiceBinder.AddIntHandler(AFormatter:IFormatterResponse);
  Var
    tmpObj : ICalculator;
    callCtx : ICallContext;
    strPrmName : string;
    procName,trgName : string;
    A : Integer;
    B : Integer;
    returnVal : TBinaryArgsResult;
    locTypeInfo : PTypeInfo;
  Begin
    callCtx := CreateCallContext();
    locTypeInfo := TypeInfo(TBinaryArgsResult);
    If ( locTypeInfo^.Kind in [tkClass,tkInterface] ) Then
      Pointer(returnVal) := Nil;
    strPrmName := 'A';  AFormatter.Get(TypeInfo(Integer),strPrmName,A);
    strPrmName := 'B';  AFormatter.Get(TypeInfo(Integer),strPrmName,B);
    tmpObj := Self.GetFactory().CreateInstance() as ICalculator;
    returnVal := tmpObj.AddInt(A,B);
    locTypeInfo := TypeInfo(TBinaryArgsResult);
    If ( locTypeInfo^.Kind = tkClass ) And Assigned(Pointer(returnVal)) Then
      callCtx.AddObject(TObject(returnVal));
    procName := AFormatter.GetCallProcedureName();
    trgName := AFormatter.GetCallTarget();
    AFormatter.Clear();
    AFormatter.BeginCallResponse(procName,trgName);
      AFormatter.Put('return',TypeInfo(TBinaryArgsResult),returnVal);
    AFormatter.EndCallResponse();
    callCtx := Nil;
  End;

Host the service into an application server.

The application server's role is to route incoming service requests to the Web Service Toolkit runtime. For the runtime to process service requests :

• The services and their implementations have to be registered ,
• The message protocol (SOAP, binary,...) have to be registered.

The runtime interface is defined in the server_service_intf unit. This unit contains :

• GetServerServiceRegistry, which returns the service registry,
• GetServiceImplementationRegistry which returns the service implementation registry,
• GetFormatterRegistry which returns the message format registry and
• HandleServiceRequest which is the unique entry point for request processing.

The toolkit is provided with a simple TCP server hosting the sample Calculator service defined early in this document located in the "\tests\tcp_server folder". The complete source files of the sample is in that directory and the client application in the "\client" sub folder. The registrations are done in the application's main form OnCreate event as printed below:

 procedure TfMain.FormCreate(Sender: TObject);
 begin
   Server_service_RegisterCalculatorService();
   RegisterCalculatorImplementationFactory();
   Server_service_RegisterSoapFormat();
   Server_service_RegisterBinaryFormat();
 end; 

Server_service_RegisterCalculatorService located in the calculator_binder unit ( generated by ws_helper ) registers the Calculator service by calling in turn GetServerServiceRegistry:

 procedure Server_service_RegisterCalculatorService();
 Begin
   GetServerServiceRegistry().Register(
     'Calculator',
     TCalculator_ServiceBinderFactory.Create() as IItemFactory
   );
 End;

RegisterCalculatorImplementationFactory located in the calculator_imp unit ( generated by ws_helper ) registers the Calculator implementation by calling in turn GetServiceImplementationRegistry:

 procedure RegisterCalculatorImplementationFactory();
 Begin
   GetServiceImplementationRegistry().Register(
     'Calculator',
     TSimpleItemFactory.Create(TCalculator_ServiceImp) as IitemFactory
   );
 End;

Server_service_RegisterSoapFormat located in the server_service_soap unit ( provided by the toolkit ) registers the SOAP implementation by calling in turn GetFormatterRegistry:

 procedure Server_service_RegisterSoapFormat();
 begin
   GetFormatterRegistry().Register(
     sSOAP_CONTENT_TYPE,
     TSimpleItemFactory.Create(TSOAPFormatter) as IitemFactory
   );
   RegisterStdTypes();
 end;

Server_service_RegisterBinaryFormat located in the server_binary_formatter unit ( provided by the toolkit ) registers the Binary message implementation by calling in turn GetFormatterRegistry:

 procedure Server_service_RegisterBinaryFormat();
 begin
   GetFormatterRegistry().Register(
     sCONTENT_TYPE,
     TBinaryFormatterFactory.Create() as IitemFactory
   );
 end;

HandleServiceRequest is invoked to process incoming request buffer in the socket client processing procedure ProcessData located in the server_unit unit.

 procedure TTcpSrvApp.ProcessData(Client : TTcpSrvClient);
 Var
   buff, trgt,ctntyp : string;
   rqst : IRequestBuffer;
   wrtr : IDataStore;
   rdr : IDataStoreReader;
   inStream, outStream, bufStream : TMemoryStream;
   i : Integer;
 begin
   inStream := Nil;
   outStream := Nil;
   bufStream := Nil;
   Try
     Client.RequestStream.Position := 0;
     Try
       inStream := TMemoryStream.Create();
       outStream := TMemoryStream.Create();
       bufStream := TMemoryStream.Create();
       rdr := CreateBinaryReader(Client.RequestStream);
       trgt := rdr.ReadStr();
       ctntyp := rdr.ReadStr();
       buff := rdr.ReadStr();
       inStream.Write(buff[1],Length(buff));
       inStream.Position := 0;
       rqst := TRequestBuffer.Create(trgt,ctntyp,inStream,bufStream);
       HandleServiceRequest(rqst);
       i := bufStream.Size;
       SetLength(buff,i);
       bufStream.Position := 0;
       bufStream.Read(buff[1],i);
       wrtr := CreateBinaryWriter(outStream);
       wrtr.WriteStr(buff);
       Client.Send(outStream.Memory,outStream.Size);
     Finally
       bufStream.Free();
       outStream.Free();
       inStream.Free();
       Client.FDataLentgh := -1;
       Client.RequestStream.Size := 0;
     End;
   Except
     On e : Exception Do
       Display('ProcessData()>> Exception = '+e.Message);
   End;
 end;

In order to give it a try one have to :

• compile the server,
• compile the client application,
• execute the server and start listening,
• execute the client.

WSDL generation

Services in the toolkit are organized into meta data repositories. Conceptually a repository corresponds :

• at compile time to the pascal unit containing the service definition
• at runtime to a name space.

The repository is the toolkit WSDL generation unit.

Services meta data

The ws_helper tool, when parsing the interface definition file, records the meta data of the services contained in the file to a Lazarus resource file. The resource file is then embedded into the generated binder's unit file( see the unit “initialization” part ). At runtime the service's recorded meta data are accessible through the interface IModuleMetadataMngr defined in the metadata_repository unit ( see below ). The GetModuleMetadataMngr function defined in the same unit returns a instance of an object supporting that interface.

  IModuleMetadataMngr = interface
    ['{B10ACF6A-A599-45A3-B083-BEEFB810C889}']
    function IndexOfName(const ARepName : shortstring):Integer;
    function GetCount():Integer;
    function GetRepositoryName(const AIndex : Integer):shortstring;
    procedure SetRepositoryNameSpace(const ARepName,ANameSpace : shortstring);
    function LoadRepositoryName(
      const ARepName,ARootAddress  : shortstring;
      out   ARepository  : PServiceRepository  ):Integer;
    procedure ClearRepository(var ARepository : PServiceRepository);
  end; 

The Metadata Service

The toolkit is provided with an easy to use metadata service implementation which in turn uses the raw interface defined in the metadata_repository unit (see above). A Lazarus GUI client application is located in the tests\metadata_browser folder.

WSDL generation API

The metadata_wsdl pascal unit contains the GenerateWSDL function for WSDL generation from a repository (see the signature below).

  PServiceRepository = ^TServiceRepository;
  TServiceRepository = record
    NameSpace        : ShortString;
    Name             : ShortString;
    RootAddress      : ShortString;
    ServicesCount    : Byte;
    Services         : PService;
  end;

  procedure GenerateWSDL(AMdtdRep : PServiceRepository; ADoc : TDOMDocument);

WSDL Customization

The WSDL generation is based on the IWsdlTypeHandler and the IWsdlTypeHandlerRegistry interfaces located in the metadata_wsdl unit. In order to customize the generated WSDL, one has to provide a class implementing the IWsdlTypeHandler interface. Then that class has to be registered in the registry. The metadata_wsdl unit contains implementations for pascal enumerations, TBaseComplexRemotable descendants, and TBaseArrayRemotable descendants.

Sample

A functional sample project is located under \tests\http_server . It is an Indy base http server.

Provided examples

The server side samples has been tested on Windows XP. The client side has been tested under Windows XP and Ubuntu. The samples are located under the “tests” folder.

Client side examples

• Google sample : It demonstrates use of class and array data types.
• Metadata Browser : This sample demonstrates use of class and array data types and mainly the toolkit metadata service.

Server side examples

• TCP server : This is a sample TCP server based on the ICS components. It uses the calculator service.
• HTTP server : This is a sample HTTP server based on the Indy10 components. It uses the calculator service and the toolkit metadata service. It demonstrates the WSDL generation.

Status

The toolkit is usable for simple types and for class types. The serialization is designed to allow customization of basic types and class types by implementing classes derived from “TBaseRemotable”. This classes have to be registered in the type registry.

Serialization

The serialization is based on the IFormatterBase interface located in the base_service_intf unit.

The toolkit has two serializers implementations : the SOAP serializer and a Binary serializer.

SOAP serializer

The SOAP serializer implements SOAP 1.1. It has support for the following pascal types:

• Available integers :
  • Byte mapped to unsignedByte
  • ShortInt mapped to byte
  • SmallInt mapped to short
  • Word mapped to unsignedShort
  • LongInt mapped to int
  • LongWord mapped to unsignedInt
  • Int64 mapped to long
  • QWord mapped to int

• String mapped to string
• Boolean mapped to boolean
• Enumerations mapped to their string representation

• Float types :
  • Single mapped to float
  • Double mapped to double
  • Extended mapped to double
  • Currency mapped to float

• Object (class intances, not TP ones ) : The toolkit has support for instances of classes derived from TBaseRemotable. TBaseRemotable is the base class used by the formatter interface to allow customization of the serialization. The toolkit provides the TBaseComplexRemotable class which implements serialization for its ( or its descendants ) published properties.

Binary serializer

The Binary serializer is more efficient in time and space compared to the SOAP serializer. It uses big endian to stream data. It has support for the following pascal types:

• Available integers :
  • Byte
  • ShortInt
  • SmallInt
  • Word
  • LongInt
  • LongWord
  • Int64
  • QWord

• String
• Boolean
• Enumerations

• Float types :
  • Single
  • Double
  • Extended
  • Currency

• Object (class intances, not TP ones ) :The toolkit has support for instances of classes derived from TBaseRemotable. TBaseRemotable is the base class used by the formatter interface to allow customization of the serialization. The toolkit provides the TBaseComplexRemotable class which implements serialization for its ( or its descendants ) published properties.

Class type serialization

The toolkit has support for instances of classes derived from TBaseRemotable. TBaseRemotable is the abstract base class used by the formatter interface to allow customization of the serialization. The toolkit provides the TBaseComplexRemotable class which implements serialization for its descendants classes published properties. It also provides TBaseObjectArrayRemotable class for serialization of array of TBaseRemotable descendant classes.

The root “TBaseRemotable” class

  TBaseRemotable = class(TPersistent)
  Public
    constructor Create();virtual;
    class procedure Save(
            AObject   : TBaseRemotable;
            AStore    : IFormatterBase;
      Const AName     : String;
      Const ATypeInfo : PTypeInfo
    );virtual;abstract;
    class procedure Load(
      Var   AObject   : TObject;
            AStore    : IFormatterBase;
      var   AName     : String;
      const ATypeInfo : PTypeInfo
    );virtual;abstract;
  End;

TBaseRemotable is the abstract base class used by the formatter interface to allow customization of the serialization. This class defines a virtual constructor and mainly two(2) virtual abstract class methods :

• Save : this method is called when the toolkit needs to serialize the AObject parameter.
• Load: this method is called when the toolkit needs to un-serialize to the AObject parameter.

The “TBaseComplexRemotable” serialization

TBaseComplexRemotable implements serialization for its descendants classes published properties. The serialization is based on runtime type information (RTTI) and can be customized to:

• ignore always some published properties.
• ignore conditionally some published properties.

The following class shows a the serialization's customization sample.

  TSampleClass = class(TBaseComplexRemotable)
  private
    FProp_Always: Integer;
    FProp_Never: Integer;
    FProp_Optional: Integer;
    function GetStoredProp_Optional: boolean;
  published
    //This property will always be serialized
    property Prop_Always : Integer read FProp_Always write FProp_Always;
    //This property will never be serialized
    property Prop_Never : Integer read FProp_Never write FProp_Never stored False;
    //This property will be serialized if "Self.GetStoredProp_Optional() = True"
    property Prop_Optional : Integer read FProp_Optional write FProp_Optional stored GetStoredProp_Optional;
  End;

Provided array implementations

The toolkit provides array implementation for basic types ( in the base_service_intf unit ) listed bellow. The implementations are based on the serialization's customization.

• Available integers :
  • Byte TArrayOfInt8URemotable
  • ShortInt TArrayOfInt8SRemotable
  • SmallInt TArrayOfInt16SRemotable
  • Word TArrayOfInt16URemotable
  • LongInt TArrayOfInt32SRemotable
  • LongWord TArrayOfInt32URemotable
  • Int64 TArrayOfInt64SRemotable
  • Qword TArrayOfInt64URemotable
• String TarrayOfStringRemotable( AnsiString )
• Boolean TArrayOfBooleanRemotable
• Float types :
  • Single TArrayOfFloatSingleRemotable
  • Double TArrayOfFloatDoubleRemotable
  • Extended TArrayOfFloatExtendedRemotable
  • Currency TArrayOfFloatCurrencyRemotable

Test cases

The toolkit uses FPCUnit for test cases. The test project is located in the \tests\test_suite folder.

TODO Client-Side

• Basic array support for SOAP
• Basic array support for Binary format
• XML-RPC formatter
• More documentation and samples !
• WSDL to pascal compiler
• Enhance the parser
  • To allow comments in the input file of ws_helper

TODO Server-Side

Extend the toolkit to Server side for :

• SOAP
• Binary serialization
• Bytes ordering in binary serialization : alaways use big-endian
• XML-RPC
• TCP transport ( first implementation).
• WSDL generation
• More documentation and samples !

Licence

• The support units are provided under modified LGPL
• ws_helper sources are provided under GPL 2 ( or later version )

Download

The lastest version can be found at http://inoussa12.googlepages.com/webservicetoolkitforfpc%26lazarus

Changes Log

• Version 0.2.2 ( 7 June 2006 )
  • All pascal basic types are supported by the SOAP serializer and the Binary serializer ( Available integers,available floats,string,boolean,Enumerations,class intances )
  • Array support for Binary serializer
  • FPCUnit test cases
    • SOAP serializer ( basic types and classes instances )
    • Binary serializer ( basic types and classes instances )
  • All interfaces now have GUID

Author

Inoussa OUEDRAOGO, http://inoussa12.googlepages.com/