Difference between revisions of "XML Tutorial"

From Lazarus wiki
Jump to navigationJump to search
(→‎Whitespace characters: Hint to free the doc created by ReadXMLFilePreserveWhitespace)
 
(22 intermediate revisions by 13 users not shown)
Line 1: Line 1:
 
{{XML Tutorial}}
 
{{XML Tutorial}}
 +
 +
The Extensible Markup Language (XML) is a World Wide Web Consortium (or "[http://www.w3.org/ W3C]") recommended language created to interchange information between different systems.
 +
 +
It is a text based way to store information as opposed to storing the information in a binary format.
 +
 +
Modern data interchange languages such as XHTML, as well as most WebServices technologies, are based on XML. This wiki can only really give a thumbnail overview of XML, with the primary focus being the parsing and use of XML files in Free Pascal applications. If you are interested in a more comprehensive explanation of XML and how it is used, see http://en.wikipedia.org/wiki/XML.
  
 
== Introduction ==
 
== Introduction ==
 
The Extensible Markup Language (XML) is a [http://www.w3.org/ W3C] recommended language created to interchange information between different systems. It is a text based way to store information. Modern data interchange languages such as XHTML, as well as most WebServices technologies, are based on XML.
 
  
 
Currently there is a set of units that provides support for XML on Free Pascal. These units are called "XMLRead", "XMLWrite" and "DOM" and they are part of the Free Component Library (FCL) from the Free Pascal Compiler. The FCL is already on the default search path for the compiler on Lazarus, so you only need to add the units to your uses clause in order to get XML support. The FCL is not documented currently (October / 2005), so this short tutorial aims at introducing XML access using those units.
 
Currently there is a set of units that provides support for XML on Free Pascal. These units are called "XMLRead", "XMLWrite" and "DOM" and they are part of the Free Component Library (FCL) from the Free Pascal Compiler. The FCL is already on the default search path for the compiler on Lazarus, so you only need to add the units to your uses clause in order to get XML support. The FCL is not documented currently (October / 2005), so this short tutorial aims at introducing XML access using those units.
Line 13: Line 17:
 
Below there is a list of XML data manipulation examples with growing complexity.
 
Below there is a list of XML data manipulation examples with growing complexity.
  
=== Units in the uses clause: Unicode or Ansi ===
+
=== Units in the uses clause ===
  
FPC comes with XML units which utilize ansi-encoded routines and therefore in each platform the encoding might be different and might not be Unicode. Lazarus comes with another separate set of XML units in the package LazUtils which fully support UTF-8 Unicode in all platforms. The units are compatible and one can change from to the other just by changing the uses clause.
+
FPC comes with XML units that supports UTF8 and UTF16. They may not have the latest updates, so there is as well a version of those units (prefixed with "laz2_") in the Lazarus package LazUtils. The units are compatible and one can change from to the other just by changing the uses clause.  
  
The units for using the FPC XML support which uses strings encoded in the system encoding are: DOM, XMLRead, XMLWrite, XMLCfg, XMLUtils, XMLStreaming.
+
The units for using the FPC XML support which uses strings encoded in the system encoding are:
  
The units for using the Lazarus XML support which has full UTF-8 Unicode support are: laz2_DOM, laz2_XMLRead, laz2_XMLWrite, laz2_XMLCfg, laz2_XMLUtils, laz_XMLStreaming.
+
* DOM
 +
* XMLRead
 +
* XMLWrite
 +
* XMLCfg
 +
* XMLUtils
 +
* XMLStreaming
  
Not all of them are needed in every example, though.
+
The units for using the Lazarus XML support which has full UTF-8 Unicode support are:
 +
 
 +
* laz2_DOM
 +
* laz2_XMLRead
 +
* laz2_XMLWrite
 +
* laz2_XMLCfg
 +
* laz2_XMLUtils
 +
* laz_XMLStreaming.
 +
 
 +
Not all of them are needed in every example, though. You will need DOM as it defines several types including TXMLDocument.
  
 
=== Reading a text node ===
 
=== Reading a text node ===
  
 
For Delphi Programmers:
 
For Delphi Programmers:
 +
 
Note that when working with TXMLDocument, the text within a Node is considered a separate TEXT Node.  As a result, you must access a node's text value as a separate node. Alternatively, the '''TextContent''' property may be used to retrieve content of all text nodes beneath the given one, concatenated together.
 
Note that when working with TXMLDocument, the text within a Node is considered a separate TEXT Node.  As a result, you must access a node's text value as a separate node. Alternatively, the '''TextContent''' property may be used to retrieve content of all text nodes beneath the given one, concatenated together.
  
Line 32: Line 51:
 
For instance, consider the following XML:
 
For instance, consider the following XML:
  
<syntaxhighlight lang="xml"><?xml version="1.0"?>
+
<syntaxhighlight lang="xml">
 +
<?xml version="1.0"?>
 
<request>
 
<request>
 
   <request_type>PUT_FILE</request_type>
 
   <request_type>PUT_FILE</request_type>
 
   <username>123</username>
 
   <username>123</username>
 
   <password>abc</password>
 
   <password>abc</password>
</request></syntaxhighlight>
+
</request>
 +
</syntaxhighlight>
  
 
The following code example shows both the correct and the incorrect ways of getting the value of the text node (add the units '''laz2_XMLRead''' and '''laz2_DOM''' to the used units list):
 
The following code example shows both the correct and the incorrect ways of getting the value of the text node (add the units '''laz2_XMLRead''' and '''laz2_DOM''' to the used units list):
  
<source>var
+
<syntaxhighlight lang=pascal>
 +
var
 
   PassNode: TDOMNode;
 
   PassNode: TDOMNode;
 
   Doc: TXMLDocument;
 
   Doc: TXMLDocument;
Line 60: Line 82:
 
     Doc.Free;
 
     Doc.Free;
 
   end;
 
   end;
end;</source>
+
end;
 +
</syntaxhighlight>
  
 
Note that ReadXMLFile(...) ignores all leading whitespace characters when parsing a document. The section  [[#Whitespace_characters|whitespace characters]] describes how to keep them.
 
Note that ReadXMLFile(...) ignores all leading whitespace characters when parsing a document. The section  [[#Whitespace_characters|whitespace characters]] describes how to keep them.
Line 66: Line 89:
 
=== Printing the names of nodes and attributes ===
 
=== Printing the names of nodes and attributes ===
  
A quick note on navigating the DOM tree: When you need to access nodes in sequence, it is best to use '''FirstChild''' and '''NextSibling''' properties (to iterate forward), or '''LastChild''' and '''PreviousSibling''' (to iterate backward). For random access it is possible to use '''ChildNodes''' or '''GetElementsByTagName''' methods, but these will create a TDOMNodeList object which eventually must be freed. This differs from other DOM implementations like MSXML, because FCL implementation is object-based, not interface-based.
+
If you want to navigate the DOM tree: when you need to access nodes in sequence, it is best to use '''FirstChild''' and '''NextSibling''' properties (to iterate forward), or '''LastChild''' and '''PreviousSibling''' (to iterate backward).
 +
 
 +
For random access it is possible to use '''ChildNodes''' or '''GetElementsByTagName''' methods, but these will create a TDOMNodeList object which eventually must be freed. This differs from other DOM implementations like MSXML, because the FCL implementation is object-based, not interface-based.
  
 
The following example shows how to print the names of nodes to a TMemo placed on a form.
 
The following example shows how to print the names of nodes to a TMemo placed on a form.
  
Bellow is the XML file called 'test.xml':
+
Below is the XML file called 'test.xml':
  
 
<syntaxhighlight lang="xml"><?xml version="1.0"?>
 
<syntaxhighlight lang="xml"><?xml version="1.0"?>
Line 78: Line 103:
 
     <Peca DestinoX="0" DestinoY="86">Pecacastelo.jpg2.swf</Peca>
 
     <Peca DestinoX="0" DestinoY="86">Pecacastelo.jpg2.swf</Peca>
 
   </imageNode>
 
   </imageNode>
</images></syntaxhighlight>
+
</images>
 +
</syntaxhighlight>
  
 
And here the Pascal code to execute the task:
 
And here the Pascal code to execute the task:
  
<source>var
+
<syntaxhighlight lang=pascal>
 +
var
 
   Doc: TXMLDocument;
 
   Doc: TXMLDocument;
 
   Child: TDOMNode;
 
   Child: TDOMNode;
Line 116: Line 143:
 
     Doc.Free;
 
     Doc.Free;
 
   end;
 
   end;
end;</source>
+
end;
 +
</syntaxhighlight>
  
 
This will print:
 
This will print:
Line 122: Line 150:
 
<pre>imageNode graphic.jpg
 
<pre>imageNode graphic.jpg
 
Peca Pecacastelo.jpg1.swf (DestinoX=0; DestinoY=0)
 
Peca Pecacastelo.jpg1.swf (DestinoX=0; DestinoY=0)
Peca Pecacastelo.jpg2.swf (DestinoX=0; DestinoY=86)</pre>
+
Peca Pecacastelo.jpg2.swf (DestinoX=0; DestinoY=86)
 +
</pre>
  
 
=== Populating a TreeView with XML ===
 
=== Populating a TreeView with XML ===
Line 130: Line 159:
 
The function below will take a XML document previously loaded from a file or generated on code, and will populate a TreeView with it´s contents. The caption of each node will be the content of the first attribute of each node.
 
The function below will take a XML document previously loaded from a file or generated on code, and will populate a TreeView with it´s contents. The caption of each node will be the content of the first attribute of each node.
  
<source>procedure TForm1.XML2Tree(tree: TTreeView; XMLDoc: TXMLDocument);
+
<syntaxhighlight lang=pascal>
 +
procedure TForm1.XML2Tree(tree: TTreeView; XMLDoc: TXMLDocument);
 
var
 
var
 
   iNode: TDOMNode;
 
   iNode: TDOMNode;
Line 166: Line 196:
 
     iNode := iNode.NextSibling;
 
     iNode := iNode.NextSibling;
 
   end;
 
   end;
end;</source>
+
end;
 +
</syntaxhighlight>
  
Another example that displays the complete XML structure including all attribute values:
+
Another example that displays the complete XML structure including all attribute values (note: the long line referencing TreeView has been split so it will word wrap for this wiki; when writing it in code you do not have to break the line unless you like the formatting) :
<source>procedure XML2Tree(XMLDoc:TXMLDocument; TreeView:TTreeView);
+
 
 +
<syntaxhighlight lang=pascal>
 +
procedure XML2Tree(XMLDoc:TXMLDocument; TreeView:TTreeView);
  
 
   // Local function that outputs all node attributes as a string
 
   // Local function that outputs all node attributes as a string
Line 185: Line 218:
 
   end;
 
   end;
  
   // Recursive function to process a node and all its child nodes
+
   // Recursive function to process a node and all its child nodes  
 +
 
 
   procedure ParseXML(Node:TDOMNode; TreeNode: TTreeNode);
 
   procedure ParseXML(Node:TDOMNode; TreeNode: TTreeNode);
 
   begin
 
   begin
Line 192: Line 226:
  
 
     // Add node to TreeView
 
     // Add node to TreeView
     TreeNode := TreeView.Items.AddChild(TreeNode,Trim(Node.NodeName+' '+GetNodeAttributesAsString(Node)+ Node.NodeValue));
+
     TreeNode := TreeView.Items.AddChild(TreeNode,  
 +
                                          Trim(Node.NodeName+' '+  
 +
                                          GetNodeAttributesAsString(Node)+  
 +
                                          Node.NodeValue)
 +
                                        );
  
 
     // Process all child nodes
 
     // Process all child nodes
Line 207: Line 245:
 
   ParseXML(XMLDoc.DocumentElement,nil);
 
   ParseXML(XMLDoc.DocumentElement,nil);
 
end;
 
end;
</source>
+
</syntaxhighlight>
  
 
=== Modifying a XML document ===
 
=== Modifying a XML document ===
Line 217: Line 255:
 
Below are some common methods from TDOMDocument:
 
Below are some common methods from TDOMDocument:
  
<source>function CreateElement(const tagName: DOMString): TDOMElement; virtual;
+
<syntaxhighlight lang=pascal>
 +
function CreateElement(const tagName: DOMString): TDOMElement; virtual;
 
function CreateTextNode(const data: DOMString): TDOMText;
 
function CreateTextNode(const data: DOMString): TDOMText;
 
function CreateCDATASection(const data: DOMString): TDOMCDATASection; virtual;
 
function CreateCDATASection(const data: DOMString): TDOMCDATASection; virtual;
function CreateAttribute(const name: DOMString): TDOMAttr; virtual;</source>
+
function CreateAttribute(const name: DOMString): TDOMAttr; virtual;
 +
</syntaxhighlight>
  
 
<tt>CreateElement</tt> creates a new element.
 
<tt>CreateElement</tt> creates a new element.
Line 232: Line 272:
 
A more convenient method to manipulate attributes is to use <tt>TDOMElement.SetAttribute</tt> method, which is also represented as the default property of <tt>TDOMElement</tt>:
 
A more convenient method to manipulate attributes is to use <tt>TDOMElement.SetAttribute</tt> method, which is also represented as the default property of <tt>TDOMElement</tt>:
  
<syntaxhighlight>
+
<syntaxhighlight lang=pascal>
 
// these two statements are equivalent
 
// these two statements are equivalent
 
Element.SetAttribute('name', 'value');
 
Element.SetAttribute('name', 'value');
Line 238: Line 278:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
And here an example method that will locate the selected item on a TTreeView and then insert a child node to the XML document it represents. The TreeView must be previously filled with the contents of an XML file using the [[Networking#Populating a TreeView with XML|XML2Tree function]].
+
And here an example method that will locate the selected item on a TTreeView and then insert a child node to the XML document it represents. The TreeView must be previously filled with the contents of an XML file using the [[XML_Tutorial#Populating a TreeView with XML|XML2Tree function]].
  
<source>procedure TForm1.actAddChildNode(Sender: TObject);
+
<syntaxhighlight lang=pascal>
 +
procedure TForm1.actAddChildNode(Sender: TObject);
 
var
 
var
 
   position: Integer;
 
   position: Integer;
Line 276: Line 317:
 
     *******************************************************************}
 
     *******************************************************************}
 
   end;
 
   end;
end;</source>
+
end;</syntaxhighlight>
  
 
=== Create a TXMLDocument from a string ===
 
=== Create a TXMLDocument from a string ===
Line 282: Line 323:
 
Given an XML document in string variable ''MyXmlString'', the following code will create it's DOM:
 
Given an XML document in string variable ''MyXmlString'', the following code will create it's DOM:
  
<source>var
+
<syntaxhighlight lang=pascal>
 +
var
 
   S: TStringStream;
 
   S: TStringStream;
 
   XML: TXMLDocument;
 
   XML: TXMLDocument;
 
begin
 
begin
   S := TStringStream.Create(MyXMLString);
+
   S := TStringStream.Create('');
 
   try
 
   try
 
     // Read complete XML document
 
     // Read complete XML document
Line 295: Line 337:
 
     S.Free;
 
     S.Free;
 
   end;
 
   end;
end;</source>
+
end;
 +
</syntaxhighlight>
  
 
=== Validating a document ===
 
=== Validating a document ===
Line 311: Line 354:
 
   <child>This is a first child.</child>
 
   <child>This is a first child.</child>
 
   <child>And this is the second one.</child>
 
   <child>And this is the second one.</child>
</root></syntaxhighlight>
+
</root>
 +
</syntaxhighlight>
  
 
This DTD specifies that 'root' element must have one or more 'child' elements, and that 'child' elements may have only character data inside. If parser detects any violations from these rules, it will report them.
 
This DTD specifies that 'root' element must have one or more 'child' elements, and that 'child' elements may have only character data inside. If parser detects any violations from these rules, it will report them.
Line 317: Line 361:
 
Loading such document is slightly more complicated. Let's assume we have XML data in a TStream object:
 
Loading such document is slightly more complicated. Let's assume we have XML data in a TStream object:
  
<source>procedure TMyObject.DOMFromStream(AStream: TStream);
+
<syntaxhighlight lang=pascal>
 +
procedure TMyObject.DOMFromStream(AStream: TStream);
 
var
 
var
 
   Parser: TDOMParser;
 
   Parser: TDOMParser;
Line 345: Line 390:
 
   if E.Severity = esError then  // we are interested in validation errors only
 
   if E.Severity = esError then  // we are interested in validation errors only
 
     writeln(E.Message);
 
     writeln(E.Message);
end;</source>
+
end;
 +
</syntaxhighlight>
  
 
=== Whitespace characters ===
 
=== Whitespace characters ===
 +
 
If you want to preserve leading whitespace characters in node texts, the above method is the way to load your XML document. Leading whitespace characters are ignored by default. That is the reason why the ReadXML(...) function never returns any leading whitespace characters in node texts.
 
If you want to preserve leading whitespace characters in node texts, the above method is the way to load your XML document. Leading whitespace characters are ignored by default. That is the reason why the ReadXML(...) function never returns any leading whitespace characters in node texts.
 
Before calling ''Parser.Parse(Src, TheDoc)'' insert the line  
 
Before calling ''Parser.Parse(Src, TheDoc)'' insert the line  
  
<source>Parser.Options.PreserveWhitespace := True;</source>
+
<syntaxhighlight lang=pascal>
 +
Parser.Options.PreserveWhitespace := True;
 +
</syntaxhighlight>
 +
 
 +
This will force the parser to return all whitespace characters including all the newline characters that exist in an XML document to make it more readable!
  
This will force the parser to return all whitespace characters. This includes all the newline characters that exist in an XML document to make it more readable!
+
The following code is similar to the <tt>ReadXMLFile()</tt> procedure, but preserves leading whitespace:
 +
 
 +
<syntaxhighlight lang=pascal>
 +
procedure ReadXMLFilePreserveWhitespace(out Doc: TXMLDocument; FileName: string);
 +
var
 +
  Parser: TDOMParser;
 +
  Src: TXMLInputSource;
 +
  InFile: TFileStream;
 +
begin
 +
  try
 +
    InFile := TFileStream.Create(FileName, fmOpenRead);
 +
    Src := TXMLInputSource.Create(InFile);
 +
    Parser := TDOMParser.Create;
 +
    Parser.Options.PreserveWhitespace := True;
 +
    Parser.Parse(Src, Doc);
 +
  finally
 +
    Src.Free;
 +
    Parser.Free;
 +
    InFile.Free;
 +
  end;
 +
end;
 +
</syntaxhighlight>
 +
 
 +
Note that the xml document <tt>Doc</tt> is an <tt>out</tt> parameter created by the local <tt>Parser</tt>. It is the user's responsibility to free the <tt>Doc</tt> himself when it is no longer needed.
 +
 
 +
=== Streamed reading ===
 +
 
 +
DOM-based processing requires the entire document loaded into memory. This may be not desirable, or not possible if document is huge. FCL provides functionality to read XML data one node at a time, using TXMLReader class and its descendants. This is similar to SAX, but works without callbacks. TXMLReader closely resembles .NET XmlReader class.
 +
A basic example follows:
 +
 
 +
<syntaxhighlight lang=pascal>
 +
uses
 +
  Classes,xmlreader,xmltextreader,xmlutils;
 +
 
 +
procedure readme(AStream: TStream);
 +
var
 +
  xtr: TXmlReader;
 +
  settings: TXMLReaderSettings;
 +
  inp: TXMLInputSource;
 +
begin
 +
  settings := TXMLReaderSettings.Create;
 +
  try
 +
    settings.PreserveWhiteSpace := True;
 +
    settings.Namespaces := True;
 +
    inp := TXMLInputSource.Create(AStream);
 +
    try
 +
      xtr := TXmlTextReader.Create(inp,settings);
 +
      try
 +
        // Here the reading starts
 +
        while xtr.Read do
 +
        begin
 +
          write(xtr.NodeType:25);
 +
          if xtr.name<>'' then
 +
            write(xtr.Name:9)
 +
          else
 +
            write('*no name* ');
 +
          write(xtr.Value);
 +
          writeln;
 +
          if xtr.NodeType=ntElement then
 +
          begin
 +
            // print attributes
 +
            if xtr.MoveToFirstAttribute then
 +
            begin
 +
              repeat
 +
                writeln('---',xtr.NodeType:21,xtr.Name:10,xtr.Value:10);
 +
              until not xtr.MoveToNextAttribute;
 +
              xtr.MoveToContent;
 +
            end; 
 +
          end;
 +
        end;
 +
      // Cleanup follows 
 +
      finally
 +
        xtr.Free;
 +
      end;
 +
    finally
 +
      inp.Free;
 +
    end;
 +
  finally
 +
    settings.Free;
 +
  end;
 +
end;
 +
</syntaxhighlight>
  
 
=== Generating a XML file ===
 
=== Generating a XML file ===
Line 361: Line 493:
 
Please, remember to include the DOM and XMLWrite units in your uses clause.
 
Please, remember to include the DOM and XMLWrite units in your uses clause.
  
<source>unit Unit1;
+
<syntaxhighlight lang=pascal>
 +
unit Unit1;
  
 
{$mode objfpc}{$H+}
 
{$mode objfpc}{$H+}
Line 433: Line 566:
 
   {$I unit1.lrs}
 
   {$I unit1.lrs}
  
end.</source>
+
end.
 +
</syntaxhighlight>
  
 
The  result will be the XML file below:
 
The  result will be the XML file below:
 +
 
<syntaxhighlight lang="xml"><?xml version="1.0"?>
 
<syntaxhighlight lang="xml"><?xml version="1.0"?>
 
<register>
 
<register>
Line 442: Line 577:
 
     <idade>32</idade>
 
     <idade>32</idade>
 
   </usuario>
 
   </usuario>
</register></syntaxhighlight>
+
</register>
 +
</syntaxhighlight>
  
 
An example where you don't need to reference an item by index.
 
An example where you don't need to reference an item by index.
  
<source>
+
<syntaxhighlight lang=pascal>
 
procedure TForm1.Button2Click(Sender: TObject);
 
procedure TForm1.Button2Click(Sender: TObject);
 
var
 
var
Line 487: Line 623:
 
     Doc.Free;
 
     Doc.Free;
 
   end;
 
   end;
</source>
+
</syntaxhighlight>
  
 
Generated XML:
 
Generated XML:
 +
 
<syntaxhighlight lang="xml">
 
<syntaxhighlight lang="xml">
 
<?xml version="1.0"?>
 
<?xml version="1.0"?>
Line 510: Line 647:
 
=== Encoding ===
 
=== Encoding ===
  
Starting from SVN revision 12582, XML reader is able to process data in any encoding by using external decoders. See [[XML_Decoders]] for more details.
+
Starting from FPC version 2.4, the XML reader is able to process data in any encoding by using external decoders. See [[XML_Decoders]] for more details.
 
 
According to the XML standard, the encoding attribute in the first line of the XML is optional in case the actual encoding is UTF-8 (presumably no BOM - Byte Order Marker) or UTF-16 (UTF-16 BOM).
 
  
As of version 2.4 of FreePascal, there is an encoding property in a TXMLDocument, but it is ignored. WriteXMLFile always uses UTF-8 and doesn´t generate an encoding attribute in the first line of the XML file.
+
According to the XML standard, the encoding attribute in the first line of the XML is optional in case the actual encoding is UTF-8 ('''without''' BOM - Byte Order Marker) or UTF-16 (UTF-16 BOM).
  
FPC versions from current trunk does explicitly write an UTF8 encoding attribute, as this is needed for some programs that cannot handle the XML without it.
+
TXMLDocument has an encoding property since FPC 2.4. It is ignored as WriteXMLFile always uses UTF-8.
 +
* FPC 2.4 doesn´t generate an encoding attribute in the first line of the XML file
 +
* FPC 2.6.0 and later explicitly write an UTF8 encoding attribute, as this is needed for some programs that cannot handle the XML without it.
  
 
== See also ==
 
== See also ==
Line 523: Line 660:
 
* [[Using INI Files]]
 
* [[Using INI Files]]
 
* [[fcl-xml]]
 
* [[fcl-xml]]
* [[Internet Tools]]
+
* [[Internet Tools]], for XPath 2 / XQuery processing
  
 
== External Links ==
 
== External Links ==
Line 530: Line 667:
  
 
* [http://www.thomas-zastrow.de/texte/fpcxml/index.php Thomas Zastrow article] [http://web.archive.org/web/20080802150722/http://www.thomas-zastrow.de/texte/fpcxml/index.php Alternate link] FPC and XML
 
* [http://www.thomas-zastrow.de/texte/fpcxml/index.php Thomas Zastrow article] [http://web.archive.org/web/20080802150722/http://www.thomas-zastrow.de/texte/fpcxml/index.php Alternate link] FPC and XML
 
[[Category:Free Component Library]]
 
[[Category:Tutorials]]
 

Latest revision as of 23:27, 5 July 2023

Deutsch (de) English (en) español (es) français (fr) magyar (hu) Bahasa Indonesia (id) italiano (it) 日本語 (ja) 한국어 (ko) português (pt) русский (ru) 中文(中国大陆)‎ (zh_CN)

The Extensible Markup Language (XML) is a World Wide Web Consortium (or "W3C") recommended language created to interchange information between different systems.

It is a text based way to store information as opposed to storing the information in a binary format.

Modern data interchange languages such as XHTML, as well as most WebServices technologies, are based on XML. This wiki can only really give a thumbnail overview of XML, with the primary focus being the parsing and use of XML files in Free Pascal applications. If you are interested in a more comprehensive explanation of XML and how it is used, see http://en.wikipedia.org/wiki/XML.

Introduction

Currently there is a set of units that provides support for XML on Free Pascal. These units are called "XMLRead", "XMLWrite" and "DOM" and they are part of the Free Component Library (FCL) from the Free Pascal Compiler. The FCL is already on the default search path for the compiler on Lazarus, so you only need to add the units to your uses clause in order to get XML support. The FCL is not documented currently (October / 2005), so this short tutorial aims at introducing XML access using those units.

The XML DOM (Document Object Model) is a set of standardized objects that provide a similar interface for using XML on different languages and systems. The standard only specifies the methods, properties and other interface parts of the object, leaving the implementation free for different languages. The FCL currently fully supports the XML DOM 2.0 and a subset of XML DOM 3.0 listed here.

Usage Examples

Below there is a list of XML data manipulation examples with growing complexity.

Units in the uses clause

FPC comes with XML units that supports UTF8 and UTF16. They may not have the latest updates, so there is as well a version of those units (prefixed with "laz2_") in the Lazarus package LazUtils. The units are compatible and one can change from to the other just by changing the uses clause.

The units for using the FPC XML support which uses strings encoded in the system encoding are:

  • DOM
  • XMLRead
  • XMLWrite
  • XMLCfg
  • XMLUtils
  • XMLStreaming

The units for using the Lazarus XML support which has full UTF-8 Unicode support are:

  • laz2_DOM
  • laz2_XMLRead
  • laz2_XMLWrite
  • laz2_XMLCfg
  • laz2_XMLUtils
  • laz_XMLStreaming.

Not all of them are needed in every example, though. You will need DOM as it defines several types including TXMLDocument.

Reading a text node

For Delphi Programmers:

Note that when working with TXMLDocument, the text within a Node is considered a separate TEXT Node. As a result, you must access a node's text value as a separate node. Alternatively, the TextContent property may be used to retrieve content of all text nodes beneath the given one, concatenated together.

The ReadXMLFile procedure always creates a new TXMLDocument, so you don't have to create it beforehand. However, be sure to destroy the document by calling Free when you are done.

For instance, consider the following XML:

<?xml version="1.0"?>
<request>
  <request_type>PUT_FILE</request_type>
  <username>123</username>
  <password>abc</password>
</request>

The following code example shows both the correct and the incorrect ways of getting the value of the text node (add the units laz2_XMLRead and laz2_DOM to the used units list):

var
  PassNode: TDOMNode;
  Doc: TXMLDocument;
begin
  try
    // Read in xml file from disk
    ReadXMLFile(Doc, 'test.xml');
    // Retrieve the "password" node
    PassNode := Doc.DocumentElement.FindNode('password');
    // Write out value of the selected node
    WriteLn(PassNode.NodeValue); // will be blank
    // The text of the node is actually a separate child node
    WriteLn(PassNode.FirstChild.NodeValue); // correctly prints "abc"
    // alternatively
    WriteLn(PassNode.TextContent);
  finally
    // finally, free the document
    Doc.Free;
  end;
end;

Note that ReadXMLFile(...) ignores all leading whitespace characters when parsing a document. The section whitespace characters describes how to keep them.

Printing the names of nodes and attributes

If you want to navigate the DOM tree: when you need to access nodes in sequence, it is best to use FirstChild and NextSibling properties (to iterate forward), or LastChild and PreviousSibling (to iterate backward).

For random access it is possible to use ChildNodes or GetElementsByTagName methods, but these will create a TDOMNodeList object which eventually must be freed. This differs from other DOM implementations like MSXML, because the FCL implementation is object-based, not interface-based.

The following example shows how to print the names of nodes to a TMemo placed on a form.

Below is the XML file called 'test.xml':

<?xml version="1.0"?>
<images directory="mydir">
  <imageNode URL="graphic.jpg" title="">
    <Peca DestinoX="0" DestinoY="0">Pecacastelo.jpg1.swf</Peca>
    <Peca DestinoX="0" DestinoY="86">Pecacastelo.jpg2.swf</Peca>
  </imageNode>
</images>

And here the Pascal code to execute the task:

var
  Doc: TXMLDocument;
  Child: TDOMNode;
  j: Integer;
begin
  try
    ReadXMLFile(Doc, 'test.xml');
    Memo.Lines.Clear;
    // using FirstChild and NextSibling properties
    Child := Doc.DocumentElement.FirstChild;
    while Assigned(Child) do
    begin
      Memo.Lines.Add(Child.NodeName + ' ' + Child.Attributes.Item[0].NodeValue);
      // using ChildNodes method
      with Child.ChildNodes do
      try
        for j := 0 to (Count - 1) do
          Memo.Lines.Add(format('%s %s (%s=%s; %s=%s)',
                                [
                                  Item[j].NodeName,
                                  Item[j].FirstChild.NodeValue,
                                  Item[j].Attributes.Item[0].NodeName,  // 1st attribute details
                                  Item[j].Attributes.Item[0].NodeValue,
                                  Item[j].Attributes.Item[1].NodeName,  // 2nd attribute details
                                  Item[j].Attributes.Item[1].NodeValue
                                ]));
      finally
        Free;
      end;
      Child := Child.NextSibling;
    end;
  finally
    Doc.Free;
  end;
end;

This will print:

imageNode graphic.jpg
Peca Pecacastelo.jpg1.swf (DestinoX=0; DestinoY=0)
Peca Pecacastelo.jpg2.swf (DestinoX=0; DestinoY=86)

Populating a TreeView with XML

One common use of XML files is to parse them and show their contents in a tree like format. You can find the TTreeView component on the "Common Controls" tab on Lazarus.

The function below will take a XML document previously loaded from a file or generated on code, and will populate a TreeView with it´s contents. The caption of each node will be the content of the first attribute of each node.

procedure TForm1.XML2Tree(tree: TTreeView; XMLDoc: TXMLDocument);
var
  iNode: TDOMNode;

  procedure ProcessNode(Node: TDOMNode; TreeNode: TTreeNode);
  var
    cNode: TDOMNode;
    s: string;
  begin
    if Node = nil then Exit; // Stops if reached a leaf
    
    // Adds a node to the tree
    if Node.HasAttributes and (Node.Attributes.Length>0) then
      s := Node.Attributes[0].NodeValue
    else
      s := ''; 
    TreeNode := tree.Items.AddChild(TreeNode, s);

    // Goes to the child node
    cNode := Node.FirstChild;

    // Processes all child nodes
    while cNode <> nil do
    begin
      ProcessNode(cNode, TreeNode);
      cNode := cNode.NextSibling;
    end;
  end;
    
begin
  iNode := XMLDoc.DocumentElement.FirstChild;
  while iNode <> nil do
  begin
    ProcessNode(iNode, nil); // Recursive
    iNode := iNode.NextSibling;
  end;
end;

Another example that displays the complete XML structure including all attribute values (note: the long line referencing TreeView has been split so it will word wrap for this wiki; when writing it in code you do not have to break the line unless you like the formatting) :

procedure XML2Tree(XMLDoc:TXMLDocument; TreeView:TTreeView);

  // Local function that outputs all node attributes as a string
  function GetNodeAttributesAsString(pNode: TDOMNode):string;
  var i: integer;
  begin
    Result:='';
    if pNode.HasAttributes then
      for i := 0 to pNode.Attributes.Length -1 do
        with pNode.Attributes[i] do
          Result := Result + format(' %s="%s"', [NodeName, NodeValue]);

    // Remove leading and trailing spaces
    Result:=Trim(Result);
  end;

  // Recursive function to process a node and all its child nodes 

  procedure ParseXML(Node:TDOMNode; TreeNode: TTreeNode);
  begin
    // Exit procedure if no more nodes to process
    if Node = nil then Exit;

    // Add node to TreeView
    TreeNode := TreeView.Items.AddChild(TreeNode, 
                                          Trim(Node.NodeName+' '+ 
                                           GetNodeAttributesAsString(Node)+ 
                                           Node.NodeValue)
                                        );

    // Process all child nodes
    Node := Node.FirstChild;
    while Node <> Nil do
    begin
      ParseXML(Node, TreeNode);
      Node := Node.NextSibling;
    end;
  end;

begin
  TreeView.Items.Clear;
  ParseXML(XMLDoc.DocumentElement,nil);
end;

Modifying a XML document

The first thing to remember is that TDOMDocument is the "handle" to the DOM. You can get an instance of this class by creating one or by loading a XML document.

Nodes on the other hand cannot be created like a normal object. You *must* use the methods provided by TDOMDocument to create them, and later use other methods to put them in the correct place in the tree. This is because nodes must be "owned" by a specific document in DOM.

Below are some common methods from TDOMDocument:

function CreateElement(const tagName: DOMString): TDOMElement; virtual;
function CreateTextNode(const data: DOMString): TDOMText;
function CreateCDATASection(const data: DOMString): TDOMCDATASection; virtual;
function CreateAttribute(const name: DOMString): TDOMAttr; virtual;

CreateElement creates a new element.

CreateTextNode creates a text node.

CreateAttribute creates an attribute node.

CreateCDATASection creates a CDATA section: regular XML markup characters such as <> are not interpreted within the CDATA section. See Wikipedia article on CDATA

A more convenient method to manipulate attributes is to use TDOMElement.SetAttribute method, which is also represented as the default property of TDOMElement:

// these two statements are equivalent
Element.SetAttribute('name', 'value');
Element['name'] := 'value';

And here an example method that will locate the selected item on a TTreeView and then insert a child node to the XML document it represents. The TreeView must be previously filled with the contents of an XML file using the XML2Tree function.

procedure TForm1.actAddChildNode(Sender: TObject);
var
  position: Integer;
  NovoNo: TDomNode;
begin
  {*******************************************************************
  *  Detects the selected element
  *******************************************************************}
  if TreeView1.Selected = nil then Exit;

  if TreeView1.Selected.Level = 0 then
  begin
    position := TreeView1.Selected.Index;

    NovoNo := XMLDoc.CreateElement('item');
    TDOMElement(NovoNo).SetAttribute('nome', 'Item');
    TDOMElement(NovoNo).SetAttribute('arquivo', 'Arquivo');
    with XMLDoc.DocumentElement.ChildNodes do
    begin
      Item[position].AppendChild(NovoNo);
      Free;
    end;

    {*******************************************************************
    *  Updates the TreeView
    *******************************************************************}
    TreeView1.Items.Clear;
    XML2Tree(TreeView1, XMLDoc);
  end
  else if TreeView1.Selected.Level >= 1 then
  begin
    {*******************************************************************
    *  This function only works on the first level of the tree,
    *  but can easily be modified to work for any number of levels
    *******************************************************************}
  end;
end;

Create a TXMLDocument from a string

Given an XML document in string variable MyXmlString, the following code will create it's DOM:

var
  S: TStringStream;
  XML: TXMLDocument;
begin
  S := TStringStream.Create('');
  try
    // Read complete XML document
    ReadXMLFile(XML, S);             
    // Alternatively: read only an XML Fragment
    ReadXMLFragment(AParentNode, S); 
  finally
    S.Free;
  end;
end;

Validating a document

Since March 2007, DTD validation facility has been added to the FCL XML parser. Validation is checking that logical structure of the document conforms to the predefined rules, called Document Type Definition (DTD).

Here is an example of XML document with a DTD:

<?xml version='1.0'?>
<!DOCTYPE root [
<!ELEMENT root (child)+ >
<!ELEMENT child (#PCDATA)>
]>
<root>
  <child>This is a first child.</child>
  <child>And this is the second one.</child>
</root>

This DTD specifies that 'root' element must have one or more 'child' elements, and that 'child' elements may have only character data inside. If parser detects any violations from these rules, it will report them.

Loading such document is slightly more complicated. Let's assume we have XML data in a TStream object:

procedure TMyObject.DOMFromStream(AStream: TStream);
var
  Parser: TDOMParser;
  Src: TXMLInputSource;
  TheDoc: TXMLDocument;
begin
  try
    // create a parser object
    Parser := TDOMParser.Create;
    // and the input source
    Src := TXMLInputSource.Create(AStream);
    // we want validation
    Parser.Options.Validate := True;
    // assign a error handler which will receive notifications
    Parser.OnError := @ErrorHandler;
    // now do the job
    Parser.Parse(Src, TheDoc);
    // ...and cleanup
  finally
    Src.Free;
    Parser.Free;
  end;
end;

procedure TMyObject.ErrorHandler(E: EXMLReadError);
begin
  if E.Severity = esError then  // we are interested in validation errors only
    writeln(E.Message);
end;

Whitespace characters

If you want to preserve leading whitespace characters in node texts, the above method is the way to load your XML document. Leading whitespace characters are ignored by default. That is the reason why the ReadXML(...) function never returns any leading whitespace characters in node texts. Before calling Parser.Parse(Src, TheDoc) insert the line

Parser.Options.PreserveWhitespace := True;

This will force the parser to return all whitespace characters including all the newline characters that exist in an XML document to make it more readable!

The following code is similar to the ReadXMLFile() procedure, but preserves leading whitespace:

procedure ReadXMLFilePreserveWhitespace(out Doc: TXMLDocument; FileName: string);
var
  Parser: TDOMParser;
  Src: TXMLInputSource;
  InFile: TFileStream;
begin
  try
    InFile := TFileStream.Create(FileName, fmOpenRead);
    Src := TXMLInputSource.Create(InFile);
    Parser := TDOMParser.Create;
    Parser.Options.PreserveWhitespace := True;
    Parser.Parse(Src, Doc);
  finally
    Src.Free;
    Parser.Free;
    InFile.Free;
  end;
end;

Note that the xml document Doc is an out parameter created by the local Parser. It is the user's responsibility to free the Doc himself when it is no longer needed.

Streamed reading

DOM-based processing requires the entire document loaded into memory. This may be not desirable, or not possible if document is huge. FCL provides functionality to read XML data one node at a time, using TXMLReader class and its descendants. This is similar to SAX, but works without callbacks. TXMLReader closely resembles .NET XmlReader class. A basic example follows:

uses
  Classes,xmlreader,xmltextreader,xmlutils;

procedure readme(AStream: TStream);
var
  xtr: TXmlReader;
  settings: TXMLReaderSettings;
  inp: TXMLInputSource;
begin
  settings := TXMLReaderSettings.Create;
  try
    settings.PreserveWhiteSpace := True;
    settings.Namespaces := True;
    inp := TXMLInputSource.Create(AStream);
    try
      xtr := TXmlTextReader.Create(inp,settings);
      try
        // Here the reading starts
        while xtr.Read do
        begin
          write(xtr.NodeType:25);
          if xtr.name<>'' then
            write(xtr.Name:9)
          else
            write('*no name* ');
          write(xtr.Value);
          writeln;
          if xtr.NodeType=ntElement then
          begin
            // print attributes
            if xtr.MoveToFirstAttribute then
            begin
              repeat
                writeln('---',xtr.NodeType:21,xtr.Name:10,xtr.Value:10);
              until not xtr.MoveToNextAttribute;
              xtr.MoveToContent;
            end;  
          end;
        end;
      // Cleanup follows  
      finally
        xtr.Free;
      end;
    finally
      inp.Free;
    end;
  finally
    settings.Free;
  end;
end;

Generating a XML file

Below is the complete code to write a XML file. (This was taken from a tutorial in the DeveLazarus blog) Please, remember to include the DOM and XMLWrite units in your uses clause.

unit Unit1;

{$mode objfpc}{$H+}

interface

uses
  Classes, SysUtils, LResources, Forms, Controls, Graphics, Dialogs, StdCtrls,
  DOM, XMLWrite;

type
  { TForm1 }
  TForm1 = class(TForm)
    Button1: TButton;
    Label1: TLabel;
    Label2: TLabel;
    procedure Button1Click(Sender: TObject);
  private
    { private declarations }
  public
    { public declarations }
  end;
  
var
  Form1: TForm1;
  
implementation

{ TForm1 }

procedure TForm1.Button1Click(Sender: TObject);
var
  Doc: TXMLDocument;                                  // variable to document
  RootNode, parentNode, nofilho: TDOMNode;                    // variable to nodes
begin
  try
    // Create a document
    Doc := TXMLDocument.Create;

    // Create a root node
    RootNode := Doc.CreateElement('register');
    Doc.Appendchild(RootNode);                           // save root node
  
    // Create a parent node
    RootNode:= Doc.DocumentElement;
    parentNode := Doc.CreateElement('usuario');
    TDOMElement(parentNode).SetAttribute('id', '001');       // create atributes to parent node
    RootNode.Appendchild(parentNode);                          // save parent node

    // Create a child node
    parentNode := Doc.CreateElement('nome');                // create a child node
    // TDOMElement(parentNode).SetAttribute('sexo', 'M');     // create atributes
    nofilho := Doc.CreateTextNode('Fernando');         // insert a value to node
    parentNode.Appendchild(nofilho);                         // save node
    RootNode.ChildNodes.Item[0].AppendChild(parentNode);       // insert child node in respective parent node
 
    // Create a child node
    parentNode := Doc.CreateElement('idade');               // create a child node
    // TDOMElement(parentNode).SetAttribute('ano', '1976');   // create atributes
    nofilho := Doc.CreateTextNode('32');               // insert a value to node
    parentNode.Appendchild(nofilho);                         // save node
    RootNode.ChildNodes.Item[0].AppendChild(parentNode);       // insert a childnode in respective parent node

    writeXMLFile(Doc, 'test.xml');                     // write to XML
  finally
    Doc.Free;                                          // free memory
  end;
end;

initialization
  {$I unit1.lrs}

end.

The result will be the XML file below:

<?xml version="1.0"?>
<register>
  <usuario id="001">
    <nome>Fernando</nome>
    <idade>32</idade>
  </usuario>
</register>

An example where you don't need to reference an item by index.

procedure TForm1.Button2Click(Sender: TObject);
var
  Doc: TXMLDocument;
  RootNode, ElementNode,ItemNode,TextNode: TDOMNode;
  i: integer;
begin
  try
    // Create a document
    Doc := TXMLDocument.Create;
    // Create a root node
    RootNode := Doc.CreateElement('Root');
    Doc.Appendchild(RootNode);
    RootNode:= Doc.DocumentElement;
    // Create nodes
    for i := 1 to 20 do
    begin
      ElementNode:=Doc.CreateElement('Element');
      TDOMElement(ElementNode).SetAttribute('id', IntToStr(i));

      ItemNode:=Doc.CreateElement('Item1');
      TDOMElement(ItemNode).SetAttribute('Attr1', IntToStr(i));
      TDOMElement(ItemNode).SetAttribute('Attr2', IntToStr(i));
      TextNode:=Doc.CreateTextNode('Item1Value is '+IntToStr(i));
      ItemNode.AppendChild(TextNode);
      ElementNode.AppendChild(ItemNode);

      ItemNode:=Doc.CreateElement('Item2');
      TDOMElement(ItemNode).SetAttribute('Attr1', IntToStr(i));
      TDOMElement(ItemNode).SetAttribute('Attr2', IntToStr(i));
      TextNode:=Doc.CreateTextNode('Item2Value is '+IntToStr(i));
      ItemNode.AppendChild(TextNode);
      ElementNode.AppendChild(ItemNode);

      RootNode.AppendChild(ElementNode);
    end;
    // Save XML
    WriteXMLFile(Doc,'TestXML_v2.xml');
  finally
    Doc.Free;
  end;

Generated XML:

<?xml version="1.0"?>
<Root>
  <Element id="1">
    <Item1 Attr1="1" Attr2="1">Item1Value is 1</Item1>
    <Item2 Attr1="1" Attr2="1">Item2Value is 1</Item2>
  </Element>
  <Element id="2">
    <Item1 Attr1="2" Attr2="2">Item1Value is 2</Item1>
    <Item2 Attr1="2" Attr2="2">Item2Value is 2</Item2>
  </Element>
  <Element id="3">
    <Item1 Attr1="3" Attr2="3">Item1Value is 3</Item1>
    <Item2 Attr1="3" Attr2="3">Item2Value is 3</Item2>
  </Element>
</Root>

Encoding

Starting from FPC version 2.4, the XML reader is able to process data in any encoding by using external decoders. See XML_Decoders for more details.

According to the XML standard, the encoding attribute in the first line of the XML is optional in case the actual encoding is UTF-8 (without BOM - Byte Order Marker) or UTF-16 (UTF-16 BOM).

TXMLDocument has an encoding property since FPC 2.4. It is ignored as WriteXMLFile always uses UTF-8.

  • FPC 2.4 doesn´t generate an encoding attribute in the first line of the XML file
  • FPC 2.6.0 and later explicitly write an UTF8 encoding attribute, as this is needed for some programs that cannot handle the XML without it.

See also

External Links