SqlDBHowto

From Lazarus wiki
Revision as of 15:58, 24 March 2012 by Vincent (talk | contribs) (Text replace - "delphi>" to "syntaxhighlight>")

Deutsch (de) English (en) español (es) français (fr) Nederlands (nl) 中文(中国大陆)‎ (zh_CN)

Introduction

This page is a translation of SqlDBHowto/nl. The Dutch text is leading, even though I took the liberty to make some corrections (minor corrections in my opinion.)

This text is setup as a 'how-to'. I want to answer a number of questions one by one, explain how you can use the various classes. All those questions are put one after the other and form a sort of tutorial.

I will try to word it in such a way that the text can be used for Lazarus as well as Freepascal. However, the examples are for Freepascal (i.e. they are console applications.)

How to connect to a database server?

SqlDB doesn't connect to a database server directly but uses a client that corresponds to the used database server. SqlDB sends the commands to the client library, the client library connects to the database and and transfers the commands. This means that a client library must be installed on the computer to make a connection to a database. Under Windows a client is usually a .dll, under Linux a .so and under OS/X a .dylib.

When the client library is installed properly you can connect to a database server using a TSQLConnection component. Various TSQLConnection components are available for different database servers. For example, use TIBConnection to connect to a Firebird/Interbase database; TPQConnection for PostgreSQL and TMySQL40Connection, TMySQL41Connection and TMySQL50Connection for MySQL databases with version number 4.0, 4.1 and 5.0 respectively. The differences between the MySQL client versions are big, to the extent that the clients and connections cannot be interchanged. If a MySQL client library version 4.1 is installed, you have to use a TMySQL41Connection. This is not related to the MySQL server: using the MySQL 4.1 client library you can probably connect to a MySQL 5.0 server.

Although details differ for the various database, in general you need to set four properties to connect to a database server:

  • the server name or ip address
  • the name of the database
  • the username
  • the password

When these properties are set, you can create a connection with the 'open' method. If the connection fails, a EDatabaseError exception is thrown. Use the property 'connected' to test if a connection has been made with the database server. Use the 'close' method to end the connection with the server.

Program ConnectDB
 
var AConnection : TSQLConnection;
 
Procedure CreateConnection;
begin
  AConnection := TIBConnection.Create(nil);
  AConnection.Hostname := 'localhost';
  AConnection.DatabaseName := '/opt/firebird/examples/employee.fdb';
  AConnection.UserName := 'sysdba';
  AConnection.Password := 'masterkey';
end;
 
begin
  CreateConnection;
  AConnection.Open;
  if Aconnection.Connected then
    writeln('Succesful connect!')
  else
    writeln('This is not possible, because if the connection failed, ' +
            'an exception should be raised, so this code would not ' +
            'be executed');
  AConnection.Close;
  AConnection.Free;
end.

If an exception is thrown, read the error message carefully. It may be that the database server is not running, the user name or password are incorrect or the database name or ip address are typed incorrectly. If the error message states that the client library cannot be found, then check if the client is installed correctly. Often the error message states literally the name of the file looked for.

How to execute direct queries/make a table?

SqlDB - the name says it all - only works with database server that make use of SQL. SQL stands for 'Structured Query Language' SQL is a language developed to allow working with relational databases. Virtually every database system has its own dialect, but a large number of SQL statements are the same for all database systems.

In FPC, there is a difference between:

  • SQL statements that return information (a dataset). For this, you have to use the TSQLQuery component; see #How to read data from a table?.
  • statements that do not return information but do something else, e.g. update data. For this, you may also use the 'ExecuteDirect' method of a TSQLConnection. (You can also use this if you get a dataset back but are not interested in the results, e.g. in a selectable stored procedure).

Most database system execute SQL statements within a transaction. If you want changes made within a transaction available in other transactions, or have those changes available even after closing the transaction(!), then you have to 'commit' the transaction.

To support transactions Sqldb contains the TSQLTransaction component. A SQL statement that is executed by Sqldb must always be executed within a transaction, even if the database system does not support transactions. Also, there are database systems that do support transaction for which TSQLConnection does not (yet) support transaction. Even then, you must use the TSQLTransaction component.

To use TSQLConnection.ExecuteDirect to execute a SQL statement you must specify which 'Transaction' must be used. In turn, to use TSQLTransaction you must specify which TSQLConnection component must be used.

The following example creates a table 'TBLNAMES' with fields 'NAME' and 'ID' and inserts two records. The used SQL statements are not explained. For more information about the SQL statements, their use and syntax, please refer to the database system documentation. The procedure 'CreateConnection' is defined in the code example in #How to connect to a database server? above.

program CreateTable;
 
var 
  AConnection : TSQLConnection;
  ATransaction : TSQLTransaction;
 
procedure CreateTransaction;
begin
  ATransaction := TSQLTransaction.Create;
  ATransaction.Database := AConnection;
end;
 
begin
  CreateConnection;
  CreateTransaction;
  AConnection.Transaction := ATransaction;
  AConnection.Open;
  ATransaction.StartTransaction;
  AConnection.ExecuteDirect('create table TBLNAMES (ID integer, NAME varchar(40));'); 
   
  // Some database-server types need a commit before you can use a newly created table. (Firebird)
  // With .Commit you also close the transaction
  ATransaction.Commit; 
 
  ATransaction.StartTransaction;
  AConnection.ExecuteDirect('insert into TBLNAMES (ID,NAME) values (1,'Name1');'); 
  AConnection.ExecuteDirect('insert into TBLNAMES (ID,NAME) values (2,'Name2');'); 
  ATransaction.Commit; 
  AConnection.Close;
  AConnection.Free;
  ATransaction.Free;
end.

How to read data from a table?

Use the TSQLQuery component to read data from a table. A TSQLQuery component must be connected to a TSQLConnection component and a TSQLTransaction component to do its work. Setting the TSQLConnection and TSQLTransaction is discussed in #How to connect to a database server? and #How to execute direct queries/make a table?.

When the TSQLConnection, TSQLTransaction and TSQLQuery are connected, then TSQLQuery needs to be further configured to work. TSQLQuery has a 'SQL' property containing a TStrings object. The 'SQL' property contains a SQL statement that must be executed. If all data from a table tablename must be read, then set the 'SQL' property to <sql>'SELECT * FROM tablename;'</sql>. Use 'open' to read the table from the server and put the data in the TSQLQuery dataset. The data can be accessed through TSQLQuery until the query is closed using 'close'.

TSQLQuery is a subclass of TDataset. TDataset has a 'Fields' collection that contains all columns of the table. The TDataset also keeps track of the current record. Use 'First', 'Next', 'Prior' and 'Last' to change the current record. 'Bof' returns 'True' if the first record is reached, and 'Eof' returns 'True' if the last record is reached. To read the value of a field in the current record, first find the right 'TField' object and then use 'AsString', 'AsInteger', etc.


Below is an example that displays all values of the table as it was made in #How to execute direct queries/make a table? above.

Program ShowData;
 
var 
  AConnection : TSQLConnection;
  ATransaction : TSQLTransaction;
 
function GetQuery : TSQLQuery;
  var AQuery : TSQLQuery;
begin
  AQuery := TSQLQuery.Create;
  AQuery.Database := FConnection;
  AQuery.Transaction := FTransaction;
  Result := AQuery;
end;
 
var 
  Query : TSQLQuery;
 
begin
  CreateConnection;
  CreateTransaction;
  Query := GetQuery;
  Query.SQL.Text := 'select * from tblNames';
  AConnection.Open;
  Query.Open;
  while not Query.Eof do
  begin
    Writeln('ID: ', Query.FieldByName('Name').AsInteger, 'Name: ' +
                                  Query.FieldByName('Name').AsString);
    Query.Next;
  end;
  Query.Close;
  AConnection.Close;
  Query.Free;
  ATransaction.Free;
  AConnection.Free;
end.

(The code above of course is not quite finished, it misses 'try...finally' blocks. However, the above code intends to show the database code and thus the finishing touches are left out.) Please note that 'TSQLTransaction.StartTransaction' is not used. This is not necessary. When TSQLQuery is opened, the SQL statement is executed and if no transaction is available then a transaction is automatically started. The programmer does not need to start the transaction explicitly. The same applies for the connection maintained by TSQLConnection. The connection is opened as needed, the line 'Aconnection.Open' is not really required. If a TSQLTransaction is destroyed, an automatic 'rollback' will be executed. Possible changes to data contained in the transaction will be lost.

Why does TSQLQuery.RecordCount always return 10?

To count the records in a datase, use '.RecordCount'. However, notice that '.RecordCount' shows the number of records that is already loaded from the server. SqlDB does not read all records when opening TSQLQuery by default, only the first 10. Only when the eleventh record is accessed will the next set of 10 records be loaded, etc. Using '.Last', all records will be loaded.

When you want to know the real number of records on the server you can first call '.Last' and then call '.RecordCount'. An alternative is available. The number of records returned by the server is set by the '.PacketRecords' property. The default value is 10; if you make it -1 then all records will be loaded at once.

Of course, you could also query the number of records in a query using another SQL query, but you would have to do that in the same transaction, as other transactions may have changed the number of records in the meanwhile.

Lazarus

Lazarus has various components to show data from a TDataset on a form. Instead of a While-loop and Writeln statements as used above, you can use the components to show the data in a table. Place the right TSQLConnection, TSQLTransaction and TSQLQuery components on a form, then connect them and set them properly. In addition you will need a TDatasource; set the 'TDatasource.Dataset' property to the TSQLQuery component you used. (Note do not set the 'TSQLQuery.Datasource' property to the TDatasource compnent you used. The 'TSQLQuery.Datasource' property is used only in master-detail tables.) Subsequently you may put a TDBGrid onto the form and set the 'Datasource' property of the grid to the TDatasource component you added before.

To see if it all works, set the 'Connected' property of the TSQLConnection to 'True' in the Lazarus IDE. The IDE will try to connect to the database server immediately. If this works you can set the 'TSQLQuery.Active' property to 'True'. If everything is right, you will see - within the IDE - all data from the table immediately on the screen.

How to change data in a table?

To change the data in a record, the TDataset (from which TSQLQuery is derived) must be set to edit mode. To enter edit mode call the '.Edit', '.Insert' or '.Append' methods. Use the '.Edit' method to change the current record. Use '.Insert' to insert a new record before the current record. Use '.Append' to insert a new record at the end of the table. In edit mode you can change field values through the 'Fields' property. Use 'Post' to validate the new data, if the data is valid then the edit mode is left. If you move to another record - for example by using '.Next' - and the dataset is in edit mode, then first '.Post' is called. Use '.Cancel' to discard all changes you made since the last '.Post' call and leave the edit mode.

Query.Edit;
Query.FieldByName('NAME').AsString := 'Edited name';
Query.Post;

The above is not the complete story yet. TSQLQuery is derived from TBufDataset which makes use of buffered updates. Buffered update means that after you called 'Post' the changes in the dataset are visible immediately, but they are not sent to the database server. What does happen is that the changes are maintained in a change log. When the '.ApplyUpdates' method is called, then all changes in the change log are sent to the database. Only then will database server know of the changes. The changes are sent to the server within a transaction of TSQLTransaction. Make sure to properly set the transaction before 'ApplyUpdates'. After applying the updates, a commit must be executed to save the changes on the database server.

The below is an example of changing the data in a table, sending the changes to the server and comitting the transaction.

Program EditData;
 
var 
  AConnection : TSQLConnection;
  ATransaction : TSQLTransaction;
  Query : TSQLQuery;
 
begin
  CreateConnection;
  CreateTransaction;
  AConnection.Transaction := ATransaction;
  Query := GetQuery;
  Query.SQL.Text := 'select * from tblNames';
  Query.Open;
  Query.Edit;
  Query.FieldByName('NAME').AsString := 'Edited name';
  Query.Post;
  Query.UpdateMode := upWhereAll;
  Query.ApplyUpdates;
  AConnection.Transaction.Commit;
  Query.Free;
  ATransaction.Free;
  AConnection.Free;
end.

For a discussion of 'UpdateMode' continue reading.

How does SqlDB send the changes to the database server?

In the code example in #How to change data in a table?, you will find the line

Query.UpdateMode := upWhereAll;

without explanation of what it does. The best way to find out what that line does is to leave it out. If you leave out the statement and the followed this howto precisely, then you will receive the following error message:

No update query specified and failed to generate one. (No fields for inclusion in where statement found)

To understand what went wrong, you must understand how changes are sent to the database server. The only way to get data in a SQL server is by executing SQL queries. SQL has three types of queries for three different ways of manupulating a record. To create a new record, change or delete a record insert, update and delete statements are executed respectively. An update statement may be as follows: <sql>update TBLNAMES set NAME='Edited name' where ID=1;</sql> To send a change to the database server, Sqldb must assemble an update query. To assemble the query, three things are needed:

The name of the table 
The table name is retrieved from parsing the select query, although this doesn't always work.
UPDATE or INSERT clause 
These contain the fields that must be changed.
WHERE clause 
This contains the fields that determine which records should be changed.

Every field (each TField in Fields) has a ProviderFlags property. Only fields with pfInUpdate in ProviderFlags will be used in the update or insert cluase of a query. By default all fields have pfInUpdate set in their ProviderFlags property.

Which fields are used in the WHERE clause depends on the UpdateMode property of the query and the ProviderFlags property of the fields. Fields with pfInkey in their ProviderFlags are always used in the WHERE clause. A field will have the pfInKey flag set automatically if the field is part of the primary key of the table and 'TSQLQuery.UsePrimaryKeyAsKey' returns 'True'.

The default value for UpdateMode of the query is upWhereKeyOnly. In this update mode only fields with pfInkey in their ProviderFlags property are used in the WHERE clause. If none of the fields have their pfInKey flag set, then no fields are available for the WHERE clause and the error message from the beginning of this section will be returned. You can solve the issue by:

  • Adding a primary key to the table and set TSQLQuery.UsePrimaryKeyAsKey to 'True', or
  • Setting the pfInkey flag for one or more fields in code.

The UpdateMode property knows two more possible values. 'upWhereAll' can be used to add all fields with the 'pfInWhere' flag set to the WHERE clause. By default all fields have this flag set. 'upWhereChanged' can be used to add only those fields that have the 'pfInWhere' flag set and that are changed in the current record.

How to execute a query using TSQLQuery?

Next to statements that return a dataset (see #How to read data from a table?) SQL has statements that do not return data. For example INSERT, UPDATE and DELETE statements do not return data. These statements can be executed using TSQLConnection.ExecuteDirect, but TSQLQuery can also be used. If you do not expect return data use TSQLQuery.ExecSQL instead of TSQLQuery.Open. As mentioned earlier, use TSQLQuery.Open to open the dataset returned by the SQL statement.

The following procedure creates a table and inserts two records using TSQLQuery.

procedure CreateTable;
  
var 
  Query : TSQLQuery;
  
begin
  Query := GetQuery;
  Query.SQL.Text := 'create table TBLNAMES (ID integer, NAME varchar(40));';
  Query.ExecSQL;
 
  Query.SQL.Text := 'insert into TBLNAMES (ID,NAME) values (1,''Name1'');';
  Query.ExecSQL;
  
  Query.SQL.Text := 'insert into TBLNAMES (ID,NAME) values (2,''Name2'');';
  Query.ExecSQL;
  
  Query.Close;
  Query.Free;
end;

How to use parameters in a query?

In the code example of #How to execute a query using TSQLQuery? the same query is used twice, only the values to be inserted differ. A better way to do this is by using parameters in the query.

The syntax of parameters in queries is different per database system, but the differences are handled by TSQLQuery. Replace the values in the query with a colon followed by the name of the parameter you want to use. For example:

Query.SQL.Text := 'insert into TBLNAMES (ID,NAME) values (:ID,:NAME);';

This query will create two parameters: 'ID' and 'NAME'. To determine the parameters, the query is parsed at the moment the text of TSQLQuery.SQL is assigned or changed. All existing parameters will be removed and the new parameters will be added to the 'TSQLQuery.Params' property. Assigning a value to a parameter is similar to assigning a value to a field in the dataset:

Query.Params.ParamByName('Name').AsString := 'Name1'

;

You can't tell from the query what kind of data must be stored in the parameter. The data type of the parameter is determined at the moment a value is first assigned to the parameter. By assigning a value using '.AsString', the parameter is assigned the data type 'ftString'. You can determine the data type directly by setting the 'DataType' property. If an incorrect datatype is assigned to the parameter, then problems will occur during opening or executing the query. See Database field type for more information on data types.

The following example creates the same table as the previous example, but now parameters are used:

procedure CreateTableUsingParameters;
  
var 
  Query : TSQLQuery;
  
begin
  Query := GetQuery;
  Query.SQL.Text := 'create table TBLNAMES (ID integer, NAME varchar(40));';
  Query.ExecSQL;

  Query.SQL.Text := 'insert into TBLNAMES (ID,NAME) values (:ID,:NAME);';
  
  Query.Params.ParamByName('ID').AsInteger := 1;
  Query.Params.ParamByName('NAME').AsString := 'Name1';
  Query.ExecSQL;
  
  Query.Params.ParamByName('ID').AsInteger := 2;
  Query.Params.ParamByName('NAME').AsString := 'Name2';
  Query.ExecSQL;
  
  Query.Close;
  Query.Free;
end;

Notice that this example requires more code than the example without the parameters. Then what is the use of using parameters? Speed is one of the reasons. The example with parameters is faster because the query is parsed only once. TSQLQuery only parses the query once, but also the database server parses the query only once. Most database systems support parameters. Whenever a query is used more than once with different values for the parameter each time, then the database server only parses the query and plans the query only once making execution considerably faster. Use 'TSQLQuery.Prepare' to determine the moment the query is parsed and planned by the database server. Use 'TSQLQuery.UnPrepare' to make sure the query is parsed and planned every time by the data server.

Another reason to use prepared statements is prevention of SQL-injection (see also Secure programming.

Finally, in some cases it just simplifies coding.

See also