Pages

Showing posts with label Databases. Show all posts
Showing posts with label Databases. Show all posts

Wednesday 6 August 2008

Storing Binary Data in PostgreSQL



PostgreSQL™ provides two distinct ways to store binary data. Binary data can be stored in a table using the data type bytea or by using the Large Object feature which stores the binary data in a separate table in a special format and refers to that table by storing a value of type oid in your table.

NOTE: YOU SHOULD NOTE THAT POSTGRESQL DOES NOT HAVE A BLOB (Binary Large Object) DATA TYPE. SO DEFINING A COLUMN AS A BLOB FOR THE STORAGE OF IMAGES OR ANY OTHER FILE FOR THAT MATTER WILL RESULT IN AN ERROR.

In order to determine which method is appropriate you need to understand the limitations of each method. The bytea data type is not well suited for storing very large amounts of binary data. While a column of type bytea can hold up to 1 GB of binary data, it would require a huge amount of memory to process such a large value. The Large Object method for storing binary data is better suited to storing very large values, but it has its own limitations. Specifically deleting a row that contains a Large Object reference does not delete the Large Object. Deleting the Large Object is a separate operation that needs to be performed. Large Objects also have some security issues since anyone connected to the database can view and/or modify any Large Object, even if they don't have permissions to view/update the row containing the Large Object reference.

Version 7.2 was the first release of the JDBC driver that supports the bytea data type. The introduction of this functionality in 7.2 has introduced a change in behavior as compared to previous releases. Since 7.2, the methods getBytes(), setBytes(), getBinaryStream(), and setBinaryStream() operate on the bytea data type. In 7.1 and earlier, these methods operated on the oid data type associated with Large Objects. It is possible to revert the driver back to the old 7.1 behavior by setting the property compatible on the Connection object to the value 7.1. More details on connection properties are available in the section called “Connection Parameters”.

To use the bytea data type you should simply use the getBytes(), setBytes(), getBinaryStream(), or setBinaryStream() methods.

To use the Large Object functionality you can use either the LargeObject class provided by the PostgreSQL™ JDBC driver, or by using the getBLOB() and setBLOB() methods.
Important

You must access Large Objects within an SQL transaction block. You can start a transaction block by calling setAutoCommit(false).

For example, suppose you have a table containing the file names of images and you also want to store the image in a bytea column:

CREATE TABLE images (imgname text, img bytea);


To insert an image, you would use:

File file = new File("myimage.gif");
FileInputStream fis = new FileInputStream(file);
PreparedStatement ps = conn.prepareStatement("INSERT INTO images VALUES (?, ?)");
ps.setString(1, file.getName());
ps.setBinaryStream(2, fis, (int)file.length());
ps.executeUpdate();
ps.close();
fis.close();

Here, setBinaryStream() transfers a set number of bytes from a stream into the column of type bytea. This also could have been done using the setBytes() method if the contents of the image was already in a byte[].
Note

The length parameter to setBinaryStream must be correct. There is no way to indicate that the stream is of unknown length. If you are in this situation, you must read the stream yourself into temporary storage and determine the length. Now with the correct length you may send the data from temporary storage on to the driver.

Retrieving an image is even easier. (We use PreparedStatement here, but the Statement class can equally be used.)

PreparedStatement ps = conn.prepareStatement("SELECT img FROM images WHERE imgname = ?");
ps.setString(1, "myimage.gif");
ResultSet rs = ps.executeQuery();

while (rs.next())
{
byte[] imgBytes = rs.getBytes(1);
// use the data in some way here
}
rs.close();
ps.close();


Here the binary data was retrieved as an byte[]. You could have used a InputStream object instead.

Alternatively you could be storing a very large file and want to use the LargeObject API to store the file:

CREATE TABLE imageslo (imgname text, imgoid oid);

To insert an image, you would use:
// All LargeObject API calls must be within a transaction block
conn.setAutoCommit(false);

// Get the Large Object Manager to perform operations with
LargeObjectManager lobj = ((org.postgresql.PGConnection)conn).getLargeObjectAPI();

// Create a new large object
long oid = lobj.createLO(LargeObjectManager.READ | LargeObjectManager.WRITE);

// Open the large object for writing
LargeObject obj = lobj.open(oid, LargeObjectManager.WRITE);

// Now open the file
File file = new File("myimage.gif");
FileInputStream fis = new FileInputStream(file);

// Copy the data from the file to the large object
byte buf[] = new byte[2048];
int s, tl = 0;

while ((s = fis.read(buf, 0, 2048)) > 0)
{
obj.write(buf, 0, s);
tl += s;
}

// Close the large object
obj.close();

// Now insert the row into imageslo
PreparedStatement ps = conn.prepareStatement("INSERT INTO imageslo VALUES (?, ?)");
ps.setString(1, file.getName());
ps.setLong(2, oid);
ps.executeUpdate();
ps.close();
fis.close();

// Finally, commit the transaction.
conn.commit();


Retrieving the image from the Large Object:
// All LargeObject API calls must be within a transaction block
conn.setAutoCommit(false);

// Get the Large Object Manager to perform operations with
LargeObjectManager lobj = ((org.postgresql.PGConnection)conn).getLargeObjectAPI();

PreparedStatement ps = conn.prepareStatement("SELECT imgoid FROM imageslo WHERE imgname = ?");
ps.setString(1, "myimage.gif");
ResultSet rs = ps.executeQuery();

while (rs.next())
{
// Open the large object for reading
long oid = rs.getLong(1);
LargeObject obj = lobj.open(oid, LargeObjectManager.READ);

// Read the data
byte buf[] = new byte[obj.size()];
obj.read(buf, 0, obj.size());
// Do something with the data read here

// Close the object
obj.close();
}

rs.close();
ps.close();

// Finally, commit the transaction.

conn.commit();

NOTE: This is an extract from Chapter 7, PostgreSQL JDBC Documentation. Please download the whole documentation from the PostgreSQL JDBC website.

Thursday 6 December 2007

MySQL

MySQL is arguably the world's most popular open source relational database management system (RDBMS). It is also the world's fastest database server (Well, on this point, this is arguable, I know Oracle pops up in your mind). MySQL is just simply too cool for anyone to ignore. Even those fellows at Oracle, Microsft and Sybase have taken serious notice of it. MySQL has many open source cousins but PostgreSQL and Firebird stand out to me.

OK, it is true that MySQL is not the most powerful database server available, but hey, who needs battleship strength (and expense too) of Oracle when you can carry out over 1000 simulatenous transactions on MySQL, all for the price of nothing? Huh? MySQL is freely downloadable from http://www.mysql.com

However, you should be able to contend with Command line interface. If you can't stand a command line interface, then you would want to download the MySQL client GUI( Graphical User Interface). The GUI tools contain a suite which includes; MySQL Administrator, MySQL QUery Browser and MySQL Systems Tray Icon and other programs. All these help you in creating schemas (databases), creating the tables, editing accout privileges and so much more. The GUI tools are also freely downloadable.

You can also manage your relationships and schemas using MySQL Workbench which is a freely but separately downloadable program from the same site as above.

You may notice that there are two different types of the MySQL server. The current server version is Version 6.0. The two types include the free GPL MySQL Commmunity Server and the commercial MySQL Enterprise Edition. The Enterprise edition is an extension of the Community Edition. This, however does not imply that the Community edition should be rubished. It is equally powerful.

The Enterprise edition is USD $200. This should be cheap enough for the enterprises and businesses as compared to the over USD$1000 for Oracle or MSSQL server.

If you try to download the free Community EDition but come up and can only find the commercial Enterprise Edition with 30 day trials, well if you intend to buy the commercial edition, then you can accept the terms and try it out before buying it. However, if you are short on finances, then locate the Community link in the home page of the MySQL website and from there you should be able to find the free editions.

If you intend to use MySQL with applications such as Java, C++, Python, PHP, Perl or any other use such as dynamic website development, then you may wany to also download the MySQL database drivers from the same site. Just read on within the site. It may also be a good idea to become a member of the site as well by registering freely.

By the way, MySQL is the most widely used database server for web applications (websites). Even Google uses it( At least I am mean't to understant so).

So, basing on all this, I strongly recommend MySQL for both commercial and non commercial use