Generating Components From Tables and Queries

Quite often I find it useful to have a ColdFusion object which reflects the data in the row of a query or database table.

I have a written a generator to accomplish this. Please note that this is CF 9 specific. But wait, you say, CF 9 has ORM built in why not just use that. There are two limitations of Hibernate as implemented in CF 9 which rule it out for my use. 1)It is limited to a single datasource and 2) it will not work on an arbitrary query.

We have an application for which different clients have different datasources. The database schema is the same for each client but the database instance, and hence datasource is different.

Also, we often move data returned by a query around the application and to the client side to be used by jQuery. It is much easier to use an object for this than using queries directly.

The Generator

The generator retrieves information about the table or query, and creates a cfc reflecting that structure and writes it to the appropriate location in the file system. The generator is designed to be run once and need not be run again unless the schema changes. If it is rerun it respects changes made to the cfc by the developer and does not overwrite them.

I will discuss the generator code a bit later but lets look at a typical cfc that is generated first. After all if you don't find the generated cfc useful why would you care about how it is generated?

Sample cfc of the artists table in cfartgallery

component datasource='cfartgallery' primarykey='ARTISTID' pkIsIdentity='true' extends="com.vawter.utilities.baseTableClass" accessors=true {
/*************Properties generated: Oct 9, 2011 Do not modify. If schema changes rerun generator *************/
    property type='numeric' name='ARTISTID' datatype='CF_SQL_INTEGER' ;
    property type='string' name='FIRSTNAME' datatype='CF_SQL_VARCHAR' ;
    property type='string' name='LASTNAME' datatype='CF_SQL_VARCHAR' ;
    property type='string' name='ADDRESS' datatype='CF_SQL_VARCHAR' ;
    property type='string' name='CITY' datatype='CF_SQL_VARCHAR' ;
    property type='string' name='STATE' datatype='CF_SQL_VARCHAR' ;
    property type='string' name='POSTALCODE' datatype='CF_SQL_VARCHAR' ;
    property type='string' name='EMAIL' datatype='CF_SQL_VARCHAR' ;
    property type='string' name='PHONE' datatype='CF_SQL_VARCHAR' ;
    property type='string' name='FAX' datatype='CF_SQL_VARCHAR' ;
    property type='string' name='THEPASSWORD' datatype='CF_SQL_VARCHAR' ;
/************init generated: Oct 9, 2011 Do not modify here. Modify in usercode************/
    public com.vawter.databaseobjects.artists function init(
        numeric ARTISTID= 0,
        string FIRSTNAME= "",
        string LASTNAME= "",
        string ADDRESS= "",
        string CITY= "",
        string STATE= "",
        string POSTALCODE= "",
        string EMAIL= "",
        string PHONE= "",
        string FAX= "",
        string THEPASSWORD= ""
    ){
        super.init(argumentCollection=arguments);
        return this;
    } //end init
/************Start User Code*************/
/****Code placed here will not be changed if you run generator again.***/
/**** You may override the update,add,delete,read,save methods here.***/
/*** You may also define any new methods or overide the init method. ***/
/*** If you override the init method be sure to call the super.init method ***/
/************End User Code*/

} //end component

Notice the metadata attached to the component: datasource, primary key, pkIsIdentity. These were collected by the generator when it was run and are used when interacting with the database. The datasource is the datasource used when the component was generated. It is used by default but you may set a new datasource at runtime if necessary.

Also notice that accessors is set to true. When CF sees this it generates getters and setters automatically. If the developer overrides one of the accessors in the "usercode" section the code "getter=false or setter=false" is added to the appropriate property when the generator is rerun.

The first thing you may notice is that the code contains no useful methods. It is basically a list of properties reflecting the structure of the table. Notice that these properties incorporate the database type from the database. The component extends com.vawter.utilities.baseTableClass which contains all the methods and does the heavy lifting.

Methods inherited from the superclass:

  1. setDatsource(string ds)
  2. getDatasource()
  3. add()
  4. update()
  5. delete()
  6. read()
  7. save()
  8. fillFromQuery(query q)
  9. fillObjectFromQueryRow(query q,numeric row)
  10. fillObjectFromStruct(struct inStruct)
  11. getJson()
  12. getJsonFromArray(array aObj)

The purpose of the first two methods is to change the datasource. The next five should be obvious. They are just the standard CRUD operations.

The next three are methods for populating the object. fillFromQuery is a little different. It creates an array of objects, one for each row of the supplied query. Each instance is of the same type as the calling object.

getJson returns a string of the serialized data in the object and getJsonFromArray returns a string of the serialized data from an array of objects (usually from fillFromQuery).

All the methods except 1,2,11,12 return a struct with elements result and errormessage. Result is true or false and if false errormessage is the reason. fillFromQuery returns an additional element "records" which is an array of objects populated from the query.

To use the read method you first populate the object with the pk you want to fetch and then call read. The method queries the database for that record and updates the object. Values other than the pk which were in the original object are ignored.

The CRUD methods are only useful for tables and views. It makes no sense to "add" to the database if the object reflects the data from an arbitrary query. Also the table must have a primary key or the CRUD operations will fail.

Sample usage of the generated code

local.o=new com.vawter.databaseobjects.artists(email='johnny@squirrel.com',firstname="Johnny",lastname="Squirrel");
    local.o.setState("CO");
    iocal.o.save();
    
    The above creates an instance of artists, setting values for email, firstname,lastname and state and then saves it to the database.
    Since we have not specified the pk this will actually do an insert into the database.
    
    local.o=new com.vawter.databaseobjects.artists();
    local.o.setArtistId(12);
    local.res=o.read();
    
    If local.res.result is true then local.o now contains the data for artistid=17 from the database.
    
    local.o=new com.vawter.databaseobjects.artists();
    local.res=local.o.fillFromQuery(q);
    
    If local.res.result is true then local.res.results is an array of objects of type artists, one element for each row in query q.
    
    local.o=new com.vawter.databaseobjects.artists();
    local.st=local.o.getJson();
    
    local.st now contains the serialized string of the data contained in local.o
    (obviously you would populate local.o before calling this method or you will just get the default values)

    local.o=new com.vawter.databaseobjects.artists();
    local.res=local.o.fillFromQuery(q);
    local.st= o.getJsonFromArray(local.res.records);
    
    local.st now contains the serialized string for all the data from the objects created from the query q

The Generator Code

The generator code is straightforward. After collecting necessary file locations and the desired component path, it collects the metadata for the table or query and information about the defaults, generates the code and writes it to the file system.

Collecting metadata for tables and views

The tag provides information about the table structure and default values.

private struct function getColumnInfo(ds,ta){
        variables._datasource=arguments.ds;
        variables._tablename=arguments.ta;
        variables._action="columns";
        include "incdbinfo.cfm" ;
        var q= variables.resultQuery;
...

Unfortunately cfscript does not have an equivalent for cfdbinfo. To use it and retain script notation we use an include statement: include "incdbinfo.cfm". incdbinfo.cfm is very simple:

<cfparam name="variables._datasource" default="">
<cfparam name="variables._action" default="columns">
<cfparam name="variables._tablename" default="">
<cfif not variables._tablename eq "">
    <cfdbinfo datasource="#variables._datasource#" name="variables.resultQuery" table="#variables._tablename#" type="#variables._action#">
<cfelse>
    <cfdbinfo datasource="#variables._datasource#" name="variables.resultQuery" type="#variables._action#">
</cfif>

If one had an action other than "columns" one would not have a table name, hence the if statement.

Collecting metadata for a supplied query

For a query, dbinfo is not available. We can, however collect information on the columns and data types by executing the query and then using the metadata.

private array function getMetaDataForTable(required string datasource,required string table,string qSQL=""){
        try{
            var q=new query();
            q.setDatasource(arguments.datasource);
            var    tablename=arguments.table;
            if(arguments.qSQL==""){
                q.setSQL("SELECT * FROM " & tablename & " where 1 = 0");
            }else{
                q.setSQL(arguments.qSQL);
            }
            var qResult=q.execute();
            var md=getMetadata(qResult.getResult());
        }catch(any ex){
            writedump(ex);
        }
        return duplicate(md);
    }

For queries qSQL is the developer supplied sql. Since this generator should not live in a production environment we are not overly concerned about sql injection.

Notice the "where 1 = 0" statement. We need no records in the result set to get the metadata so we insure that none are returned.

The metadata returned is an array of structs, one for each column. The struct contains the column name, the column data type, and whether the data is case sensitive. The column names are the same case as the database. For the examples in this post they are all upper case but in the general case they are not. We preserve the case when generating code so our properties reflect the underlying database case. This makes reading the code much easier and means that when we pass the data to javascript we are certain of the case of the variable names. Of course within CF the case of the property is not important.

Sample Usage of Generator

local.o=new com.vawter.utilities.DBObjects('cfartgallery');
local.res=local.o.tableToObject(componentpath="com.vawter.databaseobjects",table="artists");

local.res will return a struct with 3 elements a)result(true|false), b)errormessage c) the text of the cfc generated The text of the cfc generated is just for informational purposes. The real code is written to the file system You will probably want to import the generated code into your version control system.

Summary

It is easy to create objects representing tables or queries. The components generated are simple with all the methods containe in a common superclass.

You may download the code for the generator here.

In a future post I will discuss moving objects to and from the client using jQuery.

Comments and suggestions welcomed,

Comments
Andrew Scott's Gravatar ColdFusion 9 ORM does indeed work with multiple datasources, are you 100% sure you have upgraded to CF9.01?
# Posted By Andrew Scott | 10/9/11 7:29 PM
Don Vawter's Gravatar @Andrew You can now use multiple datasources but you cannot choose the datasource for a component at run time which the use case we need. To use the built in ORM for clients using 10 different datasources we would need to have an identical component (differing only by the datasource) for each client and construct a new component as new datasources were added.
# Posted By Don Vawter | 10/9/11 8:40 PM
Andrew Scott's Gravatar Actually you can, but to do this you need to modify the this.datasource at runtime as well. I have deployed 2 applications using ORM that uses the same entities across different datasources using this method.
# Posted By Andrew Scott | 10/9/11 10:16 PM
Steve's Gravatar Not sure if you have thought about this (and to be honest I haven't really thought it through 100%) but what about having your generator create 2 cfcs. One called _Artist.cfc where all the generated stuff goes, and another called Artist.cfc which extends _Artist.cfc where any user created stuff can go - including overriding default getter/setters.
This means you can re-generate the _Artist.cfc file whenever you want .. just a thought. From memory apache cayenne uses this approach (or something like it).

Cheers
Steve
# Posted By Steve | 10/9/11 11:00 PM
Don Vawter's Gravatar That certainly would work. Since the generator runs in miiliseconds regeneration is not a big issue. The only issue with the current approach is the developer must put their code between the start and end comments. I suppose that could cause problems if a developer doesn't bother to read the comments in the generated code.
# Posted By Don Vawter | 10/9/11 11:13 PM
BlogCFC was created by Raymond Camden. This blog is running version 5.9.4.001. Contact Blog Owner