Yann's Blog - Software and hardware

April 9, 2008

The database in a database

Filed under: IRM 2,Software — Yann @ 2:42 pm

I’ve been quiet about my IRM 2 activities recently, and I apologize. This article is not really a status update, but a reflection on mapping programs such as IRM 2 to a traditional RDBMS.

I’ve determined that I just built a database… inside another database. The IRM 2 core currently has a core table, which is quite literally an ID, Column, Value mapping table. The data simply looked liked this:

A100, 1, Name, Computer
A100, 1, Color, Blue
A100, 1, Owner, Bob

Every property simply gets dumped into this table, and requires extensive remapping on the application logic. The RDBMS doesn’t have much insight into schemas, as it subverts its notion of schemas. Searching for explicit property values requires joining huge amounts of data together. Data storage is inefficient, with records scattered throughout the datastore (and in many places on the disk), especially if records were updated after creation.

More and more I’m starting to feel that approaches such as BigTable, SimpleDB, and CouchDB are taking the correct approach to the large data storage problem. The data stored is as a series of documents with arbitrary and per-document schema. Each node is a replicated element, and data storage is not centralized but rather distributed.

The advent of Google’s AppEngine supporting this model of database reaffirms that this is the correct direction to move in for large scale online applications.

Now, what does all of this have to do with IRM 2 and gIRM? Stay tuned – I’m trying to find out myself :)

January 3, 2008

Whats New in girm.git (IRM 2)

Filed under: IRM 2,Software — Yann @ 11:55 pm

Just a quick update highlighting what is going on in girm.it, the Git repository for IRM 2 (on Grails).

  1. It is now possible to create, view and browse Assets based upon Schemas, which are user defined.
  2. You can create Schemas and Fields in a Schema
  3. The functionality for linking CRI (Cross Reference Identifiers) is in place – needs more work on the GUI front.

screenshot-irm-assets-list-by-type-mozilla-firefox.png
While gIRM isn’t even into alpha stages, its showing a lot of promising functionality already. Next up is a first hash of the Ticketing support system (Tracking mk 2), as well as “personalities”.

Personalities are an abstraction to Schemas which allow you to associate sub-schemas with a master schema. For instance, you can create a “Network Connectivity”personality and associate it with both a printer and a computer. Personalities can also impart custom logic to the application – such as link graphs in the case of a Network Connectivity personality. The mechanism for this isn’t quite designed yet ;)

Feature suggestions are always welcome!

December 11, 2007

gIRM Git Repository Available

Filed under: IRM 2,Software — Yann @ 11:50 pm

In addition to trying out new Kool-aid with Groovy and Grails, I’ve decided to move from tracking in Subversion to tracking in Git the gIRM code base.

The gitweb is available here: http://stackworks.net/cgi-bin/gitweb.cgi?p=girm.git;a=summary

and cloning is available at

git://stackworks.net/girm.git/

December 9, 2007

gIRM – IRM on Grails

Filed under: IRM 2 — Yann @ 11:37 pm

Just a quickie.  A few screenshots presenting IRM on Grails. Developed much faster than my previous solution, and in general making me feel a lot better about its direction

.screenshot-irm-assets-mozilla-firefox.pngscreenshot-irm-show-schema-mozilla-firefox.png

November 30, 2007

Development insights from IRM 2 and other Java Web Projects

Filed under: IRM 2,Software — Yann @ 12:22 am

Its me again, and this time I bring news of lessons learned with the current development work on IRM 2.

Firstly, while EJB 3.0 is miles better than EJB 2.1, it doesn’t match up to a proper IoC container. By IoC, I am talking both about the Tapestry 5 IoC container, and the Spring IoC container. I find that with writing service methods with EJB, I spend far too much time both massaging domain objects between the EJB layer and the Tapestry layer – the Hibernate/JPA session is no longer valid outside of the EJB layer, so forget about all the “cool” stuff it does, such as lazy fetching, and wallow in pain replicating that functionality.

Second, component based view models are powerful, but not quick to grasp. I grew up (shame shame) on PHP, doing in-line HTML output. I know, this is bad, and I know better. I’ve grown accustomed to other view technologies such as templates (Smarty, Cheetah-Template, Velocity, etc), but still find myself taking a long time developing a workable component based model. This may be a shortcoming in Tapestry, but its taken me far longer to develop views than I would like. I will say that the component model is stronger than JSPs, but sometimes quick & dirty JSP gets you there faster. Some of this may be my experiences with Tapestry 5 talking, which I will discuss further below.

Third, Hibernate is amongst the best ORM wrappers available and a major time saver. Compared to other ORM wrappers out there (like ActiveRecord), it has an unmatched feature set, often backed by very mature Java technologies and libraries.

Java is a good language, and a fantastic environment. Yes, the language isn’t that great – overly verbose, no closure, a little too static. But the Java libraries available, especially on the server-side are un-matched from other web-development languages, including my favorite language, Python.

Groovy is a very promising language. Closures, duck-typing, first class JVM language, and more. Libraries that should have existed ages ago (XmlSlurper comes to mind). If you’re going to do quick development in a Java environment, or looking at a scripting solution in Java, give Groovy a strong look.

Whatever you do, avoid the creation of your own GWF. Also known as the Ghetto Web Framework. We’ve all seem them – you started out with some servlets, and went from there, making half-baked utility functions, some copy and pasting, etc. You get a maintenance nightmare – changes which should simply be a two-liner end up being propagated in odd places in your GWF, requiring exponential time to change. Plus your GWF is so tightly coupled to your application,  you can’t use it for the next application. There are so many web frameworks out there – please use one :)

Groovy on Grails makes an incredible rapid development environment. Coupled with Groovy, and with idea taken from  Ruby on Rails amongst other systems, you get an incredible Java development environment which is more productive and quicker to develop in than EJB, Tapestry, and maybe even the venerable RoR. Plus, its pure Java nature coupled with libraries such as Hibernate and Spring, you gain all the advantages of scalability and support that you just don’t get in RoR.

Wicket and Tapestry are very well organized frameworks. While I talked somewhat negatively about component based frameworks above, I would like to mention that both of these frameworks are fantastic in their own right. Wicket seems to be gaining a lot of momentum as well. But, on a negative note, Tapestry 5 needs more documentation. You can buy 200 books on Struts (though why you’d want to use it is beyond me), 3 on Grails, 1 (and soon 2) on Wicket, and 0 on Tapestry 5. Tapestry 5 really needs more developer momentum and support to come to fruition. Howard Lewis Ship is doing a fantastic job on being very careful and deliberate in his design and crafting of Tapestry, but he is after all one person, and can not do it all.

So, you may ask,  what does this have to do with IRM 2?

Well, I’ve been once again at the crank. I’ve realized that the code I wanted to avoid writing (simple CRUD), I am writing again, primarily through my mis-choice of EJB3. Partly contributing to this is Tapestry -  I’m writing a fair number of CRUD views – none of which are in any way exciting.

I’ve been converting the existing code from Tapestry/EJB to the Groovy on Grails framework.  Its been amazing easy to accomplish, and the Grails scaffolding support has been an amazing productivity boost.

Look for a developer’s release by the end of the year .

Till then

-Yann

September 29, 2007

IRM 2 First Alpha with Tapestry 5.0.6

Filed under: IRM 2,Software — Yann @ 12:15 pm

Just a quick update: I’m trying to push IRM 2 first alpha out once I’ve converted over to using Tapestry 5.0.6 (from 5.0.5). I’m finishing up the initial storage manager in the mean time and hope to have something usable soon :)

August 15, 2007

1.6 Beta2 out, IRM2 work continues

Filed under: IRM 2,Software — Yann @ 10:27 pm

Just keeping everyone up to date – work on IRM2 is continuing. I’m trying to wrestle the beast which is the abstract storage manager into something, well, more manageable.

In the mean time, a second beta of IRM 1.6 has been pushed to SourceForge. Please test and comment!

July 23, 2007

Moving

Filed under: IRM 2,Software — Yann @ 9:55 am

So, whats been going on? Well, I’ve moved cities over this past month, and still not totally done, which has put a damper on software development efforts to say the least. I’m going to resume working on IRM1.6 and IRM2 in a short while, once the last big item, the reef aquarium, is moved.

I’m upgrading my humble little living room reef from a 55 gallon to a 90 gallon system, on a nice looking stand. I’ve built a custom acrylic sump (easier than you think) and am still in the process of plumbing it, filling it, etc. Then I get the joy of moving the whole system next weekend from one city to another.

Stay tuned!

June 13, 2007

IRM 1.6 beta/RC coming soon

Filed under: IRM 2 — Yann @ 10:07 pm

We’re just busy wrapping up some silly bugs in the IRM 1.6 codebase and trying to get an RC out the door ASAP.

Stay tuned.

May 30, 2007

IRM 1.6

Filed under: IRM 2,PHP,Software — Yann @ 10:04 pm

Work is progressing well on an interim release of IRM, to be called 1.6, from the original code base.. This is where most of my development time has been going these past two weeks. This is an effort to provide useful fixes and functionality now, instead of in “the future”. But working on it, despite many fixes from contributors such as Matt Palmer and Martin Stevens, is like trying to tame a beast. But at least we’re making progress ;)

Feel free to hop on the irm-devel mailing list in order to check out current work, or look at the regulary updated Demo: http://www.stackworks.net/irmdemo/demo/

Login in Tech with the password tech (this is case dependent).

« Older Posts

Powered by WordPress