Bendy classes and dynamic programming

; Date: Wed Feb 22 2006

Tags: Java

Jack Herrington has published this article: (web.archive.org) Going dynamic with PHP claiming to show some things PHP can do which Java can't.

Okaaaay... This is another of the articles in the meme that dynamic languages are great, and rigid languages are uncomfortable. I don't know if I got the analogy right this time, let me know please?

The problem he poses is about implementing the database interface classes. The traditional method might be to write one class per database table, expose get/set methods for each field, etc. You'd end up with a tedious implementation and a lot of typing of highly similar code. He poses three possible solutions, and claims that PHP implements the best of the three.

One problem with the article is there's more than three possible solutions. The problem with the proposed solution is it doesn't give early warning of broken code, leading to greater expense to fix coding problems.

Whever a writer presents a problem .. and then presents a selected list of solutions .. the writer (maybe unknowingly) is trying to limit the readers thinking to the given solution set. But what if there are other solutions the author doesn't present?

What's another solution? One that comes to my mind is to use Java's annotation feature. I am imagining an annotation you attach to each field in the object which declares the database column corresponding to the given field. A class-level annotation would declare the table to which the class corresponds. I can imagine this class could be autogenerated from the database without the programmer writing code. The only actual methods to write would be get/set methods. You could have a persistence library which introspects the class looking for those annotations centralizing the JDBC code to the library rather than spreading it among the database classes.

Seems pretty simple ... and it appears (web.archive.org) hibernate makes it even easier than that.

Of course using annotations requires JDK 5 (or later).

As simple as this approach might be to implement, it doesn't offer some flexibility Jack Herrington's article discusses. Namely, dynamically generating the object interface at runtime. I suppose that's even simpler, maybe. Certainly it lets the developer more quickly implement the code. But, I think this solution has problems of its own which Jack skips over.

The problem with a dynamically generated object interface is you don't know about any code mismatches until runtime. Meaning, you might not discover certain bugs in the code until the application is out in the field.

With Java you have compile time checking, and if some code refers to a field or method that's no longer there the program will fail to compile. That may seem burdensome but that's also A Good Thing. It tells the programmer up-front about a coding problem, letting the programmer more quickly fix it. It's well known that quickly identifying a bug makes it much less expensive to fix the bug than if the bug is identified out in the field (e.g. a program crash) and has to wend its way back through technical support and bug triage before it can be fixed.

With dynamically bound fields or methods ... okay, take this scenario in mind. Some code in the application refers to a field/method in a class. Then later there's a revamp of the code, a database table changes, some columns are deleted, and the class no longer has the matching fields or methods. Now, take your application that used that spiffy dynamic binding which made it a lot easier to do the initial implementation ... and what will happen. Will the programmers find all instances of references to those fields or methods?

I can almost gaurantee you the programmer won't. But the compiler would find them all -- that is, if the compiler were written to do compile time checking, like Java's is.

UPDATE: Apparently I missed this, but Jack says towards the end of the article the technique he presents isn't without its problems.

Source: (web.archive.org) weblogs.java.net

Comments

Comments are listed in date ascending order (oldest first)

I think you do this person a disservice. He's just opening people's eyes to an alternate view. Surely not a bad thing? And he does say at the end: "Dynamic objects have a lot of power, but they also carry significant risk. First, the complexity of classes increases tremendously when you start writing magic methods. These classes are harder to understand, debug, and maintain. In addition, as integrated development environments (IDEs) become more intelligent, they may experience problems with dynamic classes such as these because when they look up methods on a class, they won't find them."

Posted by: goron on February 22, 2006 at 11:56 PM

Agree. I'd put it even a little bit sharper: This kind of scripting stuff may be handy for building small and simple apps (category: my first guestbook, whoa), but it is a ludicrous toy compared to carefully crafted, really powerful tools such as Hibernate or JDO. I'm developing medium/large-scale apps (~ 100 tables) on a regular basis. For every model class, the necessary time for implementing the business and persistence logic is a magnitude of order higher than the time for writing the model classes. Typing a few dozens of POJO classes for such a project is a neglible effort compared to the rest, and the Java compiler is my best friends when it comes to the inevitable need to modify the model. Posted by: scotty69 on February 23, 2006 at 01:16 AM

There's thousands of solution for a gived problem... but why recreate the wheel ? just use existing solution or improve it...

JDO is the answer... Posted by: alois on February 23, 2006 at 02:51 AM

Uh, annotations aren't exactly statically checked and enforced by the compiler, either. Yes, you can do a lot of fun things with annotations in Java, like (java.sun.com) http://java.sun.com/developer/technicalArticles/J2SE/constraints/annotations.html shows, but they are giving you just a bunch of metadata, which you can only really use at runtime. And if you give up the benefits of static typing, you may as well use something that doesn't require a paragraph to describe the underlying toolery to make the thing work somehow. :)

cheers, dalibor topic

Posted by: robilad on February 23, 2006 at 04:15 AM

You said: "... you don't know about any code mismatches until runtime.Meaning, you might not discover certain bugs in the code until the application is out in the field.". This is true iff you do not test your code before the application is in the field. Normally you would test your code (be it java, php, ...) and detect and correct such errors. Some kind of errors (type mismatchs) can be detected earlier by the java compiler, which is true. OTOH, if you do not have to type that many getter / setter methods etc., you can start wearlier with testing. And type mismatch errors are neither the type of errors that occur that often nor are they difficult to fix (just my experience). Posted by: zincteapot on February 23, 2006 at 05:09 AM

David, a couple of month ago, I would have fully agreed with your article. But recently I decided to teach myself Ruby, mainly to broaden my horizons. I have come to the conclusion that yes, Compile time checking is a good thing but that there are other tools to detect the code mismatches you mention. I am particularly thinking about Test Driven Development. Unit tests allow, among many other things, the detection of these mismatches in a timely manner. Compile time checking is but one tool in the toolbox.

I would like to highlight that Java can suffer from the same problem when you start using reflection: Class.forName("I cannot be a classname because I contain spaces") will compile, but it will always throw an exception at runtime. Unit Tests would detect this, Compile time checking would not.

I wrote this article to illustrate (web.archive.org) what we could learn from dynamic languages.

Posted by: oansaldi on February 23, 2006 at 06:02 AM

Yes test-driven development is valuable, but you're missing the point that language to language, dynamic languages give you many many more ways to shoot yourself in the foot. If you think unit tests will give you compile-time safety, you haven't worked with any programmers in a while ;) People are notoriously independent, and even when you form conventions there are always cracks that the programmer him or her self will fall through on their own. Unit tests don't always capture these things, and at those points, you're looking at the code. In a dynamic language, you're now looking at how they think. In a language like Java, you're looking at the api they used. That's one less thing you have fudge through in order to find the problem, and it happens, A LOT.

I take pride in knowing why compile-time checking and type-safety is valuable and desirable, and I think people as well if they understand. It speaks to experience.

Yes dynamic languages are great and free. If you really want to broaden your horizons though, stay away from the hogwash of Ruby and try TCL or Clean

Posted by: ilazarte on February 23, 2006 at 08:23 AM

Incidently, if you use Eclipse you already know coding in Java is now faster than coding in a dynamic language. that IDE is like heaven. Posted by: ilazarte on February 23, 2006 at 08:42 AM

ilazarte, you wrote: If you think unit tests will give you compile-time safety. This is definitely not my point. My point is that compile-time safety is overvalued. Please refer to the Java example I gave and you may understand why.

If your unit tests don't always capture these things, I suggest you use a test coverage tool to detect what is not tested. Writing and running tests is the most efficient way of catching bugs, no matter what language.

Finally, I am only expressing my opinion, not trying to impose it. There is no need for personnal attacks.

Posted by: oansaldi on February 23, 2006 at 09:02 AM

"Yes, [TDD] is valuable, but... dynamic languages give yoy many more ways to shoot yourself in the foot." If you are actually doing TDD you aren't going to shoot yourself in the foot, because you are not going to do anything but the thing that simply and obviously solves the immediate problem. The point that you are missing is the less you say the less likely you are to say something that is wrong. Dynamic languages allow you to say less.

"Incidentally, if you use Eclipse... coding in Java is now faster than coding in a dynamic language." (web.archive.org) Have your cake and eat it too

Posted by: xagile on February 23, 2006 at 09:07 AM

has anyone ever calculated how many more unit tests you have to write when you take away the compiler(can it even be calculated?)? Ive seen arguments saying that you don't need a compiler because you can unit test all that it does. For some reason when I think about how usually anything can change at anytime in a dlanguage I don't feel very encouraged about this thinking. leouser Posted by: leouser on February 23, 2006 at 10:56 AM

Unfortunately, not everyone writes throughout unit tests for anything and everything from accessors to mutators. Test driven development works for some, but it does not work for all. No one's unit tests are perfect. Large development teams may miss areas. They may elect not to run tests, or even write through tests, because of time contraints. Some coding houses even...gasp...don't use XP programming or test driven development. Therefore, compile-time safety is of benefit to those who do not, and even of benefit to those who do, unit test. Herron is right on the money that having these checks save a lot of time and frustration up front, rather than later. Posted by: phlogistic on February 23, 2006 at 01:58 PM

I keep a phrase tacked up in my office: Testing is never finished, only abandoned ... that quip is widely understood among software quality groups. There's a budget for testing, and runing out of budget marks the cutoff point for testing.

Those of you arguing for test driven development have a point. A good one.

However it costs time (hence budget) to create those tests. I'm thinking there's an analogous budget constraint in the development end of the house, that the managers want them writing code and might see writing tests as overhead to minimize. It might not be that exact excuse, but there's going to be some budgetary argument which limits the test development done by the developer.

Further, unit testing is not a panacea that solves all forms of testing. Unit testing only tests the atomic level functions, but what about overall function? What about performance? Neither are covered by unit testing.

Posted by: robogeek on February 23, 2006 at 02:21 PM

I can't remember what percentage it is, but I remember in Code Complete that there is a limit on the amount of bugs unit testing(or is it testing in general?) can find. Its argument is that to have good protection you have to have multiple techniques. At least this is how I remember it! :D leouser Posted by: leouser on February 23, 2006 at 02:26 PM

You're right, test driven development isn't for everyone. But, neither is programming. If you can't handle TDD maybe that should tell you something.

The argument that testing takes time is the oldest one in the book. It's also patently false. Releasing buggy software (and then having to fix it) takes way more time and money. Anyone who has actually bothered to learn how to test their own code has seen their productivity increase in the process.

Posted by: xagile on February 23, 2006 at 07:05 PM

"Unit testing only tests the atomic level functions, but what about overall function? What about performance? Neither are covered by unit testing." No. They are covered by customer (aka acceptance) testing, which is equally as important as unit testing. Like unit tests, customer tests can be written before or after the implementation they are testing. The advantage of writing them first is that you then only need to create the simplest implementation that will cause them to pass (In theory that is what you want in the code first approach too, but in practice not having the tests leads to a more speculative form of design where as test first leads to an emergent form of design. In the agile community we have found the latter to be more productive.)

Posted by: xagile on February 23, 2006 at 07:17 PM

You're right, test driven development isn't for everyone. But, neither is programming. If you can't handle TDD maybe that should tell you something.

Shrug...to each their own methodology. Every team I've worked with implement some agile / xp methodologies...but they never implement every theory. Most places skip the "don't plan for the future" XP dogma, as its a little silly to say you're going to build a house without a house plan. Also makes design patterns pointless if you don't have to plan for the future, because with XP you would make every design aspect concrete.

I'm all for unit testing, but realize unit testing will never cover every potential bad situation. And that's why compile-time safety is of benefit. Posted by: phlogistic on February 24, 2006 at 09:22 AM

This has been an interesting comment thread. I'll say for myself that I've never worked in a test-driven-development fashion (while the JDK itself is developed in a pseudo-TDD fashion, I don't work on JDK development) so everything I've said about that strategy here is a guess on my part.

I just saw this article: Agitating Java and testing Windows ... while the article is largely an advertisement for this Agitator product, the intro discussion was interesting and related to the comments here.

e.g. "Creating comprehensive tests is hard work, and test-driven development represents a change in programming culture" ... I work in the Java SQE organization and can attest to the first half of the statement. It's hard to write a good test, one that actually tests something and is a valid test. As for the second half, I'll just say that looking at the TDD strategy is both very interesting (it seems like a good idea) but at the same time represents a major shift in thinking about writing applications. Despite working in the SQE organization today, I've got a long history as an application developer.

e.g. "Manually written tests are also imperfect. Test authors may fail to cover all the possible scenarios, or may include bugs in the tests themselves." and the sqlite author "calculates that 59 per cent of his code base is devoted to testing, covering 97.4 per cent of the code.". Those two points validate what I said about the overhead and budgetary constraints. That is, having 59 percent of the source base as tests indicates a large expenditure of time to develop those tests, combined with the assertion that tests are often themselve include bugs raises questions over whether this is the best use of developer time.

In the Agitator article they clearly want to portray that software as The Solution. But it also validates the assertion I made -- that having the compiler do some checks is a win by itself. Not to diss the type of analysis Agitator does. The compiler can catch some of the bugs, and save some of the programmer effort.

Posted by: robogeek on February 27, 2006 at 10:36 AM

oansaldi wrote: I would like to highlight that Java can suffer from the same problem when you start using reflection: Class.forName("I cannot be a classname because I contain spaces") will compile, but it will always throw an exception at runtime. Unit Tests would detect this, Compile time checking would not. Actually, this is caught by compile time checking.

As you've pointed out, the code you've written will always throw an exception. That exception is checked, and so the compiler will force you to catch it - otherwise your code will not compile.

The net result of this is that the compiler, though it cannot detect when your code will fail, forces you to deal with the failure scenario in case it fails. The compiler knows which statements may fail and makes you do something about it.

While I'm well aware that there's dynamic languages with exceptions, without a compiler there is no way of forcing programmers to deal with the exceptions. So Java wins again in terms of helping you to find potential problems before deployment through compile-time checking.

Graham. Posted by: grlea on February 27, 2006 at 09:21 PM

About the Author(s)

(davidherron.com) David Herron : David Herron is a writer and software engineer focusing on the wise use of technology. He is especially interested in clean energy technologies like solar power, wind power, and electric cars. David worked for nearly 30 years in Silicon Valley on software ranging from electronic mail systems, to video streaming, to the Java programming language, and has published several books on Node.js programming and electric vehicles.