Tech Life of Recht

The coming of a new year usually attracts a number of predictions, some more accurate than others. I’m no good at predicting, so I’ll give my wishes for the new year instead:

• Get my Macbook Pro to work with the DVI-VGA adapter (done)
• I’d like to be convinced that the dynamic languages are really the way forward. I see that they can do smart stuff, but I sorely miss some kind of help using whatever APIs are developed, besides the source code, and I can’t get used to working without a proper editor. I can probably live without the editor, but having methods like def initialize (options = {}) just hurts my eyes. I haven’t used Ruby/Rails that much, but I’ve already dug around more source code than I care to in order to find out what the legal options were.
• Release Java 7 with proper closures. This probably won’t happen, and even when it does, it will take a couple of years before it’s adopted.
• Already, I know that I’m going to spend a good part of my time fiddling around with huge WSDLs and schemas. Could somebody find a way to stop the black hole that is Web Services?
• As for Linux on the desktop, that will still not happen, even though it should, especially when you look at the new distributions such as Ubuntu. However, I’m pretty satisfied, now I just need xmonad to support dynamic twinview, so I can add a monitor without restarting X
• And to Apple: please take corporate customers seriously and make a docking station for the Macbook (Pro). I’m getting tired of plugging and unplugging 5 different cables every time I have to move my laptop.
• Less focus on creating big monolithic systems, more on distributing across autonomous nodes.
• Finally, it would be great with a proper vi-mode for Eclipse – like vimperator for Firefox.

I recently discovered that using the DVI adapter with the Macbook Pro wasn't as easy as I thought. The MBP comes with a DVI connector, which is great, and it works just fine when you plug a DVI monitor into it. However, when doing presentations, you almost always have to use a regular VGA connector. Even though the driver (nvidia) could see the screen, it refused to display anything.
Now I've finally worked out a solution, and luckily it's not that complicated. The trick is that even though the monitor is recognized, it is still treated as a DVI monitor. In order to get the driver to treat it as a normal VGA monitor, add this to xorg.conf:

Of course, the modes should be adjusted according to the setup. The MetaModes line should contain the valid combinations, separated by ;. In this case, I've specified that the internal display should always run at 1440x900, and then I either have a DVI monitor connected at 1680x1050 or a VGA monitor at 1280x1024. Of course, it's also possible just to specify one monitor if you're not using TwinView.

The trick now is that when running in X, xrandr can be used to switch between the monitors. When running xrandr, it should display something like this:

The tricky part now is that the nvidia driver "hides" the monitor configuration in the refresh rate. For example, the 1280x1024 resolution has only one rate, which means that this resolution is unique to one monitor. Switching to this mode with 'xrandr -s 1280x1024' will enable the VGA monitor and disable the internal display.
However, the mode 2720x1024 has two refresh rates, 52 and 50. Unfortunately there's no easy way of telling it, but the 52 rate is with the internal display and the VGA monitor on, while the 50 is with only the internal display on (and a larger desktop, something you probably don't want to use). Switching on the VGA monitor then requires two arguments: xrandr -s 2720x1024 --rate 52.

This isn't the easiest to work with, so I've created a small Perl script which can query the nvidia driver directly to find the available modes, and then call xrandr. Download the script here. To use it, the X11::Protocol Perl package must be installed, and of course xrandr and the nvidia driver. To use the script, simply call it without any arguments to get a list of the supported resolutions and monitors. Select a configuration by calling the script with -s where id is one of the numbers in the first column. That's about as simple as it gets.

xmonad 0.5 is out, and I haven't upgraded my installation in a while. It's pretty nice that configuration is now done in /.xmonad/xmonad.hs, then there's no need to recompile xmonad all the time. The configuration change did require a reorganisation of my old Config.hs, so I took a look at some of the samples to see what other people did.
I ended up with something which looks like this:

This is my xmonad.hs:

I use a simple shell script for starting xmonad:

Finally, the dzen2 status bar is run using this script:

Two awk scripts are used - one for the memory usage and one for CPU frequencies. Why awk? No idea, it just seemed like a good choice at the time.

memstatus.awk:

I like Google Web Toolkit. Well, I could probably even say love, but I don't want to go too far. However, one thing does irritate me: I usually build GWT projects using my Ant target, and I also use Ivy for dependency management. This means that I have no external dependencies, all jar files are downloaded automatically by Ivy and placed in lib/build. To make everything as easy as possible, the Ant classpath is just defined to be lib/build/*.jar, instead of singling out every one of the jar files.
This works pretty well, until you start using GWT, a recent version of Xerces, and want to do XML schema validation. Depending on the OS, different versions of Xerces are loaded, which results in spurious classpath errors - methods cannot be found, classes doesn't exist, operations not supported, and so on.
This goes on until you realize that GWT (in the form of gwt-dev-*.jar) includes all dependencies in a single jar file. This includes Ant, Xerces, other XML apis, and quite a lot of other stuff. I realize that it's easier to distribute, but why oh why can't we just get a regular jar file with the GWT classes in it? Anyways, in case anybody should be interested, the gwt-dev-*.jar file can be unpacked, anything but com.google.gwt deleted, and then these Ivy dependencies can be used to enable compilation:

This also removed the OS dependency from gwt-dev, so the same jar file can be used for all operating systems.

The project I'm working on now is using Oracle for data storage. Not only that, it's good old 9.2. Try installing that on your brand new Ubuntu. Oh well, it's doable, but recently we've begun discussing if we could run it on MySQL or PostgreSQL (I'm voting for PostgreSQL, but that's mostly because I used MySQL back when foreign keys and other advanced features weren't exactly implemented, and in the meantime, I've come to like PostgreSQL. At the very least, hitting Ctrl-c in psql does not exit the prompt like mysql does).
The way it works now is that if something goes wrong, we can either call Oracle or our strategic Oracle partner, and they'll fix it. This is just about everything I'm against: paying large amounts of money just because then we can blame somebody else when things screw up.
There are basically two issues with switching to a more light-weight database, and they're probably more or less the same on many projects. First of all, performance. Oracle probably has a good advantage there, especially in handling large and complex queries. Secondly, maintenance, especially backup and failover.
The only one of these points which I think is valid is backup - it doesn't really help if it takes a week to backup or restore data. However, when that's said, we're left with two things: Performance and reliability. The Oracle way (I'm using this term as the broadest possible. Replace Oracle with MSSQL or DB2 if you like) is to add more memory, disk, clustering, high availability. Expensive, but you get to live in your little world where you can just write code against one large database.
The other way, which I prefer (and which is probably in the Web 2.0 spirit) is to distribute. Distribute both data and processing to a number of autonomous nodes which can operate independently of each other. This is no news, and has been done many times, but it's not something that's normally considered when building good old business applications.
The result of distributing is essentially that you think about how you're accessing your data. Instead of just delegating al of the work to the database and hoping for the best, you're actually forced to analyze data relationships to discover separate components. If this succeeds, the choice of database should no longer be about whether it can optimize a query over 50 tables with subselects, type conversions, views, functions, and other stuff, but if it is efficient at looking up simple data relations. My guess is that all the popular databases can do this, so then you're free to choose the cheapest, the one which is easiest to work with, or whatever suits your environment.

Returning to the project I'm working on, we have some reservations about some of the queries we're executing. They're on Oracle syntax right now, but can probably be converted to regular SQL in a finite amount of time. That doesn't make them any smaller, but we have a good amount of pretty static data (addresses, classifications, and so on), which are retrieved together with the more dynamic data. It the static data is removed from the SQL, we end up with some much simpler queries, which shouldn't be a problem for any database engine. The problem which remains is how the static data is retrieved effeciently. At the moment, I'm leaning towards a solution where we implement a service which can take a list of data keys and return the data. Depending on the amount of data, the service can then be implemented as a in-memory map, a memcached cache, or maybe even something like Hadoop. No matter what, basing the model on a basic principle of isolating static data from the dynamic, and only querying the dynamic data seems like the way to go as a first step - and as a nice side effect, the dependency on the database's ability to perform doesn't matter that much anymore.

This probably just sound like drunken ramblings to those who have actually implemented distributed business systems, but bear with me, it's a first for me, and I need to get things out of my head before the space runs out.

I've updated the GWT Xdoclet module to support typeArgs for collections. This is used when returning a collection from a service method:

Notice that the object type is not written in <> as you would do if using GWT directly.

The new module can be downloaded here: xdoclet-gwt-module-0.5.jar.

I've been scheduled to do one of Trifork's Software Pilot JAOO meetups. It's a simple concept: we choose a topic, invite some speakers (or use one from Trifork), and people show to listen to and discuss various topics. It's free for all, so do join us. More information is available at trifork.com. The topic for the last meetup was application security, which Kresten presented, and the next time it will be about software modeling with focus on color modeling.

I find color modeling interesting because it's fun to work with, and it's very simple to learn and teach. Of course, it helps that it puts those colored post-its to use, but my experience also tells me that it's a good technique in a number of cases.

More on color modeling at the meetup. Right now, I'd like to reflect a little upon the project, I'm on at the moment. We're currently developing a new system for The Danish Medicines Agency which will be used for all medication ordinations in Denmark. Right now, if you're admitted to a hospital, you better be awake, because you're basically the only one who know what kind of medication you're on. If you can't tell the doctors yourself, there's no way to know what medication you've received recently. This is what the new system is trying to solve.

Now, the system is pretty simple, as it's basically a data repository accessible via web services. We receive medication data, store it, and send it out again when requested to do so. The web service interfaces were defined long before we started development, so that wasn't a concern to us (beyond using them). The usual (or enterprisey) way to proceed would be to

• Install Axis or something alike
• Generate code based on the WSDL and XSD files
• Receive these types at the topmost layer and convert them to some kind of value objects or data transfer objects
• Map the value objects to a database using Hibernate, EJB3 or another O/R mapper

Inspired by Steve Loughran and Edmund Smith's Rethinking the Java SOAP Stack, I suggested that we skip the code generation and value objects steps and just used XML and JDBC directly. This didn't exactly receive a warm welcome, but after discussing it more, everybody more or less agreed that the value of the generated code was not exactly clear. Also, there's been some trouble with Hibernate, especially in very large databases and when writing queries spanning a large number of tables with complex joins.

This means that the system is now based on basic JDBC (through Spring JDBC) and XML. The Java DOM API isn't exactly a pleasure to work with, so I wrote a pretty simple wrapper class for it, which can do something like this:

In other words, a very simple api for accessing XML structures. In setValue, elements are created automatically if they didn't exist, and the get methods will never throw a NullPointerException. Compare this with a similar DOM call where any number of NPEs can occur:

How do we then ensure that we actually generate valid XML? The answer is heavy unit testing and schema validation. Just like you should do in any case.

We've now reached the point where we've implemented most of our web services, and for once, I don't have the feeling that I've spent my time writing stupid value classes with no functionality. The jury is still out on whether the design is actually good and scalable, but I think it's a good approach. Should everybody do it? Probably not. But it definitely shows that you shouldn't just accept common orthodoxy and do the usual enterprise system without reflecting upon how your model will be influenced.

By the way, this system will be released under an open source license (probably Apache 2.0 or Mozilla Public License 1.1) at Softwarebørsen, the Danish government's open source repository.