The I/O of it all. Another abstraction layer example.

Recently I have been working on some "one-hit-wonders" programs that are needed just once. Working in the "COTS" like systems for years have me always thinking long term this & long term that. There is a reason the term "legacy" is used to describe existing systems; they have a VERY VERY long life, well beyond the original developers. So, my program using the latest pieces of the .Net XML Navigator and scans through a tons of flat files reacting to various things discovered in these files. My huge backlog of programming experience made me first think "Oh man, this program will take a long time to finish." Man I was so wrong.

Then I started to remember an article (ftp://ftp.research.microsoft.com/pub/tr/TR-2004-136.pdf) read a few years ago written by Jim Gray (currently in the MS R & D division). He has been fundamental in the RDBMS arena for years. This article was all about "fast" file reading and writing. For a moment I was thinking, what the heck is a RDBMS guy care about file I/O. Duh, everything is ultimately written to disk dummy! For simplicity sakes, all "good" computers must be turned off every now & then & therefore will lose all in-memory data. This made me realize that Relational Databases aren’t necessarily "faster" in the pure sense of performance (with all other things being equal - which isn't ever the case). But it is easier. That is one of the main goals of abstraction.

As I see it, lower level solutions attempt to handle the "ugly" details of it's layer leaving only exposed the common utilization scenarios for the consumer. Databases are no exception. Given the time constraints, learning curves, lack of skill sets (sorry to say), and elaborate business problems to solve; leaving the file I/O of the data to someone like SQL Server or Oracle isn't a bad idea. Why re-invent the wheel right? I agree with that, I guess I just get a little annoyed from time to time with the mystical impression people have of RDBMS. Please note, I am not saying I could do any better for a "commercial" product or that I don’t have respect for the development that went into these kinds of products. I am just not a fan of the "magical" black box mentality that exists in many development shops related to this topic.

So do you think I offended every DBA in the world? I hope I didn't offend all of them. J

Testable by Design?!?

A few years back I was at a Rational Conference in Philly and sat in on a presentation by Sam Guckenheimer titled "Quality by Design". We all have heard of Performance, Scalability, Robustness, Configurability, and even CA's "QWAN" being factored in during the design phase, but what about testability. We all have seen so many times in our career where testing gets "trimmed" in the project schedule once the rubber meets the road. Are there things that we architects can do to help preserve some dignity in this ever-shrinking phase? Modularity, Use Case driven, Unit Tests (like NUnit or the likes) and of course our lip service to the cause can all be elements to help preserve this phase, but I wonder what else can we do. With the rise of SOA architecture we see even more of a need to "prove" out a single service/layer/slice (you pick the term). Everyone's canned response is to just have the developer build the tests as they build the main code. We know that it doesn't always play out like that. The way I see it, many of the challenges with the "automated" testing support systems, is that is still requires MORE effort than management is willing/able to invest. The crux is the lack of context. We still need humans to add the context to most of the tests in order to be useful. What if, I am just thinking aloud here, MS could add some additional Attributes that a tool could reflect one, have a user enter some additional context to (in addition to message signature & the likes) so we could easily piece together a string of Unit Tests with setup & tear down to create useable system tests. These additional metadata would need to be centrally located and workable by the non-technical business analysts in order to contain any business value. Not free just cheaper. Sorry, I digressed, back to the Architecture. Any thoughts on what we could build into the core design to make testing less of a chopping block victim?

An Architect's Non-Technical Technical Challenge

When designing new computer systems (as least for businesses) I believe that most significant challenge for an architect is to match up the correct technical solution with the non-technical users. Yes it is important to get the best ROI, TCO, address key decision makers' interests, and KNOW enough of the different technologies out there to meet the functional requirements. However, what really makes or breaks a new system is whether the PEOPLE embrace it. We all know that change is uncomfortable and difficult for most to overcome but it is even more challenging when you mix techno-phobia into it. Trying to understand the person as well as the problem is very important. I have often wondered if an Industrial Sociologist would be a good addition to a company like mine. We repeatedly sell, re-engineer, and transition different clients onto new business software systems (mainly ours). We are change specialist with most of our emphasis on technical expertise and much much less on people. [Please note, I am not bashing my current employer; this observation applies to every similar company I have encounter - regardless of whether I was on the payroll or not.]