Developer Blog

The Billion Dollar Mistake

John Janzen — Fri, 31 Aug 2007 19:15:43 GMT

I am typing on a new laptop. This laptop came with Windows Vista Home Premium, but I was enticed to upgrade to Windows Vista Ultimate. I bought the Vista Ultimate license online from Microsoft. I followed all of the instructions, upgraded all my drivers, trimmed my nose hair, and ran all the tests and evaluation programs like a good OCD programmer should. I gritted my teeth, crossed my fingers, and started the upgrade.

The upgrade failed. It ran for nearly two hours, showing great progress, and then displayed a dialog box that said, “windows could not configure one or more system components” with a lone OK button. I pondered this for a time, meditating on “Ok”. It wasn’t Ok. No error codes, no help button, no web link, just Ok. So I clicked Ok, and I was restored back to where I started just like the Shute’s and Ladders game.

I searched the Internet for someone who had solved this problem. I checked for error messages in the event viewer. I worked over more drivers, more windows components, and everything else I could think of. Then I tried it again. Two hours later it was Ok and back to where I started from. At this point I began to imagine the trial of the person responsible for that dialog box. I was feeling a little vindictive.

No one is surprised to learn that Microsoft plans to make a lot of money selling Vista. They have a new money making strategy. The strategy is to sell the Operating System once to nearly everyone in the world, then sell it to nearly everyone again a second time. What is more, the price is much higher when you buy it the second time. How are they going to do this, you might ask? Well, first they put in much more license validation so Microsoft gets money from every copy. That gets everyone to buy it the first time rather than using some pirated copy. Then they offer enticing features to get people to upgrade to an improved edition of the software. The upgrade is not offered at an OEM discount price, but you get to pay Microsoft directly the full amount. Their strategy worked on me. Maybe it is time to buy Microsoft stock again. There is only one problem: the upgrade doesn’t work.

Somewhere there sits a programmer responsible for this. This programmer was given the responsibility to validate some windows components as part of the upgrade process. I imagine the code looks something like this: 1) get a list of all the components, 2) check them one by one to confirm they will work with the new OS, 3) display an error message if one doesn’t work. That is about it; it shouldn’t be very complicated. I have only one complaint. If your program knows what component is going to cause a problem, why don’t you let me in on that fact? I might be able to do something to fix the problem!

Widgets are one of the cool features of Vista. They look like oversized icons on your desktop and they display things like speedometers or maps or stock tickers scrolling by. At this point I am thinking someone should write a widget that displays the location of this programmer on a Google map with a little orange balloon following them as they look for a new job.

I called Microsoft support to resolve my Vista upgrade problem. They seemed very familiar with what I was trying to describe. In fact, I asked my friendly Indian support person if they received many calls about this issue. He said, ‘Yes, no problems, it will be ok. Everyone here is talking on the phone solving the same issue.’

I am wondering if this upgrade problem will become one of the great failures in the history of programming. There are several reasons why I think it deserves this distinction. The first is the number of people impacted by this bug. The second is that it really makes Microsoft look bad, with their new money making efforts and all. The third reason is that the problem was made so much worse because some programmer didn’t add just a little information to the error message. It is the third issue I find full of irony.

Error codes have been added by programmers to their error messages since World War II. COBOL programmers used to be famous for their ability to pour through memory dump listings produced when an error occurred. They could find a single problem in pages of green bar hexadecimal listings in seconds as if they were an idiot savant. Error messages are one of the few programmer traditions. Unfortunately, as we all can experience now with a windows upgrade, this tradition isn’t passing on to the next generation of programmers.

Modern programmers have not developed this skill because they don’t have to. We use debuggers, which are a great improvement except for one thing. Debuggers can only be used on a development environment. A debugger can help you find problems when the code is still being developed, but once it is out there in the real world and something goes wrong the developer cannot just show up in the middle of dinner like the cell phone network people and ask to borrow someone’s computer for a few days.

You may be wondering how to make the upgrade work. Here is what worked for me. From the Control Panel, Open the Programs section to Uninstall or Change a program. Then click on Turn Windows features on or off, which is text listed on the left hand side. In the dialog that opens, expand Microsoft .NET Framework 3.0 and uncheck the XPS Viewer option. Next, uncheck Remote Differential Compression. Finally, uncheck Windows DFS Replication Service. You may also want to run msconfig and check the selective startup, then choose not to start any services other than system services. If you have been good about checking all of your drivers, startup services, and stray hairs you just might successfully upgrade your computer.

Database Performance and Traffic Lights

John Janzen — Fri, 31 Aug 2007 19:14:48 GMT

In the 2003 remake of "The Italian Job", Seth Green plays the character Lyle, alias "The Real Napster". He cracks the computerized Los Angeles traffic system and brings traffic to gridlock. By turning on and off the traffic lights, he creates hopeless snarl across LA - accept for a special route that allows the good bad guys to deliver justice to the real bad guy.

My Aunt is convinced local traffic is caused by someone like "The Real Napster" changing traffic signals. The people that run the spy satellites are also a possible culprit. She likes to remind me that the spy satellites are always watching, often while looking up as if she could catch one in the act. If not these sorts, where does the traffic come from? Everyone is driving home or to work, so why do they have to slow down? Cars seem happy to drive fast at lunch time. Why is it that every once in a while, everyone is parked on the freeway some Sunday evening at 11pm?

The modern view of a database allows Ruby on Rails or the latest Java Framework to connect database tables to an AJAX web client and everyone is thrilled at how little time it took. Framework developers all claim they have finally found Fred Brooks� silver bullet. They begin to extrapolate on the trend of modern technology and wonder how many decades before vacations to the moon will be affordable to what used to be called the middle class. The Post Modern skeptic acknowledges the power of modern hardware and its ability to hide yesterday�s performance problems - up to a breaking point. The Post Modern skeptic believes design problems will always be difficult, which is what Fred Brooks actually said. The Post Modern skeptic accepts the truth.

The basic problem with traffic is that people want to be where someone else already is. Traffic slows as people wait for their turn to move their car into one of the coveted spots, usually the middle of some intersection. Traffic lights mark intersections where people are moving across a stretch of road in conflicting directions. One or more directions have to completely stop while cars move across the road in the other direction. I am convinced everything you need to know about database performance and tuning you can learn from road traffic, maybe even while waiting in traffic.

Database performance is just like street traffic. Someday I need to talk to the programmers that make the Sims games about this. The database with its tables is like a city with its roads. The processes that run on the server in response to someone�s query or update statement is like a little person in a car, traveling along the different tables. A database allows many different processes at the same time. Each process is a little car on its way to work. Traffic intersections are where different tables are responding to more than one process at a time. Congestion happens when too many cars intend to travel through the same intersection at the same time. The Sims programmers could do something with this idea.

There are three main issues affecting database performance, and each has a corollary in the traffic world. The first issue is establishing structure behind where businesses, shopping malls, and Starbucks are located relative to where people live. The second issue is the where one allows roads and more importantly where roads are restricted. The third issue is the placement of traffic lights, because not every intersection may require a light. With the database, these issues are called normalization, stored procedures, and locking.

Unfortunately, I can only hope "The Real Napster" and the satellite people can get together and fix our traffic. But I can help programmers understand how to improve database structure and create less congestion for the little Sims cars. The next three articles cover the main techniques one can use to improve database performance.

The Post Modern Database

John Janzen — Fri, 31 Aug 2007 19:11:49 GMT

Looking at vacuum ads from 1950 almost makes one want to be purchasing a new vacuum cleaner. Everything was so wonderful. People were so happy. Life was simple and getting easier with every purchase of an electric appliance. Somewhere, post war heroes were arriving at the vacuum cleaner factory early in the morning wearing hats. They would happily assemble vacuum cleaners all day so they could buy appliances for their coiffed 'home-maker' wives, who were themselves rapidly assembling future baby boomers.

Appliances truly are wonderful because I don't have to think about how they work; I don't have to solve anything. I love the dish washer. I put in dirty dishes. A few hours later I pull out clean dishes. Oh! The new mini cookies that replace dish washer soap are so cool! It nearly makes me want to do the dishes just so I can take the little cookie out of its plastic wrapper and put it to bed in its little drawer in the dishwasher door.

Database servers can be viewed as an appliance. They run on their own like a dishwasher. Four commands are all you need: insert, update, delete, and select like controls on a front panel. You put in data and then pull it out again, sorted or joined with other happy factoids. Somewhere else the DBA makes sure all the factoids get backed up regularly. It is all planned, organized, and full of happy, efficient factoids like a community designed by Walt Disney.

Popular Object Relational Mapping (ORM) tools strengthen the appliance image of the database server. These tools look at the tables you built on the database and then create the object-oriented code you need for your CRUD (hip nerd-speak for create, read, update, and delete). Tools like Ruby on Rails take the appliance image one step further, taking the database table to an AJAX web interface faster than you can run a mile.

From the beginning, database servers were intended to be trouble-free and easy to use. The father of the relational database model, Ted Codd, was trying to make a database something that could stand alone apart from the application. His primary goal was the independence of the database from the application. His paper from 1970 states in the introduction, In contrast, the problems treated here are those of data independence - the independence of application programs and terminal activities from growth in data types and changes in data representation and certain kinds of data inconsistency which are expected to become troublesome even in nondeductive systems. Admittedly, Dr Codd's idea of easy to use or easy to understand may have been different from the rest of us.

Like a good summer novel, there is a subtle, nagging problem lurking. Many programmers have experienced nothing but the database as an appliance that makes their work short and their romantic interludes long. But in smoky bars and dark alleys one finds a few programmers with very different experiences. These programmers talk in raspy voices about performance problems, object-relation impedance, and even deadlocks. These are the survivors of death marches; projects that started out as just another walk in the park but turned into nightmares of changing specifications, long hours, unhappy customers, and growing site popularity followed by horrible performance problems.

These tales are true stories. Performance demands unravel the 1950's image of the database as an appliance. Older, battle-scarred programmers are like post-modern critics of the simple, modern hope. Databases can work like appliances under reasonable demands. But the database server as an appliance is not the unadulterated truth some confident voices make it out to be.

The modern view of a database allows Ruby on Rails or the latest Java Framework to connect database tables to an AJAX web client and everyone is thrilled at how little time it took. Framework developers all claim they have finally found Fred Brooks' silver bullet. They begin to extrapolate on the trend of modern technology and wonder how many decades before vacations to the moon will be affordable to what used to be called the middle class. The Post Modern skeptic acknowledges the power of modern hardware and its ability to hide yesterday's performance problems up to a breaking point. The Post Modern skeptic believes design problems will always be difficult, which is what Fred Brooks actually said. The Post Modern skeptic accepts the truth.