It seems that Bruce Tate is falling for the "if people do it incorrectly then it should be avoided" argument. Mike Clark was using the same argument with respect to comments: "how many times have you been bitten by inconsistent or wrong comments?". (I should mention that Mr. Clark didn't advocate dropping comments completely -- only that they be used when the code isn't clear enough.)

Like Mr. Tate, I too have reviewed much code. There are two mistakes that are quite common in the code I have seen:

• The effects of multiple threads on code are not well understood. Specifically, synchronized is sprinkled apparently randomly over the code.
• Inheritance is used as a code-reuse mechanism rather than an is-a relationship.

Following Mr. Tate's and Mr. Clark's logic, we should eliminate inheritance and synchronized (and probably even threads!).

As I've said before, to me this is all a training issue. I honestly don't expect developers to use exceptions, comments, inheritance and synchronized correctly since they're never trained to do so. Most "figure it out" (coding or "software engineering" that is) as they go along. There are a number of books out there with the necessary information, but unfortunately, there are no "problems to solve at the end of the chapter". Until we address this training issue, I'm afraid that most developers will continue to incorrectly use these constructs.

This is off topic but I thought it to be a good noodler.

Wouldn't it be ironic if all of the UFO's and "alien landings" were time traveling humans from the future? The reasons for abductions were that we wanted to see the genetic makeup of early humans and determine any illnesses that they may have had. "Flying saucers" are used in the future since we're obsessed with them now. In other words, it's logical to assume that we would produce a time machine in the shape of a flying saucer since we are so interested in flying saucers now. The grandfather paradox is incorporated since history already has recorded such events.

I should point out that I am not the first to have this thought.

I'm all for UML to help communicate and serialize ideas in a consistent fashion. There have always been a number of pain points with UML that made me angry. One of them has been in-line guards -- representing conditionals in a sequence diagram. It has been nearly impossible to succinctly represent complex conditonals (e.g. multiple else ifs). UML 2 has introduced the "combined fragment". IBM Developer Works has an article that demonstrates them in action. If you're not up for reading the lengthy article at this time then just check out figure 8. This article uses them in a sequence diagram.

If you use Visio, then these UML 2.0 stencils may come in handy.

I talked about unified messaging some time ago. It seems that Microsoft was thinking the same thing. Based on the reception of this, we will see if I was right about Sendmail, Inc. foolishly sitting on their hands.

Over the years I have had much experience with the trials and tribulations of forming a company and the impact that this very important yet much overlooked step has on its well-being. My experience comes from forming my own companies (with multiple partners), various M&As, and from working from the VC's side of the table.

I often equate a business partnership with marriage. One marriage site has their top four reasons for a failed marriage as: poor communication, financial problems, a lack of commitment, and a dramatic change in priorities. These four reasons can be easily translated over to a business partnership. How many marriages last beyond one year or five years these days? We've all heard horror stories about a divorced couple fighting over possession of assets. Imagine the difficulty in dividing up a business partnership.

It is common, especially when forming your first company, to think that "everything will just work out" and that any partners that you're forming with will behave rationally and inline with your desires when necessary. This isn't always the case. Rather than rushing into a partnership, take the time to fully vet out all concerns, responsibilities, and expectations. Write everything down so that there is never the case of "well you said".

There are a number of questionnaires available online (which, of course, I cannot find at this time) that all partners should answer and then sit down and discuss. This allows all involved parties to see how well their visions for the company coincide. The questions might include:

• Where will starting capital come from and what assets are you willing to bring in?
• Where is the priority of the company in your life? (An important question in here would be in relation to existing or future marriage and/or children.)
• How will the interests of the company be divided? (It's not always equal between partners as each partner may not bring equal amounts to the table.)
• How are decisions made and, more importantly, how are disagreements handled? (This is very important in 50/50 partnerships. What happens if you both disagree? And yes, it will happen.)
• Is the goal to build the company and then sell (short term) it or grow it over many years (long term)?
• How do you decide when to throw in the towel? (Planning for success requires that you plan for failures.)
• How does a partner leave the company and what ramifications are there?

Don't jump head first into a new partnership. The time that you take in the beginning will more than amply pay itself off with the headaches it will prevent later. Take as few chances as possible and understand what you are getting into. Shop around and find an attorney and an accountant that all the partners like. They can be invaluable in helping you over all of the startup hurdles.

I have been reading a number of articles on unit tests and test driven development (TDD). I even attended a few unit test specific talks at the NFJS Java Symposium to make sure that I was up to date on all of the latest and greatest. Unit tests and TDD are all the craze these days and everyone is touting their benefits. But there are also a number of pitfalls that must be understood before entrusting your life with these safety nets.

Pitfall 1: Parallel Concerns

While writing a unit test for class Foo you think of cases A through F. These six cases represent the whole of what you are concerned about for Foo. Feeling confident in the tests you being coding. As you are coding the implementation, you're are thinking about cases A through F since that's the whole of the concerns that you know about.

In the middle of coding you realize that there is a case G. G is not in the tests you already wrote so you add a unit test for it. So now, as far as you know, the functionality is completely covered by cases A through G.

But what if there are cases H and I? Since you didn't think of them initially, they're not in the unit tests. Since they didn't become evident as code was written they are likely not accounted for. If you're lucky, cases H and I will be implicitly tested and/or coded for. But what if they're not?

The concerns of testing and coding tend to be parallel to eachother; if you thought of it, then it will be coded and tested. The test and the code will typically approach the problem in the same way. What about orthogonal concerns -- things that you didn't think of or aspects that are only uncovered when approached from a different angle?

With coding experience, one gains a sense of knowing what you don't know. These aspects can usually be boxed into a corner, documented and possibly explicitly tested for. We're all human and this knowing what you don't know certainly isn't quantifiable so there are going to be orthogonal concerns that are missed.

Pitfall 2: Invalid or not Useful

Some time ago I was working with a third party library. I wanted to provide another implementation for some custom functionality. I was in luck; the code was structured in such a way that there was an interface right where I needed it. And to top things off, there were even unit tests provided with the interface. This was going to be a good day!

I started off by running the unit tests on the existing (functional) implementation. I was rewarded with green bars; my safety net is in place. Sure, I don't know what the unit tests are acutally testing for, but there are a number of them and the current developers use them. I will learn what the tests do as need arises and I will add my own as necessary.

I started to stub out my own implementation. After a few hours, I have all of the basics in place. Now's a good time to see how far off base I am. I run the tests and all shows up green! How good can this day get?

But wait! I notice an error in my logic that should not produce a correct result. Well, maybe the unit tests don't explicitly test for that case. I'll make a note to write a test.

But something just doesn't feel right so I quickly stub out another implementation that just retuns default hardcoded values. I run the unit tests and all is green. What the ....?!?

After devoting a heafty chunk of time to weeding through the existing unit tests I discovered the problem: the unit tests were all negative tests. In other words, each test checked to ensure that something wouldn't happen. Since my code never returned null or threw an exception, they all passed. Fifteen tests that made up about one thousand lines of code served to tell me nothing useful in my case.

Pitfall 3: Requirements Match

The business logic for computing the results of some portfolio requires the use of a spatial partitioning tree structure. You approach the problem by first writing a suite of tests. These tests are robust and cover all corner cases. Due to the complexity of the algorithm, the tests even go as far as to parse the internal tree structure to provide that extra degree of certainty.

After a week of development time, you and the rest of the engineering staff are confident in the logic. The software is pushed off to quality control and the results come back: zero defects. The obligatory pizza and beer party is thrown for this extraordinary feat and the engineering staff goes home for a well deserved weekend of rest.

On Monday, still riding the high of the previous week, you arrive early only to find a post-it on your monitor telling you that the client rejected the software. You feel your heart sink. How could this be?.

After some investigation, it was determined that your understanding of the tree structure and that of the clients differed drastically. The software was certainly free from defects but the results did not match the requirements.

Conclusion

All of the pitfalls have the common theme: don't let the green light lull you into a sense of false confidence. Unit tests may provide a safety net but make sure that the net is directly under you and that it isn't just lying on the ground. Also make sure that if the tightrope is frayed and does break that the safety net isn't made of the same faulty material.

Eclipse has this wonderful feature called Dynamically marking occurrences (along with "Highlight method exit points" and "Mark locations of thrown exceptions"). (No, I don't care if IDE X has the same features.) What if we took this to the next level?

The idea is to highlight an object and select "Mark Synchronizations" (or something suitable). This would show all methods, statement, etc that are synchronized on the object. The outline and hierarchy views would also indicate other classes and methods that synchronize on the object. This would allow you to know if some other object was synchronizing on the same object. Being able to see all code that synchronizes on an object would greatly simplify writing and maintaining threadsafe code.

One could extend this idea even further by allowing the marked occurrences to be sticky. Select object one and mark its synchronization occurrences then select object two and mark its synchronization occurrences in a different color. The intersection of the two sets could be highlighted in a third (combined) color to indicate points of possible deadlock.

One of the major stubling blocks to realizing the promise of Intentional Programming (IP) is building up the data structure that abstractly represents the source. Fortunately, Eclipse's AST provides most of the necessary functionality.

The next and largest stumbling block is deriving a suitable representation for visualizing and manipulating "code" in a more abstract way. The technology is in place to implement the representation: Eclipse's GEF. Unfortunately, there is no already defined representation that provides the necessary granularity to effectively manipulate "code". UML falls short unfortunately as it does not provide the necessary degree of granularity.

I suppose that my idea is that we need an idea for an abstract, yet granular representation of "code". I will, at some point in the near future, write up some requirements for this representation and provide some example code that demonstrates the difficulty of the task at hand.

I have been thinking a bit lately about the process of coding. At this point I believe that there are two distinct phases: blank page and non-blank page. The "blank page" phase is when a new component is being added -- essentially, you have a "blank" editor window. The "non-blank page" is slightly harder to narrow down, but in essence, it is the point at which you have something coded. (I'll leave something undefined at this point.) The act of typing out language statements (i.e. "coding") appears to be most effective during the blank-page phase. You will be hard pressed to find a WYSIWYG, drag and drop editor that allows a more rapid entry of logic and expressions. (Please note that I'm not saying that a drag and drop editor would be slower in cases of building up a class stub, for example. I'm referring to for loops or general if logic and the like.)

Code and coding appears to be less effective once something (still undefined) has been written -- the non-blank-page phase. Attempting to understand large-scale structure and to perform change analysis is extremely difficult when just looking at code. Comments combined with code help facilitate change analysis quite a bit, but still fall short of a magic bullet.

Let's approach this from a different angle. The primary reason for comments is to inform you or another developer of the intent, purpose, concerns, etc of the code so that when modifications or debugging are performed, all concerns and implications are known. (Please note that I'm only concerned with non-API defining comments in this disucssion. API comments are those only used by "external" developers to use an API.) In other words, comments aren't useful in any way during the blank-page phase. Comments are only added to be useful in the non-blank-page phase.

If you think about a standard development hierarchy, more senior developers are doing the blank-page phase work (as well as some early non-blank-page work. This is where the ill-defined "coded something" will eventually derive its definition from.) whereas junior developers are typically thrown into maintenance (certainly non-blank-page work). This creates an environment whereby those that are least able to write effective comments are the ones that must be writing the comments. I say that senior developers are least able because they have had more time away from code maintenance and only by doing regular code maintenance does one know what are the most effective comments to write.

I've just thrown out a whole bunch of ideas that may seem orthogonal. I'll summarize:

• There appears to be two distinct phases of the process of coding: blank-page and non-blank-page.
• The act of coding (writing language statements) appears to be best suited for the blank-page phase.
• Code (the output of the blank-page phase) is ill-suited for the non-blank-page phase. Coding itself is of dubious use during the non-blank-page phase unless new features are being added (in which case, one could argue that this is a form of "blank-page").
• Comments provide important information that is not expressible in code.
• Comments are of little to no use in the blank-page phase but are paramount in the non-blank-page phase. (Please note that I am omitting comments used for describing an API (e.g. javadocs) in this discussion.)
• Comments must be specified in the blank-page phase and maintained during the non-blank-page phase (as part of the code maintenance).

All of this screams dichotomy. Code and comments, the output of the blank-page phase, appear to be the least suitable inputs to the non-blank-page phase. Code ("language statements") is written since it appears to be the fastest way of "data entry" for the blank-page phase but code is not suitable for change analysis during the non-blank-page phase.

Couple this dichotomy with the average times spent in each phase. If you look at "pre-1.0" vs. "post-1.0" times (development time vs. maintenance time) you can see numbers like 1:2, or 1:10. In other words, maintenance time (non-blank-page phase) far outweighs the initial development (blank-page phase'ish) time. Yet all of our techniques, our "tools", are best suited for the blank-page phase.

Perhaps we're approaching code the wrong way.

For those familiar with the concept of Intentional Programming (IP), a light blub may have lit up. I'm actually talking about a level above IP. One can make the argument that Eclipse (with the AST) combined with the EMF (well, some yet-to-be-written instance of the EMF) provides for the IP paradigm. The problem is that an AST view of IP does not effectively incorporate comments.

While I was at the NFJS Symposium I heard from a number of speakers stories about how they were bit by inaccurate comments to the point that they now use comments sparingly. As one may be able to tell from my other postings, this goes contrary to eveything that I believe in. To me, comments are a training issue. If developers are trained effectively so that comments and code are really one in the same then there's never a problem of having comments be out of date. But given that comments cannot be checked for accuracy, there's always the possibility of a refactoring (especially one done automatically though an IDE) that produces inaccurate comments. Does this mean that comments should be eradicated? No. It simply means that more diligence is needed. For example, before an automatic refactoring is checked in, each change should be checked for accuracy. This is good practice in any case since there is always the possibility that the refactoring had implications that were not initially understood. But how many developers do that? Again, solved by training.

The slides to my presentation Computing to Support Scientific Research: How to stay Focused on the Science, not the Software are available through the following link:

PowerPoint Slides

The file contains both the presentation slides and text (in the notes area of each slide). There are a number of useful references and links in the presentation to get you started on your own research.

Once I get my PDF writer back working again, I will put up a PDF of the presentation.

My background is in physics and math. I can remember my first advanced mechanics course. The professor was solving a rather innocuous looking problem. The first black board was filled with equations quickly followed by the second, third and fourth. The professor appeared to be a magician waving his wand over the board as technique after incredulous techique flowed from him. Integration by parts followed a deft use of completing the square. On and on it went until forty-five minutes and six black boards later the problem was solved. I was left breathless. Clearly the black arts had their hand in this weaving of advanced mathematics and only artists such as Jackson Pollack could see the beauty in what was created.

Was this a display of art or well practiced skill?

Before we move towards an answer, let's define succinctly what is meant by art and what is meant by skill. Using Merriam-Webster as a reference, art implies a personal, unanalyzable creative power whereas skill stresses technical knowledge and proficiency.

At the time I witnessed that display of mathematical prowess, given my limited experiences, I will admit that I believed I was watching an art. There was simply no way for me to imagine that the succession of techniques coupled with a vision of the desired solution could be known a priori.

While taking the mechanics course I was also taking differential equations and another advanced mathematics course. Over the period of the three courses I began to see that there was no man behind a curtain controlling a mystical presence. Practicing technique after technique hundreds of times in different progressions slowly built up a type of vision. Initially I could not see step B from step A unless I was told the technique to use and actually worked through to the result. Eventually, I was able to see step B without being told which technique to use. I then progressed to seeing step C and D from A without ever writing down a single equation. Directly from step A I was able to see if applying one technique would lead me to a dead end three or four steps ahead. Through the systematic practice of techniques I was able to develop a skill.

Had I taken the mathematical courses before the mechanics course, I would have never believed that I was seeing an art. It was necessary for me to first build up a skill before I could comprehend what was going on (thereby ingraining the meaning of the word prerequisite indelibly into my brain).

Software refactoring is a relatively new technique. One could argue that it has only been recently with the introduction of appropriate tools (i.e. IDE's with refactoring support) that refactoring could be effectively practiced. With such a short existence, it is no wonder that refactoring appears more like an art than the systematic practice (a skill) that it actually is. What's more is that most software engineers do not have the knowlegde of the desired end result.

Let us contrast this with the mechanics story from above. Most academic mechanics problems are stated in the following way:

Determine the equations of motion for X to move along surface defined by Y under the influence of force Z.

"equations of motion" have a succinctly defined form -- you know exactly what the form of the results should look like. One simply has to take the givens and manipulate them until they are in the desired form. But what is the desired form in software engineering?

Using quality as a basis, it is clear to me what the desired end result of refactoring looks like. Unfortunately, it is not a simple expression as in mechanics. It is a careful weighting of many factors including coupling, clarity, size and so on. To compound the problem even further, each factor is not well defined. What is "clarity"? How is "coupling" measured? What is good coupling and what is bad? None of these questions have a single answer and attempting to find the composite of these multivalued solutions may just be NP-hard.

Is there a solution to this refactoring problem? I believe that there is. But before we can begin looking for solutions, we need to better understand and define the problems. Once we finally make the jump from:

Refactoring is an art

to:

Refactoring is a skill that requires much practice and experience with maintaining the end result

then we can begin to find objective means from which solutions are derived.

At the core I believe that this is a training issue. Whereas I spent years manipulating equations so that I could see the results before ever writing a single equation, CS students do not spend years writing and maintaining code. Unless you have spent considerable time understanding and working with the implications of various results how can you being to refactor code?

I'm going to go off-topic and mention something that struck me the other day. As I was making my wife a pot pie (we alternate cooking duty) I wondered: why do we all have to speak venusian?

Allow me to briefly digress and state that unless you're a modern "enlightened" man, this entry will make little sense to you. You just go on sitting on the sofa (I almost gave away my midwestern upbringing and said "couch") drinking beer and farting followed by the statement "How about that one honey?".

I was startled by the number of situations that require all parties to conform to the venusian view. Why is the venusian view the preferred one? I came up with a number of theories about why men conform. The first starts at the lowest level of Maslow's Hierarchy of Needs. You know ... *wink* *wink* (or at least a better chance of *wink* *wink*). You can easily ascend the hierarchy to come up with the other theories.

Personally, I conform to the degree that is necessary to find a local minima in the time dependent nagging function. I should stress that this time dependence is not just hourly or monthly as most men already know (or have learned very quickly) but changes quite gradually over a period of years. That thing you do that used to be "cute" or "funny" early in a relationship has gradually increased its weight in the nagging function (typically unbeknown to you). What's interesting in this case is that in venusian the time dependence is a given. It almost seems as if the time dependence of the nagging function is trying to reach a state of maximum venusian. Early in the relationship, some marsian is accepted but as time goes on the marsian is domainated by venusian. Physical proof of this can be easily seen in a couple that has been married for a significant number of years and is deemed a "happy couple" (interestingly, now that I think about it, it's women that are deeming them "happy"). I'm a people watcher and I enjoy watching the interaction of people. One of things I have noticed is the interaction between elderly couples in the mall. The elderly men follow the women like puppies and do everything that is expected of them (wait patiently off to the side, hold the purchases, hold the woman's purse, etc). These men have clearly reached a state of minimum marsian.

Just how prevalent is this necessity to conform? The canonical example of the proper position of the toliet seat identifies how pervasive venusian is. This facinating article provides some empirical data on a unbiased solution. Personally, I simply return the seat to the state it was when I started. If it ends up in the wrong state then it was not my fault. Of course, if it was found that I was at fault I would gladly accept the blame if it minimizes the nagging function.

The other day I risked my own life in the persuit of scientific knowledge when I broached this subject with my wife. Her immediate reaction was that the jig was up -- I could see the panic in her eyes. She quickly regrouped though and started talking about the repression of women, the role of women as primary care givers, the fact that I'll never have to carry a live person in my gut and other things that I no longer remember as I was rapidly attempting to reset the wrong that I had just caused (i.e. reset the venusian dominance).

There is clearly a conspiracy afoot and I intend to reach the root of it (while still maintaining the necessary degree of venusian conformance)!

(I'd like to add a small footnote: My wife is by no means naggy. She's a wonderful person and the best friend that I have. This entry is simply an attempt to understand the inherent differences that exist between the way that men and women understand the world around them and interact. The fact that I feel it's necessary to write this underlines the differences in the way that men and women interpret situations.)