I finally took a few minutes and put up some information about the Code Views project that I am working on. You can read more about the ideas behind this concept in the entries on annotating, internationalizing, adding images, de-textifying source code and simplifying the understanding of code structure.

Please post any questions or comments about Code Views to this entry.

Because there is currently a one-to-one correlation between the code that you see in your editor and the code as it is stored on disk, a developer is severely limited in the ways that she may visualize the structure and program flow of a component. Separating the presentation of the source from its storage allows for a number of time saving enhancements while dramatically increasing a developer's ability to understand the code.

Attempting to follow the path of execution through a simple two-level hierarchy can be difficult. An example is shown below that utilizes a template (abstract) method. To follow the execution path the developer must switch back and forth between AbstractContainer and NoNestedChildrenContainer.

AbstractContainer with Template Method

NoNestedChildrenContainer implementing the Template Method

Rather than swtich between the two classes, the developer should be able to see all methods (or only the ones that they choose) at once. The developer may choose to edit the superclass's method in the unified view.

NoNestedChildrenContainer Displaying All Methods

Alpha blending has been used heavily throughout these images. While prototyping these designs it was determined that the WYSIWYG Javadocs, for example, were distracting when attempting to look at the code. When the developer moves her mouse over the dim Javadoc, alpha blending is used to brighten up the image to its full-bright state and when the mouse is moved out, the javadoc becomes dim again. Likewise, when the mouse is brought over methods from a superclass they are made full-bright.

Although it is not immediately obvious, the various blocks can be collapsed as in a conventional source editor. (The visual cues for this functionality are still being worked out.) When all blocks are collapsed the class very closely resembles a UML class digram. This unexpected consequence as lead us to some interesting ideas that will be shown in the upcoming entries.

This entry is continuing the thread on separating the presentation (view) of a programming language from its storage format (model). There are also entries on annotating, internationalizing, adding images and de-textifying source code.

Read about Code Views.

When reading source code the developer has to make a mental picture of the concepts. Likewise, when writing source code the developer has to translate her mental picture into text. What if we simplified and enhanced this process by making source code more visual (pictorial)?

There are a number of concepts in modern programming languages that have been shoe-horned into text which may be better portrayed using a pictorial representation. Two such concepts are the ternary operator and block delineators.

Ternary Operator

Imagine all of the developer time that would be made available if there was no more bickering about brace placement! (Wesner Moise has an entry about graphical source code editors and code blocks.)

Statement Block

It is possible to decorate existing textual components to help emphasize their purpose and scope. The following image combines a number of features: comments, ternary operator and if icon.

Graphical Comments, Ternary Operator and if with Icon

This decoration can be carried another step. In the previous image, the if keyword and icon appear together. If icons are used to replace keywords, if can be removed entirely. In a similar fashion, the final keyword can be replaced with a decoration on each variable reference.

The final case demonstrates that the decoration provides benefit over the keyword. When the keyword is used, there is no way to know which variables are final without looking at their declarations. With a decorated variable, each time that the variable is entered into the editor the decoration will appear indicating the constraints on it. One can imagine this decoration idea extended even further by indicating type, scope (local versus class) and access modifiers. If the decorations become too cluttered then the editor can allow decorations to be disabled as needed.

Decorated Variables

Rather than providing insight some might believe that replacing the if with an icon obfuscates the code. There are going to be a number of attempts that are made in the upcoming years that seem to bring us backwards rather than forward. We must walk before we run and we are certain to get a few scraped knees along the way.

Daniel O'Connell brings up an interesting situation in this post's comment: instead of language lock in, you could easily end up with IDE lock in. It is certainly possible that we would end up with IDE lock-in but recent history has shown us that the various IDE's feed off of one another to create relatively similar products. If Eclipse has a useful feature that IntelliJ does not have then the next version of IntelliJ will likely have it. If on the other hand, Eclipse has a feature that is not useful then it will likely receive less attention and be pulled from the product. In this time of great innovation as long as we steer well clear of "protecting" concepts by legal means (i.e. patents) then the theory of natural selection will cause a relatively level playing field and will eliminate the possibility of lock-in.

I welcome new ideas regardless of their ability to demonstrate immediately clear and attainable results. There are times when it takes two blind steps forward before we can reach our destination. And there are times when we must travel part way down a path before we determine that it is not the correct one.

The ideas that I am proposing are by no means new. (I was surprised to find that Roedy Green has written an extensive list of features that is a superset of mine.) Finding the ideal balance between textual and graphical representations will allow these ideas to flourish where they have withered before.

This entry is continuing the thread on separating the presentation (view) of a programming language from its storage format (model). There are also entries on annotating, internationalizing and adding images to source code as well as simplifying the understanding of code structure.

Read about Code Views.

It has been said that a picture is worth a thousand words. Developers have been left with out this benefit since the dawn of the computer age.

In more recent additions to the plethera of available programming languages, such as Java and C#, developers have been able to use HTML or HTML-like contructs to add references to images to the source code. In order to see the images, a developer must pass the source code through a translator which outputs HTML. This HTML is then brought up in a browser where the images are finally seen. There are quite a few steps to go through just to reap the benefits of having images available and obviously this process isn't WYSIWYG.

What a developer actually needs is to be able to see the images in her editor along with the source code. A first pass implentation of this can be done today without changing the source code storage format in any way.

Javadoc with Images

But why stop there? Why show HTML and javadoc tags for other elements? The entire javadoc can be shown and edited WYSIWYG in the source code editor.

WYSIWYG Javadoc in Source Editor

The concept can be extended even further to all comments to provide for an incredibly rich development environment. The image below shows what an inline-WYSIWYG editor might look like. This editor may be rich enough to create and manipulate the shapes seen in the image. The format in which the source is saved would need to be enhanced to store this rich media information.

WYSIWYG Inline Comment in Source Editor

This entry is continuing the thread on separating the presentation (view) of a programming language from its storage format (model). There are also entries on annotating, internationalizing and de-textifying source code as well as simplifying the understanding of code structure.

Read about Code Views.

You will be hard pressed to find a programming language whose keywords are not in English. Being an English speaker myself, I have never had to go through the hassle of having to look at a blob of apparent gibberish and attempt to make some sense of it. When I see the following code, I know exactly what each keyword means since it is in my native tounge.

    while (index < 10) {
        index = index + 1;
    }

Ideally, a Russian developer (that only speaks Russian) would write the same code using his native tounge as is seen in the following example:

Programming languages are defined by a series of character-based tokens with character-based separators. A parser is used in the compiler to read these characters and match them with predefined sequences. To support multiple spoken languages (for lack of a better phrase) multiple language specifications would need to be written each having its own set of keywords.

What if, rather than storing the literal string while and having a language specification that has that as one of its keywords, we stored something that represented the concept of a while-loop? Each developer would then be allowed to define the way that they wished to present the concept in an editor. If there was a decoupling of source code presentation and source code storage then localization of editors could be accomplished. Compilers would no longer need to parse out keywords; they would read the concept from the storage format.

This notion could be extended even further if desired. It may be possible to store identifer (e.g. variables, methods, etc) information generically and provide various translations. This would allow the JDK API for example to use identifiers meaningful to each locale. This is obviously not needed for every project but for more global APIs (e.g. Apache commons) it may be highly desirable.

This entry is continuing the thread on separating the presentation (view) of a programming language from its storage format (model). There are also entries on annotating, adding images and de-textifying source code as well as simplifying the understanding of code structure.

Thanks to Igor Fedulov for the translations.

Read about Code Views.

[To avoid confusion, annotations as I will be discussing them are not Java 5's annotations (also known as metadata). I will be using the literal definition of annotation: a note added by way of comment or explanation.]

I have always been torn about the best way to conduct a code review. Sitting down with a developer in front of a screen and pointing out various elements (e.g. changes that can be made and identifying good approaches) is one approach. Unfortunately, this requires either party to write down proposed changes so that they may be followed up on. This leads directly to another approach: sitting down with a printout of the code so that annotations can be made directly on the printout. The problem in this case is that code does not usually want to be printed (even in landscape) and let's not forget about the trees!

Ever since I used the "Reviewing" functionality in Microsoft Word a number of years ago I have wanted to apply the same principles to source code. For weekly code reviews, a developer and I would sit in front of a screen and walk through the code annotating the various issues. For daily code reviews I would be able to annotate the code and the next time the developer opened it, they would see the annotations.

Annotating Source Code

One of the primary stumbling blocks in implementing annotations in source code is storing the annotation data (e.g. the text, position, corresponding elements, and author). Given source code's textual storage format, the annotations would need to be turned into language comments. If you've ever worked with an integrated UML tool or a visual UI builder then you know the hassle of having vendor specific elements nested in comments sprinkled throughout the code. It is easy to accidentially break the comment-based binding while performing normal edits to the code.

Ideally, source code (the stuff you see in your editor) would be independent from its storage format and the annotation data would be stored safely out of harms way.

This is the first of many entries that will be written around the notion of "software without text" or more succinctly: separating the view of programming language editing from its model or storage format. This is an idea that I have been enamored with for years. There are a number of "implementations" of the idea from Intentional Programming (IP) to a recent article by Sergey Dmitriev on his Meta Programming System. There are also entries on internationalizing, adding images and de-textifying source code as well as simplifying the understanding of code structure.

Read about Code Views.

This is off topic but I thought it to be a good noodler.

Wouldn't it be ironic if all of the UFO's and "alien landings" were time traveling humans from the future? The reasons for abductions were that we wanted to see the genetic makeup of early humans and determine any illnesses that they may have had. "Flying saucers" are used in the future since we're obsessed with them now. In other words, it's logical to assume that we would produce a time machine in the shape of a flying saucer since we are so interested in flying saucers now. The grandfather paradox is incorporated since history already has recorded such events.

I should point out that I am not the first to have this thought.

I talked about unified messaging some time ago. It seems that Microsoft was thinking the same thing. Based on the reception of this, we will see if I was right about Sendmail, Inc. foolishly sitting on their hands.

Eclipse has this wonderful feature called Dynamically marking occurrences (along with "Highlight method exit points" and "Mark locations of thrown exceptions"). (No, I don't care if IDE X has the same features.) What if we took this to the next level?

The idea is to highlight an object and select "Mark Synchronizations" (or something suitable). This would show all methods, statement, etc that are synchronized on the object. The outline and hierarchy views would also indicate other classes and methods that synchronize on the object. This would allow you to know if some other object was synchronizing on the same object. Being able to see all code that synchronizes on an object would greatly simplify writing and maintaining threadsafe code.

One could extend this idea even further by allowing the marked occurrences to be sticky. Select object one and mark its synchronization occurrences then select object two and mark its synchronization occurrences in a different color. The intersection of the two sets could be highlighted in a third (combined) color to indicate points of possible deadlock.

One of the major stubling blocks to realizing the promise of Intentional Programming (IP) is building up the data structure that abstractly represents the source. Fortunately, Eclipse's AST provides most of the necessary functionality.

The next and largest stumbling block is deriving a suitable representation for visualizing and manipulating "code" in a more abstract way. The technology is in place to implement the representation: Eclipse's GEF. Unfortunately, there is no already defined representation that provides the necessary granularity to effectively manipulate "code". UML falls short unfortunately as it does not provide the necessary degree of granularity.

I suppose that my idea is that we need an idea for an abstract, yet granular representation of "code". I will, at some point in the near future, write up some requirements for this representation and provide some example code that demonstrates the difficulty of the task at hand.

When it was announced that Doug Lea's concurrent library was being added to the core set of Java classes, I had a classic movie moment:

A typical corporate development environment: cubicles with high walls, drab grey color, and harsh artificial lighting. A developer is seen happily working at his workstation. A small message appears on the screen. All that is seen is the message:

Doug Lea's concurrent library brought into the JDK

The programmer's typing has stopped and a loud thump is heard. The scene quickly shifts back from the screen to show the developer's head on the desk. The scene continues to move back and upward.

DEVELOPER

Noooooo!

The programmer's hands move up to his head as he lifts his head, leans back and continues the cry. Fade to black.

Why do I want that wonderful utility brought into one of the slowest revisioning software products of the modern era? Currently, if there's a problem with the concurrent library, I post a bug, Mr. Lea takes a look at it within a few days (a week if he's on vacation) and shortly thereafter there is a patch. Constrast this with what occurs now in the JDK: file a bug, get a cryptic response (if any) in bug parade, wait patiently for months if not years for bug parade updates, con anyone you know into voting for the bug at any chance they get, then, if the phase of the moon is just right with the angle of Mercury to the ecliptic, the bug will be listed as fixed in some version that is at least two versions ahead of the currently released JDK or in some obscure version, and finally, once the JDK version has finally come out, you download and test only to find out that something was broken because of the fix.

It was recently rumored that RFE's are being accepted and voted on for Java 1.6. There are dozens of additions that people are looking for in the JDK. Why do I need javax.swing or java.awt in my server application? Why do I need java.sql in my game? Why does anyone need org.omg.CORBA *grin*?

My RFE is this: remove everything from the JDK! Reduce "Java" down to java.lang.* (and possibly java.security). All other packages will be divided into separate projects and made open source. Sun will retain control over "Java the language". To the end developer, Java will appear unchanged.

In order to support the current uses of Java, the profiles paradigm (seen with embedded Java, for example) would be used. For example, there would be an applets profile and a server-side (J2EE) profile. Each profile will define the necessary packages and versions. To facilitate this, a robust jar versioning mechanism would be required. This would also benefit anyone that has ever had to deal with jar version hell. A first pass might be to use Package. Strict guidelines would need to be published to define how versions are numbered and updated. Sun can provide a test and certification process whereby versions are validated and signed.

There are going to be a number of snafu's that need to be rectified. For example, the current NIO implementation uses a trick in java.lang.Thread (look for blockedOn()) in order to handle some of its interruptable aspects (such as InterruptableChannel). The JDK provides an SPI to allow vendors to create their own NIO implementation, but given tricks like these, it's impossible for anyone but Sun to do. Separating java.lang from the other packages will force these issues to light and force them to be resolved. In this case, one can envision a java.lang.Interruptable, for example.

There might be a concern that this idea will cause there to be 400 java.util versions out there. To this I say: are there 400 versions of apache commons? The concurrent library? I will admit that back when Java first arrived on the scene, there was a bit of chaos. That has, for the most part, settled down and there are well understood and respected methods to producing open source software versions.

Another concern would be intra-package reuse. For example, the java.util maintainers prefer log4j whereas the java.io maintainers prefer the current Java logging API. I don't claim to have an easy answer for this but certainly a product management committee could be assembled where each package is represented. This committee would be responsible for coordinating and resolving issues such as this.

How does JSR fit into all of this? I honestly don't know. I suppose that it will do what it has always done but rather than have most of its results put into the JDK, they will become an open source project.

Kris brings up a number of reasons that dealing with the current JDK bloat is not easy.

The notion of unifying IM and email (and VoIP) is almost to the point of being trite with me. The fact that my IM messages are not stored together with my email threads (GMail-like would be simply sublime) is simply ludicrous. Not being able to just type in an IM window to an "offline" "buddy" and have it automatically send an email is mind boggling.

Given that there is already the necessary components in SIP to do both IM and email, there is no reason that a common infrastructure shouldn't be used. Using SMTP as it is used now should just be left in the corner to die like the mangy rabid dog it is.

The fact that Sendmail, Inc. sat idly by during the dot com days eating millions of dollars of VC's money while the internet raced by them at the speed of light angers me to unpreviously seen levels. Every time that I see some of Sendmail's VC's (I know a couple of them very well) I want to kick them in the teeth for letting David Anderson take what could have been the investment of a lifetime and squander it on absolutely nothing. I should mention that the reason I'm so mad has nothing to do with any investment in Sendmail, Inc. (which, thankfully, I do not have) but has everything to do with the fact that I don't have a unified messaging infrastructure.

And if you really want to see me livid, ask me about sm-X (Sendmail 10 which was previously being developed as Sendmail 9). Sendmail, in its current form, is some bastard turd left over from the "old days" of systems programming. When you look across your enterprise software, Sendmail is the only non-enterprise enterprise application. The only reason that it's still there is that it does what it needs to do quite well and that it's attached via some unseen force field to that belligerent admin that no one talks to that lives like a troll in the basement of most organizations. Attempting to wrangle Sendmail away from this admin would take the military force of a large first world country. I'm digressing. sm-X, rather than being the magic bullet that Sendmail, Inc. and the rest of the world needs to fix its messaging needs, is a complete rewrite in the same systems paradigm as the current Sendmail with essentially functional parity but they're replacing the impossible to understand sendmail.cf language with some other obfuscated language. When I asked the sm-X project leading why in the world were they taking away the only reason that the entire world hasn't migrated away from Sendmail (i.e. the sendmail.cf file and the savants that maintain it) and what was their plan to ensure that the existing installations would be able to migrate to sm-X his answer was: silence and no current plan. My current advice to the people of the world that use Sendmail is: RUN. Run and don't look back. The migration path from Sendmail 8 to Sendmail 10 is going to be a nightmare and with the well known problems that have occurred in Sendmail's past when moving just between minor revisions, you can just imagine what another full rewrite is going to look like. Start your own migration away from it now!

*heavy breathing* Alright, I finally got that all off my chest. I think I'm done for a while on the Sendmail topic. Did I mention that there's no conformance or regression suite to speak of for Sendmail? How in the world can large companies be expected to migrate to Sendmail 10? Is Sendmail, Inc. going to test, certify, charge up the wazoo for support, etc to do the migration? I would think that there is something sinister going on under the Sendmail, Inc. covers with regards to this whole Sendmail 10 thing if I didn't know for a fact that Sendmail, Inc. has no say over what happens in Sendmail (the code). They're just as surprised as the rest of us are when crap comes down the pipe.

Whew! Now I'm really done.

OK, so this was more of a long rant than an idea. Sorry. *grin*

Given the capabilities of most PC's (internal speaker, microphone, a good deal of available processing power) it seems reasonable that software can be written to cancel out the noise from a PC. To go a step further, it seems reasonable that a card can be created to detect noise from within the PC case and cancel it out (even if this required a small external speaker that needed to be mounted on the case). The hardware solution would not consume any CPU processing or the microphone port. I would glady pay $20 - $30 for this service.