Opinions on Programming

Saw an interesting article on Slashdot (here's the full article). Of course I saw it's a great article because it lines up with some opinions I've held for several years. Guess it's more of a personal philosophy, probably shaped by learning how to program in text: you've got to know how things work if want to understand what is going on. It applies to so many different aspects of life too - it's shocking at times the trivial things that people don't know. Like how to change a tire - to understand what to do, you have to have an idea of how the tire is attached to the vehicle. Programming with pictures is all well and good, but there comes a point at which you have to reach deeper understanding of what goes on behind the pictures to ensure that things are operating as you wish.

I've had this discussion with one particular person and they pointed out, quite correctly, that there is a good reason why UML programming never took off. That's because the graphical development environments were always the encoding the wrong thing - the requirements. The "what" rather than the "how". The "what" parts are often contradictory, which is why I have a job. A software development professional is there to turn the "what" into the "how" correctly. Break the paradoxes and force the machine to do what is implied in the requirements, not do them literally.

So as one of the developers quoted in the article, graphical environments are good for learning. Small projects that show how things fit together. That first frustrating few experiences when the larger project is attempted and hours are spent to make two things mesh that won't work together. Then showing the text-based programming languages - the "how" behind it all. From what I've heard, any attempts at doing UML programming actually had two steps: First, design in the graphical environment and then second, tune the generated C source code to make things actually work. Very indicative of things to come.

I also appreciated what Herb Sutter had to say - that bare-metal programming and optimization will come back into vogue in the next 10 years. Waste is waste and graphical environments and elaborate abstractions are waste. And this waste can be translated into environmentally-relevant impacts. Smaller, efficient code will use less power, less space and be better all around. I still hope for the day when every chip-based interface responds instantaneously to my input. Even if it has to tell me that what I just requested will take a long time, that initial response shapes my interaction.

Maven thoughts, then patterns thoughts

I'm still plodding away at Maven at work.  There was some discussion as to naming patterns, conventions, directory structure etc.  Most of these things weren't ground-breaking, just items that require agreement.  Without agreement, some benefits are lost.  The only point I wanted to see was that modules and Java packages corresponded.  Everyone in my group agreed with that idea, so I went to voice my concern with one of the main drivers behind the Maven efforts.  He said essentially the same thing, but pointed out an exception.  Imagine an interface is described in module I, with two concrete implementations in module A and module B.  If module I uses a factory, it must have special access to module A and module B.  Specifically, module A and module B should be non-public, package protected perhaps.  This leads to having the interface, factory and implementations in one Java package even though logically they three separate pieces.  I wasn't 100% comfortable with the example, but I couldn't identify any flaws in the reasoning.  Then I hit on why I didn't like it - I don't think I like factory classes.

I've used factory classes a few times in the recent past, but they aren't something I agree with as much.  They are hard to test and the classes that use factory are also hard to test.  I think I would prefer each of the modules be a separate Java package with a single public entry point in module A and module B.  The user would be able to determine which implementation to use, instead of putting all that logic in the factory class.  However, either implementation would work and are equally correct.  It also illustrated the point that the name conventions and layouts are guidelines, not rules.  The exceptions occur frequently enough that slavish adherence to particular rules will cause more problems than necessary.  Rather the rules are there to help remind people to really consider why they are using complex constructs.

This brings me to some thoughts on patterns.  The Abstract Factory pattern was mentioned above and my desire to avoid using it comes from a certain unease with ideas of software patterns.  I've previously had discussions with a co-worker who expressed the idea that there is something wrong with patterns, but he couldn't put his finger on it.  I suggested that it was because the patterns don't solve all problems (otherwise software development could have been fully automated by now) and the pattern requires adjustment all the time.  It seems like the problems are fit to the patterns, rather than using patterns to solve problems.  I think that patterns have been used by those that want to turn software development into more of an engineering discipline with common issues that simply need to be recognized and well-known (well-described?) solutions applied.  I agreed that these were my ideas on the topic, but didn't do the unease any justice.  My co-worker has been reading Christopher Alexander's book on pattern language, as Alexander is the source of the ideas adapted to software development.

I suppose the real issues come from those who apply the software design patterns too easily, without first considering the problem.  As time has gone on, I've been adding criteria and considerations to things I need to design.  Things like ease of understanding the result, simplicity of implementation, testability, current requirements.    Also, with the various patterns I've been involved in, they've been customized and adjusted in ways that suggest one of two things (or both): 1) pattern didn't fit and the problem is being adjusted to suit the pattern and 2) so many adjustments are made so that the pattern is only partially there.

Anyway, this is not a well-reasoned argument - it's barely written down at all!  Hopefully I will be able to devote some more thought and then more time to describing some of these things more coherently.  Not tonight though.

Build tools

I referred to the Maven change at my work the other day, but I didn't take the time to describe some of the basic principals that I was using.  These principals generate questions rather than exclude or include things.  As I mentioned in this post, any build tool can be configured to support your environment.  Using that logic, why would anyone move beyond shell scripts and batch files?  That's the first item in the philosophy:

1.  Can this project use the build tool in the way it was designed?

For make, this means try to use only the implicit rules and organize your pieces so that all the interdependent files are within the same directory.  For maven, this means organizing your project in a modular fashion where each of the configuration files describe only the dependencies between the modules.  When you move out of the comfort zone for the tool, you increase the likelihood of mistakes, or design cul-de-sacs.  Every design has certain features that shape it, and those features exclude other items or make them difficult to achieve.  Using Maven to build a project developed entirely in C raises a flag.

2. Can the build tool handle the corner case you know about now?

Pretty simple thing to remember - if the tool can't handle all your current requirements, how will you deal with the unexpected?

3. Can the tool be setup to run correctly from a sandbox?

By sandbox I mean can the tool be useful without referring to non-local items.  I understand that one of the features of Maven is that it can keep various libraries up-to-date by retrieving the latest version.  By "retrieve" it means connect via HTTP, I believe.  I've been informed that this behaviour can be disabled, which would be a plus in my view.  When building a particular project, it is preferable to be able to identify, before the build, the files that will be included.  That doesn't really provide any more or less protection from error, but it makes the process more repeatable and leads to the next point.

4. Can the tool be setup to run correctly from a source control?

Imagine you have a project in a source control system and you connect with a new machine.  Can  you do a single check, run the build tool and generate the proper output?  This is something that I value highly.  It speaks to repeatability and how fast you can get new project members up to speed.  It also means that as long as you have project source, you can build it.  Someone I knew at university was doing his PhD, but he said that he could no longer print or display his Master's thesis.  He wrote it in PostScript or LaTeX dialect and could no longer find the pieces the render the source as it appeared in the hard copy.  Many companies and individuals rely on open source tools.  If they do not keep the tools along with the project, they risk not being able to build the project from source in the future.  

5. Can developers operate effectively without modifying the configuration files?

One of the problems I've seen in various workplaces is build configuration differences.  If the build configuration files have to be checked out of source control and modified to build correctly, that is a large potential problem.  Ideally, build configurations would not need to be altered unless files are added or removed - even some tolerance of those changes would be good.  If these files have to be checked out, different groups produce slightly different builds, which leads to errors arising from these small changes.  These errors are very hard to track down and can be avoided with proper tool selection.

To summarize - the idea is that doing the right thing is easy and the wrong thing (unexpected things) is hard.  It should be easy to identify where everything came from and how something was built.  These things may take a large amount of configuration - a ramp up time learning how to use a tool properly.  As for Maven, so far I think it's right for what we are doing at work - but there are caveats.  The main one being the potential for cross-module changes, of which I know of one current example at work.  Items that touch many modules can be refactored by adding more indirection (or abstraction), but this has it's own cost.  Oh well - such complexity is what makes things difficult and what keeps me employed.  I just want to make sure that I don't forget the potential problem areas in the future.

A Friday at work

Post 177 - not prime, but at least the product of two primes: 59 and 3.  Had a pretty good at work, especially at lunch hour.  The last Friday of every month some people bring in their Rock Band and have at it over lunch - but must be done by 1pm.  Anyway, it's a pretty good time - there are quite a few people that are really good with the various instruments, especially a couple of incredible drummers.  Always impressive.

More interesting was the initial reports of some investigations into using Maven.  Maven is a development tool, used to compile/package and generally create a piece of software.  Other similar tools are "make" and "ant", with ant being more closely related.  Currently we use ant to manage our builds, but there will be a move to maven for the next release.  Given some of the problems encountered testing it out, I wanted to find out what has been motivating the move.  Especially when I found out how easy it was to retrieve libraries from Internet sources.  For an open source project, this is a tremendous feature, but for a proprietary embedded system, I'd be more comfortable with a rigidly contained system.  By that I mean one that will not attempt to search for data beyond a specified file system area.  So I went to talk to the person who has been working on this initiative for the longest and he had all the "goods", so to speak.

The reason for the move had two main motivations - the first being the default mode that Maven operates in.  Maven expects projects to be structured as a series of modules.  This encourages projects which have lots of pieces with well defined boundaries.  This is something that our company wants to promote internally, so Maven promotes it by making it the simplest choice.  Maven's strength comes in the way it describes dependencies between modules, with the simplest configurations being when most configuration files only contain the dependencies.  The second motivator was that modules can describe a dependency on an old version, so development could continue in place without breaking other modules.

After my conversation, I was pretty happy with the discussion.  The tool doesn't eliminate problems, but steps will be taken to minimize the likelihood of certain problems.  I wanted to convince myself that we won't be walking into a new set of pitfalls without enough forethought.  I believe if certain configurations are used, the future project will have fewer potential problems.

Ah well, I guess that's a first cut at that discussion.  My brain is tired and trying to get me to sleep, so it won't let me continue heavy thinking.  Tomorrow then...