My Work Daydream

1. Learn all kinds of old computing technology: OS/360, IMS, CICS, RPG-III, JCL, Cobol, PL/I; all the old stuff no one has picked up a text-book for in thirty years.
2. Set myself up as a consultant in old tech.
3. Charge slowpoke companies that couldn't be bothered to upgrade their aging systems every bloody penny the market will bear.

It might not be fun. The iOS punks would snark something fierce. But I'd flip them the bird from my Lamborghini.

No, scratch that. I'd have my manservant do it.

Stop White-Board Coding

It is part of my job to interview candidates for software development positions. We do this in the same way a lot of other companies do it (notably Amazon and Microsoft), using a long series of interviews where candidates answer coding questions in 45-minute slots, typically writing their answers on white-boards. This format is common enough that there are whole books devoted to passing such interviews [ref][ref].

The problem with this technique is that these interviews are a deeply artificial work environment. No one writes production code in 45-minute slots on white-boards. The time is much too short, which means the problems are much too small. Most problems would in fact fit well into second-year algorithms and data structures classes. Also, professional developers don't work on white-boards; they work with compilers and editors, or possibly IDEs.

This way of interviewing has few fans; we do it because coming up with anything better is hard. In my opinion, the actual underlying problem is that software developers don't have portfolios of work we can show. If we had actual pro-level code we could show to prospective employers, that would obviously be better than anything we could come up with on the spot. But we typically don't have it, because we typically do our work for large employers who guard their codebases closely.

So what to do? Really, the best solution depends on the level of the employee being hired.

For new grads, we should use internship programs, and hire only students who completed one and did well. What a student gets done during the three to six months of an internship is a far better signal than anything we can get out of a series of interviews. It is based on a far longer period of observation, covers a much larger body of work and (most crucially) that work was done under the company's actual working conditions.

At the other end of the scale, for rather senior developers, ten years into their careers or more, we should be hiring on the strength of the record. At this point, the issue should really not be the minutiae of coding; anyone with this much experience should be able to write sound code. The real issues are good design in the large, project management, and leadership.

Candidates this far into their careers should have strings of significant projects behind them. Interviews should consist of having them explain what problems their projects solved, how they designed solutions, and what choices they made and rejected during the design process. The goal of interviews like this is to establish that the candidates in fact did the work they claim to have done, and that they have the technical and social savvy expected of senior staff.

That leaves the hard case, candidates for fairly junior development jobs that are not entry level. Neither of the earlier solutions is really applicable; these candidates are not interested in internships, and can well have spent the early years of their careers doing really inglorious bug-fixing that doesn't showcase substantial design skills. And at this level, people really are hired to code, which some people don't do well.

This is where code review is most useful. Candidates should be expected to present code for review by several developers, and answer questions about why they wrote it the way they did.

The question is how to get access to actual code, when most of it belongs to the candidates' employers. This is a hard problem, but the burgeoning open-source movement offers a solution. Open-source developers have code they can show, because the code they work on is openly available. Accordingly, let it be known that mid-tier developers will be hired only on the basis of code that is available for review. Maintain a list of well-regarded open-source projects -- ideally ones that the company actually uses in-house -- and refer interested candidates to them. This is particularly useful since in-house developers already know their way around these projects, and can therefore judge not only the candidates' coding, but also their general behavior in the project forums.

So, to summarize:

• Hiring software developers on the basis of code scribbled on white-boards is bad.
• We should hire new grads based on internship performance.
• We should hire senior developers based of their records of design, implementation, and leadership.
• We should hire mid-tier developers based on code review, typically based on open-source code.

Small Changes, Big Problems

When working in a mature codebase, there is a common scenario of a small change that is OK by itself, but which aggravates an existing code health problem. For example, someone may need to add another function to a file that is already thousands of lines long, or another parameter to a list of dozens, or another cut-and-paste function that is almost but not quite like several others.

Cases like this are hard because of the duality of the problem. On the one hand, the developer is only doing what many others have done before, but on the other they are definitely making things worse.

Let's begin by considering three ways of handling the situation.

1. Found a snake? Kill it.

Under this policy, whoever needs to make changes to code that has a real code health problem is responsible for making things right. They are supposed to consider the whole problem and implement a proper solution.

The real strength of this policy is its immediacy. Code is fixed as it gets touched, meaning that the most vital portions of the codebase get updated in short order.

The problem with this policy is disproportionality. A small change can turn into a huge refactoring job. And there can be second-order problems as developers twist their designs to avoid having to deal with that crawling file of horrors two directories over.

2. The Boy Scout rule.

The old rule among the Boy Scouts was to leave the campground better than you found it. In the context of coding, this means doing a little bit of cleanup when encountering an ugly bit of code, but not necessarily rewriting the whole thing. Add a test, pull common cut-and-pasted code into a function, eliminate a redundant parameter or two -- nothing too arduous.

The strengths of this policy are the continual progress it encourages and the rather modest expectations it places on developers. These modest expectations mean that the policy is actually likely to be followed.

The real weakness is slow progress -- big problems will improve only slowly. There are also some problems that are not amenable to gradual reform.

3. For everything there is a season.

Under this policy, the right thing to do when encountering a nasty bit of code is to file a bug and enter it into the owning team's list. The team then periodically (quarterly? yearly?) runs a bug bash to clean up accumulated problems.

The strength of this policy is the opportunity for prioritization before the bash.  There are always more problems than there is time available for fixing them, and some are more important than others. This policy also avoids mixing changes for new features with changes to fix accumulated problems.

The weakness is the lack of immediacy; things get worse before they get better. These is also a real risk that some problems are never fixed. Some teams are very diligent about tending their bug lists; for others, the list is where bugs go to be forgotten.

A common policy

For my money, the best of these policies is the Boy Scout rule. It ensures continual progress without asking for too much, and is therefore likely to be actually followed. I also expect that the changes it calls for are typically in some of the most vital code in the codebase, since unimportant code tends to be left alone.

That said, there are definitely cases where the Boy Scout rule is inappropriate: developers who are unfamiliar with the codebase, problems that require large-scale fixes, and crisis times when there just isn't time. In such cases, it's better to file a bug for the next bug bash. But the more this is done, the more vital it becomes to actually hold those bug bashes regularly and intensively.

Working for a Non-Coding Boss

The Trenches is a webcomic about a gaming QA team. The site has an interesting side-column, called Tales from the Trenches, where game devs, QA, and a few other technical folks anonymously share stories about horrible, horrible jobs.

One recent entry was from an in-house developer working for an unappreciative boss:

I am the sole developer for an in-house fully custom CRM. It was developed by an amateur and was clunking along managing a mid-sized company’s affairs. It was undocumented, messy,  and riddled with tricky bugs. I was brought in to maintain and extend it. 

... 

My boss will not allow me the time to slow down and do a better job, and when asked if I could have a tiny percentage of someone (anyone!)‘s time in the office so that I could have SOME kind of QA I was told that my code shouldn’t have bugs in the first place. This was accompanied with some pointed words about my upcoming personnel review. 

... 

The lesson here, fellow trenchermen, is twofold, number one, INSIST on the time and resources you need to do your best work. If you do not get what you require, communicate that you will not be responsible for problems down the line. Put it in writing. The second lesson is don’t work for a boss that can’t code. It sucks big fat hairy monkey balls.

That's a nasty position to be in, but I think the writer is drawing the wrong conclusion. If you are working for someone who can't do your job, they are in no position to argue about how long things take. If you say this new feature will take three weeks, they can be glad about it or sad about it, but they don't have the inside knowledge to contradict with anything more than bluster.

In a situation like this, the right working relationship to establish is that the boss gets to choose what features get added, their order, and their scope. He does so based on a) his analysis of business needs and b) estimates provided by the developer about how long each feature will take. But, and this is crucial, the developer is responsible for providing those estimates and they always include the time to do the job right, including the sort of refactoring that over time will pay down the accumulated technical debt in the system.

If you can establish this relationship, there is no reason working for a non-coding boss should be a chore. They don't understand what you do, true, but that lack of understanding also provides a crucial freedom to do things your way (the right way, hopefully).