I was chatting with a friend over dinner this weekend about the impact of work environment on one's ability to excel as a software developer. We've both worked for companies where we've had to work far more than 40 hours each week for extended periods of time. We both agreed that's a good way to burn out your staff and, more importantly, a bad way to get stuff done faster. Sure, you can do an intense sprint now and then, but you don't win a marathon by running full out the whole way.
The comparison I came up with was that writing software is a lot like writing literature. It takes both discipline and creativity. Having either of those attributes will only get you so far (though it will probably put you ahead of most of the pack). To truly excel you must have both.
Discipline is not about putting in long hours (well sometimes it is, but that's not what I'm talking about). Discipline is about consistency. It's like playing blackjack and counting cards - your system won't pay off on every hand, but over time you will reap the rewards. Agile development is very much about that kind of discipline. You write your tests, go red, go green, refactor. This aspect of software is craftwork. It's methodical. It's not very exciting, but it gets you from point A to point B, and for the most part it's predictable.
Putting in 60+ hours a week is not good for your creative output. Being creative requires a brain that is rested and energized. It also takes mixing things up, changing perspective frequently. Burnout is the enemy! Sometimes when I'm stuck I'll get up and walk around the office, have a 5 minute chat with someone about something random. When I get back to my desk I can deal with the issue much better. Or if my pair and I get stuck, we'll put the story on hold for a little bit and tackle something tiny that lets us reset our perspective. That sort of thing usually helps a lot. We know enough not to try something really tough late in the day, and sometimes we save doing a fun story as a reward for finishing something that's no fun at all. Our brains naturally have attention spans that are too short to keep up with what we ask of them. Learn how to work with your brain instead of forcing it to do something it's not suited for.
There is a balance, and a way that discipline and working creatively mesh. Find a rhythm that works and that you can sustain, then stick with it. Back when I was trying to write science fiction, I got to talk to Cory Doctorow about his practice of writing. His advice was to write a page every day, then send it to people to get feedback. Pretty simple, eh? Cory produces a lot of fiction and I find it shocking that he only does a page a day. But a page a day lets him do a novel and a few short stories in a year, which is a lot more than I ever wrote when I was trying. I think he can probably keep that pace up forever. Sustainable pace wins.
One of the things I like best about working at Pivotal is that we consistently work at a sustainable pace. I can't believe how many startups advertise jobs where they say that they expect you to work "startup hours". I won't even consider working at a place like that ever again. It's not just because I don't like working that way myself, but because I think companies that expect and require that kind of pace from their developers are just going to screw themselves and burn out their developers. They'll either get real about what they can sustain, fail, or figure out how to deal with a high attrition/turnover rate.
Then there are companies that have no discipline and rely only on their "rock star" developers being able to come up with one inspired solution after another. Now, inspiration is great, but it doesn't get you even halfway there. People don't pay you for your ideas; they pay you for how you execute on them. If all you ever did was sit around and think up great solutions for difficult problems, you wouldn't ever get anything done. And if all you ever did was play Rock Band, you might be very inspired but you'd never finish writing Duke Nukem Forever.
As an ENTP (the personality type, not the company), working in a team is the only way I can consistently get things done over a long period. I can be super productive working with other people, but working on my own can be very challenging for more than a short time. It's much easier to keep up my discipline when I'm pairing since that helps me stay focused, and I can be very creative too, since I think better out loud. I know that way doesn't work for everyone, but it sure does for me. I also love tools like Pivotal Tracker that let me easily see what I have to work on next so I don't have to keep figuring it out every time I finish a task and need something else to do. On the other hand, I'm not a Calendar-About-Nothing kind of guy.
That's enough philosophy for the evening. But I'm curious to hear what other people feel about the interaction of discipline and creativity.