Category Archives: Testing

The Toughest Software Bugs

Someone recently asked me what I thought were the toughest bugs to find and fix.

I’m pretty sure they wanted a technical answer — one that would dive deep into the code and talk about the stickiest math problem, logic issue, endless loop, null pointer error, or buffer overflow.

This was one technology guy to another technology guy, so you would assume it would be something like the stuff above.

Instead, I told him that the toughest bugs are the oldest bugs. That’s right — it doesn’t matter how hard or how deep in the code the bug is, the faster you find it, the easier it is to deal with.

This is because after a developer has released their code, they treat it like a known and trusted quantity. As such, they stop thinking about it. They expect it to work just like they expect the sun to rise in the east. If the sun is up, then that code works. They don’t think about it any longer and they may actually forget that they wrote it.

So, the best way to debug your code is to find the bugs as soon as possible. … Before you write more code and forget what you were working on. Before you commit more code and you’re unsure what is causing the bug. Before you release it to the public.

With our software teams, we start that process with unit tests. Those are followed by peer reviews. This leads to an automated system that verifies the project builds without errors. It also runs the unit tests and checks the code style. This is followed by verification that the feature works as expected — including math checks. Then there’s manual testing and finally automated tests using Selenium. After all this, the whole application is tested. 

That gives us many chances to find a bug before it is released.

The majority of that testing is done on the feature itself, which reduces the amount of code that must be reviewed and debugged. The smaller the footprint, the less investigative work you need to do. The sooner you start, the more familiar you are with the affected code.

One of the tenants of the Agile Scrum development methodology is to inspect and adapt. Review your work and change it if it needs to be changed. This is often applied to project stakeholders who are supposed to get frequent demos so that they can react early and request changes before you complete the project. 

But I believe this also applies to the software development process. As software engineers, if we pause a little to inspect our code and adapt it, then we will fix our bugs more quickly. In the end, the minutes we spend finding and fixing errors now will save us hours and days later.

This isn’t about never writing a bug. It’s about finding our errors first and fixing them. Someone else once told me, “it’s not about whether you fail, it’s how you respond when you do fail that sets you apart.” Expecting bugs, looking for them and fixing them right away is the fastest and most responsible way to deal with them.

Unit Testing Your Code (aka Test-Driven Development)

A few years ago, I scheduled a meeting with the software engineers on my team to introduce them to Test-Driven Development (TDD).

I had started reading the book “How Google Tests Software” and I felt I had to share what I was learning. I started a book study with the team.

The first thing I wanted to focus on was unit tests. This was a completely new concept to the team and to me. I looked around online and found some good tutorials and a video from a teenage developer who was showing other engineers how to write their first unit test.

I built a Google slide presentation and walked through it with the team.

One of the first slides focused on the reasons (excuses) why engineers don’t write unit tests:

  • It’s too late to start
  • It’s too much work
  • It has to be maintained
  • It slows down real development

For each of these reasons, I added a translation into what it really meant:

  • I don’t know where to start.
  • This looks scary. Am I a tester or a developer?
  • I don’t want to maintain this code, too!
  • I have deadlines for real features!

My goal was to head-off the concerns early. I put myself in their shoes, thinking about what it would mean to add unit tests to my code. One thing that really hit home with me was “I don’t know where to start.”

I encouraged the team to start writing unit tests. I showed them the video of the teenager writing them. I called him “Kid Coder” and I told the team — “If Kid Coder can do it, you can, too!”

It worked, the team started writing unit tests. Most never got to the point of actual test-driven development where the tests are written first followed by the code. But we now have thousands of unit tests to go with our many projects.

The second half of my presentation — which we covered on another day — focused on the role of software engineer in test (SET). I wanted to train them on this concept because one of our next steps was to hire an SET and to set up a build server which could also run our unit tests.

It wasn’t long before we hired our first SET and set up a Jenkins server. Not only do we now automate unit tests, we’ve also automated our builds, our code style checks and more. We spawn test nodes in the cloud when we are testing certain projects and more.

It’s come a long way since that first presentation.

Now, I don’t write code too often. My team discourages it! 🙂 And I have other things to do. But I have a few side projects — some for business and some for personal use. I have a lot of code to support web projects. I write in Perl — and I’ve gotten pretty good at it

Can you guess the one thing that I had never done? You got it — I had never written a single unit test.

What a hypocrite. What had been blocking me? I didn’t know where to start.

This weekend, though, that changed. I did some research on Saturday and before I knew it I had five unit tests. Then I had eight. It’s Sunday and I’m up to 71 unit tests.

Writing the unit tests wasn’t all that hard. The hard part was fixing all the bugs I found!

Yup, I found multiple math errors, poor error-handling, incorrect date formatting and some bad regex that just didn’t work.

It’s amazing. I’m converted and I can’t imagine writing code in the future without writing my unit tests.