Boost developer productivity

Reduce time spent on debugging and shorten cycle time with LiveRecorder

8 minute read

The ugly truth in Developer Productivity Engineering (DPE)

There is a heavy focus in DPE on improving build and test cycle turnaround time, i.e. the faster the build, the more productive engineers are. And you’d be excused for thinking this is the #1 answer to improving developer productivity.

But what about the elephant in the room: time spent on debugging in the local development loop, as well as CI and system integration test failures?

Key insight

Question: Do you know how much time your software engineering team spends debugging in a 12-month period?

Depending on which research you look at, developers say they tend to spend 25–50% of their time per year on debugging. That is 25–50% of developer time not spent on more creative, enjoyable work… like programming!

A bit of arithmetic 

Let’s pose this conservative scenario. We’ll assume that:

  • your engineering team is 100 people strong 
  • the team spends an average of 30% of their time per year on debugging
  • each engineer costs an average of $100,000 per year

That’s $3,000,000 spent on debugging every year.

Do your own back-of-the-envelope calculations; and if you don't know how much time the engineering team spends on debugging, it's time to find out!

The problem of reproducibility

What if you could get the cost of debugging down by 50%? But, more critically, what if you could cut down the time engineers spent on debugging from 30% to 15% – allowing engineers to spend an additional 15% of their time on building new functionality, or delivering changes faster?

Give every engineer back > 30 days per year!

Sure! But how?

Well, first, let's look at why debugging is so time consuming.

91% of software developers admit to having unresolved defects because these defects cannot be reproduced. [see analyst report]

41% say that the biggest barrier to finding and fixing bugs in their backlog faster is to get the bug to reproduce. [see CI research report]

In other words, reproducibility is the fundamental technical problem that absorbs the bulk of the time in debugging. It also happens to be the most frustrating part of the job. When a failure in QA lands on an engineer, it takes time to figure out whether the failure is a new regression or not, then figure out where to start, and how to reproduce the problem. It’s not just frustrating: at the point when engineers pick up a bug to resolve, they often can’t know how long it will take to figure out the root cause. The uncertainty is uncomfortable. And let’s face it, the longer engineers spend on debugging tasks, the more likely they are to want to look where the grass is greener.

Time travel debugging

Time travel debugging solves the problem of reproducibility. We explain further down in what way it does this, but first, let's define what time travel debugging is.

What is time travel debugging?

Time travel debugging is the ability to wind back the clock to any point in an application’s execution and see exactly what it was doing. Among other things, developers can :

  • inspect the complete state of the application at any point in time including the contents of all variables and the heap
  • navigate their application’s execution using the full range of debugger functionality – stepping, running, breakpoints, watchpoints, catchpoints, etc. – but in reverse as well as forward (usually called reverse debugging)
  • run backward to trace when and how a variable or memory location was last changed (using reverse watchpoints)

There’s more to it, but that’s the basic principle. Learn more about time travel debugging.

How does time travel debugging reduce time spent on debugging?

Time travel debugging provides engineers with a recording of a test or production failure. Having the recording is like having the complete movie at your fingertips (instead of a bunch of still snapshots to piece together for clues). No time needs to be spent on manually trying to reproduce the problem. The bug is captured in the recording. And the recording will always behave the same way… for everyone. No more “I can't reproduce this on my machine!"

All engineers have to do is to debug the recording by traveling back in the code execution from the symptom all the way up to the root cause – in just one cycle, instead of multiple debug cycles over several days. Developers analyze the recording and see what the software really did (as opposed to what it was expected to do).

In short, time travel debugging makes debugging way more predictable, compared to using traditional debugging techniques such as logging, core dumps, printf statements, or standard debuggers.

Bonus: By allowing all developers to explore how the code is executed – every line of code in every thread, every variable, every I/O –  developers new to the codebase can debug just as efficiently as more experienced team members.

"Everyone who debugs C/C++ should be using time travel debugging. If you're not using it, you're just wasting time." 

Brian Janes, Senior Engineering Director, High Performance Computing at Altair

Time travel debugging tools

There are several time travel debugging tools available:

  • Microsoft’s time travel debugging tool for Windows C++ applications
  • Replay.io for web applications
  • The rr open source tool for debugging Linux C/C++ programs (perfect for smaller, less complex applications)
  • Undo’s LiveRecorder for debugging more complex Linux C/C++, Go, and Java programs (multithreaded or multiprocess applications, applications using shared memory or async I/O, operating in the cloud or on a VM)

Make bugs 100% reproducible with LiveRecorder

LiveRecorder makes bugs 100% reproducible.

LiveRecorder provides a one-click workflow from a test failure to a time-travel debugger placed exactly at the point of failure – skipping the tedious steps usually required to reproduce the problem and enabling developers to start debugging test failures instantly.

Developers working on complex C/C++, Go, and Java software can now save a huge amount of time diagnosing the root causes of new regressions, legacy bugs, and flaky tests.

Bugs that took days or weeks to fix can now be resolved in hours.

  • Record CI / System Test failures to capture the execution of failing test runs, including intermittent failures; store the recordings in your CI or bug tracking system for later analysis and cross-team collaboration.
  • Replay recordings: jump from the test failure (or bug report) straight into a ready-to-go, fully set up debug session in your web browser. 
  • Resolve bugs fast by tracing from symptom to root cause in one cycle: go back to any point in the execution history to inspect application state (including contents of all the variables and the heap) and see exactly what your software did.
SAP developer quote social image

Why does LiveRecorder help boost developer productivity?

  • Capturing the issue is automated (the computer does most of the work) – eliminating the need for engineers to go through the hassle of trying to manually reproduce the bug. The cognitive load of debugging is reduced because LiveRecorder automates part of the thinking of debugging – making it less taxing for developers to work out how to get started with a bug.
  • Developers can start debugging instantly by simply clicking on the recording link in the CI dashboard or bug tracker (one-click workflow).
  • Engineers can debug in a modern, easy-to-use IDE (more accessible for developers who don’t like command-line debugging).
  • Recordings are shareable for effective remote and asynchronous collaboration: QA team members can share recordings with developers, and developers can share links to moments in time with colleagues anywhere in the world, share bookmarks, and add comments in recordings, etc.
  • Context switching costs are reduced: it's expensive for developers to take a CI failure and work out a what's gone wrong. When a test fails, developers rarely have much information about what happened. LiveRecorder tightens this loop by providing faster feedback on test failures.

LiveRecorder doesn't just boost developer productivity by reducing bug-fix time; it also hugely improves the developer experience by taking the pain out of debugging. It removes all the hassle of going through fiddly steps to get to the point where developers can start debugging.

Learn more about LiveRecorder for C/C++/Go. A different version of LiveRecorder for Java applications is also available.

Request a demo

REQUEST A DEMO