Simone Concepts

Auditory Debugging

Most of us use sound now rather primitively. "Auditory Icons" are sounds used to indicate events, such as the error beep on a computer or real-world sounds like a rooster crowing for an alarm clock alert. These types of icons convey a single piece of information.

Simple uses of sound include using a beep or auditory icon to indicate entering and exiting a routine. This is useful when you don't want to set a breakpoint in the code to stop execution.

More complex meanings can be constructed using abstract sounds. Sound has several parameters than can be brought into play. These include pitch (frequency), time, spatial location, intensity, and quality. Sound is also temporal - it provides context over time which is analogous to a program progressing through lines of software.

The emerging field of "auditory display" uses sound to convey more detailed information. For debugging purposes, this requires translating arbitrary data into sound through a process called "sonification" or "audification." Through this process, a language of sound is created. What does this mean? While generating different notes to indicate that a variable = 5 or 47 may be less useful, consider using two different notes to indicate bidirectional communications between two programs. The rhythm between the notes conveys how often communications occur, and the relative sizes of the payloads of each message. Music and rhythm are innate tools that in this case would allow us to avoid sifting through pages and pages of interprocessor messaging. More detailed examples of "sonic grammar" have been termed "Earcons."

Auditory Debugging, an article by Marc EisenStadt tells a story of detecting successful compilation of software by sound.

Paul Vickers does research in "auralisation," or mapping program information to sound. He has a great website for Caitin, on the auditory external representation of software programs. For a good hands-on explanation, check out the tutorial on this page. The tutorial is well written, describing how the process and encoding work. It also provides several sound examples of basic constructs such as "IF yielding true" and "IF yielding false. For larger constructs that contain many lines of code or take time to execute, a background drone is added to encapsulate (musically) the construct.

A discussion of auditory display with some example sound files appears in this lecture, Better link for Auditory Display For Human Computer Interfacing.

Another use of audio-visual thread debugging appears in "Place phone to the PC, please" by Martin Wehlou. Martin used a different tune for each thread in a multi-threading (multi-tasking) program. If one thread hangs up (stops running), it can be difficult to trace down unless the thread is something obvious like the user display. But if each thread is associated with a tune, listening to the program until a tune disappears is one way to identify a failure.

Wehlou describes the multithreading sounds as "Freakish, but useful."

A short reference using sound for debugging appears in Auditory Display: Sonification, Audification, and Auditory Interfaces. (edited byGregory Kramer. Addison Wesley, Reading, MA 1994)

An astute reader pointed out: "Doesn't seem to me that "auditory debugging" is a new idea:

"Honey, the car is making a funny noise."
"Yeah, I know. Sounds like the gazuba is rubbing on the fenora."
(Note to the reader: You can convert this general-purpose diagnostic evaluation into a specific diagnostic evaluation suitable for finding your specific problem by substituting gazuba and fenora with names of the actual suspected flakey modules, and by

Visual Debugging

What is Visual Debugging?

Most products and references that use the term Visual Debuggging do not define it. Here are some typical statements:

• "[Product] contains ... and visual debugging"
• "...has integrated tools for ... visual debugging, ..."
• "...complete with visual debugging environment."

But what does this meeeaaannnn?

This term is used to describe several very different concepts which I will attempt to translate below.

• visualizing evidence trails from a program in order to determine correctness of execution
• profiling task execution, program control, data paths, etc.
• visualizing aspects of several processors/programs to understand interactions
• providing color coded source code listings
• graphical development/debugging environments that replace text-based ones.
• displaying normal visual information
• displaying visually-appealing presentations of software constructs
• displaying lines of code highlighted as you step through the program line by line

In my mind, visual debugging is not color coding constructs and expressions in source code listings or displaying array and structure contents is a prettier format.

I prefer the first two descriptions. . .

Visual methods are great for displaying real-life objects and data that are spatial or geographical such as images and maps. Tracing new computer viruses as they infiltrate networks is a great application. Visual methods are also good for time-series data or data having a sequence such as temperature or value of a variable. Visual methods are also very useful for representing "invisible" quantities such as intensity, densities, magnetic or electric fields, or for indicating instantaneous or timeless quantities such as vectors (3D direction of particles in an explosion).

Most implementations of visual debugging require separate application programs running on specific hardware or development platforms, or require significant instrumenting of the source code. They are generally language specific. This is great for debugging parallel or distributed processes or multi-processor implementations, but can be overkill for most programs.

Examples:

• Display time series data such as the state of hardware pins and busses.
• Anything that could potentially be displayed using an oscilloscope.

Some Individual Debugging Methods Proposed

Warning: this section could become incredibly long if I attempted to reproduce all of the extremely useful debugging information located in many other sources. Please see the Reference and Links section for other webpages. This section contains only a subset of the nearly endless combination of techniques that can be used to debug systems successfully.

The need to debug can occur at different points in the life cycle of the product from first compile to post-shipment. Generally, the debugging techniques are the same regardless of the phase, although some methods are more appropriate than others. This section contains a list of several methods, focusing on their usefulness rather than on their execution.

Binary Search Strategy

A technique proposed by some authors when the code will not compile or when attempting to narrow down the location of a bug in a file. Half the file (or function) is commented out and the compile reinitiated. If the compile works, the bug is in the removed section. This brute force method could easily send you down a time consuming pathway because there is no hypothesis about the possible source of the bug, just blind activity. What do you do if the file (program) is thousands of lines long? Explore other methods first.

Printf

In this method, printf's or similar commands to print debugging messages and variable values are sprinkled throughout the code. This allows you to monitor some activities as the program is running. If the program hangs at some point in the execution, this method can be effective in detecting where things went awry by using a series of landmark references such as "initialization complete," "got to the serial function," "saved variable fred=47," etc. However, this is a crude method that forces you to recompile the program.

Recompiling to add print statements artificially changes the state of the entire system - memory locations are different, code addresses are moved, etc. Using this method can actually mask the real problem.

A developer I knew "fixed" a problem once by adding a printf statement to the code. He intended to debug the problem by printing out the value of a variable after several math operations were performed. However, once the print statement was added, he was unable to reproduce the problem. When he removed the printf, the problem returned. His ultimate solution? Ship product with printf compiled in.

If you have access to a debugger, it is much easier and less intrusive to use the debugger to check the value of variables or to step through individual lines of code. However, sometimes using a debugger is not possible or realistic, such as when the system must respond to real time events with timing limitations. Stepping through the code cannot be done in real time. You can stop the code once to check the value of a variable, but restarting execution leaves the system in an indeterminate state with respect to external events. Using this method, you get just one snapshot of the system at a time. To check something else, you must reset the debugger and run from the beginning again.

Pin Wiggling

An interesting device I worked on was a handheld device that measured the time it takes blood to clot. This is useful for determining if people on blood thinners are taking the right dose of medicine, or if patients entering surgery are properly anti-coagulated.

The Hemochron Jr. was an 8-bit micro with 8KB of program space. That's it - 8K. And that includes all the normal text messages and error messages that can be displayed during operation. Internet Explorer is over 800KB; Microsoft Word is over 10MB.

The device ran several different tests, controlled two stepper motors, drove 6 LEDs and read intensity of light detected through moving samples of blood. It had a one-line text display capable of 16 characters. The device performed initialization, motor homing, read LED-based bar codes on disposable blood sampling cuvettes, ran the clotting tests, calculated results using equations and look up tables, and presented results to the user. And did I say that was all in 8K?

I introduced the Hemochron Jr. to tell you about another very low tech method of debugging - wiggling pins. We couldn't set breakpoints because blood continued to clot while we had the processor stopped to read registers. We needed a way to check if certain functions were working properly without stopping the program. We couldn't use a serial port to send debugging data in real time - we had used every single input and output port on the device. Every single one was vital to its proper functioning except one. One lowly output port was used to drive a buzzer that sounded when the test was complete, or when error conditions occurred.

We hijacked that lowly buzzer pin by cutting the trace on the circuit board to the buzzer, and then hooking that pin up to an oscilloscope. At useful places in the software, we could toggle the pin hi or low and watch what happened on the scope. This method was used to identify and verify several bugs, including one that kept the interrupt service routine running well beyond it's 100msec period.

Pin wiggling may seem undignified, but it is a very low cost method to measure time intervals, check the amount of time interrupts run, and measure the time it takes to process certain equations or function calls. And it doesn't require a fancy debugger or system profiler to get some of the same results.

What Did I Just Change?

This probably seems obvious.

The first response should immediately be to UNDO whatever we just did, although what we OFTEN do is to try SOMETHING ELSE!

I have no idea why this is, and I am guilty of it myself. Stop immediately and write down what you changed. After you try 14 things trying to fix the problem after thinking and lamenting, it is amazing how easy it is to FORGET which change caused the current disaster.

A corollary is to make only small changes at a time. This way, it is clearer which change caused the problem.

Reformat your Code

This technique comes with a very strong disclaimer - make a copy of your code first!

Some elusive bugs caused by syntax errors like mismatched braces and improper nesting can be identified by carefully formatting the software source file. Make sure curly brackets line up in the same column to check that they match properly. (If your coding standards dictate the "other" way, then put them back after you confirm).

Refactor long statements that extend beyond a reasonable limit (80-100 characters) to look for strange things. If several statements have a parallel structure (similar nesting or indices), line up the statements to look for inconsistencies.

#include Strong Disclaimer Here: DO NOT "clean up the code" right before burning the ROM or shipping the product. Take this advice from someone who once spent a very late night undoing the cleaning process to make the code work again. Dirty code isn't necessarily wrong, but prettying things up is a regular activity throughout the design cycle, not a last-minute pre-ship activity. Don't even add comments to fully validated software. Don't invoke bad vibes!

Stop and Document!

Before you start changing the code on a tough bug, stop and save your work. Make multiple versions of the file(s) to keep a record of what you tried. (A great use of source code control.) If the system generates any type of output (log files, displayed information, etc.) then save a copy of the output as well. One reader debugging complicated data structures recommended using a diff tool to quickly show the differences in output files when the program was working correctly with when it changed. Outputs make working backwards through time to identify a bug much easier.

Go Home

When nothing seems to work and you are out of ideas, go home. Run an errand. So many times I left work at 9 or 10 PM without getting the bug fixed, and a new idea came to me on the drive home. Or in the shower the next morning. Your brain gets tunnel vision after hours in front of the debugger, and a change of scenery is often enough to initiate a mental control-alt-delete to get you back on track.

Checking for the Obvious

An electrical engineering adage is, "Works better when you plug it in." Similarly, software folks need to look for the obvious mistakes for the language being used. Check things like matching curly braces, ending ending semicolons, etc.