[ACCEPTED]-How to debug a program without a debugger?-debugging

Actually you have quite a lot of possibilities. Either 6 with recompilation of source code or without.

With recompilation.

• Additional logging. Either into program's logs or using system logging (eg. OutputDebugString or Events Log on Windows). Also use following steps:
  • Always include timestamp at least up to seconds resolution.
  • Consider adding thread-id in case of multithreaded apps.
  • Add some nice output of your structures
  • Do not print out enums with just %d. Use some ToString() or create some EnumToString() function (whatever suits your language)
  • ... and beware: logging changes timings so in case of heavily multithreading you problems might disappear.
  • More details on this here.
• Introduce more asserts
• Unit tests
• "Audio-visual" monitoring: if something happens do one of
  • use buzzer
  • play system sound
  • flash some LED by enabling hardware GPIO line (only in embedded scenarios)

Without recompilation

• If your application uses network of any kind: Packet Sniffer or I will just choose for you: Wireshark
• If you use database: monitor queries send to database and database itself.
• Use virtual machines to test exactly the same OS/hardware setup as your system is running on.
• Use some kind of system calls monitor. This includes
  • On Unix box strace or dtrace
  • On Windows tools from former Sysinternals tools like http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx, ProcessExplorer and alike
  • In case of Windows GUI stuff: check out Spy++ or for WPF Snoop (although second I didn't use)
  • Consider using some profiling tools for your platform. It will give you overview on thing happening in your app.
• [Real hardcore] Hardware monitoring: use oscilloscope (aka O-Scope) to monitor signals on hardware lines
• Source code debugging: you sit down with your source code and just pretend with piece of paper and pencil that you are computer. Its so called code analysis or "on-my-eyes" debugging
• Source control debugging. Compare diffs of your code from time when "it" works and now. Bug might be somewhere there.

And 5 some general tips in the end:

• Do not forget about Text to Columns and Pivot Table in Excel. Together with some text tools (awk, grep or perl) give you incredible analysis pack. If you have more than 32K records consider using Access as data source.
• Basics of Data Warehousing might help. With simple cube you may analyse tons of temporal data in just few minutes.
• Dumping your application is worth mentioning. Either as a result of crash or just on regular basis
• Always generate you debug symbols (even for release builds).
• Almost last but not least: most mayor platforms has some sort of command line debugger always built in (even Windows!). With some tricks like conditional debugging and break-print-continue you can get pretty good result with obscure bugs
• And really last but not least: use your brain and question everything.

In general 4 debugging is like science: you do not create 3 it you discover it. Quite often its like 2 looking for a murderer in a criminal case. So 1 buy yourself a hat and never give up.

First of all, what does debugging actually 96 do? Advanced debuggers give you machine 95 hooks to suspend execution, examine variables 94 and potentially modify state of a running 93 program. Most programs don't need all that 92 to debug them. There are many approaches:

1. Tracing: implement 91 some kind of logging mechanism, or use an 90 existing one such as dtrace(). It usually 89 worth it to implement some kind of printf-like 88 function that can output generally formatted 87 output into a system log. Then just throw 86 state from key points in your program to 85 this log. Believe it or not, in complex 84 programs, this can be more useful than raw debugging 83 with a real debugger. Logs help you know 82 how you got into trouble, while a debugger 81 that traps on a crash assumes you can reverse 80 engineer how you got there from whatever 79 state you are already in. For applications 78 that you use other complex libraries that 77 you don't own that crash in the middle of 76 them, logs are often far more useful. But 75 it requires a certain amount of discipline 74 in writing your log messages.

2. Program/Library 73 self-awareness: To solve very specific crash 72 events, I often have implemented wrappers 71 on system libraries such as malloc/free/realloc 70 which extensions that can do things like 69 walk memory, detect double frees, attempts 68 to free non-allocated pointers, check for 67 obvious buffer over-runs etc. Often you 66 can do this sort of thing for your important 65 internal data types as well -- typically 64 you can make self-integrity checks for things 63 like linked lists (they can't loop, and 62 they can't point into la-la land.) Even 61 for things like OS synchronization objects, often 60 you only need to know which thread, or what 59 file and line number (capturable by __FILE__, __LINE__) the 58 last user of the synch object was to help 57 you work out a race condition.

3. If you are 56 insane like me, you could, in fact, implement 55 your own mini-debugger inside of your own 54 program. This is really only an option 53 in a self-reflective programming language, or 52 in languages like C with certain OS-hooks. When 51 compiling C/C++ in Windows/DOS you can implement 50 a "crash-hook" callback which is executed 49 when any program fault is triggered. When 48 you compile your program you can build a 47 .map file to figure out what the relative 46 addresses of all your public functions (so 45 you can work out the loader initial offset 44 by subtracting the address of main() from 43 the address given in your .map file). So 42 when a crash happens (even pressing ^C during 41 a run, for example, so you can find your 40 infinite loops) you can take the stack pointer 39 and scan it for offsets within return addresses. You 38 can usually look at your registers, and 37 implement a simple console to let you examine 36 all this. And voila, you have half of a 35 real debugger implemented. Keep this going 34 and you can reproduce the VxWorks' console 33 debugging mechanism.

4. Another approach, is 32 logical deduction. This is related to #1. Basically 31 any crash or anomalous behavior in a program 30 occurs when it stops behaving as expected. You 29 need to have some feed back method of knowing 28 when the program is behaving normally then 27 abnormally. Your goal then is to find the 26 exact conditions upon which your program 25 goes from behaving correctly to incorrectly. With 24 printf()/logs, or other feedback (such as 23 enabling a device in an embedded system 22 -- the PC has a speaker, but some motherboards 21 also have a digital display for BIOS stage 20 reporting; embedded systems will often have 19 a COM port that you can use) you can deduce 18 at least binary states of good and bad behavior 17 with respect to the run state of your program 16 through the instrumentation of your program.

5. A 15 related method is logical deduction with 14 respect to code versions. Often a program 13 was working perfectly at one state, but 12 some later version is not longer working. If 11 you use good source control, and you enforce 10 a "top of tree must always be working" philosophy 9 amongst your programming team, then you 8 can use a binary search to find the exact 7 version of the code at which the failure 6 occurs. You can use diffs then to deduce 5 what code change exposes the error. If 4 the diff is too large, then you have the 3 task of trying to redo that code change 2 in smaller steps where you can apply binary 1 searching more effectively.

Just a couple suggestions:

1) Asserts. This 5 should help you work out general expectations 4 at different states of the program. As well 3 familiarize yourself with the code

2) Unit 2 tests. I have used these at times to dig 1 into new code and test out APIs

One word: Logging.

Your program should write 8 descriptive debug lines which include a 7 timestamp to a log file based on a configurable 6 debug level. Reading the resultant log files 5 gives you information on what happened during 4 the execution of the program. There are 3 logging packages in every common programming 2 language that make this a snap:

Java: log4j

.Net: NLog or 1 log4net

Python: Python Logging

PHP: Pear Logging Framework

Ruby: Ruby Logger

C: log4c

I guess you just have to write fine-grain 2 unit tests.

I also like to write a pretty-printer 1 for my data structures.

I think the rest of the interview might 15 go something like this...

Candidate: So you don't buy 14 debuggers for your developers?
Interviewer: No, they 13 have debuggers.

Candidate: So you are looking for 12 programmers who, out of masochism or chest 11 thumping hamartia, make things complicated 10 on themselves even if they would be less 9 productive?
Interviewer: No, I'm just trying to see 8 if you know what you would do in a situation 7 that will never happen.

Candidate: I suppose I'd add 6 logging or print statements. Can I ask you 5 a similar question?
Interviewer: Sure.

Candidate: How would you 4 recruit a team of developers if you didn't 3 have any appreciable interviewing skill 2 to distinguish good prospects based on relevant 1 information?

Peer review. You have been looking at the 9 code for 8 hours and your brain is just 8 showing you what you want to see in the 7 code. A fresh pair of eyes can make all 6 the difference.

Version control. Especially 5 for large teams. If somebody changed something 4 you rely on but did not tell you it is easy 3 to find a specific change set that caused 2 your trouble by rolling the changes back 1 one by one.

On *nix systems, strace and/or dtrace can tell you an 2 awful lot about the execution of your program 1 and the libraries it uses.

Binary search in time is also a method: If 6 you have your source code stored in a version-control 5 repository, and you know that version 100 4 worked, but version 200 doesn't, try to 3 see if version 150 works. If it does, the 2 error must be between version 150 and 200, so 1 find version 175 and see if it works... etc.

• use println/log in code
• use DB explorer to look at data in DB/files
• write tests and put asserts in suspicious places

0

More generally, you can monitor side effects 17 and output of the program, and trigger certain 16 events in the program externally.

A Print 15 statement isn't always appropriate. You 14 might use other forms of output such as 13 writing to the Event Log or a log file, writing 12 to a TCP socket (I have a nice utility that 11 can listen for that type of trace from my 10 program), etc.

For programs that don't have 9 a UI, you can trigger behavior you want 8 to debug by using an external flag such 7 as the existence of a file. You might have 6 the program wait for the file to be created, then 5 run through a behavior you're interested 4 in while logging relevant events.

Another 3 file's existence might trigger the program's 2 internal state to be written to your logging 1 mechanism.

like everyone else said:

• Logging
• Asserts
• Extra Output

&

your favorite 2 task manager or process explorer

links 1 here and here

Another thing I have not seen mentioned 13 here that I have had to use quite a bit 12 on embedded systems is serial terminals.

You 11 can cannot a serial terminal to just about 10 any type of device on the planet (I have 9 even done it to embedded CPUs for hydraulics, generators, etc). Then 8 you can write out to the serial port and 7 see everything on the terminal.

You can 6 get real fancy and even setup a thread that 5 listens to the serial terminal and responds 4 to commands. I have done this as well and 3 implemented simple commands to dump a list, see 2 internal variables, etc all from a simple 1 9600 baud RS-232 serial port!

Spy++ (and more recently Snoop for WPF) are tremendous 1 for getting an insight into Windows UI bugs.

A nice read would be Delta Debugging from Andreas Zeller. It's 1 like binary search for debugging