For my 6809 single board computer I wondered if I could do the same for the 6809 processor. After some thought, it looked feasible, although more challenging as the 6809 has a larger and more complex instruction set than the 6502.
The basic idea is to take the instruction to be executed and store it in a buffer in RAM. The values of all CPU registers from the previous instruction trace are restored and the instruction is executed. A jump instruction is placed right after the traced instruction that goes back to the trace program. After execution, the new register values can be displayed and saved. This allows running code that is in ROM, as it gets copied to RAM when executed.
To implement this requires knowing the length of each instruction, as they can vary on the 6809 from one to five bytes depending on the instruction and addressing mode. From my previously written disassembler I already had code that could determine the instruction length and even disassemble it.
A wrinkle in this approach is handling instructions which cause a change in the flow of execution, such as a JMP (jump) instruction which would not return if simply executed in the buffer. We need to handle this instruction as a special case. We can examine the destination address and update the saved program counter accordingly. We don't need to actually execute it since it changes no other registers than the PC. We do need to check for and handle both direct (8 bit) and extended (16-bit) jump instructions. For direct, the destination address needs to be calculated by combining the instruction operand with the current value of the direct page (DP) register.
Jump to subroutine (JSR) in another special case. Here we need to push the return address on the stack (using the saved value of the stack pointer that we trace with) and then calculate the new PC value.
Similarly, we can handle BRA/LBRA and BSR/LBSR like JMP and JSR but the effective address needs to be calculated by adding the offset to the current PC value. Again we don't need to execute the instruction, just update the new PC value.
The interrupt related instructions SYNC, CWAI, SWI, SWI2, and SWI3 can all be simulated by pushing registers on the stack and setting the PC to the appropriate interrupt vector.
RTS and RTI are simulated by pulling registers (in the case of RTI) or just the PC (for RTS) off the stack. One wrinkle is that in the 6809 we need to check the E (Entire) flag in the condition code register to know of we should restore all registers from the stack.
Conditional branches are a little more tricky. They can transfer control to two different places depending on whether the condition is true or not. These are handled by writing into the execution buffer both the instruction being traced as well as different jumps depending on whether the branch is taken or not. The branch destinations can update the PC accordingly. The code in the buffer looks like below:
XXXX XX 03 Bxx $03 (Taken) ; Instruction being traced
XXXX 7E XX XX JMP BranchNotTaken
XXXX 7E XX XX Taken JMP BranchTaken
We execute the instruction in the buffer and let it perform the test. We need to handle all the possible branch instructions, both short and long versions.
TFR (transfer) and EXG (exchange) instructions are okay to run except the cases where the source or destination is the PC. We need to handle those cases manually since it would otherwise change the flow of control.
Similarly, PSHS/PHSU and PULS/PULU could potentially include the PC in the list of registers pushed or pulled. Currently I just check for this case, remove the PC from the list of registers, and warn the user in this case that it is not fully handled yet.
Indexed addressing poses a challenge: we need to handle an instruction that changes flow of control like JMP 1,X with an arbitrary index addressing mode. It might also produce side effects in the case of instructions like JMP 1,X++. These are handled using a trick: instead of JMP, we run a LEAU instruction with the same indexed operand. Then we examine value of U, which should be the new PC. Currently the code can't handle addressing modes that change the U register like JMP ,U++.
A final challenge is PCR relative index addressing. If we move the instruction to the buffer to execute it, the PC relative address is now wrong. I thought about this, and it should be possible to adjust the offset based on the difference between the original instruction location and the address of the buffer where it will be run. This would get a little complicated, so I didn't implement it (yet). For now I just ignore it and display a message at run time that it is not supported.
After working out most of the logic as pseudocode, I implemented and debugged it. I started with the basic instructions and then added all of the special cases, testing them one at a time. Once done, I tested it with some smaller complete programs.
Finally, I was able to integrate it into the "combined" ROM which also contains the ASSIST09 monitor, disassembler, and Microsoft Basic. I added the trace command as a new ASSIST09 "T" command. It took some shuffling but I was able to just get them all to fit in the 8K ROM.
It hasn't been tested exhaustively, and it can't run BASIC because it uses some PC relative instructions, but it seems to work quite well. Here is some sample output:
D000 81 30 CMPA #$30
PC=D002 A=FF B=FF X=FFFF Y=FFFF S=FFDF U=FFFF DP=FF CC=11111000 (EFHINZVC)
D002 25 04 BCS $D008
PC=D004 A=FF B=FF X=FFFF Y=FFFF S=FFDF U=FFFF DP=FF CC=11111000 (EFHINZVC)
D004 81 3C CMPA #$3C
PC=D006 A=FF B=FF X=FFFF Y=FFFF S=FFDF U=FFFF DP=FF CC=11111000 (EFHINZVC)
D006 25 C0 BCS $CFC8
PC=D008 A=FF B=FF X=FFFF Y=FFFF S=FFDF U=FFFF DP=FF CC=11111000 (EFHINZVC)
D008 30 1F LEAX $1F ,X
PC=D00A A=FF B=FF X=FFFE Y=FFFF S=FFDF U=FFFF DP=FF CC=11111000 (EFHINZVC)
D00A 34 50 PSHS U,X
PC=D00C A=FF B=FF X=FFFE Y=FFFF S=FFDB U=FFFF DP=FF CC=11111000 (EFHINZVC)
D00C 0F 41 CLR $41
PC=D00E A=FF B=FF X=FFFE Y=FFFF S=FFDB U=FFFF DP=FF CC=11111000 (EFHINZVC)
D00E CE C0 E7 LDU #$C0E7
PC=D011 A=FF B=FF X=FFFE Y=FFFF S=FFDB U=C0E7 DP=FF CC=01111110 (EFHINZVC)
D011 0F 42 CLR $42
PC=D013 A=FF B=FF X=FFFE Y=FFFF S=FFDB U=C0E7 DP=FF CC=01111110 (EFHINZVC)
D013 33 4A LEAU $0A ,U
PC=D015 A=FF B=FF X=FFFE Y=FFFF S=FFDB U=C0F1 DP=FF CC=01011001 (EFHINZVC)