world leader in high performance signal processing
Trace: » nmi

NMI and Blackfin Linux

The Blackfin processor provides for a Non Maskable Interrupt (NMI) source that has a higher priority than any other interrupt or exception. Unfortunately, this feature may not be safely utilized while running Linux. Let's find out why.

Due to workarounds for anomalies that rely on making things “atomic” by disabling interrupts, NMI usage must be strictly reviewed to make sure it is not used when such anomalies are in play (312, 473, 477, 494 to name a few)


The Blackfin processor provides for two different stacks: the supervisor (aka kernel) stack (SP) and the user stack (USP). Since each kernel thread has its own stack and may live anywhere in external memory, there is no way to guarantee that the stack is covered by a CPLB entry as you can easily have more kernel threads than the number of hardware CPLB entries (and thus you cannot lock one CPLB per thread). While you can have nested interrupts, you cannot have nested exceptions, so you have to be careful in exception context.

To address this, a small dedicated stack is created that is guaranteed to be covered by a CPLB entry. This stack space is only used by exceptions. So when an exception occurs on the Blackfin processor, the first thing you need to do is load up the exception stack into the SP register. But wait! You can't simply clobber the existing SP value as the exception may have occurred in supervisor (aka kernel) mode, so when returning from the exception, the SP needs to be restored. That's fine you think to yourself, we'll just save the old SP on the stack … err, wait, on which stack will you be saving it?

Thus we come to the chicken and egg problem. How do you save the stack pointer and restore it later without a stack in the first place?

NMI Loses

The solution utilized in Blackfin Linux is to use the RETN register as a scratch register while in exception space. When an exception occurs on the Blackfin processor:

  • load the old SP register into the RETN register
  • load the exception stack into SP
  • save any misc registers we need onto the exception stack
  • process the exception
  • restore any misc registers off of the exception stack
  • restore the old SP register from the RETN register

Now you see why NMI may not be used. If the NMI occurs while the old SP lives in the RETN register, the old SP will be clobbered. So upon return from NMI, you may have corrupted the system and Linux will blow up (kaboom!).

But what if you were planning on using NMI?

Alternatives to NMI

Usually people use NMI as an interrupt source. To this end, you can simply take a spare GPIO and use it as a generic interrupt source. For more information, see the interrupts page.

However, this still may not be sufficient if you consider the latency in taking an general interrupt to be too high for your needs. If you review the core registers that are available in the Blackfin processor, the only ones that are not used by Linux are:

Register Description Purpose in life
RETN NMI return address needed to utilize the NMI event
RETE Emulator return address needed to debug with a JTAG ICE
CYCLES free running counter needed to do performance profiling
CYCLES2 free running counter needed to do performance profiling

One of these resources needs to be reserved by Linux, so it's up to you to decide which can you live without. To make the actual selection, just change the option in the kernel configuration menu:

Blackfin Processor Options  --->
  Board customizations  --->
    Blackfin Exception Scratch Register (RETN)  --->
      (X) RETN
      ( ) RETE
      ( ) CYCLES


Once you've decided to try and write your own NMI handler, there's a few things to keep in mind. NMI has the highest priority (only Emulation and Reset are higher). While you may think to yourself that you already knew this fact, there's a few perhaps not so obvious considerations.

First, you cannot take an exception. That means you cannot call any code not covered by an ICPLB or access any data not covered by a DCPLB. You can turn off icache/dcache at the start of your handler (thus taking a performance hit due to flushing), or make sure all your code/data is located in L1.

If you need stack space in the NMI, you should preserve a location that is guaranteed to have a DCPLB covering it. Most likely, this means a small chunk of L1. Keep in mind that the same problem encountered with exceptions (needing a scratch register to cache the old SP) will need to be addressed here. The best (only?) solution is to use CYCLES for the exception stack scratch register and then use CYCLES2 for the NMI stack scratch register.

Remember to save/restore state. Since the NMI can happen at any time and the kernel does not have a dedicated register file, you need to push/pop the entire core state on the stack (or at least any register you modify).

Finally, since an NMI can literally interrupt anything running in the processor, and nothing can interrupt NMI, you cannot safely use any kernel resource that involves locks. If a resource has a spinlock associated with it and the NMI occurs while that lock is held, if the NMI handler attempts to grab the same lock, the system will obviously deadlock.

If all else fails and you still want to use NMI, you should know that there are anomaly workarounds in the kernel that rely on the processor not being interrupted (e.g. canceled speculative reads). NMI basically violates that assumption. So review your setup carefully before trying to implement support for NMI.

Custom Handler

If you still wish to define your own NMI handler, create a function evt_nmi in the boards directory for your particular board. It will automatically override the default evt_nmi function that is part of the Blackfin code.

Example Handler

# error "A safe NMI handler (that uses stack) only works with the CONFIG_BFIN_SCRATCH_REG_CYCLES option"
    .rept 512
    .long 0;
    /* save the old stack pointer */
    CYCLES2 = SP;
    /* load up our NMI stack which is in L1 */
    SP.l = _nmi_stack_top;
    SP.h = _nmi_stack_top;
    /* save all core registers */
    /* or use SAVE_ALL_SYS */
    fp = 0;
    /* workaround an anomaly */
#if ANOMALY_05000283
    cc = r7 == r7;
    p5.h = 0xffc0;
    p5.l = 0x0014;
    if cc jump 1f;
    r7.l = W[p5];
     * insert custom NMI behavior here
    sp += -12;
    /* call _my_nmi_code; */
    sp += 12;
    /* restore all core registers */
    /* or use RESTORE_ALL_SYS */
    /* dummy CYCLES read to trigger an update on the CYCLES2 shadow reg */ 
    SP = CYCLES;
    /* now read CYCLES2 from shadow */
    SP = CYCLES2;