The BSL disallows memory reads and writes until a “password” has been transmitted over the serial interface. The password is actually a copy of the interrupt vectors (32bytes) used by the micro-controller, which limits the entropy available by a significant margin: addresses are all even aligned, reset vector is generally start of flash, many interrupt vectors will be identical, general case has all interrupt vectors located in flash. The only defense against this low-entropy brute force attack is the rate limit of the BSL: some versions of the BSL prevent you from changing from 9600 baud until after the password has been verified. However, on devices with small amounts of flash (say, 8Kib), there are only about 61,000 passwords generally in use, which reduces the search time tremendously.

Some BSL versions are also susceptible to side channel attacks are discussed in this paper:
Practical Attacks against the MSP430 BSL

Texas Instruments has improved code security on the new F5xxx series devices in a drastic way: an invalid password will cause a complete device erase. Note that the flash memory also differs from the previous generation devices by being rated to erase and program as low as 1.8V (where the core voltage normally sits – the 5 series has an integrated LDO).

I can see two possible attack vectors – undervoltage to the entire processor (I have a ’5438A running as low as 1.5V, however the BSL is not yet tested down here), or “glitching” the processor (in this case, complete power down) if the start bit of the acknowledgement packet is not transmitted within an allowable window. The second scenario can be defended against in the BSL firmware (delay the acknowledge by a large value).

I have not tried either of these attacks on the 5xxx series yet (specifically, the 5438 and 5438A). Generally, there are easier ways to attack processors to capture embedded code, or simply replicate external functionality based on observed behavior (code security is not on most programmers’ minds). If small-memory variants of the 5xxx series become available (to allow the brute-force searching of the keyspace), the flash memory erase issue would have to be overcome.

In the mean time, you can use some chip disassembly techniques from Flylogic.

On another note, I will be announcing a security related, open source hardware project shortly. Its been keeping me distracted from EVE-Central (trade route tool version 2, specifically), Contribtastic, and working on the EVE-Metrics/EVE-Central unified uploader.

Full color 64×32 display, with a SPI interface, which is much easier to use than previous products (such as the ScreenKeys)

IS15DSBFP4RGB on Mouser