Yann’s Blog

I thought I’d do a small interlude into the embedded hardware and firmware space (which is generally my main area of work) from the normal EVE-Central heavy side of this blog. I have been musing about the code security protections of the MSP430 micro-controller, specifically the Boot Strap Loader (BSL), which remains enabled at all times, even if the JTAG interface is disabled (via a fuse on most MSP430s).

The BSL disallows memory reads and writes until a “password” has been transmitted over the serial interface. The password is actually a copy of the interrupt vectors (32bytes) used by the micro-controller, which limits the entropy available by a significant margin: addresses are all even aligned, reset vector is generally start of flash, many interrupt vectors will be identical, general case has all interrupt vectors located in flash. The only defense against this low-entropy brute force attack is the rate limit of the BSL: some versions of the BSL prevent you from changing from 9600 baud until after the password has been verified. However, on devices with small amounts of flash (say, 8Kib), there are only about 61,000 passwords generally in use, which reduces the search time tremendously.

Some BSL versions are also susceptible to side channel attacks are discussed in this paper:
Practical Attacks against the MSP430 BSL

Texas Instruments has improved code security on the new F5xxx series devices in a drastic way: an invalid password will cause a complete device erase. Note that the flash memory also differs from the previous generation devices by being rated to erase and program as low as 1.8V (where the core voltage normally sits – the 5 series has an integrated LDO).

I can see two possible attack vectors – undervoltage to the entire processor (I have a ‘5438A running as low as 1.5V, however the BSL is not yet tested down here), or “glitching” the processor (in this case, complete power down) if the start bit of the acknowledgement packet is not transmitted within an allowable window. The second scenario can be defended against in the BSL firmware (delay the acknowledge by a large value).

I have not tried either of these attacks on the 5xxx series yet (specifically, the 5438 and 5438A). Generally, there are easier ways to attack processors to capture embedded code, or simply replicate external functionality based on observed behavior (code security is not on most programmers’ minds). If small-memory variants of the 5xxx series become available (to allow the brute-force searching of the keyspace), the flash memory erase issue would have to be overcome.

In the mean time, you can use some chip disassembly techniques from Flylogic.

On another note, I will be announcing a security related, open source hardware project shortly. Its been keeping me distracted from EVE-Central (trade route tool version 2, specifically), Contribtastic, and working on the EVE-Metrics/EVE-Central unified uploader.

I have had several questions regarding the “rle_unpack” function in libevecache. In order to not repeat myself in e-mails, I decided to make a quick post describing the logic:

The market rows are compressed with an odd variant of run-length-encoding. In this case, the extra “0″ bytes are suppressed and encoded into one byte.

The row starts with a opcode byte, which is split as follows (high bit to low bit). You can find this broken out in the struct packer_opcap.

x x x t y y y b
7 6 5 4 3 2 1 0


There “t” is a flag to say “unpack 0s” if the bit is set, and “copy from source” if the bit is not set. So if the bit is set, “XXX” bytes of “0″ are copied to the output buffer, and if its not set, “XXX” bytes are copied from the input to the output.

This repeats with B and YYY. After one opcode byte has been processed, the next opcode byte is read, or processing is terminated if the input buffer has been exhausted.

This took a fair bit of time to figure out, with lots of sample data and market logs as the study set.

How much space does it save per order row? About 10-30%, from my samples.

You heard CCP, Dominion lands on December 1st. Hopefully this major patch will be a little less buggy than some of CCP’s previous mass roll-outs. As for EVE-Central.com, I will be working hard to make sure all of the relevant new items (namely the sovereignty devices) are in the database near the launch. I will also be launching the updated user registration system next week (since the current one is literally broken, an unintended side effect). This update hasn’t yet hit the git repository, but stay tuned.

After things stabilize with Dominion, work will progress on the EVE-Central Batching Tradefinder 2.0, followed by some UI cleanups and statistics features.

Remember to set a long skill training!

An update for those of you using the EVE-Central mail push service:

1. The CSV header is no longer CSV Header, but now actually describes the columns
2. The header has been updated to include “source” information (cache, syndication, etc). This may not be available in all cases. The exact sources aren’t set in stone yet.

Please note that each e-mail can now contain multiple types.

In addition, the daily dumps have had their Issued timestamps truncated out for a long time now. This issue has been resolved for all future dumps.

We’ve done some quick tuning to the SMTP push service to keep up with the much larger volume of traffic generated by the new uploader. You may notice that the e-mails now contain many types, concatenated together. It turns out that mailman is simply not ideal when distributing 100 e-mails a second

For now, the worst case latency from upload to e-mail is 1 minute, with a fixed upper size per e-mail.

On another note: EVE-Metrics now uses AMQP to distribute uploads. We are reading the uploads from them and incorporating it into the EVE-Central stream and site.

Looks like Yahoo and Hotmail/Live/Microsoft have synced up their e-mail blacklists. The server eve-central.com uses for outbound messages has been blacklisted by both of these entities. I am trying to get this resolved, but in the mean time, if you need a password reset, please e-mail us directly. We’ll use an alternate outbound mail method to get your password to you.

Edit: It looks like the group of IP addresses at my co-location facility has been listed in the SBL. http://www.spamhaus.org/sbl/sbl.lasso?query=SBL80992. Now I am also working with the hosting provider to help clean up this mess. Joy.

Edit 2: Things are in motion to help clean up this mess.

Edit 3: SBL entry closed up, we will see when Yahoo/Hotmail/etc pick this up.

The latest version of Contribtastic is now available. We’re up to alpha3 now.

Fixed in this release:

1. Crash when reading certain NPC standing cache files
2. Possibility of having two upload threads cruising at the same time, leading to duplicate uploads
3. No longer uploads old cache files on first startup or after reseting the configuration

Login functionality is still broken (“by design”).

The download site is still the same: http://dl.stackfoundry.org/contribtastic/. You can also file official bug reports at http://bugs.stackfoundry.org, but comments in the blog or forum are also welcome.

(tl;dr: New uploader with cache reading capability is available as an early alpha release: Download here )

When I was working on libevecache, I was imagining making a quick-turn uploader which used the functionality from the library allowing EVE-Central uploader users to no longer have to hit the magic button.

That was several months ago now. How time flies when everything else is demanding your attention: house stuff, hardware stuff, work stuff, other stuff. I wasn’t even distracted by video games or EVE-Online. My dream of a quick turn uploader clone obviously was just a dream, up until now.

I am releasing Contribtastic 2.0-alpha1, available on the download site as a Windows installer. Contribtastic is a bit different than originally intended (I am reusing the wxPython skeleton from the MarketUploader 1.3, not the pure C++ Qt framework), but this is all to get it out the door quickly: I spent more time downloading and installing the EVE client on my shiny new Windows 7 system than writing code or adapting the new framework to work with libevecache. Go Python go.

There are some known errata:

1. Contribtastic doesn’t always detect the cache folder path correctly
2. The user login functionality is basically totally broken. (Sorry, no credits for uploads yet)
3. Upload suggestions aren’t working
4. 0 item cache files will not be interpreted by the EVE-Central.com parser (a server side issue)
5. Memory usage has not been analyzed
6. Bonking upload now while already scanning for uploads can break things
7. Only tested on Windows 7

This is an alpha version, but please enjoy!

Edit: We should be back up – watch your spam folders though

Yahoo! mail is currently not being accepted from the EVE-Central.com and StackFoundry.com servers (38.113.114.184, 38.113.114.187). We are doing the correct customer support contact magic in order to re-enable delivery with Yahoo!, but cannot give definite timelines.

We would like to remind all EVE-Central.com mail feed users that you should NOT sign up webmail accounts or externally hosted accounts, including rebranded accounts (such as Yahoo or Google for your domain). The mail feed is for live processing on your server, not for batch downloading from another server. The daily CSV dumps are available for this reason.

The old IGB is no longer supported, and we are busily removing all references, hacky workarounds, and alternate styles in the code base which were brought about by its brain dead behavior. However, this means registering on the site will be problematic.

As a workaround, please visit http://eve-central.com/users/ directly to register. We will be implementing the new IGB access methods soon, so you can register from SiSi.