EVE-Central API is now running on Spray, Akka, built with Scala
The EVE-Central API service was refactored on top of a new framework, both as an aid to scale (for performance reasons), and an exercise in using new tools for web development. Except for a few growing pains, the stack has been reliable and performant.
The bits and pieces that made this possible:
- Scala – A static typed JVM-targeted language with a very good mix of functional programming. Often compared to OCaml and F#, where it shares a lot of similarity. The JVM platform gives a lot of run time performance and a huge software library, but the Java language is hobbled by years of mismanagement (see C# for something not hobbled by Sun). Enter Scala – fast, type safe, expressive, and generally fun to work in. (I still love Python too, for those of you keeping score).
- Akka – an Actor system framework, with lots of inspiration taken from Erlang (great ideas, mediocre execution). Its a message-passing system built for the JVM. While we are still using Akka 1.3, the new clustering capabilities of Akka 2+ are on my radar.
- Spray-server and Spray-can – Spray server is a Scala and Akka library for building web services, in a clean and no-nonsense fashion. Spray-can is the HTTP server component, avoiding the use of a traditional Java servlet container for managing connections. If you’re looking for a way to make RESTful (and non-RESTful) webservices in a high performance environment, give Spray a look. The community support is great.
Note that this is just the old API. No new (published) API calls are available yet, but with the speed gained by building a proper framework around some of the core technology of EVE marketing aggregating, and the inherent testability of the environment, new features should be significantly easier to bring to light.
Also, the EVE-Central user front end remains on the old Python+CherryPy stack for the time being, as the window dressing is being worked on.
Hi Yann
First I’d like to thank you for the great job you’re doing!
But: I’m working with Googlespreadsheets and it seems that the API-Calls are not working, since you made that Update
Do you have any clue what might happened?
Greetings
Goofman
Comment by Goofman — May 3, 2012 @ 5:25 am
Seconding what Goofman said – thanks for all your work
Also having issues – I currently get repeated 504 timeouts on all API accesses.
Comment by Muscaat — May 3, 2012 @ 5:34 am
Something happened late into the second day of runtime – I’m looking into the logs to see why the request rate slowed up. Thanks for the report.
Comment by Yann — May 3, 2012 @ 7:03 am
Google APIs seem to perform PROPFIND requests (which are not supported, and weren’t supported in the previous version either), but this should not cause any problems.
The API is more strict in terms of missing parameters – for instance, trailing spaces or parameters without any values (typeid=) will no longer work.
Comment by Yann — May 3, 2012 @ 7:05 am
All calls to the marketstat are not populating my excel spreadsheet any longer.
5/3/2012 17:46
34 18 Tritanium
35 18 Pyerite
36 18 Mexallon 224190995455.136.001.001.115.503.30
37 18 Isogen
38 18 Nocxium
39 18 Zydrine 424219810266.641000000000.006.05104234608.286.386.09
40 18 Megacyte
11399 18 Morphite
648410805716.121000000000.001.0091737042.176.143.79
267 89 Juggernaut Torpedo
2506 89 Mjolnir Torpedo
2508 89 Bane Torpedo 3809000188.59222.33120.0127.01217.65120.01
2510 89 Inferno Torpedo
A snippet of my data page as it appears now. I’ve made no changes to the page on this end. instead of populating the page is just types out all the data from the call
Not very smart on any of this but thought I’d pass along the change
Comment by Gary Evans — May 3, 2012 @ 2:50 pm
@Gary,
The API was tested for conformance with the previous API. Based upon logs, there are a few notes I should make:
Comment by Yann — May 3, 2012 @ 9:26 pm
And out of nowhere it worked again. It’s all Googles fault
Thumbs up Yann
o/
Comment by Goofman — May 4, 2012 @ 5:30 am
This explains a few things I’ve seen. First off, the quicklook API now returns sell orders for items in a given station in most expensive to least expensive. It’s not a major deal, I’ve worked around it, but this was a touch annoying at first until I figured out what was going on.
The default hours used to be 360 for this API, but now appears to be 24 hours.
Comment by Tom — May 9, 2012 @ 11:05 am
Did you make more changes? After a month I finally got it to work for all of one day…
Comment by Enjelus — May 16, 2012 @ 2:40 pm
There haven’t been any changes to formats or values outside of the first week.
I’m not sure if you’re still having problems or not?
Comment by Yann — May 16, 2012 @ 5:31 pm
The csv dump files are suddenly much larger beginning May 2, like 10 times more rows. What happened? I do historical research, and I’m wondering how to treat the May 2 and after history relative to that prior.
(I already posted this on the eve-central suggestions list – sorry – didn’t mean to dupe post, but hadn’t found your blog at that time)
Comment by Mark — May 24, 2012 @ 12:24 pm
Mark, I’m pretty sure that’s about the time that he started participating in Eve Market Data Relay. http://eve-market-data-relay.readthedocs.org/en/latest/index.html
Comment by Daniel — November 24, 2012 @ 12:44 pm