E-mail push update: Corrected CSV headers, fixed daily dumps
An update for those of you using the EVE-Central mail push service:
- The CSV header is no longer CSV Header, but now actually describes the columns
- The header has been updated to include “source” information (cache, syndication, etc). This may not be available in all cases. The exact sources aren’t set in stone yet.
Please note that each e-mail can now contain multiple types.
In addition, the daily dumps have had their Issued timestamps truncated out for a long time now. This issue has been resolved for all future dumps.
please make sure to at least include the source field in all rows, even when it is empty.
number of fields in row not matching up with number of fields in header == unhappy CSV parser.
and thanks for all the work you are putting into this!
Comment by N.N. — November 17, 2009 @ 2:47 am
Surprise !
)
Took me a while to notice things had changed. I only have the mail feed open for active playing periods, so when the mailfeed didnt work I assumed other things had broken, not the format. Still a good improvement, I’ve played a little with themail-identifier-subject thingy to get some indication of the original contributor (mainly myself
Comment by Msquare — November 18, 2009 @ 1:12 pm
… Wow, you’re changing the layout tonight, too? Now suddenly the lines do not contain “,evetrace” (or something like that, those mails are gone already) at the end of each line. The original eve dump contains a trailing “,” on each line, now that is gone.
What is your intention on how much you will “massage” the original EVE dump files?
Will the header line be a true CSV header (gasp, even more parsing for me) or is it still just a line of constant content to (mostly) disregard?
Comment by Msquare — November 18, 2009 @ 2:26 pm
The lack of a trailing , was due to the use of Contribtastic/libevecache sourced uploads (which doesn’t emit the trailing ,).
The header had a small snafu, but does correctly describe the fields, with the only caveat is the optional source field (at this time).
Comment by Yann — November 18, 2009 @ 3:00 pm
Date: Wed, 18 Nov 2009 01:35:01 +0000 (UTC)
Subject: [evec-upload] Upload
price,volRemaining,typeID,range,orderID,volEntered,minVolume,bid,issued,duration,stationID,regionID,solarSystemID,jumps,source
…
45.4099,48765.0,37,-1,1345142586,100000,1,True,47.99,45506.0,37,32767,1363864325,45506,1,False,2009-11-18 00:45:01.000,90,60012001,10000037,30003016,5,
…
note how in the middle of that row, it suddenly starts a new one.
this just an artefact of adding the trailing comma for no-source rows?
because parts of that mail do have a trailing comma, other parts dont.
Comment by Anonymous — November 19, 2009 @ 10:55 pm