Comments on: Data Import Test

Data Import Test

pb — Thu, 28 Aug 2008 16:18:04 -0800

Have you been waiting for fuel-up data import? Help us test it out. We finally have the first version of data import ready to try out, and we'd like to get some more testing with real data to work out the bugs before we link it from car profiles. If you're feeling adventurous, you can read about Importing Your Data Into Fuelly to see how the CSV file should be formatted. Then you can try it out by adding /import to your car profile URL. From there you can try out the new import feature. Please post any comments/problems here. Thanks!

By: stilldavid

stilldavid — Thu, 28 Aug 2008 20:08:46 -0800

Phew; I just entered my last 9 months or so worth of fill-ups by hand from a pile of old receipts. I'm sure I've lost a receipt or two, but the bulk of them are there - 31 fill-ups added!

The import worked flawlessly. Keep up the good work!

By: pb

pb — Thu, 28 Aug 2008 21:26:43 -0800

Great to hear! Thanks for trying it out. Nice idea to tag them with imported too so you can view them separately and know which ones were from a batch.

By: brentil

brentil — Fri, 29 Aug 2008 06:57:48 -0800

I just added 3 different cars worth of data ranging back to 2005, about a total of 150 data elements across all 3. No problems importing at all, went completely smoothly. The data for the most recent purchases was imported in with data I had already hand entered from 2006 and 2008 and integrated successfully.

By: cdurham

cdurham — Fri, 29 Aug 2008 07:46:56 -0800

Just imported 2 vehicles with a little less than 100 entries. Saved me a TON of time! Worked perfectly. Thanks!

By: structures282

structures282 — Fri, 29 Aug 2008 08:46:28 -0800

"Then you can try it out by adding /import to your car profile URL. From there you can try out the new import feature. "

Can you digress on these instructions. I can't seem to figure this out. Thanks

By: structures282

structures282 — Fri, 29 Aug 2008 08:54:25 -0800

Oh I figured it out. Bear with me, I'm retarded

By: Pontiac

Pontiac — Fri, 29 Aug 2008 11:14:17 -0800

What happens when one uploads the same file multiple times?

By: pb

pb — Fri, 29 Aug 2008 11:17:00 -0800

Do you want to upload the same file multiple times? Is there a reason to?

By: ObliteRon

ObliteRon — Fri, 29 Aug 2008 15:15:39 -0800

Worked for my two vehicles.

By: aruthas

aruthas — Sat, 30 Aug 2008 06:31:22 -0800

It worked smoothly for me, once I figured out that I could only import comma-separated values, and not tab-separated values as was produced by mySQL export.

By: philipsd6

philipsd6 — Sat, 30 Aug 2008 15:01:10 -0800

I need a way to do a mass delete, so I can get rid of a bad import. I imported my csv, and I ended up with these entries:

Mar 11 -- 369
Mar 09 -- 8445
Mar 07 -- 8044

Which is obviously wrong. My earliest reading should be from Mar 11.

By: mathowie

mathowie — Sat, 30 Aug 2008 15:19:14 -0800

You should be able to delete the bad entries from your log one at a time for now, at least.

By: dshamilton

dshamilton — Sat, 30 Aug 2008 18:22:02 -0800

imported 3+ years of fill ups. worked perfectly.

By: topeju

topeju — Sun, 31 Aug 2008 05:45:43 -0800

Note: The import script seems to require a "gallons" field even if you're using litres. If your units are set to litres, the gallons field is still imported as litres. (It is possible that I titled the column in my CSV "liters" rather than "litres".)

By: philipsd6

philipsd6 — Sun, 31 Aug 2008 13:12:18 -0800

@mathowie Yes, I realize that, however it can be tedious to delete 50 entries one at a time. :) It would be really nice if I could say... do a bulk delete of all entries tagged as "imported".

My current workaround is to create a new car, and test the import there, then just delete the car. But... that doesn't help me get rid of the 50 entries I added to my "real" car. :P

By: freakonaleash1187

freakonaleash1187 — Sun, 31 Aug 2008 21:12:19 -0800

I just added all my fuel-ups from 2004-2006 and it worked perfectly! Thanks!

By: IronMan

IronMan — Mon, 01 Sep 2008 07:03:11 -0800

A couple minor problems:
1 was duplicate entries. I had just entered 12 or so entries manually into Fuelly., and then I found the import test and wanted to try that. But my spreadsheet was up to date, so the most recent entries were duplicated into Fuelly. I manually deleted them, so it wasn't too bad.

The other problem, again minor, was my spreadsheet already had the mpg and other info on there. The import doesn't like anything extra and gave me errors. It also doesn't like the columns out of order.

So for a suggestion - just look at the headings and take the data under there, no matter what order. If the headings don't match what you're looking for, or are non-existent, ignore the column entirely.

And now for another, more ambitious, suggestion - I was using Google Docs for my spreadsheet. Wouldn't it be cool to use the live Google doc as input to Fuelly? We could enter the Gmail login info and then hit a "Sync" button to automatically import.

By: mathowie

mathowie — Mon, 01 Sep 2008 08:25:16 -0800

The other problem, again minor, was my spreadsheet already had the mpg and other info on there. The import doesn't like anything extra and gave me errors. It also doesn't like the columns out of order.

We figured if you're in excel or something similar, it's really easy to delete columns and move them around so we limited the format we accept, because on the flip side it's really difficult to process user input. To simplify the process we force it in one expected way.

Wouldn't it be cool to use the live Google doc as input to Fuelly? We could enter the Gmail login info and then hit a "Sync" button to automatically import.

We considered that too, but as far as I know right now there isn't a Google Docs API that would let our server grab your data without having to share your username/password. It's really bad application design to ask people to share extremely sensitive information like their gmail login and we'd never do that to members here.

By: pb

pb — Tue, 02 Sep 2008 11:03:10 -0800

Thanks for the great testing and feedback everyone. Glad to hear it's working (for the most part).

The import does fine with extra columns and the columns can be in any order. Just verified that with a test using this sample data. The columns aren't in any specific order, and the extra column foo is simply ignored.

I made a few changes to importing this morning based on your feedback:

Liters is now working as a column header (only Litres was accepted before.)
Only vehicles set for odometer tracking will import an odometer value. Vehicles set for tripometer tracking will only import the miles or kilometers column, and ignore an odometer column if one exists.
Duplicate entries will be ignored. If you upload a CSV file several times the import will only add new entries and let you know how many entries were skipped. The duplicate check is based on exact values for for fuelup_date, fuel volume, and distance columns.

Thanks again for all the testing, I think we're ready to add this feature to the site.

By: IronMan

IronMan — Tue, 02 Sep 2008 18:40:17 -0800

Wow, you guys are fast. Did you change it to accept extra columns and out-of-order columns recently? Because yesterday I had errors trying to import my file until I cleaned it up. But, as you mentioned, it wasn't a tough thing to do.

Thanks for all the work. This is a great site and the import is a real nice feature.

By: RavenFan

RavenFan — Tue, 02 Sep 2008 20:00:28 -0800

Just imported 467 records for my 94 accord, 357 records for my 98 crv and 25 records for my 08 legacy. Everything went fine. I still have some recent entries to add but I'm almost there! I've been keeping gas mileage records on all my vehicles back to my first car in 1981. Thanks for a great website.

By: zahampton

zahampton — Thu, 04 Sep 2008 07:56:41 -0800

I'm trying to import but it doesn't like my odometer column. It tells me that at least one value isn't numeric. I've got the column formatted as numeric to one decimal place. When I remove the decimal, the upload works fine. Anyone else run into this?

By: pb

pb — Thu, 04 Sep 2008 08:51:55 -0800

zahampton, I don't think it was the decimal causing the problem with your import. When I took a look at the CSV file you uploaded, it had several blank rows at the bottom of the file. You had three rows of blank values that looked like this: ,,,,,,,,,.

So when the importer hit that line, it expected some values and said, "Hey, that's not numeric!" I just updated the importer to ignore all completely blank rows in a CSV file. So you should be able to import your fuel-ups now.

By: TristanP

TristanP — Thu, 04 Sep 2008 20:04:18 -0800

Just uploaded 140 records. Once I had my columns and data done correctly, the upload went fine.

By: ryanruppe

ryanruppe — Thu, 18 Sep 2008 11:09:12 -0800

Just imported data from some other mpg tracking site. After I cleaned the data and saved it as a CSV, the upload went very smoothly. The import howto is a big help!