Throw out obviously bad data? - Page 2 - Fuelly Forums

Click here for important details on the Gas Cubby v3 Free Launch

Reply
 
Thread Tools Display Modes
 
Old 09-08-2014, 03:49 AM   #11
Registered Member
 
Join Date: May 2009
Posts: 4
Country: United States
Quote:
Originally Posted by OliverGT View Post
You can't throw out the top ones, that would be me

I don't lie honest...

But seriously, you could remove the extremes to get the average.

One of the problems though is that there are so many categories of vehicles, that most of the averages are only based on a handful of cars, this is always going to allow the extremes to influence the average more than they should.

Oliver.
Oh, I'm not advocating throwing out anyones data, but not counting a certain amount of highest and lowest ones in the group average may prevent one guy who puts that he fueled up with 11000 gallons from ruining the average. Or maybe not so much the top XX number, but by %. If I suddenly get 80 mpg in my Jetta, it means I forgot to put in a fill up or typed the mileage in wrong. That wrong data point should not be used to calculate anyones average.
__________________

evoblade is offline   Reply With Quote
Old 07-08-2016, 02:06 PM   #12
Registered Member
 
Join Date: Sep 2014
Posts: 10
Country: Canada
Location: ON
Sorry to bump an old thread, but there is a converse to this. I have the only manual transmission listed in my model year. The other cars are getting about 22 mpg, I get about 28-29. Every one of my fill-ups has been thrown out as an outlier.

1989 Saab 900.
__________________

sea_king18 is offline   Reply With Quote
Old 07-08-2016, 02:23 PM   #13
Registered Member
 
RobertV's Avatar
 
Join Date: Feb 2013
Posts: 1,900
Country: United States
Location: San Antonio, TX
Quote:
Originally Posted by sea_king18 View Post
Sorry to bump an old thread, but there is a converse to this. I have the only manual transmission listed in my model year. The other cars are getting about 22 mpg, I get about 28-29. Every one of my fill-ups has been thrown out as an outlier.

1989 Saab 900.
What makes you say/think your data has been thrown out?
RobertV is offline   Reply With Quote
Old 07-08-2016, 03:04 PM   #14
Registered Member
 
Join Date: Sep 2014
Posts: 10
Country: Canada
Location: ON
It hasn't been thrown out, that's a poor choice of words. On the chart that shows the distribution of mileage by fill-up, mine are all counted as outliers. I see the number of outliers incremented each time I add data.

The issue is that as the only standard in the group, I'm about 40% high and the algorithm flags the data.
sea_king18 is offline   Reply With Quote
Old 07-08-2016, 04:08 PM   #15
Registered Member
 
Join Date: Mar 2016
Location: Black Hills of South Dakota
Posts: 150
Country: United States
Location: Rapid City, SD
1989? Gee... The computer knows that's when dinosaurs ruled the world.
__________________
14Corolla is offline   Reply With Quote
Old 07-08-2016, 08:00 PM   #16
Registered Member
 
Join Date: Sep 2014
Posts: 10
Country: Canada
Location: ON
Quote:
Originally Posted by 14Corolla View Post
1989? Gee... The computer knows that's when dinosaurs ruled the world.
That car is replacing my '65....
sea_king18 is offline   Reply With Quote
Old 07-08-2016, 09:05 PM   #17
Site Team / Moderator
 
Jay2TheRescue's Avatar
 
Join Date: Sep 2008
Location: Northern Virginia
Posts: 4,636
Country: United States
Location: Northern Virginia
You only have 5 fillups so far. I think after you've been using Fuelly for a while, they will probably start showing up on the chart, and not be cast as an outlier.
__________________




Jay2TheRescue is offline   Reply With Quote
Old 08-02-2016, 07:07 AM   #18
Registered Member
 
Join Date: Sep 2014
Posts: 10
Country: Canada
Location: ON
Quote:
Originally Posted by Jay2TheRescue View Post
You only have 5 fillups so far. I think after you've been using Fuelly for a while, they will probably start showing up on the chart, and not be cast as an outlier.
I think this is true only if the application recalculates against the entire dataset for the vehicle each time a fillup is added. If it only calculates against the data points already flagged as good, then I'll just keep adding more and more outliers.

I assume that the application is using standard deviation to determine the outliers, but the problem for the '89 900 is that about 3/4 of the fillups come from a single vehicle and it's running 19.3 MPG (US). This vehicle should show a double peak (one for the standards at about 28 MPG, another for the autos at about 21), not a standard bell curve.
sea_king18 is offline   Reply With Quote
Old 09-06-2016, 07:49 AM   #19
Registered Member
 
Join Date: Sep 2014
Posts: 10
Country: Canada
Location: ON
11 fillups now - all have been pitched from the dataset as outliers.

It looks to me as though any given fill is checked against the rest of the data at the moment of entry and, if flagged as an outlier, never checked again. This saves processing time but ignores any emerging patterns in the data.

The '89 Saab 900 has almost 10% of the data marked as outliers.

Is there a process to periodically reanalyze the data? If not, there should be.
sea_king18 is offline   Reply With Quote
Old 09-06-2016, 04:44 PM   #20
Site Team / Moderator
 
Jay2TheRescue's Avatar
 
Join Date: Sep 2008
Location: Northern Virginia
Posts: 4,636
Country: United States
Location: Northern Virginia
There are links on the admin side to make Fuelly redo the math on your vehicle, but when I click on them, I get a no permission message. Maybe someone with higher admin permissions than me can try?
__________________

__________________




Jay2TheRescue is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Powered by vBadvanced CMPS v3.2.3


All times are GMT -8. The time now is 11:40 AM.


Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.