Throw out obviously bad data?

[evoblade]

Originally Posted by OliverGT

You can't throw out the top ones, that would be me

I don't lie honest...

But seriously, you could remove the extremes to get the average.

One of the problems though is that there are so many categories of vehicles, that most of the averages are only based on a handful of cars, this is always going to allow the extremes to influence the average more than they should.

Oliver.

Oh, I'm not advocating throwing out anyones data, but not counting a certain amount of highest and lowest ones in the group average may prevent one guy who puts that he fueled up with 11000 gallons from ruining the average. Or maybe not so much the top XX number, but by %. If I suddenly get 80 mpg in my Jetta, it means I forgot to put in a fill up or typed the mileage in wrong. That wrong data point should not be used to calculate anyones average.

[sea_king18]

Sorry to bump an old thread, but there is a converse to this. I have the only manual transmission listed in my model year. The other cars are getting about 22 mpg, I get about 28-29. Every one of my fill-ups has been thrown out as an outlier.

1989 Saab 900.

[RobertV]

Originally Posted by sea_king18

Sorry to bump an old thread, but there is a converse to this. I have the only manual transmission listed in my model year. The other cars are getting about 22 mpg, I get about 28-29. Every one of my fill-ups has been thrown out as an outlier.

1989 Saab 900.

What makes you say/think your data has been thrown out?

[sea_king18]

It hasn't been thrown out, that's a poor choice of words. On the chart that shows the distribution of mileage by fill-up, mine are all counted as outliers. I see the number of outliers incremented each time I add data.

The issue is that as the only standard in the group, I'm about 40% high and the algorithm flags the data.

[14Corolla]

1989? Gee... The computer knows that's when dinosaurs ruled the world.

[sea_king18]

Originally Posted by 14Corolla

1989? Gee... The computer knows that's when dinosaurs ruled the world.

That car is replacing my '65....

[Jay2TheRescue]

You only have 5 fillups so far. I think after you've been using Fuelly for a while, they will probably start showing up on the chart, and not be cast as an outlier.

[sea_king18]

Originally Posted by Jay2TheRescue

You only have 5 fillups so far. I think after you've been using Fuelly for a while, they will probably start showing up on the chart, and not be cast as an outlier.

I think this is true only if the application recalculates against the entire dataset for the vehicle each time a fillup is added. If it only calculates against the data points already flagged as good, then I'll just keep adding more and more outliers.

I assume that the application is using standard deviation to determine the outliers, but the problem for the '89 900 is that about 3/4 of the fillups come from a single vehicle and it's running 19.3 MPG (US). This vehicle should show a double peak (one for the standards at about 28 MPG, another for the autos at about 21), not a standard bell curve.

[sea_king18]

11 fillups now - all have been pitched from the dataset as outliers.

It looks to me as though any given fill is checked against the rest of the data at the moment of entry and, if flagged as an outlier, never checked again. This saves processing time but ignores any emerging patterns in the data.

The '89 Saab 900 has almost 10% of the data marked as outliers.

Is there a process to periodically reanalyze the data? If not, there should be.

[Jay2TheRescue]

There are links on the admin side to make Fuelly redo the math on your vehicle, but when I click on them, I get a no permission message. Maybe someone with higher admin permissions than me can try?