Who's a good mathematician/statistician?
 

I'm wondering how many miles should be accrued/gallons consumed before a statistically accurate FE value can be derived?

Certainly one mile isn't enough, and 100,000 miles is more than enough.

Certainly one cup of gas isn't enough, and 1,000 gallons is more than enough.

It's somewhere in between. Maybe it's not a single value?

Super High Mileage competitions perhaps can get away with a smaller miles/gallons sample due to their extreme efforts at measuring accuracy...? If they claim 10,000 mpg, what does that mean? I'm pretty sure if they were given one gallon of fuel they wouldn't be able to go 10,000 miles...? But then they don't have the luxury of dragging the competition out for days or weeks. Did VW use a good enough sample for their 1L claim- one trip? If someone "hypermiles" a Hummer for a specific trip segment and gets 15 instead of 10, yet a 90-day gaslog shows 10, what does that mean? Is the 90-day interval accurate?

Inquiring minds want to know...

Personal opinion ? Good cuz I'm offering mine LOL
I'd say One years entries in a gas log would be fairly representative of FE .Barring no other real mechanical/aero changes of course.

I'd like something less arbitrary and more "grounded".

Are you saying I'm "flighty"? :D or you want real numbers ? ;)

very interesting topic.... i think i will revisit when i am more awake.

i now wonder if the 90day average is the best marker to use...

anyway there has to be a good point with 6 sigma or whatever, where the standard deviation almost becomes the range....

ah i dont know if that makes sense to anyone and i am too tired to edit the post again. i will be back tomorrow :) i would say ben is the #1 mathematician here though.

Here's what I know. In statistical language, you are trying to use the standard deviation of the population, and the delta that you want to be able to quantify, to determine what sample size you would need in order to say with a certain % confidence that you have definitely seen the delta.

Practically, for me at least, my tank to tank averages bounce around so much that to observe something like a 3% mpg increase I would need probably a hundred tanks. But, during those hundred tanks, lots of other stuff changes. So I'm toast, really.

But, with a scan gauge, the standard deviation might be small enough to say that you could observe a 3% gain or loss after just a few trips.

Tomorrow at work I'll drag out my stat stuff and see if I can give you the right terminology so that you can poke around on the net a little and convince yourself. Oh wait, it came to me as I was typing.

https://www.stat.uiowa.edu/~rlenth/Power/index.html
looks like a good resource. Use the two-sample t test. Putting in my kind of fill numbers, I got a sample size of 64 tanks.

The terms to look for are "Power and Sample Size". We usually use a power (1-beta) of .8 and an alpha of .05 or .1

I like this one a little better:
https://www.chia.org/programs/2006/Po...Calculator.htm
All of the terms mean what you think. Sigma is the standard deviation of your initial data. Mu0 is your starting mean. Mu1 is the mean you want to be able to see. Oh wait, it's not running on my computer. Maybe it's something to do with my settings. Hopefully it will work on yours.

Anyway, have a look and tomorrow I'll see what I can clear up.

"But, with a scan gauge, the standard deviation might be small enough to say that you could observe a 3% gain or loss after just a few trips."

What if the Scangauge is used to take a "snapshot" of a very limited, specific occurance or set of circumstances and the "results" show FE 50% greater than any gaslog entry or average? Unless conducted properly via A-B-A, or with a great enough "field" sample, I would think such "snapshots" to be of limited utility if not worthless.

Yep, pretty much. Just like recording mileage going downhill 1000 feet of 20miles is compared to doing the same thing for a round trip of the same distance. Testing really isn't matter of math, since we should be trying to control all variables but one. Obviously, we won't get perfect results, but as MetroMPG has demonstrated, with accurate instrumentation we can determine what's better or worse up to the the standard deviation of a set of runs like Bill said. Unfortunately, if the standard deviation/background noise is too high, we may not be able to tell if certain mods work w/o changing the whole setup.

The VW 1L fuel efficiency was over the Euro urban/suburban cycles, so take that as you will. The only real world test of the 1L I know of was at an average speed of ~45mph over ~143 miles with a roughly 200ft increase in elevation in the rain, and fuel consumption was .89L/100km. If you look on googleearth you can probably map out the likely route. I'd say it could easily get 250+mpg@55mph.

Everyone should read this book.

Editorial Reviews

Amazon.com
"There is terror in numbers," writes Darrell Huff in How to Lie with Statistics. And nowhere does this terror translate to blind acceptance of authority more than in the slippery world of averages, correlations, graphs, and trends. Huff sought to break through "the daze that follows the collision of statistics with the human mind" with this slim volume, first published in 1954. The book remains relevant as a wake-up call for people unaccustomed to examining the endless flow of numbers pouring from Wall Street, Madison Avenue, and everywhere else someone has an axe to grind, a point to prove, or a product to sell. "The secret language of statistics, so appealing in a fact-minded culture, is employed to sensationalize, inflate, confuse, and oversimplify," warns Huff.
Although many of the examples used in the book are charmingly dated, the cautions are timeless. Statistics are rife with opportunities for misuse, from "gee-whiz graphs" that add nonexistent drama to trends, to "results" detached from their method and meaning, to statistics' ultimate bugaboo--faulty cause-and-effect reasoning. Huff's tone is tolerant and amused, but no-nonsense. Like a lecturing father, he expects you to learn something useful from the book, and start applying it every day. Never be a sucker again, he cries!

Even if you can't find a source of demonstrable bias, allow yourself some degree of skepticism about the results as long as there is a possibility of bias somewhere. There always is.

Read How to Lie with Statistics. Whether you encounter statistics at work, at school, or in advertising, you'll remember its simple lessons. Don't be terrorized by numbers, Huff implores. "The fact is that, despite its mathematical base, statistics is as much an art as it is a science." --Therese Littleton

I think you have two separate issues, the first being the margin of error in any given data point. So each reading can vary at least by, say, +- 0.1 gallons and +- 1 mile. The second is a set of values, each of which have an individual margin of error, that also each represent additional variables (such as driving conditions, engine changes, route changes.... you know, just about anything).

So then comes the question of just what you are looking for. Are you looking for accurate mileage such that the margins of error are reduced? Or are you looking for overall average based on all conditions? Or, at least more interesting to me, are you looking to quantify the best mpg you can achieve?

In my day job, I do performance analysis... and the "best" performing sample is acutally of most interest to me because it indicates, ruling out a bad data point and taking into account the margin of error, what is achievable under the best circumstances... For my car, I've had one tank at 58 mpg, several at 54, and many at 50-52. So unless I screwed up the 58 case, I know I can hit 58 if I want.

I am not sure statistics is the right animal in this case anyway. Statistics are used to describe samples that are often random, or have some certain distribution or arrival rate. Test scores. for example, follow a gaussian distribution. People showing up at a store follow a Poisson distribution if I recall correctly. Fuel mileage, on the other hand is simply the ideal achievable minus any negative variables in play during interval (OK, OK, this sounds like statistics, but I do think there is a distinction because one can measure each valuation with a defined margin of error, unlike real statistically described phenomena such as molecule activity is a gas or demographic sections of a census). The other funky thing where statistics can play is demonstrated by the census example, and relatedly by vote counting. Any large number of things cannot be accurately counted (if you put enough jelly beans in a jar or ballots in a box, every time you count them you will get a different answer), so stastical methods can help you understand what is there when there is too much there to get a handle in typical ways.

Or so I remember from college, but that was a long time ago :-) Observations of flaws in my..... observations..... appreciated.

Well, it all goes back to the standard deviations of the before and after situations. If you can narrow down a set of conditions that give you the same SG results on the same segment of your drive, then make your change, and then measure again, you might be able to see it. It depends how consistent you were before, how consistent you are after, and how small of a difference you are trying to see.

If you are very reliable before, and are looking for a large change, then one sample might be enough. For example, running on 4 donuts vs 4 regular tires.

But the improvement of something like a mirror delete might not be possible to prove statistically. Doesn't mean it didn't help, just that you can't statistically prove it.

>>But the improvement of something like a mirror delete
>>might not be possible to prove statistically. Doesn't mean it
>>didn't help, just that you can't statistically prove it.

Your right on the mark there. Many theories are right but cannot be proven experimentally because of precision. General Ralativity wasn't verified for years, until there was an eclipse with a know star position close to it... only then could the bending of light be experimentally measured with enough magnitude to draw a decent conclusion.

So I finally ran the numbers.
Example #1 (Bill's real data)
My tank fills for the last year had a std deviation of 1.3 mpg.

I added an air dam to my car. If I want to see a difference of .5 mpg, I need to collect 85 more samples. That's more than 3 years worth of data. In my opinion, there would be too many outside factors during those three years to say that anything was really valid.

Example #2 (made up example for "Bob")
Bob uses a SGII and on a particular trip segment he thinks he is pretty repeatable. He records data and finds that his std deviation is 0.3 mpg.

If Bob wants to see a change of 0.5 mpg, then he only needs to collect 6 samples, which would probably take just a week or two. This seems like a reasonable time period.

Example #3 (Bill's real data again)
I have heard that Acetone will bump my mileage by 10%. This is ~2.7 mpg for me. How many tanks would I need to confirm or refute the claim?

I would need 4 samples. During the two months required to get those 4 fills, there would be a lot of outside influence, but I might be okay with it if the weather and my driving patterns didn't change TOO much. Plus, for a 10% adjustment, I'm willing to put up with a little more hassle.

To confirm or refute a 5% improvement, I would need 13 samples. Too long of a time period.

Anyway, hopefully those examples were helpful. And again, something might help you, but not necessarily be statistically provable. Don't let the statistics get in the way of trying.

Bill it looks like this is pretty simple but I'm simpler still. How would this work as far as how we try to test things here. Tank to tank is out of the question IMHO.

How would this work as far as testing goes. When we test things with the SG we usually do 4-6 runs without the mod, then do the same with and then without in different directions. Most of the time in my experience the fuel results are with in .2 mpg or less when in the same direction. So what percentage would increase would you have to have to say it was a good result. Keeping in mind that most mods give somewhere around 2-3%.

I know it not that easy.

Post a data set and I'll run the stats and tell you if the A and B are statistically significantly different.

And once I have a typical set I'll poke around a little and see what the smallest difference we could prove would be.

It needs to be something with lots of runs, though. I know that already. 4 might work if you are looking at a big change and are dead-on in repeatability. Having more runs is always better, of course.

It will be tomorrow before I get to do any analysis.

No I don't have any thing with lots of runs and the % increase on the SG on most mods is around 2-3% so It won't fit the criteria. This has been an interesting thread. thanks

Okay, if anyone else has a set they want analyzed Friday, let me know.

I just read through this thread. My conclusion is that fill data and the scangauge will not convince any sceptics about the effectiveness of the Somender grooves in the combustion chamber. On the other hand I can go on believing in them no matter what the results are ;) .

Convincing skeptics is a waste of time? It's easy to be a skeptic, you just have to doubt and come up with some excuses to justify it...it's more difficult to go ahead and improve your mileage.

In the real world (at least I THINK that's where it is) I use a 3 tank avg (at least a 3/4 refill). This allows seeing some results in a reasonable amount of time and is...I believe.... an accurate reflection of mpg.

This is called a simple moving avg. When plotted...IF you are seeing real increases in mpg...you will see the plot go up and stay there RELATIVE to the time of year and so forth.

You do need to drive consistently and be accurate at fillups.

My next vehicle will probably be a Saturn...and I hope to duplicate the results others have managed. Plagiarism? :eek:

good thread clencher.

i think it's really difficult to test accounting for variables. weather/climate changes from 1 year to the next, increase in traffic, new roads and traffic lights, the tune status of tested car, etc...

stats and studies have the human error as well. i'd say it's imperfect, so do the best testing possible over a years time. the beauty is we can continue to achieve better FE w/out the worry of job security or funding to depend on.