statistics, power, binomial distribution / General Science and Electronics / Forums

Forums

4hv.org :: Forums :: General Science and Electronics

« Previous topic | Next topic »

statistics, power, binomial distribution

Move Thread

LAN_403

IamSmooth

Sun May 15 2011, 07:31PM

Registered Member #190 Joined: Fri Feb 17 2006, 12:00AM
Location:
Posts: 1567

Maybe someone is really good with stats, or has access to a statistics professor. Here we go:

I was using the calculator on this page

I am trying to determine the power for a study. The distribution is binomial. I have a device that either works or does not work. I do not know the real probability, but I think it is very good. Let's assume p = 0.75. I am going to try it 50 times. Using a binomial distrtibution program I determine that if I get 43 or more successes, this will happen by chance less than 5% (p = 0.05) of the time. So, I will accept that the device works at least 75% of the time if I get 43 successes or more. The question then is what is the power of this study? This seems to be predicated on the fact that I know the real probablility. If I think the device works 89% of the time I find that my power is 82%. If the true success rate is 90% the power goes up to 87%.

How can I get a meaningful number for the power? If my effect size is large, true I have a larger power. But, I don't know the real effect size. I can make the power of the test arbitrarily large by just stating I have large effect size. This does not seem right.

Carbon_Rod

Sun May 15 2011, 10:39PM

Registered Member #65 Joined: Thu Feb 09 2006, 06:43AM
Location:
Posts: 1155

steve516

Mon May 16 2011, 01:56AM

Registered Member #3832 Joined: Thu Apr 14 2011, 11:57PM
Location: Downtown Chicago, IL
Posts: 37

What you seem to want to be doing is a hypothesis test to determine if your device works above a certain proportion. If this is indeed the case, you are using a significance level of .05 which means your power (or beta value) is .95.

You could be doing it like this:

Null hypothesis- (H0)- p=.5
Alternative (Ha)- p>.5
significance level= .05

Now using a calculator you could find that the probability of getting 43 successes if the true proportion was .5 (as in being as random as a flip of a coin) in a sample of 50 trials is about 1.7E-7 or ~0.

In this case we would reject the null hypothesis as there is sufficient evidence to suggest that the true proportion of successes by the device is much higher than .5 at the .05 significance level. Now- your power in this case is the likelihood of rejecting a false null hypothesis, which we have done. You will notice that even at the .01 significance level (commonly used in medical experiments) that we would still reject the null, since the p-value (1.7E-7) is even less than that.

I would personally set up your experiment as such, as you can substitute your own results for the 43 number.

I'm sorry if that did not help at all.

Steve Conner

Mon May 16 2011, 06:50AM

Registered Member #30 Joined: Fri Feb 03 2006, 10:52AM
Location: Glasgow, Scotland
Posts: 6706

Iamsmooth, look up Student's T-test and the chi-squared test. They are ways of calculating the confidence level of your hypothesis, given a limited sample size. Power, effect size and so on are just these same theories with medical jargon names.

Also note: if you have a device that "either works or doesn't" then the results of a large number of trials should form a Gaussian distribution (bell curve) which is what's needed for the two above tests to work. This is the central limit theorem.

But beware: You can pick 100 devices from the population, or test the same one 100 times. If you get the same result either way, the quantity you're investigating is "ergodic". Many important things aren't, for instance looking at the death rate for 30 something white males gives me a life expectancy of 400 years.

IamSmooth

Tue May 17 2011, 12:40AM

Registered Member #190 Joined: Fri Feb 17 2006, 12:00AM
Location:
Posts: 1567

steve516 wrote ...

What you seem to want to be doing is a hypothesis test to determine if your device works above a certain proportion. If this is indeed the case, you are using a significance level of .05 which means your power (or beta value) is .95.

Power is not (1-alpha)
Power is (1-B)

steve516

Wed May 18 2011, 01:48AM

Registered Member #3832 Joined: Thu Apr 14 2011, 11:57PM
Location: Downtown Chicago, IL
Posts: 37

IamSmooth wrote ...

Power is not (1-alpha)
Power is (1-B)

Gah. I'm sorry, you are correct. My bad on that, the hypothesis test is correct though. At least the process I mean.

SK

Moderator(s): Chris Russell, Noelle, Alex, Tesladownunder, Dave Marshall, Dave Billington, Bjørn, Steve Conner, Wolfram, Kizmo, Mads Barnkob