Photo: Anna Shvets

Is .0001 a good p-value?

Conventionally, p < 0.05 is referred as statistically significant and p < 0.001 as statistically highly significant.

jcpcarchives.org - P Value, Statistical Significance and Clinical Significance

Can you be born naturally smart?

Like most aspects of human behavior and cognition, intelligence is a complex trait that is influenced by both genetic and environmental factors....

Last updated Feb 7, 2021

What country has the lowest education?

Countries with the Lowest-Ranked Educational Systems (and their estimated adult literacy rates) Niger (28.7%) Burkina Faso (28.7%) Mali (31.1%)...

Last updated Feb 11, 2019

Make Your Bookie Cry And Pull The Hair By Generating $23,187

Promotion

The p-value and null distribution

Given the value of the test statistic and the null distribution of the test statistic, it is important to see whether the test statistic is in the middle of the distribution or out in a tail of the distribution. If in the middle of it is consistent with the null hypothesis and out in the tail making the alternative hypothesis seems more plausible. The large values of the test statistic seem more plausible under the alternative hypothesis than under the null hypothesis. The larger values of test statistic corresponds to the smaller the p-value, which are indicative of the stronger evidence against the null hypothesis in favor of the alternative.

Interpretation of p-value

The p-value indicates how probable the results are due to chance.

p=0.05 means that there is a 5% probability that the results are due to random chance.

p=0.001 means that the chances are only 1 in a thousand.

The choice of significance level at which you reject null hypothesis is arbitrary. Conventionally, 5%, 1% and 0.1% levels are used. In some rare situations, 10% level of significance is also used. Statistical inferences indicating the strength of the evidence corresponding to different values of p are explained as under: Conventionally, p < 0.05 is referred as statistically significant and p < 0.001 as statistically highly significant. When presenting p values it is a common practice to use the asterisk rating system. Important in medical research, p values less than 0.05 are often reported as statistically significant as we want there to be only a 5% or less probability that the treatment results, risk factor, or diagnostic results could be due to chance alone. Results that do not meet this threshold are generally interpreted as negative.

p-value and sample size

The magnitude of p-value depends on the sample size. In view of this, following is important to note: Given a sufficiently large sample size, even an extremely small difference could be detected as significant. However, the difference may not be of clinical importance. Conversely, potentially medically important differences observed in small studies, for which the p values are more than 0.05, are ignored. All statistically significant findings at (p < 0.05) are assumed to result from real treatment effects, whereas by definition an average of one in 20 comparisons in which null hypothesis is true will result in p < 0.05.

Clinical Significance

Is there any such thing as 1000%?

"1000 percent" or "1000%" in a literal sense means to multiply by 10. In American English it is used as a metaphor meaning very high emphasis, or...

Last updated Sep 12, 2020

What is Rule 26 in golf?

The first option (Rule 26-1a) allows you to drop a ball from where you last played and try it again from there. Sometimes it may be a good option,...

Last updated Oct 15, 2022

Promotion

One should keep in mind the difference between statistical significance and clinical significance. The results of a study can be statistically significant but still be too small to be of any practical value. We might encounter a situation where new treatment shows reduction in pain as statistically significant with a p value of 0.0001. The extremely low p value in this situation indicates that we are really sure that the results are not accidental-- the improvement is really due to the treatment and not just chance. However, it is important to note the magnitude of improvement while interpreting it clinically. Usually, the extremes are easy to recognize and agree upon. If with the new treatment being investigated, patients get 90% pain relief, we can all agree that this is an effective, worthwhile treatment. But if reduction in pain is just 3%, such a paltry effect may not be of any importance. It is thus important to decide whether the observed treatment effect is large enough to make a difference. But what would constitute the smallest amount of improvement (say δ) that would be considered worthwhile. After all we want the new treatment to make a difference. This is tricky. The researchers doing the study should explicitly state what this minimal amount of clinically important improvement is. One can find the absolute amount of improvement in critical outcome measure, to decide if the improvement looked very large and can cross reference it with other sources to decide whether it met the Medically Important Clinical Difference (MICD). The null and alternative hypothesis could be appropriately framed to detect the Medically Important Clinical Difference as significant.

The Null and Alternative Hypothesis in this situation would be as under:

H0: pn = ps+δ

H1: pn > ps+δ

It is therefore advised, not to just look at the p value but also try to examine if the results are robust enough to be clinically significant with desired improvements (MICD).

References

Krikwood BR, Sterne JAC. Medical statistics, 2nd edition, Oxford Blackwell Science Ltd; 2003 The p value is calculated based on the Null distribution, which is a theatrical distribution of the test statistic when the null hypothesis is true. The commonly used null distributions and their applications are given as under:

jcpcarchives.org - P Value, Statistical Significance and Clinical Significance

Do online games really pay cash?

It's true, though, that you may make real money online by playing games on legitimate websites and programs. In addition, if you play online games...

Last updated Feb 1, 2019

Why is 16 the worst hand in blackjack?

In fact, the value 16 is said to be the worst hand one can have in blackjack. Since sixteen of the other fifty cards have a value of 10 and four...

Last updated Feb 28, 2022

Promotion

Has there ever been an actual vigilante?

From Marianne Bachmeier, the German mother who killed her daughter's murderer, to Jason Vukovich, the Alaskan man who beat up sex offenders, these...

Last updated Sep 6, 2021

Promotion

What does 1.618 mean?

golden ratio golden ratio, also known as the golden section, golden mean, or divine proportion, in mathematics, the irrational number (1 + Square...

Last updated Feb 7, 2018