next »

[pathanalysis]

Hi all,

In a majority of the papers I've read or worked on, especially ones with small Ns or prelim. data, I've always been taught to refer to p-values of .05-.1  as "nearing significance." However, my Co-I says she's never seen this before. I vote for including them, particularly as we're analyzing preliminary data with small Ns (30 and 15), but I  want know to if I'm totally out of whack by including them on our poster.

Thanks!

[shrek]

god surely loves .051 as much as .049
                                              -- looking for citation

[new_bus_prof]

Hi all,

In a majority of the papers I've read or worked on, especially ones with small Ns or prelim. data, I've always been taught to refer to p-values of .05-.1  as "nearing significance." However, my Co-I says she's never seen this before. I vote for including them, particularly as we're analyzing preliminary data with small Ns (30 and 15), but I  want know to if I'm totally out of whack by including them on our poster.

Thanks!

In my field... marginally significant, partially supported, or in alignment with... I have not seen the nearing significance terms used though.

[svenc]

"Significant at the 10% level" is an appropriate phrase from a statistical point of view.

However, in many fields p <= .05 is a magical number, worshipped by the vast hordes of folks who remember how to do the three statistical tests they need to use in their own research, but nothing else from their statistics classes.  And you risk offending many of them by talking about significance at anything other than the 5% level.  In some disciplines, this is a religion that counts many graduate program chairs, journal referees, and even journal editors among its adherents.

If you are in such a field, you're taking your chances every time you go there.

These zealots don't know god, for it is indeed true that

god surely loves .051 as much as .049

[daniel_von_flanagan]

I've never seen this term.

The reason that .05 (or even .01 in some critical situations) are "magical" is that there is every possibility that the data do not adhere strictly to the mathematical model from which the corresponding test is derived, and it is better to err on the side of conservatism.  This is especially true if you are using a test based on normal approximation and the sample size is low: if you have a p-value near 0.1, there is every possibility that the p-value God sees with her exact model is more like 0.15, and no responsible person who understands the analysis they are doing pays any attention to a result this not-unlikely.

I a personally inclined to respect a field in iverse proportion to the largest p-value to which they are willing to assign some degree of credibility. - DvF

[svenc]

This shouldn't be about binary designations of significance; it should be about interpreting p-values, in the context of sample sizes and data quality.  I was not arguing that .10 is an appropriate "magical number," but merely said that "significant at the 10% level" is a correct statement (as is "significant at the 1% level").

Far too many researchers will trumpet their p=.049, while dismissing anothers' result of p=0.53 as "insignificant."   This often translates into p=.049 == publishable.  In so many cases, all that is required to get to .049 is buying a few more observations.  It's a foolish and meaningless distinction that persists primarily because people can better remember arbitrary rules of thumb than the probability theory that underlies those rules.*

* By which I certainly don't mean you DvF.

[prof_smartypants]

My understanding is that "nearing significance" is kind of like "fairly unique". For my field, .05 is significant. I did, however, mention in a footnote variables that appeared significant at the >.1 level for the purpose of future research. Maybe in another study, these variables will be significant, thus it's important to include a mention of them so that others are aware of that. I think my wording was "While the relationship is not significant at the .05 level, the results suggest that these variables be included in future studies, blah blah blah"

As svenc points out, this is particularly important in studies (like mine) that had a relatively small n. You don't want to dismiss what could be an important relationship because of a few measly hundredths of a percentage. But you also don't want to start trumpeting your results as conclusive, either.

[qrypt]

Another way of expressing what some people here are saying: the phrase "nearing significance" makes a fetish of .05 -- it gives that threshold level a magical quality that it doesn't deserve and perpetuates a misleading way of thinking about what statistical significance means. 

I've never presented research via a poster, so no idea how to approach this kind of result in that format.  But in a paper I would simply let the p-value speak for itself and, if appropriate, discuss issues like the likely effect of sample size, prospects for future research, etc.  But I would never use the phrase "nearing significance". 

[daniel_von_flanagan]

I agree with the last three posters that such a p value can be of interest, but when people mention a p close to .1 it often seems to me to be a wistful thing, as if the result really is significant but that the statistics aren't behaving properly this time out of spite. - DvF

[dellaroux]

The "Spiteful Statistics," the latest garage band name....

:--}

[madhatter]

In psychology, I have seen this phrase used informally in the discussion section of a paper. Alternatively (and more accurately), one would say "The results, though not significant, were in the direction predicted by the hypothesis."

Even if your results crossed the .05 threshold, that's a rather weak finding of significance. These days, the expectation is that you will report on effect size to calibrate the approximate impact of the findings, rather than relying on a p value.

[kedves]

I don't remember ever having seen it.  Is there a recognized definition for it, a cut-off?  Would you give .10 a half-an-asterisk?

It would seem more credible to report the p-value and suggest that further research be done.  I understand the argument about the problem of giving a fetish status to a number,  but it doesn't make sense to act is it's all the same and 1 in 10 is the same as 1 in 100. 

I suppose it would depend on the purpose of and audience for your poster, whether these studies would be helpful or interesting to include.

[sprocket]

I've seen such p-values reported and discussed as "nonsignificant trends." And have used such terms in at least one paper I've published. This is particularly true for small-sample pilot studies and exploratory work where the objectives are not necessarily to find conclusive findings.

[locutus]

I'm in one of those .05 = magic fields.

I say just report the stats thoroughly. What you call the results is of no real consequence (or at least shouldn't be).

[sciencephd]

This discussion makes me glad, for once, that in my field statistical analysis is like the wild west.