Author(s): WIESINGER, H.P., HOPKINS, W.G., Institution: UNIVERSITY OF SALZBURG, Country: AUSTRIA, Abstract-ID: 906

INTRODUCTION:
The magnitude of an effect differs from sample to sample, owing to sampling variation. Larger effects are more likely to be statistically significant and published as important findings. Hence meta-analyzed magnitudes of published effects may suffer from substantial upward publication bias. Here we meta-analyzed simulated studies similar to those in recent meta-analyses of athlete performance to investigate such bias.
METHODS:
We simulated effects on endurance time-trial performance in an intervention group by assuming true mean effects of 1.0% (trivial-small) on males and 3.0% (small-moderate) on females, heterogeneity SD (true between-study differences in the mean effect) of 0.0%, 0.5% (trivial-small) and 1.5% (small-moderate), standard errors of measurement of ~3.0% (range 1.5-6.0%), and sample sizes of ~13 (range 10-30). Meta-analyses were performed by excluding 0%, 100%, and 50% of non-significant effects. At least 10 studies were included in each meta-analysis. At least 2000 meta-analyses were performed for each combination of study characteristics, and meta-analyzed effects were averaged to determine bias (difference from true effects). We used a meta-regression mixed model that included a fixed effect for sex, its interaction with the square of the standard error (SE, to adjust for bias), and a random effect for heterogeneity. Mean effects not adjusted and adjusted for bias were those predicted for SE squared equal to its mean and zero, respectively.
RESULTS:
The meta-analyses estimated mean effects and heterogeneity without bias when all non-significant effects were included. Complete exclusion of non-significant effects produced the greatest bias in mean effects when heterogeneity was moderate (males, +2.3%; females, +1.2%); adjustment for bias partially corrected the bias for males (to +1.2%) and almost fully corrected the bias for females (to +0.2%). Bias in mean effects was least when heterogeneity was zero (males, +1.5%; females +0.6%), and was almost fully corrected after adjustment (to +0.3% and -0.2%). Heterogeneity itself was underestimated by -0.8% for true moderate heterogeneity and by -0.2% for zero true heterogeneity. When 50% of non-significant effects were excluded, bias was reduced, but the correction for bias was largely ineffective, while heterogeneity was estimated without bias.
CONCLUSION:
Publication bias is likely to be small in small meta-analyzed mean effects on athlete endurance performance, and bias will be negligible for large effects. Meta-analysts can improve adjustment for publication bias by using meta-regression to reduce heterogeneity. The problem of publication bias would be obviated if authors submitted, and journal editors accepted, manuscripts irrespective of statistical significance of effects. Meantime, the simulation program, and versions for standardized effects and risk ratios, can be adapted to estimate the bias in mean effects and heterogeneity with any meta-analyses in sport and exercise.