
STATISTICS WITHOUT TEARS 

Year : 2009  Volume
: 18
 Issue : 2  Page : 127131 


Hypothesis testing, type I and type II errors
Amitav Banerjee^{1}, UB Chitnis^{1}, SL Jadhav^{1}, JS Bhawalkar^{1}, S Chaudhury^{2}
^{1} Department of Community Medicine, D. Y. Patil Medical College, Pune, India ^{2} Department of Psychiatry, RINPAS, Kanke, Ranchi, India
Date of Web Publication  5Jun2010 
Correspondence Address: Amitav Banerjee Department of Community Medicine, D. Y. Patil Medical College, Pune  411 018 India
Source of Support: None, Conflict of Interest: None  Check 
DOI: 10.4103/09726748.62274
Abstract   
Hypothesis testing is an important activity of empirical research and evidencebased medicine. A well worked up hypothesis is half the answer to the research question. For this, both knowledge of the subject derived from extensive review of the literature and working knowledge of basic statistical concepts are desirable. The present paper discusses the methods of working up a good hypothesis and statistical concepts of hypothesis testing. Keywords: Effect size, Hypothesis testing, Type I error, Type II error
How to cite this article: Banerjee A, Chitnis U B, Jadhav S L, Bhawalkar J S, Chaudhury S. Hypothesis testing, type I and type II errors. Ind Psychiatry J 2009;18:12731 
Karl Popper is probably the most influential philosopher of science in the 20^{ th} century (Wulff et al., 1986). Many scientists, even those who do not usually read books on philosophy, are acquainted with the basic principles of his views on science. The popularity of Popper's philosophy is due partly to the fact that it has been well explained in simple terms by, among others, the Nobel Prize winner Peter Medawar (Medawar, 1969). Popper makes the very important point that empirical scientists (those who stress on observations only as the starting point of research) put the cart in front of the horse when they claim that science proceeds from observation to theory, since there is no such thing as a pure observation which does not depend on theory. Popper states, "… the belief that we can start with pure observation alone, without anything in the nature of a theory, is absurd: As may be illustrated by the story of the man who dedicated his life to natural science, wrote down everything he could observe, and bequeathed his 'priceless' collection of observations to the Royal Society to be used as inductive (empirical) evidence.
Starting Point of Research :Hypothesis or Observation?   
The first step in the scientific process is not observation but the generation of a hypothesis which may then be tested critically by observations and experiments. Popper also makes the important claim that the goal of the scientist's efforts is not the verification but the falsification of the initial hypothesis. It is logically impossible to verify the truth of a general law by repeated observations, but, at least in principle, it is possible to falsify such a law by a single observation. Repeated observations of white swans did not prove that all swans are white, but the observation of a single black swan sufficed to falsify that general statement (Popper, 1976).
Characteristics of a Good Hypothesis   
A good hypothesis must be based on a good research question. It should be simple, specific and stated in advance (Hulley et al., 2001).
Hypothesis should be simple
A simple hypothesis contains one predictor and one outcome variable, e.g. positive family history of schizophrenia increases the risk of developing the condition in firstdegree relatives. Here the single predictor variable is positive family history of schizophrenia and the outcome variable is schizophrenia. A complex hypothesis contains more than one predictor variable or more than one outcome variable, e.g., a positive family history and stressful life events are associated with an increased incidence of Alzheimer's disease. Here there are 2 predictor variables, i.e., positive family history and stressful life events, while one outcome variable, i.e., Alzheimer's disease. Complex hypothesis like this cannot be easily tested with a single statistical test and should always be separated into 2 or more simple hypotheses.
Hypothesis should be specific
A specific hypothesis leaves no ambiguity about the subjects and variables, or about how the test of statistical significance will be applied. It uses concise operational definitions that summarize the nature and source of the subjects and the approach to measuring variables (History of medication with tranquilizers, as measured by review of medical store records and physicians' prescriptions in the past year, is more common in patients who attempted suicides than in controls hospitalized for other conditions). This is a longwinded sentence, but it explicitly states the nature of predictor and outcome variables, how they will be measured and the research hypothesis. Often these details may be included in the study proposal and may not be stated in the research hypothesis. However, they should be clear in the mind of the investigator while conceptualizing the study.
Hypothesis should be stated in advance
The hypothesis must be stated in writing during the proposal state. This will help to keep the research effort focused on the primary objective and create a stronger basis for interpreting the study's results as compared to a hypothesis that emerges as a result of inspecting the data. The habit of post hoc hypothesis testing (common among researchers) is nothing but using thirddegree methods on the data (data dredging), to yield at least something significant. This leads to overrating the occasional chance associations in the study.
Types of Hypotheses   
For the purpose of testing statistical significance, hypotheses are classified by the way they describe the expected difference between the study groups.
Null and alternative hypotheses
The null hypothesis states that there is no association between the predictor and outcome variables in the population (There is no difference between tranquilizer habits of patients with attempted suicides and those of age and sex matched "control" patients hospitalized for other diagnoses). The null hypothesis is the formal basis for testing statistical significance. By starting with the proposition that there is no association, statistical tests can estimate the probability that an observed association could be due to chance.
The proposition that there is an association  that patients with attempted suicides will report different tranquilizer habits from those of the controls  is called the alternative hypothesis. The alternative hypothesis cannot be tested directly; it is accepted by exclusion if the test of statistical significance rejects the null hypothesis.
One and twotailed alternative hypotheses
A onetailed (or onesided) hypothesis specifies the direction of the association between the predictor and outcome variables. The prediction that patients of attempted suicides will have a higher rate of use of tranquilizers than control patients is a onetailed hypothesis. A twotailed hypothesis states only that an association exists; it does not specify the direction. The prediction that patients with attempted suicides will have a different rate of tranquilizer use  either higher or lower than control patients  is a twotailed hypothesis. (The word tails refers to the tail ends of the statistical distribution such as the familiar bellshaped normal curve that is used to test a hypothesis. One tail represents a positive effect or association; the other, a negative effect.) A onetailed hypothesis has the statistical advantage of permitting a smaller sample size as compared to that permissible by a twotailed hypothesis. Unfortunately, onetailed hypotheses are not always appropriate; in fact, some investigators believe that they should never be used. However, they are appropriate when only one direction for the association is important or biologically meaningful. An example is the onesided hypothesis that a drug has a greater frequency of side effects than a placebo; the possibility that the drug has fewer side effects than the placebo is not worth testing. Whatever strategy is used, it should be stated in advance; otherwise, it would lack statistical rigor. Data dredging after it has been collected and post hoc deciding to change over to onetailed hypothesis testing to reduce the sample size and P value are indicative of lack of scientific integrity.
Statistical Principles of Hypothesis Testing   
A hypothesis (for example, Tamiflu [oseltamivir], drug of choice in H1N1 influenza, is associated with an increased incidence of acute psychotic manifestations) is either true or false in the real world. Because the investigator cannot study all people who are at risk, he must test the hypothesis in a sample of that target population. No matter how many data a researcher collects, he can never absolutely prove (or disprove) his hypothesis. There will always be a need to draw inferences about phenomena in the population from events observed in the sample (Hulley et al., 2001). In some ways, the investigator's problem is similar to that faced by a judge judging a defendant [Table 1]. The absolute truth whether the defendant committed the crime cannot be determined. Instead, the judge begins by presuming innocence  the defendant did not commit the crime. The judge must decide whether there is sufficient evidence to reject the presumed innocence of the defendant; the standard is known as beyond a reasonable doubt. A judge can err, however, by convicting a defendant who is innocent, or by failing to convict one who is actually guilty. In similar fashion, the investigator starts by presuming the null hypothesis, or no association between the predictor and outcome variables in the population. Based on the data collected in his sample, the investigator uses statistical tests to determine whether there is sufficient evidence to reject the null hypothesis in favor of the alternative hypothesis that there is an association in the population. The standard for these tests is shown as the level of statistical significance.
Type I (Also Known as 'α') and Type II (Also Known as 'β')Errors   
Just like a judge's conclusion, an investigator's conclusion may be wrong. Sometimes, by chance alone, a sample is not representative of the population. Thus the results in the sample do not reflect reality in the population, and the random error leads to an erroneous inference. A type I error (falsepositive) occurs if an investigator rejects a null hypothesis that is actually true in the population; a type II error (falsenegative) occurs if the investigator fails to reject a null hypothesis that is actually false in the population. Although type I and type II errors can never be avoided entirely, the investigator can reduce their likelihood by increasing the sample size (the larger the sample, the lesser is the likelihood that it will differ substantially from the population).
Falsepositive and falsenegative results can also occur because of bias (observer, instrument, recall, etc.). (Errors due to bias, however, are not referred to as type I and type II errors.) Such errors are troublesome, since they may be difficult to detect and cannot usually be quantified.
Effect Size   
The likelihood that a study will be able to detect an association between a predictor variable and an outcome variable depends, of course, on the actual magnitude of that association in the target population. If it is large (such as 90% increase in the incidence of psychosis in people who are on Tamiflu), it will be easy to detect in the sample. Conversely, if the size of the association is small (such as 2% increase in psychosis), it will be difficult to detect in the sample. Unfortunately, the investigator often does not know the actual magnitude of the association  one of the purposes of the study is to estimate it. Instead, the investigator must choose the size of the association that he would like to be able to detect in the sample. This quantity is known as the effect size. Selecting an appropriate effect size is the most difficult aspect of sample size planning. Sometimes, the investigator can use data from other studies or pilot tests to make an informed guess about a reasonable effect size. When there are no data with which to estimate it, he can choose the smallest effect size that would be clinically meaningful, for example, a 10% increase in the incidence of psychosis. Of course, from the public health point of view, even a 1% increase in psychosis incidence would be important. Thus the choice of the effect size is always somewhat arbitrary, and considerations of feasibility are often paramount. When the number of available subjects is limited, the investigator may have to work backward to determine whether the effect size that his study will be able to detect with that number of subjects is reasonable.
α,β,and Power   
After a study is completed, the investigator uses statistical tests to try to reject the null hypothesis in favor of its alternative (much in the same way that a prosecuting attorney tries to convince a judge to reject innocence in favor of guilt). Depending on whether the null hypothesis is true or false in the target population, and assuming that the study is free of bias, 4 situations are possible, as shown in [Table 2] below. In 2 of these, the findings in the sample and reality in the population are concordant, and the investigator's inference will be correct. In the other 2 situations, either a type I (a) or a type II (b) error has been made, and the inference will be incorrect.
The investigator establishes the maximum chance of making type I and type II errors in advance of the study. The probability of committing a type I error (rejecting the null hypothesis when it is actually true) is called a (alpha) the other name for this is the level of statistical significance.
If a study of Tamiflu and psychosis is designed with α = 0.05, for example, then the investigator has set 5% as the maximum chance of incorrectly rejecting the null hypothesis (and erroneously inferring that use of Tamiflu and psychosis incidence are associated in the population). This is the level of reasonable doubt that the investigator is willing to accept when he uses statistical tests to analyze the data after the study is completed.
The probability of making a type II error (failing to reject the null hypothesis when it is actually false) is called β (beta). The quantity (1  β) is called power, the probability of observing an effect in the sample (if one), of a specified effect size or greater exists in the population.
If β is set at 0.10, then the investigator has decided that he is willing to accept a 10% chance of missing an association of a given effect size between Tamiflu and psychosis. This represents a power of 0.90, i.e., a 90% chance of finding an association of that size. For example, suppose that there really would be a 30% increase in psychosis incidence if the entire population took Tamiflu. Then 90 times out of 100, the investigator would observe an effect of that size or larger in his study. This does not mean, however, that the investigator will be absolutely unable to detect a smaller effect; just that he will have less than 90% likelihood of doing so.
Ideally alpha and beta errors would be set at zero, eliminating the possibility of falsepositive and falsenegative results. In practice they are made as small as possible. Reducing them, however, usually requires increasing the sample size. Sample size planning aims at choosing a sufficient number of subjects to keep alpha and beta at acceptably low levels without making the study unnecessarily expensive or difficult.
Many studies set alpha at 0.05 and beta at 0.20 (a power of 0.80). These are somewhat arbitrary values, and others are sometimes used; the conventional range for alpha is between 0.01 and 0.10; and for beta, between 0.05 and 0.20. In general the investigator should choose a low value of alpha when the research question makes it particularly important to avoid a type I (falsepositive) error, and he should choose a low value of beta when it is especially important to avoid a type II error.
P Value   
The null hypothesis acts like a punching bag: It is assumed to be true in order to shadowbox it into false with a statistical test. When the data are analyzed, such tests determine the P value, the probability of obtaining the study results by chance if the null hypothesis is true. The null hypothesis is rejected in favor of the alternative hypothesis if the P value is less than alpha, the predetermined level of statistical significance (Daniel, 2000). "Nonsignificant" results  those with P value greater than alpha  do not imply that there is no association in the population; they only mean that the association observed in the sample is small compared with what could have occurred by chance alone. For example, an investigator might find that men with family history of mental illness were twice as likely to develop schizophrenia as those with no family history, but with a P value of 0.09. This means that even if family history and schizophrenia were not associated in the population, there was a 9% chance of finding such an association due to random error in the sample. If the investigator had set the significance level at 0.05, he would have to conclude that the association in the sample was "not statistically significant." It might be tempting for the investigator to change his mind about the level of statistical significance ex post facto and report the results "showed statistical significance at P < 10". A better choice would be to report that the "results, although suggestive of an association, did not achieve statistical significance ( P = .09)". This solution acknowledges that statistical significance is not an "all or none" situation.
Conclusion   
Hypothesis testing is the sheet anchor of empirical research and in the rapidly emerging practice of evidencebased medicine. However, empirical research and, ipso facto, hypothesis testing have their limits. The empirical approach to research cannot eliminate uncertainty completely. At the best, it can quantify uncertainty. This uncertainty can be of 2 types: Type I error (falsely rejecting a null hypothesis) and type II error (falsely accepting a null hypothesis). The acceptable magnitudes of type I and type II errors are set in advance and are important for sample size calculations. Another important point to remember is that we cannot 'prove' or 'disprove' anything by hypothesis testing and statistical tests. We can only knock down or reject the null hypothesis and by default accept the alternative hypothesis. If we fail to reject the null hypothesis, we accept it by default.^{[5]}
References   
1.  Daniel, W. W. (2002). Hypothesis testing. In: Biostatistics. 7^{ th} ed. John Wiley and Sons, Inc. New York; pages 204294 
2.  Hulley, S. B., Cummings, S. R., Browner, W. S., Grady, D., Hearst, N., and Newman, T. B. (2001). Getting ready to estimate sample size: Hypothesis and underlying principles In: Designing Clinical ResearchAn epidemiologic approach. 2^{ nd} ed. (pp. 5163). Philadelphia: Lippincott Williams and Wilkins. 
3.  Medawar, P. B. (1969). Induction and intuition in scientific thought. Philadelphia: American Philosophical Society. 
4.  Popper, K. (1976). Unended Quest. An Intellectual Autobiography. Fontana Collins, p 42. 
5.  Wulff, H. R., Pedersen, S. A., and Rosenberg, R. (1986). Empirism and Realism: A philosophical problem. In: (pp. 1329). Philosophy of Medicine. Oxford: Blackwell Scientific Publicatons. 
[Table 1], [Table 2]
This article has been cited by  1 
Is fatty infiltration in paraspinal muscles reversible with exercise in people with low back pain? A systematic review 

 E. O. Wesselink, J. J. M. Pool, J. Mollema, K. A. Weber, J. M. Elliott, M. W. Coppieters, A. L. PoolGoudzwaard   European Spine Journal. 2022;   [Pubmed]  [DOI]   2 
Current practice in the measurement and interpretation of intervention adherence in randomised controlled trials: A systematic review 

 Alexia Giovanazzi, Katherine Jones, Rachel M. Carr, Caroline M. Fairhurst, Michael R. Backhouse, Joy A. Adamson   Contemporary Clinical Trials. 2022; 118: 106788   [Pubmed]  [DOI]   3 
Hepatocellular carcinoma (HCC) in patients with NonAlcoholic Fatty Liver Disease (NAFLD): screening, treatment and survival analysis in a Brazilian series 

 Regiane Saraiva de Souza Melo Alencar, Claudia P. Oliveira, Aline Lopes Chagas, Leonardo Gomes da Fonseca, Claudia Maccali, Lisa Rodrigues da Cunha Saud, Mariana Pinheiro Xerfan, Jose Tadeu Stefano, Paulo Herman, Luiz Augusto Carneiro D'Albuquerque, Venâncio Avancini Ferreira Alves, Flair Jose Carrilho   Clinics. 2022; 77: 100097   [Pubmed]  [DOI]   4 
Bayesian autoencoders with uncertainty quantification: Towards trustworthy anomaly detection 

 Bang Xiang Yong, Alexandra Brintrup   Expert Systems with Applications. 2022; : 118196   [Pubmed]  [DOI]   5 
Accuracy of assessing 18, 21, and 25 years of age using Olze et al. stagebased system in an Indian sample of young adults 

 Jitendra Kumar, Anil Aggrawal, M. Sreenivas, Sujoy Ghosh, Mahesh Verma   Legal Medicine. 2022; : 102061   [Pubmed]  [DOI]   6 
Therapeutic exercise to improve motor function among children with Down Syndrome aged 0 to 3 years: a systematic literature review and metaanalysis 

 ElianaIsabel RodríguezGrande, Adriana BuitragoLópez, MarthaRocio TorresNarváez, Yannely SerranoVillar, Francisca VerdugoPaiva, Camila Ávila   Scientific Reports. 2022; 12(1)   [Pubmed]  [DOI]   7 
Visualization of the Third Ventricle, the Future Cavum Septi Pellucidi, and the Cavum Veli Interpositi at 11+3 to 13+6 Gestational Weeks on 3D Transvaginal Ultrasound Including Normative Data 

 Reinhard Altmann, Iris Scharnreitner, Christian Auer, Lena Hirtler, Claudia Springer, Stephanie Falschlehner, Wolfgang Arzt   Ultraschall in der Medizin  European Journal of Ultrasound. 2022;   [Pubmed]  [DOI]   8 
The rise to power of the microbiome: power and sample size calculation for microbiome studies 

 Tahsin Ferdous, Lai Jiang, Irina Dinu, Julie Groizeleau, Anita L. Kozyrskyj, Celia M. T. Greenwood, MarieClaire Arrieta   Mucosal Immunology. 2022;   [Pubmed]  [DOI]   9 
Increasing vitamin D levels to improve fertilization rates in cattle 

 Vanessa Peixoto de Souza, Jared Jensen, William Whitler, Charles T Estill, Cecily V Bishop   Journal of Animal Science. 2022; 100(7)   [Pubmed]  [DOI]   10 
Association between MethyleneTetrahydrofolate Reductase C677T Polymorphism and Human Immunodeficiency Virus Type 1 Infection in Morocco 

 Hanâ Baba, Meryem Bouqdayr, Asmae Saih, Rajaa Bensghir, Ahd Ouladlahsen, Mustapha Sodqi, Latifa Marih, Imane Zaidane, Anass Kettani, Omar Abidi, Lahcen Wakrim   Laboratory Medicine. 2022;   [Pubmed]  [DOI]   11 
On some fundamental challenges in monitoring epidemics 

 Vaiva Vasiliauskaite, Nino AntulovFantulin, Dirk Helbing   Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2022; 380(2214)   [Pubmed]  [DOI]   12 
A Framework of the Critical Factors for Healthcare Providers to Share Data Securely Using Blockchain 

 Ahmed G. Alzahrani, Ahmed Alhomoud, Gary Wills   IEEE Access. 2022; 10: 41064   [Pubmed]  [DOI]   13 
Deficiency in ReOrienting of Attention in Adults with AttentionDeficit Hyperactivity Disorder 

 Valentina Gumenyuk, Oleg Korzyukov, Natalie Tapaskar, Michael Wagner, Charles R Larson, Michael J. Hammer   Clinical EEG and Neuroscience. 2022; : 1550059422   [Pubmed]  [DOI]   14 
Pulsed electromagnetic fields for postappendicectomy pain management: a randomized, placebocontrolled trial 

 Muralitharan Perumal, Aizatul Isla Abdul Latib, Malarvilee Paul Samy, Mohd Rohisham bin Zainal Abidin, Elanngovan Nagandran, Tham Sim Wan, Pamela Francis, Chee Yoong Foo   Trials. 2022; 23(1)   [Pubmed]  [DOI]   15 
Interpreting Results from Statistical Hypothesis Testing: Understanding the Appropriate Pvalue 

 Eiki TSUSHIMA   Physical Therapy Research. 2022;   [Pubmed]  [DOI]   16 
The effectiveness of internetguided selfhelp interventions to promote physical activity among individuals with depression: A systematic review (Preprint) 

 Yiling Tang, Madelaine Gierc, Raymond W. Lam, Sam Liu, Guy Faulkner   JMIR Mental Health. 2022;   [Pubmed]  [DOI]   17 
Is the Rotatory Knee Stability Immediately Decreased Following a Competitive Soccer Match? 

 Alejandro Neira, Rony Silvestre, Aníbal Debandi, Daniel Darras, Iver CristiSánchez, Ignacio Barra, Luis Peñailillo, Carlos De La Fuente   Frontiers in Bioengineering and Biotechnology. 2022; 10   [Pubmed]  [DOI]   18 
The Performance, Physiology and Morphology of Female and Male OlympicDistance Triathletes 

 Paulo J. Puccinelli, Claudio A. B. de Lira, Rodrigo L. Vancini, Pantelis T. Nikolaidis, Beat Knechtle, Thomas Rosemann, Marilia S. Andrade   Healthcare. 2022; 10(5): 797   [Pubmed]  [DOI]   19 
Fundamentos Para La Elaboración De Artículos Científicos En Trauma Y Cuidado Agudo De Emergencias (Parte 5A): Bases Y Fundamentaciones De Metodología Estadística 

 Angelica Clavijo, Diana M Sánchez Parra, Juan P Ávila, Diana Urrego, Andrés M. Rubiano   Panamerican Journal of Trauma, Critical Care & Emergency Surgery. 2022; 11(1): 34   [Pubmed]  [DOI]   20 
Behavioral challenges to professional skepticism in auditors’ data analytics journey 

 Xiaoxing Li   Maandblad voor Accountancy en Bedrijfseconomie. 2022; 96(1/2): 45   [Pubmed]  [DOI]   21 
Development, Validation, and Testing of a SelfAssessment Tool to Measure Food Safety Beliefs, Attitudes, and Behaviors in Health Care Food Service Operations 

 KATHRYN FAKIER, WENQING XU   Journal of Food Protection. 2022; 85(4): 607   [Pubmed]  [DOI]   22 
Research Question, Objectives, and Endpoints in Clinical and Oncological Research: A Comprehensive Review 

 Addanki Purna singh, Praveen R Shahapur, Sabitha Vadakedath, Vallab Ganesh Bharadwaj, Dr Pranay Kumar , Venkata BharatKumar Pinnelli, Vikram Godishala, Venkataramana Kandi   Cureus. 2022;   [Pubmed]  [DOI]   23 
Statistical and Practical Significance of Articles at Sports Biomechanics Conferences 

 Uday Hasan   Annals of Applied Sport Science. 2021; 9(3): 0   [Pubmed]  [DOI]   24 
Face Validation of Database Forensic Investigation Metamodel 

 Arafat AlDhaqm, Shukor Razak, Richard A. Ikuesan, Victor R. Kebande, Siti Hajar Othman   Infrastructures. 2021; 6(2): 13   [Pubmed]  [DOI]   25 
Influence of COVID19 Restrictions on Training and Physiological Characteristics in U23 Elite Cyclists 

 Peter Leo, Iñigo Mujika, Justin Lawley   Journal of Functional Morphology and Kinesiology. 2021; 7(1): 1   [Pubmed]  [DOI]   26 
Detecting Emotions through Electrodermal Activity in Learning Contexts: A Systematic Review 

 Anne Horvers, Natasha Tombeng, Tibor Bosse, Ard W. Lazonder, Inge Molenaar   Sensors. 2021; 21(23): 7869   [Pubmed]  [DOI]   27 
Sarcopenia in the elderly versus microcirculation, inflammation status, and oxidative stress: A crosssectional study 

 Karynne Grutter Lopes, Paulo Farinatti, Daniel Alexandre Bottino, Maria das Graças Coelho de Souza, Priscila Alves Maranhão, Eliete Bouskela, Roberto Alves Lourenço, Ricardo Brandão de Oliveira   Clinical Hemorheology and Microcirculation. 2021; : 1   [Pubmed]  [DOI]   28 
Automating Visual Blockage Classification of Culverts with Deep Learning 

 Umair Iqbal, Johan Barthelemy, Wanqing Li, Pascal Perez   Applied Sciences. 2021; 11(16): 7561   [Pubmed]  [DOI]   29 
Research and the anaesthesiologist: Cutting the clutter and overcoming the odds 

 Amitav Banerjee   Indian Journal of Anaesthesia. 2021; 65(3): 183   [Pubmed]  [DOI]   30 
Early Stopping in Experimentation With RealTime Functional Magnetic Resonance Imaging Using a Modified Sequential Probability Ratio Test 

 Sarah J. A. Carr, Weicong Chen, Jeremy Fondran, Harry Friel, Javier SanchezGonzalez, Jing Zhang, Curtis Tatsuoka   Frontiers in Neuroscience. 2021; 15   [Pubmed]  [DOI]   31 
Extremely Preterm Infant Admissions Within the SafeBoosCIII Consortium During the COVID19 Lockdown 

 Marie Isabel Rasmussen, Mathias Lühr Hansen, Gerhard Pichler, Eugene Dempsey, Adelina Pellicer, Afif ELKhuffash, Shashidhar A, Salvador PirisBorregas, Miguel Alsina, Merih Cetinkaya, Lina Chalak, Hilal Özkan, Mariana Baserga, Jan Sirc, Hans Fuchs, Ebru Ergenekon, Luis Arruza, Amit Mathur, Martin Stocker, Olalla Otero Vaccarello, Tomasz Szczapa, Kosmas Sarafidis, Barbara KrólakOlejnik, Asli Memisoglu, Hallvard Reigstad, Elzbieta RafinskaWazny, Eleftheria Hatzidaki, Zhang Peng, Despoina Gkentzi, Renaud Viellevoye, Julie De Buyst, Emmanuele Mastretta, Ping Wang, Gitte Holst Hahn, Lars Bender, Luc Cornette, Jakub Tkaczyk, Ruth del Rio, Monica Fumagalli, Evangelia Papathoma, Maria Wilinska, Gunnar Naulaers, Iwona SadowskaKrawczenko, Chantal Lecart, María Luz Couce, Siv Fredly, Anne Marie Heuchan, Tanja Karen, Gorm Greisen   Frontiers in Pediatrics. 2021; 9   [Pubmed]  [DOI]   32 
Pitfalls in Study Interpretation 

 Rondi B. Gelbard, Michael W. Cripps   Surgical Infections. 2021; 22(6): 646   [Pubmed]  [DOI]   33 
Privacypreserving chisquared test of independence for small samples 

 Yuichi Sei, Akihiko Ohsuga   BioData Mining. 2021; 14(1)   [Pubmed]  [DOI]   34 
Climate change experiment suggests divergent responses of tree seedlings in eastern North America’s Acadian Forest Region over the 21st century 

 William R. Vaughn, Anthony R. Taylor, David A. MacLean, Loïc D’Orangeville, Michael B. Lavigne   Canadian Journal of Forest Research. 2021; : 1   [Pubmed]  [DOI]   35 
PLSSEM for Software Engineering Research 

 Daniel Russo, KlaasJan Stol   ACM Computing Surveys. 2021; 54(4): 1   [Pubmed]  [DOI]   36 
Hybrid Speech and Text Analysis Methods for Speaker Change Detection 

 Or Haim Anidjar, Itshak Lapidot, Chen Hajaj, Amit Dvir, Issachar Gilad   IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2021; 29: 2324   [Pubmed]  [DOI]   37 
An Empirical Investigation of the Relationship between Local Government Budgets, IT Expenditures, and Cyber Losses 

 Jay P. Kesan, Linfeng Zhang   IEEE Transactions on Emerging Topics in Computing. 2021; 9(2): 582   [Pubmed]  [DOI]   38 
Local and landscape scale drivers of
Euschistus servus
and
Lygus lineolaris
in North Carolina small grain agroecosystems


 James K. Goethe, Seth J. Dorman, Anders S. Huseth   Agricultural and Forest Entomology. 2021; 23(4): 441   [Pubmed]  [DOI]   39 
Quality objectives in management systems – their attributes, establishment and motivational function 

 Marek Bugdol, Piotr Jedynak   International Journal of Quality & Reliability Management. 2021; aheadofp(aheadofp)   [Pubmed]  [DOI]   40 
Mapping the implications and competencies for Industry 4.0 to hard and soft total quality management 

 Oluwayomi Kayode Babatunde   The TQM Journal. 2021; 33(4): 896   [Pubmed]  [DOI]   41 
Development and virtual validation of a novel digital workflow to rehabilitate palatal defects by using smartphoneintegrated stereophotogrammetry (SPINS) 

 Taseef Hasan Farook, Nafij Bin Jamayet, Jawaad Ahmed Asif, Abdul Sattar Din, Muhammad Nasiruddin Mahyuddin, Mohammad Khursheed Alam   Scientific Reports. 2021; 11(1)   [Pubmed]  [DOI]   42 
Preliminary evaluation of a multicomponent youth development program for siblings separated by foster care: pandemic related impacts to service delivery and youth wellbeing 

 Jeffrey Waid, Cynthia Dantas   Journal of Public Child Welfare. 2021; : 1   [Pubmed]  [DOI]   43 
Innovative regressionbased methodology to assess the technoeconomic performance of photovoltaic installations in urban areas 

 Enrique FusterPalop, Carlos PradesGil, X. Masip, Joan D. VianaFons, Jorge Payá   Renewable and Sustainable Energy Reviews. 2021; 149: 111357   [Pubmed]  [DOI]   44 
Interrelationship between micronutrients and cardiovascular structure and function in type 2 diabetes 

 Grace W. M. Walters, Emma Redman, Gaurav S. Gulsin, Joseph Henson, Stavroula Argyridou, Thomas Yates, Melanie J. Davies, Kelly Parke, Gerry P. McCann, Emer M. Brady   Journal of Nutritional Science. 2021; 10   [Pubmed]  [DOI]   45 
The Effect of SelfPaced Exercise Intensity and Cardiorespiratory Fitness on Frontal Grey Matter Volume in Cognitively Normal Older Adults: A Randomised Controlled Trial 

 Natalie J. Frost, Michael Weinborn, Gilles E. Gignac, Ying Xia, Vincent Doré, Stephanie R. RaineySmith, Shaun Markovic, Nicole Gordon, Hamid R. Sohrabi, Simon M. Laws, Ralph N. Martins, Jeremiah J. Peiffer, Belinda M. Brown   Journal of the International Neuropsychological Society. 2021; : 1   [Pubmed]  [DOI]   46 
Heart rate variability responses determined by photoplethysmography in people with spinal cord injury 

 Luiz Henrique Rufino Batista, Wagner Jorge Ribeiro Domingues, Anselmo de Athayde Costa e Silva, Kathya Augusta Thomé Lopes, Minerva Leopoldina de Castro Amorim, Mateus Rossato   Biomedical Signal Processing and Control. 2021; 69: 102845   [Pubmed]  [DOI]   47 
Effectiveness of fascial manipulation on pain and disability in musculoskeletal conditions. A systematic review 

 Karthik Arumugam, Karvannan Harikesavan   Journal of Bodywork and Movement Therapies. 2021; 25: 230   [Pubmed]  [DOI]   48 
A simulationbased framework for modulating the effects of subjectivity in greenfield Mineral Prospectivity Mapping with geochemical and geological data 

 Mohammad Parsa, Amin Beiranvand Pour   Journal of Geochemical Exploration. 2021; 229: 106838   [Pubmed]  [DOI]   49 
Examining reciprocal associations between parent depressive symptoms and child internalizing symptoms on subsequent psychiatric disorders: An adoption study 

 Camille C. Cioffi, Leslie D. Leve, Misaki N. Natsuaki, Daniel S. Shaw, David Reiss, Jody M. Ganiban, Jenae M. Neiderhiser   Depression and Anxiety. 2021;   [Pubmed]  [DOI]   50 
Patient safety, quality of care and missed nursing care at a cardiology department during the COVID19 outbreak 

 Carolin Nymark, AnnChristin Vogelsang, AnnCharlotte Falk, Katarina E Göransson   Nursing Open. 2021;   [Pubmed]  [DOI]   51 
Digital forensic readiness intelligence crime repository 

 Victor R. Kebande, Nickson M. Karie, KimKwang Raymond Choo, Sadi Alawadi   Security and Privacy. 2021; 4(3)   [Pubmed]  [DOI]   52 
Statistical data presentation: a primer for rheumatology researchers 

 Durga Prasanna Misra, Olena Zimba, Armen Yuri Gasparyan   Rheumatology International. 2021; 41(1): 43   [Pubmed]  [DOI]   53 
Macrolides for the prevention and treatment of feeding intolerance in preterm low birth weight infants: a systematic review and metaanalysis 

 Sriparna Basu, Susan Smith   European Journal of Pediatrics. 2021; 180(2): 353   [Pubmed]  [DOI]   54 
Modulating the Impacts of Stochastic Uncertainties Linked to Deposit Locations in DataDriven Predictive Mapping of Mineral Prospectivity 

 Mohammad Parsa, Emmanuel John M. Carranza   Natural Resources Research. 2021; 30(5): 3081   [Pubmed]  [DOI]   55 
An Empirical Validation Method for Narrowing the Range of Poverty Thresholds 

 Geranda Notten, Julie Kaplan   Social Indicators Research. 2021;   [Pubmed]  [DOI]   56 
The effects of feedback valance and progress monitoring on goal striving 

 Leah Borovoi, Kelly Schmidtke, Ivo Vlaev   Current Psychology. 2020;   [Pubmed]  [DOI]   57 
SelfProtection versus Fear of Stricter Firearm Regulations: Examining the Drivers of Firearm Acquisitions in the Aftermath of a Mass Shooting 

 Maurizio Porfiri, Roni BarakVentura, Manuel Ruiz Marín   Patterns. 2020; 1(6): 100082   [Pubmed]  [DOI]   58 
Multivariate log file analysis for multileaf collimator failure prediction in radiotherapy delivery 

 Arkadiusz Mariusz Wojtasik, Matthew Bolt, Catharine H. Clark, Andrew Nisbet, Tao Chen   Physics and Imaging in Radiation Oncology. 2020; 15: 72   [Pubmed]  [DOI]   59 
On comparing locations of twoparameter exponential distributions using sequential sampling with applications in cancer research 

 Yan Zhuang, Sudeep R. Bapat   Communications in Statistics  Simulation and Computation. 2020; : 1   [Pubmed]  [DOI]   60 
The tradeoff between adult size and development time due to different feeding regimes in the scorpion Tityus neibae 

 Michael Seiter, Laurin Mosetig, Norbert Milasowszky   Invertebrate Reproduction & Development. 2020; 64(4): 274   [Pubmed]  [DOI]   61 
Focusing on fidelity: narrative review and recommendations for improving intervention fidelity within trials of health behaviour change interventions 

 E. Toomey, W. Hardeman, N. Hankonen, M. Byrne, J. McSharry, K. MatvienkoSikar, F. Lorencatto   Health Psychology and Behavioral Medicine. 2020; 8(1): 132   [Pubmed]  [DOI]   62 
Methodologic concerns regarding the evidence of a higher prevalence of apical periodontitis and endodontic treatment need in tobacco smokers 

 E. J. N. L. Silva, K. P. Pinto, C. M. Ferreira, L. C. Maia, L. M. Sassone, T. K. S. Fidalgo   International Endodontic Journal. 2020; 53(12): 1744   [Pubmed]  [DOI]   63 
Don’t touch: Developmental trajectories of toddlers’ behavioral regulation related to older siblings’ behaviors and parental discipline 

 Sheila R. Berkel, JuHyun Song, Richard Gonzalez, Sheryl L. Olson, Brenda L. Volling   Social Development. 2020; 29(4): 1031   [Pubmed]  [DOI]   64 
Investigating the Effect of Prompt Treatment on Malaria Prevalence in Children Aged below Five Years in Zambia: A Nested CaseControl Study in a CrossSectional Survey 

 Mukumbuta Nawa   Advances in Public Health. 2020; 2020: 1   [Pubmed]  [DOI]   65 
Pharmacists' viewpoint towards their professional role in healthcare system: a survey of hospital settings of Pakistan 

 Nabeel Khan, Ken McGarry, Atta Abbas Naqvi, Muhammad Shahid Iqbal, Zaki Haider   BMC Health Services Research. 2020; 20(1)   [Pubmed]  [DOI]   66 
Staffordshire Bull Terriers in the UK: their disorder predispositions and protections 

 Camilla Pegram, Katie Wonham, Dave C. Brodbelt, David B. Church, Dan G. O’Neill   Canine Medicine and Genetics. 2020; 7(1)   [Pubmed]  [DOI]   67 
Life history aspects of the buthid scorpion Tityus forcipula (Gervais, 1843) with remarks on its thermal limits 

 Michael Seiter, Nathalie Friedl, Michiel A.C. Cozijn   The Journal of Arachnology. 2020; 48(2)   [Pubmed]  [DOI]   68 
Enhancing Executive Control: Attention to Balance, Breath, and the Speed Versus Accuracy Tradeoff 

 Varsha Singh, Vaishali Mutreja   Frontiers in Psychology. 2020; 11   [Pubmed]  [DOI]   69 
Country and Sex Differences in Decision Making Under Uncertainty and Risk 

 Varsha Singh, Johannes Schiebener, Silke M. Müller, Magnus Liebherr, Matthias Brand, Melissa T. Buelow   Frontiers in Psychology. 2020; 11   [Pubmed]  [DOI]   70 
Effects of an actual insulin injection demonstration on insulin acceptance among patients with T2DM: a pragmatic randomised controlled trial 

 Atthayaporn Choomai, Apichai Wattanapisit, Orathai Tiangtam   Romanian Journal of Internal Medicine. 2020; 0(0)   [Pubmed]  [DOI]   71 
The concept of motivation for effective credit risk management 

 M.V. Pomazanov   Finance and Credit. 2020; 26(11): 2567   [Pubmed]  [DOI]   72 
Intubation Setting, Aspiration, and VentilatorAssociated Conditions 

 Steven Talbert, Christine Wargo Detrick, Kimberly Emery, Aurea Middleton, Bassam Abomoelak, Chirajyoti Deb, Devendra I. Mehta, Mary Lou Sole   American Journal of Critical Care. 2020; 29(5): 371   [Pubmed]  [DOI]   73 
Raiders of the Lost Correlation: A Guide on Using Pearson and Spearman Coefficients to Detect Hidden Correlations in Medical Sciences 

 Alessandro Rovetta   Cureus. 2020;   [Pubmed]  [DOI]   74 
Neoplasia associada ao tratamento das doenças reumáticas 

 Gustavo Guimarães Moreira Balbi   Revista Paulista de Reumatologia. 2020; (2020 janm): 13   [Pubmed]  [DOI]   75 
Comparison of Accuracy and Reliability of Working Length Determination Using Cone Beam Computed Tomography and Electronic Apex Locator: A Systematic Review 

 Amar Sholapurkar, Janki Amin, Jordan Lines, Maxim P Milosevic, Andrew Park   The Journal of Contemporary Dental Practice. 2019; 20(9): 1118   [Pubmed]  [DOI]   76 
Automatic Texture Based Classification of the Dynamics of OneDimensional Binary Cellular Automata 

 Marcelo Arbori Nogueira, Pedro Paulo Balbi de Oliveira   International Journal of Natural Computing Research. 2019; 8(4): 41   [Pubmed]  [DOI]   77 
Biases in feature selection with missing data 

 Borja SeijoPardo, Amparo AlonsoBetanzos, Kristin P. Bennett, Verónica BolónCanedo, Julie Josse, Mehreen Saeed, Isabelle Guyon   Neurocomputing. 2019; 342: 97   [Pubmed]  [DOI]   78 
Differentiating conscientious from indiscriminate responders in existing NEOFive Factor Inventory3 data 

 Zdravko Marjanovic, Ronald R. Holden   Journal of Research in Personality. 2019; 81: 127   [Pubmed]  [DOI]   79 
Addressing fidelity within complex health behaviour change interventions: A protocol of a scoping review of intervention fidelity frameworks and models. 

 Rebekah Roy, Heather Colquhoun, Molly Byrne, Fabiana Lorencatto, Karen MatvienkoSikar, Nicola McCleary, Nicole Nathan, Elaine Toomey   HRB Open Research. 2018; 1: 25   [Pubmed]  [DOI]   80 
Confidence Interval Width for Pearson’s Correlation Coefficient: A GaussianIndependent Estimator Based on Sample Size and Strength of Association 

 Tiago Olivoto, Alessandro D. C. Lúcio, Velci Q. Souza, Maicon Nardino, Maria I. Diel, Bruno G. Sari, Dionatan K. Krysczun, Daniela Meira, Carine Meier   Agronomy Journal. 2018; 110(2): 503   [Pubmed]  [DOI]   81 
Validating Factors That Impact the Acceptance and Use of eAssessment among Academics in Saudi Universities 

 Nuha Alruwais,Gary Wills,Mike Wald   International Journal of Information and Education Technology. 2017; 7(10): 716   [Pubmed]  [DOI]   82 
Addressing low health literacy with “Talking Pill Bottles”: A pilot study in a community pharmacy setting 

 Annie Y. Lam,Juliet K. Nguyen,Jason J. Parks,Donald E. Morisky,Donna L. Berry,Seth E. Wolpin   Journal of the American Pharmacists Association. 2017; 57(1): 20   [Pubmed]  [DOI]   83 
An exploratory study for investigating the critical success factors for cloud migration in the Saudi Arabian higher education context 

 Abdulrahman Alharthi,Madini O. Alassafi,Robert J. Walters,Gary B. Wills   Telematics and Informatics. 2017; 34(2): 664   [Pubmed]  [DOI]   84 
A Hybrid Computeraideddiagnosis System for Prediction of Breast Cancer Recurrence (HPBCR) Using Optimized Ensemble Learning 

 Mohammad R. Mohebian,Hamid R. Marateb,Marjan Mansourian,Miguel Angel Mañanas,Fariborz Mokarian   Computational and Structural Biotechnology Journal. 2017; 15: 75   [Pubmed]  [DOI]   85 
Why Psychologists Should by Default Use Welch’s ttest Instead of Student’s ttest 

 Marie Delacre, Daniël Lakens, Christophe Leys   International Review of Social Psychology. 2017; 30(1): 92   [Pubmed]  [DOI]   86 
Advantages of Computer Simulation in Enhancing Studentsæ Learning About Landform Evolution: A Case Study Using the Grand Canyon 

 Wei Luo,Jon Pelletier,Kirk Duffin,Carol Ormand,Weichen Hung,David J. Shernoff,Xiaoming Zhai,Ellen Iverson,Kyle Whalley,Courtney Gallaher,Walter Furness   Journal of Geoscience Education. 2016; 64(1): 60   [Pubmed]  [DOI]   87 
Systematic review of published studies on aquatic exercise for balance in patients with multiple sclerosis, Parkinsonæs disease, and hemiplegia 

 Pichanan Methajarunon,Chachris Eitivipart,Claire J. Diver,Anchalee Foongchomcheay   Hong Kong Physiotherapy Journal. 2016; 35: 12   [Pubmed]  [DOI]   88 
Power and Confounding in Diffuse Alveolar Hemorrhage Secondary to Antineutrophil Cytoplasmic AntibodyAssociated Vasculitis: Comment on the Article by CartinCeba et al 

 Mauricio RestrepoEscobar,Johanna HernándezZapata   Arthritis & Rheumatology. 2016; 68(11): 2827   [Pubmed]  [DOI]   89 
An Overview of Bootstrapping Method Applicable to Survey Researches in Rehabilitation Science 

 Bongsam Choi   Physical Therapy Korea. 2016; 23(2): 93   [Pubmed]  [DOI]   90 
Gender differences in the neural response to acupuncture: clinical implications 

 Sujung Yeo,Bruce Rosen,Peggy Bosch,Maurits van den Noort,Sabina Lim   Acupuncture in Medicine. 2016; 34(5): 364   [Pubmed]  [DOI]   91 
Correlates of NearInfrared Spectroscopy Brain–Computer Interface Accuracy in a MultiClass Personalization Framework 

 Sabine Weyand,Tom Chau   Frontiers in Human Neuroscience. 2015; 9   [Pubmed]  [DOI]   92 
On the performances of the flower pollination algorithm – Qualitative and quantitative analyses 

 Amer Draa   Applied Soft Computing. 2015; 34: 349   [Pubmed]  [DOI]   93 
Bacteremia after piezocision 

 Zehra Ileri,Mehmet Akin,Emire Aybuke Erdur,Hatice Turk Dagi,Duygu Findik   American Journal of Orthodontics and Dentofacial Orthopedics. 2014; 146(4): 430   [Pubmed]  [DOI]   94 
Grandmothers’ Smoking in Pregnancy and Grandchildren’s Birth Weight: Comparisons by Grandmother Birth Cohort 

 Eileen RillamasSun,Siobán D. Harlow,John F. Randolph   Maternal and Child Health Journal. 2013;   [Pubmed]  [DOI]  



