Publications about misinterpretations and critique on NCA

This section refers to publications about misinterpretations and critique on NCA. 

Sorjonen, K., Wikström Alex, J., & Melin, B. (2017). Necessity as a Function of Skewness. Frontiers in psychology, 8, 2192.

Reply: Dul, J., van der Laan, E., & Kuik, R. (2020). A statistical significance test for Necessary Condition Analysis. Organizational Research Methods, 23(2), 385–395.

Sorjonen et al. (2017) show by simulation that two unrelated variables can produce an empty space in the upper left corner. This is a valid observation. However, this situation can be detected by NCA’s significance test (Dul, van der Laan, & Kuik, 2020); see also Dul, van der Laan, Kuik, & Karwowski, 2019) , which tests the null hypothesis that the two variables are unrelated. Note that NCA’s 'approximate permutation test'  is also called a 'randomness test' of the empty space. 

Sorjonen, K., & Melin, B. (2019). Predicting the significance of necessity. Frontiers in psychology, 10, 283.

Reply: Dul, J., van der Laan, E., Kuik, R, & Karwowski, M. (2019). Necessary Condition Analysis: Type I error, power, and over-interpretation of test results. A reply to a comment on NCA, Frontiers in Psychology, 10, 1493. (see also the long version by Dul, van der Laan, Kuik and Karwowski, 2019)

Sorjonen & Melin (2019) show by simulation that the probability that NCA’s significance test results in p < 0.05 when X and Y are related. Sorjonen and Melin (2019) criticize NCA’s significance test because of its inability to specify which alternative hypothesis of relatedness between X and Y resulted in p < 0.05. However, no statistical null hypothesis test can test a specific alternative hypothesis; expecting this from a null hypothesis test is a common misconception. NCA’s significance test is a null-hypothesis test, not a test of an alternative hypothesis.

The same misconception about the ability of a null-hypothesis test is made in Sorjonen, K., & Melin, B. (2023). Necessary condition analysis has either low specificity or low sensitivity: Results from simulations and empirical analyses of grit, depression, and anxiety. Heliyon, 9(4). In this article two additional misinterpretations are (1) the claim that NCA's statistical test is a general randomness test, although it is a test of the randomness of the expected empty corner, and (2) that NCA is just a statistical test when it tests necessity, although NCA has two other requirements as well for concluding about necessity: theoretical support (directing the attention to the right expected empty corner), and minimum effect size (e.g. d ≥  0.1, ensuring practical relevance). With the correct interpretation of NCA, simulations show that NCA has high sensitivity (true positive rate) and high specificity (true negative rate) for identifying necessity.

Thiem, A. (2021). The Logic and Methodology of “Necessary but Not Sufficient Causality” A Comment on Necessary Condition Analysis (NCA). Sociological Methods & Research. 50(2), 913-925.

Reply: Dul, J., Vis, B., & Goertz, G. (in press). Necessary Condition Analysis (NCA) Does Exactly What It Should Do When Applied Properly: A Reply to a Comment on NCA. Sociological Methods & Research. 50(2), 926-936.

Thiem (2021) criticizes NCA as a valid method for identifying necessary conditions. He argues that QCA is better equipped to do so. The reply to this article  shows that the article is based on wrong assumptions about what NCA aims to do (necessity not sufficiency; focus on necessary conditions not on INUS conditions), and applies NCA incorrectly.

A similar misconception about the aims of NCA can be found in Ilagan, M. J., & Patungan, W. (2018). The relationship between intelligence and creativity: On methodology for necessity and sufficiency. Archives of Scientific Psychology, 6(1), 193.) These authors critisize NCA because it does not specify the full data generation process (DGP). This is a correct observation, but NCA's goal is to identify necessity (the empty space above the ceiling) and not about probabilistic sufficiency (the full space below the ceiling) and therefore does not specify the full DGP. Expecting this from NCA is a misconception.

Dul, J. (2022). Problematic applications of necessary condition analysis (NCA) in tourism and hospitality research. Tourism Management, 104616. 

This article identifies two problematic applications of NCA that are primarily published in the tourism and hospitality research field when NCA is combined with QCA. The first problem is a logical misinterpretation that a factor that is a necessary condition for the outcome must be part of each sufficient configuration (which is correct), also implies the opposite: a factor that is part of the sufficient configuration must be a necessary condition (which is not correct). The second problem is the statement that NCA is the necessity analysis of fsQCA.