RT @CraccoEmiel: @lakens Lack of correlations with cognitive tasks may be consequence of these tasks minimizing between-subject variability…
@lakens Lack of correlations with cognitive tasks may be consequence of these tasks minimizing between-subject variability, though. See: https://t.co/ytECIO02XH
@danengber @max12elliott @talyarkoni @BrentWRoberts this is a nice paper on this topic: https://t.co/0W7RbdA6hb
RT @max12elliott: @the_mindwanders @JamieLarsH @ajshackman @INM7_ISN In addition to our paper, I would highly recommend https://t.co/uRHoxY…
@the_mindwanders @JamieLarsH @ajshackman @INM7_ISN In addition to our paper, I would highly recommend https://t.co/uRHoxYne5d For more discussion of the tension between within-subject and between subject variance
@bradpwyble One problem might be the low test-retest reliability of many tasks that use averaged RTs to estimate effects (see: https://t.co/c3MtWHWoGD). Then, any inference on a given individual (i.e. spy vs. no spy) will be very noisy.
RT @stevenjluck: Another great paper! Bottom line: Low between-subjects variability in an effect gives us high statistical power in experim…
RT @stevenjluck: Another great paper! Bottom line: Low between-subjects variability in an effect gives us high statistical power in experim…
Another great paper! Bottom line: Low between-subjects variability in an effect gives us high statistical power in experimental studies but low power in looking for individual differences. The flanker interference effect, e.g., is super robust but has medi
@stevenjluck This paper adds to the discussion: https://t.co/2t8UNsQTga
What I'm reading today: https://t.co/R8qHV2vwjh #lectureprep #readallthethings
RT @neurofractal: Really interesting - I always thought the opposite. Robust paradigms are useful for individual differences as any devianc…
Really interesting - I always thought the opposite. Robust paradigms are useful for individual differences as any deviance from the 'normal' is reliable and interpretable.. or you just end up analysing noise. Good suggestions for alternative approaches in
Using robust experimental tasks to study individual differences may be suboptimal. Here's why... https://t.co/NMFujJ86ug https://t.co/EdVhyFKfGC Thanks to @lucycragg for highlighting this literature! https://t.co/4JdyIh7O3D
@rj_redden @OSFramework All data from the reliability paradox paper (https://t.co/22I0D4zAgc) are available on the OSF: https://t.co/X3BF2NW8IL Unfortunately no code and materials.
@KMKing_Psych @MattPalmatier @NeuroPolarbear @minzlicht indeed, that's exactly the point that this paper makes: https://t.co/kEAK5QFvju
@ManonIronside well, one can in principle have a measure that is valid but not reliable in the test/retest sense, because there is too little between-subject variability - see https://t.co/4TT1JhIDD9 - but that's probably too deep in the weeds for this boo
The “problem with the instrument” argument has been explicitly put forward by the authors of the studies. According to this hypothesis, the low external validity and reliability of the behavioral tasks derive from intrinsic limitations of the tasks (https:
@LHuntNeuro @sampendu @DrGBuckingham @chrisdc77 @jonroiser @talyarkoni This a nice article which outlines why some of our most precise measurements may make terrible predictors for subject-level variance. https://t.co/uPI0Vhoefv
@KMKing_Psych Great reading suggestions. I like the recent empirical paper by Craig Hedge et al. (2018) on this topic, “The reliability paradox” https://t.co/eAI4p7QDlH
@AnaM_Triana And I add that it's not structure/function dualism, I think that there are some structural functions that we share a lot across individuals and are not making us "unique"... I still haven't made my mind on that but it follows the "reliability
There's evidence that these tasks might have poor reliability by *design*, because they are intended to maximize an experimental effect (like the Stroop effect) which wipes out individual differences. https://t.co/c4VCeUCcEw /7
@KinglaKing I think relatedly, we have the development of experimental paradigms that try to wipe out individual differences so EVERYONE shows the effect of the manipulation (like the Stroop). This ends up focusing on really extreme and un-generalizable ef
@keanan_joyner @Psychologician This is probably in part because they are often designed to maximize an experimental effect (like the Stroop contrast), making it difficult to measure between person variance https://t.co/LEP8aPO88s https://t.co/c4VCeUCcEw 6
@aaronjfisher This is it: https://t.co/aC7Kt7ICQs
RT @neurobot01: And now @glassybrain reiterating the point that reliable (at the individual level) is in some ways the enemy of robust (at…
And now @glassybrain reiterating the point that reliable (at the individual level) is in some ways the enemy of robust (at the group level); see also: https://t.co/O6q2s8d8wP #OHBM2018
@flourneuro And related to this important paper on the paradox reliability: https://t.co/GVxffuay6l by Hedge et al.
RT @KMKing_Psych: @JeffRouder There's a great paper by Hedge that made the same point. When you design your task (like the Stroop) to alway…
@JeffRouder There's a great paper by Hedge that made the same point. When you design your task (like the Stroop) to always show the within person effect, you often kill your power to show individual differences. https://t.co/c4VCeUCcEw
@andrewang91 @JkayFlake @EikoFried @mijkenijk Neat; I see your topic ("connecting unreliable measurements to power"); presumably you're going to talk about how reliability is pretty much useless as a proxy for power? (ex. https://t.co/utpOKgz82W & http
@MikeLwrnc @t_awkr @profelainefox this rings true with Hedge's approach https://t.co/KJYI3IWUNc in presenting the proportion of variance explained by within Ss, Between Ss, and error. But, I don't know if we can make this distinction without test-retest, e
@Sam_D_Parsons @t_awkr @profelainefox These tasks also often measure within *and* between person effects (like the Stroop) and little thought is often given to whether they measure both reliably (or how to construct tasks to be a good measure of within per
@EikoFried @JkayFlake For cognitive measures; Hedge et al. The reliability paradox https://t.co/veGFqjNdpL and Cooper et al. The Role of Psychometrics in Individual Differences Research in Cognition https://t.co/I4WCDsVrY3
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @MacBuRo: @robinnkok And even when I think a measure is reliable because it produces replicable effects, I see this in my timeline and c…
@robinnkok And even when I think a measure is reliable because it produces replicable effects, I see this in my timeline and cry: https://t.co/rxggb6MVeX (I also cry out of guilt for not reporting or testing the reliability of the measures I use, but it'
blog post came about when I began to play with the idea that the reliability of our measures should impact power. Reading Hedge et al. also confirmed this. https://t.co/KJYI3IWUNc
cognitive scientists https://t.co/MWiO4KtjBL
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @JelteWicherts: The reliability of several popular cognitive tasks is surprisingly low. https://t.co/H93x85ZUyj
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @athanatizein: While obvious, this is something that we really need to pay more attention to. It's important that we don't sculpt our pa…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @neuroconscience: I think this is one of the major challenges facing #ComputationalPsychiatry - task needs to do more than just index me…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @CompBrainBeh: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of indiv…
RT @mvugt: must-read for cognitive scientists: tasks that are very reliable across sessions by definition are poor measures of individual d…