Diabetes self-management (DSM) is a crucial part in the management of diabetic patients. Most randomized controlled clinical trials reported significant benefits by diabetes self-management education (DSME) on DSM behaviors and metabolic control. Although the randomized clinical trials are the gold standard method in assessing the effectiveness of any intervention, including DSME interventions, the outcomes of these studies may reflect exaggerated effects; because in most of these studies, subjects in control group receive usual (standard) care with no any DSME. The lack (...) of such education in the control group had many drawbacks: ( 1 ) make blindness impossible; at which non blinded studies may result in false-positive findings; ( 2 ) it may be unethical to recruit patients for participation in DSME studies as control subjects, most of these studies with long periods of follow up, without any direct benefit to them during the study period; the lack of direct benefit to control subjects in any clinical trial may results in high attrition (drop-out) rate. The high attrition rate in clinical trials especially in the control group can lower the statistical power of the study results; thus, the DSME program would appear to have positive effects even if the program had no real benefit. This problem of the DSME exaggerated effects can be solved partially through providing control subjects with education about some DSM behaviors or comparing the effect of DSME through different educational approaches (for example DSME by didactic vs interactive approach). (shrink)

A recent study of moral intuitions, performed by Joshua Greene and a group of researchers at Princeton University, has recently received a lot of attention. Greene and his collaborators designed a set of experiments in which subjects were undergoing brain scanning as they were asked to respond to various practical dilemmas. They found that contemplation of some of these cases (cases where the subjects had to imagine that they must use some direct form of violence) elicited greater activity in certain (...) areas of the brain associated with emotions compared with the other cases. It has been argued (e.g., by Peter Singer) that these results undermine the reliability of our moral intuitions, and therefore provide an objection to methods of moral reasoning that presuppose that they carry an evidential weight (such as the idea of reflective equilibrium). I distinguish between two ways in which Greene's findings lend support for a sceptical attitude towards intuitions. I argue that, given the first version of the challenge, the method of reflective equilibrium can easily accommodate the findings. As for the second version of the challenge, I argue that it does not so much pose a threat specifically to the method of reflective equilibrium but to the idea that moral claims can be justified through rational argumentation in general. (shrink)

Background: Screen time among adults represents a continuing and growing problem in relation to health behaviors and health outcomes. However, no instrument currently exists in the literature that quantifies the use of modern screen-based devices. The primary purpose of this study was to develop and assess the reliability of a new screen time questionnaire, an instrument designed to quantify use of multiple popular screen-based devices among the US population. -/- Methods: An 18-item screen-time questionnaire was created to quantify use (...) of commonly used screen devices (e.g. television, smartphone, tablet) across different time points during the week (e.g. weekday, weeknight, weekend). Test-retest reliability was assessed through intra-class correlation coefficients (ICCs) and standard error of measurement (SEM). The questionnaire was delivered online using Qualtrics and administered through Amazon Mechanical Turk (MTurk). -/- Results: Eighty MTurk workers completed full study participation and were included in the final analyses. All items in the screen time questionnaire showed fair to excellent relative reliability (ICCs = 0.50–0.90; all < 0.000), except for the item inquiring about the use of smartphone during an average weekend day (ICC = 0.16, p = 0.069). The SEM values were large for all screen types across the different periods under study. -/- Conclusions: Results from this study suggest this self-administered questionnaire may be used to successfully classify individuals into different categories of screen time use (e.g. high vs. low); however, it is likely that objective measures are needed to increase precision of screen time assessment. (shrink)

Recent research introduced the Discounting Inventory that allows the measurement of individual differences in the delay, probabilistic, effort, and social discounting rates. The goal of this investigation was to determine several aspects of the reliability of the Discounting Inventory using the responses of 385 participants (200 non-smokers and 185 current-smokers). Two types of reliability are of interest. Internal consistency and test-retest stability. A secondary aim was to extend such reliability measures beyond the non-clinical participant. The current study (...) aimed to measure the reliability of the DI in a nicotine-dependent individuals and non-nicotine-dependent individuals. It is concluded that the internal consistency of the DI is excellent, and that the test-retest reliability results suggest that items intended to measure three types of discounting were likely testing trait, rather than state, factors, regardless of whether “non-smokers” were included in, or excluded from, the analyses (probabilistic discounting scale scores being the exception). With these cautions in mind, however, the psychometric properties of the DI appear to be very good. (shrink)

There are many theories surrounding infant imitation; however, there is no research to our knowledge evaluating the reliability of listener perception of vocal imitation in prelinguistic infants. This paper evaluates intra- and inter-rater judgments on the degree of “imitativeness” in utterances of infants below 12 months of age. 18 listeners were presented audio segments selected from naturalistic recordings to represent in each case a parent vocal model followed by an infant utterance ranging from low to high degrees of imitativeness. (...) The naturalistic data suggested vocal imitation occurred rarely across the first year, but strong intra- and inter-rater correlations were found for judgments of imitativeness. Our results suggest salience of the infant’s vocal imitation despite its rare occurrence as well as active perception by listeners of the imitative signal. We discuss infant vocal imitation as a potential signal of well-being and as an indicator of language learning and vocal communication skills as perceived by caregivers. (shrink)

BackgroundThe BelRAI Screener is a short-form assessment consolidating internationally validated interRAI items focusing on physical and psychological aspects of functioning and problems with activities of daily living. It was fully implemented in the Flemish home care setting as of June 2021. In a biopsychosocial model for developing a personalized and effective care plan social and contextual aspects are considered equally important to biomedical ones. Thus, a social supplement to the BelRAI Screener was collaboratively developed with stakeholders and tested to gather (...) additional information on the social context of community-dwelling adults with care needs.Objective and methodsTo examine the interrater reliability of the BelRAI Social Supplement in Flanders, Belgium, an observational study was conducted using a convenience sample. The method of simultaneous rating was used due to strict COVID-19 guidelines at the time and to minimize assessment burden. Fifty two community-dwelling adults requesting home care support were simultaneously assessed by two independent assessors during home visits. Interrater reliability was tested on all 80 items of the BelRAI Social Supplement using observed agreement, kappa coefficients, and intraclass correlation coefficients.ResultsThe kappa mean and median values for nominal items, show substantial agreement, while the kappa mean and median values for ordinal items were 0.81 and 0.90, which correspond to almost perfect agreement. Following the traditional cut-off points for the interpretation of the kappa statistic, reliability was almost perfect for 49% of all items, substantial for 33%, moderate for 8%, and poor for 10%. The majority of items with poor kappa value, showed a high observed agreement, reflecting homogeneity of the sample rather than poor agreement.ConclusionThe strength of kappa agreement for the items in this version of the BelRAI Social Supplement is generally substantial to almost perfect, with high proportions of observed agreement. COVID-19 restrictions had a large impact on the planning and execution of the home visits. A final optimization of the instrument and accompanying manual according to the findings will result in an improved version ready for nation-wide implementation. (shrink)

This article investigates whether investors consider the reliability of companies’ sustainability information when determining the companies’ market value. Specifically, we examine market reactions (in terms of abnormal returns) to events that increase the reliability of companies’ sustainability information but do not provide markets with additional sustainability information. Controlling for competing effects, we regard companies’ additions to an internationally important sustainability index as such events and consider possible determinants for market reactions. Our results suggest that first, investors take (...) into account the reliability of sustainability information when determining the market value of a company and second, the benefits of increased reliability of sustainability information vary cross-sectionally. More specifically, companies that carry higher risks for investors (e.g., higher systematic investment risk, higher financial leverage, and higher levels of opportunistic management behavior) react more strongly to an increase in the reliability of sustainability information. Finally, we show that the benefits of an increase in the reliability of sustainability information are higher in times of economic uncertainty (e.g., during economic downturns and generally high stock price volatilities). (shrink)

Timothy Williamson contends that our primary cognitive heuristic for prospectively assessing conditionals, i.e., the suppositional procedure, is provably inconsistent. Our diagnosis is that stipulations about the nature of suppositional rejection are the likely source of these results. We show that on at least one alternative, and quite natural, understanding of the suppositional attitudes, the inconsistency results are blocked. The upshot is an increase in the reliability of our suppositional heuristics across a wider range of contexts. One interesting (...) consequence of the increased reliability is a proportional decrease in the plausibility of an error-theory that Williamson employs against widespread intuitions about the truth values of counterpossible conditionals. (shrink)

PURPOSE: The aim of this study was to assess the reliability and validity of the new judging system in DanceSport. METHODS: Eighteen judges rated the 12 best placed adult dancing couples competing at an international competition. They marked each couple on all judging criteria on a 10 level scale. Absolute agreement and consistency of judging were calculated for all main judging criteria and sub-criteria. RESULTS: A mean correlation of overall judging marks was 0.48. Kendall’s coefficient of concordance for (...) overall marks (W = 0.58) suggesting relatively low agreement among judges. Slightly lower coefficients were found for the artistic part (Partnering skills (W = 0.45) and Choreography and performance (W = 0.49)) compared to the technical part (Technical qualities (W = 0.56) and Movement to music (W = 0.54)). ICC for overall criteria was low for absolute agreement (ICC(2,3) = 0.62) but higher for consistency (ICC(3,3) = 0.80). CONCLUSION: The relatively large differences between judges’ marks suggest that judges either disagreed to some extent on the quality of the dancing or used the judging scale in different ways. The biggest concern was standard error of measurement (SEM) which was often larger than the difference between dancers scores suggesting that this judging system lacks validity. This was the first research to assess judging in DanceSport and offers suggestions to potentially improve both its objectivity and validity in the future. (shrink)

This article analyses the relationship between deduction and induction by focusing on Aristotle's knowledge acquisition processes. The deductive and inductive processes in Aristotelian science are analysed in depth, and it is emphasised that these two processes are, in fact, interrelated. It is claimed that induction and deduction use logical inference but are not themselves an inference. The structure of inductive inference is determined, and the deductive inference and the inferential part of the scientific process are given. Furthermore, the article addresses (...) the reliability of the inductive process, emphasising the difficulties in obtaining accurate results. In conclusion, it is shown that Aristotelian science has a fundamentally intuition-based part and plays an important role in the scientific process. (shrink)

An analysis is presented of published methods that have been used by experimenters to justify the reliability of the theory of invasion of microorganisms into cultured cells. The results show that, to demonstrate this invasion, many experimenters used two or more methods that were based on independent technical and theoretical principles, and by doing so improved the reliability of the theory. Subsequently I compare this strategy of 'multiple derivability' with other strategies, discussed in the literature in relation (...) to the mesosome, a bacterial organelle that had been detected with the electron microsope, but which appeared later to be an artifact. I propose that different strategies have been applied in this problem, and multiple derivability may have been the decisive one. Finally I discuss the idea that multiple derivability may help to anchor theories in a larger network of theories. (shrink)

The main aim of this paper is to revisit the curve fitting problem using the reliability of inductive inference as a primary criterion for the ‘fittest' curve. Viewed from this perspective, it is argued that a crucial concern with the current framework for addressing the curve fitting problem is, on the one hand, the undue influence of the mathematical approximation perspective, and on the other, the insufficient attention paid to the statistical modeling aspects of the problem. Using goodness-of-fit as (...) the primary criterion for ‘best', the mathematical approximation perspective undermines the reliability of inference objective by giving rise to selection rules which pay insufficient attention to ‘accounting for the regularities in the data'. A more appropriate framework is offered by the error-statistical approach, where (i) statistical adequacy provides the criterion for assessing when a curve captures the regularities in the data adequately, and (ii) the relevant error probabilities can be used to assess the reliability of inductive inference. Broadly speaking, the fittest curve (statistically adequate) is not determined by the smallness if its residuals, tempered by simplicity or other pragmatic criteria, but by the nonsystematic (e.g. white noise) nature of its residuals. The advocated error-statistical arguments are illustrated by comparing the Kepler and Ptolemaic models on empirical grounds. ‡I am grateful to Deborah Mayo and Clark Glymour for many valuable suggestions and comments on an earlier draft of the paper; estimating the Ptolemaic model was the result of Glymour's prompting and encouragement. †To contact the author, please write to: Department of Economics, Virginia Tech 3019 Pamplin Hall (0316), Blacksburg, VA 24061; e-mail: [email protected]. (shrink)

Online experimental methods are used in psychological studies investigating the perceptions and impressions of facial photographs, even without substantial evidence supporting their reliability and validity. Although, the quality of visual stimuli is more difficult to control remotely, the methods might allow us to obtain a large amount of data. Then the statistical analysis of a larger volume of data may reduce errors and suggest significant difference in the stimuli. Therefore, we analyzed the reliability and validity of online surveys (...) in investigating the perceptions and impressions of facial photographs created from averaged faces with skin tones modified using computer graphics. In this study, we conducted online and laboratory experiments with well-controlled conditions. For each experiment, 50 participants completed the same questionnaire regarding their impressions of the same 28 CG facial photographs. The results showed significant correlations between the two experiments for all 19 items in the questionnaire. SD in the Online1 compared to the Control from the stimuli and individual differences were 56–84 and 88–104% in each questionnaire items, respectively. Moreover, the rates of mismatching perceptual evaluations to the corresponding physical features demonstrated in the photographs were 4.9–9.7% on average in an additional online survey of another 2,000 participants. These results suggest that online surveys can be applied to experiments to investigate impressions from CG facial photographs instead of general laboratory experiment by obtaining an appropriate number of participants to offset larger statistical errors that may result from the increased noise in the data from conducting the experiment online. (shrink)

High prevalence of musicians’ physical and mental performance-related health issues (PRHI) has been demonstrated over the last 30 years. To address this, health promotion strategies have been implemented at some post-secondary music institutions around the world, yet the high prevalence of PRHI has persisted. In 2018, an international group of researchers formed the Musicians’ Health Literacy Consortium to determine how best to decrease PRHI, and to examine the relationship between PRHI and health literacy. An outcome of the Consortium was the (...) development of a new health literacy tool for musicians, the MHL-Q19, which drew from the theoretical framework of the European health literacy suite of tools, HLS-EU. The aim of the current study was to evaluate the validity and reliability of the MHL-Q19. Participants completed a battery of questionnaires, including the HLS-EU-Q16 for the assessment of general health literacy; the Musculoskeletal Pain Intensity and Interference Questionnaire for Musicians (MPIIQM); the RAND-12 quality of life questionnaire; and the General Self-Efficacy scale (GSE). We hypothesized that the MHL-Q19 would have a weak correlation with the HLS-EU-Q16; moderate correlation with the physical component scale and weak correlation with the mental component scale of the RAND-12; moderate correlation with the GSE; and finally, moderate correlation with pain interference and weak correlation with pain intensity of the MPIIQM. A total of 549 post-secondary music students from six English-speaking countries completed the battery of questionnaires, and 328 of these participants provided valid responses to the MHL-Q19 alone 2 weeks later. The tool showed acceptable internal consistency and test–retest reliability. Three of our hypotheses were supported, although the strength of the correlations varied from what we had predicted. The fourth hypothesis was not supported; our findings indicate that lower health literacy scores were weakly related to higher MPIIQM pain intensity and interference scores. The results of this study support the notion that musicians’ health literacy is a distinct construct that cannot be fully evaluated with existing health literacy tools. Given that this is a new instrument, the evidence presented is positive and promising. Further studies will be needed to refine the tool. (shrink)

Utilitarianism is a normative ethical theory that evaluates moral decisions based on their consequences, emphasizing the maximisation of overall well-being/outcomes for the greatest number. It serves as an important framework in clinical settings, where ethical dilemmas frequently arise. This study aimed to assess the validity and reliability of the Turkish version of the Oxford Utilitarianism Scale (OUS) among obstetricians and gynecologists (OB/GYNs). A total of 194 obstetricians and gynecologists participated in this online study and completed the OUS alongside a (...) brief sociodemographic questionnaire. Construct validity was assessed using confirmatory factor analysis (CFA). Reliability was evaluated via internal consistency coefficients (Cronbach’s alpha) and corrected item–total correlations. Additionally, measurement invariance analyses by sex were performed. CFA confirmed a two-dimensional structure— impartial beneficence and instrumental harm—with good fit indices. High reliability was indicated by both internal consistency coefficients and corrected item-total correlations. The results of the measurement invariance analyses showed similar measurement properties across genders. The Turkish OUS demonstrated adequate construct validity and acceptable internal consistency in this OB/GYN sample. It can be used as a reliable and valid tool to support research, ethics education, and structured ethical reflection in obstetric practice. (shrink)

Statistical evidence is pervasive in medicine. In this chapter we will focus on the reliability of randomized clinical trials (RCTs) conducted to test the safety and efficacy of medical treatments. RCTs are scientific experiments and, as such, we expect them to be replicable: if we repeat the same experiment time and again, we should obtain the same outcome (Norton 2015). The statistical design of the test should guarantee that the observed outcome is not a random event, but rather a (...) real effect of the treatments administered. However, for more than a decade now, we have been discussing a replicability crisis across different experimental disciplines including medicine: the outcomes of trials published in very prestigious journals often disappear when the experiment is repeated (see, e.g., Lehrer 2010, Begley and Ellis 2012, Horton 2015). There are different accounts of the reason for this replicability crisis, ranging from scientific fraud to lack of institutional incentives to double-check someone else’s results. In this chapter we will use the replicability crisis as a thread to introduce some central issues in the design of scientific experiments in medicine. First, in section 1, we will see how replicability and statistical significance are connected: we can only make sense of the p-value of a trial outcome within a series of replications of the test. But in order to conduct these replications properly, we need to agree on the proper design of the experiment we are going to repeat. In particular, we need to prevent the preferences of the experimenters from biasing the outcome of the experiment. If there is such a bias, when the experiment is replicated by a third party, the observed outcome will vanish. In section 2, we will argue that trialists need to agree on the debiasing procedures and the statistical quality controls that feature in the trial protocol, if they want the outcome to be replicable. In section 3, we will make two complementary points. On the one hand, replicability per se is not everything: we need trial outcomes that are not only statistically significant but also clinically relevant. On the other hand, trials are not everything: the experts analyzing the evidence can improve the reliability of statistical evidence, although they sometimes fail; we need to further study how they make their decisions. In section 4, we will use a controversy about the overprescription of statins to show how non-replicable effects are obtained in trials and how experts may fail at detecting such flaws, if the commercial interests are big enough. (shrink)

The author analyses the reliability and accuracy of the results of sociological research conducted using quota sampling. The author discusses the advantages and disadvantages of using quota samples compared to random samples. The main focus is on the representativeness of the obtained data and the possibility of extrapolating them to the general population.In the problem statement, the author noted that a mechanical method of forming a random sample is most often used to study public opinion in Ukraine. The (...) main feature of this method is to ensure an equal probability of selection of each element of the general population. The author emphasizes that although this method allows for easy calculation of possible errors and analysis of collected information, it has a disadvantage in representing specific groups of the population.To address this problem, the author proposes using an equal probability method of forming quota samples, which allows selecting respondents according to their quotas in the general population, thus providing a more representative model of the studied population. However, in the scientific community, there is no consensus on the use of indicators in forming quota samples, and all existing works remain at the level of theoretical assumptions.The author attempts to empirically substantiate the effectiveness of quota samples through experimentation. The hypothesis of the experiment assumes that the results of studies conducted according to quotas can be extrapolated to the general population with high accuracy and reliability using mathematical and statistical methods. During the experiment, both quota and random samples were used to compare the results.The general population consisted of 489 musical compositions represented in different proportions to eliminate the subjective component from the experiment and provide the opportunity to clearly assess and record the parameters of the initial general population for further comparison with the results of different types of samples. The musical compositions had two parameters: the name of the performer for identification and the duration of the track for digital and statistical evaluation. The analysis was carried out using a specially developed program for statistical data analysis. The collected data allowed for the assessment of the average track duration, median, skewness, standard deviation, and other statistical indicators.The results of the experiment showed that the quota sample provides more accurate results when the quotas are followed during the selection of elements of the general population. The sample obtained by random selection without adhering to quotas showed greater variability in indicators such as mean, median, skewness, and standard deviation relative to the general population. Thus, it was concluded that quota samples have advantages in sociological research, especially when specific groups of the population need to be considered.The author emphasizes the need for further research and improvement of quota sampling methods to enhance the accuracy and reliability of sociological studies, which are important for examining particular social issues. (shrink)

Conditional constructions (if–then) enable us to express our thoughts about possible states of the world, and they form an important ingredient for our reasoning and argumentative capabilities. Different types and argumentative uses have been distinguished in the literature, but their applicability to actual language use is rarely evaluated. This paper focuses on the reliability of applying classifications of connections between antecedents and consequents of conditionals to discourse, and three issues are identified. First, different accounts produce incompatible results when (...) applied to language data. Second, a discrepancy between theory and data was observed in previous studies, which sometimes discard existing classifications for being detached from actual language use. Finally, language users construct various cognitive relations between clauses of conditionals without being able to rely on overt linguistic features, which poses problems for the annotation of conditionals in argumentation and discourse. This paper addresses these issues by means of comparing theoretical types and actual uses of conditionals, by inspecting the dispersion of types in natural-language corpora, and by conducting an experiment in which the inter-rater reliability of classifications was assessed. The results show that the reliability of classifications of conditionals when applied to language data is low. With respect to the aforementioned issues, different classifications produced incompatible results, a discrepancy between theory and data was indeed observed, and low reliability scores indicated a largely interpretative nature of types of conditionals. Given these results, suggestions for the enhancement of reliability in corpus studies of conditionals and beyond are provided to enhance future classification design. (shrink)

As a result of the complexity inherent in some natural systems, mathematical models employed in ecology are often governed by a large number of variables. For instance, in the study of population dynamics we often find multiregional models for structured populations in which individuals are classified regarding their age and their spatial location. Dealing with such structured populations leads to high dimensional models. Moreover, in many instances the dynamics of the system is controlled by processes whose time scales are very (...) different from each other. For example, in multiregional models migration is often a fast process in comparison to the growth of the population.Approximate reduction techniques take advantage of the presence of different time scales in a system to introduce approximations that allow one to transform the original system into a simpler low dimensional system. In this way, the dynamics of the original system can be approximated in terms of that of the reduced system. This work deals with the study of that approximation. In particular, we work with a non-autonomous discrete time model previously presented in the literature and obtain different bounds for the error we incur when we describe the dynamics of the original system in terms of the reduced one. (shrink)

Who is a native signer? Since around 95% of deaf infants are born into a hearing family, deaf signers are exposed to a sign language at various moments of their life, and not only from birth. Moreover, the linguistic input they are exposed to is not always a fully fledged natural sign language. In this situation, is the notion of native signer as someone exposed to language from birth of any use? We review the results of the first large-scale (...) cross-linguistic investigation on the effects of age of exposure to sign language. This research involved about 45 Deaf adult signers in each of three sign languages (Catalan Sign Language, French Sign Language, and Italian Sign Language). Across the three languages, participants were divided into three groups – those exposed from birth, those between 1 and 5 years of age, and those exposed between 6 and 15 years of age – and received a battery of tests designed for each language targeting various aspects of morphosyntactic competence. In particular, the tests focused on both those morphosyntactic phenomena that are known from the spoken language literature to be good detectors of language impairment or delay (i.e.,wh-interrogatives and relative clauses) and on morphosyntactic phenomena that are sign language specific (i.e., role shift and directional verbs). The results showed a clear effect of being native, with significant differences across languages and tests between signers exposed to sign language from birth and those exposed in the 1st years of life. This confirms the life-long importance of language exposure from birth and the reliability of the notion of “nativeness”, at least for syntax. On the other hand, while in most domains the differences observed between populations might be differences in performance, for some specific constructions, signers belonging to the three groups may have different grammars. This latter finding challenges the generalized use of native signers’ grammar as the baseline for language description and language assessment. (shrink)

In people with chronic low back pain, maladaptive structural and functional changes on a cortical level have been identified. On a functional level, somatosensory cortical excitability has been shown to be reduced in chronic pain conditions, resulting in cortical disinhibition. The occurrence of structural and/or functional maladaptive cortical changes in people with CLBP could play a role in maintaining the pain. There is currently no measurement protocol for cortical excitability that employs stimulation directly to the lower back. We developed a (...) protocol for the measurement of single pulse somatosensory evoked potential waveforms and paired-pulse behavior generated from sensory nerves of the lower back and quantified its test–retest reliability in a sample of 30 healthy individuals to gain insights into the normal variability of cortical responses, which could then be compared to results from people with CLBP. We investigated cortical excitability by measuring SEPs and PPB. PPB was defined as the ratio of the amplitude of the second cortical response divided by the first cortical response. A2s was determined by subtracting the response to single-pulse stimuli from the paired pulse stimuli response to account for linear superposition effects. The test–retest reliability of the protocol was very poor with no evidence of systematic bias but a high amount of random variability between sessions. There was no significant difference in the right side PPB for session 1 and session 2 ; mean session difference [ = −0.44 ; t = −1.17, p = 0.26]. The ICC3.1 for the outlier-removed right side PPB were 0.19 and 0.43 for left side PPB. This finding potentially has wider implications for PPB protocols. If these findings were replicated in other groups and other nerves, it would question the validity of this measure more generally. However, these findings are restricted to healthy people and sensory nerves of the lower back and may not be generalizable. (shrink)

There is much discussion in the media and some of the scientific literature of how many of the conclusions from scientific research should be doubted. These critiques often focus on studies – typically in non‐experimental spheres of biomedical and social sciences – that search large datasets for novel correlations, with a risk that inappropriate statistical evaluations might yield dubious conclusions. By contrast, results from experimental biological research can often be interpreted largely without statistical analysis. Typically: novel observation(s) are reported, (...) and an explanatory hypothesis is offered; multiple labs undertake experiments to test the hypothesis; interpretation of the results may refute the hypothesis, support it or provoke its modification; the test/revise sequence is reiterated many times; and the field moves forward. I illustrate this experimental/non‐experimental dichotomy by examining the contrasting recent histories of: (a) our remarkable and growing understanding of how several inositol‐containing phospholipids contribute to the lives of eukaryote cells; and (b) the difficulty of achieving any agreed mechanistic understanding of why consuming dietary supplements of inositol is clinically beneficial in some metabolic diseases. (shrink)

Background There is increasing use of the Internet by patients as an educational tool prior to a surgical procedure. However, the quality of information is highly variable, and it is imperative that as clinicians, we are aware of the information available to ensure that consent is valid and avoid unrealistic expectations. Our aim is to assess the quality of medical information available on the Internet related to common surgical procedures. Methods Analysis of the quality of patient information on 10 websites (...) for six common surgical conditions ( n = 54 following exclusions) was assessed using the DISCERN questionnaire and, for readability, using the Flesch reading ease test. Results There was high variation in the DISCERN score from 1.34 to 4.4 (mean 2.75). Overall, poor quality of patient information is available. Consistent scoring below 55 on Flesch reading ease score suggests a reading level beyond the majority of the patient population. Conclusion The majority of patient-centred health education on the Internet is poor. Therefore, improved verbal and written information during consultation combined with local, national and international websites is important to meet patient’s expectations and avoid issues regarding consent. (shrink)

The puzzle of resultant moral luck arises when we are disposed to think that an agent who caused a harm deserves to be blamed more than an otherwise identical agent who did not. One popular perspective on resultant moral luck explains our dispositions to produce different judgments with regard to the agents who feature in these cases as a product not of what they genuinely deserve but of our epistemic situation. On this account, there is no genuine resultant moral luck; (...) there is only luck in what evidence becomes available to observers. In this paper, I develop an evolutionary account of our inclination to take the results of actions as evidence for the mental states of agents, thereby explaining why the resulting intuitions are recalcitrant to correction. The account explains why the puzzle of resultant moral luck arises: because our disposition to take the harms agents cause as evidence of their mental states can produce intuitions which conflict with those that arise when we examine agents’ mental states without reference to the results of their actions. The account also helps to solve the puzzle of resultant moral luck, by providing a strong reason to ignore the intuitions caused by our disposition to regard actual harms as evidence of mental states. Since these intuitions arise using an unreliable proxy for agents’ mental states, they ought to be trumped by more reliable evidence. (shrink)

This study aimed to conduct the cultural adaptation, validity, and reliability assessment of the Ethical Sensitivity Scale in Turkish. The 28-item, 7-factor scale, developed by Tirri and Nokelainen (2011), was translated into Turkish and tested on 591 medical students. Construct validity was examined using exploratory and confirmatory factor analyses on two independent samples to avoid overfitting. The suitability of the scale content and sample for factor analysis was confirmed by Bartlett’s test of sphericity (χ2 = 2655.489, df = 378, (...) p <.001) and a Kaiser-Meyer-Olkin value of 0.867. Items with low factor loadings were removed, resulting in a 23-item, 7-factor model with good fit indices (χ2/df = 1.773, GFI = 0.902, AGFI = 0.870, CFI = 0.941, NNFI = 0.929, RMSEA = 0.051, SRMR = 0.049). Cronbach’s alpha confirmed good internal consistency. Findings indicate that the Turkish version of Ethical Sensitivity Scale is a valid and reliable tool for assessing ethical sensitivity among medical students. (shrink)

OBJECTIVES: This reliability generalization study aimed to estimate the mean and variance of the interrater reliability coefficients (ryy) of supervisory ratings of overall, task, contextual, and positive job performance. The moderating effect of the appraisal purpose and the scale type was examined. It was hypothesized that the ratings collected for research purposes and multi-item scales have higher ryy. It was also examined whether ryy was similar for the four performance dimensions. METHOD: A database consisting of 224 independent samples (...) was created and hierarchical sub-grouping meta-analyses were conducted. RESULTS: The appraisal purpose was a moderator of ryy for the four performance dimensions. Scale type was a moderator of ryy for overall and task performance collected for research purposes. The findings also suggest that supervisors seem to have less difficulty evaluating overall job performance than task, contextual, and positive performance. The best estimates of the observed ryy for overall job performance are.61 for research-collected ratings and.45 for administrative-collected ratings. MAIN CONCLUSIONS: (1) Appraisal purpose moderates ryy and researchers and practitioners should be aware of its effects before collecting ratings or using empirically-derived interrater reliability distributions, (2) Scale type seems to moderate ryy in the case of the ratings collected for research purposes, only, (3) overall job performance is more reliably rated than task, contextual, and positive performance. Implications for research and practice are discussed. (shrink)

Datasets that ship automatically generated feature annotations invite a question that is rarely asked of them: would a human agree with those labels? This report answers that question for the Objective Projection corpus, a Turkish narrative dataset whose scenes carry a per-scene applied_rules field produced by a rule-based detector over six craft features two prohibitions (explicit emotion labelling, simile) and four positive techniques (materialized metaphor, microfocus, temporal anchor, atmosphere contradiction). -/- Three studies are reported. Study 1 (n = 120) scores (...) the detector against blind labels produced by the scheme’s own author. Study 2 (n = 100, a disjoint scene set) scores the detector plus Gemini 2.5 Flash and Grok against an independent non-expert human rater whose labels were locked before any machine ran. Study 2b re-runs the identical protocol with Claude Fable 5 (High) and ChatGPT 5.5. The central result concerns one rule. On materialized metaphor the feature closest to the methodology’s theoretical core the five machine labellers returned positive rates of 0, 1, 40, 72 and 78 out of 100 scenes, against a human count of 9. Cohen’s k was at or indistinguishable from chance for five of the six machine labellers, across both human references, on both scene sets: 0.004, 0.015, 0.000, 0.019, 0.027. Raw agreement, by contrast, ranged from 74.7% to 84.5%, an artefact of class imbalance rather than a sign of competence. -/- We deliberately do not resolve the finding into a single story. Two readings survive the data: that the feature is genuinely inferential and beyond current automatic detection, or that the 1 rule’s definition is not yet operational enough for any rater to apply consistently including the human. Distinguishing them requires a second independent human rater, which this report does not have and therefore does not claim. -/- Keywords: inter-rater reliability, Cohen’s kappa, annotation quality, class imbalance, LLM-asannotator, computational narratology, show-don’t-tell, Turkish corpus, Bulut Doctrine. (shrink)

This paper focuses on the ideals of scientif c objectivity as they emerge in discussions concerning meta-analyses and medical research. Stegenga (2011) has argued that meta-analyses fail to be objective because conducting them involves making judgments. I show that his reasoning is based on the so-called procedural ideal of objectivity, which can be questioned: this ideal is unattainable and does not capture some of the problematic issues of medical research. By introducing a case in research on selective serotonin reuptake inhibitors, (...) I demonstrate why the so-called social view on objectivity succeeds better in accommodating 1) the way in which scientif c research necessarily involves judgments, 2) the possible risks involved in research, and 3) the inf uence that the institutional context has on research activities. Adopting this ideal of objectivity helps us better appreciate the virtues of meta-analyses and pinpoint which practices threaten the reliability of meta-analyses’ results. (shrink)

Psycholinguistic theories traditionally assume similar cognitive mechanisms across different speakers. However, more recently, researchers have begun to recognize the need to consider individual differences when explaining human cognition. An increasing number of studies have investigated how individual differences influence human sentence processing. Implicitly, these studies assume that individual‐level effects can be replicated across experimental sessions and different assessment methods such as eye‐tracking and self‐paced reading. However, this assumption is challenged by the Reliability Paradox. Thus, a crucial first step for (...) a principled investigation of individual differences in sentence processing is to establish their measurement reliability, that is, the correlation of individual‐level effects across multiple measurement occasions and methods. In this work, we present the first naturalistic eye movement corpus of reading data with four experimental sessions from each participant (two eye‐tracking sessions and two self‐paced reading sessions). We deploy a two‐task Bayesian hierarchical model to assess the measurement reliability of individual differences in a range of psycholinguistic phenomena that are well‐established at the population level, namely, effects of word length, lexical frequency, surprisal, dependency length, and number of to‐be‐integrated dependents. While our results indicate high reliability across measurement occasions for the word length effect, it is only moderate for higher‐level psycholinguistic predictors such as lexical frequency, dependency distance, and the number of to‐be‐integrated dependencies, and even low for surprisal. Moreover, even after accounting for spillover effects, we observe only low to moderate reliability at the individual level across methods (eye‐tracking and self‐paced reading) for most predictors, and poor reliability for predictors of syntactic integration. These findings underscore the importance of establishing measurement reliability before drawing inferences about individual differences in sentence processing. (shrink)

Background: Few robust autism-specific outcome assessments have been developed specifically for use by teachers in special schools. The Assessment of Barriers to Learning in Education – Autism (ABLE-Autism) is a newly developed teacher assessment to identify and show progress in barriers to learning for pupils on the autism spectrum with coexisting intellectual disabilities. Aims: This study aimed to conduct a preliminary validity and reliability evaluation of the ABLE-Autism. Methods and procedures: Forty-eight autistic pupils attending special schools were assessed using (...) the ABLE-Autism. Multi-level modelling was used to evaluate test-retest reliability, internal consistency and convergent validity with the Teacher Autism Progress Scale. Outcomes and results: Results showed excellent test-retest reliability and internal consistency. A large effect size suggested that the ABLE-Autism is strongly correlated with the Teacher Autism Progress Scale. Teacher feedback was positive and suggested that the ABLE-Autism is easily understood by teachers, relevant to autistic pupils in special schools, and adequately covers the skills and behaviours that teachers believe are important to assess for these pupils. Conclusions and implications: Although further validation is recommended, the preliminary evaluation of the ABLE-Autism suggests that it is a useful and has the potential to be an effective outcome assessment for autistic pupils in special schools. (shrink)

The paper addresses the evaluation of climate models and gives an overview of epistemic uncertainties in climate modeling; the uncertainties concern the data situation as well as the causal behavior of the climate system. In order to achieve reasonable results nonetheless, multimodel ensemble studies are employed in which diverse models simulate the future climate under different emission scenarios. The models jointly deliver a robust range of climate prognoses due to a broad plurality of theories, techniques, and methods in climate (...) research; the range reliably indicates the future development of the global climate. Nevertheless, the uncertainties are widely used by skeptics to challenge the IPCC’s prognoses. Such skeptical allegations can well be distinguished from points of fruitful epistemological criticism: in spite of the enduring range of prognoses, the epistemic uncertainties should not play a role in finding agreements on climate change mitigation. (shrink)

Currently, there are no computerized tools enabling objective interpretation of observational gait assessment based on Wisconsin Gait Scale, which is a reliable and well-tested tool. The solution envisaged by us may provide a practical tool for assessing gait deviations in patients with hemiparesis after stroke. The present study assessed agreement between a new application software for computerized WGS and 3-dimensional gait analysis, and reliability of the application. The study involved 33 individuals with hemiparesis after stroke. The software was developed (...) based on a model designed taking into account components of the WGS and incorporating auxiliary lines passing through the relevant anthropometric points on the patient’s body, as well as measurements of angular values, distances and duration of the specific gait phases, which make it possible to substantiate assessment based on this scale. Series of videos were made to record gait of the qualified patients. After the gait evaluation was carried out using the app, the data were retrieved from the software. The gait assessment was performed separately by three independent examiners who reviewed the video recording using the new app twice. Additionally, 3DGA was carried out for all the subjects, and the results of the app-aided assessment were compared to those acquired using 3DGA. The findings show statistically significant correlations between majority of the WGS items measured using the new app, and the relevant spatiotemporal and kinematic parameters identified by 3DGA. Agreement between the scores reported by the three examiners was high in both measurements, as reflected by Cronbach’s alpha exceeding 0.8. The findings reflect very good intra-observer reliability and inter-observer reliability of the new application software for computerized WGS. The opportunities offered by the observational gait scale objectified through our new software for computerized WGS result from the fact that the tool provides a useful low-cost and time-effective feedback to monitor ongoing treatments or formulate hypotheses. (shrink)

The variety of evidence thesis in confirmation theory states that more varied supporting evidence confirms a hypothesis to a greater degree than less varied evidence. Under a very plausible interpretation of this thesis, positive test results from multiple independent instruments confirm a hypothesis to a greater degree than positive test results from a single instrument. We invoke Bayesian Networks to model confirmation on grounds of evidence that is obtained from less than fully reliable instruments and show that the (...) variety of evidence thesis is not sacrosanct when testing is conducted with less than fully reliable instruments: under certain conditions, a hypothesis receives more confirmation from evidence that is obtained from one rather than from more independent instruments. In the appendix, we prove certain convergence results for large numbers of positive test results from single versus multiple less than fully reliable instruments. (shrink)

In computer simulations of physical systems, the construction of models is guided, but not determined, by theory. At the same time simulations models are often constructed precisely because data are sparse. They are meant to replace experiments and observations as sources of data about the world; hence they cannot be evaluated simply by being compared to the world. So what can be the source of credibility for simulation models? I argue that the credibility of a simulation model comes not only (...) from the credentials supplied to it by the governing theory, but also from the antecedently established credentials of the model building techniques employed by the simulationists. In other words, there are certain sorts of model building techniques which are taken, in and of themselves, to be reliable. Some of these model building techniques, moreover, incorporate what are sometimes called “falsifications.” These are contrary-to-fact principles that are included in a simulation model and whose inclusion is taken to increase the reliability of the results. The example of a falsification that I consider, called artificial viscosity, is in widespread use in computational fluid dynamics. Artificial viscosity, I argue, is a principle that is successfully and reliably used across a wide domain of fluid dynamical applications, but it does not offer even an approximately “realistic” or true account of fluids. Artificial viscosity, therefore, is a counter-example to the principle that success implies truth – a principle at the foundation of scientific realism. It is an example of reliability without truth. (shrink)

Fully automated vehicles (AVs) hold promise toward providing numerous societal benefits including reducing road fatalities. However, we are uncertain about how individuals’ perceptions will influence their ability to accept and adopt AVs. The 28-item Automated Vehicle User Perception Survey (AVUPS) is a visual analog scale that was previously constructed, with established face and content validity, to assess individuals’ perceptions of AVs. In this study, we examined construct validity, via exploratory factor analysis and subsequent Mokken scale analyses. Next, internal consistency was (...) assessed via Cronbach’s alpha (α) and 2-week test–retest reliability was assessed via Spearman’s rho (ρ) and intraclass correlation coefficient (ICC). The Mokken scale analyses resulted in a refined 20-item AVUPS and three Mokken subscales assessing specific domains of adults’ perceptions of AVs: (a)Intention to use; (b)perceived barriers; and (c)well-being. The Mokken scale analysis showed that all item-coefficients of homogeneity (H) exceeded 0.3, indicating that the items reflect a single latent variable. The AVUPS indicated a strong Mokken scale (Hscale= 0.51) with excellent internal consistency (α = 0.95) and test–retest reliability (ρ = 0.76, ICC = 0.95). Similarly, the three Mokken subscales ranged from moderate to strong (rangeHscale= 0.47–0.66) and had excellent internal consistency (range α = 0.84–0.94) and test–retest reliability (range ICC = 0.84–0.93). The AVUPS and three Mokken subscales of AV acceptance were validated in a moderate sample size (N= 312) of adults living in the United States. Two-week test–retest reliability was established using a subset of Amazon Mechanical Turk participants (N= 84). The AVUPS, or any combination of the three subscales, can be used to validly and reliably assess adults’ perceptions before and after being exposed to AVs. The AVUPS can be used to quantify adults’ acceptance of fully AVs. (shrink)

Citizen science has been one of the main forms of public participation in scientific research. The inclusion of non-professional scientists in this context has, however, generated concerns about the reliability of its results, and a great part of the literature has been investigating it. In this paper, we aim to provide a more comprehensive epistemic evaluation of citizen science by discussing standards other than reliability. To do this, we refer to Alvin Goldman’s social veristic epistemology and its (...) fivestandards for epistemically evaluating social practices or institutions: reliability (the ratio of results to total number of results and errors fostered by a practice), power (the practice’s ability to help cognizers find results that answer the questions that interest them), fecundity (the practice’s ability to lead to large numbers of results for many practitioners), speed (how quickly a practice leads to results), and efficiency (how well a practice limits the cost of getting results). By applying it to the citizen science literature and by relying upon its contributions, this framework enables the integration of various theoretical reflections on citizen science, encompassing trade-offs among these standards and addressing ethical concerns. Furthermore, it also allows one to respond to common criticism about citizen science and to draw some important implications. In particular, we show how citizen science's fecundity is crucial for mitigating the challenges associated with unseen science, i.e., research whose results are never shared outside an institution boundary, thus highlighting its significance in the growing landscape of public participation in scientific research. (shrink)

Pseudo-profound bullshit refers to statements designed to impress but lacking in meaning. Due to the discrepancy between philosophical and psychological perspectives on bullshit, there is limited empirical evidence supporting its defining characteristics. This study aimed to test whether these characteristics are indeed distinctive to bullshit and influence the tendency to perceive it as profound. Additionally, recognising that context shapes bullshit receptivity, the second aim was to investigate whether bullshit features influence changes in receptivity when the reliability of an added (...) source varies. Results indicate that participants can identify both meaninglessness and intention to impress in different types of statements. Furthermore, the intention to impress elicits greater changes in profundity evaluations than the meaningfulness of the statement. Finally, both reliable and unreliable sources affect profundity ratings, with these effects observed across all statement categories. (shrink)

Citizen science has been one of the main forms of public participation in scientific research. The inclusion of non-professional scientists in this context has, however, generated concerns about the reliability of its results, and a great part of the literature has been investigating it. In this paper, we aim to provide a more comprehensive epistemic evaluation of citizen science by discussing standards other than reliability. To do this, we refer to Alvin Goldman’s social veristic epistemology and its (...) fivestandards for epistemically evaluating social practices or institutions: reliability (the ratio of results to total number of results and errors fostered by a practice), power (the practice’s ability to help cognizers find results that answer the questions that interest them), fecundity (the practice’s ability to lead to large numbers of results for many practitioners), speed (how quickly a practice leads to results), and efficiency (how well a practice limits the cost of getting results). By applying it to the citizen science literature and by relying upon its contributions, this framework enables the integration of various theoretical reflections on citizen science, encompassing trade-offs among these standards and addressing ethical concerns. Furthermore, it also allows one to respond to common criticism about citizen science and to draw some important implications. In particular, we show how citizen science's fecundity is crucial for mitigating the challenges associated with unseen science, i.e., research whose results are never shared outside an institution boundary, thus highlighting its significance in the growing landscape of public participation in scientific research. (shrink)

A measure of coherence is said to be truth conducive if and only if a higher degree of coherence (as measured) results in a higher likelihood of truth. Recent impossibility results strongly indicate that there are no (non-trivial) probabilistic coherence measures that are truth conducive. Indeed, this holds even if truth conduciveness is understood in a weak ceteris paribus sense (Bovens & Hartmann, 2003, Bayesian epistemology. New York, Oxford: Oxford University Press; Olsson, 2005, Against coherence: Truth probability and (...) justification. Oxford: Oxford University Press). This raises the problem of how coherence could nonetheless be an epistemically important property. Our proposal is that coherence may be linked in a certain way to reliability. We define a measure of coherence to be reliability conducive if and only if a higher degree of coherence (as measured) results in a higher probability that the information sources are reliable. Restricting ourselves to the most basic case, we investigate which coherence measures in the literature are reliability conducive. It turns out that, while a number of measures fail to be reliability conducive, except possibly in a trivial and uninteresting sense, Shogenji’s measure and several measures generated by Douven and Meijs’s recipe are notable exceptions to this rule. (shrink)

Background: An association between gaming disorder and the symptoms of common mental disorders is unraveled yet. In this preregistered study, we quantitatively synthesized reliability, convergent and discriminant validity of GD scales to examine association between GD and other constructs.Methods: Five representative GD instruments were chosen based on recommendations by the previous systematic review study to conduct correlation meta-analyses and reliability generalization. A systematic literature search was conducted through Pubmed, Proquest, Embase, and Google Scholar to identify studies that reported (...) information on either reliability or correlation with related variables. 2,124 studies were full-text assessed as of October 2020, and 184 were quantitatively synthesized. Conventional Hedges two-level meta-analytic method was utilized.Results: The result of reliability generalization reported a mean coefficient alpha of 0.86 and a mean test-retest estimate of 0.86. Estimated effect sizes of correlation between GD and the variables were as follows: 0.33 with depression, 0.29 with anxiety, 0.30 with aggression, –0.22 with quality of life, 0.29 with loneliness, 0.56 with internet addiction, and 0.40 with game playtime, respectively. The result of moderator analyses, funnel and forest plots, and publication bias analyses were also presented.Discussion and Conclusion: All five GD instruments have good internal consistency and test-retest reliability. Relatively few studies reported the test-retest reliability. The result of correlation meta-analysis revealed that GD scores were only moderately associated with game playtime. Common psychological problems such as depression and anxiety were found to have a slightly smaller association with GD than the gaming behavior. GD scores were strongly correlated with internet addiction. Further studies should adopt a rigorous methodological procedure to unravel the bidirectional relationship between GD and other psychopathologies.Limitations: The current study did not include gray literature. The representativeness of the five tools included in the current study could be questioned. High heterogeneity is another limitation of the study.Systematic Review Registration: [https://www.crd.york.ac.uk/PROSPERO/], identifier [CRD42020219781]. (shrink)

This research aims to evaluate the calculation accuracy and efficiency of the artificial neural network-based important sampling method on reliability of structures such as drum brakes. The finite element analysis result is used to establish the ANN sample in ANN-based reliability analysis methods. Because the process of FEA is time-consuming, the ANN sample size has a very important influence on the calculation efficiency. Two types of ANNs used in this study are the radial basis function neural network and (...) back propagation neural network. RBF-IS and BP-IS methods are used to conduct reliability analysis on training samples of three different sizes, and the results are compared with several reliability analysis methods based on ANNs. The results show that the probability of failure of the RBF-IS method is closer to that of the Monte-Carlo simulation method than those of other methods. In addition, the RBF-IS method has better calculation efficiency than the other methods considered in this study. This research demonstrates that the RBF-IS method is well suited to structure reliability problems. (shrink)

The inductive reliability of Bayesian methods is explored. The first result presented shows that for any solvable inductive problem of a general type, there exists a subjective prior which yields a Bayesian inductive method that solves the problem, although not all subjective priors give rise to a successful inductive method for the problem. The second result shows that the same does not hold for computationally bounded agents, so that Bayesianism is "inductively incomplete" for such agents. Finally a consistency proof (...) shows that inductive agents do not need to disregard inductive failure on sets of subjective probability 0 in order to be ideally rational. Together the results reveal the inadequacy of the subjective Bayesian norms for scientific methodology. (shrink)

The reliability of clinical practice guidelines has been disputed because guideline panel members are often burdened with financial conflicts of interest. Current recommendations for COI regulation advise not only detailed declaration but also active management of conflicts. To continuously assess COI declaration and management in German guidelines we established the public database LeitlinienWatch. We analyzed all German guidelines at the highest methodological level that included recommendations for pharmacological therapy according to five criteria: declaration and assessment of COI, composition of (...) the guideline development group, independence of the coordinators and lead authors, imposed abstentions because of COI and public external review. Each criterion was assessed using predefined outcome categories. Most guidelines contained a detailed declaration of COI. However, none of the guidelines provided full transparency of COI assessment results. The guideline group was composed of a majority of participants with COI in 55% of the guidelines, no guideline was free of participants with COI. Only 9% of guidelines had coordinators and lead authors without any financial COI. Most guidelines did not provide a rule for abstentions for participants with COI. In 21% of guidelines there was a rule, but abstentions were either not practiced or not documented, whereas in 7% partial abstentions and in 2% complete abstentions were documented. Two thirds of the guideline drafts were not externally reviewed via a public website. COI are usually documented in detail in German guidelines of the highest methodological level. However, considerable improvement is needed regarding active management of COI, including recruitment of independent experts for guideline projects, abstention from voting for participants with COI and external review of the guideline draft. We assume that the publicly available ratings on GuidelineWatch will improve the handling of conflicts of interest in guideline development. (shrink)

Information generally comes from less than fully reliable sources. Rationality, it seems, requires that one take source reliability into account when reasoning on the basis of such information. Recently, Bovens and Hartmann (2003) proposed an account of the conjunction fallacy based on this idea. They show that, when statements in conjunction fallacy scenarios are perceived as coming from such sources, probability theory prescribes that the “fallacy” be committed in certain situations. Here, the empirical validity of their model was assessed. (...) The model predicts that statements added to standard conjunction problems will change the incidence of the fallacy. It also predicts that statements from reliable sources should yield an increase in fallacy rates (relative to unreliable sources). Neither the former (Experiment 1) nor the latter prediction (Experiment 3) was confirmed, although Experiment 2 showed that people can derive source reliability estimates from the likelihood of statements in a manner consistent with the tested model. In line with the experimental results, model fits and sensitivity analyses also provided very little evidence in favor of the model. This suggests that Bovens and Hartmann’s present model fails to explain fully people’s judgements in standard conjunction fallacy tasks. (shrink)

In the literature, the question regarding how to axiomatize the transitive logic of false belief is thought of as hard and left as an open problem. In this paper, among other contributions, we deal with this problem. In more detail, although the standard doxastic operator is undefinable with the operator of false belief, the former is almost definable with the latter. On one hand, the involved almost definability schema guides us to find the desired core axioms for the transitive logic (...) and the Euclidean logic of false belief. On the other hand, inspired by the schema and other considerations, we propose a suitable canonical relation, which can uniformly handle the completeness proof of various logics of false belief, including the transitive logic. We also extend the results to the logic of reliable belief, due to the interdefinability of the operators of false belief and reliable belief. (shrink)

Possible measures to mitigate climate change require global collective actions whose impacts will be felt by many, if not all. Implementing such actions requires successful communication of the reasons for them, and hence the underlying climate science, to a degree that far exceeds typical scientific issues which do not require large-scale societal response. Empirical studies have identified factors, such as the perceived level of consensus in scientific opinion and the perceived reliability of scientists, that can limit people's trust in (...) science communicators and their subsequent acceptance of climate change claims. Little consideration has been given, however, to recent formal results within philosophy concerning the relationship between truth, the reliability of evidence sources, the coherence of multiple pieces of evidence/testimonies, and the impact of independence between sources of evidence. This study draws on these results to evaluate exactly what has been established in the empirical literature about the factors that bias the public's reception of scientific communications about climate change. (shrink)

The coherentist theory of justification provides a response to the sceptical challenge: even though the independent processes by which we gather information about the world may be of dubious quality, the internal coherence of the information provides the justification for our empirical beliefs. This central canon of the coherence theory of justification is tested within the framework of Bayesian networks, which is a theory of probabilistic reasoning in artificial intelligence. We interpret the independence of the information gathering processes (IGPs) in (...) terms of conditional independences, construct a minimal sufficient condition for a coherence ranking of information sets and assess whether the confidence boost that results from receiving information through independent IGPs is indeed a positive function of the coherence of the information set. There are multiple interpretations of what constitute IGPs of dubious quality. Do we know our IGPs to be no better than randomization processes? Or, do we know them to be better than randomization processes but not quite fully reliable, and if so, what is the nature of this lack of full reliability? Or, do we not know whether they are fully reliable or not? Within the latter interpretation, does learning something about the quality of some IGPs teach us anything about the quality of the other IGPs? The Bayesian-network models demonstrate that the success of the coherentist canon is contingent on what interpretation one endorses of the claim that our IGPs are of dubious quality. (shrink)

According to a simple version of the reliability theory of epistemic justification, a belief is justified if and only if the process leading to that belief is reliable. The idea behind this theory is simple and attractive. There are a variety of mental or cognitive processes that result in beliefs. Some of these processes are reliable—they generally yield true beliefs—and the beliefs they produce are justified. Other processes are unreliable and the beliefs they produce are unjustified. So, for example, (...) reliable processes such as perception and memory yield justified beliefs while an unreliable process such as wishful thinking—believing something as a result of wishing that it is true—yields unjustified beliefs. (shrink)

Background. The events of 9/11 and the October 2002 National Intelligence Estimate on Iraq's Continuing Programs for Weapons of Mass Destruction precipitated fundamental changes within the US Intelligence Community. As part of the reform, analytic tradecraft standards were revised and codified into a policy document – Intelligence Community Directive (ICD) 203 – and an analytic ombudsman was appointed in the newly created Office for the Director of National Intelligence to ensure compliance across the intelligence community. In this paper we investigate (...) the untested assumption that the ICD203 criteria can facilitate reliable evaluations of analytic products. Method. Fifteen independent raters used a rubric based on the ICD203 criteria to assess the quality of reasoning of 64 analytical reports generated in response to hypothetical intelligence problems. We calculated the intra-class correlation coefficients for single and group-aggregated assessments. Results. Despite general training and rater calibration, the reliability of individual assessments was poor. However, aggregate ratings showed good to excellent reliability. Conclusions. Given that real problems will be more difficult and complex than our hypothetical case studies, we advise that groups of at least three raters are required to obtain reliable quality control procedures for intelligence products. Our study sets limits on assessment reliability and provides a basis for further evaluation of the predictive validity of intelligence reports generated in compliance with the tradecraft standards. (shrink)