As AI systems advance in capabilities, measuring their safety and alignment to human values is becoming paramount. A fast-growing field of AI research is devoted to developing such assessments. However, most current advances therein may be ill-suited for assessing AI systems across real-world deployments. Standard methods prompt large language models (LLMs) in a questionnaire-style to describe their values or behavior in hypothetical scenarios. By focusing on unaugmented LLMs, they fall short of evaluating AI agents, which could actually perform relevant (...) behaviors, hence posing much greater risks. LLMs' engagement with scenarios described by questionnaire-style prompts differs starkly from that of agents based on the same LLMs, as reflected in divergences in the inputs, possible actions, environmental interactions, and internal processing. As such, LLMs' responses to scenario descriptions are unlikely to be representative of the corresponding LLM agents' behavior. We further contend that such assessments make strong assumptions concerning the ability and tendency of LLMs to report accurately about their counterfactual behavior. This makes them inadequate to assess risks from AI systems in real-world contexts as they lack construct validity. We then argue that a structurally identical issue holds for current AI alignment approaches. Lastly, we discuss improving safety assessments and alignment training by taking these shortcomings to heart. (shrink)

When experimental participants are given the chance to reflect and revise their initial judgments in a dynamic conversational context, do their responses to philosophical scenarios differ from responses to those same scenarios presented in a traditional static survey? In three experiments comparing responses given in conversational contexts with responses to traditional static surveys, we find no consistent evidence that responses differ in these different formats. This aligns with recent findings that various manipulations of reflectiveness have no effect on participants’ judgments (...) about philosophical scenarios. Although we did not find a consistent quantitative effect of format (conversation vs. static survey), conversational experiments still provide qualitative insights into debates about how participants are understanding (or misunderstanding) the scenarios they read in experimental studies, whether they are replacing difficult questions with questions that are more easily answered, and how participants are imagining the scenarios they read in ways that differ from what is explicitly stated by experimenters. We argue that conversational experiments—"Socratic questionnaires"—help show what is going on “under the hood” of traditional survey designs in the experimental investigation of philosophical questions. (shrink)

Aim: This research aims to analyze and compare the perspectives of high school teachers and students regarding the physical education class, as well as the impact of the introduction of non-specific equipment on their interest and involvement. Methods: The responses were automatically centralized by Google Forms, then exported and processed using Microsoft Excel. The questionnaire was completed between May 15 and June 15, 2022 by 143 teachers and 1357 high school students in Romania. The questionnaire included 4 demographic (...) questions and 14 questions covering the topic studied. The questions addressed to teachers and students focus on the perception of physical education classes and their perspective on the implementation of new equipment during the physical education lesson. Results: The results of the questionnaire indicate a significant interest on the part of both teachers and students in the new methods and means introduced in physical education classes. According to the data collected, 65% of the interviewed teachers believe that the introduction of modern fitness equipment increases students' interest and involvement during classes. Also, 43.7% of students appreciate that the use of new equipment considerably amplifies their interest in physical education classes. Conclusions: The analysis of the answers obtained from physical education teachers and high school students reveals the consensus on the importance of physical education classes in the harmonious development of students. Both teachers and students believe that one hour a week is insufficient to meet this need. In order to achieve the objectives of the physical education class, it is essential to allocate additional time dedicated to the development of the motor qualities necessary to meet the standards specific to high school age. In addition, the introduction of modern and non-specific equipment during physical education classes can significantly increase the involvement and attractiveness of these activities, thus contributing to a more enjoyable and effective educational experience for students. (shrink)

In interwar Vienna, Paul F. Lazarsfeld and his colleagues developed an approach to survey research that used the questionnaire and the direct, face-to-face interview to gather data about subjective experience for aggregative analysis. For these young researchers, the questionnaire-based interview emerged from a contradictory set of Central European intellectual traditions and political concerns. Enthusiasm on the political left for quantification and the gathering of social data encouraged survey research; yet, local political allies and intellectual mentors also opposed the (...) study of individual attitudes and the quantitative aggregation of such material. Academic psychology legitimized the use of “introspection” and facilitated the extension of this method to populations of untrained subjects. The methodological concept of the “model” helped overcome the Verstehen/Erklären dichotomy within debates over the proper methods of the human and social sciences. This article examines methodological and philosophical statements, study designs, and questionnaires to explain how the interview gained particular importance within this setting. (shrink)

This paper reports the results of questionnaire-based research conducted at an exhibition of interactive humanoid robots that was held at the Osaka Science Museum, Japan. The aim of this exhibition was to investigate the feasibility of communication robots connected to a ubiquitous sensor network, under the assumption that these robots will be practically used in daily life in the not-so-distant future. More than 90,000 people visited the exhibition. A questionnaire was given to the visitors to explore their opinions (...) of the robots. Statistical analysis was done on the data of 2,301 respondents. It was found that the visitors’ opinions varied according to age; younger visitors did not necessarily like the robots more than elderly visitors; positive evaluation of the robots did not necessarily conflict with negative evaluations such as anxiety; there was no gender difference; and there was almost no correlation between opinions and the length of time spent near the robots. (shrink)

Background: Knowledge about assent or dissent of children to non-therapeutic research is poor.Objectives: To assess sociodemographic characteristics in healthy children and adolescents who were invited to participate in non-therapeutic research, to evaluate their motives for assent or dissent and their understanding of the information given.Methods: A total of 1281 healthy children and adolescents six to sixteen years of age were invited to participate in a non-therapeutic study and a questionnaire.Results: Assenting children were motivated by a desire to help sick (...) children and to gain experience with participating in a research study . Dissenting children made their decision because of worries about having a blood or a urine sample taken or because of worries about a doctor’s examination . Fewer children in the assent group than in the dissent group worried about the doctor’s examination . In the assent and dissent group, 568 and 343 children, respectively, said they were able to understand some or all of the written information , and 650 and 330 , respectively, were able to understand some or all of the verbal information .Conclusions: Sociodemographic characteristics may not influence healthy children’s decision to volunteer for non-therapeutic research. Assenting children have altruistic and educational motives, whereas worries about procedures may cause children to dissent. A great majority of school children and adolescents feel capable of understanding and giving assent or dissent to non-therapeutic research. (shrink)

In the late 19th century, the questionnaire was one means of taking the case study into the multitudes. This article engages with Forrester’s idea of thinking in cases as a means of interrogating questionnaire-based research in early American psychology. Questionnaire research was explicitly framed by psychologists as a practice involving both natural historical and statistical forms of scientific reasoning. At the same time, questionnaire projects failed to successfully enact the latter aspiration in terms of synthesizing masses (...) of collected data into a coherent whole. Difficulties in managing the scores of descriptive information questionnaires generated ensured the continuing presence of individuals in the results of this research, as the individual case was excerpted and discussed alongside a cast of others. As a consequence, questionnaire research embodied an amalgam of case, natural historical, and statistical thinking. Ultimately, large-scale data collection undertaken with questionnaires failed in its aim to construct composite exemplars or ‘types’ of particular kinds of individuals; to produce the singular from the multitudes. (shrink)

ObjectiveRelative deprivation is associated with collective and individual variables in psychology. However, so far, there are few studies on measuring the relative deprivation of university students. Therefore, this study designs the University Students’ Relative Deprivation Questionnaire, verifies its validity and reliability, and then uses it to measure the mental health of students.MethodsAfter reviewing the relevant studies and conducting a theoretical analysis and an open questionnaire survey, this article determined the structural dimension of USRDQ. A total of 103 university (...) students were selected to take the open questionnaire survey, 200 were selected to engage in the item analysis, exploratory factor analysis, and internal consistency reliability test, 257 were selected to engage in the confirmatory factor analysis, and 287 were selected to take the retest reliability.ResultsThe USRDQ includes 19 items under the three dimensions, namely, social comparison, cognitive evaluation, and emotional experience. Factor loads range from 0.49 to 0.87, which accounted for 63.39% of the total variation. The questionnaire has good fitting indicators. The Cronbach’s α coefficient of the questionnaire is 0.916, and the coefficients of the three factors range from 0.805 to 0.934. The results of the survey show that the relative deprivation of students is quite high with a mean of 76.78 and a standard deviation of 16.96. (shrink)

-/- Policy regulations of ethically controversial genetic technologies should, on the one hand, be based on ethical principles. On the other hand, they should be socially acceptable to ensure implementation. In addition, they should align with ethical theory. Yet to date we lack a reliable and valid scale to measure the relevant ethical judgements in laypeople. We target this lacuna. -/- We developed a scale based on ethical principles to elicit lay judgments: the Genetic Technologies Questionnaire (GTQ). In two (...) pilot studies and a pre-registered main study, we validated the scale in a representative sample of the US population. -/- The final version of the scale contains 20 items but remains highly reliable even when reduced to five. It also predicts behaviour; for example, ethical judgments as measured by the GTQ predicted hypothetical donations and grocery shopping. In addition, the GTQ may be of interest to policymakers and ethicists because it reveals coherent and ethically justified judgments in laypeople. For instance, the GTQ indicates that ethical judgments are sensitive to possible benefits and harms (in line with utilitarian ethics), but also to ethical principles such as the value of consent-autonomy. -/- In conclusion, the GTQ can be recommended for research in both experimental psychology and applied ethics, as well as a tool for ethically and empirically informed policymaking. (shrink)

The article presents the Identity Dynamics Questionnaire (IDQ) as a method for measuring identity processes and modes included in the identity dynamics model by Oleś. The model refers to general processes of personality development that underlie identity functioning located on two dimensions: the internal experience and the external activity. The new tool was created to validly and reliably operationalize the theoretical proposition and to verify its assumptions and relations between components of the model. Four studies were conducted as successive (...) stages of constructing the IDQ. Studies 1 and 2 aimed to exploratory setting the content of subscales designed to measure four identity processes (integration, reconstruction, adaptation, expansion – part A of the IDQ) and four identity modes (stabilization, fluidity, amplification, pliability – part B of the IDQ). Studies 3 and 4 served to confirmatory analyses concerning the structure of the IDQ and show its validity. The studies showed satisfactory psychometric properties of current version of the new method. (shrink)

Research on the etiology of adult psychopathology and its relationship with childhood trauma has focused primarily on specific forms of maltreatment. This study developed an instrument for the assessment of childhood and adolescence trauma that would aid in identifying the role of co-occurring childhood stressors and chronic adverse conditions. The Complex Trauma Questionnaire (ComplexTQ), in both clinician and self-report versions, is a measure for the assessment of multi-type maltreatment: physical, psychological, and sexual abuse; physical and emotional neglect as well (...) as other traumatic experiences, such rejection, role reversal, witnessing domestic violence, separations, and losses. The four-point Likert scale allows to specifically indicate with which caregiver the traumatic experience has occurred. A total of 229 participants, a sample of 79 nonclinical and that of 150 high-risk and clinical participants, were assessed with the ComplexTQ clinician version applied to Adult Attachment Interview (AAI) transcripts. Initial analyses indicate acceptable inter-rater reliability. A good fit to a 6-factor model regarding the experience with the mother and to a 5-factor model with the experience with the father was obtained; the internal consistency of factors derived was good. Convergent validity was provided with the AAI scales. ComplexTQ factors discriminated normative from high-risk and clinical samples. The findings suggest a promising, reliable, and valid measurement of early relational trauma that is reported; furthermore, it is easy to complete and is useful for both research and clinical practice. (shrink)

Hodroj, Latham, and Miller’s target article is the latest in a slew of recent studies that seek to investigate temporal experience using a method that involves participants reading vignettes then completing questionnaires. I have significant misgivings about this method, and in this commentary, I discuss its limitations. I start by discussing the kinds of proposals that can, and cannot, be tested in principle using the questionnaire method, suggesting that some prominent proposals fall outside its scope. I illustrate this through (...) an analogy with the opponent process theory of colour experience, where I suggest that the tools of cognitive neuroscience are needed. I then draw attention to some of the pitfalls of the questionnaire method in general, and to some problems specific to the current study. One major issue concerns the question of whether it has been shown that participants really understood their task. I give reasons for doubting this. First, I show that it is too easy to answer the comprehension questions correctly without understanding. Second, I argue that it is not credible that participants actually held the world view entailed by the combination of responses given by a significant minority of them, suggesting that there was significant confusion. I raise several other objections before concluding with a modest suggestion for future work. (shrink)

There is currently a dearth of a culturally sensitive instrument for measuring job satisfaction among university lecturers in Africa, specifically Nigeria. To bridge this gap, we developed and psychometrically tested the Lecturers' Job Satisfaction Questionnaire (LJSQ) in two Nigerian universities. Content validity evidence was gathered through experts' ratings, with a quantitative approach followed in computing content validity indices at the item and scale level. The researchers refined and tested the Lecturers’ Job Satisfaction Questionnaire (LJSQ) through a systematic process. (...) A focus group discussion with 10 university lecturers guided its initial development, followed by pilot testing with 3,122 lecturers divided into three subsamples: 262 for Exploratory Factor Analysis (EFA), 1,300 for Confirmatory Factor Analysis (CFA), and 1,560 for bifactor modelling. Participants were allocated systematically across the subsamples to ensure a balanced distribution. Results of data analysis revealed a five-factor structure encompassing satisfaction with work conditions, workload allocation, remuneration, career advancement opportunity, and research support. The bifactor model demonstrated the best fit, among four competing models supported by reliability measures such as Cronbach’s alpha, Omega coefficients, and split-half reliability, alongside convergent and discriminant validity, confirming its effectiveness in assessing job satisfaction. Conclusively, the applicability of the LJSQ transcends disciplinary and institutional boundaries, providing a foundation for future cross-cultural validation and longitudinal studies. Keywords:. (shrink)

‘Scientific dishonesty’ implies the fabrication, falsification or plagiarism in proposing, performing or reviewing research or in reporting research results. A questionnaire was given to postgraduate students at the medical faculties in Sweden who attended a course in research ethics during the academic year 2008/2009 and 58% answered (range 29%–100%). Less than one-third of the respondents wrote that they had heard about scientific dishonesty in the previous 12 months. Pressure, concerning in what order the author should be mentioned, was reported (...) by about 1 in 10 students. We suggest that all departments conducting research should have a written policy about acceptable research behaviour and that all doctoral students should be informed of the content of this policy. Participants in the research groups concerned should also be required to analyse published articles about scientific dishonesty and critically discuss what could be done about unethical conduct. (shrink)

This study examined the relationship between unethical employee behavior and the dimensions of the Ethical Climate Questionnaire (ECQ). In order to explore the relationship between the dimensions of the ECQ and unethical behavior, the factor structure of five previously identified empirical models and the hypothesized nine-dimension model for the ECQ was tested with a confirmatory factor analysis. The analysis revealed that the hypothesized nine-dimension model provided as good or even better fit to the data than the five empirically derived (...) models. Therefore, the nine-dimensional model was used to examine the criterion-related validity of the ECQ. The results demonstrated that the nine ethical climate dimensions were correlated with some of the unethical behaviors examined in this study, but not others. However, the results clearly demonstrated that most of the ethical climate dimensions were significantly related to an aggregate measure of unethical behavior. It was suggested that these results might account for the differences in previous studies on the criterion-related validity of the ECQ. The results also replicated a previous report that the association between unethical behavior and ethical climate is stronger in organizations that do not have a code of ethics. Finally, a difference was observed in the ethical climates for organizations with a code of ethics and organizations without a code of ethics. (shrink)

The primary objective of the research was to validate the Emotional Leadership Questionnaire for Training, as proposed by McKee et al. (2008). The study included a sample of 97 school principals. An analysis of the original questionnaire was conducted, which comprises four factors and 25 items. Of these, three items and 13 factors were retained. The results demonstrated an optimal fit of the model and high-reliability indices. This instrument provides a practical self-perception tool to assess emotional leadership, underscoring (...) the importance of empathy, adaptability, and self-knowledge in the training of school principals. The content validity was established through the assessment of 15 experts using Lawshe's formula, which demonstrated the validity of all items and dimensions. The reliability was evaluated through Cronbach's alpha and Omega indices, which yielded values between 0.77 and 0.86 and between 0.79 and 0.87, respectively. These results indicate high reliability for all factors. Construct validity was confirmed through confirmatory factor analysis. (shrink)

School principals world-wide report high levels of strain and attrition resulting in a shortage of qualified principals. It is thus, crucial to identify psychosocial risk factors that reflect principals’ occupational wellbeing. For this purpose, we used the Copenhagen Psychosocial Questionnaire (COPSOQ-II), a widely used self-report measure covering multiple psychosocial factors identified by leading occupational stress theories. We evaluated the COPSOQ-II regarding factor structure and longitudinal, discriminant, and convergent validity using latent structural equation modeling in a large sample of Australian (...) school principals (N = 2,049). Results reveal that confirmatory factor analysis produced marginally acceptable model fit. A novel approach we call set exploratory structural equation modeling (ESEM-set), where cross-loadings were only allowed within a priori defined sets of factors, fit well, and was more parsimonious than a full ESEM. Further multitrait-multimethod models based on the set-ESEM confirm the importance of a principal’s psychosocial risk factors; Stressors and depression were related to demands and ill-being, while confidence and autonomy were related to wellbeing. We also show that working in the private sector was beneficial for showing a low psychosocial risk, while other demographics have little effects. Finally, we identify five latent risk profiles (high risk to no risk) of school principals based on all psychosocial factors. Overall the research presented here closes the theory application gap of a strong multi-dimensional measure of psychosocial risk-factors. (shrink)

Urban spaces require increasing their attractiveness by exploring the social and spatial implications of new lifestyles. Broad civic knowledge is the basis for placemaking to shape user-centred and inclusive spaces of everyday life. Gathering information on the sense of the place is crucial to finding out and understanding the place-related identity of its users to make the place more appealing and usable. The most popular survey tool is a questionnaire constructed as a series of questions and used for collecting (...) information about a population’s attributes, attitudes or actions. Administered in person or online, questionnaire surveys may provide broad coverage of urban communities – however, they require significantly large human resources to carry out. The survey based on personal contact with the respondent may limit time allocated to its implementation, thus affecting the sample selection. Also, the researcher’s involvement in the questionnaire-filling process can be problematic if he or she reveals the topic of interest and if the sequencing of questionnaire themes influences responses. Similarly, an online questionnaire may not reach some potential respondents due to their reduced access to the internet etc. These limitations may affect the results’ reliability. Taking into account the above-mentioned aspects, the idea of this chapter is to present selected types of questionnaires (textual, visual, in-VR and survey gamification). The main aim is to discuss these questionnaires in the context of their main advantages and disadvantages resulting from the specifics of a type and a form of survey (in-depth and online) to facilitate their use in the process of data collection in placemaking research. The results show that the variety of types of questionnaires, as well as the form of the conducted survey, allow their better selection to the specificity of the community, increasing the place-related identity, thus may provide a more reliable and complete set of information possible to be used in placemaking. (shrink)

I started out as a student of physics, hard-working, interested, but alas, not ‘in love’ with my subject. Then logic struck, and having become interested in this subject for various reasons – including the fascinating personality of my first teacher –, I switched after my candidate’s program, to take two master’s degrees, in mathematics and in philosophy. The beauty of mathematics was clear to me at once, with the amazing power, surprising twists, and indeed the music, of abstract arguments. As (...) our professor of Analysis wrote at the time in our study guide “Mathematics is about the delight in the purity of trains of thought”, and oldfashioned though this phrasing sounded in the revolutionary 1960s, it did resonate with me. Then I had the privilege of being taught set-theoretic topology by a group of brilliant students around De Groot, our leading expert around the time, who worked with Moore’s method of discovering a subject for oneself. Topology unfolded from a few definitions and examples to real theorems that we had to prove ourselves – and the take-home exam took sleepless nights, as it included proving some results from scratch which came from a recent dissertation (as it turned out later). Only at the very end did De Groot appear, to give one lecture on Tychonoff’s Theorem where an application was made of the Axiom of Choice, a sacral act only to be performed by tenured full professors. (shrink)

(1985). Development of a Questionnaire to Measure Secondary School Pupils’ Attitudes to Computers and Robots. Educational Studies: Vol. 11, No. 1, pp. 33-40.

The sexological research questionnaire, which became a central research tool in twentieth-century sexology, has a methodological-developmental history stretching back into mid-nineteenth century Germany. It was the product of a prolonged, disruptive encounter between sexual scientists constructing sexual case studies along with newly assertive homosexual men supplying self-penned sexual autobiographies. Homosexual autobiographies were intensely interesting to these men of science but lacked the brevity, structure, and discipline of a formal clinical case study. In the closing decades of the century, efforts (...) to harness and regularize this self-penned material resulted in a series of methodological adaptations. By the turn of the century this process had resulted in the first use of a formal sexual research questionnaire. (shrink)