People can discriminate the synchrony between audio-visual scenes. However, the sensitivity of audio-visual synchrony perception can be affected by many factors. Using a simultaneity judgment task, the present study investigated whether the synchrony perception of complex audio-visual stimuli was affected by audio-visual causality and stimulus reliability. In Experiment 1, the results showed that audio-visual causality could increase one's sensitivity to audio-visual onset asynchrony (AVOA) of both action stimuli and speech stimuli. Moreover, participants were more (...) tolerant of AVOA of speech stimuli than that of action stimuli in the high causality condition, whereas no significant difference between these two kinds of stimuli was found in the low causality condition. In Experiment 2, the speech stimuli were manipulated with either high or low stimulus reliability. The results revealed a significant interaction between audio-visual causality and stimulus reliability. Under the low causality condition, the percentage of “synchronous” responses of audio-visual intact stimuli was significantly higher than that of visual_intact/auditory_blurred stimuli and audio-visual blurred stimuli. In contrast, no significant difference among all levels of stimulus reliability was observed under the high causality condition. Our study supported the synergistic effect of top-down processing and bottom-up processing in audio-visual synchrony perception. (shrink)

Audio description remains the cornerstone of accessibility for visually impaired audiences to all sorts of audiovisual content, including porn. Existing work points to the efficacy of audio description to guarantee immersion and emotional engagement, but evidence on its role in sexual arousal and engagement in porn is still scant. The present study takes on this challenge by comparing sighted and visually impaired participants’ experiences with porn in terms of their physiological response [i.e., cortisol and heart rate ] and (...) self-report measures of affect [Positive and Negative Affect Schedule ; Watson et al., 1988], anxiety [State-Trait Anxiety Inventory ; Spielberger et al., 1970], sexual reactivity and arousal [Sexual Inhibition/Sexual Excitation Scale ; Moyano and Sierra ; and the Ratings of Sexual Arousal ; Mosher ], and narrative engagement or transportation [The Transport Narrative Questionnaire, Green and Brock ]. 69 Spanish participants were allocated into three different groups: 25 sighted participants who watched and heard the porn scenes in their audio-visual version ; 22 sighted participants who listened to the audio described version without images ; and 22 visually impaired participants who also listened to the audio described version without images. Overall, results on physiological and self-report measures revealed no significant differences between groups or different versions of the clips. The analysis of cortisol reactivity to porn as the maximum increase or decrease in cortisol with respect to baseline values revealed no significant differences between the groups, but pointed to a higher percentage of non-responders than responders in the three groups, the highest being found in the ONCE group. As for participants’ cardiac response to the clips, no significant differences were found across the groups, with the highest HR levels being registered in the baseline phase. Self-report measures revealed significant between-group differences in negative affect. The ONCE group displayed the highest pre-task levels of negative affect and was the only group that showed a decrease in negative affect after exposure to the clips. Sighted and visually impaired participants reported to be moderately aroused and immersed in the films, regardless of exposure to AV or AD porn. In addition, correlations found between participants’ levels of self-report sexual arousal and transportation and post-task affect pointed to a positive relationship between exposure to porn and perceived levels of sexual arousal and affect. Results from the study reflected the efficacy of audio description in providing sighted and visually impaired audiences with a similar experience to that offered by original AV porn scenes. This study is exploratory but provides valid, initial groundwork for further research on the impact of audio description on porn reception. (shrink)

We examined whether facial expressions of performers influence the emotional connotations of sung materials, and whether attention is implicated in audio-visual integration of affective cues. In Experiment 1, participants judged the emotional valence of audio-visual presentations of sung intervals. Performances were edited such that auditory and visual information conveyed congruent or incongruent affective connotations. In the single-task condition, participants judged the emotional connotation of sung intervals. In the dual-task condition, participants judged the emotional connotation of intervals while performing (...) a secondary task. Judgements were influenced by melodic cues and facial expressions and the effects were undiminished by the secondary task. Experiment 2 involved identical conditions but participants were instructed to base judgements on auditory information alone. Again, facial expressions influenced judgements and the effect was undiminished by the secondary task. The results suggest that visual aspects of music performance are automatically and preattentively registered and integrated with auditory cues. (shrink)

In lieu of an abstract, here is a brief excerpt of the content:Reading Audio BooksWilliam IrwinI hide my audio book habit because most of my colleagues, and even some of my snobbier students, regard audio books as a sign of an impending dark age of mass illiteracy. Feeling uneasy, I wonder: when The Brothers Karamazov comes up in conversation am I obliged to "confess" that I listened to the unabridged audio book, but did not silently read (...) the massive tome? Is there a difference that makes a difference between listening to an audio book and reading a printed book?IPeople read less today, partly because reading silently is not social; we crave the company of loved ones at the end of the day, and so prefer to watch television with them. Silent reading does not fit today's multitasking lifestyle; you can listen to music while you drive or jog, but you cannot read a book. Or, then again, perhaps you can, if listening to an audio book is reading after all. Time does not need to be specifically set aside for an audio book the way it does with a conventional book—the lights don't even have to be on. And surprisingly, it is satisfying to listen to a book ten minutes at a time while running errands. Perhaps this is because getting started is as easy as pressing a button. By contrast, few people read printed books in ten minute blocks, presumably because it takes something for most people to warm up to reading. If they are going to read, then they are going to do it for a sustained time—something longer than a ten-minute block. It is easier to "push through" and finish an audio book. I nearly gave up on Catch-22; it's too thick and over the top with satire for my taste. And I surely would have given up on it if I had been reading the book silently. I stuck with [End Page 358] the audio book, however, largely because it was easy to continue; I just pressed play every time I got in my car.Books on tape have been available for quite some time, but, despite often being abridged, they were too clunky and cumbersome for most people. The availability of books on CD and for MP3 download makes it all much more convenient. Audio books are now the fastest growing segment of the book industry and are likely to become even more common and popular, perhaps some day eclipsing the sale of conventional books, perhaps some day becoming the normal way of "reading" literature. And far from being abridged, some audio books today include extras. For example, the audio version of The Amber Spyglass includes epigrams before each chapter that are not in the print version.IIHarold Bloom says, "Deep reading really demands the inner ear as well as the outer ear. You need the whole cognitive process, that part of you which is open to wisdom. You need the text in front of you."1 Before listening to audio books myself I would have agreed with Bloom; but now I suspect he simply has never listened to an audio book. Skeptical of literature on audio, I started my audio experience with nonfiction. Pleasantly surprised with nonfiction, I tried listening to fiction and found the presentation very different. Nonfiction books are simply read, whereas works of fiction are typically performed—characters get distinct voices and the reader really puts on a show. Most audio book performances of literature are much like a one-man play, taking us back to the days of traveling bards and rhapsodes.In The Performance of Reading Peter Kivy highlights that for most of its history Western literature has been experienced out loud.2 As is well known, Homer's Iliad and Odyssey were performed out loud by bards, as Beowulf was centuries later. But less well known is the fact that even after the invention of the printing press, when books were still scarce and expensive, even people in possession of books generally read them out loud to themselves and to others. So it is odd that along with Harold Bloom, my colleagues and students, look with mistrust at... (shrink)

In the Internet age, information security is threatened anytime and anywhere and the copyright protection of audio and video as well as the need for matching detection is increasingly strong. In view of this need, this paper proposes a zero-watermarking algorithm for audio and video matching based on NSCT. The algorithm uses NSCT, DCT, SVD, and Schur decomposition to extract video features and audio features and generates zero-watermark stream through synthesis, which is stored in a third-party organization (...) for detection and identification. The detection algorithm can obtain zero watermark from the audio and video to be tested and judge and locate tampering by comparing with the zero watermark of the third party. From the experimental results, this algorithm can not only detect whether the audio and video are mismatched due to tampering attacks but also locate the mismatched audio and video segments and protect the copyright. (shrink)

In this paper we offer a theory of cross-modal objects. To begin, we discuss two kinds of linkages between vision and audition. The first is a duality. The the visual system detects and identifies surfaces ; the auditory system detects and identifies sources . Surfaces are illuminated by sources of light; sound is reflected off surfaces. However, the visual system discounts sources and the auditory system discounts surfaces. These and similar considerations lead to the Theory of Indispensable Attributes that states (...) the conditions for the formation of gestalts in the two modalities. The second linkage involves the formation of audiovisual objects, integrated cross-modal experiences. We describe research that reveals the role of cross-modal causality in the formation of such objects. These experiments use the canonical example of a causal link between vision and audition: a visible impact that causes a percussive sound. (shrink)

IntroductionThe idea of video recording in the operating room with panoramic cameras and microphones is a new concept that is changing the approach to medical activities in the OR. However, VR in the OR has brought up many concerns regarding patient privacy and has highlighted legal and ethical issues that were never previously exposed.AimTo review the literature concerning these aspects and provide a better ethical and legal understanding of the new challenges concerning VR in the OR.ConclusionsThere is a disparity between (...) the two main legal models concerning VR in the OR, namely the European legal system ) and the American legal framework ). This difference mainly deals with two distinct bioethical paradigms: GDPR places a strong emphasis on protecting patients’ privacy to improve the public health system, whereas HIPAA indicates the need to generate protocols to safeguard the risks connected to medical activity and patient privacy. Following from this point, we may argue that, at the ethical and bioethical level, GDPR and HIPAA depend mainly on two different ethical models: a perspective based on moral acquaintances and weak proceduralism, respectively. It is worth noting the importance of developing additional guidelines concerning different world regions to avoid the ethical problems that may emerge when simply applying a foreign paradigm to a very different culture. (shrink)

Vision allows perception of ordinary material objects; audition, the sources of sounds. Often, perception of objects involves vision and audition—seeing and hearing the perceived things. Perception here is multi-sensory. This chapter addresses how multi-sensory perception should be understood. It distinguishes two different models for multi-sensory perception—the “cross-modal” model (different senses each represent objects in a modality-specific way), and the “amodal” model (different senses contribute to a single amodal representation of an object)—and asks which model best explains multi-sensory awareness of objects. (...) It then examines several different kinds of auditory-visual interaction to determine whether any support the amodal model. Although many interactions can be explained by the cross-modal model, some can be explained only by the amodal model suggesting that in some instances multi-sensory perception is amodal. The chapter ends by describing some consequences this might have for understanding auditory perception and perception more generally. (shrink)

State shyness refers to a transient emotional experience characterised by simultaneous approach and avoidance tendencies when individuals encounter novel stimuli or perceive social evaluations, representing a common phenomenon in daily social interactions. Current methods for eliciting state shyness, such as public speaking tasks, often involve complex indirect experimental designs and depend on post-hoc behavioural coding, making it difficult to precisely induce and synchronously measure state shyness, thus limiting their application in controlled experimental settings. To address this methodological limitation, this research (...) developed and validated an audio-based corpus specifically designed for the direct induction of state shyness. The corpus consists of two subsets, each containing 25 standardised social scenarios. In Study 1, semi-structured interviews with 126 university students informed the creation of these scenarios, which were subsequently refined through thematic analysis. The efficacy of the corpus was then validated through self-reported state shyness ratings from 58 participants. Study 2 further validated the corpus’s reliability by employing Electrodermal Activity as an objective physiological measure with 37 participants. This research offers a standardised and easily deployable tool for inducing state shyness in controlled experimental contexts, enabling direct and precise inducing state shyness. (shrink)

Experimental research has shown that pairs of stimuli which are congruent and assumed to 'go together' are recalled more effectively than an item presented in isolation. Will this multisensory memory benefit occur when stimuli are richer and longer, in an ecological setting? In the present study, we focused on an everyday situation of audio-visual learning and manipulated the relationship between audio guide tracks and viewed portraits in the galleries of the Tate Britain. By varying the gender and narrative (...) style of the voice-over, we examined how the perceived congruency and assumed unity of the audio guide track with painted portraits affected subsequent recall. We show that tracks perceived as best matching the viewed portraits led to greater recall of both sensory and linguistic content. We provide the first evidence that manipulating crossmodal congruence and unity assumptions can effectively impact memory in a multisensory ecological setting, even in the absence of precise temporal alignment between sensory cues. (shrink)

This paper has already been published in Sociological Research Online, 21/3, August 2016. All rights reserved. © SOCIOLOGICAL RESEARCH ONLINE. It is available from Kent Academic Repository. We thank Dawn Lyon for the permission to republish it here.: This article documents, shows and analyses the everyday rhythms of Billingsgate, London's wholesale fish market. It takes the form of a short film based an audiovisual montage of time-lapse photography and sound recordings, and a - Sociologie – Nouvel article.

Utilizing a pre-experimental pre-test post-test design, this study investigated the effect of an authentic audio-visual American sitcom on the intercultural sensitivity and intercultural effectiveness of a sample of male and female upper-intermediate English students. To this aim, 34 Iranian EFL students were selected through convenient non-random sampling. In order to assure the participants' homogeneity in English proficiency, the selected students were given the Oxford Quick Placement Test prior to the intervention. Over a 10-week period, the participants were presented with (...) nearly 30-min-long episodes of Friends in a flipped context, two episodes every week uploaded to the accompanying website. An online 10-item quiz on the day of watching and a next-day meeting were held. During the online discussion meetings, the participants discussed cultural questions tailored to the aims of the study. The required data were collected through the administration of the intercultural sensitivity scale and intercultural effectiveness scale in pre- and post-test assessments. The analyses of the data indicated that both ICS and ICE improved significantly during the intervention. However, ICS indicated more improvement. In addition, a semi-structured interview was administered to the participants to examine their perception of the flipped classroom experience. Having analyzed the data using MAXQDA 2020, some significant themes emerged which are reported. On the whole, the participants portrayed positive attitudes toward using technology in flipped classes. The educational and practical implications are discussed. (shrink)

We investigate whether smile mimicry and emotional contagion are evident in non-text-based computer-mediated communication (CMC). Via an ostensibly real-time audio-visual CMC platform, participants interacted with a confederate who either smiled radiantly or displayed a neutral expression throughout the interaction. Automatic analyses of expressions displayed by participants indicated that smile mimicry was at play: A higher level of activation of the facial muscle that characterises genuine smiles was observed among participants who interacted with the smiling confederate than among participants who (...) interacted with the unexpressive confederate. However, there was no difference in the self-reported level of joviality between participants in the two conditions. Our findings demonstrate that people mimic smiles in audio-visual CMC, but that even though the diffusion of emotions has been documented in text-based CMC in previous studies, we find no convincing support for the phenomenon of emotional contagion in non-text-based CMC. (shrink)

The prevalence of violence particularly bullying in elementary schools represents a significant challenge to students’ well-being and educational achievement. The negative effects of bullying can hinder academic success and personal development. Therefore, integrating anti-bullying values through educational interventions such as an audio fairy tale book is crucial for fostering awareness and promoting positive attitudes among students. This study aims to develop and assess the effectiveness of an audio fairy tale book as an educational tool for bullying prevention in (...) elementary schools. The Research and Development (R&D) methodology employing the ADDIE model (Analysis, Design, Development, Implementation, and Evaluation) was utilized in this study. The participants included teachers and students from SDN 2 Krapyak and SDN 1 Sowan Lor in Jepara. Data were collected through questionnaires, interviews, observations, and expert validation. Both quantitative and qualitative data analysis were employed. The results indicate that the developed audio fairy tale book is a highly feasible and effective medium for anti-bullying education. Expert validation scores were 90% for media experts and 94% for material experts. Teacher feedback averaged 90 and 92, respectively, with trial results showing an average score of 4.6 and 4.48. The audio fairy tale book has proven to be an effective tool in preventing bullying, enhancing student engagement, and fostering a deeper understanding of anti-bullying values. The inclusion of a reflection guide at the end of each story further strengthens its impact, offering students a meaningful experience in developing positive attitudes. (shrink)

RESUMO O projeto BiRCh (The Corpus of Bilingual Russian Child Speech, Corpus de fala de crianças bilíngues em russo) envolve a construção de um corpus longitudinal composto de gravações de fala em russo produzida por crianças e suas famílias na Rússia, Ucrânia, Alemanha, EUA e Canadá. Estamos construindo um corpus de larga escala com base no conjunto dessas gravações, o ‘Parsed and Audio-aligned Corpus of Bilingual Russian Child and Child-directed Speech (BiRCh)’, com os dois componentes básicos: (1) as transcrições (...) de um milhão de palavras alinhadas com os arquivos de áudio, em que pode ser realizada a busca textual, e (2) as transcrições de 500 mil palavras anotadas morfologicamente e analisadas sintaticamente, também alinhadas com os arquivos de áudio. Estamos utilizando o corpus para investigar os diversos fenômenos no input linguístico e na trajetória do desenvolvimento de falantes de herança, tais como o uso de caso, gênero, construções passivas e impessoais, marcadores de polidez, disfluências e marcadores discursivos. Este artigo enfoca os desafios e soluções no processo da construção do BiRCh e as implicações para a pesquisa com base nos dados detalhadamente anotados fornecidos pelo corpus. (shrink)

The purpose of this study is to discuss the effects of audiovisual input on second language acquisition and the factors that influence the difficulty of audiovisual learning through a systematic literature review. Prior to this systematic review, in this paper, we searched papers on related topics for the past 10 years from 2012 to 2022, and found 46 journal papers that met the research criteria. They can basically represent the scholarly work in this field. The 46 studies were published in (...) journals indexed in Google Scholar, Eric, Scopus, and Wiley Library. Databases were selected according to a set of inclusion and exclusion criteria. The following conclusions are drawn from the literature review: Audiovisual input can provide more authentic language input and more adequate and richer multimodal cultural and situational contexts, which can better promote learners' understanding of the content and stimulate learners' interest in participating in listening comprehension tasks. The influencing factors of multimodal input on listening difficulty include subtitles, video type, and the relationship between the audio and visual input. (shrink)

Previous research with speech and non-speech stimuli suggested that in audiovisual perception, visual information starting prior to the onset of corresponding sound can provide visual cues, and form a prediction about the upcoming auditory sound. This prediction leads to audiovisual interaction. Auditory and visual perception interact and induce suppression and speeding up of the early auditory event-related potentials such as N1 and P2. To investigate AV interaction, previous research examined N1 and P2 amplitudes and latencies in response to audio (...) only, video only, audiovisual, and control stimuli, and compared AV with auditory perception based on four AV interaction models. The current study addresses how different models of AV interaction express N1 and P2 suppression in music perception. Furthermore, the current study took one step further and examined whether previous musical experience, which can potentially lead to higher N1 and P2 amplitudes in auditory perception, influenced AV interaction in different models. Musicians and non-musicians were presented the recordings of a keyboard /C4/ key being played, as well as CO stimuli. Results showed that AV interaction models differ in their expression of N1 and P2 amplitude and latency suppression. The calculation of model and has consequences for the resulting N1 and P2 difference waves. Furthermore, while musicians, compared to non-musicians, showed higher N1 amplitude in auditory perception, suppression of amplitudes and latencies for N1 and P2 was similar for the two groups across the AV models. Collectively, these results suggest that when visual cues from finger and hand movements predict the upcoming sound in AV music perception, suppression of early ERPs is similar for musicians and non-musicians. Notably, the calculation differences across models do not lead to the same pattern of results for N1 and P2, demonstrating that the four models are not interchangeable and are not directly comparable. (shrink)

This article argues that errors in audio data processing should be examined to explore and expose the underlying components that enable linguistic communication and cross-cultural understanding. Examples of errors in the transcription of a Mexican social network’s conversations are analyzed to demonstrate the potential of such data in the development of sociocognitive language-processing theories. It is suggested that researchers working with audio-recorded data should expand the scope of what is considered useful data for the sake of both methodological (...) reflexivity and identifying the underlying cognitive processes enabling linguistic understanding. (shrink)

This article investigates the hypothesis of creating non-linear audio-visual narratives, through an unanticipated use of traditional print-based games, enriched with videos, via augmented reality (AR) possibilities. A ludic system has been created and presented. Based on a traditional card game, a non-linear cinematic narrative occurs. We attempt to examine the following questions: in which way can we bring together different forms of visual communication, such as graphic design and video? Can the above forms create a complex narrative whole and (...) what kind of rules will be needed for this? How can we enrich traditional forms of gaming with the potentials of AR? Gaming itself demands a set of rules. Can these rules play the role of algorithms in the combined universe that we have designed and created? In which way can the designer on the one hand and the user on the other influence the overall output of the system? What will the user experience be like? The printed card system chosen for this is Tarot and more precisely the Great Arcana, which makes use of the 22 fundamental Tarot figures. (shrink)

Podcasts have been described as “a cinema for the ears” and this application of a visual rhetoric to describe an audio-only experience results in an attempt to define what is still a relatively new medium. I argue that it is possible to consider something cinematic without the presence of moving images. Assertions in favour of the cinematic nature of podcasts often employ the visual imagination of listeners evoked by heightened audio characteristics that a particular podcast may possess. By (...) focusing on film-centred podcasts specifically, which, in terms of content and form, are implicitly and often explicitly concerned with properties of the cinematic, I argue for a more conceptual analysis of the idea of a visual form of audio. While many film-oriented podcasts provide a supplementary celebration of cinema culture rather than manifesting a unique cinematic experience of their own, there are examples of film-centred podcasts that attempt to actualise what I will call an “audio-cinematic” experience, deploying the creative potential of the podcast to manifest an experiential aura that evokes a cinematic imagination. I analyse the sonic dimensions of audio-cinematic podcasts including my work with Neil Fox on The Cinematologists Podcast. (shrink)

Following Cass Sunstein's popular treatment of the concept, echo chambers are often defined as environments which exclude contrary opinions through omission. C. Thi Nguyen contests the popular usage and defines echo chambers in terms of in-group trust and out-group distrust. In this paper, I argue for a more comprehensive treatment. While both exclusion by omission and out-group distrust help sustain echo chambers, neither defines the phenomenon. I develop a social network model of echo chambers which focuses on the role of (...) belief-reinforcing echoes. First, I argue that the model allows us to incorporate Nguyen's main point about distrust without construing other commentators as deeply mistaken about the nature of echo chambers. Second, I use the model to develop an account of collaborative resistance and use it to clarify the role echo chambers play in spreading misinformation. (shrink)

In this position paper, we have used Alan Cooper’s persona technique to illustrate the utility of audio- and video-based AAL technologies. Therefore, two primary examples of potential audio- and video-based AAL users, Anna and Irakli, serve as reference points for describing salient ethical, legal and social challenges related to use of AAL. These challenges are presented on three levels: individual, societal, and regulatory. For each challenge, a set of policy recommendations is suggested.

Despite a predominantly digital, 21st century music production landscape, analogue hardware professional audio technologies persist. In the discoursal throes of the leading online audio technology message forum Gearslutz, such technologies are routinely objectified, sexualized, fetishized and socialized into gear. Situated in a contemporary critical, interdisciplinary framework of fetish, masculinity and sexuality studies, this research interrogates how audio technologies manufactured and intended for music production contexts become sexy. Applying a mixed-mode methodology, including an intensive discourse, image and material-semiotic (...) analysis of an ‘epic’ sexy gear thread, we collated extensive data about technological fetishization. Sexy gear discourse articulates themes of voyeurism, acquisition, control and animation – linking the fetish value of technological objects and their connoisseurship with the erotic potential of sexualized objects. Such discourses ultimately serve to maintain social order, and become sites for performing the maintenance work of hegemonic masculine formations. This research provides new insights into how hegemonic masculinities depend upon the organization of online and offline sociability around fetishized material objects. Furthermore, our findings align with those of current scholarship focused on representational politics of technoculture. (shrink)

It is a matter of fact that Europe is facing more and more crucial challenges regarding health and social care due to the demographic change and the current economic context. The recent COVID-19 pandemic has stressed this situation even further, thus highlighting the need for taking action. Active and Assisted Living technologies come as a viable approach to help facing these challenges, thanks to the high potential they have in enabling remote care and support. Broadly speaking, AAL can be referred (...) to as the use of innovative and advanced Information and Communication Technologies to create supportive, inclusive and empowering applications and environments that enable older, impaired or frail people to live independently and stay active longer in society. AAL capitalizes on the growing pervasiveness and effectiveness of sensing and computing facilities to supply the persons in need with smart assistance, by responding to their necessities of autonomy, independence, comfort, security and safety. The application scenarios addressed by AAL are complex, due to the inherent heterogeneity of the end-user population, their living arrangements, and their physical conditions or impairment. Despite aiming at diverse goals, AAL systems should share some common characteristics. They are designed to provide support in daily life in an invisible, unobtrusive and user-friendly manner. Moreover, they are conceived to be intelligent, to be able to learn and adapt to the requirements and requests of the assisted people, and to synchronise with their specific needs. Nevertheless, to ensure the uptake of AAL in society, potential users must be willing to use AAL applications and to integrate them in their daily environments and lives. In this respect, video- and audio-based AAL applications have several advantages, in terms of unobtrusiveness and information richness. Indeed, cameras and microphones are far less obtrusive with respect to the hindrance other wearable sensors may cause to one’s activities. In addition, a single camera placed in a room can record most of the activities performed in the room, thus replacing many other non-visual sensors. Currently, video-based applications are effective in recognising and monitoring the activities, the movements, and the overall conditions of the assisted individuals as well as to assess their vital parameters. Similarly, audio sensors have the potential to become one of the most important modalities for interaction with AAL systems, as they can have a large range of sensing, do not require physical presence at a particular location and are physically intangible. Moreover, relevant information about individuals’ activities and health status can derive from processing audio signals. Nevertheless, as the other side of the coin, cameras and microphones are often perceived as the most intrusive technologies from the viewpoint of the privacy of the monitored individuals. This is due to the richness of the information these technologies convey and the intimate setting where they may be deployed. Solutions able to ensure privacy preservation by context and by design, as well as to ensure high legal and ethical standards are in high demand. After the review of the current state of play and the discussion in GoodBrother, we may claim that the first solutions in this direction are starting to appear in the literature. A multidisciplinary debate among experts and stakeholders is paving the way towards AAL ensuring ergonomics, usability, acceptance and privacy preservation. The DIANA, PAAL, and VisuAAL projects are examples of this fresh approach. This report provides the reader with a review of the most recent advances in audio- and video-based monitoring technologies for AAL. It has been drafted as a collective effort of WG3 to supply an introduction to AAL, its evolution over time and its main functional and technological underpinnings. In this respect, the report contributes to the field with the outline of a new generation of ethical-aware AAL technologies and a proposal for a novel comprehensive taxonomy of AAL systems and applications. Moreover, the report allows non-technical readers to gather an overview of the main components of an AAL system and how these function and interact with the end-users. The report illustrates the state of the art of the most successful AAL applications and functions based on audio and video data, namely lifelogging and self-monitoring, remote monitoring of vital signs, emotional state recognition, food intake monitoring, activity and behaviour recognition, activity and personal assistance, gesture recognition, fall detection and prevention, mobility assessment and frailty recognition, and cognitive and motor rehabilitation. For these application scenarios, the report illustrates the state of play in terms of scientific advances, available products and research project. The open challenges are also highlighted. The report ends with an overview of the challenges, the hindrances and the opportunities posed by the uptake in real world settings of AAL technologies. In this respect, the report illustrates the current procedural and technological approaches to cope with acceptability, usability and trust in the AAL technology, by surveying strategies and approaches to co-design, to privacy preservation in video and audio data, to transparency and explainability in data processing, and to data transmission and communication. User acceptance and ethical considerations are also debated. Finally, the potentials coming from the silver economy are overviewed. (shrink)

The rapid advancement of neural text-to-speech and voice conversion technologies has led to synthetic speech of unprecedented quality, raising urgent concerns over its misuse in disinformation, fraud, and impersonation. Despite growing awareness and notable contributions from the research community, synthetic speech detection remains a fundamentally open problem. This chapter adopts a holistic perspective to analyse existing detection strategies and propose future directions. We begin with a detailed analysis of data-driven state-of-the-art methods, providing an historical perspective and highlighting their reliance on (...) vocoding artefacts—traces that are currently effective but are likely to disappear as neural codecs replace traditional audio compression. We then explore content-aware and voice-based approaches, which aim to identify inconsistencies intrinsic to the speech signal rather than to the vocoding process. To address generalisation and long-term robustness, we investigate viable generalisation paradigms to detect anomalies due to the synthesis algorithms without needing access to representative synthetic examples. Finally, we discuss broader non-technical challenges, including data availability, financial sustainability, and impact of regulatory frameworks, to then outline a sustainable path forward for future research on synthetic speech detection. (shrink)

The term “Northeastern Renaissance” carries a dual meaning: on the one hand, it represents an artistic movement that emerged in the late 2010s in the three Northeastern provinces of the People’s Republic of China (as well as the eastern regions of Inner Mongolia), bearing a manifesto-like quality; on the other hand, it is also ironic, as the Northeastern provinces have long been perceived as a region that was historically beyond the reach of China’s traditional elite culture, thus having little to (...) revive or restore. This duality is reflected in the audio-visual works of Northeastern post-industrial culture, particularly in their depiction of sex and violence. These works exhibit an ambiguous attitude toward sex and violence—both critical and humorous. Building upon the absurdist foundation of Northeastern post-industrial culture, as discussed in the author’s 2024 academic presentation, this paper further explores the dual nature of sex and violence. The structure of the paper is as follows: The first section reviews my 2024 discussion on the absurdity in Northeastern post-industrial audio-visual works. The subsequent chapters focus on Drifting Away, a crime drama that aired in 2025, whose narrative elicited polarized reactions from the audience—some fervently idolized the four butcher-like serial killers, while others found the moral implications deeply unsettling. Finally, the author will analyze this contrast through the lens of the duality in Northeastern post-industrial culture, demonstrating how the intersection of Northeastern migrant culture and post-industrial conditions has given rise to a satirical art form that challenges traditional Chinese moral values. (shrink)

Since 2019, Northeastern post-industrial culture has been a popular topic of discussion; the general public refers to it as the Northeastern Renaissance. Crises of identity, honor, and faith have been recurring themes in several Northeastern films released in recent years. Furthermore, these cinematic narratives frequently generate somber humor by presenting an enormous contrast between ideals and actuality. The article examines how the post-industrial narrative context of Northeast China has influenced audio-visual cultural products and contemporary Chinese popular culture. To elucidate (...) the fundamental aspect of the Northeast narrative, the article’s author employs the absurd theory of Albert Camus. The main structure and viewpoints of the article are as follows: The first is after presenting individuals’ contemplations on the Northeastern Renaissance-instigated post-industrial narrative of the Northeast. In the following sections, the article explores the narrative context of post-industrial audio-visual works in Northeast China, focusing on the absurdity depicted in the documentary Tie Xi Qu: West of the Tracks (2002). The authors’ analysis reveals how the documentary captures the socio-economic decline and the resulting human experiences of absurdity and dislocation. Key findings indicate that the narrative structure and visual style effectively convey the bleak realities of post-industrial life, offering critical insights into the region’s socio-cultural transformation. These findings suggest that future research should explore the intersection of visual media and social change in other post-industrial contexts. -/- . (shrink)

Audio features such as inharmonicity, noisiness, and spectral roll-off have been identified as correlates of “noisy” sounds. However, such features are likely involved in the experience of multiple semantic timbre categories of varied meaning and valence. This paper examines the relationships of stimulus properties and audio features with the semantic timbre categories raspy/grainy/rough, harsh/noisy, and airy/breathy. Participants rated a random subset of 52 stimuli from a set of 156 approximately 2-s orchestral instrument sounds representing varied instrument families, registers, (...) and both traditional and extended playing techniques. Stimuli were rated on the three semantic categories of interest, as well as on perceived playing exertion and emotional valence. Correlational analyses demonstrated a strong negative relationship between positive valence and perceived physical exertion. Exploratory linear mixed models revealed significant effects of extended technique and pitch register on valence, the perception of physical exertion, raspy/grainy/rough, and harsh/noisy. Instrument family was significantly related to ratings of airy/breathy. With an updated version of the Timbre Toolbox, we used 44 summary audio features, extracted from the stimuli using spectral and harmonic representations, as input for various models built to predict mean semantic ratings for each sound on the three semantic categories, on perceived exertion, and on valence. Random Forest models predicting semantic ratings from audio features outperformed Partial Least-Squares Regression models, consistent with previous results suggesting that non-linear methods are advantageous in timbre semantic predictions using audio features. Relative Variable Importance measures from the models among the three semantic categories demonstrate that although these related semantic categories are associated in part with overlapping features, they can be differentiated through individual patterns of audio feature relationships. (shrink)