Article
Computer Science, Information Systems
Gerda Ana Melnik-Leroy, Jolita Bernataviciene, Grazina Korvel, Gediminas Navickas, Gintautas Tamulevicius, Povilas Treigys
Summary: This paper is the first attempt to gather research on intonation in Lithuanian from both the Lithuanian and the Western traditions, the structuralist and generativist points of view, and the linguistic and modelling perspectives. It identifies issues in existing research and proposes directions for future investigations in linguistics and modelling.
Article
Acoustics
Li Wang, Sanrong Xiao, Cunmei Jiang, Qingqi Hou, Alice H. D. Chan, Patrick C. M. Wong, Fang Liu
Summary: Studies on how autistic individuals process tone and intonation in regards to form and function have mainly focused on speakers of non-tonal languages, with varying results. This study examined Mandarin-speaking autistic children and found that their abilities to discriminate tone and intonation were not impaired at either the form or function level. These abilities were positively associated with each other and were worse in autistic children with more severe symptoms of autism. However, enhanced tone and intonation processing may not be a general characteristic of the autistic population with long-term tone language experience.
JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA
(2023)
Article
Acoustics
Seeon Kim, Hsiao-Hsiuan Chou, Xin Luo
Summary: Training with enhanced amplitude envelope could improve tone recognition for CI users, especially for those who attended more to amplitude envelope cues before training. Collaboration of auditory training and speech enhancement can bring maximum benefits to CI users.
JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA
(2021)
Article
Psychology, Developmental
Li Wang, C. Philip Beaman, Cunmei Jiang, Fang Liu
Summary: Individuals with ASD show intact abilities in discriminating, identifying, and imitating statement-question intonation, with sensitivity to pitch direction predicting performance. Both groups exhibit similar developmental changes in intonation processing.
JOURNAL OF AUTISM AND DEVELOPMENTAL DISORDERS
(2022)
Article
Acoustics
Siddique Latif, Abdullah Shahid, Junaid Qadir
Summary: Despite the lack of speech emotion datasets, this paper suggests using synthetic emotional speech generated by an end-to-end text-to-speech (TTS) system to augment speech emotion recognition (SER) systems. The proposed TTS system includes encoders for speaker and emotion embeddings, a sequence-to-sequence text generator, and a WaveRNN for audio generation. Experimental results show that the generated emotional speech significantly improves SER performance on multiple datasets and effectively augments SER performance.
Article
Computer Science, Artificial Intelligence
John Harvill, Seong-Gyun Leem, Mohammed AbdelWahab, Reza Lotfian, Carlos Busso
Summary: This study proposes a new formula for measuring emotional similarity between speech recordings. Instead of predicting emotional attributes or recognizing emotional categories, this formulation explores the ordinal nature of emotions by comparing emotional similarities. The study addresses questions about which emotional descriptors provide the most suitable space to assess emotional similarities and whether deep neural networks can learn representations to quantify emotional similarities robustly. By creating alternative emotional spaces using attribute-based descriptors and categorical emotions, the study shows that a meaningful embedding can be learned to assess emotional similarities, outperforming human evaluators in the same task.
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING
(2023)
Article
Acoustics
Muhammad Qasim, Tania Habib, Saba Urooj, Benazir Mumtaz
Summary: This paper presents the design and development of a dyadic emotional speech corpus for the Urdu language. Emotion recognition experiments have been conducted using various classifiers, and the utterance-level features have shown better performance than frame-level features and spectrograms. The combined feature set of cepstral, spectral, prosodic, and voice quality features performs better than individual feature sets, achieving high recall rates for emotion recognition.
SPEECH COMMUNICATION
(2023)
Article
Computer Science, Information Systems
Pawel Powroznik, Piotr Wojcicki, Slawomir W. Przylucki
Summary: The article discusses the challenges of implementing emotion recognition systems based on spoken text and presents a method using scalograms and natural language processing algorithms to accomplish this task. Testing was done on emotional speech recordings in Polish, English, German, and Danish, with results ranging from 62% to over 94% accuracy depending on language and classifier used. The use of fuzzy classifiers was found to greatly improve the efficiency of classification.
Article
Acoustics
C. Hema, Fausto Pedro Garcia Marquez
Summary: Emotions are an important part of human life and can be expressed through speech. Speech Emotion Recognition (SER) systems can extract and predict the emotional tone of a speaker using audio signals. Emotions can be categorized into types such as anger, happiness, sadness, and neutral, and a system can be developed based on this criterion with proper training and resources.
Article
Chemistry, Multidisciplinary
Behnam Faghih, Joseph Timoney
Summary: This article introduces a new smoother algorithm for pitch contours and compares it with 15 other existing algorithms. The proposed algorithm is shown to smooth the contours more accurately, highlighting the importance of designing smoother algorithms based on contour types and intended applications.
APPLIED SCIENCES-BASEL
(2022)
Article
Mathematics
Dmitry Ryumin, Elena Ryumina, Denis Ivanko
Summary: This article presents a novel approach called EMOLIPS for emotional speech lip-reading. It utilizes visual data processing and deep learning techniques for speech to text recognition. By using trained emotional lip-reading models, the approach successfully addresses the issue of multi-emotional lip-reading in real-life scenarios. Experimental results show a significant improvement in phrase recognition accuracy.
Article
Chemistry, Analytical
Gulmira Bekmanova, Banu Yergesh, Altynbek Sharipbay, Assel Mukanova
Summary: This article presents an emotional speech recognition method for recognizing student emotions during online exams in distance learning due to COVID-19. The method achieves an accuracy of 79.7% for the Kazakh language and can be widely applied to recognize emotions in different languages. It analyzes human speech using emotionally charged words stored in a code book to determine the presence of emotions.
Article
Multidisciplinary Sciences
Xiaoying Zhang, Zuliyaer Talifu, Jianjun Li, Xiaobing Li, Feng Yu
Summary: Music-based melodic intonation therapy (MIT) shows promise as a treatment for non-fluent aphasia after stroke, with positive effects on reorganization and activation of the arcuate fasciculus in the brain.
Article
Multidisciplinary Sciences
Annaliese Micallef Grimaud, Tuomas Eerola
Summary: This study investigates the relationship between seven musical cues and seven emotions using perception and production approaches. The results show that the two approaches yield similar findings in most cue-emotion combinations, with dynamics and instrumentation cues showing the most variation. In terms of prediction accuracy, the production approach outperforms the perception approach, suggesting that it may be a more efficient method to explore how cues shape different emotions in music.
Article
Engineering, Electrical & Electronic
Akshita Abrol, Nisha Kapoor, Parveen Kumar Lehana
Summary: Analyzing a low complexity fractal-based technique is believed to provide better emotional content estimation due to the nonlinear nature of speech signals, reliably identifying emotional content within speech signals by calculating fractal dimension and loop area parameters, and being able to withstand up to 10 dB of signal noise level.
CIRCUITS SYSTEMS AND SIGNAL PROCESSING
(2021)
Article
Computer Science, Artificial Intelligence
Reza Lotfian, Carlos Busso
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING
(2019)
Article
Acoustics
Reza Lotfian, Carlos Busso
IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING
(2019)
Article
Acoustics
Najmeh Sadoughi, Carlos Busso
SPEECH COMMUNICATION
(2019)
Article
Acoustics
Fei Tao, Carlos Busso
SPEECH COMMUNICATION
(2019)
Article
Psychiatry
Daniel Fulford, Jasmine Mote, Rachel Gonzalez, Samuel Abplanalp, Yuting Zhang, Jarrod Luckenbaugh, Jukka-Pekka Onnela, Carlos Busso, David E. Gard
Summary: Social impairment is prevalent in schizophrenia spectrum disorders, with individuals often having difficulty accurately reporting their social behaviors. Smartphone sensors may offer more objective indicators of social activity, showing promise for understanding individuals with schizophrenia.
JOURNAL OF PSYCHIATRIC RESEARCH
(2021)
Article
Oncology
Elvis Duran-Sierra, Shuna Cheng, Rodrigo Cuenca, Beena Ahmed, Jim Ji, Vladislav V. Yakovlev, Mathias Martinez, Moustafa Al-Khalil, Hussain Al-Enazi, Yi-Shing Lisa Cheng, John Wright, Carlos Busso, Javier A. Jo
Summary: The combination of multispectral autofluorescence lifetime imaging (maFLIM) and machine learning allows for automated discrimination of dysplastic and cancerous oral tissue from healthy tissue, potentially improving outcomes for oral cancer patients by facilitating maximal tumor resection.
Article
Computer Science, Artificial Intelligence
Georgios N. Yannakakis, Roddy Cowie, Carlos Busso
Summary: This paper discusses the theoretical reasons for using ordinal labels to represent and annotate emotions, emphasizing the appropriateness of preference learning methods in treating ordinal labels, and demonstrates the advantages of ordinal annotation in affective computing through case studies.
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING
(2021)
Article
Computer Science, Information Systems
Fei Tao, Carlos Busso
Summary: The study introduces a novel multitask learning audiovisual automatic speech recognition system that generalizes across conditions, improves performance, and solves two key speech tasks.
IEEE TRANSACTIONS ON MULTIMEDIA
(2021)
Proceedings Paper
Engineering, Electrical & Electronic
Yuning Qiu, Teruhisa Misu, Carlos Busso
2020 IEEE 23RD INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS (ITSC)
(2020)
Proceedings Paper
Computer Science, Artificial Intelligence
Ali N. Salman, Carlos Busso
2020 15TH IEEE INTERNATIONAL CONFERENCE ON AUTOMATIC FACE AND GESTURE RECOGNITION (FG 2020)
(2020)
Proceedings Paper
Imaging Science & Photographic Technology
Ali N. Salman, Carlos Busso
2020 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP)
(2020)
Proceedings Paper
Acoustics
Kusha Sridhar, Carlos Busso
2020 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING
(2020)
Article
Acoustics
Srinivas Parthasarathy, Carlos Busso
IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING
(2020)
Article
Computer Science, Information Systems
Andrea Vidal, Jorge F. Silva, Carlos Busso
Proceedings Paper
Acoustics
Sumit Jha, Carlos Busso
2019 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP)
(2019)