Refine
Document Type
- Article (4) (remove)
Language
- English (4)
Has Fulltext
- yes (4)
Keywords
- 3D Recovery (1)
- Ambisonics (1)
- Beamforming (1)
- Beamforming <Signalverarbeitung> (1)
- Binaural Rendering (1)
- Binaural Reproduction (1)
- COVID-19 Pandemic (1)
- Computer Science (1)
- GPU (1)
- Higher Education (1)
Faculty
- Fakultät 07 / Institut für Medien- und Phototechnik (4) (remove)
In this work, we propose a novel data-driven approach to recover missing or corrupted motion capture data, either in the form of 3D skeleton joints or 3D marker trajectories. We construct a knowledge-base that contains prior existing knowledge, which helps us to make it possible to infer missing or corrupted information of the motion capture data. We then build a kd-tree in parallel fashion on the GPU for fast search and retrieval of this already available knowledge in the form of nearest neighbors from the knowledge-base efficiently. We exploit the concept of histograms to organize the data and use an off-the-shelf radix sort algorithm to sort the keys within a single processor of GPU. We query the motion missing joints or markers, and as a result, we fetch a fixed number of nearest neighbors for the given input query motion. We employ an objective function with multiple error terms that substantially recover 3D joints or marker trajectories in parallel on the GPU. We perform comprehensive experiments to evaluate our approach quantitatively and qualitatively on publicly available motion capture datasets, namely CMU and HDM05. From the results, it is observed that the recovery of boxing, jumptwist, run, martial arts, salsa, and acrobatic motion sequences works best, while the recovery of motion sequences of kicking and jumping results in slightly larger errors. However, on average, our approach executes outstanding results. Generally, our approach outperforms all the competing state-of-the-art methods in the most test cases with different action sequences and executes reliable results with minimal errors and without any user interaction.
High-quality rendering of spatial sound fields in real-time is becoming increasingly important with the steadily growing interest in virtual and augmented reality technologies. Typically, a spherical microphone array (SMA) is used to capture a spatial sound field. The captured sound field can be reproduced over headphones in real-time using binaural rendering, virtually placing a single listener in the sound field. Common methods for binaural rendering first spatially encode the sound field by transforming it to the spherical harmonics domain and then decode the sound field binaurally by combining it with head-related transfer functions (HRTFs). However, these rendering methods are computationally demanding, especially for high-order SMAs, and require implementing quite sophisticated real-time signal processing. This paper presents a computationally more efficient method for real-time binaural rendering of SMA signals by linear filtering. The proposed method allows representing any common rendering chain as a set of precomputed finite impulse response filters, which are then applied to the SMA signals in real-time using fast convolution to produce the binaural signals. Results of the technical evaluation show that the presented approach is equivalent to conventional rendering methods while being computationally less demanding and easier to implement using any real-time convolution system. However, the lower computational complexity goes along with lower flexibility. On the one hand, encoding and decoding are no longer decoupled, and on the other hand, sound field transformations in the SH domain can no longer be performed. Consequently, in the proposed method, a filter set must be precomputed and stored for each possible head orientation of the listener, leading to higher memory requirements than the conventional methods. As such, the approach is particularly well suited for efficient real-time binaural rendering of SMA signals in a fixed setup where usually a limited range of head orientations is sufficient, such as live concert streaming or VR teleconferencing.
Microphone arrays consisting of sensors mounted on the surface of a rigid, spherical scatterer are popular tools for the capture and binaural reproduction of spatial sound scenes. However, microphone arrays with a perfectly spherical body and uniformly distributed microphones are often impractical for the consumer sector, in which microphone arrays are generally mounted on mobile and wearable devices of arbitrary geometries. Therefore, the binaural reproduction of sound fields captured with arbitrarily shaped microphone arrays has become an important field of research. In this work, we present a comparison of methods for the binaural reproduction of sound fields captured with non-spherical microphone arrays. First, we evaluated equatorial microphone arrays (EMAs), where the microphones are distributed on an equatorial contour of a rigid, spherical 1.
Second, we evaluated a microphone array with six microphones mounted on a pair of glasses. Using these two arrays, we conducted two listening experiments comparing four rendering methods based on acoustic scenes captured in different rooms2. The evaluation includes a microphone-based stereo approach (sAB stereo), a beamforming-based stereo approach (sXY stereo), beamforming-based binaural reproduction (BFBR), and BFBR with binaural signal matching (BSM). Additionally, the perceptual evaluation included binaural Ambisonics renderings, which were based on measurements with spherical microphone arrays. In the EMA experiment we included a fourth-order Ambisonics rendering, while in the glasses array experiment we included a second-order Ambisonics rendering. In both listening experiments in which participants compared all approaches with a dummy head recording we applied non-head-tracked binaural synthesis, with sound sources only in the horizontal plane. The perceived differences were rated separately for the attributes timbre and spaciousness. Results suggest that most approaches perform similarly to the Ambisonics rendering. Overall, BSM, and microphone-based stereo were rated the best for EMAs, and BFBR and microphone-based stereo for the glasses array.
Due to the COVID-19 pandemic, university students worldwide have experienced drastic changes in their academic and social lives, with the rapid shift to online courses and contact restrictions being reported among the major stressors. In the present study, we aimed at examining students’ perceived stress over the course of the pandemic as well as individual psychological and social coping resources within the theoretical framework of the Transactional Model of Stress and Coping in the specific group of STEM students. In four cross-sectional studies with a total of 496 computer science students in Germany, we found that students reported significantly higher levels of perceived stress at both measurement time points in the second pandemic semester (October/November 2020; January/February 2021) as compared to the beginning of the first (April/May 2020), indicating that students rather became sensitized to the constant pandemic-related stress exposure than habituating to the “new normal”. Regarding students’ coping resources in the higher education context, we found that both high (a) academic self-efficacy and (b) academic online self-efficacy as well as low (c) perceived social and academic exclusion among fellow students significantly predicted lower levels of students’ (d) belonging uncertainty to their study program, which, in turn, predicted lower perceived stress at the beginning of the first pandemic semester. At the beginning of the second pandemic semester, we found that belonging uncertainty still significantly mediated the relationship between students’ academic self-efficacy and perceived stress. Students’ academic online self-efficacy, however, no longer predicted their uncertainty about belonging, but instead had a direct buffering effect on their perceived stress. Students’ perceived social and academic exclusion among fellow students only marginally predicted their belonging uncertainty and no longer predicted their perceived stress 6 months into the pandemic. We discuss the need and importance of assessing and monitoring students’ stress levels as well as faculty interventions to strengthen students’ individual psychological and social coping resources in light of the still ongoing pandemic.