——Leveraging algorithms to enhance the connection between teachers and students
About the project
Practice Advisor is a project I am excited to be a part of, having been invited by saxophonist Juchen Wang to collaborate alongside Jimmy Zhu, a fellow student from CMU. This project transforms my initial research into a robust musical instrument practice software. Our goal is to begin with the saxophone and eventually expand our software to encompass all areas of instrumental education, addressing the challenge of unequal access to musical educational resources.
In this endeavor, my role involves algorithmic coding and data analysis, leveraging various analytical tools developed during my research on clarinet timbre. After discussions with the team, we’ve decided to primarily focus on analyzing aspects of timbre, especially its brightness. We are methodically advancing the project, developing each module individually, with timbre being a key focus. Additionally, we are exploring potential applications for these modules in various contexts.
Our initial approach used spectral centroid to represent the overall trend in timbre brightness. We are now looking to refine this by integrating additional spectral points that were previously averaged out by the spectral centroid, providing more accurate corrections to timbre brightness. Furthermore, we plan to use variations in amplitude and fundamental frequency to estimate the current pitch and note, aiming for effective score following. This step-by-step development reflects our commitment to creating a comprehensive and impactful tool for instrumental learning.
Completed in collaboration with project founder Juchen Wang and fellow researcher Qixiao Zhu.
Research Process and Intermediate Results
Figure: The latest version of our structure , developed after extensive consultations and refinements
Figure: The initial project idea from Juchen
Figure: The first part of the project
Codes
About the Codes
Currently, we have two main codes. The first is a real-time recognition program I wrote in Python, where I utilized PyAudio to achieve real-time FFT and performed basic pitch estimation. The second involves analyzing the timbre of sustained notes in six dimensions during their stable phase. In the later stages, we plan to collect data to enable these six dimensions to work together in determining the color of sound.
Six Dimensions Color Analysis
Click or drag to the floating window to download
Real-time Recognition
Figure: Real-Time Analysis
Click or drag to the floating window to download