Physics-Based Sound Synthesis for Games and Interactive Systems

Session 1: The Time Domain: Sound, Digital Audio, PCM Files, Noise Vs. Pitch, A Hint Of Spectra 
a) Sound in Air, Traveling Waves b) Digital Audio, Sampling, Quantization, Aliasing c) Soundfiles, Wavetables, Manipulating PCM d) Pitch (vs. Noise), Spectral Analysis 0.1 e) Time-domain Pitch/Noise Detection: ZeroXings, AMDF, Autocorrelation

Session 2: Physics, Oscillators, Sines & Spectra, Spectral/Additive Synthesis 
a) Mass-Spring-Damper system, also simple Pendulum b) Fourier analysis/synthesis, Spectrum Analysis 1.0 c) More on additive Sine-wave synthesis

Session 3: Digital Filters, Modal Synthesis 
a) Digital Filters, Finite Impulse Response (FIR) b) Linearity, Time-invariance, Convolution c) Infinite Impulse Response (IIR) Digital Filters d) BiQuad Resonator Filter, Modal Synthesis

Session 4: Physical Modeling Synthesis: 1D Systems 
a) 1-D systems, Strings, Modal (Fourier) Solution b) Strings II: Waveguide (D’Alembert) Solution c) 1-D systems, Bars, Tubes, solutions d) Advanced Waveguide Synthesis for 1-D systems

Session 5: Physical Modeling II: 2 And 3-D Systems 
a) 2-D systems, plates, drums, higher-order modes Fourier (Sine and/or Modal) Solutions, Waveguide Solutions b) 3-D systems, rooms, resonators, Meshes, Waveguide synthesis c) Resonator/Modal view and solution of 3-D systems Pop bottles and other lumped resonators

Session 6: Subtractive Synthesis, Vocal Sounds And Models 
a)  Subtractive Synthesis, Voice Synthesis, Formants b) Linear Prediction, LPC c) FOFs d) FM Synthesis: Horns, Bells, Voices

Session 7: Grains, Particles And Statistical Models 

a) Wavelets (just for completeness) b) Granular Synthesis c) Particle Models, Statistical Modal Synthesis d) Wind, Water, Surf, and Other Whooshing Sounds

Session 8: Extending And Refining Physical Synthesis Models 
a) Waveshaping Synthesis, Distortion Modeling b) Time-Varying Systems c) Stiffness, All-Pass Filters, Banded Waveguides d) Commuted Synthesis e) JULIUS on KS, strings, demos

Session 9: Tying It All Together: Applications, Sonification, Interactions, And Control 
a) Scanned Synthesis b)  Don’t forget the laptop!!! SMELT:   c) Controlling Synthesis with game controllers (Wii, mobile TouchOSC, more) d) Walking Synthesis, a complete system e) Procedural Audio: Driving synthesis from process, game state, etc. f) Data set Sonification

• Software: ChucK (also optionally STK, PeRColate for Max/MSP, Processing, GL/Glut)
• Recommended (highly) Textbook:

Real Sound Synthesis for Interactive Applications (Kadenze discount available soon!)

• Familiarity with ChucK programming language

Introduction to Programming for Musicians and Digital Artists (Kadenze ChucK course)

Programming for Musicians and Digital Artists (ChucK book, Kadenze Discount)

• Operating system: Mac OS X, Windows, or Linux (Planet CCRMA recommended)
• Desired: familiarity with algebra. no calculus required.
• Helpful to have: some personal sound-making things: a guitar or other stringed instrument, a drum, a kitchen pan, a prayer bowl, glasses, bowls, voice...

Instructors:

Perry Cook

Perry R. Cook is Emeritus Professor of Computer Science (also Music) at Princeton University, founding advisor/consultant to social music company SMule, and consulting professor at CalArts, Stanford CCRMA, and University of Arizona. With Dan Trueman, he co-founded the Princeton Laptop Orchestra, which received a MacArthur Digital Learning Initiative Grant in 2005. With Ge Wang, Cook is co-author of the ChucK Programming Language. His newest book is “Programming for Digital Musicians and Artists,” with Ajay Kapur, Spencer Salazar, and Ge Wang. The recipient of a 2003 Guggenheim Fellowship, Cook is (still) working on a new book, "La Bella Voce e La Macchina (the Beautiful Voice and the Machine), A History of Technology and the Expressive Voice." Perry is also co-founder of Kadenze.

Julius Smith

Julius O. Smith normally teaches a music signal-processing course sequence and supervises related research at the Center for Computer Research in Music and Acoustics (CCRMA). He is formally a professor of music and (by courtesy) electrical engineering. In 1975, he received his BS/EE degree from Rice University, where he got started in the field of digital signal processing and modeling for control. In 1983, he received the PhD/EE degree from Stanford University, specializing in techniques for digital filter design and system identification, with application to violin modeling. His work history includes the Signal Processing Department at Electromagnetic Systems Laboratories, Inc., working on systems for digital communications, the Adaptive Systems Department at Systems Control Technology, Inc., working on research problems in adaptive filtering and spectral estimation, and NeXT Computer, Inc., where he was responsible for sound, music, and signal processing software for the NeXT computer workstation. Prof. Smith is a Fellow of the Audio Engineering Society and the Acoustical Society of America. He is the author of four online books and numerous research publications in his field.