\documentclass[12pt]{article}
\usepackage{graphicx}
\usepackage{float}
\usepackage{hyperref}
\usepackage{multirow}
\usepackage{amsmath}
\usepackage{enumitem}
\usepackage{subcaption}
\usepackage{url}
\usepackage{array}

\usepackage{listings}
\lstset{%
%backgroundcolor=\color{cyan!10},
basicstyle=\ttfamily,
numbers=left,numberstyle=\scriptsize
}

% adjust margins
\addtolength{\oddsidemargin}{-.5in}
\addtolength{\evensidemargin}{-.5in}
\addtolength{\textwidth}{1in}

\addtolength{\topmargin}{-.5in}
\addtolength{\textheight}{1in}

\renewcommand{\arraystretch}{2}


\title{Impulse Response and Auralization}
\author{EE 224: Signals and Systems I}
\date{}

\begin{document}

\maketitle

\section{Overview}
``Auralization'' is the process of using a computer to create or reproduce sound, often to achieve a desired sound effect. This is done by simulating characteristics of a room or other environment in order to produce sound that seems to have been recorded in that environment. Auralization is an increasingly popular technique for accurately simulating reverberation using acoustic impulse responses (IRs) from interesting buildings, spaces, and other sources. It is also used in video game and film production to create realistic sound effects.

\emph{Bring headphones to this lab!}

\section{Learning Objectives}

By the end of this lab, students will be able to:
\begin{enumerate}
\item Use a real-world impulse response to simulate different environments using discrete-time convolution
\item Describe FM modulated signals and chirp signals
\item Interpret a spectogram
\end{enumerate}

\section{Pre-Lab Reading}

\subsection{Auralization and Background}

EchoThief (\url{http://www.echothief.com}) collects interesting IRs from ``unusually clamorous places'' for anyone to use.  You can read about how these IRs are created \href{http://www.echothief.com/echothief/}{here}.  A similar process is described in Section~\ref{sec:sigacq}  This lab will use the computed impulse responses to approximate the sound of a signal in different environments. 

\subsubsection{Signal Acquisition}
\label{sec:sigacq}

Ideally, we could obtain the acoustic impulse response of a space by producing an impulse signal and measuring the sound it produces in a given space. Unfortunately, impulse functions can only be approximated in the real world. One way to do this is to produce a short, tall (i.e., loud) pulse, but this requires a large amount of power (equal to the energy of the pulse divided by the pulse width). Requiring both more energy and a shorter pulse makes electrical power handling a limiting factor in such a system. The output stage of a transmitter can only handle so much power without destroying itself.

Chirp signals provide a way of breaking this limitation while still exciting a broad range of frequencies. (Knowing the impulse response is equivalent to knowing how the system responds to a sinusoid at any frequency.) A chirp signal is a sinusoid whose frequency changes from a starting frequency to an ending one. To generate a chirp, an impulse is passed through a chirp system. After the chirp echoes are received, the signal is passed through an antichirp system which converts the chirp echoes to an impulse response as shown in Figure~\ref{fig:waveshape} below. This phenomenon is called waveshaping. Often a logarithmically swept sine wave is used as a chirp signal. 




%In order to characterize an acoustic space, the experimenter needs to provide an excitation signal at a wide range of frequencies. For realistic responses, the acquisition process looks like Figure~\ref{fig:sigacq}. We will ignore the non-linear memoryless piece of the system as the theory is beyond what we can do in EE~224. 

%\begin{figure}[h]
%\centering
%\includegraphics[width=5.5in]{IRmeasurement.PNG}
%\caption{An impulse response measurement system where $h(t)$ represents the desired impulse response.}
%\label{fig:sigacq}
%\end{figure}


\begin{figure}[h]
\centering
\includegraphics[width=5.5in]{waveshape.PNG}
\caption{An impulse response measurement system}
\label{fig:waveshape}
\end{figure}

\subsubsection{Chirp Signals}

Many interesting signals can be produced by changing the argument of a generalized sinusoid as different functions of time. This is called \emph{FM synthesis} or \emph{Frequency Modulation}\footnote{In FM radio, the argument of the sinusoid contains the audio signal.} and is used to create instrument simulations, interesting sound effects, improve performance of radar systems, etc. A chirp signal is a sinusoid whose frequency sweeps from a starting frequency to an ending one. A ``standard'' sinusoid has 
\begin{align*}
x(t) &= A \cos{(\Psi(t))} & \Psi(t) = \omega t + \phi
\end{align*}
However, much more interesting signals can be created with different functions such as a quadratic, logarithmic, or sinusoid as shown below. 
\begin{align*}
\Psi(t) &= 2\pi \mu t^2 + 2 \pi f_0 t + \phi \\
\Psi(t) &= e^{a t} + 2 \pi f_0 t + \phi \\
\Psi(t) &= \cos{(2\pi f_1 t)} + 2 \pi f_0 t + \phi
\end{align*}

The frequency spectra of these signals are hard to analyze; however, a decent approximation of signal behavior can be found by looking at the first derivative of the argument function. This is known as the \emph{instantaneous frequency}, $f_i$, of the signal.  The instantaneous frequency of the linear, quadratic, and logarithmic chirp are:
\begin{align*}
Linear: && f_i(t) = f_0 + \beta t && \beta = \frac{f_1-f_0}{t_1} & \\
Quadratic: && f_i(t) = f_0 + \beta t^2 && \beta = \frac{f_1-f_0}{t_1^2} & \\
Logarithmic: && f_i(t) = f_0 \beta^t && \beta = \left(\frac{f_1}{f_0}\right)^{1/t_1} &
\end{align*}

Figure~\ref{fig:chirpsys} below shows the linear chirp system. For the problem of auralization, a logarithmically swept sinusoid is used.
\begin{figure}[h]
\centering
\includegraphics[trim = 0 40 0 0, clip, width=6in]{chirpsystem.PNG}
\caption{Impulse response of a linear chirp signal (from Figure 11-10 of The Scientist and Engineer’s Guide to Signal Processing by Steven W. Smith, \cite{smith})}
\label{fig:chirpsys}
\end{figure}

These signals can be generated using the Matlab command \texttt{chirp}.

\texttt{y = chirp(t,f0,t1,f1,`method',phi)}

\noindent
The command generates samples of a swept-frequency cosine signal at the time instances defined in array \texttt{t}, where \texttt{$f_0$} is the instantaneous frequency at time $0$, and \texttt{$f_1$} is the instantaneous frequency at time $t_1$. \texttt{$f_0$} and \texttt{$f_1$} are both in hertz. If unspecified, $f_0 = e^{-6}$ for logarithmic chirp and $0$ for all other methods, $t_1=1$, and $f_1=100$.

\subsubsection{Spectrograms}

In addition to looking at signals in the time domain, it is often useful to look at the spectrum of a signal. A signal's spectrum shows which frequencies are present in the signal. A constant frequency sinusoid spectrum consists of two impulse functions at $\pm 2 \pi f_0$. For more complicated signals, there may be many spikes. For even more complicated signals such as music or frequency modulated signals, the spectrum changes with time. In these cases, the spectrogram of a signal is used instead of a spectrum. A spectrogram is found by estimating the spectrum over multiple short windows of time. The magnitude of the spectrum over these time ``windows'' is plotted as intensity or color on a two dimensional plot with time on one axis and frequency on the other.

\begin{sloppypar}
In Matlab, the function \texttt{spectrogram} will be used to compute the spectrogram.  The spectrogram function divides a signal into segments. Longer segments provide better frequency resolution; shorter segments provide better time resolution. For more information, see \url{http://www.mathworks.com/help/signal/examples/practical-introduction-to-time-frequency-analysis.html}. There are theoretical limits on how well short pieces of the signal can represent the frequency content in a signal. Generally, longer time windows give better frequency resolution. The spectrogram function can be called as follows: \texttt{spectrogram(xx, 1024, 512, [], Fs)}. The first argument, \texttt{xx}, is the time signal, 1024 is the number of samples in the time window, 512 is the number of samples of overlap between the time windows, and \texttt{Fs} is the sampling frequency. \emph{NOTE: the spectrogram function requires the Matlab signal processing toolbox.}
\end{sloppypar}

In order to see what the spectrogram function does, run the following code.
\begin{lstlisting}
N = 1024;
n = (0:N-1);
w0 = 2*pi/5;
x = sin(w0*n)+10*sin(2*w0*n);
spectrogram(x,128,64,[],100);
\end{lstlisting}
To see the spectrogram of a chirp, replace \texttt{x = sin(w0*n)+10*sin(2*w0*n);} with a chirp signal.

%You are now ready to answer the pre-lab questions on Canvas.

\section{Lab Exercises}

\subsection{Using a Computed Impulse Response and Convolution}
\label{sec:IRconv}

Implement sound effects on a provided sound clip and impulse responses of your choice.

\begin{enumerate}[label=(\alph*)]
\item Go to \href{http://www.echothief.com}{EchoThief} and select an interesting location.  Download the impulse response associated with the place you chose.  The file you downloaded should be a \emph{.wav} file.
\item Examine your impulse response.  To do this, you will first need to load it into MATLAB using \texttt{[v, Fs] = audioread(file.wav)}.  What is the sampling rate of this impulse response?  How many samples long is your IR?  How long in seconds is your IR?  Plot your IR as a function of time, with the time in seconds.  Comment on the plot in your report.  Is this a mono or stereo recording?  You can listen to your IR by typing \texttt{soundsc(v, Fs)} in MATLAB.
\item Download one of the two voice files from Canvas (female voice: \emph{spfe49\_1.wav} or male voice: \emph{spme50\_1.wav}).  Load this audio file into MATLAB.  What is the sampling rate of this file?  Is this a mono or stereo recording?  How many samples are in the signal?  Listen to the audio.  What is the voice talking about?
\item Now for the fun part!  We want to make it sound like the voice is coming from the location corresponding to your IR.  We can do this by convolving the speach signal with the impulse response.  Suppose your voice signal is $v[n]$ and the IR is $h[n]$.  The convolution $y[n] = v[n]*h[n]$ will produce a speach signal that sounds like it was recorded at the location you picked.  It MATLAB, you can type \texttt{y = conv(v,h)} to convolve the signals.
\begin{enumerate}[label=(\roman*)]
\item NOTE 1: make sure your voice signal and IR are recorded at the same sampling rate, otherwise you will need to use \texttt{resample} on one of them.
\item NOTE 2: if your voice signal or IR (or both) were recorded in stereo, you will have to pick one of the two channels to use before calling \texttt{conv}.  You can select the first column of a matrix $\mathbf{h}$ by typing \texttt{h1 = h(:,1)}.
\end{enumerate}
\item You can play back your new voice by typing \texttt{soundsc(y,Fs)}.\footnote{Be sure to use \texttt{soundsc}.  The \texttt{sound} function expects to get data ranging from -1.0 to 1.0 (see the help documentation), so if your audio signal exceeds 1.0 in magnitude, you will here additional distortion not caused by your IR.}  Describe what you hear in your report.
\item Feel free to experiment with IRs from different locations and different sound files, but leave enough time to finish Section~\ref{sec:FM}.
\end{enumerate}

%Demonstrate your result to your TA and 

\subsection{Investigating Frequency Modulated (FM) Signals}
\label{sec:FM}

A chirp signal is also known as a swept frequency cosine. The frequency of the cosine changes in time. The frequency can change in many different ways as seen in Figure~\ref{fig:spectrograms} for linear and quadratic sweeps. We will focus on a linear sweep where the instantaneous frequency is linear as shown in Figure~\ref{fig:linchirp}. Figure~\ref{fig:quadchirp} shows the case where the instantaneous frequency is a quadratic.
\begin{figure}[h]
\centering
\begin{subfigure}[b]{0.45\textwidth}
         \centering
         \includegraphics[width=\textwidth]{linearChirp.PNG}
         \caption{Linear chirp}
         \label{fig:linchirp}
\end{subfigure}
%\hfill
\begin{subfigure}[b]{0.45\textwidth}
         \centering
         \includegraphics[width=\textwidth]{quadraticChirp.PNG}
         \caption{Quadratic chirp}
         \label{fig:quadchirp}
\end{subfigure}
\caption{Spectrograms of a linear and quadratic chirp}
\label{fig:spectrograms}
\end{figure}

\begin{enumerate}[label=(\alph*)]
\item {Use the Matlab \texttt{chirp} command to generate a signal. Find the spectrogram using a window length of 2048 using the command:
\begin{lstlisting}
spectrogram(x,2048,1024,[],Fs,`yaxis'); 
\end{lstlisting}
Include this plot in your lab report and comment on the shape of the chirp.}
\item What is the frequency value at $t=1.0$ sec for the chirps in Figures~\ref{fig:linchirp} and~\ref{fig:quadchirp}?
\item As a test case, generate a chirp sound whose frequency starts at 400~Hz and ends at 4000~Hz with a duration of seven seconds and a sampling rate of 44.1~kHz. Listen to the sound that the chirp makes using the \texttt{soundsc} command in Matlab and include a listing of the commands that you used to generate the chirp signal and spectrogram.
\item Synthesize a ``chirp'' signal with the following parameters: time duration of five seconds with a sampling frequency of 11,025 Hz, starting frequency of 5000~Hz, and ending frequency of 20~Hz. Listen to the signal; what does it sound like (e.g., is the frequency movement linear)? Does it chirp down or up? Create a spectrogram of the signal to verify that you have the correct instantaneous frequency.
\item Synthesize a ``chirp'' signal with the following parameters: time duration of ten seconds with a sampling frequency of 11,025~Hz, starting frequency of 20~Hz, and the frequency at time $t=1$ sec should be 1200~Hz. Listen to the signal; what does it sound like (e.g., is the frequency movement
linear)? Does it chirp down or up? Create a spectrogram of the signal to verify that you have the correct instantaneous frequency. Speculate on what is happening with the signal.
\end{enumerate}
 
%Show your spectrograms to a TA and have the TA sign the verification sheet.






\subsection{Report Checklist}

Be sure the following are included in your report.

\vspace{\baselineskip}

\noindent
Section~\ref{sec:IRconv}
\begin{enumerate}
\item {Characteristics of the impulse response you chose (length in sec, length in samples, sampling rate)}
\item {Plot of your IR and comments about it}
\item {Description of the voice you chose (sampling rate, mono/stero, number of samples, etc.)}
\item {Description of your sound effect}
\end{enumerate}

\noindent
Section~\ref{sec:FM}
\begin{enumerate}
\item {Plot of your chirp with comments}
\item {Frequency values from Figure~\ref{fig:spectrograms}}
\item {Spectrogram of seven second chirp with code in the appendix}
\item {Description of five second chirp with spectrogram}
\item {Description of ten second chirp with spectrogram}
%\item {Include your spectrograms in your report}
\end{enumerate}


%\section{References}

\begin{thebibliography}{9}
\bibitem{echo}
``EchoThief Impulse Response Library,''
\textit{EchoThief},
SuperHoax,
\\\url{http://www.echothief.com/}
 
\bibitem{smith}
Steven W. Smith.
\textit{The Scientist and Engineer's Guide to Digital Signal Processing},
California Technical Publishing, San Diego, CA, 1997.

\bibitem{vorlander}
Michael Vorl\"{a}nder.
\textit{Auralization : Fundamentals of Acoustics, Modelling, Simulation, Algorithms and Acoustic Virtual Reality},
Springer, Berlin, Germany, 2008.


\end{thebibliography}


\end{document}