\documentclass[12pt]{article}
\usepackage[pdftex]{color, graphicx}
\usepackage{amsmath, amsfonts, amssymb, mathrsfs}
\usepackage{lscape}
\usepackage{hangcaption}
\usepackage{dcolumn}
\usepackage{natbib}
\bibpunct{(}{)}{;}{a}{}{,}

\oddsidemargin=0.25in
\evensidemargin=0.25in
\textwidth=6in
\textheight=8.75in
\topmargin=-.5in
\footskip=0.5in

\newcounter{fig}
\newcounter{tab}
\newcounter{app}
\date{}


\title{\begin{center}Statistics 5114: HW 0\end{center}}

\begin{document}
\maketitle
\newcommand{\argmin}{\text{argmin}}
\noindent This assignment will help you get over some of the calculus and probability hurdles that often present themselves in class. 
We won't require formal proofs in class, but being able to prove things formally is a useful skill. Even drawing pictures can be acceptable if the ideas are right, 
and the concepts are reasonably well explained. During class, I'll make clear what needs to be proved/etc.
Enjoy!!! {\bf Select \underline{5 problems of your choosing} and write up solutions (neatly; \LaTeX  is preferred). You'll turn these in (on paper) on the due date assigned}.


\bibliographystyle{bioinformatics}

\begin{enumerate}

\item Show that $lim_{n\rightarrow \infty}(1+\frac{1}{n})^n\rightarrow e$.
\item Show that $lim_{n\rightarrow \infty}(1+\frac{f(x)}{n})^n\rightarrow e^{f(x)}$.
\newline\newline
Note that these result can be shown several different ways, and in some classes is taken as ``definition". Just review the result and know it:) A math major might disagree with a proof being correct/incorrect, but we won't be that rigid in class.
\item Review Jensen's inequality which shows a relationship between $E[g(x)]$ and $g(E[x])$  (either $\geq$ or $\leq$), where $g(\cdot)$ is either a convex or concave function.
\item Prove that the inequality switches to an equality in Jensen's inequality if and only if $g(\cdot)$ is a linear function. Note that you don't need to use Jensen's to prove that, since $E[x]$ is a ``linear operator", but this will help you to think through Jensen's.
\item Let $x_1$ and $x_2$ have a bivariate Normal distribution, which can be expressed by:
\[\begin{pmatrix}
x_1 \\
x_2
\end{pmatrix}\sim N\left(\begin{pmatrix}
\mu_1 \\
\mu_2
\end{pmatrix},\begin{pmatrix}
\Sigma_{11} & \Sigma_{12} \\
\Sigma_{21} & \Sigma_{22}
\end{pmatrix}\right),
\] where $\Sigma_{21}=\Sigma_{12}^T$. Note that I'm being really sloppy about the dimensionality of $x_1, x_2, \mu_1, \mu_2, \Sigma_{11}, \Sigma_{12}, \Sigma_{21}, $ and $\Sigma_{22}$. I'd like you to label the dimensions and make sure everything is ``conformable" (look that word up if you don't know it:)).\newline\newline
Find the conditional distribution for $x_1|x_2$, which can be verbally expressed as $x_1$ given $x_2$. Note that when we say the words ``given" $x_2$, $x_2$ is fixed/constant.
\item Prove that the variance for $x_1|x_2$  is smaller or equal to the marginal variance of $x_1$.
\item Relate the mean/expected value of $x_1|x_2$  to what you have learned in your Regression/Anova class. 
\item This is a new problem, and the probability distribution does not matter. Prove that $f(x_1|x_2) = f(x_1)$ is an equivalent to saying $f(x_1,x_2)=f(x_1)f(x_2)$ (i.e $x_1$ and $x_2$ are independent).
\item Review Bayes' Theorem.
\item Review likelihood functions. Here's a result: if $x_i\sim N(\mu,\sigma^2)$ ($x_i's are independent$ and let $\sigma^2$ is a fixed known constant,; Note that if I don't say they are independent in my upcoming class, and I don't explain any dependence ``mechanism", you can assume everything is independent:)), the likelihood function for $\mu$ is $\prod_{i=1}^ne^{-\frac{1}{2}(x_i-\mu)^2/\sigma^2}$, which a function of $\mu$ (this is not a function of the $x_i's$ since for likelihood functions ``condition" on the data. That is, the $x_i's$ are considered fixed/observed). 
\newline\newline
Why did I drop the leading term $\frac{1}{\sqrt{2\pi\sigma^2}}$? 
\item Show that the above likelihood function is equivalent to $Ce^{-\frac{1}{2}(\mu-\bar x)^2/(\sigma^2/N)}$, where $C$ can be any positive constant.
\item Review Taylor's Series/Power Series, which states something like:
$$ f(x) = a_0 + a_1(x-a) + a_2(x-a)^2 + a_3(x-a)^3 + ... + a_n(x-a)^{n},$$
where the coefficients are:
$$
a_n = \frac{f^{(n)}(a)}{n!}.
$$ 
\item What does Taylor's \underline{Theorem} say about the \underline{remainder term}? What is the ``radius of convergence"?
%\text{[where }\left|x-a\right| < R\text{]}

%\newline\newline
\textbf{Let's spend a little effort to get ahead on some problems:) This is optional.}

\item This is probably a new result for you, but it's worth studying:) Use Google/Stack Exchange/Quora to help you look up the terms and help you through the result. This is not in Casella and Berger. \newline\newline
Prove that
if $\theta\sim p(\theta)$ and $Y_1,\dots, Y_n$ are conditionally i.i.d. given $\theta$, then marginally (unconditionally on $\theta$),
$Y_1,\dots, Y_n$ are \underline{exchangeable}. I like this result a lot:)
\item An assignment that usually gives students a bit of trouble is my EM algorithm project which can be found at (https://leman.stat.vt.edu/VTCourses/5114EM.pdf). Have a look at Casella and Berger, and see if you can understand the algorithm. The explanation in the book is a little difficult to follow. Have a look at my video series on this, which can be found starting from the link (https://www.youtube.com/watch?v=zrZjZ7Sp6-Q). I have lots of videos on that YouTube page, so if you're really looking to get ahead, you can scroll through my previous lectures.
\item Have a look at last years HW assignments, which can all be found at:\newline 
https://leman.stat.vt.edu/VTCourses/5114Schedule.html
\end{enumerate}


\end{document}




