<s>[INST] How to evaluate $\int_{0}^{\infty} \frac{x^{-\mathfrak{i}a}}{x^2+bx+1} \,\mathrm{d}x$ using complex analysis? We were told today by our teacher (I suppose to scare us) that in certain schools for physics in Soviet Russia there was as an entry examination the following integral given 
$$\int\limits_{0}^{\infty} \frac{x^{-\mathfrak{i}a}}{x^2+bx+1} \,\mathrm{d}x\,,$$ 
where $a \in \mathbb{R}$, $b \in [0,2)$, and $\mathfrak{i}$ is the imaginary unit. And since we are doing complex analysis at the moment, it can, according to my teacher, be calculated using complex methods. 
I was wondering how this could work? It seems hard to me to find a good curve to apply the residue theorem for this object, I suppose. Is there a trick to compute this integral?
 [/INST] $\newcommand{\Res}{\text{Res}}$
Firstly define:
\begin{align}
f(z) = \frac{1}{z^2+bz+1}
\end{align}
Secondly define:
\begin{align}
g(z)= (-z)^{-ia}f(z)
\end{align}
We use the Principal Log to define $(-z)^{-ia}$. The main reason for the minus sign is that I want to work with the principal Log. Consider now the keyhole contour $K_R$ consisting of $C_R \cup C_R^+ \cup C_R^-$. The circle part is $C_R$ with radius $R$, and $C_R^+$ is the segment that connects $0$ to $R$ on the right half plane from above and $C_R^-$ the same as $C_R^+$ but from below.
So our contour looks like the one below:

Note that $\int_{C_R} g(z)dz \to 0$ as $R\to \infty$ (why?). Now the poles of this function $g(z)$ is at $z_1= -\frac{1}{2}b - i\  \sqrt[]{1-\frac{1}{4}b^2}$ and $z_2 = -\frac{1}{2}b + i\  \sqrt[]{1-\frac{1}{4}b^2}$. Note that for large enough $R$ both poles will be in the area enclosed by our contour.
On $C_R^+$ we have:
\begin{align}
\lim_{R\to \infty} \int_{C_R^+} g(z) dz = e^{-a\pi}\int^{\infty}_0\frac{x^{-ia}}{x^2+bx+1} dx
\end{align}
Similarly on $C_R^-$ we have:
\begin{align}
\lim_{R\to \infty} \int_{C_R^-} g(z) dz = -e^{a\pi}\int^{\infty}_0\frac{x^{-ia}}{x^2+bx+1} dx
\end{align}
So:
\begin{align}
\lim_{R\to\infty} \int_{K_R} g(z) dz &= (-e^{a\pi} + e^{-a\pi} )\int^\infty_0 \frac{x^{-ia}}{x^2+bx+1} dx \\
&=-2\sinh(a\pi) \int^\infty_0 \frac{x^{-ia}}{x^2+bx+1} dx
\end{align}
On the other hand we have by the Residue theorem:
\begin{align}
\lim_{R\to\infty} \int_{K_R} g(z) dz = 2\pi i\left( \Res_{z=z_1}g(z)+\Res_{z=z_2}g(z)\right)
\end{align}
Let's calculate the residues:
\begin{align}
\Res_{z=z_1} g(z) = \frac{\left(\frac{1}{2}b + i\  \sqrt[]{1-\frac{1}{4}b^2}\ \right)^{-ia}}{-2i \ \sqrt[]{1-\frac{1}{4}b^2} } 
\end{align}
The other one:
\begin{align}
\Res_{z=z_2} g(z) = \frac{\left(\frac{1}{2}b - i\  \sqrt[]{1-\frac{1}{4}b^2}\ \right)^{-ia}}{2i \ \sqrt[]{1-\frac{1}{4}b^2} } 
\end{align}
Define $\beta:=\frac{1}{2}b + i\  \sqrt[]{1-\frac{1}{4}b^2}$. So:
\begin{align}
\lim_{R\to\infty} \int_{K_R} g(z) dz = \pi \frac{\bar\beta^{-ia}-\beta^{-ia}}{\sqrt[]{1-\frac{1}{4}b^2}} = 2\pi \frac{\bar\beta^{-ia}-\beta^{-ia}}{ \sqrt[]{4-b^2}}
\end{align}
This means:
\begin{align}
\int^{\infty}_0 \frac{x^{-ia}}{x^2+bx+1}dx &= \frac{2\pi}{-2\sinh(a\pi)}  \frac{\bar\beta^{-ia}-\beta^{-ia}}{ \sqrt[]{4-b^2}} 
\end{align}
Note that $\beta = \exp(i\arccos(b/2))$. So $\beta^{-ia}=\exp(a\arccos(b/2))$ and $\bar\beta^{-ia}=\exp(a\arccos(b/2)).$ Substituting $\beta$ gives us the final result:

\begin{align}
\int^{\infty}_0 \frac{x^{-ia}}{x^2+bx+1}dx = \color{red}{\frac{2\pi\sinh(a\cdot\arccos(b/2))}{\sinh(a\pi) \ \sqrt[]{4-b^2}} }
\end{align}

This integral is real.

Remark
It may be interesting to note the following. For positive $x$ one has: $x^{-ia}=e^{-ia\text{Log}(x)}=e^{-ia\ln(x)}=\cos(a\ln(x))-i\sin(a\ln(x))$. So we have:
\begin{align}
\int^\infty_0 \frac{x^{-ia}}{x^2+bx+1}  dx = \int^\infty_0 \frac{\cos(a\ln(x))}{x^2+bx+1}  dx - i\int^\infty_0 \frac{\sin(a\ln(x))}{x^2+bx+1}  dx
\end{align}
Since our integral is real we get the following identities for free:
\begin{align}
\color{blue}{\int^\infty_0 \frac{\cos(a\ln(x))}{x^2+bx+1}  dx =  \frac{2\pi\sinh(a\cdot\arccos(b/2))}{\sinh(a\pi) \ \sqrt[]{4-b^2}}}
\end{align}
And:
\begin{align}
\color{blue}{\int^\infty_0 \frac{\sin(a\ln(x))}{x^2+bx+1}  dx = 0 }
\end{align}
</s>