<s>[INST] Integrals involving multi-valued functions Show that 
$$\int_{0}^{\infty}{\frac{\cos{x}}{x^\alpha}dx}=\mathrm{\Gamma}\left(1-a\right)\sin{\left(\frac{\mathrm{\pi\alpha}}{2}\right)}\ ,\ \ \ \ 0<\alpha<1$$
what would be the contour of this integration if I want to solve this integration using residue theorem
 [/INST] Assuming you wish to show
$$\int_0^\infty \frac{\cos x}{x^\alpha} \, dx = \Gamma (1 - \alpha) \sin \left (\frac{\pi \alpha}{2} \right ), \qquad 0 < \alpha < 1,$$
what follows is a real method that can be used.
Let
$$I = \int_0^\infty \frac{\cos x}{x^\alpha} \, dx.$$
By employing the following useful property for the Laplace transform:
$$\int_0^\infty f(x) g(x) \, dx = \int_0^\infty \mathcal{L} \{f(x)\} (t) \cdot \mathcal{L}^{-1} \{g(x)\} (t) \, dt.$$
Noting that
$$\mathcal{L} \{\cos x\}(t) = \frac{t}{1 + t^2},$$
and
$$\mathcal{L}^{-1} \left \{\frac{1}{x^\alpha} \right \} (t)= \frac{1}{\Gamma (\alpha)} \mathcal{L}^{-1} \left \{\frac{\Gamma (\alpha)}{x^{(\alpha - 1) + 1}} \right \} (t) = \frac{t^{\alpha - 1}}{\Gamma (\alpha)},$$
then
\begin{align}
I &= \int_0^\infty \cos x \cdot \frac{1}{x^\alpha} \, dx\\
&= \int_0^\infty \mathcal{L} \{\cos x\} (t) \cdot \mathcal{L}^{-1} \left \{\frac{1}{x^\alpha} \right \} (t) \, dt\\
&= \frac{1}{\Gamma (\alpha)} \int_0^\infty \frac{t^\alpha}{1 + t^2} \, dt.
\end{align}
Setting $u = t^2$, one has
\begin{align}
I &= \frac{1}{2 \Gamma (\alpha)} \int_0^\infty \frac{u^{\frac{\alpha}{2} - \frac{1}{2}}}{1 + u} \, du\\
&= \frac{1}{2 \Gamma (\alpha)} \operatorname{B} \left (\frac{1}{2} - \frac{\alpha}{2}, \frac{1}{2} + \frac{\alpha}{2} \right ) \tag1\\
&= \frac{1}{2 \Gamma (\alpha)} \Gamma \left (\frac{1}{2} - \frac{\alpha}{2} \right ) \Gamma \left (\frac{1}{2} + \frac{\alpha}{2} \right ) \tag2\\
&= \frac{1}{2 \Gamma (\alpha)} \Gamma \left [1 - \left (\frac{1}{2} + \frac{\alpha}{2} \right ) \right ] \Gamma \left (\frac{1}{2} + \frac{\alpha}{2} \right )\\
&= \frac{1}{2 \Gamma (\alpha)} \frac{\pi}{\sin \left (\frac{\pi}{2} - \frac{\pi \alpha}{2} \right )} \tag3\\
&= \frac{1}{2 \Gamma (\alpha)} \frac{\pi}{\cos \left (\frac{\pi \alpha}{2} \right )}\\
&= \frac{\Gamma (1 - \alpha) \sin (\pi \alpha)}{2 \pi} \cdot \frac{\pi}{\cos \left (\frac{\pi \alpha}{2} \right )} \tag4\\
&=\frac{\Gamma (1 - \alpha) \sin \left (\frac{\pi \alpha}{2} \right ) \cos \left (\frac{\pi \alpha}{2} \right )}{\cos \left (\frac{\pi \alpha}{2} \right )}\tag5\\
&= \Gamma (1 - \alpha) \sin \left (\frac{\pi \alpha}{2} \right ),
\end{align}
as required to show.
Explanation
(1) Using $\operatorname{B} (x,y) = \displaystyle{\int_0^\infty \frac{t^{x - 1}}{(1 + t)^{x + y}} \, dt}$.
(2) Using $\operatorname{B}(x,y) = \dfrac{\Gamma (x) \Gamma (y)}{\Gamma (x + y)}$.
(3) Using the reflexion formula for the gamma function: $\Gamma (1 - z) \Gamma (z) = \dfrac{\pi}{\sin (\pi z)}$.
(4) Again using the reflexion formula for the gamma function.
(5) Using the double angle formula for sine.
</s>