<s>[INST] Question related to the following function $f\left( x \right) = \frac{{b - x}}{{1 - bx}}$ Let $f:\left( {0,1} \right) \to R$ be defined by $f\left( x \right) = \frac{{b - x}}{{1 - bx}}$
, where 'b' is a constant such that $0 < b < 1$. Then
(A) $f$ is not invertible on (0,1)
(B) $f\ne f^{-1}$ on (0,1)and $f'(b)=\frac{1}{f'(0)}$
(C) $f= f^{-1}$ on (0,1)and $f'(b)=\frac{1}{f'(0)}$
(D) $f^{-1}$ is differentiable on (0,1)
My approach is as follow
$f:\left( {0,1} \right) \to R$, $f\left( x \right) = \frac{{b - x}}{{1 - bx}}$,$0 < b < 1$
$f'\left( x \right) = \frac{{\left( {1 - bx} \right)\frac{d}{{dx}}\left( {b - x} \right) - \left( {b - x} \right)\frac{d}{{dx}}\left( {1 - bx} \right)}}{{{{\left( {1 - bx} \right)}^2}}}$
$f'\left( x \right) = \frac{{\left( {1 - bx} \right) \times  - 1 - \left( {b - x} \right) \times \left( { - b} \right)}}{{{{\left( {1 - bx} \right)}^2}}} = \frac{{\left( {1 - bx} \right) \times  - 1 + \left( {b - x} \right) \times b}}{{{{\left( {1 - bx} \right)}^2}}} = \frac{{\left( { - 1 + bx} \right) + \left( {{b^2} - bx} \right)}}{{{{\left( {1 - bx} \right)}^2}}} = \frac{{{b^2} - 1}}{{{{\left( {1 - bx} \right)}^2}}}$
$x\in(0,1)$ and $b\in(0,1)$, then $bx\in(0,1)$, hence $f'(x)<0$, hence my assumption is that $f$ is invertible as it is decreasing function when $0<x<1$ but official answer is (A) that is $f$ is not invertible on (0,1)
 [/INST] Just observe that there is no $x$ such that $f(x) =\frac{1}{b}$. If any such $x$ exists then it would imply that $b^2=1$ which is certainly not possible. This shows that $f$ is not surjective which implies that $f$ is not invertible.
Edit:
One more way to see that $f$ is not surjective is to observe that $-1 < f(x) < b$ by simply manipulating the inequalites $0<x<1$ and $0<b<1$.
</s>