smith loved his brothers and sisters , but only with his mother had he ever admitted just how difficult his extreme fame had been for him at times , especially when it had reached a point where he could no longer go where he wanted , where he felt trapped under a magnifying glass .