Gerrard will leave the Reds when his current deal expires next summer, despite being offered a new contract.
Carragher said the 34-year-old could have been made to feel more "wanted" or been tempted by a coaching role to go with his playing responsibilities.
Liverpool boss Brendan Rodgers said he wanted Gerrard to stay and that the midfielder wanted to carry on playing.
Gerrard will now move to the United States and has been linked with a move to Major League Soccer side LA Galaxy.
He joined Liverpool aged nine and has gone on to win the Champions League, Uefa Cup, two FA Cups, three League Cups, one Community Shield and two Uefa Super Cups during his time at the club.
However, Rodgers has rested him on several occasions this season when he has started with his captain on the bench.
Media playback is not supported on this device
"Steven is not someone who would be comfortable just sitting on the bench and, in that sense, he has made the right decision," said Carragher, writing in his column for the Daily Mail.
"Still, I cannot help feeling Liverpool's hierarchy should have done more to ensure he remained at Anfield."
The 36-year-old added:  "Over the next 12 months Steven could have been given a role on the staff to combine with his playing duties.
"It would have been like work experience, with him shadowing Brendan Rodgers, looking at how the academy is run - all the different aspects of the club.
"At the end of the year, it might have been that Steven wasn't at the right level to be a coach or he could have decided that coaching wasn't for him.
"But I look at what is happening with Ryan Giggs at Manchester United now and I am dismayed that Liverpool are letting that experience leave."
Rodgers, who took over as Liverpool boss in June 2012, said Gerrard wanted to play for a couple more years.
"Naturally, at this level that was going to be tapered a little bit over the next couple of years," the Reds manager said.
"He wasn't ready to move into coaching or anything like that yet. He still sees himself very much as a player."
Former Liverpool defender Jamie Carragher believes his old club should have done more to keep Steven Gerrard.