Media playback is not supported on this device
The Stoke City midfielder scored with a stunning long-range strike in Wales' 2-2 draw with Austria in Vienna, but came off with a hamstring injury.
Manager Chris Coleman has confirmed Allen has joined Aaron Ramsey on the sidelines.
"Joe is a massive loss because he is a massive player for us," Taylor said.
"Obviously Joe is going to be missed. He's in great form and scored a great goal and will be a huge miss.
"But hopefully we can get the three points against Georgia, ready for when Joe slots back into the team."
Skipper Ashley Williams admits the absence of Allen is a bitter pill to swallow, but believes that whoever replaces him will produce the required level of performance.
"Obviously we want Joe to play, he is massive for us," Williams told BBC Radio Wales Sport.
"But whoever comes in will step up and do a job as required. That's how it is with us.
"Two-years ago we played against Cyprus and Bosnia without Joe and Aaron [Ramsey] and we will do so again.
"Whoever steps in will have been part of the group for a number of years and know what is required."
Striker Sam Vokes feels Wales need to beat Georgia to ensure they make the most of their hard-fought point in Vienna.
"We are happy with a point, of course. Not many people take points in Austria," he said.
"For us, we got the point and now we have something to set us up for Sunday.
"It was a good point in the end. We weren't at our best and still picked up a point. We take a positive from that.
"It is important to get the win to maximise the point in Austria. It is early doors in the group and we need to put our stamp on the group."
The absence of midfielder Joe Allen for Sunday's clash with Georgia is a 'massive' blow to Wales, according to defender Neil Taylor.