After coaching the Paraguay National team, Markarian returned to Greece and became the head coach of Panathinaikos in 2002 reaching the quarter finals in the UEFA Champions League. Markarian resigned his post as head coach in May, 2002 only to return in October saying that he "needed time off to collect his thought and relax". The following season, Markarian led them to the quarter finals of the UEFA Cup but lost to Jose Mourinho's FC Porto. In 2005, the Iraklis Thessaloniki Greek team hired him to help improve their first division standings.