Rigby Graham, who died in May, aged 84 , was a teacher, writer, painter, woodworker and printmaker.
Festival organisers said he was also known for his letters which could be so scathing recipients were unsure whether to bin or frame them.
The artist once described someone's art as having the structure of a digestive biscuit which had been urinated on.
The University of Leicester's eighth Literary Leicester festival is a celebration of the written and spoken word.
Publisher of Graham's autobiography Mike Goldmark is one of the speakers.
He worked with Graham for nearly three decades and described him as "unlike any man I have ever encountered".
"The artist's biography, Against the Grain, summed up the way in which he had lived his life", he said.
Mr Goldmark said: "Rigby Graham had taught education until his mid-50s, a subject about which he said he knew little and cared for even less.
"Yet for years after his retirement ex-students visiting the gallery in Uppingham [where some of his art was on show] would ask after him, and he remembered every one of them."
However, he said: "Graham was not one to suffer fools, once describing someone's art as having the structure of a p****d-on digestive biscuit".
"Nothing was ever rushed and there was no compromise.
"Nor did he ever cost his time - he would be more likely to ask the colour of the buyer's socks," said Mr Goldmark.
The talk was taking place at the University of Leicester on Saturday, accompanied by a display of Graham's work.
Source: Goldmark Gallery
A literary festival is celebrating the "extraordinary contribution to British art" of a Leicester artist and writer.