however , unfortunately , he knew it was too soon and julia would have been furious at such an action perpetrated in front of charlotte and oliver , so he kept control of himself , but only just barely .