Title: Lifelong Learning the Task-Parameter Relationships for Knowledge Transfer. Abstract: In this work, we motivate the focus on knowledge transfer in the lifelong learning setting. We propose a heuristic that learns \textit{`task-parameter relationships'} as the expected adaptation rate over K steps of a (parameter, task) pair. We leverage this heuristic to exploit the similarities between the optimal weight spaces of tasks, allowing for the possibility of forward and backward knowledge transfer. These task-parameter relationships are learned by an auxiliary network trained on intermittent explorations of parameter space during the continual learning stream. We show that the benefits to knowledge transfer come with no increase in overall model capacity, while naturally mitigating catastrophic forgetting.