Its the horse that makes me nervous. He's the type that's really really lazy and laid back... but then when something scares him he's really explosive and I can never tell when he's going to do it.
have you checked him for pain? Sometimes when a horse has fallen, been ridden with ill-fitting tack or for whatever other reason becomes "out" somewhere, a good tweak by a chiropractor or a massage therapist can "cure" a horse that is laid back but tends to be explosive.
He sounds like he can use some real emotional training. It's not enough sometimes to just go through the regular training route with the steering, stopping, backing up,....sometimes some horses need more.
I'm talking about some desensitizing, some spook in place lessons, and to really be pushed to relax/give to pressure when nervous.
Sometimes it's necessary to raise his emotional level and then drop it and raise it and drop it...repeatedly through proper despooking....to get rid of the boogies....to teach him to think rather than just react.
Also, loads of bending exercises that require lots of disengaging of the hips to the point where the horse moves that hip (crosses over in the back feet) by a light touch of the rein and leg cue.
All of this,...I'd get a trainer who is familar with more in-depth type training, rather than just the go forward, steering, stopping, etc... type of trainer.
Then once the horse has learned to chill out in the face of adversity....then you'd come in the picture and get lessons on how to one rein stop, how to handle the reins one rein at a time....riding in a simple snaffle....learning to speed up and stop the horse eventually with your seat first....etc....
He needs more emotional training and then you need to be trained on how to raise your confidence level. A good way to do that is by understanding horses (why they do what they do...instinct vs training, etc)....and taking lessons so you are confident that you can stop your horse any time (one rein stop to the point of cueing the horse with your seat first)....can do wonders.