If you want to improvise with great ease, you HAVE to learn the notes on the fretboard as well as possible. That’s a process that can take some time, but with the right exercises it doesn’t take as long as most guitarists think. Although patterns are good and helpful it will help you extremely to think mostly about notes (and not too much about patterns) when you improvise. So 2 are the very basic tools you need. 1) learn the notes on your instrument EXTREMELY well and 2) learn some theory (scales, chord construction etc). Learning chord construction and being able to think (almost) immediately which notes belong to each chord that you try to solo over will make your solos more musical. (Learning arpeggios will also help very much)
I hope this helps a bit. IT HELPED ME! You can check out some older stuff of mine at http://www.dimitriskyriakidis.com to see wether these hints helped or not.