Two things come to mind - the spring on the ebrake shoes isn't pulling the shoes back from the drum reliably, and your ebrake may be adjusted too tight. There should be 7-9 clicks as you pull out the ebrake handle.
I would drop the driveshaft and study the ebrake. Drum brakes tend to be...