Your idle solenoid should cut off fuel to the idle circuit when the ignition is shut off. In the case of run on or dieseling, the solenoid would be stuck open and it is normally closed. You might try removing the thing and cleaning it with some carb/choke spray cleaner.
Now when you say "removing the thing" do you mean the carburetor? I was just reading up on the carb adjusting and I think I will have to set aside some time to tinker with it. Maybe even do a rebuild. :-[