Do you have mogrify on your server? If so you would set the correct path in the config options in admin and in global options set your processor to Imagemagick. If not all you can do is limit filesizes and raise memory in php. Basically with GD2 it is about compromise to what the server handles.
http://www.photopost.com/forum/showp...50&postcount=2