Aaron,

Re: your TTFB problem if you still have it.

You might try the Comet Cache (my preferred WP caching) or other caching and Autopimize plugins in tandem. Works for me.