The buffer vs. true bypass will not effect the overall volume output, so that is not the issue. You need to check your "gain setting" resistors. If you used an incorrect value that could lower the output significantly.
Double check these values: R9, R10, R11, R15, R19, R22.
Also, make sure you aren't using 56k or 560k on R24 instead of 560R.
If these all check out, then you may need to audio probe the outputs of the ICs to see where the volume is dropping off. This would be pin8, 14, 7 and 1 in that order (of the quad amp, not the charge pump).