You should use reservoir capacitors in parallel with the batteries - the current from the batteries relies on a chemical reaction, this inevitably will not be as instantaneous as may be required.
You might want to get some ideas about existing amplifiers notably the hiraga design using this battery/capacitor arrangement from;