An output stage serves 2 or 3 purposes.

1) it filters out digital hash; high frequency noise from the DAC

2) it provides an impedance buffer for the DAC's output stages

3, if the stage is a proper I/V stage) if provides a low-impedance load to the DAC, so that it acts as a current-source (rather than a voltage source), yielding increased performance.

Having said that, it is possible to connect the Buffalo DAC directly to an amplifier. You will lose the filtering (#1). The amplifier will generally have a higher input impedance, so #2 will not be much of an issue, but by the same token, for #3, the performance is still very good when acting as a voltage-source, but higher-impedance loads can cause noise to be generated in the DAC chips output stages, so you do take a performance hit.

Also, you will want the amplifier to have balanced inputs, so you can take advantage of the cancellation of common-mode distortion/noise artifacts. Without that, performance will be significantly worse.

The bottom line is, if you like the way it sounds, that's all that really matters.