Backward error analysis has become an important tool for understanding the long time behavior of numerical integration methods. This is true in particular for the integration of Hamiltonian systems where backward error analysis can be used to show that a symplectic method will conserve energy over exponentially long periods of time. Such results are typically based on two aspects of backward error analysis: (i) It can be shown that the modified vector fields have some qualitative properties which they share with the given problem and (ii) an estimate is given for the difference between the best interpolating vector field and the numerical method. These aspects have been investigated recently, for example, by Benettin and Giorgilli in [ J. Statist. Phys., 74 (1994), pp. 1117--1143], by Hairer in [Ann. Numer. Math., 1 (1994), pp. 107--132], and by Hairer and Lubich in [Numer. Math., 76 (1997), pp. 441--462]. In this paper we aim at providing a unifying framework and a simplification of the existing results and corresponding proofs. Our approach to backward error analysis is based on a simple recursive definition of the modified vector fields that does not require explicit Taylor series expansion of the numerical method and the corresponding flow maps as in the above-cited works. As an application we discuss the long time integration of chaotic Hamiltonian systems and the approximation of time averages along numerically computed trajectories.