We construct the supersymmetric completion of quartic R + R4-actions in the ten-dimensional effective action of the heterotic string. Two invariants, of which the bosonic parts are known from one-loop string amplitude calculations, are obtained. One of these invariants can be generalized to an R + F2 + F4-invariant for supersymmetric Yang-Mills theory coupled to supergravity. Supersymmetry requires the presence of B AND R AND R AND R AND R-terms, (B AND F AND F AND F AND F for Yang-Mills) which correspond to counterterms in the Green-Schwarz anomaly cancellation. Within the context of our calculation the zeta(3)R4-term from the tree-level string effective action does not allow supersymmetrization.