A unified theory is developed to predict the elastic moduli, yield stress, and stress-strain curve of polymer composites. In addition to the elastic moduli of filler and matrix, two more constants, related to the hole and activation volume, are needed as input parameters to describe the nonequilibrium mechanical properties of composites. This paper is a generalization of our earlier composite theory to incorporate the effect of structural relaxation. We have derived the compositional dependent relaxation time, and have applied it to understand the deformation kinetics, and to determine the compressive stress-strain behaviour as a function of the filler concentration, strain rate, and temperature. The type of stress applied to a system plays an important role in nonlinear deformations. We shall discuss uniaxial compression, which is not sensitive to cracks, and gives the characteristics of the pure material. As the volume fraction of filler increases, both the effective elastic modules and yield stress increases. However, the system becomes more brittle at the same time.