We performed a series of three-dimensional hydrodynamic calculations of binary coalescence using the smoothed particle hydrodynamics (SPH) method. The initial conditions are exact polytropic equilibrium configurations on the verge of dynamical instability. We consider synchronized equilibria only, focusing on the effects of varying the compressibility of the fluid and the binary mass ratio. We concentrate here on stiff equations of state, with adiabatic exponents GAMMA > 5/3, and we assume that the polytropic constants (K =- P/rho(GAMMA)) are the same for both components. These conditions apply well to models of neutron star binaries. Accordingly, we discuss our results in the context of the LIGO project, and we calculate the emission of gravitational radiation in the quadrupole approximation. The fully nonlinear development of the instability is followed using SPH until a new equilibrium configuration is reached by the system. We find that the properties of this final configuration depend sensitively on both the compressibility and mass ratio. An axisymmetric merged configuration is always produced when the adiabatic exponent T less than or similar to 2.3. As a consequence, the emission of gravitational radiation shuts off abruptly right after the onset of dynamical instability. In contrast, triaxial merged configurations are obtained when GAMMA greater than or equal to 2.3, and the system continues to emit gravitational waves after the final coalescence. Systems with mass ratios q not-equal 1 typically become dynamically unstable before the onset of mass transfer. Stable mass transfer from one neutron star to another in a close binary is therefore probably ruled out. For a mass ratio q less than or similar to 0.5, however, dynamical mass transfer can temporarily retard the coalescence by causing a rapid reexpansion of the binary into a new, slightly eccentric but dynamically stable orbit. The maximum amplitude h(max) and peak luminosity L(max) of the gravitational waves emitted during the final coalescence are nearly independent of GAMMA but depend sensitively on the mass ratio q. The approximate scalings we find are h(max) is-proportional-to q2 and L(max) is-proportional-to q6 for q close to unity. These are much steeper dependences than would be expected for a system containing two point masses, where h is-proportional-to q and L is-proportional-to q2(1 + q)
