A delay-differential equation modelling an artificial neural network with two neurons is investigated. A linear stability analysis provides parameter values yielding asymptotic stability of the stationary solutions: these can lose stability through either a pitchfork or a Hopf bifurcation, which is shown to be supercritical. At appropriate parameter values, an interaction takes place between the pitchfork and Hopf bifurcations. Conditions are also given for the set of initial conditions that converge to a stable stationary solution to be open and dense in the functional phase space. Analytic results are illustrated with numerical simulations.