In this paper, we study further the problem of designing controllers that optimally reject persistent disturbances. The focus in the present paper is on the case where the plant to be controlled has either zeros or poles on the stability boundary, i.e., the unit circle in the discrete-time case and the extended j-omega axis in the continuous-time case. This situation arises, for example, when the plant is of type 1 or more, which it needs to be in order to track a step input. We tackle first the discrete-time case, and study the problem of minimizing a cost functional of the form parallel-to f - rg parallel-to 1, where the transform g approximately of g has some unit circle zeros. The dual problem formulation introduced by Dahleh and Pearson is extended to the present situation, and it is shown that an optimal controller need not exist. Then we study the construction of a sequence of suboptimal controllers whose performance approaches the unattainable infimum of the cost function, and show that two results which hold in the case of H infinity optimization do not hold in the present situation. Specifically, the introduction of unit circle zeros can, in fact, increase the value of the infimum, even when every unit circle zero of g approximately is also a zero of f approximately, and a sequence of controllers constructed in an "obvious" fashion fails to be an optimizing sequence. Similar results are then obtained for the continuous-time case.