Bandwidth-efficient coded modulation with optimized linear partial-response signals

We study the design of optimal signals for bandwidth-efficient linear coded modulation, Recent results show that for linear channels with intersymbol interference (ISI), reduced-search decoding algorithms have near-maximum-likelihood error performance, but with much smaller complexity than the Viterbi decoder. Consequently, the controlled ISI introduced by a lowpass filter can be practically used for bandwidth reduction, Such spectrum shaping filters comprise an explicit coded modulation, for which we seek the optimal design, We simultaneously constrain the bandwidth and maximize the minimum Euclidean distance between signals, We show that under quite general assumptions the problem can be formulated as a linear program, and solved with well-known efficient optimization techniques, Numerical results are presented, and the performance of the optimal signals, measured by their combined bandwidth and noise immunity, is analyzed, The new codes are comparable to set-partition (TCM) trellis codes, Tests of an M-algorithm decoder confirm this and show that the performance occurs at small detection complexity.