We explore the phase diagram of neutral quark matter at high baryon density as a function of the temperature T and the strange quark mass M-s. At T=0, there is a sharp distinction between the insulating color-flavor locked (CFL) phase, which occurs where M-s(2)/mu<2Delta, and the metallic gapless CFL phase, which occurs at larger M-s(2)/mu. Here, mu is the chemical potential for quark number and Delta is the gap in the CFL phase. We find this distinction blurred at Tnot equal0, as the CFL phase undergoes an insulator to metal crossover when it is heated. We present an analytic treatment of this crossover. At higher temperatures, we map out the phase transition lines at which the gap parameters Delta(1), Delta(2), and Delta(3) describing ds pairing, us pairing and ud pairing, respectively, go to zero in an Nambu-Jona-Lasinio (NJL) model. For small values of M-s(2)/mu, we find that Delta(2) vanishes first, then Delta(1), then Delta(3). We find agreement with a previous Ginzburg-Landau analysis of the form of these transitions and find quantitative agreement with results obtained in full QCD at asymptotic density for ratios of coefficients in the Ginzburg-Landau potential. At larger M-s(2)/mu, we find that Delta(1) vanishes first, then Delta(2), then Delta(3). Hence, we find a "doubly critical" point in the (M-s(2)/mu,T) plane at which two lines of second order phase transitions (Delta(1)-->0 and Delta(2)-->0) cross. Because we do not make any small-M-s approximation, if we choose a relatively strong coupling leading to large gap parameters, we are able to pursue the analysis of the phase diagram all the way up to such large values of M-s that there are no strange quarks present.