We present deep U -, V -, and I - band images of the ultracompact blue dwarf galaxy POX 186 obtained with the Planetary Camera 2 of the Hubble Space Telescope. We have also obtained a near-ultraviolet spectrum of the object with the Space Telescope Imaging Spectrograph and combine this with a new ground-based optical spectrum. The images confirm the galaxy to be extremely small, with a maximum extent of only 300 pc, a luminosity of similar to10(-4) L*, and an estimated mass of similar to10(7) M-.. Its morphology is highly asymmetric, with a tail of material on its western side that may be tidal in origin. The U - band image shows this tail to be part of a stream of material in which stars have recently formed. Most of the star formation in the galaxy is, however, concentrated in a central, compact (d similar to 10-15 pc) star cluster. We estimate this cluster to have a total mass of 10 5 M, to be forming stars at a rate of less than 0.05 yr(-1), and to have a maximum age of a few million years. The outer regions of the galaxy are significantly redder than the cluster, with V-I colors consistent with a population dominated by K and M stars. From our analysis of the optical spectrum we find the galaxy to have a metallicity Z similar or equal to 0.06 Z(.) and to contain a significant amount of internal dust [ E (B-V) similar or equal to 0.28]; both values agree with previous estimates. While these results rule out earlier speculation that POX 186 is a protogalaxy, its morphology, mass, and active star formation suggest that it represents a recent (within similar to 10(8) yr) collision between two clumps of stars of subgalactic size (similar to100 pc). POX 186 may thus be a very small dwarf galaxy that, dynamically speaking, is still in the process of formation. This interpretation is supported by the fact that it resides in a void, so its morphology cannot be explained as the result of an encounter with a more massive galaxy. Clumps of stars this small may represent the building blocks required by hierarchical models of galaxy formation, and these results also support the recent downsizing picture of galaxy formation in which the least massive objects are the last to form.