The effects of PK11195, a high-affinity peripheral benzodiazepine receptor (PBR) ligand, on protein phosphorylation in isolated purified rat brain mitochondria were investigated. The isoquinoline carboxamide ligand of PBR, PK11195, but not the benzodiazepine ligand Ro5-4864, in the nanomolar concentration range strongly increased the phosphorylation of 3.5 and 17 kDa polypeptides. The effect of PK11195 was seen in the presence of elevated Ca2+ levels (3 x 10(-7) to 10(-6) M), but not at very low Ca2+ levels (10(-8) to 3 x 10(-8) M). This indicates that PBR involves Ca2+ as a second messenger in the regulation of protein phosphorylation. Staurosporine, an inhibitor of protein kinase activity was able to suppress the PK11195promoted protein phosphorylation. When the permeability transition pore (PTP) was opened by threshold Ca2+ load, phosphorylation of the 3.5- kDa polypeptide was diminished, but strong phosphorylation of the 43-kDa protein was revealed. The 43-kDa protein appears to be a PTP- specific phosphoprotein. If PTP was opened, PK11195 did not increase the phosphorylation of the 3.5 and 17-kDa proteins but suppressed the phosphorylation of the PTP- specific 43-kDa phosphoprotein. The ability of PK11195 to increase the protein phosphorylation, which was lost under Ca2+-induced PTP opening, was restored again in the presence of calmidazolium, an antagonist of calmodulin and inhibitor of protein phosphatase PP2B. These results show a tight interaction of PBR with the PTP complex in rat brain mitochondria. In conclusion, a novel function of PBR in brain mitochondria has been revealed, and the PBR- mediated protein phosphorylation has to be considered an important element of the PBR-associated signal transducing cascades in mitochondria and cells.