Oracles and queries that are sufficient for exact learning

We show that the class of all circuits is exactly learnable in randomized expected polynomial time using weak subset and weak superset queries. This is a consequence of the following result which we consider to be of independent interest: circuits are exactly learnable in randomized expected polynomial time with equivalence queries and the aid of an NP-oracle. We also show that circuits are exactly learnable in deterministic polynomial time with equivalence queries and a Sigma(3)(p)-oracle. The hypothesis class for the above learning algorithms is the class of circuits of larger-but polynomially related-size. Also, the algorithms can be adapted to learn the class of DNF formulas with hypothesis class consisting of depth-3 boolean AND-boolean OR-boolean AND formulas (by the work of Angluin this is optimal in the sense that the hypothesis class cannot be reduced to DNF formulas, i.e., depth-2 boolean OR-boolean AND formulas). We also investigate the power of an NP-oracle in the context of learning with membership queries. We show that there are deterministic learning algorithms that use membership queries and an NP-oracle to learn: monotone boolean functions in time polynomial in the DNF size and CNF size of the target formula; and the class of O(log n)-DNF boolean AND O(log n)-CNF formulas in time polynomial in n. We also show that, with an NP-oracle and membership queries, there is a randomized expected polynomial time algorithm that learns any class that is learnable from membership queries with unlimited computational power. Using similar techniques, we show the following both for membership and for equivalence queries (when the hypotheses allowed are precisely the concepts in the class); any class learnable with unbounded computational-power is learnable in deterministic polynomial time with a Sigma(5)(p)-oracle. Furthermore, we identify the combinatorial properties that completely determine learnability in this information-theoretic sense. Finally we point out a consequence of our result in structural complexity theory showing that if every NP set has polynomial-size circuits then the polynomial hierarchy collapses to ZPP(NP). (C) 1996 Academic Press, Inc.