Characterization of stochastic viability of any nonsmooth set involving its generalized contingent curvature

Thanks to the Stroock and Varadhan "Support Theorem" and under convenient regularity assumptions, stochastic viability problems are equivalent to invariance problems for control systems (also called tychastic viability), as it has been singled out by Doss in 1977 for instance. By the way, it is in this framework of invariance under control systems that problems of stochastic viability in mathematical finance are studied. The Invariance Theorem for control systems characterizes invariance through first-order tangential and/or normal conditions whereas the stochastic invariance theorem characterizes invariance under second-order tangential conditions. Doss's Theorem states that these first-order normal conditions are equivalent to second-order normal conditions that we expect for invariance under stochastic differential equations for smooth subsets. We extend this result to any subset by defining in an adequate way the concept of contingent curvature of a set and contingent epi-Hessian of a function, related to the contingent curvature of its epigraph. This allows us to go one step further by characterizing functions the epigraphs of which are invariant under systems of stochastic differential equations. We shall show that they are (generalized) solutions to either a system of first-order Hamilton-Jacobi equations or to an equivalent system of second-order Hamilton-Jacobi equations.