Fractional Backward Stochastic Differential Equations and Fractional Backward Variational Inequalities

In the framework of fractional stochastic calculus, we study the existence and the uniqueness of the solution for a backward stochastic differential equation, formally written as: $$\begin{aligned} \left\{ \begin{array}{l} -\hbox {d}Y(t)= f(t,\eta (t),Y(t),Z(t))\hbox {d}t-Z(t)\delta B^{H}\left( t\right) ,\quad t\in [0,T],\\ Y(T)=\xi , \end{array} \right. \end{aligned}$$ { − d Y ( t ) = f ( t , η ( t ) , Y ( t ) , Z ( t ) ) d t − Z ( t ) δ B H ( t ) , t ∈ [ 0 , T ] , Y ( T ) = ξ , where $$\eta $$ η is a stochastic process given by $$\eta (t)=\eta (0) +\int _{0}^{t}\sigma (s) \delta B^{H}(s)$$ η ( t ) = η ( 0 ) + ∫ 0 t σ ( s ) δ B H ( s ) , $$t\in [0,T]$$ t ∈ [ 0 , T ] , and $$B^{H}$$ B H is a fractional Brownian motion with Hurst parameter greater than $$1/2$$ 1 / 2 . The stochastic integral used in above equation is the divergence-type integral. Based on Hu and Peng’s paper, Backward stochastic differential equation driven by fractional Brownian motion, SIAM J Control Optim (2009), we develop a rigorous approach for this equation. Moreover, we study the existence of the solution for the multivalued backward stochastic differential equation $$\begin{aligned} \left\{ \begin{array}{l} -\hbox {d}Y(t)+\partial \varphi (Y(t))\hbox {d}t\ni f(t,\eta (t),Y(t),Z(t))\hbox {d}t-Z(t)\delta B^{H}\left( t\right) ,\quad t\in [0,T],\\ Y(T)=\xi , \end{array}\right. \end{aligned}$$ { − d Y ( t ) + ∂ φ ( Y ( t ) ) d t ∋ f ( t , η ( t ) , Y ( t ) , Z ( t ) ) d t − Z ( t ) δ B H ( t ) , t ∈ [ 0 , T ] , Y ( T ) = ξ , where $$\partial \varphi $$ ∂ φ is a multivalued operator of subdifferential type associated with the convex function $$\varphi $$ φ .