A compact embedding result and its applications to a nonlocal Schrödinger equation

Compactness is a fundamental issue in nonlinear analysis. Assume s∈(0,1)$s\in (0,1)$ and Ω⊂RN$\Omega \subset \mathbb {R}^N$ with N>2s$N>2s$, we consider a subspace X0,Gs(Ω)$X^s_{0,G}(\Omega )$ of X0s(Ω)$X_0^s(\Omega )$, which is the space of invariant functions corresponding to a subgroup G of O(N)$O(N)$, where X0s(Ω)$X_0^s(\Omega )$ is a kind of function spaces including fractional Sobolev spaces H0s(Ω)$H_0^s(\Omega )$. We show that the embeddings X0,Gs(Ω)↪Lp(Ω)$X_{0,G}^s(\Omega )\hookrightarrow L^p(\Omega )$ are compact for p∈(2,2s∗)$p\in (2,2_s^*)$ provided Ω is compatible with G, where 2s∗=2NN−2s$2^*_s=\frac{2N}{N-2s}$ is the fractional Sobolev critical exponent. Moreover, the existence and multiplicity of radial and nonradial solutions were obtained of the following nonlocal Schrödinger equation: −Lku+V(x)u=f(x,u),x∈RN,$$\begin{equation*} -{\mathcal {L}_k}u+V(x)u=f(x,u),\ x\in \ \mathbb {R}^N, \end{equation*}$$where −Lk$-{\mathcal {L}_k}$ is an integro‐differential operator has order 2s and can be given by Lku(x):=∫RNu(x+y)+u(x−y)−2u(x)K(y)dy,x∈RN,$$\begin{equation*} {\mathcal {L}_k}u(x):=\int _{\mathbb {R}^N}{\left(u(x+y)+u(x-y)-2u(x)\right)}K(y)dy,\ x\in \mathbb {R}^N, \end{equation*}$$and the kernel K satisfies the following properties: (K1)there is θ>0$\theta > 0$ and s∈(0,1)$s\in (0,1)$ such that K(x)≥θ|x|−(N+2s)$K(x)\ge \theta |x|^{-({N+2s})}$ for any x∈RN∖{0}$x\in \mathbb {R}^N\backslash \lbrace 0\rbrace$; (K2)mK∈L1(RN)$mK\in L^1(\mathbb {R}^N)$, where m(x):=min{|x|2,1}$m(x):=\min \lbrace |x|^2,1\rbrace$.