The method of potential operators for anisotropic helmholtz operators on domains with smooth unbounded boundaries

The paper is devoted to the method of potential operators for boundary and transmission problems in domains in Rⁿ with smooth unbounded boundaries for the anisotropic Helmholtz operators Hu(x) = Δ. a(x)Δu(x) + b(x)u(x), x∈ Rn with variable coefficients, where a = (a^k,l)ⁿ k,l=1 are real-valued symmetric matrices on Rⁿ with entries a^k,l ∈ C^∞_b (Rn), k, l = 1, . . . , n. We assume that the operator H is strongly elliptic, b(x) = ω²b₀(x), ω > 0 is the frequency of the harmonic vibrations, b0(x) is the refractive index which satisfies the following conditions: b₀ ∈ C^∞_b (Rⁿ), Rb₀(x) > 0, Jb₀(x) ≥ 0, x∞ Rⁿ, lim inf Rⁿ∞x→∞ Jb0(x) > 0. We introduce single and double layer potentials associated with the operator H, and reduce by means of these potentials the Dirichlet, Neumann, Robin, and transmission problems for a domain with unbounded smooth boundary ∂D to pseudodifferential equations on ∂D. Applying the method of limit operators, we study Fredholm properties and the invertibility of the boundary pseudodifferential operators in the Sobolev spaces H^s(∂D), s ∈ R.