First-principles study of the effective Hamiltonian for Dirac fermions with spin-orbit coupling in two-dimensional molecular conductor $$\alpha $$ α -(BETS) $$_2 \hbox {I}_3$$ 2 I 3

We employed first-principles density-functional theory (DFT) calculations to characterize Dirac electrons in quasi-two-dimensional molecular conductor $$\alpha $$ α -(BETS) $$_2 \hbox {I}_3$$ 2 I 3 [= $$\alpha $$ α -(BEDT–TSeF) $$_2 \hbox {I}_3$$ 2 I 3 ] at a low temperature of 30 K. We provide a tight-binding model with intermolecular transfer energies evaluated from maximally localized Wannier functions, where the number of relevant transfer integrals is relatively large due to the delocalized character of Se p orbitals. The spin–orbit coupling gives rise to an exotic insulating state with an indirect band gap of about 2 meV. We analyzed the energy spectrum with a Dirac cone close to the Fermi level to develop an effective Hamiltonian with site potentials, which reproduces the spectrum obtained by the DFT band structure. Graphic abstract