A Monday night sweep of the D.C. Jail discovered several weapons, including a 9 mm Glock found in a pay phone accessible to inmates.
Several knives and cell phones also were recovered by corrections officer.
Officials said the handgun may have been in the jail since 2003, the Washington Post reported.
The weapon may be linked to another handgun smuggled into the facility that was part of a 2003 scheme in which four inmates agreed to shoot themselves in the hopes of suing the District and winning a monetary judgment for not keeping them safe, officials said.
Those inmates carried out the plan and were injured, but two were convicted and two pleaded guilty in the plot.
A second gun has been rumored to have been part of the plan, the Post reported, and corrections officials have trying to find the gun ever since.