The scheme, which was designed by architect Jeremy Dixon.Edward Jones, would enlarge and restore the entrance lobby on the first floor.
Charles Saumarez Smith, director of the gallery, said it would now have to raise private funds. He said: "We are slightly bemused by the HLF saying our application doesn't meet its criteria. It provides better access and opens up the gallery to its 4.5m visitors a year."
The HLF said that it had already awarded the gallery £938,000 towards the refurbishment and £11m for the purchase of a painting by Raphael, The Madonna of the Pinks.