HRTF is Head Related Transfer Function. It describes how you perceive sound. Every person perceives sound differently because the individual shape of your head, ears, nasal & mouth cavity, etc. all affect how the sound reaches your ears. In short: different people listening to the same thing, hear it differently.
What most HRTFs have in common is the range from 2 – 5 kHz is amplified by 15 dB or more. The ear’s resonance is typically +17 dB at 2.7 kHz. That is a huge non-linearity. Here is a typical HRTF curve from Tyll Herstens at Inner Fidelity.
Another way to think about this: Suppose you’re standing at the seashore listening to waves crashing on the beach. That sound is similar to white noise: it has roughly equal energy across a wide frequency range. The sound you actually perceive, however, is 10 – 20 dB louder in the 2 -5 kHz range because those frequencies were amplified (or frequencies outside that range attenuated) by your head, ears, ear canals before it hit your eardrums.
You can easily test how the size & shape of your head & ears affects sound. While listening to music on speakers, gently push your ears forward or open your mouth really wide. The sound changes. And that only gives a small taste of what the real differences are – imagine how much more different it might be if you could change the size & shape of your head, ears, etc.! That different sound you hear would be what another person hears normally.
The astute reader will wonder – if this variation is due to individual variance in body size & shape, how can it be measured? The answer is simple. Take 2 tiny microphones small enough to fit inside your ear canal. Position them in the open air and use them to record sound. Now build a fake life-size human head using materials that approximate the density & reflectivity of human tissue and skin, and insert these same mics deep into the ear canals, facing outward. Now measure the same sound again. The difference between the two recordings is the HRTF of your dummy head.
Every person has an individual HRTF and the variance from person to person is significant. Since headphones bypass the HRTF, in order to sound natural they must have a frequency response that matches the HRTF. Put differently, a headphone with flat frequency response would sound quite dull, down 15+ db in the 2 – 5 kHz range.
This doesn’t apply to loudspeakers. If a speaker has objectively flat FR, every person will perceive that however they perceive natural sounds. Speakers don’t have to reproduce the HRTF because the sound comes from a distant source and your HRTF transforms it when it hits your body. Headphones play sounds directly into your ears, bypassing your body, head and HRTF.
This means there is an absolute reference FR for speakers: perfectly flat. But there is no absolute reference FR for headphones. A headphone has to mimic the HRTF which is different for every person. The best a well-engineered headphone can do is mimic the most common or average HRTF across the population. Each individual will be a little different.
Thus, different people will disagree on what headphone has the most natural FR reproducing sounds most realistically. For example, the Sennheiser HD-800 has a big response rise around 5 – 7 kHz. For me personally, it’s artificially bright, almost skull-jarring. But for others it may sound natural. At the opposite end of the spectrum, the Audeze LCD-2 has a dip from 2 – 9 kHz (its raw response has a rise, but it rises a bit less than the typical HRTF does). For me personally, it sounds natural and realistic. My HRTF probably lifts this frequency range less than average. But for others this headphone sounds dull.