Heads-Up Display (also known as HUD) is the visual display on screen that relays important information to the player. These can include health, weapons, ammo, and time left among other game specific information. Some games have tried to limit or hide the HUD so that the players immersion isn't ruined.
Games like Metroid Prime and Metroid Prime 2: Echoes have notable HUDs for their realism and attention to detail. Because these games display the HUD on the inside of Samus's Visor, which can be affected by rain or fog, as well as presenting information in the way you'd expect a robotic suit to present it, the immersion is powerful.
However, having a detailed HUD is not the only way to immerse players. Games like Peter Jackson's King Kong feature no HUD at all when playing a human, giving no clues to health or ammo, but still make it a realistic first-person experience.
A HUD is different from an interface in that it is not interactive, but only displays (sometimes irrelevant) information. In Kingdom Hearts, there was no HUD telling the player the targeted opponent's health, however, the player could later learn the move 'Scan', which automatically displayed the opponents health at the cost of ability points. Most people would equip Scan and lose ability points, even though knowing the opponents health, in no way, benefits the player in combat.
HUDs are often necessary to help the player know that they're losing or making progress and are sometimes viewed as more important than realism. Small, abrasive, and well-designed HUDs can sometimes be easily forgotten by the player, and not affect the immersion at all.
One of the focuses of the Nintendo DS system, was to allow developers to put the HUD on a second screen, that way the player could focus on one screen and not see the HUD at all, but still have it at his disposal whenever he chose to look at it. Many games for the DS use the second screen to display the map or ammo and health meters.