A weapon overheats as it is used. This is indicated by the yellow bar on the weapon icon on your weapons tray, which turns orange and eventually red when it is at maximum. While the heat gauge is at maximum, every shot fired has a chance of overheating your gun, at which point it will jam and needs to be reloaded to unjam it.
Using a weapon while at maximum heat will also wear out its' durability quicker, reducing its' effectiveness. A weapon constantly cools -- when it isn't in use, the heat gauge goes down quickly. Some weapons overheat more slowly than others; some overheat after a few shots, some cool down so quickly that it's difficult to overheat them, while some don't overheat at all.
To avoid overheating, one tactic is to have two weapons you alternate between, using one when the other's hot. Another idea would be to reload a bit more often, giving the weapon a chance to cool off.