You can have self-confidence without having self-esteem. Self-confidence is believing in your ability to do something; self-esteem is believing in your worthiness as a person.
More and more frequently I run into people who know they are intelligent, but don't believe it; know they are attractive, but don't believe it; know they are successful, but don't believe it; know they are good people, but don't believe it and/or know they are worthwhile and don't believe it.
The positive side to not feeling intelligent, attractive, successful, good or worthwhile is that it can keep pushing you to become better at all of these. The negative side is that it can cause you to feel anxious, depressed and rob you of peace of mind. At the extreme it can cause you to feel despair, which I define as des-pair, i.e. feeling unpaired with intelligence, attractiveness, success, goodness or worthiness. And feeling despair can cause you to give up.