Sample-Efficient Algorithms For Hard-Exploration Problems In Reinforcement Learning