Autonomous navigation in extraterrestrial environments comes with a variety of challenges, including difficult terrain, limited communication, and unpredictable obstacles. This thesis investigates the use of reinforcement learning (RL) techniques to enable arover to explore and navigate the lunar environment autonomously. A multi-input RL agent was developed that uses visual data from instance segmentation, along with distance and orientation to goal measurements, to move towards randomly placed goal points while avoiding obstacles. The agent was trained and tested within the Gazebo simulator with various training methods, including reward shaping, hyperparameter tuning, and input configurations, to optimize its performance. Additional testing was done in a Unity based lunar simulator that was built along side the thesis in order to create a more photorealistic environment for the RL agent. Simulation challenges, such as inconsistent terrain physics and synchronization issues, impacted the learning process. However, the agent showed promising signs of learning to safely navigate the environment, indicating its potential for real-world applications. Future work will focus on improving the lunar simulator and looking further into the local map and intrinsic motivation approaches. This research provides valuable insights into the development of RL-driven exploration agents and contributes toward autonomous robotic systems capable of resiliently navigating extraterrestrial surfaces.