When conceptualizing the algorithm running on an intelligent robot through the glasses of a reductionist, a subsystem responsible for getting a sense of the location of the agent shows to be among the intuitively deduced modules. Whether we view location as a parameter that defines our position with respect to a fixed origin, or we define it through a relative perspective, a robot needs to have a notion of its placement in the world to be able to make appropriate decisions and perform the necessary actions while reacting to the dynamic world around it. However, this dynamic nature of the world presents problems such as flawed describers and unreliable observations that make it challenging to use straightforward solutions to perform localization through low-level information gathered by the sensors mounted on the robot. Therefore, approaches are needed that are able to exploit the semantic information embedded in observed scenes and extract higher-level information about the world around them that are robust to such issues. Moreover, by leveraging multiple sensors, information from modalities where the source of data varies to a suitable extent inbetween the said modalities may be gathered and fused to form a joint high-level representation of the state of the robot, further adding to the reliability of the localization system. In this thesis, our goal is to design and experiment with neural network architectures and create learning paradigms that incentivize the extraction of robust features through a representation learning procedure where the inputs to the network are not preprocessed. We propose mechanisms and objectives that allow the network to disregard faulty input information while achieving interpretability that allows the system to communicate its uncertainty about the estimates based on the provided inputs. Moreover, we take a hybrid approach to global localization of the robot where physical and learning based models are combined to form a multilevel localization approach in order to increase the flexibility of the pipeline. We perform comprehensive experiments to show our motivation while comparing our approaches to the state-of-the-art methods quantitatively and qualitatively. We analyze the proposed approaches through custom designed interpretation methods to get in-depth intuition on how our algorithms add to the literature and improve upon the state-of-the-art algorithms. Thereafter, we provide an overview of the branches of our work that can be explored further while delineating the potential future of the field.