Robots capable of performing manipulation tasks in a broad range of missions in unstructured environments can develop numerous applications to impact and enhance human life. Existing work in robot learning has shown success in applying conventional machine learning algorithms to enable robots for replicating rather simple manipulation tasks in manufacturing, service and healthcare applications, among others. However, learning robust and versatile models for complex manipulation tasks that are inherently multi-faceted and naturally intricate demands algorithmic advancements in robot learning. Our research supports the long-term goal of making robots more accessible and serviceable to the general public by expanding robot applications to real-world scenarios that require systems capable of performing complex tasks. To achieve this goal, we focus on identifying and investigating knowledge gaps in robot learning of complex manipulation tasks by leveraging upon human-robot interaction and robot learning from human instructions. This document presents an overview of the recent research developments in the Persistent Autonomy and Robot Learning (PeARL) lab at the University of Massachusetts Lowell. Here, I briefly discuss different research directions, and present a few proposed approaches in our most recent publications. For each proposed approach, I then mention potential future directions that can advance the field.