Successful human-robot teaming will require robots to adapt autonomously to a human teammate's internal state, where a critical element of such adaptation is the ability to estimate the human's workload in unknown situations. Existing workload models use machine learning to model the relationship between physiological signals and workload. These methods often struggle to generalize to unknown tasks, as the relative importance of various physiological signals change significantly between tasks. Many of these changes constitute a meaningful shift in the data's distribution, which violates a core assumption made by the underlying machine learning approach. A survey of machine learning techniques designed to overcome these challenges is presented, where common techniques are evaluated using three criteria: portability, model complexity, and adaptability. These criteria are used to analyze each technique's applicability to estimating workload during unknown tasks in dynamic environments and guide future empirical experimentation.