This article concerns the challenge of reliable broadband passive sonar target detection and tracking in complex acoustic environments. Addressing this challenge is becoming increasingly crucial for safeguarding underwater infrastructure, monitoring marine life, and providing defense during seabed warfare. To that end, a solution is proposed based on a vector-autoregressive model for the ambient noise and a heavy-tailed statistical model for the distribution of the raw hydrophone data. These models are integrated into a Bernoulli track-before-detect (TkBD) filter that estimates the probability of target existence, target bearing, and signal-to-noise ratio (SNR). The proposed solution is evaluated on both simulated and real-world data, demonstrating the effectiveness of the proposed ambient noise modeling and the statistical model for the raw hydrophone data samples to obtain early target detection and robust target tracking. The simulations show that the SNR at which the target can be detected is reduced by 4 dB compared to when using the standard constant false alarm rate detector-based tracker. Further, the test with real-world data shows that the proposed solution increases the target detection distance from 250 m to 390 m. The presented results illustrate that the TkBD technology, in combination with data-driven ambient noise modeling and heavy-tailed statistical signal models, can enable reliable broadband passive sonar target detection and tracking in complex acoustic environments and lower the SNR required to detect and track targets.