Photonic in-memory computing is a high-speed, low-energy alternative to traditional transistor-based digital computing that utilizes high photonic operating frequencies and bandwidths. In this work, we develop a comprehensive system-level performance model for photonic in-memory computing, capturing the effects of key latency sources such as external memory access and opto-electronic conversion. We perform algorithm-to-hardware mapping across a range of workloads, including the Sod shock tube problem, Matricized Tensor Times Khatri-Rao Product (MTTKRP), and the Vlasov-Maxwell equation, to evaluate how the latencies impact real-world high-performance computing workloads. Our performance model shows that, while accounting for system overheads, a compact 1x256 bit single-wavelength photonic SRAM array, fabricated using the standard silicon photonics process by GlobalFoundries, sustains up to 1.5 TOPS, 0.9 TOPS, and 1.3 TOPS on the Sod shock tube problem, MTTKRP, and the Vlasov-Maxwell equation with an average energy efficiency of 2.5 TOPS/W.