In this paper we introduce a novel Neural Networks-based approach for approximating solutions to the (2D) incompressible Navier--Stokes equations, which is an extension of so called Deep Random Vortex Methods (DRVM), that does not require the knowledge of the Biot--Savart kernel associated to the computational domain. Our algorithm uses a Neural Network (NN), that approximates the vorticity based on a loss function that uses a computationally efficient formulation of the Random Vortex Dynamics. The neural vorticity estimator is then combined with traditional numerical PDE-solvers, which can be considered as a final implicit linear layer of the network, for the Poisson equation to compute the velocity field. The main advantage of our method compared to the standard DRVM and other NN-based numerical algorithms is that it strictly enforces physical properties, such as incompressibility or (no slip) boundary conditions, which might be hard to guarantee otherwise. The approximation abilities of our algorithm, and its capability for incorporating measurement data, are validated by several numerical experiments.