SDN provides a programmable command and control networking system in a multi-tenant cloud network using control and data plane separation. However, separating the control and data planes make it difficult for incorporating some security services (e.g., firewalls) into SDN framework. Most of the existing solutions use SDN switches as packet filters and rely on SDN controllers to implement firewall policy management functions, which is impractical for implementing stateful firewalls since SDN switches only send session's initial packets and statistical data of flows to their controllers. For a data center networking environment, applying a Distributed FireWall (DFW) system to prevent attacker's lateral movements is highly desired, in which designing and implementing an SDN-based Stateful DFW (SDFW) demand a scalable distributed states management solution at the data plane to track packets and flow states. Our performance results show that SDFW achieves scalable security against data plane attacks with a marginal performance hit ~ 1.6% reduction in network bandwidth.