In online auctions, fraudulent behaviors such as shill bidding pose significant risks. This paper presents a conceptual framework that applies dynamic, behavior-based penalties to deter auction fraud using blockchain smart contracts. Unlike traditional post-auction detection methods, this approach prevents manipulation in real-time by introducing an economic disincentive system where penalty severity scales with suspicious bidding patterns. The framework employs the proposed Bid Shill Score (BSS) to evaluate nine distinct bidding behaviors, dynamically adjusting the penalty fees to make fraudulent activity financially unaffordable while providing fair competition. The system is implemented within a decentralized English auction on the Ethereum blockchain, demonstrating how smart contracts enforce transparent auction rules without trusted intermediaries. Simulations confirm the effectiveness of the proposed model: the dynamic penalty mechanism reduces the profitability of shill bidding while keeping penalties low for honest bidders. Performance evaluation shows that the system introduces only moderate gas and latency overhead, keeping transaction costs and response times within practical bounds for real-world use. The approach provides a practical method for behaviour-based fraud prevention in decentralised systems where trust cannot be assumed.