With the advent of virtualization technology, cloud computing realizes on-demand computing. The capability of dynamic resource provisioning is a fundamental driving factor for users to adopt the cloud technology. The aspect is important for cloud service providers to optimize the expense for running the infrastructure as well. Despite many technological advances in related areas, however, it is still the case that the infrastructure providers must decide hardware configuration before deploying a cloud infrastructure, especially from the storage's perspective. This static nature of the storage provisioning practice can cause many problems in meeting tenant requirements, which often come later into the picture. In this paper, we propose a system called IOArbiter that enables the dynamic creation of underlying storage implementation in the cloud. IOArbiter defers storage provisioning to the time at which a tenant actually requests a storage space. As a result, an underlying storage implementation, e.g., RAID-5, 6 or Ceph storage pool with 6+3 erasure coding, will be materialized at the volume creation time. Using our prototype implementation with Openstack Cinder, we show that IOArbiter can simultaneously satisfy a number of different tenant demands, which may not be possible with a static configuration strategy. Additionally QoS mechanisms such as admission control and dynamic throttling help the system mitigate a noisy neighbor problem significantly.