Decentralized optimization for non-convex problems are now demanding by many emerging applications (e.g., smart grids, smart building, etc.). Though dramatic progress has been achieved in convex problems, the results for non-convex cases, especially with non-linear constraints, are still largely unexplored. This is mainly due to the challenges imposed by the non-linearity and non-convexity, which makes establishing the convergence conditions bewildered. This paper investigates decentralized optimization for a class of structured non-convex problems characterized by: (i) nonconvex global objective function (possibly nonsmooth) and (ii) coupled nonlinear constraints and local bounded convex constraints w.r.t. the agents. For such problems, a decentralized approach called Proximal Linearizationbased Decentralized Method (PLDM) is proposed. Different from the traditional (augmented) Lagrangian-based methods which usually require the exact (local) optima at each iteration, the proposed method leverages a proximal linearization-based technique to update the decision variables iteratively, which makes it computationally efficient and viable for the non-linear cases. Under some standard conditions, the PLDM global convergence and local convergence rate to the epsilon-critical points are studied based on the Kurdyka-Lojasiewicz property which holds for most analytical functions. Finally, the performance and efficacy of the method are illustrated through a numerical example and an application to multi-zone heating, ventilation and air-conditioning (HVAC) control.