Nonlinear structural analyses in engineering often require extensive finite element simulations, limiting their applicability in design optimization and real-time control. Conventional deep learning surrogates often struggle with complex, non-parametric three-dimensional (3D) geometries and directionally varying loads. This work presents Point-DeepONet, an operator-learning-based surrogate that integrates PointNet into the DeepONet framework to learn a mapping from non-parametric geometries and variable load conditions to physical response fields. By leveraging PointNet to learn a geometric representation from raw point clouds, our model circumvents the need for manual parameterization. This geometric embedding is then synergistically fused with load conditions within the DeepONet architecture to accurately predict three-dimensional displacement and von Mises stress fields. Trained on a large-scale dataset, Point-DeepONet demonstrates high fidelity, achieving a coefficient of determination (R^2) reaching 0.987 for displacement and 0.923 for von Mises stress. Furthermore, to rigorously validate its generalization capabilities, we conducted additional experiments on unseen, randomly oriented load directions, where the model maintained exceptional accuracy. Compared to nonlinear finite element analyses that require about 19.32 minutes per case, Point-DeepONet provides predictions in mere seconds--approximately 400 times faster--while maintaining excellent scalability. These findings, validated through extensive experiments and ablation studies, highlight the potential of Point-DeepONet to enable rapid, high-fidelity structural analyses for complex engineering workflows.