Modulating the stiffness of soft actuators is crucial for improving the efficiency of interaction with the environment. However, current stiffness modulation mechanisms are hard to achieve high lateral stiffness and a wide range of bending stiffness simultaneously. Here, we draw inspiration from the anatomical structure of the finger and propose a bi-directional tunable stiffness actuator (BTSA). BTSA is a soft-rigid hybrid structure that combines air-tendon hybrid actuation (ATA) and bone-like structures (BLS). We develop a corresponding fabrication method and a stiffness analysis model to support the design of BLS. The results show that the influence of the BLS on bending deformation is negligible, with a distal point distance error of less than 1.5 mm. Moreover, the bi-directional tunable stiffness is proved to be functional. The bending stiffness can be tuned by ATA from 0.23 N/mm to 0.70 N/mm, with a magnification of 3 times. The addition of BLS improves lateral stiffness up to 4.2 times compared with the one without BLS, and the lateral stiffness can be tuned decoupling within 1.2 to 2.1 times (e.g. from 0.35 N/mm to 0.46 N/mm when the bending angle is 45 deg). Finally, a four-BTSA gripper is developed to conduct horizontal lifting and grasping tasks to demonstrate the advantages of BTSA.