Kalai conjectured that if $P$ is a simplicial $d$-polytope that has no missing faces of dimension $d-1$, then the graph of $P$ and the space of affine $2$-stresses of $P$ determine $P$ up to affine equivalence. We propose a higher-dimensional generalization of this conjecture: if $2\leq i\leq d/2$ and $P$ is a simplicial $d$-polytope that has no missing faces of dimension $\geq d-i+1$, then the space of affine $i$-stresses of $P$ determines the space of affine $1$-stresses of $P$. We prove this conjecture for (1) $k$-stacked $d$-polytopes with $2\leq i\leq k\leq d/2-1$, (2) $d$-polytopes that have no missing faces of dimension $\geq d-2i+2$, and (3) flag PL $(d-1)$-spheres with generic embeddings (for all $2\leq i\leq d/2$). We also discuss several related results and conjectures. For instance, we show that if $P$ is a simplicial $d$-polytope that has no missing faces of dimension $\geq d-2i+2$, then the $(i-1)$-skeleton of $P$ and the set of sign vectors of affine $i$-stresses of $P$ determine the combinatorial type of $P$. Along the way, we establish the partition of unity of affine stresses: for any $1\leq i\leq (d-1)/2$, the space of affine $i$-stresses of a simplicial $d$-polytope as well as the space of affine $i$-stresses of a simplicial $(d-1)$-sphere (with a generic embedding) can be expressed as the sum of affine $i$-stress spaces of vertex stars. This is analogous to Adiprasito's partition of unity of linear stresses for Cohen--Macaulay complexes.