Here is a partial answer elaborating on my comments:
Proposition 1: Let $X \subset \mathbb{R}$. Then TFAE:
- $f^{-1}(X)$ is Lebesgue-measurable for all Borel $f$;
- $X$ is universally measurable, i.e., $X$ is $\mu$-measurable for all Borel probability measure $\mu$ on $\mathbb{R}$.
We need the following two lemmas:
Lemma 1: Let $f: \mathbb{R} \to \mathbb{R}$ be Borel, $\mu$ be the completion of a Borel probability measure on $\mathbb{R}$. Then $f_\ast\mu$, defined on the pushforward along $f$ of the $\sigma$-algebra of $\mu$-measurable sets, is the completion of the Borel probability measure $f_\ast\mu|_{\mathcal{B}(\mathbb{R})}$.
Proof: We first prove that $f_\ast\mu$ is complete. Indeed, assume $Y \subset X$ and $f_\ast\mu(X) = 0$, i.e., $\mu(f^{-1}(X)) = 0$. Then $f^{-1}(Y) \subset f^{-1}(X)$. As $\mu$ is complete, $f^{-1}(Y)$ is $\mu$-measurable, so $Y$ is in the domain of $f_\ast\mu$, i.e., the pushforward along $f$ of the $\sigma$-algebra of $\mu$-measurable sets.
We now note that, as $f$ is Borel, $f_\ast\mu$ is indeed defined on all Borel sets, so $\phi = f_\ast\mu|_{\mathcal{B}(\mathbb{R})}$ is a well-defined Borel probability measure. We denote its completion by $\bar{\phi}$ and note that, as $f_\ast\mu$ is complete, its domain contains all $\phi$-measurable sets. Now, we need the following:
Claim 1: Suppose $X \subset \mathbb{R}$ is $\phi$-measurable. Then $\bar{\phi}(X) = f_\ast\mu(X)$.
Proof of Claim: Since $X$ is $\phi$-measurable, $X = A \cup B$ where $A$ is Borel and $B \subset C$ where $C$ is Borel and $\phi(C) = 0$, i.e., $\mu(f^{-1}(C)) = 0$. Furthermore, $\bar{\phi}(X) = \phi(A) = \mu(f^{-1}(A))$. Now, $B \subset C$ so $f^{-1}(B) \subset f^{-1}(C)$ so $\mu(f^{-1}(B)) = 0$. Thus,
$$f_\ast\mu(X) = \mu(f^{-1}(A) \cup f^{-1}(B)) = \mu(f^{-1}(A)) = \bar{\phi}(X).$$
$\square$
Claim 2: The image of a Borel set $X$ under a Borel function $f$ is universally measurable, so in particular $\phi$-measurable.
Proof of Claim: The graph of a Borel function is Borel, so $f(X)$ is the projection of the Borel set
$$(X \times \mathbb{R}) \cap \operatorname{graph}(f)$$
onto the second coordinate, whence it is analytic and therefore universally measurable. $\square$
Now, we need to show any $X \subset \mathbb{R}$ that is $f_\ast\mu$-measurable must be $\phi$-measurable. Note that $X = f(f^{-1}(X)) \sqcup (X \setminus f(\mathbb{R}))$. By Claim 2, $\mathbb{R} \setminus f(\mathbb{R})$ is $\phi$-measurable. Then, by Claim 1, it is $\phi$-null. As $X \setminus f(\mathbb{R}) \subset \mathbb{R} \setminus f(\mathbb{R})$, $X \setminus f(\mathbb{R})$ is $\phi$-null so in particular $\phi$-measurable as well.
Thus, it suffices to consider the case where $X = f(f^{-1}(X))$. Since $X$ is $f_\ast\mu$-measurable, $f^{-1}(X)$ is $\mu$-measurable, whence $f^{-1}(X) = A \cup B$ where $A$ is Borel and $B \subset C$ where $C$ is Borel and $\mu(C) = 0$. Then $X = f(A) \cup (X \setminus f(A))$. Observe that
$$f^{-1}(X \setminus f(A)) \subset f^{-1}(X) \setminus A \subset B \subset C.$$
By Claim 2, $f(A)$ is $\phi$-measurable and so is $f(\mathbb{R} \setminus C)$. Thus, $\mathbb{R} \setminus f(\mathbb{R} \setminus C)$ is $\phi$-measurable as well. Note that $\mathbb{R} \setminus f(\mathbb{R} \setminus C)$ consists of exactly those $x \in \mathbb{R}$ s.t. $f^{-1}(\{x\}) \subset C$, so in particular $X \setminus f(A) \subset \mathbb{R} \setminus f(\mathbb{R} \setminus C)$. Moreover, by Claim 1,
$$\bar{\phi}(\mathbb{R} \setminus f(\mathbb{R} \setminus C)) = \mu(f^{-1}(\mathbb{R} \setminus f(\mathbb{R} \setminus C))) \leq \mu(C) = 0.$$
That is, $\mathbb{R} \setminus f(\mathbb{R} \setminus C)$ is $\phi$-null and thus so is $X \setminus f(A)$. In particular, $X \setminus f(A)$ is $\phi$-measurable. Hence, $X = f(A) \cup (X \setminus f(A))$ is $\phi$-measurable as well. $\square$
Lemma 2: Let $\phi$ be a Borel probability measure on $\mathbb{R}$ and let $\mu$ be a Borel probability measure on $\mathbb{R}$ that is absolutely continuous w.r.t. the Lebesgue measure. Then there exists a Borel $f: \mathbb{R} \to \mathbb{R}$ s.t. $\phi = f_\ast\mu|_{\mathcal{B}(\mathbb{R})}$.
Proof: Standard decomposition results about probability measures show that there exists a (possibly empty) countable set $\{r_n\}_n \subset \mathbb{R}$ and a corresponding collection of positive numbers $\{d_n\}_n$ s.t. $\phi|_{\mathbb{R} \setminus \{r_n\}_n}$ is atomless and $\phi(\{r_n\}) = d_n$ for all $n$. By absolute continuity of $\mu$ w.r.t. the Lebesgue measure, we may choose (possibly empty) disjoint intervals $I_0$, $(I_n)_n$ s.t. $\mu(I_0) = \phi(\mathbb{R} \setminus \{r_n\}_n)$, $I_0 = \varnothing$ if $\phi(\mathbb{R} \setminus \{r_n\}_n) = 0$, $\mu(I_n) = d_n$ for all $n$, and $I_0 \cup \left(\bigcup_n I_n\right) = \mathbb{R}$. Since any two atomless probability measures on Polish spaces are isomorphic via a measure-preserving Borel isomorphism, we see that, in case $\phi(\mathbb{R} \setminus \{r_n\}_n) \neq 0$ or equivalently $I_0 \neq \varnothing$, there is a $\mu$-to-$\phi$-measure-preserving Borel isomorphism $g: I_0 \to \mathbb{R} \setminus \{r_n\}_n$. It is then easy to check that the following $f: \mathbb{R} \to \mathbb{R}$ satisfies the requirements of the lemma:
$$f(x) = \begin{cases}
g(x)\text, &\text{if }x \in I_0\\
r_n\text, &\text{if }x \in I_n\text.
\end{cases}$$
$\square$
Proof of Proposition 1: Fix a Borel probability measure $\mu$ on $\mathbb{R}$ that is equivalent to the Lebesgue measure (e.g., the Gaussian measure). Then $X \subset \mathbb{R}$ is in $\mathcal{E}$, the $\sigma$-algebra of Lebesgue-measurable sets iff it is $\mu$-measurable. Combining Lemma 1 and Lemma 2 then shows that
$$\{f_\ast\mathcal{E}: f: \mathbb{R} \to \mathbb{R} \text{ is Borel}\}$$
equals
$$\{\Sigma: \Sigma\text{ is the }\sigma\text{-algebra of }\phi\text{-measurable sets for some Borel probability measure }\phi\text{ on }\mathbb{R}\}.$$
Thus, $X \subset \mathbb{R}$ is universally measurable iff $X \in f_\ast\mathcal{E}$ for all Borel $f: \mathbb{R} \to \mathbb{R}$, i.e., iff $f^{-1}(X) \in \mathcal{E}$ for all Borel $f: \mathbb{R} \to \mathbb{R}$. $\square$
Remark: The same argument shows that $f^{-1}(X)$ is Lebesgue-measurable for all continuous $f$ (i.e., $X$ is persistently measurable, in your terminology) iff $X$ is measurable w.r.t. every Borel probability measure $\mu$ on $\mathbb{R}$ which is the pushforward of the Gaussian measure (or any Borel probability measure equivalent to the Lebesgue measure, for that matter) along a continuous function. This shows universal measurability implies persistent measurability. My guess is that the converse is also true. To prove this, one only needs to show that given any Borel probability measure $\mu$ on $\mathbb{R}$, there is a Borel probability measure $\phi$ on $\mathbb{R}$ s.t. it is the pushforward of the Gaussian measure along a continuous function and $\mu$ is absolutely continuous w.r.t. $\phi$. This is at least true whenever $\mu$ is such that its diffuse part is absolutely continuous w.r.t. the Lebesgue measure. I don't know how to prove the general case, though.
