If you expose your private database's raw SQL access to public web, i bet people will find a way.

The same way here, i see the main issue isn't prompt injection, it is publicly accessible agent having access to private repos. What is the important use case for such a config that it warrants such basic security violation?