We may be talking across each other here. Or I may be wrong about the details.
Instead of
ports:
- 8080:9090
You can use
expose:
- 9090
And that port will only be usable inside the docker network, so caddy could still reverse proxy to it but nothing from outside will be able to access it.