https://www.ory.sh/ logo
Join Slack
Powered by
We are trying to setup oathkeeper and kratos on gk...
# ory-selfhosting
p
We are trying to setup oathkeeper and kratos on gke and we cannot figure out how to setup ingress controller health check for oathkeeper. We have setup two services one exposed on port 4455 as NodePort and the other 4456 as ClusterIP. The health check is HTTP based on port 4455 and it keeps failing. When we try to update it to TCP, it gets rolled back to HTTP. Has anyone faced a similar issue before? Does anyone have an example of ingress configuration for oathkeeper?
m
hello @User while i am not familiar with your particular error, this discussion could be helpful: https://github.com/ory/oathkeeper/issues/624 we should include it in the documentation 🤓
p
Thanks Vincent. Do you have a similar example or something for nginx ingress controller too?
The only thing I am looking for is to be able to configure nginx ingress to be able to perform health check on the oathkeeper service
m
Sorry Aman, I dont know of an example for Nginx, there might be something hidden in the github discussions. If you are able to figure it out, please open a similar discussion, that would help a lot thanks!
p
I understand. Thanks for your response. I realize that this is an nginx problem. I should find a place related to nginx.
🙏 1
w
Hi there, the k8s healthchecks are on the pod/localhost level and should not interfere with services at all
unless you are talking about some external check?
p
These checks are on the oathkeeper service that exposes both port 4455, 4456.
The challenge here is that the ingress controller needa just one rule tp forward traffic to 4455
And it automatically setups a health check on 4455.
w
are those some GKE based healtchecks, and not those defined for the k8s deployment?
p
Correct.
The k8s health check on the deployment works fine
I'm on GKE, and I have not been able to figure out a way to setup either a TCP health check on 4455 port, or ask GKE to check port 4456 instead.
It keeps hitting 4455 and keeps getting a 400 or something because there is no rule for health check endpoint in oathkeeper rules
w
I see, i think in this scenario you might want to try to restructure the stack a bit. Have you tried exposing oathkeeper via ingress and not directly by external facing services? AFAIK gke healthchecks are created for external facing services, like ingress gateway pods
so the network stack should be : user -> ingress-gateway -> oathkeeper -> target application
p
Yeah, that is how it is. Ingress gateway is our ingress nginx controller
The ingress controller needs a health check and I'm not sure why it creates a backend service health check for it
On the same port
w
then you can change the service types to ClusterIP
p
They are set to NodePort right now. Do you think that'd help?
Can ingress controller reach a service on ClusterIP?
w
NodePort would expose the application on the ports of the actual node, which you’d hardly want to do
yes, the ingress is supposed to be a single point of entry to the cluster
you should not need to expose anything else
p
I see. Makes sense. How'd that fix the health check though?
w
do you use our charts for deployment, or some homebrew stack?
p
I created k8s deployement and service files myself
I can share those with you, if it helps
(Share here)
w
I’d recommend to take a look here https://github.com/ory/k8s/tree/master/helm/charts/oathkeeper 😉
p
I tried that and I'll try again. I felt like they were too many variables to understand those.
w
sure i can understand that 🙂 You can use helm to create the files for you, and then modify them to suit your needs/simplify, but they are a good point of reference
helm template ./path
will print out the manifests which you can then manage on your own
🙏 1
p
Ooh, I didn't think of that. It'll simplify the files a lot. Let me try. Thank you so much.
👍 1
2 Views
Open in Slack
PreviousNext
Powered by