[hydra v2.2.0 performance bottlenecking our system...
# ory-selfhosting
m
[hydra v2.2.0 performance bottlenecking our system] This is from a trace while doing a casual k6 load test after upgrading hydra from 1.10.5 to v2.2.0. What I noticed is the requests are significantly slower due to hydra introspect (no other operations here aside from the sdk call) Seeing this, I did some quick 1m duration load testing (via k6) over hydra directly, the results are as follows: • hydra v1 on
/oauth2/introspect
RPS - 34.779 • hydra v2 on
/oauth2/introspect
RPS - 16.27 Of course, I've controlled the cpu/mem resources for both deployments to make the experiment consistent. infra details: postgres, k8s, both hydras deployed from helm with pretty much the same values.yaml Now hesitant to push this upgrade to production. šŸ˜„ So far I'm getting 2x less RPS instead of the promised over 6x improvement
šŸ”“ 1
šŸ‘€ 1
official otel trace, but not sure what to make of this. but as seen in the pic,
Fosite.IntrospectToken
runs for
12.5 seconds
under high load
so far I've tried recreating the db as well as changing
?max_conns
in postgres dsn, but to no avail. cc: @magnificent-energy-493 sorry for the tag. been stuck investigating this for days already
m
Hey carlos, no worries! I am not sure what is the cause here to be honest, and also not that familiar with Hydra performance eval šŸ¤” For context, you are using Hydra 1.10.5. in production and are looking to upgrade for the performance improvements?
m
Appreciate the reply! Pretty much yep. We've been using hydra for about 4 years now. (as well as keto actually -- I guess we're early adopters in a sense šŸ™‚ ). We saw promising improvements in hydra v2 so we decided to check it out on staging first. Did some performance sanity checks, and then boom. got these results. Hopefully I'm just missing something simple and I don't have to revert all my code changes. šŸ™