https://www.ory.sh/ logo
Join Slack
Powered by
Trying to terraform up <keto> from helm chart but ...
# talk-keto
m
Trying to terraform up keto from helm chart but getting an error saying the chart does not exist. Any ideas?
Copy code
resource "helm_release" "keto" {
  name       = "ory"
  repository = "<https://k8s.ory.sh/helm/charts>"
  chart      = "ory"
  ...
}
Copy code
Error: could not download chart: chart "ory" not found in <https://k8s.ory.sh/helm/charts> repository
t
Copy code
chart      = "ory"
This should be
Copy code
chart      = "keto"
m
ah of course, that worked. Thanks!
I've deployed a managed cloudSql postgres in gcp and following the instructions by deploying both keto and gcloud-sqlproxy chart. Got a crashloopbackoff when trying to deploy keto tho and the error mentioned a database.yml file. Any ideas how to configure this file? Could not find it under any doumentation
Copy code
[POP] 2023/09/08 13:10:56 warn - One or more of connection details are specified in database.yml. Override them with values in URL.
time=2023-09-08T13:10:56Z level=error msg=Unable to ping the database connection, retrying. audience=application error=map[message:failed to connect to `host=pg-sqlproxy-gcloud-sqlproxy user=postgres database=db`: hostname resolving error (lookup pg-sqlproxy-gcloud-sqlproxy on 10.10.11.10:53: no such host)] service_name=Ory Keto service_version=v0.11.1-alpha.0
t
Are you running the cloudsql-proxy as a sidecar container, or a standalone pod?
m
Originally as a standalone pod but now as a sidecar containers. Although, I'm unsure how to correctly configure it since the helm chart for keto does not offer a mechanism to pass additional arguments when setting up the extra container. Created issue
t
You can pass any YAML to 
extraContainers
Here's an example that we use for our 
cloudsql-proxy
sidecar containers
Copy code
extraContainers: |
        - name: cloud-sql-proxy
          image: <http://gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.2.0|gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.2.0>
          imagePullPolicy: Always
          args:
          - "--structured-logs"
          - "--health-check"
          - "--http-address=0.0.0.0"
          - "--port=5432"
          - "--private-ip"
          - <INSTANCE>
          securityContext:
            runAsNonRoot: true
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
          livenessProbe:
            httpGet:
              path: /liveness
              port: 9090
            initialDelaySeconds: 0
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 2
          readinessProbe:
            httpGet:
              path: /readiness
              port: 9090
            initialDelaySeconds: 0
            periodSeconds: 10
            timeoutSeconds: 5
            successThreshold: 1
            failureThreshold: 2
          startupProbe:
            httpGet:
              path: /startup
              port: 9090
            periodSeconds: 1
            timeoutSeconds: 5
            failureThreshold: 20
          resources:
            requests:
              memory: 128Mi
              cpu: 50m
            limits:
              memory: 512Mi
              cpu: 250m
You'll need the same on the migrations container too, if you're running migrations as a 
job
m
Updated the dsn as well with your suggestion and now that error has gone away
Copy code
dsn: postgres://${local.db_username}:${local.password}@localhost:${local.sql_port}
Tho adding in this extraConfiguration to jobs has surfaced another issue. Despite the logs for the job resulting in a 
Successfully applied all migrations
message, the kubernetes status on the pod is set to NotReady. Is there some documentation with details I am missing on handling these error cases / recommendations on setting up I have not read up?
Copy code
time=2023-09-10T12:12:40Z level=error msg=Unable to ping the database connection, retrying. audience=application error=map[message:failed to connect to `host=127.0.0.1 user=postgres database=`: dial error (dial tcp 127.0.0.1:5432: connect: connection refused)] service_name=Ory Keto service_version=v0.11.1-alpha.0
[POP] 2023/09/10 12:12:47 warn - One or more of connection details are specified in database.yml. Override them with values in URL.
time=2023-09-10T12:12:47Z level=info msg=No tracer configured - skipping tracing setup audience=application service_name=Ory Keto service_version=v0.11.1-alpha.0
Current status:
Version			Name					Status
20150100000001000000	networks				Pending
20201110175414000000	relationtuple				Pending
20201110175414000001	relationtuple				Pending
20210623162417000000	relationtuple				Pending
20210623162417000001	relationtuple				Pending
20210623162417000002	relationtuple				Pending
20210623162417000003	relationtuple				Pending
20210914134624000000	legacy-cleanup				Pending
20220217152313000000	nid_fk					Pending
20220512151000000000	indices					Pending
20220513200300000000	create-intermediary-uuid-table		Pending
20220513200400000000	create-uuid-mapping-table		Pending
20220513200400000001	uuid-mapping-remove-check		Pending
20220513200500000000	migrate-strings-to-uuids		Pending
20220513200600000000	drop-old-non-uuid-table			Pending
20220513200600000001	drop-old-non-uuid-table			Pending
20230228091200000000	add-on-delete-cascade-to-relationship	Pending
Applying migrations...
Successfully applied all migrations:
Version			Name					Status
20150100000001000000	networks				Applied
20201110175414000000	relationtuple				Applied
20201110175414000001	relationtuple				Applied
20210623162417000000	relationtuple				Applied
20210623162417000001	relationtuple				Applied
20210623162417000002	relationtuple				Applied
20210623162417000003	relationtuple				Applied
20210914134624000000	legacy-cleanup				Applied
20220217152313000000	nid_fk					Applied
20220512151000000000	indices					Applied
20220513200300000000	create-intermediary-uuid-table		Applied
20220513200400000000	create-uuid-mapping-table		Applied
20220513200400000001	uuid-mapping-remove-check		Applied
20220513200500000000	migrate-strings-to-uuids		Applied
20220513200600000000	drop-old-non-uuid-table			Applied
20220513200600000001	drop-old-non-uuid-table			Applied
20230228091200000000	add-on-delete-cascade-to-relationship	Applied
Although the kubernetes status on the job pod is set to NotReady, the underlying reason was that the startup probe was failing. The logs did indicate correctly that the migration applied correctly. To get keto working, here was my flow: 1. enable autoMigration 2. bring up keto with jobs with cloudSQL proxy containers on both 3. see the keto migration pod logs: "Successfully applied all migrations" a. The kubernetes status will be set to NotReady with the startup probe failing for these pods b. Migration was applied to database 4. Disable autoMigration 5. Bring up keto which will now work as expected a. If autoMigration is enabled, then it will fail for above reason I tried doubling the startupProbe values but no luck so have this workaround for now. Strange enough, deployment.extraContainers was perfectly fine but job.extraContainers showed this behavior
Created Github Issue for job always resulting in NotReady status
5 Views
Open in Slack
PreviousNext
Powered by