After upgrading `kratos to 0.9`, sending emails do...
# talk-kratos
r
After upgrading
kratos to 0.9
, sending emails doesn't work on local mail server (
mailhog
or
mailslurper
) in docker. Is anyone having similar problem? it worked in
0.8
for me. logs say that it sent the mail but i don't receive anything. docker-compose:
# --------------------------------------------------------------------------------
mailhog:
image: mailhog/mailhog:latest
ports:
- "8025:8025"
- "1025:1025"
networks:
- intranet
# --------------------------------------------------------------------------------
mailslurper:
image: oryd/mailslurper:smtps-latest
ports:
- "4436:4436"
- "4437:4437"
- "1025:1025"
networks:
- intranet
# --------------------------------------------------------------------------------
i tried many things for `connection_uri`:
<smtp://mailhog:1025?disable_starttls=true&skip_ssl_verify=true>
<smtp://test:test@mailhog:1025?disable_starttls=true>
<smtp://mailslurper:1025?disable_starttls=true&skip_ssl_verify=true>
<smtp://test:test@mailslurper:1025?disable_starttls=true>
๐Ÿ‘ 1
f
I have exactly the same problem here, tried with both mailhog & mailslurper and fumbled with connection_uri just like @ripe-alarm-21163 ๐Ÿ˜ข
h
What errors do you get?
Copy code
<smtps://test:test@localhost:1025/?skip_ssl_verify=true>
this we use in the e2e tests together with:
Copy code
docker run --name mailslurper -p 4436:4436 -p 4437:4437 -p 1025:1025 oryd/mailslurper:latest-smtps
f
I have no errors at all: kratos says
Sending out verification email with verification link
, but the courier just doesn't pick it up ๐Ÿค”
There are new entries in "courier_messages" table though, but all of them have status 1
h
Is the courier running?
๐Ÿ‘ 1
f
I've tried adding a courier to docker-compose (as kratos courier watch), and now I'm getting errors:
Copy code
kratos_courier_1        | time=2022-03-29T09:48:32Z level=info msg=Courier worker started. audience=application service_name=Ory Kratos service_version=v0.9.0-alpha.3
kratos_courier_1        | time=2022-03-29T09:48:32Z level=error msg=Unable to send email using SMTP connection. audience=application error=map[message:dial tcp 127.0.0.1:1025: connect: connection refused stack_trace:
kratos_courier_1        | <http://github.com/ory/mail/v3.(*Dialer).Dial|github.com/ory/mail/v3.(*Dialer).Dial>
kratos_courier_1        | 	/go/pkg/mod/github.com/ory/mail/v3@v3.0.0/smtp.go:92
kratos_courier_1        | <http://github.com/ory/mail/v3.(*Dialer).DialAndSend|github.com/ory/mail/v3.(*Dialer).DialAndSend>
kratos_courier_1        | 	/go/pkg/mod/github.com/ory/mail/v3@v3.0.0/smtp.go:211
kratos_courier_1        | <http://github.com/ory/kratos/courier.(*courier).dispatchEmail|github.com/ory/kratos/courier.(*courier).dispatchEmail>
kratos_courier_1        | 	/project/courier/smtp.go:173
kratos_courier_1        | <http://github.com/ory/kratos/courier.(*courier).DispatchMessage|github.com/ory/kratos/courier.(*courier).DispatchMessage>
kratos_courier_1        | 	/project/courier/courier_dispatcher.go:13
kratos_courier_1        | <http://github.com/ory/kratos/courier.(*courier).DispatchQueue|github.com/ory/kratos/courier.(*courier).DispatchQueue>
kratos_courier_1        | 	/project/courier/courier_dispatcher.go:65
kratos_courier_1        | <http://github.com/ory/kratos/courier.(*courier).watchMessages.func1|github.com/ory/kratos/courier.(*courier).watchMessages.func1>
kratos_courier_1        | 	/project/courier/courier.go:85
kratos_courier_1        | <http://github.com/cenkalti/backoff.RetryNotify|github.com/cenkalti/backoff.RetryNotify>
kratos_courier_1        | 	/go/pkg/mod/github.com/cenkalti/backoff@v2.2.1+incompatible/retry.go:37
kratos_courier_1        | <http://github.com/cenkalti/backoff.Retry|github.com/cenkalti/backoff.Retry>
kratos_courier_1        | 	/go/pkg/mod/github.com/cenkalti/backoff@v2.2.1+incompatible/retry.go:24
kratos_courier_1        | <http://github.com/ory/kratos/courier.(*courier).watchMessages|github.com/ory/kratos/courier.(*courier).watchMessages>
kratos_courier_1        | 	/project/courier/courier.go:84
kratos_courier_1        | runtime.goexit
kratos_courier_1        | 	/usr/local/go/src/runtime/asm_amd64.s:1581] message_from=no-reply@test.io service_name=Ory Kratos service_version=v0.9.0-alpha.3 smtp_server=localhost:1025 smtp_ssl_enabled=false
@high-optician-2097 yes, courier was running together with main kratos service before, but it wasn't picking up emails. It started throwing errors when I ran the courier separately
h
looks like the port is not open
f
Here's what I've tried (not both of them at the same time of course)
Anyway, when I don't run courier as a singleton but via
watch-courier: true
, it says it's sending out an email, but mailhog/mailslurper doesn't react at all :<
h
Itโ€™s really hard to say anything without the full context or a full reproducible example. In the kratos repository, the e2e test pass fine - they do test whether emails arrive or not. We have tests in both the go code for email sending as well as full e2e. So it has to be some other issue but itโ€™s very difficult to say from screenshot excerpts. Can you reproduce this issue with the ory kratos quickstart as well? If so, there might be an issue. If not, itโ€™s probably local
f
Ok, I'll try, thank you!
Hmm, quickstart is having some db issues ๐Ÿ‘€ I've been following this guide: https://www.ory.sh/docs/kratos/quickstart#clone-ory-kratos-and-run-it-in-docker
Copy code
kratos_1                      | time=2022-03-29T12:19:47Z level=debug msg=Connecting to SQL Database func=<http://github.com/ory/kratos/driver.(*RegistryDefault).Init.func1|github.com/ory/kratos/driver.(*RegistryDefault).Init.func1> file=/project/driver/registry_default.go:537 audience=application connMaxLifetime=0s idlePool=8 pool=16 service_name=Ory Kratos service_version=v0.9.0-alpha.3
kratos_1                      | time=2022-03-29T12:19:48Z level=warning msg=Unable to determine network, retrying. func=<http://github.com/ory/x/logrusx.(*Logger).Logf|github.com/ory/x/logrusx.(*Logger).Logf> file=/go/pkg/mod/github.com/ory/x@v0.0.358/logrusx/helper.go:118 audience=application error=map[debug: message:Unable to locate the table reason: stack_trace:
kratos_1                      | <http://github.com/ory/x/sqlcon.handleSqlite|github.com/ory/x/sqlcon.handleSqlite>
kratos_1                      | 	/go/pkg/mod/github.com/ory/x@v0.0.358/sqlcon/error_sqlite.go:26
kratos_1                      | <http://github.com/ory/x/sqlcon.HandleError|github.com/ory/x/sqlcon.HandleError>
kratos_1                      | 	/go/pkg/mod/github.com/ory/x@v0.0.358/sqlcon/error.go:89
kratos_1                      | <http://github.com/ory/x/networkx.(*Manager).Determine|github.com/ory/x/networkx.(*Manager).Determine>
kratos_1                      | 	/go/pkg/mod/github.com/ory/x@v0.0.358/networkx/manager.go:42
kratos_1                      | <http://github.com/ory/kratos/persistence/sql.(*Persister).DetermineNetwork|github.com/ory/kratos/persistence/sql.(*Persister).DetermineNetwork>
kratos_1                      | 	/project/persistence/sql/persister.go:80
kratos_1                      | <http://github.com/ory/kratos/driver.(*RegistryDefault).Init.func1|github.com/ory/kratos/driver.(*RegistryDefault).Init.func1>
kratos_1                      | 	/project/driver/registry_default.go:582
kratos_1                      | <http://github.com/cenkalti/backoff.RetryNotify|github.com/cenkalti/backoff.RetryNotify>
kratos_1                      | 	/go/pkg/mod/github.com/cenkalti/backoff@v2.2.1+incompatible/retry.go:37
kratos_1                      | <http://github.com/cenkalti/backoff.Retry|github.com/cenkalti/backoff.Retry>
kratos_1                      | 	/go/pkg/mod/github.com/cenkalti/backoff@v2.2.1+incompatible/retry.go:24
kratos_1                      | <http://github.com/ory/kratos/driver.(*RegistryDefault).Init|github.com/ory/kratos/driver.(*RegistryDefault).Init>
kratos_1                      | 	/project/driver/registry_default.go:523
kratos_1                      | <http://github.com/ory/kratos/driver.New|github.com/ory/kratos/driver.New>
kratos_1                      | 	/project/driver/factory.go:15
kratos_1                      | <http://github.com/ory/kratos/cmd/serve.NewServeCmd.func1|github.com/ory/kratos/cmd/serve.NewServeCmd.func1>
kratos_1                      | 	/project/cmd/serve/root.go:33
kratos_1                      | <http://github.com/spf13/cobra.(*Command).execute|github.com/spf13/cobra.(*Command).execute>
kratos_1                      | 	/go/pkg/mod/github.com/spf13/cobra@v1.3.0/command.go:860
kratos_1                      | <http://github.com/spf13/cobra.(*Command).ExecuteC|github.com/spf13/cobra.(*Command).ExecuteC>
kratos_1                      | 	/go/pkg/mod/github.com/spf13/cobra@v1.3.0/command.go:974
kratos_1                      | <http://github.com/spf13/cobra.(*Command).Execute|github.com/spf13/cobra.(*Command).Execute>
kratos_1                      | 	/go/pkg/mod/github.com/spf13/cobra@v1.3.0/command.go:902
kratos_1                      | <http://github.com/ory/kratos/cmd.Execute|github.com/ory/kratos/cmd.Execute>
kratos_1                      | 	/project/cmd/root.go:48
kratos_1                      | main.main
kratos_1                      | 	/project/main.go:35
kratos_1                      | runtime.main
kratos_1                      | 	/usr/local/go/src/runtime/proc.go:255
kratos_1                      | runtime.goexit
kratos_1                      | 	/usr/local/go/src/runtime/asm_amd64.s:1581 status:Internal Server Error status_code:500] service_name=Ory Kratos service_version=v0.9.0-alpha.3
h
Thatโ€™s ok, it means the migrations did not run yetโ€ฆ
f
Just above it it says
Copy code
kratos-migrate_1              | Successfully applied SQL migrations!
kratos_kratos-migrate_1 exited with code 0
The UI works, but when I click on login/signup/recover/verify buttons, I get "The connection was reset" :<
Oh, my bad - my system doesn't have /var/lib/sqlite ๐Ÿ˜„ I've updated the docker-compose sqlite volume paths and now it works
h
Oh I see ๐Ÿ™‚ Ok cool that it works!
f
And the email has indeed arrived
Oh, I think I've found out what the problem was... There was
watch-courier: true
in my YAML config, but apparently it stopped working, because when I've added
--watch-courier
flag to my
kratos serve
command in docker-compose, it started sending out emails!
h
Oh, yes, that is a regression! @ripe-alarm-21163 this might be your problem too
๐Ÿ‘ 1
Is
watch-courier
in the config file in the official resources somewhere?
f
Yeah, it's still in the configuration reference: https://www.ory.sh/docs/kratos/reference/configuration
h
Oh I see, but it does say:
This is a CLI flag and environment variable and can not be set using the config file.
๐Ÿ˜„
๐Ÿ˜… 1
f
lol ๐Ÿ˜… Wouldn't it make sense to remove it altogether though? And maybe add some note to changelog, because I believe a lot of people still rely on this config value, at least in dev environments? ๐Ÿ™ˆ
Oh, I think I get it now... It has to be there in order for the environment variable to work, right?
Anyway, I think it's worth mentioning in the changelog as people can easily overlook this comment in config reference just as me and @ripe-alarm-21163 did ๐Ÿ˜
h
yeah the problewm is that this is autogenerated and itโ€™s not properly being removed from the document there ๐Ÿ˜…
so it should have never worked as a config value in the first place
r
@high-optician-2097 yes exactly, i also had watch-courier: true in my kratos.yaml file
a
I just ran into this issue with the upgrade from v0.8.0-alpha.3 to v0.9.0-alpha.3. I have the watch courier set in cli and config, but still can't get emails. Also tried the various connection string options. Still no emails in mailslurper and no errors.
f
Try running courier as a standalone service:
kratos courier watch
, this might help
r
this is what i have in my docker-compose and it works fine with the --watch-courier flag:
kratos:
command: ["serve", "--watch-courier", "--config", "/etc/config/kratos/kratos.yaml", "--dev"]
a
Running courier didn't seem to have any effect. I saw that message when it ran with the same command as above.
Start command:
Copy code
kratos serve -c "$CONFIG_PATH" --dev --watch-courier
I added this to the config file and still no change.
Copy code
watch-courier: true
latest try with the smtp setting for testing...
Copy code
<smtp://test:test@localhost:1025/?skip_starttls=true&skip_ssl_verify=true>
r
watch-courier: true has no effect in kratos.yml conf, only CLI flag works, not sure what's going on in your case
t
I actually got the same error for AWS SES. I use smtps:// port 587 with no params.
Copy code
{
  "audience": "application",
  "error": {
    "message": "tls: first record does not look like a TLS handshake"
  },
  "level": "error",
  "message_from": "<mailto:no-reply@mail.mydomain.com|no-reply@mail.mydomain.com>",
  "msg": "Unable to send email using SMTP connection.",
  "service_name": "Ory Kratos",
  "service_version": "v0.9.0-alpha.3",
  "smtp_server": "<http://email-smtp.us-east-1.amazonaws.com:587|email-smtp.us-east-1.amazonaws.com:587>",
  "smtp_ssl_enabled": true,
  "time": "2022-04-06T23:19:18-04:00"
}
And running this command works for me
Copy code
nc -vz <http://email-smtp.us-east-1.amazonaws.com|email-smtp.us-east-1.amazonaws.com> 587
Connection to <http://email-smtp.us-east-1.amazonaws.com|email-smtp.us-east-1.amazonaws.com> port 587 [tcp/submission] succeeded!