Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when sending an email in Java with CompletableFuture.runAsync #104

Open
disbrain opened this issue Jul 21, 2016 · 19 comments
Open

Error when sending an email in Java with CompletableFuture.runAsync #104

disbrain opened this issue Jul 21, 2016 · 19 comments

Comments

@disbrain
Copy link

disbrain commented Jul 21, 2016

I just migrated from Play 2.3 to Play 2.5 but now I can't send mails anymore.

  • I Modified my conf file entry accordingly to the new format
  • I created an Injector Constructor for injecting mailer
    @Inject
    public SmtpConnector(MailerClient mailer)
    {
        this.mailerClient = mailer;
        MailcapCommandMap mc = (MailcapCommandMap) MailcapCommandMap.getDefaultCommandMap();
        mc.addMailcap("text/html;; x-java-content-handler=com.sun.mail.handlers.text_html");
        mc.addMailcap("text/xml;; x-java-content-handler=com.sun.mail.handlers.text_xml");
        mc.addMailcap("text/plain;; x-java-content-handler=com.sun.mail.handlers.text_plain");
        mc.addMailcap("multipart/*;; x-java-content-handler=com.sun.mail.handlers.multipart_mixed");
        mc.addMailcap("message/rfc822;; x-java-content-handler=com.sun.mail.handlers.message_rfc822");
        CommandMap.setDefaultCommandMap(mc);
    }
  • I execute the sending in an async task:
CompletableFuture.runAsync( () -> mailerClient.send(email) ).exceptionally(exc -> {exc.printStackTrace(); return null;});

But now I get the following exception:

java.util.concurrent.CompletionException: org.apache.commons.mail.EmailException: Sending the email to the following server failed : 127.0.0.1:25
    at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
    at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
    at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1629)
    at java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1618)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: org.apache.commons.mail.EmailException: Sending the email to the following server failed : 127.0.0.1:25
    at org.apache.commons.mail.Email.sendMimeMessage(Email.java:1421)
    at org.apache.commons.mail.Email.send(Email.java:1448)
    at play.api.libs.mailer.SMTPMailer$$anon$2.send(MailerPlugin.scala:100)
    at play.api.libs.mailer.CommonsMailer.send(MailerPlugin.scala:130)
    at play.api.libs.mailer.SMTPMailer.send(MailerPlugin.scala:110)
    at play.api.libs.mailer.SMTPDynamicMailer.send(MailerPlugin.scala:117)
    at play.api.libs.mailer.MailerClient$class.send(MailerPlugin.scala:56)
    at play.api.libs.mailer.SMTPDynamicMailer.send(MailerPlugin.scala:114)
    at assets.SmtpConnector.lambda$send_mail$1(SmtpConnector.java:57)
    at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
    ... 5 more
Caused by: javax.mail.MessagingException: IOException while sending message;
  nested exception is:
    javax.activation.UnsupportedDataTypeException: no object DCH for MIME type multipart/alternative; 
    boundary="----=_Part_0_1284684208.1469102367572"
    at com.sun.mail.smtp.SMTPTransport.sendMessage(SMTPTransport.java:1177)
    at javax.mail.Transport.send0(Transport.java:195)
    at javax.mail.Transport.send(Transport.java:124)
    at org.apache.commons.mail.Email.sendMimeMessage(Email.java:1411)
    ... 14 more
Caused by: javax.activation.UnsupportedDataTypeException: no object DCH for MIME type multipart/alternative; 
    boundary="----=_Part_0_1284684208.1469102367572"
    at javax.activation.ObjectDataContentHandler.writeTo(DataHandler.java:896)
    at javax.activation.DataHandler.writeTo(DataHandler.java:317)
    at javax.mail.internet.MimeBodyPart.writeTo(MimeBodyPart.java:1485)
    at javax.mail.internet.MimeMessage.writeTo(MimeMessage.java:1773)
    at com.sun.mail.smtp.SMTPTransport.sendMessage(SMTPTransport.java:1121)
    ... 17 more

I touched NOTHING except the mailer/play version in build.sbt. Any clues please?

@disbrain disbrain changed the title [Java] Error sending mail Error sending mail Jul 21, 2016
@ggrossetie
Copy link
Member

Looks like a classpath issue but you already added MailcapCommandMap.
References:

Do you have a dependency on javax.mail or commons-email in your build.sbt. Could you please provide a sample project to reproduce this issue ?

@disbrain
Copy link
Author

No, I don't have such dependencies.
I attached a very minimal sample project, thanks a lot
IssueTest.zip

@ggrossetie
Copy link
Member

I think this issue is related to the ForkJoinPool.commonPool()
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ForkJoinPool.html#commonPool--

I don't know why but threads in this pool have a "special" classpath and javax.mail dependency is not loaded correctly.

If you use another Executor then the code is working. For instance I'm using a Executors.newSingleThreadExecutor():

CompletableFuture.runAsync(() -> mailerClient.send(email), Executors.newSingleThreadExecutor())
  .exceptionally(exc -> {
    exc.printStackTrace();
    return null;
});

The following is also working:

private static ForkJoinPool commonExecutor = new ForkJoinPool(ForkJoinPool.getCommonPoolParallelism());

CompletableFuture.runAsync(() -> mailerClient.send(email), commonExecutor)
  .exceptionally(exc -> {
    exc.printStackTrace();
    return null;
});

NOTE: In this cases you don't need to redefine MailcapCommandMap

CAUTION: I've never used this API before so maybe there's a better, simpler workaround

@disbrain
Copy link
Author

disbrain commented Jul 22, 2016

Hi Mogztter, I tried, but the behavior is the same, I still get the exception
I added debug to javax.activation and as extra info now I get:

Can't load DCH com.sun.mail.handlers.multipart_mixed; Exception: java.lang.ClassNotFoundException: com/sun/mail/handlers/multipart_mixed

Could be all this related about the new play classloader? I use oracle java 8

@ggrossetie
Copy link
Member

Are you running Play! in debug mode ? in production mode ? I'm running the application with activator run.

Here's the modified files:

build.bt

name := """IssueTest"""

version := "1.0-SNAPSHOT"

lazy val root = (project in file(".")).enablePlugins(PlayJava)

scalaVersion := "2.11.7"
routesGenerator := StaticRoutesGenerator
libraryDependencies ++= Seq(
  javaJdbc,
  cache,
  javaWs,
  "org.webjars" %% "webjars-play" % "2.3.0",
  "org.webjars" % "bootstrap" % "3.1.1-2",
  "com.typesafe.akka" % "akka-remote_2.11" % "2.4.8",
  "com.typesafe.play" %% "play-mailer" % "5.0.0"
)

SmtpConnector.java

package goodies;

import play.libs.mailer.Email;
import play.libs.mailer.MailerClient;

import javax.inject.Inject;
import java.util.Arrays;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ForkJoinPool;

public class SmtpConnector {

    private final static String DEFAULT_EMAIL_SENDER = "[email protected]";
    private final MailerClient mailerClient;

    private static ForkJoinPool commonExecutor = new ForkJoinPool(ForkJoinPool.getCommonPoolParallelism());

    @Inject
    public SmtpConnector(MailerClient mailer) {
        this.mailerClient = mailer;
    }

    public void send_mail(String[] recipients, String subject, String textBody, Optional<String> htmlBody) {
        send_mail(DEFAULT_EMAIL_SENDER, recipients, subject, textBody, htmlBody);
    }

    public void send_mail(String sender, String[] recipients, String subject, String textBody, Optional<String> htmlBody) {
        if (sender == null || recipients == null) {
            return;
        }
        Email email = new Email()
                .setFrom(sender)
                .setSubject(subject)
                .setBodyText(textBody);
        Arrays.stream(recipients).forEach(recipient -> email.addTo(recipient));

        if (htmlBody.isPresent()) {
            email.setBodyHtml(htmlBody.get());
        }
        CompletableFuture.runAsync(() -> mailerClient.send(email), commonExecutor)
                .exceptionally(exc -> {
                    exc.printStackTrace();
                    return null;
                });
    }
}

Could be all this related about the new play classloader?

Probably

I use oracle java 8

Me too

java version "1.8.0_31"
Java(TM) SE Runtime Environment (build 1.8.0_31-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode)

@disbrain
Copy link
Author

disbrain commented Jul 22, 2016

I found that specifying executor works, but only if the email sending method is called directly from the Play controller method.
In my setting it's called in an async reply handlng method (to an akka request) and within a class instantiated through reflection:

.thenApplyAsync(new Function<Message, JsonNode>() {
  @Override
  public JsonNode apply(Message response) {
    try
    {
      Object json_reply = output_parametrized_constructor.newInstance(response, parameter.get(),http_ctx);
    }

and in the class construtor

EmailService.send_async_account_validation_email(...)

I don't know if even moving out the sending from the constructor and reflectively calling a method for that on the just built object would fix the problem because the classloader context should not be Play's anyway, right?

@ggrossetie
Copy link
Member

ggrossetie commented Jul 22, 2016

This seems rather complex but I don't know your use case.
I think we need to find the root cause, so far we just found a workaround.

If we send an email synchronously everything is working fine. So I'm pretty sure that the classloader context is different when using CompletableFuture.
Could you please try without using runtime DI ? https://github.com/playframework/play-mailer/blob/master/user-manual.adoc#new-instances

@disbrain
Copy link
Author

disbrain commented Jul 22, 2016

I tried, with and w/o explicitly setting MailcapCommandMap before sending the email, but it's still not working.

Here the class I used:

public class SmtpConnector {

    private final static MailerClient mailerClient = new SMTPMailer(
            new SMTPConfiguration(
                    "127.0.0.1",
                    25,
                    false,
                    false,
                    scala.Option.apply(null),
                    scala.Option.apply(null),
                    false,
                    scala.Option.apply(null),
                    scala.Option.apply(null),
                    false));
    private final static ForkJoinPool mail_pool = new ForkJoinPool(ForkJoinPool.getCommonPoolParallelism());

    public static void send_mail(String[] recipients, String subject, String textBody, Optional<String> htmlBody) {
        send_mail(Preferences.DEFAULT_EMAIL_SENDER, recipients, subject, textBody, htmlBody);
    }

    public static void send_mail(String sender, String[] recipients, String subject, String textBody, Optional<String> htmlBody) {
        if (sender == null || recipients == null)
        {
            return;
        }

        Email email = new Email()
                .setFrom(sender)
                .setSubject(subject)
                .setBodyText(textBody);
        Arrays.stream(recipients).forEach(recipient -> email.addTo(recipient));

        if (htmlBody.isPresent())
            email.setBodyHtml(htmlBody.get());

        //async call for sending email
        CompletableFuture.supplyAsync( () -> mailerClient.send(email),mail_pool).exceptionally(exc -> {exc.printStackTrace(); return null;});

    }
}

@ggrossetie
Copy link
Member

This is working for me. Are you testing with your sample projet or with your full project ? Did you tried to use Executors.newSingleThreadExecutor() instead of a ForkJoinPool ?

@disbrain
Copy link
Author

disbrain commented Jul 22, 2016

Yeah, but nothing changed.
I was able to replicate the issue within a new test project. Now accessing the index page triggers an async email sending and it fails w/wo the MailcapCommandMap
IssueTest-v2.zip

EDIT
I attach the -verbose:class and getClass().getClassLoaders() dumps
classloader-dump.zip
verbose-class.zip

@ggrossetie
Copy link
Member

I'm not familiar with this Java API but I think you are doing too much asynchronous:

  • supplyAsync runs in a ForkJoinPool.
  • thenApplyAsync runs in a ForkJoinPool.
  • send_mail runs in a newSingleThreadExecutor.

Why not simplify:

HomeController.java

    public static CompletionStage<Result> index() {
        try {
            return CompletableFuture.supplyAsync(() -> build_me)
                    .thenApply(new Function<Constructor<?>, Result>() {
                        @Override
                        public Result apply(Constructor<?> constructor) {
                            try {
                                constructor.newInstance(null);
                            } catch (Exception exc) {
                                exc.printStackTrace();
                                return internalServerError("D'OH");
                            }
                            return ok("YAY");
                        }
                    });
        } catch (Exception exc) {
            exc.printStackTrace();
            return CompletableFuture.completedFuture(internalServerError());
        }
    }

SmtpConnector.java

    public void send_mail(String sender, String[] recipients, String subject, String textBody, Optional<String> htmlBody) {
        if (sender == null || recipients == null)
        {
            return;
        }
        Email email = new Email()
                .setFrom(sender)
                .setSubject(subject)
                .setBodyText(textBody);
        Arrays.stream(recipients).forEach(recipient -> email.addTo(recipient));

        if (htmlBody.isPresent()) {
            email.setBodyHtml(htmlBody.get());
        }
        mailerClient.send(email);
    }

Also why are you using reflection ?

@disbrain
Copy link
Author

disbrain commented Jul 22, 2016

That was just an example on how to reproduce the issue, In my real setting I have a generic webservice handling system that validates requests, queries backend and build responses automatically and asynchrously, so I have no idea about the types that are going to be built/returned by the system. Basically every webservice have to specify only which kind of object it wants as a reply and parsing/validation/querying is all automatic:

    private final static String SET_TASK_PARAMETERS_ROUTER = play.Play.application().configuration().getString("actor.set_task_parameters");
    private final static GenericBrainAsyncer set_task_parameters_ws = new GenericBrainAsyncer(
            SetTaskParametersInput.class,
            GenericRestOutputGenerator.class,
            SET_TASK_PARAMETERS_ROUTER,
            WebServicesHotCache.GENERIC_TIMEOUT_REPLY);

    @CorsRest
    @VerboseRest
    @RequireAuthentication
    @BodyParser.Of(BodyParser.Json.class)
    public static CompletionStage<Result> set_task_parameters() {
        return set_task_parameters_ws.WS_behave(request().body().asJson());
    }

In this example, the ws request will be validated using SetTaskParametersInput class model, the resulting querying object will be sent to the backend, and the reply from the backend will be generated/analyzed using GenericRestOutputGenerator. All the async chaining stuff since is a pattern common to every service (validation, querying, analysis, output generation), is done be the generic ws_behave. But since I have very different validation models for every service, I need to be generic. Actually I could not use routes (I could create a service dispatcher with the same logic), I use them just for readability

Since the code of input builders and validators/output analyzers can be automatically generated too with a special compiler, in this way I can autogenerate the code of every new webservice regardless of what it takes, gives or what it does simply using a name convention in the actor conf file

In my case I have to be asynchronous with the email sending because I don't want to add the email-sending delay while generating the response object the user, and only inside the reply validator I will have the infos required for the email sending, since they are been validated and extracted from the request/reply by an automatic validator

@ggrossetie
Copy link
Member

Thanks for this detailed explanation.
Just to make sure that this issue is related to classpath context in an async environment (and not specific to play-mailer), could you please try to send an email directly with commons-email ?

@ggrossetie ggrossetie changed the title Error sending mail UnsupportedDataTypeException: no object DCH for MIME when sending an email in Java with CompletableFuture.runAsync Jul 22, 2016
@ggrossetie ggrossetie changed the title UnsupportedDataTypeException: no object DCH for MIME when sending an email in Java with CompletableFuture.runAsync Error when sending an email in Java with CompletableFuture.runAsync Jul 22, 2016
@ggrossetie
Copy link
Member

Since you are using Akka you could also try to send an email asynchronously with:

system.scheduler().scheduleOnce(
    Duration.create(10, TimeUnit.MILLISECONDS),
    () -> // Send email here,
    system.dispatcher()
);

@disbrain
Copy link
Author

disbrain commented Jul 25, 2016

I confirm that even using commons-email it doesn't work due to the same error.

I found out that
CompletableFuture.runAsync( () -> { Thread.currentThread().setContextClassLoader( getClass().getClassLoader() ); mailerClient.send(email);}, Executors.newSingleThreadExecutor()).exceptionally(exc -> {exc.printStackTrace(); return null;});

seems another workaround for the issue

@ggrossetie
Copy link
Member

I don’t know if you already read this page but you could try to use the application class loader to load play-mailer: https://www.playframework.com/documentation/2.5.x/ThreadPools#class-loaders-and-thread-locals

@gmethvin any idea about the root cause ? These workarounds are not perfect especially for a Play! plugin. I think plugin integration (classpath context...) should be easier in Java.

@gmethvin
Copy link
Member

Hi @disbrain,

Looking up the exception you got, it appears the same solution has been mentioned on stackoverflow quite long ago, though there are no references to CompletableFuture. You can use the application class loader if you want but it's going to basically do the same thing as your workaround.

I'm not that familiar with JavaMail so I don't know if I can help much. Seems like there are certain expectations JavaMail has about how the context class loader is set up. I'd suggest asking on the Play mailing list; maybe someone there has a better solution.

@disbrain
Copy link
Author

Yes @Mogztter but even now that I perfectly understand what it means I think that's not useful at all how is written, just confusing.
Yes @gmethvin thank you, I figured it out: It wasn't working because I put the setContext in the email sending method, not in the runAsync body

@me-ydv-5
Copy link

me-ydv-5 commented Jul 9, 2018

Sorry for spamming everyone here, but looks like the task gets completed only with workaround told by @Mogztter. Is anyone able to reason out why it works only in case of new Execution Context. Thanks for the workaround though, it has literally solved my long pending problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants