Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22172

Worker hangs when the external shuffle service port is already in use

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.3.0
    • Spark Core
    • None

    Description

      When the external shuffle service port is already in use, Worker throws the below BindException and hangs forever, I think the exception should be handled gracefully.

      17/09/29 11:16:30 INFO ExternalShuffleService: Starting shuffle service on port 7337 (auth enabled = false)
      17/09/29 11:16:30 ERROR Inbox: Ignoring error
      java.net.BindException: Address already in use
              at sun.nio.ch.Net.bind0(Native Method)
              at sun.nio.ch.Net.bind(Net.java:433)
              at sun.nio.ch.Net.bind(Net.java:425)
              at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
              at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:128)
              at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:500)
              at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1218)
              at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:495)
              at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:480)
              at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:965)
              at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:209)
              at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:355)
              at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
      
      

      Attachments

        Activity

          People

            devaraj Devaraj Kavali
            devaraj Devaraj Kavali
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: