I just discovered jenkinsci/docker#805, which sounds similar to my scenario on the surface. There, Pedro Rodrigues found that the JNLP port number is first set to a random value and then a Groovy initialization script in the Docker image changes to it the desired value. This opens up a race where Jenkins is responding to the wrong JNLP port for a short amount of time during initialization. Pedro fixed this in the Docker image by using a Java system property rather than a Groovy initialization script to set the port. The Java system property is used when Jenkins first initializes JNLP, so this theoretically closes the race.
I wonder if this phenomenon could explain my observations above, where Jenkins was replying to HTTP request but the JNLP port was not available. It seems like the following sequence of events is possible:
Jenkins advertises the old JNLP port
Swarm/Remoting picks this up
Jenkins executes the Groovy initialization script to change the port
Swarm/Remoting tries to connect to the old port
I'll be deploying Pedro's changes to the Docker image next week. I don't know for sure they will fix this problem, but I am hopeful that they might.