WebSocket Agents and Archiving Artifacts Performance

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

WebSocket Agents and Archiving Artifacts Performance

Tim Black
Is there any reason to believe that using the new -webSocket mode for agents would be any less sluggish at archiving artifacts from agent to master than ssh mode? 

Using the ssh-slaves-plugin I'm getting abysmal throughput (~13Mbps) when artifacts are being copied from agent to master, despite their 10GBps link: https://issues.jenkins-ci.org/browse/JENKINS-7921

Reading up on some of the conversations on this long-standing issue over the last decade, I'm not confident that this performance is going to be improved. So, I'm considering alternatives, like using WebSocket agents. I read here that the problem might be that "Jenkins archives via its control channel (e.g. ssh slave - using java SSH implementation JSCH). The java ssh just can't get anywhere near 1Gb/s network speed that native SSH can manage easily"

So, I was just wondering if WebSocket Agents might perform better at archiving artifacts bc they are implemented so differently.

Thanks,
Tim Black  

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/d41f3894-ed12-4594-ba08-2dee57fd4181n%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: WebSocket Agents and Archiving Artifacts Performance

jn...@cloudbees.com
Hi Tim,

it is reasonable to assume that the performance may change.

Basically, a lot of this has to do with the chattyness of the channel, the latency, and buffer sizes setup etc etc..  
It may well be that the websocket channels are better sized by default in the regard (it could also be the inverse).  It also may depend heavily on the underlying infrastructure.

I would suggest you perform a quick test as the only way to be sure (and report back for others!)

but if you are using large artifacts perhaps you want to look at an external artifact manager instead such as s3 / azure etc   (e.g. https://www.jenkins.io/blog/2019/07/25/azure-artifact-manager/) ?

/James

On Wednesday, August 26, 2020 at 9:42:03 PM UTC+1 [hidden email] wrote:
Is there any reason to believe that using the new -webSocket mode for agents would be any less sluggish at archiving artifacts from agent to master than ssh mode? 

Using the ssh-slaves-plugin I'm getting abysmal throughput (~13Mbps) when artifacts are being copied from agent to master, despite their 10GBps link: https://issues.jenkins-ci.org/browse/JENKINS-7921

Reading up on some of the conversations on this long-standing issue over the last decade, I'm not confident that this performance is going to be improved. So, I'm considering alternatives, like using WebSocket agents. I read here that the problem might be that "Jenkins archives via its control channel (e.g. ssh slave - using java SSH implementation JSCH). The java ssh just can't get anywhere near 1Gb/s network speed that native SSH can manage easily"

So, I was just wondering if WebSocket Agents might perform better at archiving artifacts bc they are implemented so differently.

Thanks,
Tim Black  

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/3036fbd2-0169-4a7c-b2d2-2ae2cc84b9e4n%40googlegroups.com.