Ok, time for some fun. I've done some digging in Jenkins and found several places where the version number is exposed in situations with no permission checks.
HudsonAuthenticationEntryPoint exposes the version in a header. This class is used for handling permission errors to redirect the user to login in the first place, so clearly there are no permission checks involved.
TcpSlaveAgentListener exposes that header when you send a "GET /" request to it.
UDPBroadcastThread exposes the version number as an XML tag in the metadata included in the multicast packet sent for autodiscovery. I'm not sure how important the version number is to include here, though it might be required.
VersionCommand only requires Overall/Read to execute.
Api instances include setting the version header, though typically, these objects are already guarded by other permission checks. However, if even a single API endpoint is unsecured, this would leak the version number.
AbstractBuild tracks the version of Jenkins that build was executed in.
DownloadService adds a script to the footer of any page that includes the version number in the body of an Ajax request as long as the user has Overall/Read permissions.
IncompatibleVMDetected shows information about the running JVM as well, but no Jenkins version other than the footer that's not specific to this error page.
l:layout and l:html both include the version number as a header, and l:layout includes the version number in the footer itself.
Jenkins/login.jelly exposes the version in headers.