home | list info | list archive | date index | thread index

[OCLUG-Tech] ssh, X11 forwarding not working (CentOS 6.3)

Hi everyone.  For years, I've been using ssh -X just fine to connect to
our various remote machines and interact with X applications on the
remote machines via the display in front of me, but recently I installed
a fresh CentOS 6.3 x64 on real hardware and it isn't working.  The
DISPLAY environment variable is never populated on the connected session
and even manually setting it fails.

"hope" is my local machine, "dy" is an established CentOS 5.x machine
that works fine and "dz" is the problematic one.

dz was installed with "X Windows System" and indeed on the console, X
works just fine and I have a Gnome desktop too.

    [larocque@hope ~]$ echo $DISPLAY
    :0
    [larocque@hope ~]$
    [larocque@hope ~]$ ssh -Xvv root [ at ] dz [ dot ] cms [ dot ] math [ dot ] ca
    OpenSSH_5.6p1, OpenSSL 1.0.0j-fips 10 May 2012
    debug1: Reading configuration data /home/larocque/.ssh/config
    debug1: Reading configuration data /etc/ssh/ssh_config
    debug1: Applying options for *
    debug2: ssh_connect: needpriv 0
    debug1: Connecting to dz.cms.math.ca [10.5.7.201] port 22.
    debug1: Connection established.
    debug1: identity file /home/larocque/.ssh/id_rsa type -1
    debug1: identity file /home/larocque/.ssh/id_rsa-cert type -1
    debug1: identity file /home/larocque/.ssh/id_dsa type -1
    debug1: identity file /home/larocque/.ssh/id_dsa-cert type -1
    debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
    debug1: match: OpenSSH_5.3 pat OpenSSH*
    debug1: Enabling compatibility mode for protocol 2.0
    debug1: Local version string SSH-2.0-OpenSSH_5.6
    debug2: fd 3 setting O_NONBLOCK
    debug1: SSH2_MSG_KEXINIT sent
    debug1: SSH2_MSG_KEXINIT received
    debug2: kex_parse_kexinit:
    diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
    debug2: kex_parse_kexinit:
    ssh-rsa-cert-v01 [ at ] openssh [ dot ] com,ssh-dss-cert-v01 [ at ] openssh [ dot ] com,ssh-rsa-cert-v00 [ at ] openssh [ dot ] com,ssh-dss-cert-v00 [ at ] openssh [ dot ] com,ssh-rsa,ssh-dss
    debug2: kex_parse_kexinit:
    aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc [ at ] lysator [ dot ] liu [ dot ] se
    debug2: kex_parse_kexinit:
    aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc [ at ] lysator [ dot ] liu [ dot ] se
    debug2: kex_parse_kexinit:
    hmac-md5,hmac-sha1,umac-64 [ at ] openssh [ dot ] com,hmac-ripemd160,hmac-ripemd160 [ at ] openssh [ dot ] com,hmac-sha1-96,hmac-md5-96
    debug2: kex_parse_kexinit:
    hmac-md5,hmac-sha1,umac-64 [ at ] openssh [ dot ] com,hmac-ripemd160,hmac-ripemd160 [ at ] openssh [ dot ] com,hmac-sha1-96,hmac-md5-96
    debug2: kex_parse_kexinit: none,zlib [ at ] openssh [ dot ] com,zlib
    debug2: kex_parse_kexinit: none,zlib [ at ] openssh [ dot ] com,zlib
    debug2: kex_parse_kexinit:
    debug2: kex_parse_kexinit:
    debug2: kex_parse_kexinit: first_kex_follows 0
    debug2: kex_parse_kexinit: reserved 0
    debug2: kex_parse_kexinit:
    diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
    debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
    debug2: kex_parse_kexinit:
    aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc [ at ] lysator [ dot ] liu [ dot ] se
    debug2: kex_parse_kexinit:
    aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc [ at ] lysator [ dot ] liu [ dot ] se
    debug2: kex_parse_kexinit:
    hmac-md5,hmac-sha1,umac-64 [ at ] openssh [ dot ] com,hmac-ripemd160,hmac-ripemd160 [ at ] openssh [ dot ] com,hmac-sha1-96,hmac-md5-96
    debug2: kex_parse_kexinit:
    hmac-md5,hmac-sha1,umac-64 [ at ] openssh [ dot ] com,hmac-ripemd160,hmac-ripemd160 [ at ] openssh [ dot ] com,hmac-sha1-96,hmac-md5-96
    debug2: kex_parse_kexinit: none,zlib [ at ] openssh [ dot ] com
    debug2: kex_parse_kexinit: none,zlib [ at ] openssh [ dot ] com
    debug2: kex_parse_kexinit:
    debug2: kex_parse_kexinit:
    debug2: kex_parse_kexinit: first_kex_follows 0
    debug2: kex_parse_kexinit: reserved 0
    debug2: mac_setup: found hmac-md5
    debug1: kex: server->client aes128-ctr hmac-md5 none
    debug2: mac_setup: found hmac-md5
    debug1: kex: client->server aes128-ctr hmac-md5 none
    debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
    debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
    debug2: dh_gen_key: priv key bits set: 129/256
    debug2: bits set: 513/1024
    debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
    debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
    debug1: Host 'dz.cms.math.ca' is known and matches the RSA host key.
    debug1: Found key in /home/larocque/.ssh/known_hosts:153
    debug2: bits set: 533/1024
    debug1: ssh_rsa_verify: signature correct
    debug2: kex_derive_keys
    debug2: set_newkeys: mode 1
    debug1: SSH2_MSG_NEWKEYS sent
    debug1: expecting SSH2_MSG_NEWKEYS
    debug2: set_newkeys: mode 0
    debug1: SSH2_MSG_NEWKEYS received
    debug1: Roaming not allowed by server
    debug1: SSH2_MSG_SERVICE_REQUEST sent
    debug2: service_accept: ssh-userauth
    debug1: SSH2_MSG_SERVICE_ACCEPT received
    debug2: key: general ssh key for larocque at CMS
    (larocque [ at ] cms [ dot ] math [ dot ] ca) (0x7f599d07efb0)
    debug2: key: /home/larocque/.ssh/id_rsa ((nil))
    debug2: key: /home/larocque/.ssh/id_dsa ((nil))
    debug1: Authentications that can continue:
    publickey,gssapi-keyex,gssapi-with-mic,password
    debug1: Next authentication method: gssapi-keyex
    debug1: No valid Key exchange context
    debug2: we did not send a packet, disable method
    debug1: Next authentication method: gssapi-with-mic
    debug1: Unspecified GSS failure.  Minor code may provide more
    information
    Credentials cache file '/tmp/krb5cc_500' not found

    debug1: Unspecified GSS failure.  Minor code may provide more
    information
    Credentials cache file '/tmp/krb5cc_500' not found

    debug1: Unspecified GSS failure.  Minor code may provide more
    information


    debug1: Unspecified GSS failure.  Minor code may provide more
    information


    debug2: we did not send a packet, disable method
    debug1: Next authentication method: publickey
    debug1: Offering RSA public key: general ssh key for larocque at CMS
    (larocque [ at ] cms [ dot ] math [ dot ] ca)
    debug2: we sent a publickey packet, wait for reply
    debug1: Server accepts key: pkalg ssh-rsa blen 646
    debug2: input_userauth_pk_ok: SHA1 fp
    9c:0c:da:c8:f0:4f:64:cd:59:27:d3:3f:a4:72:c2:fd:cc:63:9b:83
    debug1: Authentication succeeded (publickey).
    Authenticated to dz.cms.math.ca ([10.5.7.201]:22).
    debug1: channel 0: new [client-session]
    debug2: channel 0: send open
    debug1: Requesting no-more-sessions [ at ] openssh [ dot ] com
    debug1: Entering interactive session.
    debug2: callback start
    debug2: x11_get_proto: /usr/bin/xauth  list :0 2>/dev/null
    debug1: Requesting X11 forwarding with authentication spoofing.
    debug2: channel 0: request x11-req confirm 0
    debug2: client_session2_setup: id 0
    debug2: channel 0: request pty-req confirm 1
    debug1: Sending environment.
    debug1: Sending env XMODIFIERS = @im=none
    debug2: channel 0: request env confirm 0
    debug1: Sending env LANG = en_CA.utf8
    debug2: channel 0: request env confirm 0
    debug2: channel 0: request shell confirm 1
    debug2: fd 3 setting TCP_NODELAY
    debug2: callback done
    debug2: channel 0: open confirm rwindow 0 rmax 32768
    debug2: channel_input_status_confirm: type 99 id 0
    debug2: PTY allocation request accepted on channel 0
    debug2: channel 0: rcvd adjust 2097152
    debug2: channel_input_status_confirm: type 99 id 0
    debug2: shell request accepted on channel 0
    Last login: Tue Oct  2 11:40:38 2012 from hope.ott.cms
    [root@dz ~]#
    [root@dz ~]#
    [root@dz ~]#
    [root@dz ~]#
    [root@dz ~]# echo $DISPLAY

    [root@dz ~]# xclock
    Error: Can't open display:
    [root@dz ~]#
    [root@dz ~]#
    [root@dz ~]#
    [root@dz ~]#
    [root@dz ~]# grep X /etc/ssh/*_config
    /etc/ssh/ssh_config:#   ForwardX11 no
    /etc/ssh/ssh_config:# If this option is set to yes then remote X11
    clients will have full access
    /etc/ssh/ssh_config:# to the original X11 display. As virtually no
    X11 client supports the untrusted
    /etc/ssh/ssh_config:    ForwardX11Trusted yes
    /etc/ssh/ssh_config:    SendEnv XMODIFIERS
    /etc/ssh/sshd_config:AcceptEnv XMODIFIERS
    /etc/ssh/sshd_config:#X11Forwarding no
    /etc/ssh/sshd_config:X11Forwarding yes
    /etc/ssh/sshd_config:#X11DisplayOffset 10
    /etc/ssh/sshd_config:#X11UseLocalhost yes
    /etc/ssh/sshd_config:#    X11Forwarding no
    [root@dz ~]#
    [root@dz ~]#
    [root@dz ~]#
    [root@dz ~]# rpm -qa|grep -i x11|sort
    ConsoleKit-x11-0.4.1-3.el6.x86_64
    dbus-x11-1.2.24-7.el6_3.x86_64
    libX11-1.3-2.el6.x86_64
    libX11-common-1.3-2.el6.noarch
    pulseaudio-module-x11-0.9.21-14.el6_3.x86_64
    qt-x11-4.6.2-24.el6.x86_64
    xorg-x11-apps-7.4-10.el6.x86_64
    xorg-x11-drivers-7.3-13.3.el6.x86_64
    xorg-x11-drv-acecad-1.5.0-3.el6.x86_64
    xorg-x11-drv-aiptek-1.4.1-2.el6.x86_64
    xorg-x11-drv-apm-1.2.3-2.el6.x86_64
    xorg-x11-drv-ast-0.91.10-1.el6.x86_64
    xorg-x11-drv-ati-6.14.2-9.el6.x86_64
    xorg-x11-drv-ati-firmware-6.14.2-9.el6.noarch
    xorg-x11-drv-cirrus-1.3.2-2.el6.x86_64
    xorg-x11-drv-dummy-0.3.4-1.el6.x86_64
    xorg-x11-drv-elographics-1.3.0-2.el6.x86_64
    xorg-x11-drv-evdev-2.6.0-2.el6.x86_64
    xorg-x11-drv-fbdev-0.4.2-2.el6.x86_64
    xorg-x11-drv-fpit-1.4.0-2.el6.x86_64
    xorg-x11-drv-glint-1.2.5-1.el6.x86_64
    xorg-x11-drv-hyperpen-1.4.1-2.el6.x86_64
    xorg-x11-drv-i128-1.3.4-1.el6.x86_64
    xorg-x11-drv-i740-1.3.2-2.el6.x86_64
    xorg-x11-drv-intel-2.16.0-4.el6.x86_64
    xorg-x11-drv-keyboard-1.6.0-1.el6.x86_64
    xorg-x11-drv-mach64-6.9.0-1.el6.x86_64
    xorg-x11-drv-mga-1.4.13-7.el6.x86_64
    xorg-x11-drv-mouse-1.7.0-4.el6.x86_64
    xorg-x11-drv-mutouch-1.3.0-2.el6.x86_64
    xorg-x11-drv-nouveau-0.0.16-13.20110719gitde9d1ba.el6.x86_64
    xorg-x11-drv-nv-2.1.18-2.el6.x86_64
    xorg-x11-drv-openchrome-0.2.904-4.el6.x86_64
    xorg-x11-drv-penmount-1.5.0-2.el6.x86_64
    xorg-x11-drv-qxl-0.0.14-13.el6_2.x86_64
    xorg-x11-drv-r128-6.8.1-3.el6.x86_64
    xorg-x11-drv-rendition-4.2.4-1.el6.x86_64
    xorg-x11-drv-s3virge-1.10.4-2.el6.x86_64
    xorg-x11-drv-savage-2.3.2-1.el6.x86_64
    xorg-x11-drv-siliconmotion-1.7.5-1.el6.x86_64
    xorg-x11-drv-sis-0.10.3-1.el6.x86_64
    xorg-x11-drv-sisusb-0.9.4-1.el6.x86_64
    xorg-x11-drv-synaptics-1.4.1-3.el6.x86_64
    xorg-x11-drv-tdfx-1.4.3-2.el6.x86_64
    xorg-x11-drv-trident-1.3.4-1.el6.x86_64
    xorg-x11-drv-v4l-0.2.0-4.el6.x86_64
    xorg-x11-drv-vesa-2.3.0-2.el6.x86_64
    xorg-x11-drv-vmmouse-12.7.0-1.el6.x86_64
    xorg-x11-drv-vmware-11.0.3-1.el6.x86_64
    xorg-x11-drv-void-1.4.0-1.el6.x86_64
    xorg-x11-drv-voodoo-1.2.4-1.el6.x86_64
    xorg-x11-drv-wacom-0.13.0-6.el6.x86_64
    xorg-x11-drv-xgi-1.6.0-11.el6.x86_64
    xorg-x11-font-utils-7.2-11.el6.x86_64
    xorg-x11-server-common-1.10.6-1.el6.centos.x86_64
    xorg-x11-server-utils-7.5-5.2.el6.x86_64
    xorg-x11-server-Xorg-1.10.6-1.el6.centos.x86_64
    xorg-x11-utils-7.4-8.el6.x86_64
    xorg-x11-xauth-1.0.2-7.1.el6.x86_64
    xorg-x11-xinit-1.0.9-13.el6.x86_64
    xorg-x11-xkb-utils-7.4-6.el6.x86_64
    [root@dz ~]#


Oddly, as you can see above, SSH seems to properly negotiate the X11
Forwarding, yet I still have no DISPLAY.  I compared the ssh -vv output
between hope and dy to the one shown above between hope and dz and they
match very closely, but only the dy destination actually gives me the
DISPLAY setting and properly forwards X11.

I tried "ssh -Y" rather than "ssh -X" but got the same symptom.

I also tried clearing IP tables entirely.


What should I try next, oh wizards?

-Steve