Skip to main content

VMware ESXi 5.5 Purple Diagnostic Screen Exception 14 in SEsparse and LibAIO (LibAIODrainMergeQueue, LibAIOMergedIODone, SESparseAsyncDataDone) (2073516)

Symptoms
  • VMware ESXi 5.5 host fails with a purple diagnostic screen
  • You see backtrace similar to:

    cpu0:33101)@BlueScreen: #PF Exception 14 in world 33101:memMap-0 IP 0x4180182f4948 addr 0x4108fffffff0 PTEs:0x100088063;0x80000020ad5bf063;0x0;
    cpu0:33101)Code start: 0x418018000000 VMK uptime: 1:09:27:02.593
    cpu0:33101)0x4123c535cb20:[0x4180182f4948]LibAIODrainMergeQueue@vmkernel#nover+0x150 stack: 0x4130002a85c0
    cpu0:33101)0x4123c535cb80:[0x4180182f53fd]LibAIOMergedIODone@vmkernel#nover+0x211 stack: 0x412ec622ef90
    cpu0:33101)0x4123c535cbb0:[0x41801802d21f]AsyncPopCallbackFrameInt@vmkernel#nover+0xe7 stack: 0x1
    cpu0:33101)0x4123c535cbe0:[0x418018bb9798]SESparseAsyncDataDone@esx#nover+0x15c stack: 0x41300007b0c0
    cpu0:33101)0x4123c535cc10:[0x41801802d21f]AsyncPopCallbackFrameInt@vmkernel#nover+0xe7 stack: 0x4123c535cc70
    cpu0:33101)0x4123c535cc40:[0x4180182cdc22]FS_IOAccessDone@vmkernel#nover+0x56 stack: 0x412ec043e340
    cpu0:33101)0x4123c535cc70:[0x41801802d21f]AsyncPopCallbackFrameInt@vmkernel#nover+0xe7 stack: 0x4123c535cc90
    cpu0:33101)0x4123c535cca0:[0x4180182fd5e1]FDSAsyncTokenIODone@vmkernel#nover+0xdd stack: 0x4123c535cdb0
    cpu0:33101)0x4123c535cda0:[0x418018320593]SCSICompleteDeviceCommand@vmkernel#nover+0x46f stack: 0x4123c535cdc0
    cpu0:33101)0x4123c535cdd0:[0x4180187dbebe]nmp_CompleteCommandForDevice@com.vmware.vmkapi#v2_2_0_0+0x82 stack:
    cpu0:33101)0x4123c535cee0:[0x4180187dc18e]nmp_CompleteCommandForPath@com.vmware.vmkapi#v2_2_0_0+0x12a stack: 0
    cpu0:33101)0x4123c535cf60:[0x41801834153a]SCSICompletePathCommand@vmkernel#nover+0x292 stack: 0xe958503533d2
    cpu0:33101)0x4123c535cfd0:[0x41801832eaf3]SCSICompleteAdapterCommand@vmkernel#nover+0x13f stack: 0x4180184a69f
    cpu0:33101)0x4123c535d0f0:[0x4180186d9c13]SCSILinuxWorldletFn@com.vmware.driverAPI#9.2+0x3e3 stack: 0x4123c535
    cpu0:33101)0x4123c535d250:[0x4180180eb521]WorldletProcessQueue@vmkernel#nover+0x779 stack: 0x417f00000000
    cpu0:33101)0x4123c535d290:[0x4180180eb93c]WorldletBHHandler@vmkernel#nover+0x54 stack: 0x4123c535d2d0
    cpu0:33101)0x4123c535d320:[0x41801802e94f]BH_DrainAndDisableInterrupts@vmkernel#nover+0xf3 stack: 0x4123c535d4
    cpu0:33101)0x4123c535d360:[0x418018063e83]IDT_IntrHandler@vmkernel#nover+0x1af stack: 0x4123c535d480
    cpu0:33101)0x4123c535d370:[0x4180180f1064]gate_entry@vmkernel#nover+0x64 stack: 0x4018
    cpu0:33101)0x4123c535d480:[0x4180183a329a]Power_HaltPCPU@vmkernel#nover+0x1fe stack: 0x0
    cpu0:33101)0x4123c535d4f0:[0x41801824e859]CpuSchedIdleLoopInt@vmkernel#nover+0x4bd stack: 0x4123c535d580
    cpu0:33101)0x4123c535d650:[0x418018254930]CpuSchedDispatch@vmkernel#nover+0x1630 stack: 0x0
    cpu0:33101)0x4123c535d6c0:[0x418018255c65]CpuSchedWait@vmkernel#nover+0x245 stack: 0x1
    cpu0:33101)0x4123c535d740:[0x4180182565c4]CpuSched_TimedWait@vmkernel#nover+0xec stack: 0x0
    cpu0:33101)base fs=0x0 gs=0x418040000000 Kgs=0x0
  • You observe this issue on a View environment with ESXi 5.5 using SESparse or Reclaim disk space.
Purpose
This article provides information on troubleshooting hosts that fail with a purple diagnostic screen referencing Linux native Asynchronous I/O Interface library LibAIO in the stack trace.
Cause
This issue occurs due to a new feature introduced in the AIO Library to improve SESparse performance and may occur when using SESparse snapshots.
Resolution
This is a known issue affecting VMware ESXi 5.5.
Currently, there is no resolution.

Work around

To work around this issue, you must disable LibAIO coalesce Read and Write.

To disable LibAIO coalesce Read and Write:
  1. Run this command on the affected ESXi host to verify current settings for coalesce Read and Write:

    esxcfg-advcfg -g /FDS/FDSEnableCoalesceWrites
    esxcfg-advcfg -g /FDS/FDSEnableCoalesceReads

    Note: Value 1 indicates Enabled and value 0 indicates Disabled. 
  2. If you see the value 1, run this command on the ESXi host to disable the setting:

    esxcfg-advcfg -s 0 /FDS/FDSEnableCoalesceWrites
    esxcfg-advcfg -s 0 /FDS/FDSEnableCoalesceReads

    Note: Changes to these advanced parameters can be changed in a live system and they are persistent.


Comments

Post a Comment

Popular posts from this blog

Visual Studio 2012 / 2013 Update 1 2 3 4 Offline Installer

Visual Studio 2012 Update 2 was released about a week ago.  This update includes lots of fixes and some features – you can see the list   here .  The only problem with the update is that Microsoft does not offer an offline installer.  If you are installing this on your own PC or for one person, you may not have a need for one.  But…  if your entire team needs to install this (or you just want to have it for later for a PC rebuild, you can download all 1.8 GB and have an offline installer for you or your team to share.  Here’s how: Get the update from Microsoft  here . (updated with Update 4 link) Save the file to a folder. open the folder Pro tip – Shift + right-click the background of the folder and choose ‘Open command window here’ in the command window type  VS2013.4.exe /Layout (or VS2013.1.exe /Layout or VS2013.2.exe /Layout depending on your update) It will then ask you w...
Post a Comment
Read more

Deployment of the Microsoft Windows Malicious Software Removal Tool in an enterprise environment

INTRODUCTION Microsoft has released the Microsoft Windows Malicious Software Removal Tool to help you remove specific, prevalent malicious software from a computer.  Skip the details and download the tool For more information about how to download the tool, go to the following Microsoft webpage: http://www.microsoft.com/security/pc-security/malware-removal.aspx The information that is contained in this article is specific to the enterprise deployment of the tool. We highly recommend that you review the following Microsoft Knowledge Base article. It contains general information about the tool and about the download locations.  The tool is primarily intended for noncorporate users who do not have an existing, up-to-date antivirus product installed on their computers. However, the tool can also be deployed in an enterprise environment to enhance existing protection and as part of a defense-in-depth strategy. To deploy the tool in an enterprise environmen...
1 comment
Read more

Enabling VNC Server on Solaris 10

By default, In Solaris 10, VNC service is almost configured. Below is the process to enable it. Check VNC service root#  svcs -a | grep -i vnc disabled 13:47:12 svc:/application/x11/xvnc-inetd:default We need to enable the VNC service root#  svcadm enable svc:/application/x11/xvnc-inetd:default By default, it broken we need to      so some changes. root#  svcs svc:/application/x11/xvnc-inetd:default STATE STIME FMRI maintenance 14:22:41 svc:/application/x11/xvnc-inetd:default Need to append vnc to the      /etc/services root#  echo “vnc-server\t5900/tcp\t\t\t# Xvnc” >>/etc/services Cross verify /etc/services root#  tail /etc/services snmpd 161/udp snmp # SMA snmp daemon vnc-server 5900/tcp # Xvnc Need to customizethe gnu display      manager root#  ls -al /etc/X11/gdm/custom.conf /etc/X11/gdm/custom.conf: No such file o...
Post a Comment
Read more