Freshly updated OSX: 10.9.2
CUDA Driver Version: 5.5.28 (one before the latest)
GPU Driver Version: 8.24.9 310.40.25f01
While doing heavy CUDA ray tracering get GUI freeze followed by system crash.
screen fills:
cuMemFree failed: launch timeout
PyCUDA WARNING: a clean-up operation failed (dead context maybe?)
cuMemFree failed: launch timeout
PyCUDA WARNING: a clean-up operation failed (dead context maybe?)
... message suggesting use of Context.pop() to avoid issue
sometimes cursor reponds to input, sometime not, but nevertheless unable to do anything essentially a GUI freeze
fans spin up, then down, then up
SSH in from another machinem, using top observe process with name beginning DumpGPU:
delta:~ blyth$ mdfind DumpGPU
/System/Library/Sandbox/Profiles/com.apple.DumpGPURestart.sb
/System/Library/LaunchDaemons/com.apple.DumpGPURestart.plist
/System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/DumpGPURestart
delta:~ blyth$
delta:~ blyth$
sometimes get after 5 minutes a blank white screen and then the grey multi lingual death message:
You computer restarted because of a problem
Normally dont get to see the start of the error, but somehow (potentially different conditions):
INFO:chroma:MOUSEBUTTONDOWN 4 [ 330.98363295 330.98363295 330.98363295]
INFO:chroma:MOUSEBUTTONUP 4
INFO:chroma:MOUSEBUTTONDOWN 4 [ 330.98363295 330.98363295 330.98363295]
Traceback (most recent call last):
File "./simplecamera.py", line 285, in <module>
camera._run()
File "./simplecamera.py", line 271, in _run
self.process_event(event)
File "./simplecamera.py", line 136, in process_event
self.translate(v)
File "./simplecamera.py", line 119, in translate
self.update()
File "./simplecamera.py", line 125, in update
pixels = self.pixels_gpu.get()
File "/usr/local/env/chroma_env/lib/python2.7/site-packages/pycuda/gpuarray.py", line 252, in get
drv.memcpy_dtoh(ary, self.gpudata)
pycuda._driver.LaunchError: cuMemcpyDtoH failed: launch timeout
system.log:
Mar 10 19:27:14 delta.local python[2861]: CPSSetForegroundOperationState(): This call is deprecated and should not be called anymore.
Mar 10 19:27:56 delta.local python[2886]: CPSGetCurrentProcess(): This call is deprecated and should not be called anymore.
Mar 10 19:27:56 delta.local python[2886]: CPSSetForegroundOperationState(): This call is deprecated and should not be called anymore.
Mar 10 19:37:41 delta.local ReportCrash[2969]: Saved crash report for python[2886] version 0 to /Users/blyth/Library/Logs/DiagnosticReports/python_2014-03-10-193741_delta.crash
Mar 10 19:37:58 delta kernel[0]: process WindowServer[108] caught causing excessive wakeups. Observed wakeups rate (per sec): 570; Maximum permitted wakeups rate (per sec): 150; Observation period: 300 seconds; Task lifetime number of wakeups: 381526
Mar 10 19:37:58 delta.local hidd[84]: IOHIDEventQueue unable to get policy for event of type 11. (e00002e8)
Mar 10 19:37:58 --- last message repeated 87 times ---
Mar 10 19:37:58 delta.local ReportCrash[2972]: Invoking spindump for pid=108 wakeups_rate=570 duration=79 because of excessive wakeups
Mar 10 19:38:06 delta.local sshd[2975]: Accepted publickey for blyth from fe80::214:51ff:fe82:2425%en0 port 58074 ssh2
Mar 10 19:38:06 delta.local sshd: blyth [priv][2975]: USER_PROCESS: 2977 ttys003
Mar 10 19:38:55 delta kernel[0]: NVDA(OpenGL): Channel timeout!
Mar 10 19:39:16 delta kernel[0]: process Terminal[163] thread 1624 caught burning CPU! It used more than 50% CPU (Actual recent usage: 93%) over 180 seconds. thread lifetime cpu usage 91.563823 seconds, (52.291935 user, 39.271888 system) ledger info: balance: 90007955052 credit: 91555275475 debit: 1547320423 limit: 90000000000 (50%) period: 180000000000 time since last refill (ns): 95903014198
Mar 10 19:39:16 delta.local ReportCrash[3242]: Invoking spindump for pid=163 thread=1624 percent_cpu=93 duration=97 because of excessive cpu utilization
Mar 10 19:39:23 delta kernel[0]: NVDA(OpenGL): Channel timeout!
Mar 10 19:40:20 --- last message repeated 1 time ---
Mar 10 19:40:20 delta kernel[0]: NVDA(OpenGL): Channel timeout!
Mar 10 19:41:17 --- last message repeated 1 time ---
Mar 10 19:41:17 delta kernel[0]: NVDA(OpenGL): Channel timeout!
Mar 10 19:42:14 --- last message repeated 1 time ---
Mar 10 19:42:14 delta kernel[0]: NVDA(OpenGL): Channel timeout!
Mar 10 19:43:10 --- last message repeated 1 time ---
Mar 10 19:43:10 delta kernel[0]: NVDA(OpenGL): Channel timeout!
kernel panic log:
Anonymous UUID: 32BCAB7F-2AEA-A951-3785-013ECFB913EA
Mon Mar 10 19:47:26 2014
panic(cpu 0 caller 0xffffff7f9ba70fb0): "GPU Panic: [<None>] 5 3 5f 1d 0 8 0 3 : NVRM[0/1:0:0]: Read Error 0x00000144: CFG 0x0fe910de 0x00100406 0xc0000000, BAR0 0x101000000 0xffffff8200014000 0x0e7290a2, D0, P0/4\n"@/SourceCache/AppleGraphicsControl/AppleGraphicsControl-3.4.35/src/AppleMuxControl/kext/GPUPanic.cpp:127
Backtrace (CPU 0), Frame : Return Address
0xffffff81db8fabe0 : 0xffffff8019622fa9
0xffffff81db8fac60 : 0xffffff7f9ba70fb0
0xffffff81db8fad30 : 0xffffff7f99ffb22c
0xffffff81db8fadf0 : 0xffffff7f9a0c5106
0xffffff81db8fae30 : 0xffffff7f9a2e8e54
0xffffff81db8fae50 : 0xffffff7f9a001b92
0xffffff81db8faef0 : 0xffffff7f99fb1330
0xffffff81db8faf10 : 0xffffff8019ab033b
0xffffff81db8faf40 : 0xffffff7f99cd3a27
0xffffff81db8faf50 : 0xffffff7f9bb079af
0xffffff81db8faf60 : 0xffffff7f9bb0f7d8
0xffffff81db8faf80 : 0xffffff80196db3fb
0xffffff81db8fafd0 : 0xffffff80196f34e9
0xffffff81ed5abd30 : 0xffffff7f9a301d0b
0xffffff81ed5abda0 : 0xffffff7f9a13bdb1
0xffffff81ed5abe20 : 0xffffff7f99fe3aea
0xffffff81ed5abee0 : 0xffffff7f99fb1635
0xffffff81ed5abf20 : 0xffffff801964a23a
0xffffff81ed5abfb0 : 0xffffff80196d6ff7
Kernel Extensions in backtrace:
com.apple.iokit.IOPCIFamily(2.9)[EDA75271-4E9D-34E7-A2C5-14F0C8817D37]@0xffffff7f99cbb000->0xffffff7f99ce5fff
com.apple.driver.AppleACPIPlatform(2.0)[5D6DA288-1289-3FD4-BFA5-09C109930AE7]@0xffffff7f9bb01000->0xffffff7f9bb59fff
dependency: com.apple.iokit.IOACPIFamily(1.4)[045D5D6F-AD1E-36DB-A249-A346E2B48E54]@0xffffff7f9a8de000
dependency: com.apple.iokit.IOPCIFamily(2.9)[EDA75271-4E9D-34E7-A2C5-14F0C8817D37]@0xffffff7f99cbb000
com.apple.driver.AppleMuxControl(3.4.35)[1BFF66C1-65E4-3BB3-9DEE-B61C3137019B]@0xffffff7f9ba63000->0xffffff7f9ba75fff
dependency: com.apple.driver.AppleGraphicsControl(3.4.35)[09897896-ACBD-36B5-B1D4-0CCC4000E3B3]@0xffffff7f9ba5b000
dependency: com.apple.iokit.IOACPIFamily(1.4)[045D5D6F-AD1E-36DB-A249-A346E2B48E54]@0xffffff7f9a8de000
dependency: com.apple.iokit.IOPCIFamily(2.9)[EDA75271-4E9D-34E7-A2C5-14F0C8817D37]@0xffffff7f99cbb000
dependency: com.apple.iokit.IOGraphicsFamily(2.4.1)[4421462D-2B1F-3540-8EEA-9DFCB0565E39]@0xffffff7f99f58000
dependency: com.apple.driver.AppleBacklightExpert(1.0.4)[E04639C5-D734-3AB3-A682-FE66694C6653]@0xffffff7f9ba5e000
com.apple.nvidia.driver.NVDAResman(8.2.4)[3D591202-DD24-3441-925A-F6808ABDF185]@0xffffff7f99fab000->0xffffff7f9a20ffff
dependency: com.apple.iokit.IOPCIFamily(2.9)[EDA75271-4E9D-34E7-A2C5-14F0C8817D37]@0xffffff7f99cbb000
dependency: com.apple.iokit.IONDRVSupport(2.4.1)[999E29DA-D513-3544-89D1-9885B728A098]@0xffffff7f99f9b000
dependency: com.apple.iokit.IOGraphicsFamily(2.4.1)[4421462D-2B1F-3540-8EEA-9DFCB0565E39]@0xffffff7f99f58000
com.apple.nvidia.driver.NVDAGK100Hal(8.2.4)[ACFCEA0C-4C80-36C0-8636-D10EE7D2DE17]@0xffffff7f9a21a000->0xffffff7f9a3c6fff
dependency: com.apple.nvidia.driver.NVDAResman(8.2.4)[3D591202-DD24-3441-925A-F6808ABDF185]@0xffffff7f99fab000
dependency: com.apple.iokit.IOPCIFamily(2.9)[EDA75271-4E9D-34E7-A2C5-14F0C8817D37]@0xffffff7f99cbb000
BSD process name corresponding to current thread: kernel_task
Mac OS version:
13C64
Kernel version:
Darwin Kernel Version 13.1.0: Thu Jan 16 19:40:37 PST 2014; root:xnu-2422.90.20~2/RELEASE_X86_64
Kernel UUID: 9FEA8EDC-B629-3ED2-A1A3-6521A1885953
Kernel slide: 0x0000000019400000
Kernel text base: 0xffffff8019600000
System model name: MacBookPro11,3 (Mac-2BD1B31983FE1663)
System uptime in nanoseconds: 6796404552470
last loaded kext at 243252292556: com.apple.filesystems.msdosfs 1.9 (addr 0xffffff7f9bb8e000, size 65536)
last unloaded kext at 303464511146: com.apple.filesystems.msdosfs 1.9 (addr 0xffffff7f9bb8e000, size 57344)
loaded kexts:
com.nvidia.CUDA 1.1.0
com.apple.driver.AppleHWSensor 1.9.5d0
com.apple.filesystems.autofs 3.0
com.apple.driver.AudioAUUC 1.60
com.apple.iokit.IOBluetoothSerialManager 4.2.3f10
com.apple.driver.ApplePlatformEnabler 2.0.9d1
com.apple.driver.AGPM 100.14.15
com.apple.driver.X86PlatformShim 1.0.0
com.apple.iokit.IOUserEthernet 1.0.0d1
com.apple.Dont_Steal_Mac_OS_X 7.0.0
com.apple.driver.AppleHWAccess 1
com.apple.driver.AppleHDA 2.6.0f1
com.apple.driver.AppleUpstreamUserClient 3.5.13
com.apple.GeForce 8.2.4
com.apple.iokit.BroadcomBluetoothHostControllerUSBTransport 4.2.3f10
com.apple.driver.AppleIntelHD5000Graphics 8.2.4
com.apple.driver.AppleSMCLMU 2.0.4d1
com.apple.driver.AppleMCCSControl 1.1.12
com.apple.driver.AppleMuxControl 3.4.35
com.apple.driver.AppleLPC 1.7.0
com.apple.driver.AppleIntelFramebufferAzul 8.2.4
com.apple.driver.AppleThunderboltIP 1.1.2
com.apple.driver.AppleCameraInterface 4.26.0
com.apple.driver.AppleUSBTCButtons 240.2
com.apple.driver.AppleUSBTCKeyboard 240.2
com.apple.driver.AppleUSBCardReader 3.4.1
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0d1
com.apple.BootCache 35
com.apple.driver.AppleUSBHub 666.4.0
com.apple.driver.XsanFilter 404
com.apple.iokit.IOAHCIBlockStorage 2.5.1
com.apple.driver.AppleAHCIPort 3.0.0
com.apple.driver.AirPort.Brcm4360 831.21.63
com.apple.driver.AppleUSBXHCI 670.4.0
com.apple.driver.AppleSmartBatteryManager 161.0.0
com.apple.driver.AppleACPIButtons 2.0
com.apple.driver.AppleRTC 2.0
com.apple.driver.AppleHPET 1.8
com.apple.driver.AppleSMBIOS 2.1
com.apple.driver.AppleACPIEC 2.0
com.apple.driver.AppleAPIC 1.7
com.apple.nke.applicationfirewall 153
com.apple.security.quarantine 3
com.apple.kext.triggers 1.0
com.apple.iokit.IOSerialFamily 10.0.7
com.apple.iokit.IOBluetoothFamily 4.2.3f10
com.apple.driver.DspFuncLib 2.6.0f1
com.apple.vecLib.kext 1.0.0
com.apple.iokit.IOAudioFamily 1.9.5fc2
com.apple.kext.OSvKernDSPLib 1.14
com.apple.iokit.IOAcceleratorFamily 98.14
com.apple.nvidia.driver.NVDAGK100Hal 8.2.4
com.apple.nvidia.driver.NVDAResman 8.2.4
com.apple.iokit.IOBluetoothHostControllerUSBTransport 4.2.3f10
com.apple.iokit.IOSurface 91
com.apple.driver.X86PlatformPlugin 1.0.0
com.apple.driver.AppleSMC 3.1.8
com.apple.driver.AppleSMBusController 1.0.11d1
com.apple.driver.AppleBacklightExpert 1.0.4
com.apple.driver.AppleGraphicsControl 3.4.35
com.apple.iokit.IONDRVSupport 2.4.1
com.apple.driver.IOPlatformPluginFamily 5.7.0d10
com.apple.AppleGraphicsDeviceControl 3.4.35
com.apple.iokit.IOAcceleratorFamily2 98.14
com.apple.driver.AppleHDAController 2.6.0f1
com.apple.iokit.IOGraphicsFamily 2.4.1
com.apple.iokit.IOHDAFamily 2.6.0f1
com.apple.driver.AppleUSBMultitouch 240.9
com.apple.iokit.IOUSBHIDDriver 660.4.0
com.apple.driver.AppleThunderboltDPInAdapter 3.1.7
com.apple.driver.AppleThunderboltDPAdapterFamily 3.1.7
com.apple.driver.AppleThunderboltPCIDownAdapter 1.4.5
com.apple.iokit.IOSCSIBlockCommandsDevice 3.6.6
com.apple.iokit.IOUSBMassStorageClass 3.6.0
com.apple.iokit.IOSCSIArchitectureModelFamily 3.6.6
com.apple.driver.AppleUSBMergeNub 650.4.0
com.apple.driver.AppleUSBComposite 656.4.1
com.apple.iokit.IOUSBUserClient 660.4.2
com.apple.iokit.IOAHCIFamily 2.6.5
com.apple.driver.AppleThunderboltNHI 2.0.1
com.apple.iokit.IOThunderboltFamily 3.2.7
com.apple.iokit.IO80211Family 630.35
com.apple.driver.mDNSOffloadUserClient 1.0.1b5
com.apple.iokit.IONetworkingFamily 3.2
com.apple.iokit.IOUSBFamily 675.4.0
com.apple.driver.AppleEFINVRAM 2.0
com.apple.driver.AppleEFIRuntime 2.0
com.apple.iokit.IOHIDFamily 2.0.0
com.apple.iokit.IOSMBusFamily 1.1
com.apple.security.sandbox 278.11
com.apple.kext.AppleMatch 1.0.0d1
com.apple.security.TMSafetyNet 7
com.apple.driver.AppleKeyStore 2
com.apple.driver.DiskImages 371.1
com.apple.iokit.IOStorageFamily 1.9
com.apple.iokit.IOReportFamily 23
com.apple.driver.AppleFDEKeyStore 28.30
com.apple.driver.AppleACPIPlatform 2.0
com.apple.iokit.IOPCIFamily 2.9
com.apple.iokit.IOACPIFamily 1.4
com.apple.kec.corecrypto 1.0
com.apple.kec.pthread 1
CUDA Mac Driver
Latest Version: CUDA 5.5.47 driver for MAC
Release Date: 03/05/2014
Previous Releases:
CUDA 5.5.28 driver for MAC
Release Date: 10/23/2013
CUDA driver 5.5.47 is required for CUDA support on Mac OS X 10.9 Mavericks. Please see supported products tab.
An alternative method to download the latest CUDA driver is within Mac OS environment. Access the latest driver through System Preferences > Other > CUDA. Click ‘Install CUDA Update’.