VMware Cloud Community
clouddstarr
Contributor
Contributor
‎07-15-2019 04:26 AM

vROPS Nvidia GPU Adapter not collecting DATA / ERROR java.lang.IndexOutOfBoundsException

Hello Community,

I'm having trouble with the NVIDIA GPU Adapter not collecting/showing Data for the NVIDIA GPU Dashboards.

Screenshot 2019-07-15 at 12.58.08.png

Screenshot 2019-07-15 at 13.10.07.pngScreenshot 2019-07-15 at 13.10.38.png

The NVIDIA Adapter is configured correctly and its showing "Collecting":

Screenshot 2019-07-15 at 12.51.59.png

Screenshot 2019-07-15 at 12.52.10.png

But there are no information collected whatsoever especially if you look at the "last collection time"

Screenshot 2019-07-15 at 12.52.41.png

If I check the Nvidia Adapter Logs I can see a Java ERROR:

Screenshot 2019-07-15 at 13.07.55.png

[178] 2019-05-14 04:15:16,356 ERROR [Collector worker thread 6] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[179] 2019-05-14 04:15:16,908 ERROR [Collector worker thread 6] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[180] 2019-05-14 04:15:17,177 ERROR [Collector worker thread 6] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[181] 2019-05-14 04:15:17,904 ERROR [Collector worker thread 6] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[182] 2019-05-14 04:15:18,079 ERROR [Collector worker thread 6] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[183] 2019-05-14 04:15:18,361 ERROR [Collector worker thread 6] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[184] 2019-05-14 04:15:18,955 ERROR [Collector worker thread 6] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[185] 2019-05-14 04:15:19,435 ERROR [Collector worker thread 6] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[186] 2019-05-14 04:15:19,788 ERROR [Collector worker thread 6] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[187] 2019-05-14 04:15:21,137 ERROR [Collector worker thread 6] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[188] 2019-05-15 03:20:19,794 ERROR [Collector worker thread 9] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[189] 2019-05-15 04:15:18,035 ERROR [Collector worker thread 9] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[190] 2019-05-15 04:15:18,309 ERROR [Collector worker thread 9] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[191] 2019-05-15 04:15:19,075 ERROR [Collector worker thread 9] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[192] 2019-05-15 04:15:19,243 ERROR [Collector worker thread 9] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[193] 2019-05-15 04:15:19,797 ERROR [Collector worker thread 9] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[194] 2019-05-15 04:15:20,216 ERROR [Collector worker thread 9] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[195] 2019-05-15 04:15:20,537 ERROR [Collector worker thread 9] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[196] 2019-05-15 04:15:21,721 ERROR [Collector worker thread 9] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[197] 2019-05-16 03:20:17,429 ERROR [Collector worker thread 15] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[198] 2019-05-16 04:15:20,305 ERROR [Collector worker thread 23] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[199] 2019-05-16 04:15:21,004 ERROR [Collector worker thread 23] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[200] 2019-05-16 04:15:21,323 ERROR [Collector worker thread 23] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[201] 2019-05-16 04:15:22,148 ERROR [Collector worker thread 23] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[202] 2019-05-16 04:15:22,310 ERROR [Collector worker thread 23] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[203] 2019-05-16 04:15:22,864 ERROR [Collector worker thread 23] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[204] 2019-05-16 04:15:23,344 ERROR [Collector worker thread 23] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[205] 2019-05-16 04:15:25,092 ERROR [Collector worker thread 23] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[206] 2019-05-17 04:15:19,191 ERROR [Collector worker thread 4] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[207] 2019-05-17 04:15:19,645 ERROR [Collector worker thread 4] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[208] 2019-05-17 04:15:19,882 ERROR [Collector worker thread 4] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[209] 2019-05-17 04:15:20,904 ERROR [Collector worker thread 4] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[210] 2019-05-17 04:15:21,591 ERROR [Collector worker thread 4] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[211] 2019-05-17 04:15:22,191 ERROR [Collector worker thread 4] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[212] 2019-05-18 03:20:17,756 ERROR [Collector worker thread 7] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[213] 2019-05-18 03:20:17,875 ERROR [Collector worker thread 7] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

[214] 2019-05-19 03:20:19,921 ERROR [Collector worker thread 10] (951) com.nvidia.nvvgpu.adapter.client.DcgmClient.getVgpuUtilizationInfo - java.lang.IndexOutOfBoundsException

Anybody got an idea what the meaning behind this error is?Even a pointer in the right direction would be wonderful.

It seems it has something to do with the requested Size of the Array but I'm not sure 😕

Screenshot 2019-07-15 at 13.17.25.png

Thanks for the help in advance. I can provide further information if needed.

It's my first post so I hope its alright:)

Cheers,

clouddstarr

0 Kudos
2 Replies
daphnissov
Immortal
Immortal
‎07-16-2019 06:23 PM

Are you sure that you're connecting this to a supported GPU? Check the documentation for that adapter and see what is and what is not supported. That's the first place to start.

------------------
How to Ask for Help on Tech Forums
https://neonmirrors.net
0 Kudos
clouddstarr
Contributor
Contributor
‎07-19-2019 04:09 AM

Hello Guys,

thank you for your Reply.

This should definitely be possible for our Environment from what I know.

We are using the ESXi 6.7

Screenshot 2019-07-17 at 10.40.02.png

and got an M60 and a P40-Cluster (Windows 7) running that is supported for vGPU

Screenshot 2019-07-17 at 10.40.29.png

The only possible mismatch I can see is that we are using the vGPU Windows Driver 412.31 with vGPU Software 8.0

Screenshot 2019-07-17 at 10.46.51.png

Screenshot 2019-07-17 at 10.46.56.png

Do we need to update our Windows Driver to Version 425.31 or is this not the cause?

Anything else I can check?

I'm quite new to the topic so they could be some misunderstandings from my side.

Sorry for that Smiley Happy

Greetings,

Nikolai

0 Kudos