Test Explorer intermittently never finishes finding tests
On occasion, after building a relatively large solution, Test Explorer takes forever to discover tests. By forever I mean that I wait a couple minutes, eventually loose patience, and restart Visual Studio to solve the problem. I've hit this issue at least 3 times so far.
Usually when this happens I suspect that I took some sort of action in Visual Studio that trips it up somehow. Maybe something gets paused but never resumed? Or it's just the UI that fails to update after Visual Studio finishes to find tests? In any case: This never happened in Visual Studioi 2015. :(
Add Comment
Thank you for your feedback!
How many test projects and tests does the solution has?
In successful scenarios how much time it takes to discover all tests?
What is the kind of test projects (Target framework: net46, netcoreapp10, Test framework:mstest, xunit, project type: Desktop, web, UWP)?
Can you please provide call stack for devenv.exe, vstest.console.exe and vstest.discoveryengine.exe following URL: https://superuser.com/questions/462989/how-can-i-view-the-call-stack-of-a-running-process-thread if same issue happens again?
We look forward to hearing from you!
The solution has a total of 53 projects, of which:
- 4 are test projects.
- 3 are projects that support test execution, but I don't believe they have tests themselves.
When VisualStudio successfully detects the tests, it takes <1min... Probably more like 15 seconds or so.
We're developing a service, but I suppose that counts as a desktop app? I'm building x64 debug. For the most part we're on .NET 4.6. At least one of the projects still uses .NET 4.5.
Thanks for the information Denise.
Please let us know the Test Framework as well: MSTest, Xunit etc.
Also capture and share process dump of following processes when Visual Studio hangs: devenv.exe, vstest.console.exe and vstest.discoveryengine.exe
Steps (Repeat for each process one by one): Open Task Manager -> Details tab -> locate one of the above mentioned process -> right click -> Create dump file. Edit: Use SystemInternals process explorer to create Full dump.( Find process -> right click -> Create dump -> Create Full dump. )
Once files are saved, share them with us. We look forward to hearing from you!
I believe we're using MSTest.
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Moq;
VisualStudio doesn't hang in its entirety. The UI is plenty responsive. It's just that the Test Explorer UI shows that it's still trying to find tests... the progress bar is moving... and it keeps looking forever.
I have the same problem after updating to 15.2 (previous version did not have this issue for me). Test explorer shows a loading bar that never stops. Build or Rebuild does not help. Output panel does not show anything intereseting (exception informational messages). I have restarted VS several times and it never stops loading after update.
I have noticed the same, using NUnit, and size of sln doesn't matter. The issue is also reported here: https://github.com/nunit/nunit3-vs-adapter/issues/352
Hi @Terje Sandstrom, Thanks for reporting issue here. Can you please share following:
Logs from test platform: Follow https://github.com/Microsoft/vstest-docs/blob/master/docs/diagnose.md#collect-trace-using-config-file. You find config file at "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe.config".
Full dump files of devenv.exe, vstest.console.exe and dotnet.exe(child process of vstest.console.exe), To Create dump files,
Steps (Repeat for each process one by one): Edit: Use SystemInternals process explorer application to create Full dump.( Open process explorer -> Find process -> right click -> Create dump -> Create Full dump. )
Does running "dotnet test --list-tests" always works in Nunit test project directory from powershell?
Test explorer logs: Since you are on 15.2 can you set VS_UTE_Diagnostics system wide environment variable to 1 and help us with the logs from Output->Tests pane please? Turning this logging on should be possible through Tools->Options in 15.3.
Also if you could share a repro, that would help greatly.
Hi @Terje Sandstrom (AAD) @Gian Maria Ricci : Can you please provide information requested by Satya.
Without this information, this feedback will not be actionable and we will have to close this at our end.
It happens quite often for me in a solution where I have Mspec test runner, but it happens sometimes even with only NUnit tests. It is absolutely random, so it is not reproducible.
Can you please share info asked in https://developercommunity.visualstudio.com/comments/76957/view.html?
Where would I submit the dump files without sharing them with the general public?
I have the same issue when using Visual Studio Professional 2017 15.2 (26430.16) with NUnit adapter. I have to restart VS multiple times a day which has become extremely annoying. The workaround is to enable ReSharper again which doubles CPU consumption and triples memory consumption, and as a result has the added side effect of Visual Studio crashing at least once a day with OutOfMemoryException. So I'm between a rock and a hard place....
Change comment visibility from "Viewable by all users" to required visibility.
Thanks @Manfred Lange for all dumps. I will analyze these dump and come back to you with update.
It appears as if occasionally if this happens rebuilding the solution helps. However, it may take several minutes. Note that I do not have enough data points to confirm this is always the case. There are cases where this does not solve the problem. Right now I was hoping that I could send another set of dump files. By the time I had sorted out the correct login for this site (there is room for improvements!!) VS finished finding tests. I'll continue to try provide more dump files as soon as I can.
I see this very often, and it's very annoying. I'm using XUnit 2.2.0.3545, Visual Studio 15.3.5, in Windows 7 (company decision, not mine). The "Test Explorer" works after starting VS; it might even work a couple of times. But something changes, and then it just spins (searching for tests) forever [I've left it for a couple of days and it still spins]. The only way to fix it is to restart VS, sometimes more than once.
Couple of requests: Do you see any message in the output window ( under test tab) when the spinner keeps spinning?
Does this issue still repro in VS 2017 Update 5, preview 2?
This is what TestExplorer shows, right after a successful build. It just spins forever. I'm running VS2017 v15.4.1
Output window only shows:
[11/1/2017 12:06:18 PM Informational] ========== Discover test finished: 183 found (0:00:01.3764734) ==========
[11/1/2017 12:06:18 PM Informational] ------ Discover test started ------
The Test Explorer spins as per Jerry's picture above. I'm also on v15.4.1 Enterprise.
I'll add one more tidbit: If I close the solution when the test explorer is stuck in this state, the solution closes but the test explorer still shows whatever tests it previously discovered and it still spins.
- I have started visual studio
- test explorer is set to Auto Hide
- Opened test explorer (no tests, progress active)
- Build the solution Ctrl+Shift+B (some projects build)
- Run without debugging
- Opened test explorer (no tests, progress active)
- The progress bar stays active forever
- Test Output contents attached visualstudiotestoutput.txt
@Paulius Zaliaduonis at DanskeBank : Thanks for the log. In the event viewer do you see a process crash report during this time? If these are .Net Framework projects then the process of interest is vstest.discoveryengine*.exe/testhost*.exe. If this is a .Net Core project then it would be a dotnet.exe.
Hi, I have not noticed any processes crashing yet. Where do I look for them? Windows Event Log? Whitch section?
Solution by
Add Comment
@Satya Madala [MSFT] @Abhitej Anoop John Bandi [MSFT] - FYI
@Hunter Nance - We have investigated this and have a fix for it. We expect to ship a public release with the fix soon.
Solution by
Add Comment
@Simon Liddington: Thanks for the report. Just a clarification: If you disable that test, does test discovery always complete? For .Net Framework based tests Nunit, like other adapters, creates an app domain to discover/execute tests which could potentially be having issues with tear down. When you see devenv in this state, could you let us know if there is a "testhost*.exe" process alive and if yes, share its dump with us please? If it isn't alive then that's most likely a similar issue as this thread (only that it's happening on completion and not on init) - we would be interested in the dumps of "vstest.console.exe" and "devenv.exe" in that case to narrow this down.
Thanks,
Abhitej.
Solution by
Add Comment
@Marc Levesque: Thanks for the report. The log file does not seem to have come through. Can you try attaching it again? I hope the logging is set to Diagnostics in Tools -> Options -> Tests.
@Abhitej Anoop John Bandi [MSFT] I haven't had the problem yet but I wanted to make sure I got as much data as possible when it does happen. I turned on Diagnostics logging mode, but the Output window doesn't show any more data than it did before I made the change. Is this where I should be getting the data or is it logged somewhere else?
@Marc Levesque Yes, That's where it should show up. There was one issue in logging that I think you might be running into. You might want to navigate to Tools -> Options -> Test once on the VS instance you are running tests on just to ensure that the logging level is set properly for that instance. This should output more logs. Do let us know if that doesn't work.
@Abhitej Anoop John Bandi [MSFT] Of course now VS is making a liar out of me. I haven't had the problem with the tests hanging since I posted initially, but only once (today in fact) have I seen the more verbose text in the output window. Every other time it's the simple logging, and I am constantly checking the option in VS and it is set to "Diagnostic".
Solution by
Add Comment
Unfortunately we're still experiencing this issue with Visual Studio 2017 version 15.7.1. Attached are our logs. Let me know if more info is needed.
@Austin Salgat - the logs indicate multiple issues. Firstly it seems that nUnit engine itself is crashing. Can you isolate the test which is causing this? It also seems like the assemblies are getting built as x64 but the test setting expect x86 assemblies?
If you can share a small repro solution where the discovery is hanging for you then we can try to investigate.
Solution by
Add Comment
I just noticed today that I had the same issue happen after I added a new test and did a build. The Test Explorer appears to be scanning for new tests, but never recovers, and appears to sit indefinitely scanning. I rectify the problem by restarting Visual Studio. Annoying, but consistently works.
@Michael Scott Forbes: Can you tell us what framework these tests are built against please(mstest/xunit/nunit)? And the .Net Framework they target? The Output->Tests pane logs would really help too. Some folks on this thread seem to have Lightweight Solution Load enabled. Do you have so too?