Test Explorer intermittently never finishes finding tests

Reported by Denise Oquist [MSFT] Jun 05, 2017 at 06:31 PM unit-test fixed in: visual studio 2017 version 15.7 preview 3 windows 10.0 Fixed In: Visual Studio 2017 version 15.7 Visual Studio 2017 version 15.2 visual studio

On occasion, after building a relatively large solution, Test Explorer takes forever to discover tests. By forever I mean that I wait a couple minutes, eventually loose patience, and restart Visual Studio to solve the problem. I've hit this issue at least 3 times so far.

Usually when this happens I suspect that I took some sort of action in Visual Studio that trips it up somehow. Maybe something gets paused but never resumed? Or it's just the UI that fails to update after Visual Studio finishes to find tests? In any case: This never happened in Visual Studioi 2015. :(

Add comment Show 36

Satya Madala [MSFT] · Jun 06, 2017 at 09:01 AM 0

Thank you for your feedback!

How many test projects and tests does the solution has?

In successful scenarios how much time it takes to discover all tests?

What is the kind of test projects (Target framework: net46, netcoreapp10, Test framework:mstest, xunit, project type: Desktop, web, UWP)?

Can you please provide call stack for devenv.exe, vstest.console.exe and vstest.discoveryengine.exe following URL: https://superuser.com/questions/462989/how-can-i-view-the-call-stack-of-a-running-process-thread if same issue happens again?

We look forward to hearing from you!

Denise Oquist [MSFT] Satya Madala [MSFT] · Jun 06, 2017 at 04:29 PM 0

The solution has a total of 53 projects, of which:
- 4 are test projects.
- 3 are projects that support test execution, but I don't believe they have tests themselves.
When VisualStudio successfully detects the tests, it takes <1min... Probably more like 15 seconds or so.
We're developing a service, but I suppose that counts as a desktop app? I'm building x64 debug. For the most part we're on .NET 4.6. At least one of the projects still uses .NET 4.5.

Shyam Gupta [MSFT] · Jun 07, 2017 at 08:17 AM 0

Thanks for the information Denise.

Please let us know the Test Framework as well: MSTest, Xunit etc.

Also capture and share process dump of following processes when Visual Studio hangs: devenv.exe, vstest.console.exe and vstest.discoveryengine.exe

Steps (Repeat for each process one by one): Open Task Manager -> Details tab -> locate one of the above mentioned process -> right click -> Create dump file. Edit: Use SystemInternals process explorer to create Full dump.( Find process -> right click -> Create dump -> Create Full dump. )

Once files are saved, share them with us. We look forward to hearing from you!

Denise Oquist [MSFT] Shyam Gupta [MSFT] · Jun 07, 2017 at 04:58 PM 0

I believe we're using MSTest.

using Microsoft.VisualStudio.TestTools.UnitTesting;
using Moq;

VisualStudio doesn't hang in its entirety. The UI is plenty responsive. It's just that the Test Explorer UI shows that it's still trying to find tests... the progress bar is moving... and it keeps looking forever.

Fredrik Magnusson · Jun 11, 2017 at 09:36 PM 0

I have the same problem after updating to 15.2 (previous version did not have this issue for me). Test explorer shows a loading bar that never stops. Build or Rebuild does not help. Output panel does not show anything intereseting (exception informational messages). I have restarted VS several times and it never stops loading after update.

Terje Sandstrom (AAD) · Jul 05, 2017 at 08:12 PM 0

I have noticed the same, using NUnit, and size of sln doesn't matter. The issue is also reported here: https://github.com/nunit/nunit3-vs-adapter/issues/352

Satya Madala [MSFT] Terje Sandstrom (AAD) · Jul 06, 2017 at 01:58 PM 0

Hi @Terje Sandstrom, Thanks for reporting issue here. Can you please share following:

Logs from test platform: Follow https://github.com/Microsoft/vstest-docs/blob/master/docs/diagnose.md#collect-trace-using-config-file. You find config file at "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe.config".

Full dump files of devenv.exe, vstest.console.exe and dotnet.exe(child process of vstest.console.exe), To Create dump files,

Steps (Repeat for each process one by one): Edit: Use SystemInternals process explorer application to create Full dump.( Open process explorer -> Find process -> right click -> Create dump -> Create Full dump. )

Does running "dotnet test --list-tests" always works in Nunit test project directory from powershell?

Test explorer logs: Since you are on 15.2 can you set VS_UTE_Diagnostics system wide environment variable to 1 and help us with the logs from Output->Tests pane please? Turning this logging on should be possible through Tools->Options in 15.3.

Also if you could share a repro, that would help greatly.

Jayarani Garg [MSFT] Terje Sandstrom (AAD) · Jul 21, 2017 at 12:23 PM 0

Hi @Terje Sandstrom (AAD) @Gian Maria Ricci : Can you please provide information requested by Satya.

Without this information, this feedback will not be actionable and we will have to close this at our end.

Gian Maria Ricci · Jul 06, 2017 at 10:08 AM 0

It happens quite often for me in a solution where I have Mspec test runner, but it happens sometimes even with only NUnit tests. It is absolutely random, so it is not reproducible.

Satya Madala [MSFT] Gian Maria Ricci · Jul 06, 2017 at 02:06 PM 0

Can you please share info asked in https://developercommunity.visualstudio.com/comments/76957/view.html?

Manfred Lange · Aug 10, 2017 at 05:37 AM 0

Where would I submit the dump files without sharing them with the general public?

I have the same issue when using Visual Studio Professional 2017 15.2 (26430.16) with NUnit adapter. I have to restart VS multiple times a day which has become extremely annoying. The workaround is to enable ReSharper again which doubles CPU consumption and triples memory consumption, and as a result has the added side effect of Visual Studio crashing at least once a day with OutOfMemoryException. So I'm between a rock and a hard place....

Satya Madala [MSFT] Manfred Lange · Aug 10, 2017 at 06:26 AM 0

Change comment visibility from "Viewable by all users" to required visibility.

Satya Madala [MSFT] Satya Madala [MSFT] · Aug 21, 2017 at 08:06 AM 0

Thanks @Manfred Lange for all dumps. I will analyze these dump and come back to you with update.

Manfred Lange · Aug 17, 2017 at 08:20 AM 0

It appears as if occasionally if this happens rebuilding the solution helps. However, it may take several minutes. Note that I do not have enough data points to confirm this is always the case. There are cases where this does not solve the problem. Right now I was hoping that I could send another set of dump files. By the time I had sorted out the correct login for this site (there is room for improvements!!) VS finished finding tests. I'll continue to try provide more dump files as soon as I can.

Jerry · Oct 23, 2017 at 06:51 PM 0

I see this very often, and it's very annoying. I'm using XUnit 2.2.0.3545, Visual Studio 15.3.5, in Windows 7 (company decision, not mine). The "Test Explorer" works after starting VS; it might even work a couple of times. But something changes, and then it just spins (searching for tests) forever [I've left it for a couple of days and it still spins]. The only way to fix it is to restart VS, sometimes more than once.

Manish Jayaswal [MSFT] · Nov 01, 2017 at 03:13 AM 0

Couple of requests: Do you see any message in the output window ( under test tab) when the spinner keeps spinning?

Does this issue still repro in VS 2017 Update 5, preview 2?

Jerry Manish Jayaswal [MSFT] · Nov 01, 2017 at 12:41 PM 0

This is what TestExplorer shows, right after a successful build. It just spins forever. I'm running VS2017 v15.4.1

capture1.jpg (19.4 kB)

Oz Solomon [MSFT] Manish Jayaswal [MSFT] · Nov 01, 2017 at 04:18 PM 0

Output window only shows:

[11/1/2017 12:06:18 PM Informational] ========== Discover test finished: 183 found (0:00:01.3764734) ==========
[11/1/2017 12:06:18 PM Informational] ------ Discover test started ------

The Test Explorer spins as per Jerry's picture above. I'm also on v15.4.1 Enterprise.

Oz Solomon [MSFT] Manish Jayaswal [MSFT] · Nov 01, 2017 at 04:19 PM 0

I'll add one more tidbit: If I close the solution when the test explorer is stuck in this state, the solution closes but the test explorer still shows whatever tests it previously discovered and it still spins.

Paulius Zaliaduonis at DanskeBank · Nov 08, 2017 at 07:33 AM 1

I have started visual studio
test explorer is set to Auto Hide
Opened test explorer (no tests, progress active)
Build the solution Ctrl+Shift+B (some projects build)
Run without debugging
Opened test explorer (no tests, progress active)
The progress bar stays active forever
Test Output contents attached visualstudiotestoutput.txt

visualstudiotestoutput.txt (267.5 kB)

Abhitej Anoop John Bandi [MSFT] Paulius Zaliaduonis at DanskeBank · Dec 22, 2017 at 10:31 PM 0

@Paulius Zaliaduonis at DanskeBank : Thanks for the log. In the event viewer do you see a process crash report during this time? If these are .Net Framework projects then the process of interest is vstest.discoveryengine*.exe/testhost*.exe. If this is a .Net Core project then it would be a dotnet.exe.

Paulius Zaliaduonis at DanskeBank Abhitej Anoop John Bandi [MSFT] · Dec 27, 2017 at 07:56 AM 0

Hi, I have not noticed any processes crashing yet. Where do I look for them? Windows Event Log? Whitch section?

Show more comments

Added a solution by Oz Solomon [MSFT] · Oct 30, 2017 at 02:16 PM

One thing I've noticed is that this happens often when "Lightweight Solution Load" is enabled and projects the unit tests depend on aren't loaded. You can either disable "Lightweight Solution Load" or open the Configuration Manager to force the entire solution to load.

Add comment Show 1 · Share

Jerry · Oct 30, 2017 at 04:09 PM 0

No, we are not using Lightweight Solution Load.

Added a solution by Hunter Nance · Jan 16 at 05:49 PM

I updated to VS2017 today and now this is happening consistently every time I right click->debug a Selenium test in Test Explorer.. if I close the browser or hit the stop button it will act like it is continuing, so I will hit cancel at which point it will switch to saying "Canceling" and it will hang forever until I close and reopen VS.

I just updated to VS2017 and this now happens every single time I right click to debug a Selenium test. When I click cancel it will say "Canceling" and will stay in this state indefinitely

Add comment Show 1 · Share

Manish Jayaswal [MSFT] · Jan 16 at 06:03 PM 0

@Satya Madala [MSFT] @Abhitej Anoop John Bandi [MSFT] - FYI

@Hunter Nance - We have investigated this and have a fix for it. We expect to ship a public release with the fix soon.

Added a solution by Simon Liddington · Mar 24 at 12:59 PM

Same here with NUnit tests in a large solution with 229 Projects. Test Explorer keeps "spinning". Test output pane shows NUnit test adapter has finished but VS does not i.e. I get the first line below but NOT the second (from a successful completion). Disabling source

[19/03/2018 12:22:19 PM Informational] NUnit Adapter 3.10.0.21: Test discovery complete
[19/03/2018 12:22:20 PM Informational] ========== Discover test finished: 2901 found (0:00:11.7327234) ==========

...

By process of elimination I narrowed it down to this test that was showing up in the test output pane with the following warning. Giving it an explicit name using SetName() instead of relying on the long string parameter fixed the problem. Although oddly other similar warnings for shorter strings were not causing the same issue.

[24/03/2018 11:51:22 PM Warning] Contents of string 'SoftFurnishings.UnitTests.CurtainMFPricingConfigurationsUnitTest.FromJson_MigrationSample2_Matches(SoftFurnishings.CurtainMFPricingConfigurations,"..<long string>..")' exceeds max size of '1024'. String has been truncated.

Using

VS Pro 15.6.4

NUnit Test Adapter 3.10.0

NUnit 3.10.1

Add comment Show 1 · Share

Abhitej Anoop John Bandi [MSFT] · Mar 26 at 05:53 PM 0

@Simon Liddington: Thanks for the report. Just a clarification: If you disable that test, does test discovery always complete? For .Net Framework based tests Nunit, like other adapters, creates an app domain to discover/execute tests which could potentially be having issues with tear down. When you see devenv in this state, could you let us know if there is a "testhost*.exe" process alive and if yes, share its dump with us please? If it isn't alive then that's most likely a similar issue as this thread (only that it's happening on completion and not on init) - we would be interested in the dumps of "vstest.console.exe" and "devenv.exe" in that case to narrow this down.

Thanks,
Abhitej.

Added a solution by Marc Levesque · Apr 19 at 02:16 PM

I have the same issue, discover tests never finishes. The project is a WAP. I'm using XUnit. The solution only has 4 projects in it; the WAP, 2 dlls, and the tests. There are only 39 tests in the test project. All projects target .Net 4.5.2.

I have included the current text from the Tests output window. I don't know what is meant by "None of the specified source(s)...is valid. Fix the above errors/warnings and then try again." because the indicated files are there.

In this instance at least, I don't believe the tests had not run since I had opened the solution. I started a build, then noticed that I didn't have the option selected to "Run tests after build", so while it was building, I clicked the button. I have done this in the past without issue. In this case I just did it a few minutes ago, maybe 10-15 now, but definitely not around 8:24am, which is the last timestamp on the output. In fact I don't know why the tests would have tried running at all this morning since as I mentioned, I didn't have the option turned on to run the tests after build.

I can confirm that the path ~\AppData\Local\Temp\VisualStudioTestExplorerExtensions\xunit.runner.visualstudio.2.1.0/build is empty as indicated in the output.

Closing and opening VS is also my solution. Although the same path mentioned above is still empty (the same error appeared in the output window after restarting VS), the tests were discovered and ran (automatically) without a problem after restarting VS.

I didn't notice anything out of sorts in Event Viewer that looks like it would be related.

I have Visual Studio Enterprise 2017, 15.6.4, and I don't even see an option for Lightweight Solution Load.

Add comment Show 5 · Share

Abhitej Anoop John Bandi [MSFT] · Apr 19 at 07:06 PM 0

@Marc Levesque: Thanks for the report. The log file does not seem to have come through. Can you try attaching it again? I hope the logging is set to Diagnostics in Tools -> Options -> Tests.

Marc Levesque · Apr 20 at 01:25 PM 0

@Abhitej Anoop John Bandi [MSFT] I haven't had the problem yet but I wanted to make sure I got as much data as possible when it does happen. I turned on Diagnostics logging mode, but the Output window doesn't show any more data than it did before I made the change. Is this where I should be getting the data or is it logged somewhere else?

Abhitej Anoop John Bandi [MSFT] Marc Levesque · Apr 23 at 05:26 AM 0

@Marc Levesque Yes, That's where it should show up. There was one issue in logging that I think you might be running into. You might want to navigate to Tools -> Options -> Test once on the VS instance you are running tests on just to ensure that the logging level is set properly for that instance. This should output more logs. Do let us know if that doesn't work.

Marc Levesque Abhitej Anoop John Bandi [MSFT] · May 01 at 07:50 PM 0

@Abhitej Anoop John Bandi [MSFT] Of course now VS is making a liar out of me. I haven't had the problem with the tests hanging since I posted initially, but only once (today in fact) have I seen the more verbose text in the output window. Every other time it's the simple logging, and I am constantly checking the option in VS and it is set to "Diagnostic".

Show more comments

Added a solution by Blair Wang [MSFT] · May 08 at 07:49 AM

This issue has been fixed and is now available in our latest update. You can download the update via the in-product notification or from here: https://www.visualstudio.com/vs/ Thanks to everyone who provided feedback, votes and solutions for this problem.

Add comment Show 2 · Share

Austin Salgat · May 10 at 08:35 PM 0

Unfortunately we're still experiencing this issue with Visual Studio 2017 version 15.7.1. Attached are our logs. Let me know if more info is needed.

testdiscoveryfailure.txt

testdiscoveryfailure.txt (37.1 kB)

Manish Jayaswal [MSFT] Austin Salgat · May 10 at 10:38 PM 0

@Austin Salgat - the logs indicate multiple issues. Firstly it seems that nUnit engine itself is crashing. Can you isolate the test which is causing this? It also seems like the assemblies are getting built as x64 but the test setting expect x86 assemblies?

If you can share a small repro solution where the discovery is hanging for you then we can try to investigate.

Added a solution by Michael Scott Forbes · Sep 14, 2017 at 11:50 AM

I have a similar problem as well. It usually happens to me upon ending a test, like it is caught in an infinite loop and can't recover. I usually have to close Visual Studio and restart. And I do believe it has happened exclusively when I am debugging a test. This has been happening through several updates of Visual Studio 2017. I am currently on 15.3.4, and the problem still exists.

Add comment Show 2 · Share

Michael Scott Forbes · Sep 22, 2017 at 02:04 PM 0

I just noticed today that I had the same issue happen after I added a new test and did a build. The Test Explorer appears to be scanning for new tests, but never recovers, and appears to sit indefinitely scanning. I rectify the problem by restarting Visual Studio. Annoying, but consistently works.

Abhitej Anoop John Bandi [MSFT] Michael Scott Forbes · Dec 22, 2017 at 10:19 PM 0

@Michael Scott Forbes: Can you tell us what framework these tests are built against please(mstest/xunit/nunit)? And the .Net Framework they target? The Output->Tests pane logs would really help too. Some folks on this thread seem to have Lightweight Solution Load enabled. Do you have so too?

Developer Community

Get help from our community supported forum

Test Explorer intermittently never finishes finding tests

Solutions

Try Microsoft Edge

Developer Community

Get help from our community supported forum

Test Explorer intermittently never finishes finding tests

Solutions

Related Problems