More on PowerShell multithreading via runspace pools

It turned out to be easier to write the Lock-Object function than it was to test it.

There has been some great work done on figuring out the use of Runspaces for multithreading by the PowerShell community over the past couple of years (by Boe Prox, Dr. Tobias Weltner, rambling cookie monster, Tome Tanasovski, jrich, etc.)

The post I linked for Boe Prox was particularly helpful, in this case: it’s the one that showed me that it’s possible to have concurrent access to objects in PowerShell in the first place, and also became my starting point for figuring out how to test Lock-Object. In that blog post, he mentioned two methods for enabling this type of shared access: Runspace.SessionStateProxy.SetVariable() (for working with single runspace objects), and PowerShell.AddArgument() (when working with Runspace pools.) Both of these were fine for sharing objects, but I was finding that my Lock-Object function wasn’t loaded in the other runspaces, even though I had loaded it in my session.

I went through a few code changes before I arrived at my preferred approach, and while doing so, also found a more convenient way of sharing objects when using Runspace pools:

  • First, I just added Import-Module calls to each background job to make sure they could access the Lock-Object function. I was still also defining a param block and using PowerShell.AddArgument() to pass in the variables.
  • I took out the calls to Import-Module and installed the module into my PSModulePath. This also works fine, if you’re running PowerShell 3.0 or later and haven’t disabled module auto-loading.
  • Finally, I started looking at the properties of the various objects in play (RunspacePool, InitialSessionState, PowerShell, etc), and it was in the InitialSessionState class that I struck gold. It has methods named ImportPSModule(), ImportPSModulesFromPath(), and ImportPSSnapin() which can be used to make sure all of the background runspaces have access to the Lock-Object command. It also has a Variables property which can be used to share variables across every runspace in the pool, without having to define arguments and param blocks for each individual command.

Here’s how it works. (I’m trying out Gists for sharing the code, since the sourcecode tag on wordpress creates such a narrow window with the need for side-scrolling.)

Here’s the complete test code. By commenting out the calls to Lock-Object in each of the background runspaces as I’ve done, you can see that you’ll get an error from the second runspace of “Collection was modified; enumeration operation may not execute.” By uncommenting the calls to Lock-Object, the code works as intended, with each thread waiting its turn to work with the list.

Note: By writing code that requires synchronized access to objects, you’ll always be losing at least some of the performance benefit of using multithreading in the first place. If a thread has to sit there and do nothing while it waits to execute (as in the test code), there was no benefit at all over just executing the two script blocks one after the other in the same thread. Designing your code so the threads can do as much as possible without interfering with one another is a discussion beyond the scope of this post.

On a side note, Runspace, RunspacePool and PowerShell objects are Disposable. In production code, you should use try/finally to make sure Dispose() is called after your jobs are done, to release the various resources that those objects create. In the test code, I didn’t want to distract from what I was focused on, and any leaked resources would be released after either closing my PowerShell session or rebooting my computer anyway.

Advertisements

About Dave Wyatt

Microsoft MVP (PowerShell), IT professional.
This entry was posted in PowerShell and tagged , , , . Bookmark the permalink.

3 Responses to More on PowerShell multithreading via runspace pools

  1. Good stuff!

    Thanks for the mention, but credit goes to Boe, Tobias, Tome, and other folks I borrowed from who know what they are doing : )

    You mentioned looking through the objects / classes in question – I found this quite helpful, would recommend it for anyone diving into working with runspaces! Run through the example runspace code, flip through the MSDN documentation on initialsessionstate, runspacefactory, runspacepool, and powershell, and explore and experiment with various properties and methods at the prompt.

    Thanks for posting the Lock-Object function, it’s quite handy!

    Like

  2. Pingback: Adding a Multithreaded DataGridView to a Powershell form | Sys Admin Jam

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s