Possible to execute ForEach-Object commands in Parallel

Question

I have been attempting to execute parallel commands on data (such as a list of Server Names) that is imported via CSV and called by ForEach-Object. I have found that foreach has a parallel method but it appears that ForEach-Object does not. I am concerned about good performance as I hit multiple servers to read log data, and have the following options:

1) PowerShell Workflow 
2) PowerShell Background Job 
3) Invoke-Command

When I tried option 3, it failed as I tried to send arguments via -ArgumentList, but it didn't seem to like the variables I was pushing from the CSV in the form of:

$($_.'CSVColumnValue')

Is it possible to execute commands in parallel using ForEach-Object, and if so, what is the recommended method to achieve this?

please add your code to your question, otherwise it's difficult to give advice — SimonS
– SimonS, Commented Mar 5, 2016 at 12:25
You also may want to look at Invoke-ScriptAsync which uses RunSpace Pools to quickly run a script against a list of servers. — user4317867
– user4317867, Commented Mar 13, 2016 at 2:01

M.M. · Accepted Answer · 2016-03-13 00:06:39Z

4

There is no such thing as 'ForEach-Object -Parallel' since it is performed in the pipeline. Check Erik Meijer and Jeffrey Snover at MSDN Channel 9; those folks are gettin' nitty-gritty on the pipeline of PowerShell. It's a vid from 2008, lasts over an hour, but still... excellent on how the pipeline works.

The 'ForEach -Parallel' is a construct and not a pipeline cmdlet. It is part of the Workflow module and introduced in Windows PowerShell 3.0.

Compare:

$Items = 1..42
ForEach -Parallel ($Item in $Items) {
 $Item
}

With:

1..42 | ForEach-Object -Process { $_ }

answered Mar 13, 2016 at 0:06

M.M.

1514 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

duct_tape_coder Over a year ago

So it's not possible to use any of the other constructs from ForEach-Object such as -Begin {} then?

js2010 · Accepted Answer · 2019-10-08 00:20:44Z

As far as invoke-command goes, you would need to use the 'using' scope to access local variables:

$a = 'hi there'
invoke-command comp001 { echo $using:a }

Or to pipe something in:

1,2,3 | invoke-command comp001 { echo $input }

-argumentlist only works if there's a param defined in the scriptblock. The docs aren't good on this point.

$a = 'hi'
invoke-command comp001 { param($a) echo $a } -ArgumentList $a

Unless you use args:

invoke-command comp001 { echo $args[0] } -ArgumentList $a

Powershell 7 preview 3 actually has a multi-thread "foreach-object -parallel":

 1..5 | ForEach-Object -Parallel { sleep 10; 'done' }

Collectives™ on Stack Overflow

Possible to execute ForEach-Object commands in Parallel

2 Answers 2

1 Comment

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Related