Post data to NIFI using PowerShell, or more broadly, using powershell to post data to a URL using Rest.

Today I wanted to display how using PowerShell I can iterate through a directory and send the files in it to a NIFI instance for further processing.

I wanted to use Windows as my Operating Example and create a simple PowerShell script to accomplish this using the Invoke-RestMethod of PowerShell.

I also wanted to send headers on my post message so that I can use them within my NIFI instance to make routing and processing decisions.

For instance, I wanted to assign my data to a site ID and a FEED name. I can send those attributes within my post message and then I can build a data flow pipeline that can send data from X site ID’s down one data flow path, say HADOOP ( NIFI DOC ).

Below is my PowerShell script.

For latest copy please get it from GITHUB <<<<

Here is an example

$cert =”c:path\to\cert\file.cer”
$siteId = “020f9730-4f3e-440a-bad6-a3e2654250f4”
$feed=”logs
$uri = “URL/STRING”
$dataDirectory = “c:\path\to\your\directory”
$files = Get-ChildItem -Path $dataDirectory
foreach ($file in $files)
{
Invoke-RestMethod -uri $uri -Method POST -Header @{“site.id”=”$siteId”;”filename”=”$file”;”feed”=”$feed”} -Infile “$dataDirectory\$file”
} -Certificate $cert

Once the data is posted on NIFI it will retain the headers I passed through my post message represented as http.header.{header} as NIFI attributes on the FlowFile.

The documentation is found here: HandleHttpRequest

EXAMPLE: