There are many instances when working with DataFlows where you need to execute some executable or command. In this post we will explore two examples where the process executor was used to run a program not native to Composable from a DataFlow.
Building a Web App
The first task we will look is at building an automated pipeline for building an AngularJS web app. Our goal here is to run the gulp prod command in the same directory as our web app code.
To achieve this we can use the Process Executor. This module will allow us to spawn a command prompt and run a command in some working directory. The first input we will focus on is the FileName. This input specifies the executable that we want to run. In our case this is the command prompt which can be found at the file path seen above. The next input is Arguments. This specifies that will be passed to the executable when it is started. In our case we will use the /C parameter and the gulp command we want to execute. Here we are fully specifying the path to gulp although you could also just as easily add gulp to the path on your server and not need the fully qualified path. We use the /C argument here so that cmd.exe will run the command and then terminate. Without this argument the process executor would hang for the period of time specified in the WaitTimeMillis input (or forever if the input was -1) as cmd.exe would be waiting for further input. Here we require an admin Windows credential because of the location of the repository, however this may not always be required. Lastly we need the working directory. In another part of this DataFlow we are pulling down the web app from a Git repo. We pass this directory into the working directory input of the module so that we can build this web app.
While setting up a DataFlow you might ask yourself how you can follow what the external process is doing. One nice feature of the process executor is that all log messages are piped directly into the trace log of the DataFlow so that you can follow along in real-time.
Executing Go Code From A DataFlow
The second task that we will look at is execution of a program written in Go from a DataFlow. Recently we had a problem that required some complex logic which was difficult to model in a functional way and therefore did not fit well into the DataFlow paradigm. We chose to solve this problem using Go but still needed to be able to run this code from a DataFlow. As was the case with building web apps, the process executor is perfect for solving this problem.
You can see here that the paradigm is similar to the example above. Here we are running go.exe with arguments specified through the string formatter and specify the working directory as the directory which contains our Go code.
Executing A DataFlow From Go Code
In the above DataFlow, you may notice the External DataFlow Input here. Within the Go program we have provisions for calling a web activated DataFlow.
The External DataFlow Input will give us the id of the DataFlow to run. In this program we have already loaded an auth token that is stored in a .env file and set authToken on ComposableCaller to the value, so we can just add the Authorization header to web request and then run the request. Finally we check that the request did not fail and there were no errors.
If you are replicating this project for yourself, you can get your own auth token by visiting your user’s profile, selecting the “Generate API Token” button, giving your token a name, and selecting “Generate.”
As you can see with these two examples, there are so many ways to extend DataFlows and integrate them with many different applications whether the integrations are pre-built into Composable or not.
