Composable DataOps Blog
Composable DataOps Blog
Microsoft Message Queuing (MSMQ) is a robust messaging platform for reliable and low latency asynchronous communication between distributed application components. It offers exactly-once delivery, transactional coordination with databases, priority queues, and peek-lock semantics — making it well-suited for mission-critical workloads…
In order to meet the demand of incoming requests to a Composable cluster, you many need to scale out and run multiple web nodes. In this topology, a network load balancer is required to sit in front of the web…
There are many instances when working with DataFlows where you need to execute some executable or command. In this post we will explore two examples where the process executor was used to run a program not native to Composable from…
1. Create an App Registration in the Azure Portal. 2. In the App Registration, you’ll need to add Composable redirect urls. There are 2 different urls that Composable may request Microsoft Online to redirect back to. One is the Composable…
Composable now supports running Python scripts from DataLabs using the DataLab Notebook Runner Module. This is useful in production environments where you wish to run DataLab code without the overhead of spinning up a notebook. The above script and notebook…
The purpose of this Spotlight is to demonstrate the use of the DoWhile module to loop within a Composable DataFlow. Our scenario: The Fibonacci sequence is one of the best-known series in Mathematics. The series is composed of Fibonacci numbers,…
Composable has an SDK for deploying resources between two instances of Composable. For example, a source instance may be a development environment and the target instance may be the production environment. Resources can include Dataflows, QueryViews, DataPortals, Keys, WebApps, etc.…
Composable has an API for every action that you can perform through its UI. All services are documented here: Below is an example script for executing operations on the UserService. The first part of the script loads the necessary…
The purpose of this Spotlight is to demonstrate the mechanics of a simple, classic For loop within a Composable DataFlow. Our scenario: We are told to expect delivery of a plain-text dataset with the following column names: first name, last…
The WebService we’ll be creating is a simple API endpoint that accepts a Time Zone as a parameter and returns current date and time. To create a Web Service, first create a new DataFlow and add two Modules: “Web Receive”…