Composable DataOps Blog
Composable DataOps Blog
As organizations scale their usage of Databricks, they often encounter challenges related to governance and management of the various components within the environment. In this blog post, we will explore the importance of implementing a monitoring tool to maintain control…
Organizations are constantly seeking new ways to access and leverage valuable data. One emerging solution that has gained significant attention is the concept of data marketplaces. Data marketplaces provide a platform for securely sharing data, enabling organizations to tap into…
Composable has an SDK for deploying resources between two instances of Composable. For example, a source instance may be a development environment and the target instance may be the production environment. Resources can include Dataflows, QueryViews, DataPortals, Keys, WebApps, etc.…
Composable has an API for every action that you can perform through its UI. All services are documented here: Below is an example script for executing operations on the UserService. The first part of the script loads the necessary…
The WebService we’ll be creating is a simple API endpoint that accepts a Time Zone as a parameter and returns current date and time. To create a Web Service, first create a new DataFlow and add two modules: “Web Receive”…
Composable’s DataFlow designer is a powerful tool for rapid development of data pipelines in an intuitive, visual setting. However, some users with programming experience may prefer developing DataFlows in code instead of visually, which is made possible by Composable’s FluentAPI.…
Tutorial “Effective October 1, 2022, we will begin to permanently disable Basic Auth in all tenants, regardless of usage, with the exception of SMTP Auth.” Due to the recent deprecation of both Exchange Online and support for Basic Authentication,…
What is a Predicate? A predicate is an operation on an Input object which produces a boolean (true/false) result. It can be helpful to think of a predicates as “Yes or No” questions; “Is this car red?” “Is this building…
Have you ever received the error System.IO.PathTooLongException: The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.? If so,…
In this post, we’ll be going over how to configure an ODBC connection to DataBricks within Composable so you can run spark queries from Composable resources (i.e. QueryViews and DataFlows). The first thing you’ll need to do is install the…