As we close out 2024, we wanted to share some of the latest updates and enhancements to the Composable platform. These features, ranging from major upgrades to subtle yet powerful enhancements, are designed to streamline your processes and supercharge your DataOps productivity.
Highlights
Test Suites
Testing is a cornerstone of reliable analytics. Our new Test Suites feature allows you to define and run comprehensive tests across your DataFlows. This ensures your pipelines function as intended, helping you catch issues early and maintain consistency in your data operations.
Network View of DataFlow, Run, and Folder Relationships
Understanding complex relationships between DataFlows, Runs, and Folders is now easier than ever. The Network View visualizes these interconnections, providing a clear, interactive map of dependencies and processes. This feature makes debugging and optimization intuitive and efficient.
Interactive Table Output in DataLabs Cells
DataLabs now supports interactive table outputs, giving you the ability to explore, filter, and sort data directly within your development environment. Say goodbye to exporting data for inspection. This update puts powerful interactivity at your fingertips.
Additional Updates
New Modules for Enhanced Functionality
- Set Operations on Arrays : Perform union, intersection, and difference operations on arrays with ease.
- Speech-to-Text Modules: Simplify transcription tasks with these lightweight modules.
- JSON Key Deleter: Recursively strip unwanted fields from JSON objects, ideal for cleaning and refining data.
- Kafka Message Reader: Read specific messages from Kafka streams, enhancing your real-time data capabilities.
- Composable Resource Searcher: Quickly find and navigate to specific resources, streamlining large-scale projects.
Enhanced Views and Visualizations
- Resource Network View in TestSuite and Key Pages: View resource interdependencies directly from these pages for streamlined debugging and analysis.
- Video and Audio Rendering in DataFlow Runs: View multimedia content as part of your DataFlow results.
Improved Workflow Optimization
- Option to Compress DataFlow Run Results: Save storage space and improve performance with an easy compression option.
- Automated DataLab Log and Execution Cleaner: Maintain a tidy development environment effortlessly.
Unit Testing Framework for DataFlows
Bolster the reliability of your DataFlows with our Unit Testing Framework, providing automated validation for your pipelines. Test small components independently and ensure your logic holds up at every stage.
Looking Ahead
These updates reflect our commitment to addressing user feedback and staying ahead of industry trends. Whether you’re diving into advanced testing, leveraging interactive outputs, or exploring new modules, the latest enhancements to the Composable platform are here to elevate your workflows.
