Statistics of runs

Ylem is a full-cycle data streaming software, therefore not only does it help manage and execute pipelines but also collects detailed statistics of runs for each pipeline and task.

What information is collected:

• Starting time

• Ending time

• Duration in milliseconds

• Task ID and type

• Task output

• Success or failure

Having this information for each of the pipelines Ylem provides users with multiple different ways of working with the statistics:

• A main dashboard that contains accumulated statistics about all pipelines and metrics

• Log of runs for every pipeline and tasks

• Aggregated statistics for every pipeline and task within the selected date range

• Powerful API to pull statistics in different forms that can be integrated with your BI software if you want to use these historical data for your reports and dashboards

• Slow task profiler that allows you to have constant control over your bottlenecks

Task output example in the list of logs

Dashboard graph representing the number of new pipelines per month