From the latest news to upcoming events and interesting topics, “In the Pipeline” is overflowing with interesting updates for the Kedro community.
This month: a roundup of June’s Kedro news, some release updates, and our top picks from recent articles.
Kedro news
June was an exciting month for Kedro as we celebrated four-years as an open-source project. In a blog post, we looked back to the launch in June 2019 and reflected on the milestones we have passed since then. Part of our celebration unveiled Kedro's new branding, with changes to the identity that align it with our vision of simplicity and collaboration.
Besides looking back over our achievements to date, we also looked forward, with the news that Kedro will sit at the heart of the QuantumBlack Horizon, an integrated and flexible suite of AI development tools announced in June 2023.
We want to express our gratitude to the community, contributors, and users who have supported and shaped our software over the last few years. As we enter a dynamic and exciting period in the field of data science and machine learning, we can together embrace the future and strive to innovate and create lasting impact.
Release news
There have been a few releases since the last issue of In the Pipeline.
We’re working to a new release cadence so Kedro versions 0.18.9, 0.18.10 and 0.18.11 have all shipped in recent weeks.
The headline news is that we have reworked the micropackaging workflow to use standard Python packaging practices. We made a batch of changes for Kedro’s rebranding and some additions for OmegaConfigLoader (such as support for variable interpolation in the catalog and some bugfixes). In those new releases, we also added substantial improvements to our deployment documentation. In particular, we’ve added a brand new set of pages about working with Kedro and Databricks.
Mirroring the cadence of Kedro framework, we’ve also released released a number of updates to Kedro-Viz; versions 6.2.0, 6.3.0, 6.3.1 and 6.3.2 have been released.
The headline news in Kedro-Viz 6.2.0 was the addition of collaborative experiment tracking so multiple users can store their experiment data in centralised remote storage and share it. For more details, check out the documentation and blog post. Kedro-Viz 6.3.0 introduced the new Kedro brand, support for a new way to configure data layers, and data preview. The later minor releases have fixed bugs and made small improvements.
Finally, kedro-datasets versions 1.3.0, 1.4.0, 1.4.1 and 1.4.2 were also shipped in recent weeks. We introduced a SparkStreamingDataSet and a ManagedTableDataSet for managed delta tables on Databricks, plus pandas 2.0 support, and SQLAlchemy 2.0 support (dropping support for versions below 1.4). We also fixed some bugs and tidied up some docstrings. Check out the full description available in the release notes and the list of datasets in our documentation. It's worth pointing out that kedro-datasets is a separate PyPI package for Kedro datasets and that kedro.extras.datasets is deprecated (and will be removed in Kedro 0.19 when that release arrives later in 2023).
Recently on the Kedro blog
Recently published on the Kedro blog:
We’re always looking for collaborators to write about their experiences using Kedro. Get in touch with us on our Slack workspace to tell us your story!
What we’ve learned
We asked on one of our community Slack channels where Kedroids get their news and updates. Some of the useful resources shared include these newsletters:
The Gradient — collects articles, interviews, and news coverage about AI.
The Sequence — a weekly digest about developments in machine learning, artificial intelligence, and data science.
We also revisited some blog posts about Kedro as we updated our fork of the awesome-kedro GitHub repo to collect articles, videos and podcasts about Kedro. We liked “Deploying a recommendation system the Kedro way” and “From concept to production in 2 months: sales forecasting Machine Learning model for dema.ai” by GetinData | Part of Xebia.
If you have an article about Kedro that should be on the awesome-kedro list, please do add it!
Coming up soon
Over the next few weeks, we hope to take delivery of a big bundle of Kedro swag that showcases our new brand. Look out for opportunities to collect stickers, t-shirts and more! We also expect to have news of upcoming Kedro releases and some virtual meetings in the coming month.
That’s it for this edition
And that’s a wrap for this month. Make sure to bookmark this blog or add our RSS feed to your favorite reader to stay in the loop and join us in the first week of August for another update from the Kedro team.
If you can’t wait for next month’s In the Pipeline news, we also toot out regular updates onto Mastodon (https://social.lfx.dev/@kedro) and across the popular channels of the Slack community. Let us know on Slack if there's something you'd like us to feature on this blog or newsletter!
