Cloud Machine LearningJuly 29th, 2016 by Matthew Ross
Part 2: Cloud Machine Learning
Over the course of three weeks, Energyworx will take its readers on a journey to better understand Machine Learning and what it means for the Energy Industry. Part 1 provided the basics and now we turn to the best in Cloud Machine Learning with Google.
Energyworx strongly believes in Pure Play Cloud Computing. We also believe that we should leverage the best products available on the market to strengthen our Software as a Service. This has been the case with many Google Cloud products including Cloud Dataflow, BigQuery and BigTable which helps us stream, store, process and analyze large volumes of data. This means Energyworx can focus on tools and use cases for the energy industry rather than developing methods for managing big data. Our customers thus experience best in class energy data management and a faster time to market for their new business models.
This same approach is applied to Machine Learning as Google has been a leader in the field for many years applying ML to Search, YouTube, Play and many other Google products.
Cloud ML & The Energyworx Datalab
Cloud ML provides pre-trained models and a platform to generate your own tailored models – so no need to start from scratch. Cloud ML also seamlessly integrates with other Google Cloud products.
The Energyworx Datalab environment is an amazing resource for our data scientists as they can create and analyze models using the familiar Jupyter notebook. The Energyworx Datalab is built on top of Jupyter but heavily customized in a multi-tenant environment, running on cloud allowing large scale usage. We integrated our API with Energyworx Datalab to create our own custom notebook that can be accessed by authorized users. We also developed sample libraries for analyzing energy data and improved the underlying code for faster processing.
One of the best features is that we allow you to run your models directly on our Energyworx cloud environment, using the same data that is accessible for your business applications, instantly and running at large scale. So developing a model with training sets and sequentially running it from a cloud environment on millions of datasets is not a problem – the Energyworx SaaS does the job for you. Once the right results are achieved, you can simply store the results in Energyworx and make it available for your business applications. Do you want to run and apply your models daily (or even streaming) on your ingested data? Just enable this on a single or group of datasources and the Energyworx system will do this automatically.
Now our customers can use our very own Exploratory Data Analysis (EDA) product that provides them with a user-friendly environment to load data from our platform, visualize and apply analytics. The new (ML) algorithms developed in Datalab can then be implemented in the production environment and applied to business applications. All of this is free and part of the package for our pay-per-use customers.
TensorFlow – open source software library for numerical computation
Energyworx Datalabs is great for developing ML algorithms but how do you train the models at scale? TensorFlow which was developed by the Google Brain Team is a very promising ML resource that was recently open-sourced. TensorFlow is much more applicable to the Energy Sector (and time-series data) than many of the other Google ML tools devoted to imagery, speech and text.
TensorFlow uses data flow graphs which describe mathematical computations with nodes and edges. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API. This design facilitates the transition from research prototype to production system.
We’re excited about TensorFlow because of its flexibility for creating data flows, language options and the ability to integrate with our API and production platform.
Part 3 will explore specific use cases for the Energy Sector and where we believe Machine Learning can make the biggest impact. For a sneak preview, check out the presentation we delivered for the Internet of Energy Virtual Summit 2016. See you next week!