Have you ever wondered how gaming companies build their analytics pipelines? Particularly scalable ones that are able to collect terabytes of data every day? At Outfit7, this is done with a little help from Google Cloud's top services, including Kubernetes, Dataflow, BigQuery, and Cloud Composer. In this presentation, you'll see how the pipeline is built, starting from ingestion in Kubernetes, through to ending in Jupyter, Tableau, and other BI dashboards. You’ll also find out how the team fights downtime with proactive monitoring and integration tests. And last but not least, you’ll hear about the challenges that Outfit7 faced when the amount of data it had to handle skyrocketed during the peak of the COVID-19 quarantine.

Data Quality is paramount -we all agree on that- and isn't straightforward even with small data sets. When working with over 120PB of data on Hadoop and thousands of jobs, I can tell you firsthand, it's a challenge! We started to tackle this at Criteo 2 years ago, and I have some tangible results I'll be happy to share. We'll go through this journey, from collecting data, detecting suspect behaviors, and alerting users on data quality incidents, while integrating these new checks into the Criteo Data Platform.

In this session you will learn about how H&M evolves reference architecture covering entire MLOps stack addressing a few common challenges in AI and Machine learning product, like development efficiency, end to end traceability, speed to production, etc. This architecture has been adapted by multiple product teams managing 100”s of models across the entire H&M value chain and enables data scientists to develop model in a highly interactive environment, enable engineers to manage large scale model training and model serving pipeline with fully traceability.
The team presenting is currently responsible for ensuring that best practices and reference architecture are implemented on all product teams to accelerate H&M groups’ data driven business decision making journey.

Machine learning-powered systems have become an essential part of most businesses. One such example are recommender systems that adapt to customer behavior to provide an organic way to make domains like clothes, books, or music explorable. In order to successfully put such systems into production, we need to bridge the gap from the raw mathematical models and algorithms to robust and scalable software systems. In this talk, we start out with core approaches to recommender systems like collaborative filtering or click probability prediction, and follow this journey to explore how theory and practice come together.

Machine Learning is an essential part to solve use cases in cybersecurity, operations and various lines of business. This talk provides you with an overview of Splunk’s big data and machine learning technologies that are used to solve real-world use cases. For example, we dive into the technical details of two selected customer examples of applied machine learning. First, we explain how a datacenter division of Siemens uses Splunk with unsupervised and supervised machine learning approaches in cybersecurity to uncover anomalies and to automate the classification of security events. We highlight the technical details of how this use case has been addressed with Splunk’s Machine Learning Toolkit. Second, we explain how BMW Group’s Innovation Lab developed a Predictive Testing Strategy with Splunk and a deep learning approach. Details are provided on how the Deep Learning Toolkit App for Splunk was used to build and evaluate a TensorFlow based model to solve that use case. We conclude the session with an outlook and a wrap up on all available technical resources.

At Nielsen Identity, we leverage Druid to provide our customers with real-time analytics tools for various use-cases, including inflight
analytics, reporting and building target audiences. The common challenge of these use-cases is counting distinct elements in real-time at scale. We’ve been using Druid to solve these problems for the past 5 years, and gained a lot of experience with it.
In this talk, we will share some of the best practices and tips we’ve gathered over the years. We will cover the following topics:
* Data modeling
* Ingestion
* Retention and deletion
* Query optimization

You will learn about a tool that can improve your big data projects on GCP. Unified structure, configuration, versioning, build, deployment, and more, available for Dataflow/Dataproc/BigQuery.

In my presentation I would like to present some approaches, good practices and Google Cloud components that we use in Sotrender to effectively train and deploy our machine learning models, which are used to analyze Social Media data. I will discuss which aspects of DevOps we focus on when developing machine learning models (MLOps), and how these ideas can be easily implemented in your company or startup using Google Cloud Platform.

During this session, we aim to review the technical and organisational challenges we faced, while building a complex AI-based app with a short time-to-market. How it was additionally influenced by the dispersion of involved teams around the world (10 time zones)? What unpredictable events that affected our plans, e.g. the COVID pandemic or the development vendor changing in the middle
of the project? We will evaluate the examples of failures and successes. Lessons we have learned in the process.  We invite you to a broader discussion.

Data is increasingly becoming core to many products. Whether to improve recommendations for users, getting insights on how they use the product or using machine learning to improve the experience. This creates a critical need for understanding how data is flowing through our systems. Data pipelines must be auditable, reliable and run on time. Tracking lineage and metadata is the underlying foundation that enables many use cases related to data. It provides understanding of dependencies between many teams consuming and producing data and how constant changes impact them. It enables governance and compliance and generally helps you keep you data running. Marquez is an open source project part of the LF AI which instruments data pipelines to collect lineage and metadata and enable those use cases. It provides context by making visible dependencies across organisations and technologies and enables lineage governance and discovery.

Successful data projects are built on solid foundations. What happens when we’re misled or unaware of what a solid foundation for data teams means? When a data team is missing or understaffed, the entire project is at risk of failure. This talk will cover the importance of a solid foundation and what management should do to fix it.

Data has to be processed fast, so that a firm can react to changing business conditions in realtime. Streaming SQL gives us a
possibility to make stream processing available for a broader audience but also makes it easier to access data streams. This
presentation will not only give you an brief overview of the data and streaming architecture at InnoGames but also introduces you
to the idea of Streaming SQL in general and how it is implemented in Apache Flink. Furthermore it shows actual examples of how
to use Flink SQL so that you hopefully are inspired to consider this rather new technology to tackle your data challenges.

Review of existing frameworks/approaches to data quality and principles behind designing data quality system at OLX.

During this session we will reflect upon the importance of personalization in e-commerce. What challenges accurred as a results of bridging the gap between Google AlphaGo and real world? Furthermore, we will discuss the Vowpal Wabbit: the Swiss army knife of ML algos.

To sum up, we will introduce a case study exercise, during which participants will be creating a personalized user experience on a webpage.

Presto, an open source distributed SQL engine, is widely recognized for its low-latency queries, high concurrency, and native ability to query multiple data sources. Proven at scale at organizations including, Airbnb, Comcast, Fcaebook, FINRA, GrubHub, LinkedIn, Lyft, Netflix, Twitter, Uber, and Zalando, Presto has experienced unprecedented growth in popularity in both on premise and cloud deployments over Object Stores, HDFS, NoSQL, and RDBMS data stores.

Delta Lake, a storage layer originally invented by Databricks and recently open sourced, brings ACID capabilities to big datasets held in Object Storage. While initially designed for Spark, Delta Lake now supports multiple query compute engines. In particular, Starburst, developed a native integration for Presto that leverages Deltaspecific performance optimizations.

Join this session and hear how Starburst Presto deployed on Azure Kubernetes Service (AKS) serves as a fast SQL query engine versus data in ADLS, and enables query-time correlations between IoT data in Delta Lake, customer data in SQL Server, and web log data in Elasticsearch.

You will also gain best practice and real-life insights and lessons learned from production deployment of this integration.

At Simply Business we have built a rich stateful application using Kafka Streams to manage the leads that will be served by our 300-people strong UK call centre. This application combines X different data points from different services and has many internal data stores to operate and has been in production for over 2 years. Thanks to the increased observability and the ability to ingest lead scoring data from another streaming ML project, we have been able to increase the return of each call by over 60%.

Weather forecasting is important in many fields, and minor improvements in accuracy can have a considerable business impact. Today, weather forecasting is performed using computationally expensive mathematical models based on the Navier-Stokes and mass continuity equations, the first law of thermodynamics, and the ideal gas law. These models simulate the physical world and are making use of expensive supercomputers. Alternatively, AI can be used to learn from data and produce forecasts in a fraction of the time the physical simulations require, and in many cases at a higher degree of accuracy. In this presentation, I will show how an AI was used to produce competitive forecasts, using state-of-the-art AI models and neural architecture search, and how I used React to prototype a weather forecasting service.

Digital Transformation takes a lot of our processes to new levels. Especially, since the eruption of the pandemic, we need to look for new solutions for old problems. For example, if we open for remote IT specialists, we can easily get 10x-100x more candidate, while having the same recruitment procedures.
During my presentation, I would like to share ideas of how AI-driven tools are and can be used in HR and Talent Management:
- Current possibilities of what is feasible (including some demos),
- Where we can take these tools in the forseeable future,
- What are the current technical challenges,
- Impact challenges, especially from ethical standpoint.

Analytical data processing has become the cornerstone of today's businesses success, and it is facilitated by Big Data platforms that offer virtually limitless scalability. However, minimizing the total cost of ownership (TCO) for the infrastructure can be challenging. By analyzing spot instance price history using ARIMA models, it is feasible to leverage the discounted prices of the cloud spot market with a limited risk of analytical job termination. In particular, we evaluated savings opportunities when using Amazon EC2 spot instances comparing to on-demand resources. During the presentation we show the evaluation of univariate spot price regression models to forecast future prices, and we confirm feasibility of short-term spot price prediction through real data from AWS. This confirms cost savings opportunities up to 80% compared to on-demand and within 1% of the absolute minimum.

We would like to invite you to discuss how Bolt migrated from batch and synchronous to real-time and asynchronous. During our session we will review and evaluate the obstacles we faced along the way and lessons we have learned. We will also focus on the unleashed value of real-time data.

We would like to invite you to discuss how Bolt migrated from batch and synchronous to real-time and asynchronous. During our session we will review and evaluate the obstacles we faced along the way and lessons we have learned. We will also focus on the unleashed value of real-time data.

Unity Ads helps publishers and advertisers reach their business goals, and machine learning is at the core of our product. In this presentation, I will first give an overview of the machine learning systems we built for real-time ads bidding, which process tens of thousands of ad auction requests per second. Then, I will share several generalizable lessons we learned in making our systems performant from machine learning perspective and scalable from engineering perspective.