<iframe src="//www.googletagmanager.com/ns.html?id=GTM-MXN9JJ" height="0" width="0" style="display:none;visibility:hidden">

The Smaato Blog

The Smaato Blog

Dr. Stefan Shadwinkel

Stefan works for Smaato's development team as a Senior Big Data Developer. He has extensive experience in big data analytics as well as in the field of fraud prevention.
Find me on:

Recent Posts

Spark on Docker on Amazon EC2: Only the Code Tells You Everything

Posted by Dr. Stefan Shadwinkel on November 13, 2015

Our global real-time advertising platform processes vast amounts of data per second. Therefore managing, supporting, and enhancing all its tools and processes with data-driven solutions is crucial to our success.

Developing these solution requires a flexible setup that can also be easily scaled to allow testing on reasonable data sizes. One part in our current setup is to run Apache Spark on Docker on Amazon EC2 instances.

Using straight EC2 instances instead of EMR has the benefits of lower costs and being able to directly run the latest version or development builds of Spark.

In this blog post, we will look into the peculiarities of configuring Spark on Docker on EC2 and dive into some Spark code excerpts to understand Spark's behavior.

Read more »

Big Data & NoSQL Meetup Hamburg with Apache Flink at Smaato

Posted by Dr. Stefan Shadwinkel on July 17, 2015

Smaato was very happy to host the spring to summer edition of the Big Data and NoSQL Hamburg (BDNSHH) meetup with two great guests from Berlin: Aljoscha Krettek and Maximilian Michels from dataArtisans, the company behind Apache Flink.

Apache Flink is an open source platform for scalable batch and stream data processing. At its core is a streaming dataflow engine that provides data distribution, communication, and fault tolerance for distributed computations over data streams. Interesting features are its custom dataflow optimizer, custom memory management, and its strategies to perform well when memory runs out.

We’ve interviewed our guests to dig deeper into Apache Flink:

Read more »