Food deliveries, Bayes and Computational Statistics

Marton Trencseni - Sat 22 June 2019 • Tagged with python, math, fetchr

I was grabbing a burger at Shake Shack, Mall of the Emirates in Dubai, when I noticed this notebook on the counter. The staff is using it to track food deliveries and each service (Carriage, Talabat, UberEats, Deliveroo) has its own column with the order numbers. Let's assume this is the only page for the day, and ask ourselves: given this data, what is the probability that UberEats is the most popular food delivery service?.

Shake shack food deliveries

Continue reading

The Collatz conjecture

Marton Trencseni - Sun 02 June 2019 • Tagged with python, math

The Collatz conjecture is a conjecture in mathematics that concerns a sequence defined as follows: start with any positive integer n. Then each term is obtained from the previous term as follows: if the previous term is even, the next term is one half the previous term. If the previous term is odd, the next term is 3 times the previous term plus 1. The conjecture is that no matter what value of n, the sequence will always reach 1.

Collatz

Continue reading

MNIST pixel attacks with Pytorch

Marton Trencseni - Sat 01 June 2019 • Tagged with python, pytorch, cnn, torchvision, mnist, skl

It’s easy to build a CNN that does well on MNIST digit classification. How easy is it to break it, to distort the images and cause the model to misclassify?

MNIST attack accuracy

Continue reading

Solving CIFAR-10 with Pytorch and SKL

Marton Trencseni - Tue 14 May 2019 • Tagged with python, pytorch, cnn, torchvision, cifar, skl

CIFAR-10 is a classic image recognition problem, consisting of 60,000 32x32 pixel RGB images (50,000 for training and 10,000 for testing) in 10 categories: plane, car, bird, cat, deer, dog, frog, horse, ship, truck. Convolutional Neural Networks (CNN) do really well on CIFAR-10, achieving 99%+ accuracy. The Pytorch distribution includes an example CNN for solving CIFAR-10, at 45% accuracy. I will use that and merge it with a Tensorflow example implementation to achieve 75%. We use torchvision to avoid downloading and data wrangling the datasets. Like in the MNIST example, I use Scikit-Learn to calculate goodness metrics and plots.

CIFAR examples

Continue reading

Solving MNIST with Pytorch and SKL

Marton Trencseni - Thu 02 May 2019 • Tagged with python, pytorch, cnn, torchvision, mnist, skl

MNIST is a classic image recognition problem, specifically digit recognition. It contains 70,000 28x28 pixel grayscale images of hand-written, labeled images, 60,000 for training and 10,000 for testing. Convolutional Neural Networks (CNN) do really well on MNIST, achieving 99%+ accuracy. The Pytorch distribution includes a 4-layer CNN for solving MNIST. Here I will unpack and go through this example. We use torchvision to avoid downloading and data wrangling the datasets. Finally, instead of calculating performance metrics of the model by hand, I will extract results in a format so we can use SciKit-Learn's rich library of metrics.

MNIST example digits

Continue reading

SVM with Pytorch

Marton Trencseni - Tue 16 April 2019 • Tagged with pytorch, svm, iris

I use the standard Iris dataset for supervised learning with a Support Vector Machine model using Pytorch's autograd.

SVM

Continue reading

Hacker News Embeddings with PyTorch

Marton Trencseni - Tue 12 March 2019 • Tagged with pytorch, embedding

A PyTorch model is trained on public Hacker News data, embedding posts and comments into a high-dimensional vector space, using the mean squared error (MSE) of dot products as the loss function. The resulting model is reasonably good at finding similar posts and recommending posts for users.

Vector space

Continue reading

rxe: literate and composable regular expressions

Marton Trencseni - Sat 02 March 2019 • Tagged with python

rxe is a thin wrapper around Python's re module. The various rxe functions are wrappers around corresponding re patterns. For example, rxe.digit().one_or_more('a').whitespace() corresponds to \da+\s. Because rxe uses parentheses but wants to avoid unnamed groups, the internal (equivalent) representation is actually \d(?:a)+\s. This pattern can always be retrieved with get_pattern().

rxe example code

Continue reading

PyTorch Basics: Solving the Ax=b matrix equation with gradient descent

Marton Trencseni - Fri 08 February 2019 • Tagged with pytorch

I will show how to solve the standard A x = b matrix equation with PyTorch. This is a good toy problem to show some guts of the framework without involving neural networks.

PyTorch computational graph

Continue reading

Automating a Call Center with Machine Learning

Marton Trencseni - Sun 27 January 2019 • Tagged with fetchr, machine-learning, call-center

Over a period of 6 months, we rolled out a Machine Learning model to predict a customer’s delivery (latitude, longitude). During the recent holiday peak, this ML model handled most of Fetchr’s order scheduling.

Share of ML scheduled versus Call center scheduled deliveries

Continue reading

5 things that happened in Data Science in 2018

Marton Trencseni - Wed 09 January 2019 • Tagged with data, openai, waymo, deepmind, tesla, reinforce

2018 was a hot year for Data Science and AI. Here we picked out 5 highlights, which in our opinion shaped the field in the past year.

Deepmind playing CTF

Continue reading

Warehouse locations with k-means

Marton Trencseni - Wed 26 September 2018 • Tagged with data, data-science, metrics, fetchr

Sometimes, the seven gods of data science, Pascal, Gauss, Bayes, Poisson, Markov, Shannon and Fisher, all wake up in a good mood, and things just work out. Recently we had such an occurence at Fetchr, when the Operational Excellence team posed the following question: if we could pick our Saudi warehouse locations, where would be put them? What is the ideal number of warehouses, and, what does ideal even mean? Also, what should our “delivery radius” be?

.

Continue reading

Growth Accounting and Backtraced Growth Accounting

Marton Trencseni - Sun 16 September 2018 • Tagged with data, data-science, metrics, growth-accounting, fetchr

Previously I wrote two articles about data infra and data engineering at Fetchr. This time I want to move up the stack and talk about a simple piece of metrics engineering that proved to be very impactful: Growth Accounting and Backtraced Growth Accounting.

Backtraced Growth Accounting

Continue reading

Fetchr Data Science Infra at 1 year

Marton Trencseni - Tue 14 August 2018 • Tagged with data, etl, workflow, airflow, fetchr, model, ml

A description of our Analytics+ML cluster running on AWS, using Presto, Airflow and Superset.

Fetchr Data Science Infra

Continue reading

What not to spend time on

Marton Trencseni - Mon 23 July 2018 • Tagged with warren, buffett, self, help, physics, haskell

Warren Buffett says deciding what not to spend time on is just as important as deciding what to spend time on.

Warren Buffett

Continue reading

Beat the averages

Marton Trencseni - Sat 07 July 2018 • Tagged with statistics, data

When working with averages, we have to be careful. There are pitfalls lurking to pollute our statistics and results reported.

Probability distribution

Continue reading

Building the Fetchr Data Science Infra on AWS with Presto and Airflow

Marton Trencseni - Wed 14 March 2018 • Tagged with data, etl, workflow, airflow, fetchr

We used Hive/Presto on AWS together with Airflow to rapidly build out the Data Science Infrastructure at Fetchr in less than 6 months.

Warehouse DAG

Continue reading

Don’t build cockpits, become a coach

Marton Trencseni - Wed 09 November 2016 • Tagged with data, science, product, analytics

I used to think that a good analogy for using data is the instrumentation of a cockpit in an airliner. Lots of instruments, and if they fail, the pilot can’t fly the plane and bad things happen. There’s no autopilot for companies. The problem with this analogy is that planes aren’t built in mid-air. Product teams and companies constantly need to build and ship new products.

A big complicated cockpit

Continue reading

Beautiful A/B testing

Marton Trencseni - Sun 05 June 2016 • Tagged with ab-testing, strata, statistics, data

I gave this talk at the O’Reilly Strata Conference London in 2016 June, mostly based on what I learned at Prezi from 2012-2016.

14. slide

Continue reading

Hack, HHVM and avoiding the Second-system effect

Marton Trencseni - Sat 14 May 2016 • Tagged with books, programming, hhvm, brooks

I read this book on my first vacation after I started working at Facebook and thus became a semi-regular Hack/HHVM user. I highly recommend reading (parts of) it. But not to learn Hack/PHP, which is irrelevant to most people. Instead, it’s to learn about how Facebook improved it’s www codebase and performance without rewriting the old PHP code in one big effort, and thus avoided the famous Second-system effect.

Hack book

Continue reading