Can algorithms help you find love? Many happy couples successfully brought together via online dating websites show us that data science can help you find love. I'm joined this week by Thomas Levi, Senior Data Scientist at Plenty of Fish , to discuss some of his work which helps people find one another as efficiently as possible. Matchmaking is a truly non-trivial problem, and one that's dynamically changing all the time as new users join and leave the "pool of fish". This episode explores the a...
Dec 05, 2014•59 min
Economist Peter Backus put forward "The Girlfriend Equation" while working on his PhD - a probabilistic model attempting to estimate the likelihood of him finding a girlfriend. In this mini episode we explore the soundness of his model and also share some stories about how Linhda and Kyle met.
Nov 28, 2014•16 min
I'm joined this week by Alex Boklin to explore the topic of magical thinking especially in the context of Rhonda Byrne's "The Secret", and the similarities it bears to The Global Consciousness Project (GCP). The GCP puts forward the hypothesis that random number generators elicit statistically significant changes as a result of major world events.
Nov 21, 2014•42 min
What is randomness? How can we determine if some results are randomly generated or not? Why are random numbers important to us in our everyday life? These topics and more are discussed in this mini-episode on random numbers. Many readers will be vaguely familar with the idea of "X number of monkeys banging on Y number of typewriters for Z number of years" - the idea being that such a setup would produce random sequences of letters. The origin of this idea was the mathemetician Borel who was inte...
Nov 14, 2014•3 min
This week's episode explores the possibilities of extracting novel insights from the many great social web APIs available. Matthew Russell's Mining the Social Web is a fantastic exploration of the tools and methods, and we explore a few related topics. One helpful feature of the book is it's use of a Vagrant virtual machine. Using it, readers can easily reproduce the examples from the book, and there's a short video available that will walk you through setting up the Mining the Social Web virtua...
Nov 07, 2014•50 min
This episode explores the basis of why we can trust encryption. Suprisingly, a discussion of looking up a word in the dictionary (binary search) and efficiently going wine tasting (the travelling salesman problem) help introduce computational complexity as well as the P ?= NP question, which is paramount to the trustworthiness RSA encryption. With a high level foundation of computational theory, we talk about NP problems, and why prime factorization is a difficult problem, thus making it a great...
Oct 31, 2014•26 min
Jeff Stanton joins me in this episode to discuss his book An Introduction to Data Science , and some of the unique challenges and issues faced by someone doing applied data science. A challenge to any data scientist is making sure they have a good input data set and apply any necessary data munging steps before their analysis. We cover some good advise for how to approach such problems....
Oct 24, 2014•37 min
The t-test is this week's mini-episode topic. The t-test is a statistical testing procedure used to determine if the mean of two datasets differs by a statistically significant amount. We discuss how a wine manufacturer might apply a t-test to determine if the sweetness, acidity, or some other property of two separate grape vines might differ in a statistically meaningful way. Check out more details and examiles found in the show notes linked below. https://dataskeptic.com/blog/episodes/2014/t-t...
Oct 17, 2014•17 min
This week I'm joined by Karl Mamer to discuss the data behind three well known urban legends. Did a large blackout in New York and surrounding areas result in a baby boom nine months later? Do subliminal messages affect our behavior? Is placing beer alongside diapers a recipe for generating more revenue than these products in separate locations? Listen as Karl and I explore these claims.
Oct 10, 2014•48 min
The Data Skeptic Podcast is launching a contest- not one of chance, but one of skill. Listeners are encouraged to put their data science skills to good use, or if all else fails, guess! The contest works as follows. Below is some data about the cumulative number of downloads the podcast has achieved on a few given dates. Your job is to predict the date and time at which the podcast will recieve download number 27,182. Why this arbitrary number? It's as good as any other arbitrary number! Use wha...
Oct 08, 2014•12 min
A discussion about conducting US presidential election polls helps frame a converation about selection bias.
Oct 03, 2014•15 min
Commute times and BBQ invites help frame a discussion about the statistical concept of confidence intervals.
Sep 26, 2014•12 min
A discussion about getting ready in the morning, negotiating a used car purchase, and selecting the best AirBnB place to stay at help frame a conversation about the decision theoretic principal known as the Value of Information equation.
Sep 19, 2014•14 min
In this bonus episode, guest Louis Zocchi discusses his background in the gaming industry, specifically, how he became a manufacturer of dice designed to produce statistically uniform outcomes. During the show Louis mentioned a two part video listeners might enjoy: part 1 and part 2 can both be found on youtube. Kyle mentioned a robot capable of unnoticably cheating at Rock Paper Scissors / Ro Sham Bo. More details can be found here . Louis mentioned dice collector Kevin Cook whose website is Di...
Sep 17, 2014•47 min
Marick Sinay from ZestFianance is our guest this weel. This episode explores how data science techniques are applied in the financial world, specifically in assessing credit worthiness.
Sep 12, 2014•31 min
Linhda and Kyle talk about Decision Tree Learning in this miniepisode. Decision Tree Learning is the algorithmic process of trying to generate an optimal decision tree to properly classify or forecast some future unlabeled element based by following each step in the tree.
Sep 05, 2014•13 min
Our guest this week is Hamilton physics professor Kate Jones-Smith who joins us to discuss the evidence for the claim that drip paintings of Jackson Pollock contain fractal patterns. This hypothesis originates in a paper by Taylor, Micolich, and Jonas titled Fractal analysis of Pollock's drip paintings which appeared in Nature. Kate and co-author Harsh Mathur wrote a paper titled Revisiting Pollock's Drip Paintings which also appeared in Nature. A full text PDF can be found here , but lacks the ...
Aug 29, 2014•50 min
Our topic for this week is "noise" as in signal vs. noise. This is not a signal processing discussions, but rather a brief introduction to how the work noise is used to describe how much information in a dataset is useless (as opposed to useful). Also, Kyle announces having recently had the pleasure of appearing as a guest on The Conspiracy Skeptic Podcast to discussion The Bible Code. Please check out this other fine program for this and it's many other great episodes....
Aug 22, 2014•16 min
Our guest this week is Susan Gerbic. Susan is a skeptical activist involved in many activities, the one we focus on most in this episode is Guerrilla Skepticism on Wikipedia , an organization working to improve the content and citations of Wikipedia. During the episode, Kyle recommended Susan's talk a The Amazing Meeting 9 which can be found here . Some noteworthy topics mentioned during the podcast were Neil deGrasse Tyson's endorsement of the Penny for NASA project. As well as the Web of Trust...
Aug 15, 2014•1 hr 10 min
In this week's mini episode, Linhda and Kyle discuss Ant Colony Optimization - a numerical / stochastic optimization technique which models its search after the process ants employ in using random walks to find a goal (food) and then leaving a pheremone trail in their walk back to the nest. We even find some way of relating the city of San Francisco and running a restaurant into the discussion.
Aug 08, 2014•15 min
Our guest this week is Shahid Shah. Shahid is CEO at Netspective , and writes three blogs: Health Care Guy , Shahid Shah , and HitSphere - the Healthcare IT Supersite . During the program, Kyle recommended a talk from the 2014 MIT Sloan CIO Symposium entitled Transforming "Digital Silos" to "Digital Care Enterprise" which was hosted by our guest Shahid Shah . In addition to his work in Healthcare IT, he also the chairperson for Open Source Electronic Health Record Alliance , an non-profit organi...
Aug 01, 2014•57 min
This miniepisode discusses the technique called Cross Validation - a process by which one randomly divides up a dataset into numerous small partitions. Next, (typically) one is held out, and the rest are used to train some model. The hold out set can then be used to validate how good the model does at describing/predicting new data.
Jul 25, 2014
This episode features a discussion with statistics PhD student Zach Seeskin about a project he was involved in as part of the Eric and Wendy Schmidt Data Science for Social Good Summer Fellowship. The project involved exploring the relationship (if any) between streetlight outages and crime in the City of Chicago. We discuss how the data was accessed via the City of Chicago data portal, how the analysis was done, and what correlations were discovered in the data. Won't you listen and hear what w...
Jul 18, 2014•33 min
This episode loosely explores the topic of Experimental Design including hypothesis testing, the importance of statistical tests, and an everyday and business example.
Jul 11, 2014•16 min
In this week's episode, we discuss applied solutions to big data problem with big data engineer Jay Shankar. The episode explores approaches and design philosophy to solving real world big data business problems, and the exploration of the wide array of tools available.
Jul 07, 2014•50 min
In this minisode, we discuss Bayesian Updating - the process by which one can calculate the most likely hypothesis might be true given one's older / prior belief and all new evidence.
Jun 27, 2014•11 min
In the second full length episode of the podcast, we discuss the current state of personalized medicine and the advancements in genetics that have made it possible.
Jun 20, 2014•57 min
In this mini, we discuss p-values and their use in hypothesis testing, in the context of an hypothetical experiment on plant flowering, and end with a reference to the Particle Fever documentary and how statistical significance played a role.
Jun 13, 2014•17 min
A conversation with Convertro's Nathan Janos about methodologies used to help advertisers understand the affect each of their marketing efforts (print, SEM, display, skywriting, etc.) contributes to their overall return.
Jun 06, 2014•1 hr 16 min
In this first mini-episode of the Data Skeptic Podcast, we define and discuss type i and type ii errors (a.k.a. false positives and false negatives).
May 30, 2014•11 min
