65
What has become more apparent to me as the years have passed is that the thing missing from it is the ability to convey a finding, or relevant information once an analysis is complete, to a non-technical audience. A large amount of the hard work that most data scientists do is not necessarily all data wrangling and modeling and coding. Instead, once you have a result, it’s about figuring out how to explain that result to people who are not necessarily technical or who are either making business decisions or making engineering decisions.
Really, it’s all about conveying a finding. You can use words to do that, you can use visualization to do that, or you can develop a presentation to do it. A well- rounded data science team will have someone who is very competent at this. If your organization is making decisions
based on your analysis, you need to be sure they understand why.
This echoes parts of what we’ve heard when we talked with Hilary Mason and Mike Dewar. Both of them emphasized the storytelling part and how to carefully communicate the analysis part.
It’s something that receives the least amount of thought, but turns out to be one of the most important things once you’re doing this in the wild. Even the people who have had success in data science up to this point have just been naturally good at it, whether they were blogging about it or giving good presentations. Both Mike and Hilary are examples of people who are good at doing that. They are naturally good at it. People who are not naturally good at it can learn about it through coaching, and mentorship.
In just the same way, if you’re not a good coder you can become a better coder through coaching and mentorship.
You said on a Strata panel: “Human problems won’t be solved by root mean square error.” What did you mean by that?
I think when people think about data science, or even machine learning applied to data science, people think that we have a well-defined problem, and we have our data set. We need to find a way of taking that problem and that data set and producing an answer that is better than the one that we currently have.
66
For example, Kaggle does a really good job of finding a problem definition, finding the data set, saying that it’s connected to that problem and ramping up to a competition. That way people can try and achieve a very specific thing such as having a better predictor, or having a better classifier so your errors are small.
But the really hard problems are ones for which we don’t have good well-defined definitions for yet. Or we recognize the problem but it’s not obvious how to find the relevant data that goes with it. Those are really hard problems to me. I’m a social scientist by training, so I think about how human behavior could be observed, what it is that I want to learn about institutions or policies or government and interventions to help keep a lid on our lives.
Those problems are very hard to model. So they require more creative thinking.
Particularly at first, or at the onset where you have no idea if there’s even any relevant data out there. You might have to go on and run an experiment, run a data collection experiment. Then try from there. “Ok, what are the models and methods that might work in this context?” At the end, you’re going to spend a lot more time thinking, “Alright, what are the intended and unintended consequences that might result by implementing my idea?”
Take New York City, for example. Let’s say you wanted to optimize the snow removal routes in New York City when there’s a snowstorm. Those who were in New York when there was a big snowstorm might remember — there were a lot of people who complained because the snow ploughs couldn’t get to certain neighborhoods fast enough.
So technically it’s probably a pretty easy problem to solve. It’s like a rough optimization problem. You could do that. But if you take a snow plough that’s expected to be in one place and reroute it to another place, the people who live in that block will have a negative effect on optimization. Or at least there will be a perceived negative effect.
This is a long-winded answer, but it’s much easier if you’re only thinking about minimizing error. If you have a broader perspective on how your application or your problem or the solution to it actually impacts people, it becomes harder and therefore much more interesting and useful to the discipline of data science.
How have you found working at the intersection of social science and data science?
What are the problems that you’ve really chewed on and how did you come to arrive at those sorts of problems?
For me, it started where you are, in my undergraduate times. I was a computer science student but I went to a liberal arts college so I got to take lots of other classes. I always
DREW CONWAY
67
found questions that were being asked in my political science or sociology classes to be the ones I was really interested in: “How do groups of people make choices? How do markets move? Why is one group of people making different choices than another group of people? What motivates people to do bad things? What motivates people to do good things?” These sorts of questions were much more interesting to me at the time than writing a faster compiler or a different programming language.
At that stage I actually ended up double majoring in Computer Science and Political Science so I had to write two theses. My political science thesis was back in 2004. Keep in mind that when I went to college, 9/11 was a big part of my experience. So I became really interested in terrorism and terrorist groups. I was reading trying to learn more about it. At the time peer-to-peer file sharing networks were still prominent. I was reading about how those file-sharing networks were used and the way data went through them and I observed that they were structured in very similar ways to nefarious networks or terrorist networks. I wrote my thesis on mirrors between these two things. There are weaknesses in the file-sharing network. If it was possible to replicate those weaknesses in a human network, maybe you could exploit the same weaknesses that people use to try to intercept communications on a file-sharing network.
I actually got invited to present that paper at West Point when I was a senior. This set me on the first part of my career path. I started my career in the intelligence community and there were people at the conference from various intelligence agencies who were really interested in the idea that you could model human behavior in the same way you model computer traffic.
Part of it for me was that I felt a connection to the 9/11 event and I was interested in learning more about why people would do that. So between the knowledge that I had learned in Computer Science and my interest in Social Sciences, I landed a job as a computational social scientist working inside the intelligence community. The problems I was working on there were exactly an extension of the work that got me there: understanding networks, working out how people make choices in non command- and-control structures.
Ever since then, I’ve always been fascinated by computer science, math, and statistics as a tool belt. I find these technical things really interesting to apply to human problems.
DREW CONWAY
If you have a broader perspective on how your application or your problem or the solution to it actually impacts people, it becomes harder and therefore much more interesting and useful to the discipline of data science.
68
I’m not working in the intelligence community any more, but since then, I’ve worked on my Ph.D. and have done research in the space, and have even started an organization like Data Kind, which tries to scope out the intersection of where the human problems are, where the technical talent is and then put them together. And now at Project Florida, I’ve always wanted to do take these learnings and apply them within the sensor market and with healthcare. It’s always been the classic problem that’s excited and motivated me.
How is it that you were able to come straight out of undergrad and begin working in this domain?
For me, I’m not sure my career path is one that I would recommend for other people. I loved my career, I can’t complain about any step. But we’ll call it an outlier situation.
I was working with a lot of “reformed” academics. The people who were mentors to me had been professors at big research universities and it was very multidisciplinary.
I had colleagues and bosses who were PhDs in math, computer science, economics and sociology. I was working with a large group of really smart people.
I started my career as a very junior analyst. The way that DC works, in a sense, is that in order to reach the next “level” you have to have at least a Masters degree. Well, I got to that point around 2007, so I was thinking about what I wanted to do. I was reaching out to my colleagues and mentors for advice. They sat me down and they said I had two choices: “You can do the typical DC thing which is to go to night school, get your Masters degree and then do the next thing. Or you could think about becoming a professional researcher. Go back to school full-time, see if you’re interested and do a doctorate.”
For me they were saying, “We know you, we know what you like doing. You should really consider the Ph.D. because we think it would be good for you.”
To be honest, I didn’t really want to do it. It’s such a huge opportunity cost. If I did it, that was five years I could have been making money and building a career. However, on their counsel I started looking around at some programs. I knew I didn’t want to go back to school for a computer science degree or a math degree, because I definitely wasn’t the greatest computer scientist or mathematician that ever lived. And also, at the end those are not the problems I want to solve. If you’re going to do a PhD., you have to contribute back to the discipline. I wasn’t interested in contributing back to those two disciplines, so I thought about various Political Science programs. I wanted to find one that was very quantitative. I ended up at NYU Political Science, which was one of three or four political science departments in the world that was heavily quantitative right from the start.
It was also in New York.
DREW CONWAY
69
I felt that being in New York and in a large urban area opened up a lot of different things and wouldn’t limit me to focusing specifically on my academic endeavor. I could be exposed to other things while I was there.
I also decided that I wanted to talk more publicly about the work I was doing. Part of this is colored by the fact that for years, by being in the intelligence community, I couldn’t talk at all about the work I was doing. So moving on from there, I was really eager to start blogging or going to the media to talk about the work.
As soon as I got to graduate school, I started doing those things. That helped balance the work I was doing as a graduate student with running the Meetup in New York, giving talks, advising start-ups and getting involved. That doubled my work but it was all fun work and I really loved it.
The decision to go back to school was basically, “Well, I think this would be good for my career.” I didn’t even really know if I wanted to be a professor. It was something I was interested in, but I knew if I was going to become a professor I was going to be the kind of professor that had one-and-a-half foot in the university, and the other half foot out doing stuff.
From my experience at graduate school I decided I definitely didn’t want to be a professor.
My father was a professor so I’m sort of a university brat. I know the lifestyle is fantastic
— there’s nothing wrong with it. However, the realm of a university is teaching and publishing and not building software or data science.
Given that you had the experience of working in industry before going back to graduate school, do you feel that you had a significantly different perspective?
Were you looking at the academic problems you were facing in grad school differently because you’ve had a chance to dig your teeth into them already in the
“real world”?
One thing I always say, and I tell this to people all the time, is that I highly recommend not going directly from undergraduate to graduate school. Even if it’s just to work for a year, I think it provides you so much more insight and experience in the kind of problems that are interesting to industry versus the problems that are interesting to researchers.
My early industry experience was unique in that the work I was doing in the intelligence community was split between two halves. One half was the classic intelligence aspect:
studying people for short-term projects that have to be turned around in a very narrow time window.
DREW CONWAY
70
I highly recommend not going directly from undergraduate to graduate school.
The other half of my job looked much more academic. These were long-term research projects; we were working with specific agencies that had the capacity to do high-risk research. Through that experience I decided that I really enjoyed and was interested in solving hard problems. One of the problems we worked on is how to enable non command-and-control structures (e.g., organizations without coherent org charts) to make choices.
For example, in a command-and-control organization like the army, if you’re Lieutenant Colonel and you’re promoted to full Colonel, everybody understands how that works. However, when you’re in a non command-and-control structure, different
people in each part of the network have different responsibilities. One does fundraising, one does surveillance, and one does operations. Suddenly there’s a person from the operations cell who gets captured; how does that operations cell make a choice about who will become the new leader? Or does someone get taken from another cell and worked through the system that way?
We’ve thought a lot about how to solve that problem and we didn’t solve it at all.
However, I got really excited about the thought of solving longer-term problems. So I had another reason to go to graduate school. There was a lot of freedom to think about solving problems that I found interesting.
I think the basic difference there is in industry is that it’s about always solving someone else’s problem for them. Now, that’s not an absolute truth, but certainly when you’re starting your career you’re almost always solving someone else’s problem. Then when you get to graduate school, you get to think of those problems on your own. The issue is sometimes those problems are really boring or they’re not interesting because you don’t have enough experience or enough knowledge to recognize good problems. That’s where mentorship as a graduate student becomes important.
If you’re going to go to graduate school, you’ve got to trust in and work really hard with your advisor because if you don’t, you’re probably going to produce bad research. It’s way easier to produce bad research than it is to produce good research. In industry the objective function is set by someone else; that objective function typically is profit and the problems are usually smaller and more attainable.
So if you have experience on either side of that, you can be more reflective about how it might be on the other side. I think that there is a certain strictness versus a freedom component and there’s positives and negatives on both sides. It’s really about what
DREW CONWAY
71
motivates the person doing the work, what kind of stuff you like to do, and how you see your own self-worth measured in terms of what you’re contributing. Because in either case, you’re never truly independent. That’s a fallacy that’s built in at graduate school.
In reality, you’re the furthest possible from being autonomous during graduate school.
You’re certainly more autonomous than you would be working for a big company on a team. But you have many masters as a graduate student, the least of which is yourself, and you have to be really good at maintaining your own schedule and solving a problem on your own.
Your book “Machine Learning for Hackers” is in the canon of data science now.
Given that, can we talk about the tools that you have found to be useful in your career and also while doing data science? How do you discover useful tools for data science work?
Personally, I am not as much a lover of languages as some computer scientists are. Have you ever heard of the Strange Loop Conference in St Louis? It’s in St Louis every year; it’s a fantastic conference and I highly recommend it. But it’s for people who love tools and love programming. So I went there and was doing an introduction to machine learning programming. I found I was very much a fish out of water there. I was surrounded by people who I respect and who do interesting work and all they cared about talking about was the hot new programming language.
So my approach to tools is: is the cost-benefit of me taking the time to learn the tool going to have a significant impact on getting my work done more efficiently or effectively?
For example, I’m now known as an R programmer because my book heavily uses R.
The truth is, I’d never written a line of R code before I got to graduate school. I was a Java, Python, command line programmer from undergraduate, along with a little bit of MATLAB. When I went to graduate school all the statistics classes were taught in Stata.
It’s a point-and-click statistics program and you have to play by the rules. Eventually what the program allows you to do is, you have to use this highly stylized, domain- specific language for Stata called Mata. During graduate school, we were writing our own optimization functions in Mata. I was looking at the syntax and I didn’t know how I was going to do it. It was so far afield from any relevant training I’d had in computer science.
So I raised my hand and asked, “Can we do our problems in R?” And the guy teaching the class said, “Sure, I don’t care.”
Since I’d never programmed anything in R, I set out to teach myself how to program in R simply so I could finish my problem sets for my Intro to Statistics class. For me, once I’m committed to doing it I really want to learn it all and go really deep.