"Data" Professions - what's the shittiest

Data masseuse. People are always massaging data.

Data birther

Data Finagler

Data Purchaser

Data Flinger

Data Hood

Data Mensch

Data Barber

Data Angler

g'night

data scientist is just a trendy name for a data analyst as far as i can tell

― ciderpress, Thursday, October 27, 2016 10:48 AM (yesterday)

haha... if only my analysts had stats, applied maths and strong coding backgrounds... not to even broach machine learning, proper feature engineering and data viz outside of a standard dashboard.

my issue with data scientists coming from academia is they tend to over-rely on quant skills to make up for lack of soft skills when communicating abstract proposals to management. my problem with career-pivoters entering data science in the past couple years is that they rarely become competent coders and (tragically) rarely grow out of their analyst methods of processing/problem-solving... that said, these folks are usually a fantastic resource for obscure excel tips.

senior data scientists with 10+ years of industry experience are typically a delight to work with.

heuristic-infested

<3

yeehaw!! gitalong, little datas

Automatic thread bump. This poll is closing tomorrow.

Data Yoda

https://www.youtube.com/watch?v=KuTSAeFhdZU

https://twitter.com/johnmyleswhite/status/811721143396679680

necessary but not sufficient condition for being a data scientist: your "best" language is python or R.

apart from that it's a totally incoherent cohort of people with a huge range of skills. i have literally been asked "what kind of data scientist" are you in phone interviews. this question seems to be getting at the spectrum that runs from between "decision science"/"business intelligence" data scientists to production machine learning software engineers.

for my money (except in certain professions like banking where analyst has a specific meaning) a data analyst is either a junior data scientist or a data scientist who can't code. these people don't get paid as much but they find work.

data scientists who can't write production code or work with software engineers, of which there is a glut thanks to literally thousands of science phds deciding to move into the profession, are suddenly finding it extremely tough to get jobs in tech. i spoke at one of the boot camps for phds yesterday and there is a "oh shit i'm going to end up at an insurance company" fear in the air.

the HN comment linked in that tweet is OTM https://news.ycombinator.com/item?id=13233352.

the situation in europe is a bit more forgiving as far as i can tell. the bubble never got as big there.

That Shingy clip is killing me.

a "oh shit i'm going to end up at an insurance company" fear in the air

i hear they got money tho

compared to glamour tech roles not really. insurance companies are claiming to be becoming technology companies but are only just starting to pay technology salaries for their tech roles yet. obv they have a ton of middle management so there's a lot more scope to move into those roles, which are better paid. but first job out: a mid-stage tech startup is going to pay a data scientist at least as much as an insurance company.

ctrl-f "insurance" and look at the second result in this pdf http://www.oreilly.com/data/free/files/2016-data-science-salary-survey.pdf, and then adjust for the fact that many people who call themselves data scientists at insurance companies are managers.

i was measuring relative to the bottom quintile, which i am in : P

yeah, i guess i'm v focussed on a cohort of science phds who thought they weren't going to be in the bottom quintile but are now realizing they probably are

call yourselves what you like, you're all nerds

:D

xp everyone thinks they're above average

I guess the trick is to get some practical experience with some data sets specific to a field, and then you get to change your title to something a bit more prolific cashflow-wise?
Most people I know who do "data science" type things are probably half-decent ("scrappy") coders who are especially proficient with a subset of data types and use cases and exactly as smart on math as they need to be to filter what they want out of the trough. In some cases they're also into going out and getting new piles of data from other institutions, which can become a self-licking ice cream cone of a career if you do it well enough. But it's all very constrained to specific problems and nobody would suggest that a Ph.D would be necessary to do any of it.

Also the people I'm talking about are all making $120K+

rather, almost all the people I'm talking about - I shouldn't overgeneralize

can walk out of a 6 month night course in analytics into 65k job in dublin

yeah that describes my work pretty accurately though I don't have a masters yet so im not making quite as much $

i would never call myself a data scientist though. im an analyst

Automatic thread bump. This poll's results are now in.

Crowds are intelligent

data scientists who can't write production code or work with software engineers, of which there is a glut thanks to literally thousands of science phds deciding to move into the profession, are suddenly finding it extremely tough to get jobs in tech.

i work in research for a large US technology corporation and i tend to agree with the above. in our projects related to ML the main problems we face everyday are for the most part practical: what is the most efficient way to implement a given algorithm? how do we scale out across many machines? how do we learn on very large amounts of streaming data? how do we deploy and administer an application that is running on hundreds of VMs in the cloud.

having a deep understanding of the theory is definitely a huge advantage when it comes to solving these problems relative to someone from a software development background. BUT i really think it's limiting to consider oneself above writing serious code and i am working at convincing our phds of this. i speak from experience because i also used to think of myself as a "theory" person - i did mathematics phd. since i decided to get more involved in development, i have been reaping the rewards both in terms of scientific results and career development. like, someone who can understand the theory and actually build stuff is always going to be more useful that someone who can do one or the other.

moving on from discussion here - Democratic (Party) Direction - regarding Kamala Harris tweeting about "risk assessment" and the Weapons of Math Destruction book that talks about how fucked up those risk assessments can be, I feel it is important to note that building models and algorithms for social applications is just as fraught as the self-driving car shit. GIGO definitely still applies and the amount and relative quality of data is still way too sparse and shitty to be using it as a substitute for ground truth in policy making.

the point is removing some human/judge discretion can lead to more fairness and less punishment, which is the opposite of cathy oneill's fears (that the algorithms will just reify discrimination in a black-boxed gigo way and amplify it potentially in some sort of positive feedback loop)

― flopson

let's talk about the work the word "can" is doing in your first sentence. understand that cathy o'neil is a data scientist and not a luddite, and she certainly is not saying "let's just do everything manually". shit, back when they were imposing mandatory minimums people made the same sort of arguments about them, that "objective standards" would lead to greater fairness in sentencing. "objective standards", or "algorithms", or whatever terminology you want to use, _can_ and _have been_ racist, and starry-eyed technocratic idealists have consistently, and continue to consistently, ignore the racist ways in which their "objective" technological "innovations" can be and are put to use.

Algorithms were created by people, and embody the values and biases of the people who created and implement them.

It's just another way of saying you, personally, your cadre, your class, has the "objective truth". God, weapons, science, economics, now algorithms.

xp- u seem mad

if u want to "look at the work "can" is doing" in that sentence (u can fuck off with this angry scare-quoting and bad faith sub-undergraduate "close-reading" of my posts btw) maybe read the working paper i posted that literally showed an algorithm was less racist than judges in determining whether defendants should await trial at home or in jail

fwiw i think Weapons of Math Destruction is a great book and take its warnings extremely seriously, but it absolutely has flaws and was written very "early" in this process of shifting-human-discretion-to-algorithms so a lot of future research will inevitably prove some of its concerns wrong (while also elucidating others). Cathy herself takes a similar position on it fwiw

sry didn't mean to get aggro--just, pls don't jump down my throat or read something into what i was writing that wasn't there

no worries i'm disinclined to get into an online shouting match over, well... it's a semantic argument. we don't actually disagree about anything specific here as far as i can see and there's not really a lot to talk about.