ℝolliℵg M∀th Thr∑a∂

I've tried Rodeo, but it's just a pale facsimile of R Studio.

It's not! I really like Jupyter notebook, but I wish there were a development environment as good as R Studio for Python! In fact, I just wish R studio would include full python support (technically you can run python from inside it, but not very well). I was so spoiled coming from R, I assumed there was something as good as R Studio for every language.

PyCharm (maybe)?

(I like RStudio too! I just find it difficult to organize projects.)

There's some exciting R/Python crossover going on. Hadley and Wes McKinney just created Feather together: http://blog.rstudio.org/2016/03/29/feather/

They re-implemented HDF? I guess? Nonetheless, I like Hadley and Wes. :D

Oh, Spyder is worth a look too. https://pythonhosted.org/spyder/

fuck i'm getting crucified on StackOverflow for my noob R questions lol

i'm finding weening myself off of for-loops is pretty difficult... all my intuitions about how to approach a problem kind of go out the window. but i'm slowly getting the hang of the 'just put it in a list and lapply' philosophy. someone on SO directed me to this which i read last night stoned in my room at like 2 am and it blew my mind. my scripts are gorgeous now, even though it takes me twice as long to write them

oh is R a functional language? my wife is learning it because she's moving into ~~~ data science ~~~ (she has a doctorate in math) & all the jobs want R. was thinking it would be a week-long project b/c I thought it was more a tool for stat analysis than for developing apps, but I dunno.

it is functional and a tool for stat analysis

And can generate lots of graphs quickly

That SO answer is really weird to me! Putting dfs into a list to apply a function over all of them is natural, but do you really need to do that so often that you're going to keep all your dfs in a list right from the beginning just in case? Reflects a very unorthodox (but not necessarily wrong, I guess) mindset!

It's possible that I'm missing some fundamental advantage of the approach. I just can't remember the last time I needed to do the same thing to an entire list of dfs.

Ah, okay, a comment on a question linked to that question helps me understand: "If the data.frames have a similar structure, it is a good idea to keep them in a list."

that makes sense

Typically dreadful article on Gödel in the New Yorker

surprised, i think roberts is usually pretty good

Promulgated in Vienna in the early nineteen-thirties, the notion of incompleteness threw mathematics into a hall of mirrors, where it reflected upon itself to alluring, if disorienting, effect: the theorem proved, using mathematics, that mathematics could not prove all of mathematics. Of course, it has a proper and technically precise formulation, but the late logician Verena Huber-Dyson paraphrased it for me as follows: “There is more to truth than can be caught by proof.” Or, as the British novelist Zia Haider Rahman put it in his award-winning début, “In the Light of What We Know,” “Within any given system, there are claims which are true but which cannot be proven to be true.”

until the last sentence, that's just heartbreakingly ugh.

& even the last sentence is super wrong.

Whee shimmering metaphors for incompleteness what fun

i had a friend who was always horrified by explanations of incompleteness theorems intended for popular audiences. those quotes don't look that bad to me and the author is explicit about quoting a logician who is himself paraphrasing, and a novelist who why would you even ask a novelist. idk it never bothered me that much but i only ever read 'Godel's Proof' by Ernst Nagel

imo exactingly precise statements of the gödel thing can seem trivial without historical background, like why would anyone care that you can encode statements with arithmetic etc. but people really believed in Hilbert's program, and iirc even kg himself intended to prove a positive result of 2nd problem

would love to read a good history of devastating negative results in math

i read the first 50 pages of Zia Haider Rahman's book due to a favorable James Wood review, but found it insufferable, partly because of how it tossed math around as this complicated thing you couldn't possibly understand except in the most banal metaphors. i just read Cryptonomicon and was wondering if a real logic/cs/cryptography person would be rolling their eyes at some of the technical stuff in it. but contra Rahman Stephenson really takes time to explain stuff; the gears of a bike analogy for prime factors was great

Don't have to be too exacting to point out that eg elementary geometry is immune to incompleteness

I myself have an allergic reaction to popularizations that go overboard with the far flung analogies and "connections" -which reminds me to ask- why does every other book related to computability have to have an intro by Douglas Hofstadter?- so I can only imagine how a professional logician like Euler feels. Would like to read Janna Levin novel about Gödel and Turing, though.

If I had to summarize Gödel's theorem in four words I would type "Formalize. Apply diagonal argument."

Also, guy who inspired me to start this thread Math & Music: The Severed Alliance. Some Recent Academic Approaches (Do Not Read If You Hate Drums) who is a topologist at Lehman College is playing a jazz piano gig tonight at Mezzrow. Would like to go sometime but probably not tonight.

it is functional and a tool for stat analysis

― de l'asshole (flopson), Sunday, June 26, 2016 11:02 AM Bookmark Flag Post Permalink

R is not really that functional. It's a weird hybrid designed by people in an ad-hoc way. It's got amazing libraries and tooling, but as a language if you think "functional" you'll get confused after a while (but probably if you don't think 'functional' you'll get confused too -- lots of things just don't make much sense outside of 'it was easier to implement this way').

Hadley Wyckham says R is "at its heart, a functional programming language"

idk it never bothered me that much but i only ever read 'Godel's Proof' by Ernst Nagel
When I studied this as an undergraduate, we spent a term going through Boolos and Jeffrey, but this Nagel book looks short and sweet.

Hadley Wyckham is right and wrong in the same way as if you said that about say Javascript. If you _can_ have closures, you _can_ be functional. But that's not how most libraries are written, and the language has lots else going on

http://jasp.ism.ac.jp/kinou2sg/contents/R-ism-dec-8-no-anim.pdf

http://r.cs.purdue.edu/pub/ecoop12.pdf

http://community.haskell.org/~ndm/temp/EGMitchell-ExperienceReport.pdf

i read the first 50 pages of Zia Haider Rahman's book due to a favorable James Wood review, but found it insufferable

OK I looked at the Wikipedia page for this and I can't think of a time extravagant praise for something made it sound so terrible

from paper sclover linked

As a language, R is like French; it has an elegant core, but every rule comes with a set of ad-hoc exceptions that directly contradict it.

sick burn lol

and this seems like a good answer to Euler's question:

The R user community roughly breaks down into three groups. The largest groups are the end users. For them, R is mostly used interactively and R scripts tend to be short sequences of calls to prepackaged statistical and graphical routines. This group is mostly
unaware of the semantics of R, they will, for instance, not know that arguments are passed by copy or that there is an object system (or two). The second, smaller and more savvy, group is made up of statisticians who have a reasonable grasp of the semantics
but, for instance, will be reluctant to try S4 objects because they are “complex”. This group is responsible for the majority of R library development. The third, and smallest, group contains the R core developers who understand both R and the internals of the
implementation and are thus comfortable straddling the native code boundary. One of the reasons for the success of R is that it caters to the needs of the first group, end users. Many of its features are geared towards speeding up interactive data analysis.
The syntax is intended to be concise. Default arguments and partial keyword matches reduce coding effort. The lack of typing lowers the barrier to entry, as users can start working without understanding any of the rules of the language. The calling convention
reduces the number of side effects and gives R a functional flavor.

sorry 4 butchered formatting

first set of slides were incomprehensible (although i liked how the code was typeset in comic sans lol) to me and the paleontology one i didn't really get but the middle one seems spot on, from a skim by an extremely non-CS person. a lot of the R gods on SO constantly admit flaws and inconsistencies in the language due to weird implementation

Was trying to remember earlier what mathematician had no hands and was going to post to this thread for help but then it finally came to me.

Hadley Wyckham says R is "at its heart, a functional programming language"

I like Hadley. But he’s wrong. It’s iteration and selection throughout. You should use apply because R’s loop optimizations are horrible to non-existant.

I liked R. And I’ll still occasionally use R when it’s a collaborators preference. But I can’t imagine a student starting with R in 2016 when Python’s scientific community is so far ahead in practically every area.

Nonetheless, when someone asks for R advice, I usually tell them to read Patrick Burns’ “The R Inferno:”

http://www.burns-stat.com/pages/Tutor/R_inferno.pdf

And nobody should ever use S4 objects! Woof!

I liked ggplot2. But Hadley’s post-ggplot2 work is a reminder of Maslow’s hammer. Your work suffers when you become too attached to a familiar tool.

And, frankly, ggplot2 feels archaic in 2016. gnuplot and matplotlib too.

"data science" is this meaninglessly general term that is starting to be usefully divided up in to "product data science" (e.g. machine learning in the product) and "analytics" (e.g. decision science/business intelligence).

R is virtually useless in the first, but much more useful in the second, which is more traditional stats and batch/static reporting.

yes the work my wife is looking at is in "data analytics" particularly. the company she's looking at right now wants (in addition to a doctorate in math or stats, and English fluency) capacity with SQL, and R and/or Python and/or Excel. I lolled at Excel but I think that says well what they want.

"data science" is this meaninglessly general term that is starting to be usefully divided up in to "product data science" (e.g. machine learning in the product) and "analytics" (e.g. decision science/business intelligence).

R is virtually useless in the first, but much more useful in the second, which is more traditional stats and batch/static reporting.

― 𝔠𝔞𝔢𝔨 (caek), Thursday, June 30, 2016 8:31 AM (1 hour ago) Bookmark Flag Post Permalink

i work in analytics but there's tonnes of ML in R

i used to lol at Excel when i was in school but it's the least pain in the ass way to just look at data quickly imo, which is extremely useful in the job

ty for R inferno, this is hilarious

R has ML libraries, sure. so does javascript. they don't get used in product though.

what does that mean?

as far as i've experienced, r doesn't get used as the backend for web apps, for collaborative filtering at web scale, for CNNs, etc. these are the use cases i mean when i say "product".

you can probably do all those things in r (write an api, collaborative filtering, train a neural network, etc.), but i don't know anybody who does in production.

it doesn't seem that needs chez moi involve developing apps of any kind, that's for the developers afaict, not the analysts, but I dunno. from what I've read of R it seems silly to do development there.

i once got asked in an interview "what kind of data scientist are you" and it turned out he was getting at this product/production vs analyst distinction. i think it's real, and IME r definitely falls on one side of it in practice, and that's at least in part because of the design of the language (rather than mere social network effects). but to be clear there are tons of jobs where r is far and away the most useful language you can know.

yeah I mean we're just reading ads but it seems to me if you want a doctorate in math/stats then you're not just looking for a developer. but I dunno.

this is extremely reductive and misses out on tons of factors/complications, but gives a very rough idea of what's most valuable to know. valuable != necessary of course.

https://duu86o6n09pv.cloudfront.net/reports/2015-data-science-salary-survey.pdf

huh that's interesting and helpful

here's a very stupid question: is there some recommended "certification" for having learned these tools, or can you just pick them up on your own and then list it on your cv/resumé ? my own CS degree is like 20 years old & I don't remember anything about that (& my wife doesn't have any CS degrees, just math, though she used Matlab a lot for her dissertation, in applied math). like what do self-trained people in these tools have to do to convince employers that they can use them? or will this come out in some test in an interview?

for data science, it's less of a problem to be a self taught coder in "tech" businesses than in more traditional business. the discipline is mature enough that there's a fairly good change you end up being interviewed by someone who themselves has a strong quant but non-CS phd.

so, given a maths phd, i don't think further credentials are strictly necessary.

that said, there's a cottage industry of boot camps/recruitment things that make the transition quite a lot easier (and perhaps more lucrative), either by formally teaching stuff and providing credentials, providing an environment in which your "job" is to learn for a few weeks, or helping with applications/interviews. http://insightdatascience.com/ is the best known of these.

if your wife knows matlab already, then i recommend andrew ng's coursera machine learning course. it's intellectually interesting but it's also excellent interview prep. the only thing i didn't like about it was that the exercises were in matlab, because i had to waste time learning that. i put that (and a couple of other coursera courses) on my resume my first time out, but i don't think anyone noticed or cared about how i'd acquired the knowledge.

ok super, we'll have a look. she's got plenty of time for coursera courses; right now she's working through an O'Reilly book on R and it's going easily as expected.

(major caveat with any advice i give: my experience and network is all tech/startup, which is an unusual industry and is not where most of the jobs are, i.e. healthcare, insurance, finance, etc.)

right, she's looking at the tech/startup industry in Paris, which is quite weird as you can imagine.

(though one startup in Paris last year hired more mathematicians in France than all universities in France combined, and this is the current target)

caek does your ilxmail work? my wife has questions for you if you'd be willing.

i read this book

http://www-bcf.usc.edu/~gareth/ISL/

which does all the examples in R. the methods are outdated but perfect for getting the intuition, and the big themes bias-variance tradeoff are really well-developed. it's extremely easy and i got through it in a week. it's the baby version (created for an MBA class iirc) of Elements Of Statistical Learning, which i'm reading now

i hear v good things about ESL and ISL

euler i think so, and sure!