ℝolliℵg M∀th Thr∑a∂

i hear v good things about ESL and ISL

euler i think so, and sure!

you can probably do all those things in r (write an api, collaborative filtering, train a neural network, etc.), but i don't know anybody who does in production.

ha, having said that, i saw on twitter this talk is happening today

http://schedule.user2016.org/event/7Sq2/gradient-boosted-trees-model-deploying-r-models-into-production-environments

Allen/etaeoe, what's your favorite plotting library (in any language) right now?

I use ggplot2 all day every day, and while I try to keep my eye on new developments, I haven't yet found anything else yet that lets me get what's in my mind's eye onto a realized plot as quickly and easily. Lately I've been using Plotly with it, and wrapping ggplots in ggplotly() for some quick and easy interactivity (zooming, tool tips, etc)

-1 "yet"

Another good applied intro-level book along the lines of ISL is Max Kuhn's Applied Predictive Modeling, which gets into some hairier stuff that other sources tend to skip like how to deal with extreme class imbalance. He also touches on response surface methodology and multiobjective optimization, which is potentially so useful but I never see anybody else talking about (then again I don't come from an engineering background). Again, though, the book is R-based, so don't read it if you hate R.

Allen/etaeoe, what's your favorite plotting library (in any language) right now?

“It’s complicated.”

Typically, I use visualizations either as descriptive statistics or as figures.

When I need a descriptive statistic (e.g. histogram, Q-Q, or scatter), I’ll continue to use Seaborn from Python and ggplot2 from R. I find them too verbose. Especially when compared to R’s default plotting functions. But they work.

When I need a figure, I’ll use D3 to render an SVG suitable for publication. I’ve tried Cytoscape too. If a figure is computationally expensive to render (e.g. more than one hundred thousand observations), I’ll use SVG or WebGL directly.

I’ve used TikZ too. It works.

Everything I’ve mentioned feels inadequate. When I used ggplot2 (matplotlib too) in 2005, it was a major revelation. TikZ too. However, it’s been an insane decade for mathematics and statistics. 2005’s tools feel way too limiting for the ideas I want to express in 2016.

Conceptually, D3 is fantastic. And Mike Bostock has been a champion for articulating the transition we’re undergoing. Unfortunately, I don’t think D3 should become the default option. It feels antithetical to both standard and emerging web technologies. And it’s isolated from the larger web ecosystem (e.g. D3 uses custom selection and data-binding operations).

I think Plot.ly’s Plotly.js library is sensible as a curated collection of D3 visualizations. But venture-backed visualization software makes me nervous.

I also feel burdened by the lack of contemporary visualization tools for common problems (e.g. volumetric images).

Don't want to appear uncharitable, but feel like this software angle should perhaps have its own thread.

Unless you are using to calculate Catalan numbers, of course:)

Don't want to appear uncharitable, but feel like this software angle should perhaps have its own thread.

Yeah. Someone should start a “statistics” (or “data science” or whatever) thread.

Unless you are using to calculate Catalan numbers, of course:)

Or,

http://i.stack.imgur.com/ceazj.png

Don't want to appear uncharitable, but feel like this software angle should perhaps have its own thread.

― Tarzan v. BMI (James Redd and the Blecchs), Sunday, July 3, 2016 3:47 PM (Yesterday) Bookmark Flag Post Permalink

Unless you are using to calculate Catalan numbers, of course:)

― Tarzan v. BMI (James Redd and the Blecchs), Sunday, July 3, 2016 3:54 PM (Yesterday) Bookmark Flag Post Permalink

Maybe Catalan numbers should have their own thread, they rule

They definitely have their own book or two.

During the "grande affaire" of the earlier twentieth century debate on The Theory of Relativity between Albert Einstein and Henri Bergson, Paul Valéry, the French poet, diarist, and general man of ideas and letters, who corresponded with both on friendly terms, acted as a middleman on at least one occasion, accompanying Einstein on a visit in 1922 to Bergson's home.

Ha, wrong thread, mostly.

i'm against a separate 'data science' thread via apprehension of other ilxors posting their 'opinions' on it. everyone except us seems to ignore this one B-)

ive successfully avoided doing just that so far fwiw :/

RIP Kalman. almost broke my brain trying to understand your filter in time-series stats class :-)

http://hungarytoday.hu/news/renowned-hungarian-scientis-rudolf-kalman-dies-aged-86-46732

RIP

My "aha" moment in getting the Kalman filter was when deriving a simple version of it myself as a special case of the Bayes theorem, iirc.

can you show us?

Proof is left to the readers.

The proof is obvious

Or is it?

*leaves thread*

*time passes*

Yes, it's obvious

Been a long time, I'll see if I can reproduce the aha. :-)

https://twitter.com/AnalysisFact

http://www.johndcook.com/blog/twitter_page/

Euler buy this for your wife for xmas ;-) http://r4ds.had.co.nz/introduction-1.html

looks good!

Still pondering making a mod request to delete my miscalculation of the first few Catalan numbers.

FYI Artificial intelligence still has some way to go

https://www.youtube.com/watch?v=b0HzWMqLeiE

In a lecture that started talking about https://en.m.wikipedia.org/wiki/Classical_Wiener_space#Classical_Wiener_measure

And finding it very hard not to bust out giggling

hope you don't run into Tits groups then

Nah this is way funnier

I asked unthread for a readable analysis book and I chanced upon a pretty good one somehow. I'm reading Abbott, Understanding Analysis just for comprehension and it's going pretty well, in that a bunch of half-remembered things from high school math suddenly seem important in light of the careful construction of R and proofs about sequences and limits. On to continuity.

yeah, i had high school math flashbacks when i took intro analysis. "wait, haven't i done this before? oh wait, it all fits together."

Makes me wish they just taught me Analysis in 9th and 10th grade, it would have all seemed less directionless.

I have a question re: the hairy ball theorem.

We know that combing the hairs on one ball flat will leave at least one tuft of hair sticking out unable to be flattened. But what about two hairy balls that are touching? Can we perform a smooth combing over them?

i hate podcasts but this is good imo

http://www.csbookclub.com/

crüt: depends what you mean by "touching" and how you define a vector field on the resulting space.

If by touching you mean, cut a hole in each ball and glue them together along the boundary, then the resulting surface is still a sphere so the hairy ball theorem still applies

If by touching you mean identified at a single point, then it gets a bit tricky.

You can comb each ball in the following way, such that they have only one "pole" or cowlick:

https://upload.wikimedia.org/wikipedia/commons/6/6e/Hairy_ball_one_pole_animated.gif

then you can identify the pole on each ball. The resulting space is no longer a manifold, so the question then becomes, how do you define a tangent vector at that point (the wedge)?

One possibility is to just ignore the "bad point", the complement of which *is* a manifold. If you mean that, then by construction (taking a vector field on each sphere whose only zero is at the bad point), you do indeed get a non-vanishing vector field.

On the other hand: if the two spheres are touching tangentially, then the tangent planes to the two spheres at the bad point line up, so we can still talk about a tangent vector at that point, and so it still makes sense to talk about a vector field on the whole thing.

OK i have a math question. something I should know how to do and surely learned at some pt at school but I forgot and I don't know what topic to look up

Let's say you have a function F from integers between 1 and 100 to R. generally a well-behaved function but locally can get spiky

I want to approximate it by a function G that is

1) as similar to F as possible
2) as smooth as possible
3) integrates to the same value as F: G(1) + G(2) + ... + G(100) = F(1) + F(2) + ... + F(100)

I know we did this stuff in Numerical Analysis and had to write Matlab scripts that did this sort of thing all the time. Condition (3) is straightforward, but (1) and (2) seem to be in tension; maybe I have to minimize some loss function of MSE and some measure of spikiness?

This seems as good a place as any to mention that Don Knuth has announced his experimental work for pipe organ.

http://www-cs-faculty.stanford.edu/~uno/fant.html

flopson: maximum entropy seems like one tack to take? http://www.lacan.upc.edu/arroyo/Site1/Research/Entries/2012/9/12_Maximum_entropy_approximation.html

let me restate to be sure i understand tho -- "i have an assignment of the integers from 1 to 100 to 100 respective values in R. i would like to make a new assignment of the same form N_[1-100] -> R, but with the condition that the sum of values in the codomain agree with the prior one, and with some sort of smoothing applied."

If that's correct, yeah, you need to decide the tradeoff yr willing to make, towards what end, between similarity and smoothness. i mean a lazy and decent thing to do is just to take some sort of quadratic or cubic interpolation on the points, then "bump" it into shape with a second pass that makes the sum tie out while minimizing MSE as you describe.

that's correct. looking for something simple and quick

friend linked me to: https://en.wikipedia.org/wiki/Mollifier

neat!