StreetEYE Blog

The Top 100 People To Follow To Discover Financial News On Twitter, May 2017


It’s been a year since we posted our last list of people to follow on Twitter for financial news. Time for an update!

I posted earlier about some of the trends in the financial Twittersphere.

  • More churn
  • Less growth
  • Politics (i.e. Trump) drowning everything else out

See below the table for a little technical discussion, and a fun interactive map.

Screen name Quality Centrality Frequency
valuewalk 10.00 1.97 9.04
JohnLothian 8.54 0.05 10.00
blakehounshell 8.28 4.55 6.63
lrozen 7.86 2.60 8.05
pdacosta 6.70 6.02 7.43
ClaraJeffery 6.49 2.60 5.94
HotlineJosh 6.42 0.77 6.32
fmanjoo 5.73 5.41 4.52
JohnJHarwood 5.69 3.48 4.49
mathewi 5.50 2.12 4.74
moorehn 5.47 4.64 6.47
activiststocks 5.45 0.41 3.81
niubi 5.26 2.13 4.32
michaelsderby 4.96 1.77 3.28
Frances_Coppola 4.92 2.69 3.71
acrossthecurve 4.79 1.07 3.03
TimOBrien 4.77 3.97 2.90
BruceBartlett 4.75 2.49 3.76
MarkThoma 4.74 5.49 3.05
dandrezner 4.64 4.24 2.89
mattyglesias 4.58 6.37 2.96
sdonnan 4.57 3.07 4.29
AmyResnick 4.53 2.71 4.59
kimmaicutler 4.52 1.70 3.06
crampell 4.51 6.52 3.24
davemcclure 4.50 1.31 4.02
WillauerProsky 4.40 0.65 4.16
kairyssdal 4.33 2.93 2.32
AdamPosen 4.26 4.39 2.88
gabrielsnyder 4.24 2.24 2.03
sarahcuda 4.23 1.23 3.36
DavidClinchNews 4.19 0.54 2.96
BobBrinker 4.16 1.11 2.73
Noahpinion 4.16 4.94 3.09
Susan_Hennessey 4.16 1.73 2.60
AnnieLowrey 4.10 5.48 2.21
ComfortablySmug 4.08 1.54 3.34
EpicureanDeal 4.07 2.68 2.40
brianstelter 4.06 5.51 2.91
kadhimshubber 4.00 1.50 2.22
LaurenLaCapra 4.00 2.03 2.49
hblodget 3.98 6.76 2.06
NickatFP 3.98 2.05 2.56
djrothkopf 3.94 1.99 2.71
TimDuy 3.94 2.57 1.95
ZekeJMiller 3.93 1.91 2.17
davewiner 3.88 0.79 3.65
JohnCassidy 3.87 5.08 2.29
FGoria 3.84 1.74 1.79
BarbarianCap 3.76 1.61 2.46
BaldwinRE 3.72 2.33 2.24
carlquintanilla 3.69 4.94 1.91
BrendanNyhan 3.66 3.34 2.19
jbarro 3.65 6.71 1.97
lizzieohreally 3.63 3.91 2.53
sspencer_smb 3.59 0.26 2.59
NKingofDC 3.58 1.91 1.69
MissTrade 3.58 0.00 4.66
JesseDrucker 3.58 1.96 1.90
jyarow 3.57 2.42 1.34
harrisj 3.56 0.71 2.46
ReformedBroker 3.53 6.26 1.68
markgongloff 3.50 2.21 1.72
MattGoldstein26 3.47 2.81 2.28
FlitterOnFraud 3.42 0.91 2.33
jeffjarvis 3.35 1.66 2.27
davidmwessel 3.35 6.59 1.55
raju 3.35 2.91 2.89
PekingMike 3.30 1.52 1.71
TonysAngle 3.29 1.20 2.31
NickTimiraos 3.28 3.85 1.43
ritholtz 3.22 6.44 2.03
GreekAnalyst 3.21 1.58 2.17
ObsoleteDogma 3.21 7.91 1.07
prchovanec 3.21 2.60 2.29
HBoushey 3.17 1.74 1.60
rationalwalk 3.11 0.03 1.98
karaswisher 3.07 4.39 1.72
jessefelder 3.06 1.03 1.46
BuzzFeedBen 3.02 5.59 1.55
edwardnh 3.00 3.30 1.70
RobertMackey 2.99 0.95 1.83
mims 2.98 4.31 1.46
TimAeppel 2.97 1.65 1.66
M_C_Klein 2.95 5.98 1.17
mekosoff 2.92 1.37 1.68
qhardy 2.90 1.99 1.55
jamessaft 2.88 1.55 1.44
davidjoachim 2.86 1.19 1.54
mbaram 2.82 0.58 1.48
alex 2.81 0.86 1.93
MikeIsaac 2.81 2.75 1.31
NoahShachtman 2.78 1.92 1.19
BCAppelbaum 2.77 8.05 0.81
mccarthyryanj 2.77 3.16 1.14
EddyElfenbein 2.76 2.64 1.64
NinjaEconomics 2.71 2.82 2.21
JHWeissmann 2.71 2.16 1.22
CarlBialik 2.68 0.72 1.60
inafried 2.66 1.30 1.74

Here’s our map of ~350 Twitter accounts, using a force-directed network. Everyone who follows an account or posts similar links pulls it toward them, so we see Twitter accounts arranged into broad areas: Asia, Europe, US markets, tech, media, econ, politics. People who are in the center are pretty equally followed by folks in all parts of the map, whatever their varied interests. Bigger labels represent people with larger followings.

Here’s a link to a fun interactive version.

How does this work?

We start with a network centrality analysis, building a graph of who follows whom and finding the people with the largest number of most influential followers. Those are probably people who will play an important role in transmitting important news.

Then we trim the list to people who are good curators: they post links to a variety of good content in a frequent and timely manner. We cull people who may be central, but don’t post a lot of good timely news links.

Who do we cull?

  • Non-relevant accounts. @DonaldTrump (the new champion!) and @BarackObama are widely followed, but the stuff they post is only occasionally a news link directly relevant to financial markets.
  • People who don’t curate, filter, and signal the most important things to read. @LHSummers is a good example. He’s a really important guy to follow but he only tweets out his own stuff, he doesn’t help people discover financial news.

We rank the quality of each account’s curation on how much financial news they tweet, how much of it gets picked up and generates a lot of buzz, and how early they post it. It’s a subjective, but hopefully useful formula. Everytime you tweet something, and your followers view it and retweet it, traffic flow downstream, more likely than not from the most central folks to the periphery, and whenever attention flows downstream, influence flows back upstream in a sort of Newton’s Third Law of social media.

And of course, see the most shared financial stories updated continuously on the StreetEYE home page, and follow us on Twitter.

A final P.S. and caveat: Like all such lists, it’s closer to a fun parlor trick than a definitive ranking. I’m delighted if people I respect get a kick out of it and even find it useful to discover new people to follow, but keep in mind that 1) I just used a small sample of Twitter data and 2) it’s just a simplistic formula. If you’re on here then you’re probably pretty widely followed, post a lot of timely and relevant financial news, that goes pretty viral. Let’s just say it’s inherently has a lot of bias and variance and tilts mainstream by design. I wouldn’t read much into why X is higher than Y or Z isn’t on here. Don’t take it too seriously!

Frequently asked questions:

  1. WTF is StreetEYE? Since 2011 we’ve tried to systematically find the best people to follow for financial news, and create a crowd-sourced front page of the financial Internet. You can read more here.
     
  2. Why isn’t <brilliant pundit or technical commentator> on the list? Assuming they are very widely followed, probably because they don’t share other people’s content very much. There are a lot of great people to follow, who only share their own original content, or that of their colleagues. They don’t add a lot of signal to help discover news, so they don’t meet the relevance bar. 
     
  3. Why isn’t ZeroHedge on here? Everyone knows they should be #1! ZeroHedge blocks our bot on Twitter. There are three accounts I would like to follow that block us. Maybe they hate being aggregated in this way. Maybe I said something snarky. I have been known to be a jerk. Sorry! Ask them.
     
  4. What about Drudge Report, Memeorandum, Techmeme, Mediagazer? Those are aggregators in politics, politics, tech, media respectively, the latter three in the great Gabe Rivera’s empire. They are fantastic and a lot of people follow them, I recommend you check them out. But even though by the numbers they rank pretty highly, they’re not specifically market-related and people thought they looked weird on this sort of list, so I took them off as not directly relevant, despite decent numbers.
     
  5. Why isn’t <other great account> on here. Maybe they are just below the 100 cutoff. I am the first to admit the formula is subjective and arbitrary. If they post a lot of timely relevant headlines, they are probably on our radar. If there is someone who is relevant and widely followed who is not on the list, we would like to know. Contact us! We love feedback.
     
  6. Can I see the whole list? Here are the top 300. If you want to see where others rank, or follow even more of them, go to town!
     
  7. If you have other questions, tweet at us at @StreetEYE or contact us via the contact form.
     

Rethinking the marketplace of ideas

I was recently listening to Fred Wilson and Howard Lindzon talk about, among others, news sources and curation, which is a topic dear to my heart. (Which I wrote about before here and here).

Curation is hard, people tend to fall into confirmation bias circle jerks, and we are in a crisis of media legitimacy. (More generally, the legitimacy of the establishment and objective reality.)

If you think the moon is made of green cheese, you have a problem. If a lot of people think the moon is made of green cheese, society has a much bigger problem. (Or that climate change is a hoax invented by the Chinese, or that vaccinations cause autism, etc.)

The marketplace for ideas is faltering. I’m not sure if it’s only the arrival of the unwashed masses, Facebook likes, adverse selection, filtering tools that aren’t fit for purpose, or manipulation and ‘fake news.’

What seems to have gone missing is the sharing ethic that existed when the blogosphere and Twittersphere and social bookmarking were young.

What works to surface quality content is human curation, like Memeorandum, Abnormal Returns, and an engaged community that actively promotes and shares quality and feels that it’s important… like AVC, Hacker News, the better subreddits, Open Culture, Arts and Letters Daily, Brain Pickings, etc.

Ray Dalio seems to think a new market design with some kind of regulatory component is needed. I don’t know what he has in mind, but one could imagine a self-regulatory body that agrees on journalistic standards like CFA ethics: clearly separating fact from opinion, having a reasonable basis for any statement of fact or conclusion, showing your sources and investigative processes, promptly correcting errors, etc.

And then people and media sources that meet those standards get a seal of approval, and it investigates violations, censures or even expels people and organizations that don’t meet the standard. Sort of a credit rating agency for journalists.

Government regulation of the press isn’t compatible with the USA’s Constitution and even press self-regulation wouldn’t be in line with American’s traditional sense of the free press. It’s not clear that the most prominent publications e.g. the New York Times and Wall Street Journal would submit to a regime like that or that it would have much sway without them.

But Dalio is not wrong either, we need better consensus on what we expect from journalists and tools to signal credibility and hold people accountable. If the New York Times wants to be credible, it can’t rely purely on prestige, it has to be open about its standards and practices and uphold what it means for something to be published in the newspaper of record.

David Siegel and Cathy O’Neil think personalization and big data are the problem.

Siegel frames the problem as one of micro-personalization. The picture I have is, in the old days everyone watched Cronkite and read the New York Times, and elite media institutions set the agenda. With the Internet and cable TV, the media fragmented. Everyone is more receptive toward media outlets that reflect their own values and point of view, and gravitates towards outlets that reflect them. But the more you hear mostly news and points of view that confirm your own biases, the stronger those biases get. And personalization and news recommendations are the ultimate silo or filter bubble. You only hear the news you like to hear. The end result is a singularity of polarization, where anything that doesn’t toe a narrow line triggers cognitive dissonance and a strong emotional reaction of ‘OMG mainstream media bias’/’fake news’.

Siegel’s argument as I understand it seems completely plausible. But the root issue is fragmentation, not algorithms per se. Eliminate algorithms and you still have the problem that some people get all their information from Democratic Underground and some from Free Republic. You can make an algorithm optimize for whatever you want, for instance try to surface quality from a variety of points of view. The algorithm genie is not going back in the bottle. To the extent there’s an algorithm problem, the answer is to improve the algorithm.

O’Neil’s “Weapons of Math Destruction” point is that any manipulation that can fool humans can fool the algorithms. We’re stuck with misinformation, and the solution is to turn to trusted sources. But what happens if sinister forces fool your trusted source, or make you think a source can be trusted when in fact they are bought and sold?

Furthermore, if I only believe stuff once it’s fully fact-checked and published on Bloomberg or in the New York Times, I’m ignoring a lot of potentially useful information and sources. It’s a bias/variance problem. I want a news filter that is sensitive enough to surface the viral story about United Airlines if I happen to own their stock, while being selective enough to bury hoaxes.

If the argument is that any signal a filter might use can be gamed by fake news information operations, that is generally true, but only up to a point.

Indeed, Goodhart’s law is in play, and any statistical regularity that let’s one filter fake news will tend to collapse once pressure is applied on it for control purposes and it starts being gamed.

But good predictors don’t usually collapse to zero. Even though just about anything can be faked with sufficient skill, museums are probably not mostly full of fakes, even though the incentives to manufacture forgeries are enormous. Our relative safety lies in the fact that it’s hard to commit a perfect crime. The entropy of reality is impossibly complex to forge. With enough fact-checkers, lies are generally detectable. With enough resources you can make anything go viral, but you can’t make anything withstand scrutiny. You can fool all the people some of the time and some of the people all the time, but you cannot fool all the people all the time. And (fake news alert!) Lincoln never actually said that.

The crude manipulation of the form ‘Pope endorses Trump’ is straightforward for the top quartile of readers to detect and debunk. So it is feasible for algorithms to filter that out based on where the story originated, the ratio of credible sources spreading it, spam flagging. Just as Google filters spam, news algorithms can and should filter egregiously fake news.

Platforms like Facebook and Google are going to build algorithms, and it’s incumbent on them to build ones that can’t be fooled all of the time. To cry censorship is a bad-faith, burn-it-all-down argument. The answer to bad algorithms is better algorithms. There is no argument for letting people vote based on ‘Pope endorses Trump’-level news. Either participate in truth-based news and policy, or GTFO, because democracy needs to be nurtured, and the crude distortions of reality, reminiscent of Nazis and Soviets, herald the end of democracy.

Powerful forces will use big data and machine learning to manipulate you. But big data and machine learning algorithms are also very powerful tools to fight manipulation and help surface quality. And we should be doing that instead of decrying personalization, and building a market design which is more resistant to manipulation and provides tools to surface quality.

I think if we could somehow build a pervasive ‘pay it forward’ karma and reputation ecosystem that rewarded people for sharing quality and burying fake news and garbage, and somehow get back some of the old sharing ethic, it would go a long way. That’s what I’d be thinking about if I were a VC or online community entrepreneur. Tools to let people signal quality and build credibility and fight the noise machines.

If credible news organizations open their work, tag facts vs. opinions, open up source materials like interview transcripts, link to sources, then a crowdsourced fact-checking Wikipedia / Genius could do a pretty good job debunking stuff that’s complete garbage. Contributors recognized as fair fact-checkers could get karma and monetary rewards.

Something like WikiTribune seems like a promising approach. Maybe even an industry news integrity initiative.

It’s a huge problem and solving it is critical for democracy and free market capitalism.

That being said, if you have a large chunk of people who want to tear everything down, you’re not going make progress building trust and credibility.

And fish rots from the head. If you’re led by someone with a habit of lying, denying reality, and calling everything he doesn’t like “fake news,” you’re going to continue to have a crisis of legitimacy and reality-based institutions.

Come back Kelly Evans! We’ll be good this time! I promise!

A digressive rant on the rot in the financial Twittersphere in the Trump era

If we’d been born where they were born and taught what they were taught, we would believe what they believe. – attributed to Abraham Lincoln

(who also said, “The love you take is equal to the love you make”, but that’s another story)

Spring is in the air! On the East Coast, anyway.

Last week I was fortunate to get out to the West Coast, dodging storms on this side of the country and also over there. And this week I spent a little time sprucing up StreetEYE sources, deleting people who haven’t tweeted in a while, adding the popular, influential, prolific, and relevant new sources. Every year or so around this time I’ll do an updated ranking of the top people to follow.

So I’m in the frame of mind to step back and take a look at the evolution of the financial Twittersphere.

And what I see is not great.

  • Churn. Losing great people like Kelly Evans last summer.
  • Lack of growth. Churn has always been a factor, people fall in love with social media and burn out after a couple of years. But the all-stars like Kelly Evans are not being replaced like they used to be.
  • There’s a subset of influential FinTwit people who protect accounts, allow only approved people to follow them. Or who regularly delete all their tweets, so no one can start a shitstorm over something they said a year ago. Which hurts discovery. And discovery is hard enough. Which is one of the reasons I created StreetEYE, so I would have a way of systematically finding the top people to follow.
  • And the usual garbage troll accounts devoted to stirring up bullshit. Some of which are surprisingly popular. It turns out a judicious mix of clickbait bullshit and timely entertaining commentary is a good way to amass a huge following. (Even if following those folks is a money- and sanity-losing game).
  • And the elephant in the room is Trump. Politics is all people talk about, even in the financial Twittersphere. Those Trump posts are great for engagement, but they suck out all the oxygen for intelligent conversation about markets and economics. (And are possibly terrible for Twitter, if the non-hyperpartisans start tuning out. Ever-increasing vitriol and engagement, ever-diminishing reach.)

The relatively low quality of online discussion is the thread that brings all of those together. Tragedy of the commons, adverse selection, I guess.

Twitter is great. So why does it suck so much?

We generally think most people think more or less similarly to us. We are astounded when we encounter cargo-cultists, flat-earth believers, whole societies of magical thinkers.

When we go online, we find that while our own process of social construction of reality is pretty similar to other people’s, it takes us to very different places.

If we’d been born where they were born and experienced what they experienced, would we really believe what they believe?

It can be mind-expanding that social media takes you outside your bubble, brings opposites together, like some virtual A train to Times Square. But it leads to conflict.

Now, to me, it’s pretty obvious that women often get a raw deal from society. I follow some smart, funny women who are pretty feminist. And not gonna lie, even though my left brain mostly agrees with them, the nursing of a litany of petty grievances, the constant mocking of white male privilege, ‘mansplaining’, ‘manspreading’, and whatnot, can get really annoying.1

There are also some males who are into, shall we say, non-female-friendly male culture. Pickup artists, ‘red pill’ and whatnot. Immature maybe. Lacking self-awareness and empathy. Assholes.

When those two mindsets encounter each other on social media, they’re not gonna have a good time.

Cognitive dissonance arises. Sparks fly. Much heat is generated and little light. And both sides walk away with even more strongly confirmed priors, that men and women on the other side are mean and nasty and out to oppress or emasculate them.

I share an alma mater with Barack Obama, I was a freshman when he was a senior. He talks like me, thinks like me. Well, I wish, because he’s smart and cool and funny. I’m predisposed to like him.

That same intellectual approach apparently offends a lot of people who see it as condescending.

When Donald Trump sees Obama, clearly he sees something totally different from me. I take personal offense at the whole birther thing and view it as an original sin that can never be expunged. But clearly it resonates with a lot of people, to my disgust and amazement.2

And the Trump supporters confronted with what is apparently my sort of ‘condescension’ just dig in, double down, and reinforce their views.

I don’t really know why Trump supporters support him. As Pascal said, “The heart has its reasons, which reason cannot know.” Or JP Morgan, “A man always has two reasons for the things he does – a good reason and the real reason.” I really don’t really believe ‘liberal condescension’ is the reason. That sounds a lot like the cognitive dissonance someone without self-awareness might experience when untenable positions meet inconvenient facts and reason.

Like being virulently anti-Muslim, and at the same time not understanding why Christian fundamentalist values arouse opposition.

You can’t reason someone out of an opinion they didn’t arrive at through reason in the first place. The answers are more likely to be found in mass psychology, Gustave Le Bon, in Goebbels, in the inability of Hillary Clinton to construct a narrative with mass appeal, to run as a candidate of change instead of as a machine politician of the establishment, to do electoral math (just like in 2008 vs. Obama).

As the world gets smaller, the ability to put yourself in others’ shoes a little bit is more important. It would help if people dialed down the online vitriol, learned to roll their eyes and go about their business, instead of going bananas over a dongle joke or someone wanting to tone down the pubescent male fantasy world of video games.

But civility has to be a two way street. The online hate against Obama was really something. Aligning yourself with that, positioning yourself as the leader of that, and calling people who disagree with you ‘enemies of the American people’ sure isn’t going to help.

People say there’s a problem with social ‘filter bubbles’. The underlying problem is parochialism, intolerance. People go bonkers when they encounter anything outside their norms. Was there really less of a filter bubble 40 years ago, when different parts of America were even more like different countries, with different languages, foods, music, brands, and there were only 3 networks with anodyne non-culture that filtered out anything controversial?

Is social media making us worse people? Is it making us dumber?

I’ve written a little bit about ‘fake news’, since I think I know a little about news and machine learning.

Cathy O’Neil has made a schtick out of saying that big data is a ‘weapon of math destruction‘ and problems like ‘fake news’ can’t be solved by machine learning.

She’s mostly wrong…the ‘fake news’ we’re talking about, of the ‘Pope endorses Trump’ variety, is easily detected by the 20% of the informed, critically thinking population, and so it can also be detected by robots with much better information about where it came from and how it spread. Google does a good job with spam and it’s essentially the same problem.

Cathy O’Neil is mostly right that big data relies on patterns of human-created data, that data will reflect human biases (an even more appalling example), big data is garbage in, garbage out, and you can’t dispense with old-fashioned evidence-based critical thinking, gumshoe reporting, to prime that pump of evidence-based reasoning.

And also that it may concentrate winners and losers and wealth. (Same may apply to passive investing, machine learning-based investing).

There’s a classic bias/variance tradeoff.

If you say, I’m only going to use Bloomberg-vetted information in investment decisions, you’re going to be slower to respond to new information than if you react to every tweet and blog and market rumor.

You need a filter that is adaptive enough to surface good social media experts without necessarily waiting til they become Bloomberg pundits, while not trusting every source of flackery, disinformation, and idiocy.

If everybody followed Cathy O’Neil’s advice, no one would ever have started reading some obscure but clever blogger like Cathy O’Neil, who owes her career as a big data pundit to social media, and maybe some big data-assisted discovery like Google searches, Twitter’s recommendation engine.

You need both shoe leather and tech. You need to be selective in which sources you trust, and you need technology to deploy against the armies of bots and data scientists looking to spam, deceive, and manipulate you.

Social media can make you smarter and quicker. You need both, the Cathy O’Neils and big data.

If everybody just watches CNBC, everybody gets super herdy (bias). Everybody watches different social media filters, that’s less herdy, maybe inefficient and noisy (variance).

Big data filters some of the noise, aggregators aggregate, Bloomberg goes and hires the Cathy O’Neils, that might eliminate some variance, bring back some bias, but maybe with an overall better level of discussion.

The noise level of the chattering and flaming on social media continues, occasionally something rises above the noise, gets picked up by aggregators, Bloomberg etc.

Big data isn’t inherently bad, in fact you need it on your side to to get maximum benefit from all the noise, to defeat the dark forces of spam and fake news. It’s an arms race, the forces of evil use the magic of big data and you need your own magic to counter their spell.

But it’s imperfect magic, when sophisticated, well-funded people finance CNS and Breitbart, and use sophisticated personalized marketing to raise them to a higher profile among their target audience than more balanced, fact-based sources, you need machine learning just to level the playing field a bit. And so much of the media is so self-serving and bought and sold by vested interests and you’re bombarded with so much garbage that there is no substitute for critical thinking.

On the whole these days, I’m probably more in agreement with Cathy O’Neil about big data tilting the playing field toward the forces of evil than I used to be, when it seemed everyone having access to all information anytime anywhere would be great for well-informed democracy.

Is the financial Twittersphere destined to be the Mos Eisley cantina of financial media? Does social media make participants more vile, primitive, and unhappy?

Facebook and social media are the McDonald’s of social interaction. Ubiquitous, convenient, enjoyable, not necessarily unhealthy if consumed mindfully and in moderation. But they are engineered to be highly addictive and appeal to most basic tastes and impulses.

The types of social interaction they favor are single-serving emo BS for ‘likes’. Extreme views. Comment wars. Trolling.

Sometimes there are positive viral movements like the Ice Bucket Challenge, petitions, GoFundMes for Jo Cox and Pulse victims.3

But often it’s pretty mean and nasty stuff.

One thing I think we should have learned is that Facebook’s real names work better than Twitter’s pseudonymity. (Which maybe Twitter is moving away from, gradually). Social media needs reputation management. People should be able to control whether random trolls can interact with them. Maybe people should need to accumulate reputation to post stuff, or for their posts to have reach, or people should be able to filter who can interact with them based on reputation. Maybe people should accumulate mod points to bump or bury others’ posts. But it should be transparent (which Facebook is not, at all).

Another thing we should have learned is, folks who spend their entire social lives in this highly engineered environment, are like people who only eat at McDonald’s, or never get off their couch from watching Fox or CNBC, or out of their cars.

It’s not just a ‘fake news’ problem. The whole social media ecosystem bias/variance tradeoff needs to be re-tuned for more quality and less noise.

Maybe social media would be better if there were mechanisms that encouraged people to limit their usage. Maybe it should cost reputation if you are constantly tweeting. Maybe there should be options to remind you if you’ve been online more than an hour a day, or to cut yourself off entirely when you hit your daily budget.

There are ways to optimize for quality over quantity both in what people see, and how they contribute.4

The reach and activity might go down a little, and the quality might go up a lot, paradoxically increasing reach and activity in the long run.

I don’t know how I feel about Zuck’s manifesto until it produces some real features and products (see here and here) but social media, like a mall, is a highly engineered experience, and it needs some intelligent design to not be terrible and not make the world worse.

Maybe there’s a happy medium between the Twitter free-for-all and more closed communities like SumZero and Value Investors Club. Where the worst noise is disincentivized and good stuff rises above the noise.

Social media gave us Trump. And Trump is shaping up to be a disaster. Ergo, social media is a disaster for civilization?

When I created StreetEYE, I thought social media, people freely sharing information, with the best information and the greatest people percolating to the top, was the way of the future. It hasn’t happened yet. The tools may need to evolve. It may take a new generation of platforms and tools. But it’s going to happen.

When we can give people like Kelly Evans the reason to come back.

On life’s vast ocean diversely we sail. Reason’s the card, but passion the gale. – Alexander Pope

Those who will not reason, are bigots, those who cannot, are fools, and those who dare not, are slaves. – Lord Byron


1 I can appreciate some gentle witty mocking, but to the more extreme women, maleness seems something offensive per se, while a dose of estrogen will excuse just about any misdeed. The black activist community picks battles judiciously by comparison. I’d go so far as to say most blacks are more judgmental about black lawbreakers than most white folks. Anyway, that’s one way I read those statistics showing white cops are less likely to shoot blacks than black cops. You pretty much have to kill an unarmed child to get a black protest movement going, whereas some women seem on the edge of violent uprising over a dongle joke or ‘manspreading’.


2 Blinding glimpses of the obvious: am not a Trump fan, and he’s a very weird guy, he’s outside the norms of politics, reality, decency. And not in the good way that shakes things up.

If a political opponent is going to say Obama may not be a citizen, or didn’t go to Columbia, or didn’t deserve his spot at Harvard, it doesn’t tell me anything about Obama, it tells me about the person who’s spreading that.

I have friends with personal experience with selecting editors of school newspapers, law review, etc., and your peers don’t select you unless they think 1) you’re one of the top people and 2) a decent guy they’re going to get along with. Anyone who says Obama was somehow undeserving or not legit is 1) ignorant, 2) of poor moral character, or 3) pandering to people in those groups. Which group do you think Trump is in?

One can not like Obama’s politics or as a human being but a lot of the conversation about him reflects very poorly on us as a country.

I get personally offended by people who promote the notion Obama was not legit, and Trump based his whole career off that. Where the f*** are Trump’s Fordham transcripts that somehow got him into Wharton? We know something about Ivy affirmative action for the rich too. Did Trump have the grades or scores to get into any grad school, let alone Harvard, let alone be President of Law Review? The guy doesn’t read books.

If Trump believes 10% of what comes out of his own mouth he’s 100% delusional. People really see what they want to see, and Trump is Exhibit 1 of that kind of crazy-ass magical thinking, and his supporters are in the same category … how do they not see he’s a promoter, has no deep interest in policy or ideology, knows practically nothing about economics or foreign affairs, cares about no principle or ethic beyond gratifying his ego and the chips on his shoulder, has angry, divisive views, indecent behavior that should have disqualified him, is supported by unabashed Nazis. He says stuff politicians don’t say, because when they say it and people believe it, it sinks us as a nation.

And yeah, seems kind of odd that he picks feuds with NATO, EU, Merkel, China, the Fed, the CIA, Mexico, but one country loves him and he has nothing bad to say about them…Weird! As someone once said, ‘there’s something going on there.’

He’s got the ‘B’ and ‘C’ string appointed to the Cabinet. I mean, can you even imagine Dimon or Blankfein or Paulson working for this guy? They wouldn’t even lend the guy a fiver. And on the foreign policy side it’s even worse, he has the entire national security establishment on his blacklist. And the guys who aren’t blacklisted are wary of working for him. I’m not even so worried about Trump because he’s a clown. He could start a war but I think the military and Congress wouldn’t let him. Unless there’s a dirty bomb in NY or DC and then all bets are off.

But when Trump craters, a lot of people are going to be very very angry, and I don’t think they’re going to turn to an establishment type, they’re going to turn to another candidate of change on the left or right. And I’m worried about stability in the rest of the world. If the US and the UK that had fairly rational economic policy are going populist, what the hell is going to happen in southern Europe in the next downturn? What about India, and Turkey, and the rest of the Middle East?


3 Viral memes giveth, and viral memes taketh away. Social media creates a Jo Cox murder or a Comet Ping Pong or a Dylann Roof, and then mobilizes to helps the family of the victims. Doesn’t quite even out.


4 Facebook has all the data, they can optimize for anything: duration of interaction on items, clicks through to items, people you frequently message, are tagged with, are in the same location with via your smartphone. They have an incentive to optimize for things that give them more ad revenue in the short run, but in the long run also for things that increase the depth and quality of interaction.

 

Some fun data-mining of StreetEYE headlines

Well, congrats to the Patriots and all my Boston homies. That was a catch and a comeback for the ages. Huger than Joe Montana back in the 80s, maybe GOAT. Y’all still should still learn to talk and to drive like normal human beings, but enjoy a legendary victory.

So, last year I did a word cloud of most common terms in 2015 StreetEYE headlines. Somehow I never got around to it around New Years this year. So here it is! (click to embiggen)

2016 StreetEYE Headline Word Cloud

Word cloud 2016

Interesting to compare … ‘Trump’ was yuge, and ‘Brexit’ was the other big one. ‘Greece’ was big in 2015, and faded like <cough> the Atlanta Falcons.

OK, while we’re clearing up unfinished business, here are the top clicked stories of 2016.

1. Economics on Buying vs Renting a House
2. A hedge fund has laid out why it is closing — and it is enough to set alarm bells ringing everywhere
3. Goldman Sachs Says It May Be Forced to Fundamentally Question How Capitalism Is Working
4. Trader exposes sexist horrors of the Wall Street ‘frat house’
5. FANG Is So 2015… BARF In 2016
6. With a single vote, England just screwed us all
7. Let Me Remind You Fuckers Who I Am, by @shitHRCcantsay
8. My very peculiar and speculative theory of why the GOP has not stopped Donald Trump
9. How Does This Hedge-Fund Manager Make So Much Money
10. Are US Stocks Overvalued?

Finally, if you’re REALLY into mad science…here’s a semantic analysis of > 1,000,000 headlines on StreetEYE (not just front page, everything that was shared by anyone on social media that we follow…this app does the analysis in your browser, so it will take a minute to download the data, and needs an up-to-date computer).

In the top right search, type a term, like ‘Trump’, then click on a completion, then click on ‘Isolate 101 data points’, and you’ll see something like the below (click to embiggen):

To try another search, choose “Show all data”, type in something new like “jpmorgan”, click on the completion, and you’ll see one like this.

What is the point? This is a way for computers to infer meaning of text based on context. Possibly it gives insight into how humans do it. A good representation of meaning can let us cluster related stories together. It can be used as an input to predict which stories will go viral, and improve the relevance and timeliness of headlines, or for other purposes like machine translation.

For the code that was used to generate the visualization inputs, see here.

Is Silicon Valley Truly Libertarian? Is Politics Society’s OS, Ripe For Disruption?

I heard it was you
Talkin’ ’bout a world
Where all is free
It just couldn’t be
And only a fool would say that – Steely Dan

A hero is someone who understands the responsibility that comes with his freedom. – Bob Dylan

Here’s a ramble that’s been hanging in my drafts for a while: Why do so many tech folks identify as libertarian? What do they mean? Is politics ripe for tech disruption? It’s a brand new year, and all old posts must go!

Now, I tend to lean, as the cliché goes, socially liberal, fiscally conservative, market-oriented. So I have affinity for libertarian thinking. Libertarians emphasize human rights, think outside the box and reason from first principles. I love that.

But a lot of the people who identify as libertarian just take it too far. What gets me eye-rolling is the person who, whenever something is messed up, thinks “OMG, if the government would step out of the way, a market solution would appear and work so much better!” … Even for activities which historically have a strong government role, roads, schools, money and banking, safety standards, pollution, etc.

I’ve written (here and here) about why I’m not a fan of that strand of libertarianism.

If you think the notion of immaculate, spontaneous order is a straw man, oh boy, just go to a Gary Johnson event and hear him questioned about whether it’s OK for the state to issue drivers licenses and speeding tickets. Not to mention debates about whether the Civil Rights Act of 1964 is a worse infringement on liberty than Jim Crow and segregation, or it’s moral for the government to tax people to save them from an asteroid impact, or whether parents should have the legal right to not feed kids and let them die. These aren’t just thought experiments to them, the libertarian scene is a hotbed of non-seriousness to downright kookiness.

I like freedom as much or more than most. But the idea that freedom is, in and of itself, a sufficient condition for desirable outcomes for everything has never seemed a compelling, complete theory that can deal with public goods, coordination problems, financial stability, most of the things we consider economic policy. It just begs the question, it’s the ‘assume a can opener’ of political ideologies. Once you give everyone freedom, you still have to design a market system and system of laws to solve social problems, define and assign property rights, decide what gets traded and how1.

Policies are market designs. Markets are sophisticated mechanisms, complex social institutions that have highly co-evolved with economic and political frameworks.

There is no such thing as a pure free market, just as there’s no such thing as a pure human ‘state of nature’. Free markets depend on laws, cultural norms, technology which are like David Foster Wallace’s water. They may seem natural and invisible because we’ve internalized them…but try explaining them to an alien or an uncontacted Amazon tribe.

Laissez-faire is not even wrong, it’s not even a thing. Free markets are games within a game, they are institutions humans create to serve objectives. They have to obey natural laws but they don’t exist in a social and cultural vacuum, they are intensely developed to be fit for purpose.

And market designs involve values and choices. Freedom is not the only human value that matters. Policy involves difficult tradeoffs between freedom, some notion of effectiveness, and some notion of fairness. Here’s a fancy graph:

Venn diagram of ideologies
(3D version which seemed more logical but turned out pretty hard to read)

There ain’t no such thing as a free lunch, but also, there ain’t no such thing as a true free market.

That’s my political science ‘theory of everything’. (Inspired by Keynes) People organize to pursue their individual and group values2. They can choose to maximize different values. One universal value is freedom, or maximum power to the individual. Another universal value is fairness within the in-group according to some value system… each accorded status, power, material reward in accordance to their contribution and merit, with no one taking undue advantage of anyone else. Another value is ‘efficiency’, which is a catch-all for other possibly non-universal group values: economic efficiency and growth, national power and world supremacy, religious or sectarian values.

Extreme pursuit of individual freedom leads to libertarianism. Extreme pursuit of fairness leads to communism. Extreme pursuit of narrow group values at the expense of freedom and fairness leads to fascism.

Are tech types libertarian in that extreme sense of primarily maximizing freedom? Clearly not!

The vast majority of Silicon Valley political contributions go to Democrats. A tiny sliver of CEOs identifies as Republican, sometimes keeps it secret, or wears it as a badge of contrarianism. Someone like Travis Kalanick used Ayn Rand as an avatar, but loves Obamacare, which lets gig-ers get healthcare without putting the onus on their employer/platform.

The simplistic stereotype is that the tech elite are Randian technocrats who just want the government to get out of the way while they solve the big problems that confound the politicians, lobbyists, and bureaucrats, and make the world a better place.

But I think the proper analogy is that Silicon Valley techies view government as code, the operating system that defines the operation of society…an operating system maybe overdue for disruption and a major upgrade. And Silicon Valley types think they know code, know complex systems, and are just the ones to do it.

Let’s explore the analogy a little.

• Government defines rules between the people, corporations, the administrations, foreign policy. An operating system defines rules, protocols, APIs between the users, administrators, programs and processes, hardware, networking with other computers.

• Government defines a market design that allocates public goods between competing interests. An operating system defines rules that allocate hardware (e.g. CPU time, peripherals) between different processes and users.

• Like an operating system, you want government to have as small a footprint as possible, use as few resources as possible, and give users and developers maximum freedom to fully utilize all the computing resources at their disposal.

• Like computers, economies have grown bigger, more powerful, and more complex. Both operating systems and governments have taken on greater roles over time. Add multiprocessing, now you need a good scheduler, memory protection. Add multi-user and networking, need much better permissioning and security.

As systems get better and more complex, users expect more and more from the OS in terms of services, coordinating and scheduling complex tasks, security, supporting complex peripherals. So, as the system gets better, the percentage of work done by the OS vs. individual programs tends to grow, and the resources it uses grow as well.

That’s an essential paradox of operating systems and government. A better society is one where individuals are maximally empowered, and can also coordinate with others in complex ways. It’s a hallmark of improving civilization to give more people access to a more complex network of interactions, because they are safe, they have good norms of behavior and trust each other, they have clear laws well enforced, they have access to basic services. The more complexity is supported, the better the society. More complexity and interaction tends to mean more complex laws and regulation, and more shared resources supported by government.

That government is best which governs least, but as society grows more complex, the bar will tend to rise.

• In code, you want to abstract problems into modules that solve parts of a problem well. In politics you have the concept of subsidiarity: national, state, local governments; solve every problem at the lowest level of government that can effectively deal with it.3

• The qualities that make good code are agreed upon…efficient use of resources, easy to use, simple to understand, easy to read and maintain. But code involves hard tradeoffs…fast, good, or cheap? Pick any two. What actually is good code is partly subjective. People get into religious wars about methodologies and languages…agile vs. extreme…functional vs. object-oriented, etc.

Premature optimization is the root of all evil. Government changes very slowly, tech changes very rapidly. So any optimization in government, i.e. writing really detailed regs applicable to the current state of technology and the economy, is generally a premature optimization. Therefore government is the root of all evil? Well anywhere there’s something really wrong it’s probably a government failure, by definition. But it’s a catch-22, you don’t optimize, government is bad; you optimize, and eventually you end up in an unplanned-for situation where the optimization makes things worse.

Checks and balances slow things down. They’re sort of like bias and variance. You want to tune your system so that it adapts to change…but if it responds too quickly you can get instability and overfitting. You want your government to respond to the people and to changing circumstances, but not pivot abruptly on every whim. Direct democracy can be a double-edged sword. <cough>Brexit, Proposition 134<cough>.

• Sometimes you need to patch code pretty quickly and you accumulate technical debt, expedient solutions which need to be more carefully implemented later, or create maintenance and other problems in the long run. Hard cases make bad law…the solution that seems just in a complex case doesn’t always generalize. You don’t want to optimize code too much for the current use case, you need to be flexible. And you need laws that reflect universal principles. Sometimes worse is better…a simple, cheap solution everyone understands is better than a highly optimized, over-engineered one.

The euro is the ultimate in political ‘technical debt’. Build something that works today, even though it will need major re-engineering to be robust over the long term, and hope you can implement version 2 in time to avoid collapse.

• As you add complexity, a well-designed system can iteratively get better and better. Up to a point. There comes a tipping point where it becomes impossible to maintain and iterate on. Sometimes you accumulate too much technical debt, or change is just too fast, and you need to start over from scratch. Security exploits, bugs multiply. The same may be true of society, the tree of liberty, etc.

• Any computer system can get hacked. When you have a good market design, it’s a good basis for powerful interaction and coordination. But all systems can be gamed and exploited. Unless people are mostly decent and bad actors are sanctioned, you can get moral hazard and a tragedy of the commons.

• We don’t write systems or build hardware in which the components are told, here’s what we want to do, you guys work out the protocols5. APIs and protocols are specified in gory detail using things like RFCs.

• You can write awful systems in any language, and you can write pretty good systems in any language. It’s the same with ideologies. There are countries with moderately socialist orientations that work pretty well, and there are socialist countries which are disasters. There are countries with mostly market-oriented solutions that work pretty well, and there are free-market countries of weak-state and strong-state varieties which are disasters.

Personally, I’ll take Sweden over Pinochet’s Chile. In my opinion there are no countries at the extremes of any axis that work well over a long period of time. Somalia, with no working government, is in some sense the apotheosis of small government and maximum private freedom, and not exactly paradise on earth.

Ideologies are like development methodologies, people get pretty worked up about them, you need some methodology, but any reasonable methodology that isn’t applied so strictly it gets in the way is probably OK, and having decent developers is more important than which methodology they use.

If you’re concerned about Obamacare trampling on liberties, and not about people unable to get care, being bankrupted, facing impossible choices and preventable deaths, you might be an architecture astronaut caring more about buzzwords and ideological purity than actual solutions, outcomes, lives. The person who is free from government interference, because they’re dead, doesn’t notch it up as a win for liberty. The user wants code that works, and if it doesn’t work it doesn’t matter if it was made using scrum or Ruby.

Ultimately, I think the Silicon Valley brand of libertarianism is really the cult of disruption; healthy skepticism of government as premature or partial optimization and technical debt; nerd suspicion of wooly MBA/JD pointy-headed boss types; desire to empower people with tools, knowledge, ability to make their own choices, build their own solutions; when that requires a strong guiding hand in the form of code, or government intervention (education, net neutrality), so be it.

When you translate the rules of society into code, is it always a win for the freedom and power of the individual? Not unless you’re careful. Technology can be sustaining for totalitarianism as well as a disruptor. It can be a tool for surveillance, social control, databases of Muslims.

I like Uber, but a lot of the code is a pricing black box designed to maximize the value of Uber, not a free auction market. There are still questions of fairness to e.g. the disabled, blacks. You can use big data to understand customers better and offer them more and better choices, or selectively charge people more.

Build a true 2-way market where drivers can offer services and riders can bid for them transparently, maybe even based on a blockchain ledger, and that’s a true libertarian solution. If you work on free software, you are a true libertarian and a scholar and I salute you. Until then, you’re paying lip service to freedom while creating a new layer of code to regulate behavior, while maximizing your own rent-seeking.

It can sometimes look like tech hubris, liberty for me, the nerd, and not for thee, the unwashed mashes.

Since we’ve expanded the scope to theories of everything, what really separates left and right? Some interrelation between interests, learned values, psychological makeup:

  • How you feel about people making good choices on their own;
  • How you feel about authorities coming up with good policies and market designs, avoiding deadweight losses;
  • How you feel about markets, whether they are generally fair or some participants accumulate one-sided market power; whether externalities and market failures are important and common, relative to deadweight losses from interventions that interfere with markets like taxes, regs;
  • In particular whether labor market outcomes for individuals are fair; whether people generally get what they deserve in the market economy, or whether the distribution of wealth and income is unfair; whether people generally rise or fall to the level they deserve, or whether initial endowments of wealth, educational opportunity, ethnic biases are determining factors6;
  • How you feel about democracy and the likelihood of corruption and capture vs. the ability of government to work for common good;
  • How much slack we should give to the ‘losers’; how you feel about free riders;
  • How you feel about authority; are you oriented toward rules, and motivated by a sense of moral outrage, or outcomes, thinking moral rules are heuristics to lead to good outcomes; is your personality high in ‘conscientiousness’?
  • How you feel about the ‘other’; do you strongly identify with a group, are group values more important than individual values; is your personality high in ‘openness’ to the values of others?
  • Are you motivated by fear, or hope, is the world a dangerous, zero-sum place with internal and external threats, or do people generally work together for progress?

Decent people can solve problems using left and right solutions, just as programmers can solve problems with different languages. But there is good code and bad code. Indecent, extreme, and disagreeable people aren’t going to solve anything.

We’ve become a nation of architecture astronauts, spending our energy on ideological flame wars instead of shipping code that works.

There’s a fine line between being a critic and a contrarian, and being a hater tearing down the progress other people are trying to achieve. Once you get to the extreme of opposing all mandatory vaccinations, I would say you’ve left reason behind. Given what we understand about network effects, advocating freedom to engage in antisocial behavior without sanction seems like rejecting the categorical imperative and reason-based Western morality.

Elevating individual freedom to the highest and only moral good, and following that to its extreme conclusion on matters of asteroids and vaccinations, seems like bad code. In the extreme, it leads somewhere between indifference to the possibility bad choices may create human suffering, and outright cruelty.

Of course, gentle libertarian-oriented friends, I don’t ascribe cruelty to you, I merely urge you to ask yourself how far extremism in the name of liberty should be taken before it becomes a vice, or at best self-defeating.

Men of intemperate minds cannot be free. Their passions forge their fetters. — Edmund Burke

To suppose that any form of government will secure liberty or happiness without any virtue in the people, is a chimerical idea. — James Madison


1 Suppose individuals solve problems by freely agreeing on institutions like democratic governments with civil liberties and also taxes, police forces, prisons, some middle ground between non-violence and slavery? If people freely enter into a contract that imposes sanctions for violations, is it still freedom?

2 Yes, Virginia, people identify, and act with their tribe, nation, other institutions, not just as individuals. People have aspirations as individuals and also for their social unit. Have you ever been to a football game or other ‘sacred’ ritual where people dress, paint their face, eat and drink, sing, chant and play music in the proper manner to appease the universe and bring success to their in-group (and disaster to the out-group)? The original sin of communism and totalitarianism is not respecting the individual and private property, evaluating every individual choice as political and what it means for the state; the original sin of libertarianism is not respecting the drive to identify and find meaning through an in-group, ‘something bigger than me’, viewing every interaction as a trade, significant only to people as individual unitary atomistic me-bubbles.

Leftists tend to view market forces as somehow illegitimate when they are laws of human nature and game theory math. Libertarians tend to view everything as a market, and politics as somehow illegitimate. Markets and politics are just a couple of important, well-defined games of organizational and social control among many, such as social status.

3 If you think this is an easy problem, think again. Refactoring is a bitch. Some argue the 2nd Ave Subway costs more because it had to go deep below all existing infrastructure. If one entity owned all the buried infrastructure, you wouldn’t dig up the road one week to fix electricity and the next to fix water, you would presumably do everything in one shot, or redesign an integrated subway/water/gas/electric/telecom package and have something more maintainable that saved money. But then every time you need an electric fix, you have to go through the monolithic infrastructure agency, maybe wait until you can schedule water/gas/electric simultaneously. Sometimes you need agencies that cross jurisdictions, like the Port Authority which coordinates transport activities that impact the NYC area with NY and NJ. Maybe guns are the poster boy for this problem. You can’t regulate guns in Chicago if they are unregulated in Mike Pence’s Indiana. So I would prefer a national registration system, with local authorities in charge of local regs, but then at least they would be able to look up what residents bought in another state, require training, insurance, follow the trail of guns used in crime.

4 Proposition 13, which freezes California property taxes at 1975 levels, achieves the unusual feat of being rash direct democracy (variance), and locking in an outdated policy over the long term (bias). Premature sub-optimization.

5 Actually, that’s what machine learning is. But even in supervised learning, you define how the components can interact, and what they are trying to optimize and iteratively improve. Sci-Fi time! Imagine a society of the future where a giant computer is taught the human happiness ‘loss function’ and some kind of mother ‘hello Google’ and smartphone notifications directs everyone in the most optimal way.

6 Case in point: Civil rights. Was Jim Crow really more tolerable than the Civil Rights Act? Under normal circumstances a law targeting specific groups and specific outcomes, as opposed to universal principles, is bad code. But it’s perverse to deny blacks the right to participate in democracy, enforce a system of segregation by extralegal means and unequal application of the law, and then say that system is less of a cruel insult to freedom than a law that ends it. When that system was maintained in many cases by unequal application of equal laws such as bogus ‘literacy tests’, I fail to see an alternative to mandating reasonable outcomes as a last resort in this sort of extreme situation. It’s hard to do social science without thinking about outcomes; Noble principles and intentions should be eventually be checked against results. It’s called evidence-based policy. Ambrose Bierce said politics is a contest of interests, masquerading as a contest of principles. Better people acknowledge they are fighting for their interests (outcomes), but also think of others and shared principles. The worst sort actually believe they are fighting on principle, and only their opponents are fighting for their own narrow interests. Whatever your ideology, at the end of the day you have to ship code that works.

Buyer’s markets, seller’s markets and the hedge fund hype cycle

First come the innovators…Then come the imitators…And then come the idiots — Warren Buffett

I was playing poker with hedge fund CTO types over the holidays and they were debating whether they were underpaid.

Given some of the skills, years of training, responsibilities, trust, hours involved, some argued, it’s lamentable that senior IT folks get paid mid 6 figures, when mid-level investment staff make 7 figures.

(A rule of thumb in a well-established hedge fund might be, in a good year the top guy makes 9 figures, the most senior trader/analyst/PM types make 8 figures, the workhorse mid-level folks make 7 figures. While there are a few exceptional hedge fund CTOs who are partners and earn on a par with senior management, most earn far less).

So, time for some game theory / micro-economics 101.

Suppose there are 100 hedge funds. They each need a CTO. Suppose there are 101 guys who have the right skills and training to be CTOs. If you hire a guy who doesn’t have the chops, the fund has a 25% chance of a crippling crash, cyber-attack, trading system outage that puts you out of business.

Now, 100 guys will get jobs, and 1 will be left unemployed. That guy will send out his résumé and basically be willing to take someone else’s job for any meaningful raise over his best alternative job. And any fund who gets that guy’s résumé is going to tell the guy with a huge partnership package to take a pay cut, or take a figurative hike. And now a new guy will be in the same position of sending out his résumé, and so on until every CTO in the industry is getting a really basic pay package.

Now suppose there are 100 hedge funds, but now there are 99 guys with the qualifications to be CTO. One hedge fund is going to be left out in the cold bearing that risk. How much should they bid to steal away a CTO from the other firm? The answer is, A LOT. In the short run you could justify giving them anything up to 25% of the value of firm every year, and still come out ahead.

OK, no one is going to get 25% of the value of the firm every year, it’s not really a stable long term solution. The IT guy ends up with 95% of the equity after 10 years. And then the PM says, hey, if I leave there’s a 100% chance the place is out of business. So in practice the equilibrium is, the top guys whose talents are scarce, ‘A’ player portfolio managers, traders, etc., end up partners, and the equity share each one gets is proportional to their ‘marginal product’: the value of the firm with an ‘A’ player in that spot compared to the value of the firm with a ‘B’ player in that spot.

And that’s the difference between a buyer’s market and a seller’s market. If there are more buyers than sellers, the sellers set the price. If there are more sellers than buyers, the buyers set the price. 1

It doesn’t matter how big the market imbalance is. A small shift in either supply or demand can tip the balance. And once market participants realize the other side now has the upper hand, the impact on price can be enormous.

And when it becomes a buyer’s market for the hedge funds themselves, everybody’s gonna have a bad time.

For a while, from the 90s to the end of the dot-com crash, superstar hedge fund managers seemed to have an edge. Everybody wanted in on that deal. The business got crowded. Returns suffered. And maybe a shakeout, a popping of the bubble, is at hand.

There’s a parallel to the technology ‘hype cycle.’ In the old days large investment managers had an edge. You needed big budget IT, fund accounting, marketing and distribution, scale to get management to talk to you. The advent of PCs, Bloombergs, the Internet changed that. An ecosystem sprang up of technology partners for hedge fund order management systems, portfolio accounting systems, CRMs, research analytics…along with hedge technology integrators, administrators, prime brokers.

In the 80s and 90s you had a small number of pioneers who did extremely well, Soros, Steinardt, Robertson. You had a slope of hope, which eventually leads to a peak of inflated expectations, followed by a trough of disillisionment. Finally, you reach an equilibrium plateau of productivity.

Hedge funds are a star vehicle. Stars come and go, and only a few have staying power. Sometimes, when there’s enough demand, stardom can be manufactured, and even mediocrity gets rewarded. When there’s no demand, talent has to fight harder to be recognized. But talent usually finds its own level, genius usually gets rewarded, and mediocrity usually becomes a commodity.

1 In the classical economic model, wage earners get paid their marginal product of labor. In the economics described by Marx, wage earners get paid a subsistence wage. The difference is simply between a buyer’s market or a seller’s market for labor. Typically, we assume that as technology improves, increasing the productivity of both labor and capital, they stay in balance, and the increased productivity of labor means employers are willing to pay more for labor. In the Solow growth model or Piketty, if capital is easily substituted for labor, as technology improves continuously, eventually you end up with a buyer’s market for labor, and owners of capital appropriate more and more of the gains from growth. We’ve never seen this sort of economic singularity, where overall wealth grows as robots build more robots while wage earners sink to a subsistence level. But just because something never happened before doesn’t mean it can’t happen.

Christmas Story, by H. L. Mencken

THIS STORY was first printed in the New Yorker, to whose editors I am indebted for permission to republish it. In writing it I had valuable suggestions from my brother, August Mencken.

H.L.M

CHRISTMAS STORY

Despite all the snorting against them in works of divinity, it has always been my experience that infidels— or free-thinkers, as they usually prefer to call themselves— are a generally estimable class of men, with strong overtones of the benevolent and even of the sentimental. This was certainly true, for example, of Leopold Bortsch, Totsaufer (1) for the Scharnhorst Brewery, in Baltimore, forty-five years ago, whose story I have told, alas only piecemeal, in various previous communications to the press. If you want a bird’s-eye view of his character, you can do no better than turn to the famous specifications for an ideal bishop in I Timothy III, 2-6. So far as I know, no bishop now in practice on earth meets those specifications precisely, and more than one whom I could mention falls short of them by miles, but Leopold qualified under at least eleven of the sixteen counts, and under some of them he really shone. He was extremely liberal ( at least with the brewery’s money ) , he had only one wife ( a natural blonde weighing a hundred and eighty-five pounds ) and treated her with great humanity, he was ( I quote the text ) “no striker . . . not a brawler,” and he was preeminently ‘Vigilant, sober, of good behavior, given to hospitality, apt to teach.” Not once in the days I knew and admired him, c. 1900, did he ever show anything remotely resembling a bellicose and rowdy spirit, not even against the primeval Prohibitionists of the age, the Lutheran pastors who so often plastered him from the pulpit, or the saloonkeepers who refused to lay in Scharnhorst beer. He was a sincere friend to the orphans, the aged, all blind and one-legged men, ruined girls, opium fiends, Chinamen, oyster dredgers, ex-convicts, the more respectable sort of colored people, and all the other oppressed and unfortunate classes of the time, and he slipped them, first and last, many a substantial piece of money.

(1) A Totsaufer (literally, dead-drinker) is a brewery’s customers’ man. One of his most important duties is to carry on in a wild and inconsolable manner at the funerals of saloonkeepers.

Nor was he the only Baltimore infidel of those days who thus shamed the churchly. Indeed, the name of one of his buddies, Fred Ammermeyer, jumps into my memory at once. Fred and Leopold, I gathered, had serious dogmatic differences, for there are as many variations in doctrine between infidels as between Christians, but the essential benignity of both men kept them on amicable terms, and they often cooperated in good works. The only noticeable difference between them was that Fred usually tried to sneak a little propaganda into his operations— a dodge that the more scrupulous Leopold was careful to avoid. Thus, when a call went out for Bibles for the paupers lodged in Bayview, the Baltimore almshouse, Fred responded under an assumed name with a gross that had to be scrapped at once, for he had marked all the more antinomian passages with a red, indelible pencil— for example, Proverbs VII, 18-19; Luke XII, 19; I Timothy V, 23; and the account of David’s dealing with Uriah in II Samuel XI.

Again, he once hired Charlie Metcalfe, a small-time candy manufacturer, to prepare a special pack of chocolate drops for orphans and ruined girls with a deceptive portrait of Admiral Dewey on the cover and a print of Bob Ingersoll’s harangue over his brother’s remains at the bottom of each box. Fred had this subversive exequium reprinted many times, and distributed at least two hundred and fifty thousand copies in Baltimore between 1895 and 1900. There were some Sunday-school scholars who received, by one device or another, at least a dozen. As for the clergy of the town, he sent each and every one of them a copy of Paine’s “Age of Reason” three or four times a year— always disguised as a special-delivery or registered letter marked “Urgent.” Finally, he employed seedy rabble rousers to mount soap boxes at downtown street corners on Saturday nights and there bombard the assembled loafers, peddlers, and cops with speeches which began seductively as excoriations of the Interests and then proceeded inch by inch to horrifying proofs that there was no hell.

But in the masterpiece of Fred Ammermeyer’s benevolent career there was no such attempt at direct missionarying; indeed, his main idea when he conceived it was to hold up to scorn and contumely, by the force of mere contrast, the crude missionarying of his theological opponents.

This idea seized him one evening when he dropped into the Central Police Station to pass the time of day with an old friend, a police lieutenant who was then the only known freethinker on the Baltimore force. Christmas was approaching and the lieutenant was in an unhappy and rebellious frame of mind— not because he objected to its orgies as such, or because he sought to deny Christians its beautiful consolations, but simply and solely because he always had the job of keeping order at the annual free dinner by the massed missions of the town to the derelicts of the waterfront, and that duty compelled him to listen politely to a long string of pious exhortations, many of them from persons he knew to be whited sepulchres.

“Why in hell,” he observed impatiently, “do all them goddam hypocrites keep the poor bums waiting for two, three hours while they get off their goddam whim wham? Here is a hall full of men who ain’t had nothing to speak of to eat for maybe three, four days, and yet they have to set there smelling the turkey and the coffee while ten, fifteen Sunday-school superintendents and W.C.T.U. sisters sing hymns to them and holler against booze. I tell you, Mr. Ammermeyer, it ain’t human. More than once I have saw a whole row of them poor bums pass out in faints, and had to send them away in the wagon. And then, when the chow is circulated at last, and they begin fighting for the turkey bones, they ain’t hardly got the stuff down before the superintendents and the sisters begin calling on them to stand up and confess whatever skulduggery they have done in the past, whether they really done it or not, with us cops standing all around. And every man Jack of them knows that if they don’t lay it on plenty thick there won’t be no encore of the giblets and stuffing, and two times out of three there ain’t no encore anyhow, for them psalm singers are the stingiest outfit outside hell and never give a starving bum enough solid feed to last him until Christmas Monday. And not a damned drop to drink! Nothing but coffee— and without no milk! I tell you, Mr. Ammermeyer, it makes a man’s blood boil.”

Fred’s duly boiled, and to immediate effect. By noon the next day he had rented the largest hall on the water-front and sent word to the newspapers that arrangements for a Christmas party for bums to end all Christmas parties for bums were under way. His plan for it was extremely simple. The first obligation of hospitality, he announced somewhat prissily, was to find out precisely what one’s guests wanted, and the second was to give it to them with a free and even reckless hand. As for what his proposed guests wanted, he had no shade of doubt, for he was a man of worldly experience and he had also, of course, the advice of his friend the lieutenant, a recognized expert in the psychology of the abandoned.

First and foremost, they wanted as much malt liquor as they would buy themselves if they had the means to buy it. Second, they wanted a dinner that went on in rhythmic waves, all day and all night, until the hungriest and hollowest bum was reduced to breathing with not more than one cylinder of one lung. Third, they wanted not a mere sufficiency but a riotous superfluity of the best five-cent cigars on sale on the Baltimore wharves. Fourth, they wanted continuous entertainment, both theatrical and musical, of a sort in consonance with their natural tastes and their station in life. Fifth and last, they wanted complete freedom from evangelical harassment of whatever sort, before, during, and after the secular ceremonies.

On this last point, Fred laid special stress, and every city editor in Baltimore had to hear him expound it in person. I was one of those city editors, and I well recall his great earnestness, amounting almost to moral indignation. It was an unendurable outrage, he argued, to invite a poor man to a free meal and then make him wait for it while he was battered with criticism of his ways, however well intended. And it was an even greater outrage to call upon him to stand up in public and confess to all the false steps of what may have been a long and much troubled life.

Fred was determined, he said, to give a party that would be devoid of all the blemishes of the similar parties staged by the Salvation Army, the mission helpers, and other such nefarious outfits. If it cost him his last cent, he would give the bums of Baltimore massive and unforgettable proof that philanthropy was by no means a monopoly of gospel sharks— that its highest development, in truth, was to be found among freethinkers.

It might have cost him his last cent if he had gone it alone, for he was by no means a man of wealth, but his announcement had hardly got out before he was swamped with offers of help. Leopold Bortsch pledged twenty-five barrels of Scharnhorst beer and every other Totsdufer in Baltimore rushed up to match him. The Baltimore agents of the Pennsylvania two-fer factories fought for the privilege of contributing the cigars. The poultry dealers of Lexington, Fells Point, and Cross Street markets threw in barrel after barrel of dressed turkeys, some of them in very fair condition. The members of the boss bakers’ association, not a few of them freethinkers themselves, promised all the bread, none more than two days old, that all the bums of the Chesapeake littoral could eat, and the public-relations counsel of the Celery Trust, the Cranberry Trust, the Sauerkraut Trust, and a dozen other such cartels and combinations leaped at the chance to serve.

If Fred had to fork up cash for any part of the chow, it must have been for the pepper and salt alone. Even the ketchup was contributed by social-minded members of the Maryland canners’ association, and with it they threw in a dozen cases of dill pickles, chowchow, mustard, and mincemeat. But the rent of the hall had to be paid, and not only paid but paid in advance, for the owner thereof was a Methodist deacon, and there were many other expenses of considerable size— for example, for the entertainment, the music, the waiters and bartenders, and the mistletoe and immortelles which decorated the hall. Fred, if he had desired, might have got the free services of whole herds of amateur musicians and elocutionists, but he swept them aside disdainfully, for he was determined to give his guests a strictly professional show. The fact that a burlesque company starved out in the Deep South was currently stranded in Baltimore helped him here, for its members were glad to take an engagement at an inside rate, but the musicians’ union, as usual, refused to let art or philanthropy shake its principles, and Fred had to pay six of its members the then prevailing scale of four dollars for their first eight hours of work and fifty cents an hour for overtime. He got, of course, some contributions in cash from rich freethinkers, but when the smoke cleared away at last and he totted up his books, he found that the party had set him back more than a hundred and seventy-five dollars.

Admission to it was by invitation only, and the guests were selected with a critical and bilious eye by the police lieutenant. No bum who had ever been known to do any honest work— even such light work as sweeping out a saloon— was on the list. By Fred’s express and oft-repeated command it was made up wholly of men completely lost to human decency, in whose favor nothing whatsoever could be said. The doors opened at 11 a.m. of Christmas Day, and the first canto of the dinner began instantly. There were none of the usual preliminaries- no opening prayer, no singing of a hymn, no remarks by Fred himself, not even a fanfare by the band. The bums simply shuffled and shoved their way to the tables and simultaneously the waiters and sommeliers poured in with the chow and the malt. For half an hour no sound was heard save the rattle of crockery, the chomp- chomp of mastication, and the grateful grunts and “Oh, boy!”s of the assembled underprivileged.

Then the cigars were passed round (not one but half a dozen to every man ) , the band cut loose with the tonic chord of G major, and the burlesque company plunged into Act I, Sc. 1 of “Krausmeyer’s Alley.” There were in those days, as old-timers will recall, no less than five standard versions of this classic, ranging in refinement all the way from one so tony that it might have been put on at the Union Theological Seminary down to one so rowdy that it was fit only for audiences of policemen, bums, newspaper reporters, and medical students. This last was called the Cincinnati version, because Cincinnati was then the only great American city whose mores tolerated it. Fred gave instructions that it was to be played a outrance and con fuoco, with no salvo of slapsticks, however brutal, omitted, and no double entendre, however daring. Let the boys have it, he instructed the chief comedian, Larry Snodgrass, straight in the eye and direct from the wood. They were poor men and full of sorrow, and he wanted to give them, on at least one red-letter day, a horse doctor’s dose of the kind of humor they really liked.

In that remote era the girls of the company could add but little to the exhilarating grossness of the performance, for the strip tease was not yet invented and even the shimmy was still only nascent, but they did the best they could with the muscle dancing launched by Little Egypt at the Chicago World’s Fair, and that best was not to be sneezed at, for they were all in hearty sympathy with Fred’s agenda, and furthermore, they cherished the usual hope of stage folk that Charles Frohman or Abe Erlanger might be in the audience. Fred had demanded that they all appear in red tights, but there were not enough red tights in hand to outfit more than half of them, so Larry Snodgrass conceived the bold idea of sending on the rest with bare legs. It was a revolutionary indelicacy, and for a startled moment or two the police lieutenant wondered whether he was not bound by his Hippocratic oath to raid the show, but when he saw the whole audience leap up and break into cheers, his dubieties vanished, and five minutes later he was roaring himself when Larry and the other comedians began paddling the girls’ cabooses with slapsticks.

I have seen many a magnificent performance of “Krausmeyer’s Alley” in my time, including a Byzantine version called “Krausmeyer’s Dispensary,” staged by the students at the Johns Hopkins Medical School, but never have I seen a better one. Larry and his colleagues simply gave their all. Wherever, on ordinary occasions, there would have been a laugh, they evoked a roar, and where there would have been roars they produced something akin to asphyxia and apoplexy. Even the members of the musicians’ union were forced more than once to lay down their fiddles and cornets and bust into laughter. In fact, they enjoyed the show so vastly that when the comedians retired for breath and the girls came out to sing “Sweet Rosie O’Grady” or “I’ve Been Workin’ on the Railroad,” the accompaniment was full of all the outlaw glissandi and sforzandi that we now associate with jazz.

The show continued at high tempo until 2 p.m., when Fred shut it down to give his guests a chance to eat the second canto of their dinner. It was a duplicate of the first in every detail, with second and third helpings of turkey, sauerkraut, mashed potatoes, and celery for everyone who called for them, and a pitcher of beer in front of each guest. The boys ground away at it for an hour, and then lit fresh cigars and leaned back comfortably for the second part of the show. It was still basically “Krausmeyer’s Alley,” but it was a “Krausmeyer’s Alley” adorned and bedizened with reminiscences of every other burlesque- show curtain raiser and afterpiece in the repertory. It went on and on for four solid hours, with Larry and his pals bending themselves to their utmost exertions, and the girls shaking their legs in almost frantic abandon. At the end of an hour the members of the musicians’ union demanded a cut-in on the beer and got it, and immediately afterward the sommeliers began passing pitchers to the performers on the stage. Meanwhile, the pitchers on the tables of the guests were kept replenished, cigars were passed round at short intervals, and the waiters came in with pretzels, potato chips, celery, radishes, and chipped beef to stay the stomachs of those accustomed to the free-lunch way of life.

At 7 p.m. precisely, Fred gave the signal for a hiatus in the entertainment, and the waiters rushed in with the third canto of the dinner. The supply of roast turkey, though it had been enormous, was beginning to show signs of wear by this time, but Fred had in reserve twenty hams and forty pork shoulders, the contribution of George Wienefeldter, president of the Wienefeldter Bros. & Schmidt Sanitary Packing Co., Inc. Also, he had a mine of reserve sauerkraut hidden down under the stage, and soon it was in free and copious circulation and the guests were taking heroic hacks at it. This time they finished in three-quarters of an hour, but Fred filled the time until 8 p.m. by ordering a seventh-inning stretch and by having the police lieutenant go to the stage and assure all hands that any bona-fide participant found on the streets, at the conclusion of the exercises, with his transmission jammed would not be clubbed and jugged, as was the Baltimore custom at the time, but returned to the hall to sleep it off on the floor. This announcement made a favorable impression, and the brethren settled down for the resumption of the show in a very pleasant mood. Larry and his associates were pretty well fagged out by now, for the sort of acting demanded by the burlesque profession is very fatiguing, but you’d never have guessed it by watching them work.

At ten the show stopped again, and there began what Fred described as a Bierabend, that is, a beer evening. Extra pitchers were put on every table, more cigars were handed about, and the waiters spread a substantial lunch of rye bread, rat-trap cheese, ham, bologna, potato salad, liver pudding, and Blutwurst. Fred announced from the stage that the performers needed a rest and would not be called upon again until twelve o’clock, when a midnight show would begin, but that in the interval any guest or guests with a tendency to song might step up and show his or their stuff. No less than a dozen volunteers at once went forward, but Fred had the happy thought of beginning with a quartet, and so all save the first four were asked to wait. The four laid their heads together, the band played the vamp of “Sweet Adeline,” and they were off. It was not such singing as one hears from the Harvard Glee Club or the Bach Choir at Bethlehem, Pennsylvania, but it was at least as good as the barbershop stuff that hillbillies now emit over the radio. The other guests applauded politely, and the quartet, operating briskly under malt and hop power, proceeded to “Don’t You Hear Dem Bells?” and “Aunt Dinah’s Quilting Party.” Then the four singers had a nose-to-nose palaver and the first tenor proceeded somewhat shakily to a conference with Otto Strauss, the leader of the orchestra.

From where I sat, at the back of the hall, beside Fred, I could see Otto shake his head, but the tenor persisted in whatever he was saying, and after a moment Otto shrugged resignedly and the members of the quartet again took their stances. Fred leaned forward eagerly, curious to hear what their next selection would be. He found out at once. It was “Are You Ready for the Judgment Day?,” the prime favorite of the period in all the sailors’ bethels, helping-up missions, Salvation Army bum traps, and other such joints along the waterfront. Fred’s horror and amazement and sense of insult were so vast that he was completely speechless, and all I heard out of him while the singing went on was a series of sepulchral groans. The man was plainly suffering cruelly, but what could I do? What, indeed, could anyone do? For the quartet had barely got halfway through the first stanza of the composition before the whole audience joined in. And it joined in with even heartier enthusiasm when the boys on the stage proceeded to “Showers of Blessings,” the No. 2 favorite of all seasoned mission stiffs, and then to “Throw Out the Lifeline,” and then to “Where Shall We Spend Eternity?,” and then to “Wash Me, and I Shall Be Whiter Than Snow.”

Halfway along in this orgy of hymnody, the police lieutenant took Fred by the arm and led him out into the cold, stinging, corpse-reviving air of a Baltimore winter night. The bums, at this stage, were beating time on the tables with their beer glasses and tears were trickling down their noses. Otto and his band knew none of the hymns, so their accompaniment became sketchier and sketchier, and presently they shut down altogether. By this time the members of the quartet began to be winded, and soon there was a halt. In the ensuing silence there arose a quavering, boozy, sclerotic voice from the floor. “Friends,” it began, “I just want to tell you what these good people have done for me— how their prayers have saved a sinner who seemed past all redemption. Friends, I had a good mother, and I was brought up under the in- fluence of the Word. But in my young manhood my sainted mother was called to heaven, my poor father took to rum and opium, and I was led by the devil into the hands of wicked men— yes, and wicked women, too. Oh, what a shameful story I have to tell! It would shock you to hear it, even if I told you only half of it. I let myself be . . .”

I waited for no more, but slunk into the night. Fred and the police lieutenant had both vanished, and I didn’t see Fred again’ for a week. But the next day I encountered the lieutenant on the street, and he hailed me sadly.

“Well,” he said, “what could you expect from them bums? It was the force of habit, that’s what it was. They have been eating mission handouts so long they can’t help it. Whenever they smell coffee, they begin to confess. Think of all that good food wasted! And all that beer! And all them cigars!”

On ‘fake news’, market designs, and the fascist/libertarian nexus

Arbitrary power is most easily established on the ruins of liberty abused to licentiousness. – George Washington

To suppose that any form of government will secure liberty or happiness without any virtue in the people, is a chimerical idea. – James Madison

Reportedly, a CNN screenshot?

Reportedly, a CNN screenshot?

It seems richly ironic that the utopian, nominally libertarian visionaries of Silicon Valley, the folks of the Whole Earth Catalog and Think Different, in creating a many-to-many Internet platform where everyone can communicate with everyone via social media, have provided the tools for mass manipulation, mass surveillance, and mob rule on a mass scale.

Social media is turning into a cesspool of tar-and-pitchfork mobs, hate, false but truthy viral memes, spam, and lame celebrity and corporate shilling.

Surely we can agree that it would be a bad thing for democracy if a demagogue, through effective control of mass media, were to persuade people he won an election when he didn’t, and that people who say otherwise are unreliable and that only he can solve the country’s problems, etc., etc.?

Is that actually happening? Seems a question of degree. If you post that Clinton won the popular vote, some moron will come back to say she didn’t really because of illegals, or uncounted absentee ballots, or some other crazy argument that boils down to, “we won all the votes if you don’t count the people who voted for her, who shouldn’t count anyway.”

Which sounds like a great excuse for further disenfranchisement, convenient purges of the voter rolls, voter ID laws. “Always accuse your adversary of whatever it is you yourself are attempting.”

Which sounds like an un-American disrespect for our democracy, and an excuse for further cynicism, and decay.

Then, if people want to take common sense steps to prevent the spread of lies, people cry media manipulation.

If someone wants to be a troll and write that Trump won the popular vote, I certainly don’t advocate censoring them or sending them to jail. If someone wants to pay Facebook to run an ad pointing to a site writing similar nonsense, that’s also fine, as long as it’s obvious it’s an ad and Facebook is not presenting it as journalism.

However, once a lot of people start sharing that nonsense as news and Facebook presents it as news, that’s a real problem. Facebook is presenting false and misleading fiction as news adhering to journalism school norms.

At that point Facebook is misleading people. Facebook should let people flag it as false. Their tools to do that are shite, protestations notwithstanding. And if something is consistently flagged as false, and someone is sharing it as news, Facebook should show a disclaimer, ‘the accuracy of this item/site has been disputed, do you want to learn more before you share?’ with stats on who and how many people are flagging it, and the links to evidence they cite. A little nudge goes a long way with most decent people. And if people still want to share it, then fine. I’m not advocating Chinese style memory-holing Tienanmen Square here. Just don’t present conspiracy theories, wingnut fiction, and blatantly manipulative propaganda in context as real news.

Commingling false propaganda with news diminishes the credibility of all news media, it reduces the value of Facebook’s platform, and it’s malware infecting the operating system of democracy.

If you want to believe nonsense, like that Trump won the popular vote or that the moon landing was faked and it’s made of green cheese, it’s your own problem. But if everyone believes nonsense, it’s a sick society. Mark Zuckerberg and Facebook need to decide if the ad money, engagement, and sidestepping criticism are worth poisoning the well.

There is no valid argument against a minimal news filter for fraud and manipulation. Is it OK that people searching for election results get as a the top result a fake page showing Trump won the popular vote, and Google should just let it slide? We take it for granted that Google should care about search quality.

It’s impossible to avoid applying a minimal filter to avoid spam, illegal or offensive content. Surely blatantly false and misleading content shouldn’t be given a pass, while napalm children are blocked.

Facebook already optimizes for engagement, they need to tune it a bit for anti-spam.

The objections boil down to, the only truth that matters is, if Facebook were to stop presenting nonsense under the rubric of news, it would hurt one party more than another. Truth is what serves the party line and the Great Leader.

The argument that it’s a slippery slope to Stalinist media control is a fallacy. The alternative to simple common sense, not calling demonstrably false things news, looks exactly like Pravda. (And I don’t think it’s a coincidence.) And if you follow the slippery slope argument, you can never do anything to improve markets, or help a starving person, because it’s a slippery slope to socialism, communism and dependence.

The good-faith argument that we should err on the side of freedom of speech is a good instinct. But you can’t allow lies to drown out truth, and dark, extreme forces and foreign actors to manipulate narratives so easily.

A problem I have with libertarian arguments in general is that they take free markets as given, when in fact they are extraordinarily complex institutions which depend on norms and laws enforced by government.

Before you can have a free market, you need a market design that works. And this applies to the marketplace for ideas as well as the free market for goods and services. (And it’s a key function of a news aggregator like StreetEYE.)

Otherwise, the libertarian argument boils down to, “I’ve got the money, I’ve got the power, I know how to game the system, eff you and your superior attitude and your ‘fairness’ and your market design, I’m going to impose the market design that benefits me.”

When really the libertarian argument should be, how do we create a market design with a minimum of rules and arbitrary government intervention that achieves the objectives of the market, where government can’t abuse its power, and where bad-faith actors and big market players can’t abuse their power.

Which is extremely hard. And takes a complex understanding of government, markets, and the true meaning of liberty as maximum freedom for responsible actors. And willingness to do the hard work of constantly improving markets and rules, instead of throwing up hands and obstructing the people willing to do the work.

And if taken too far, the attitude that laws are always the problem, not the solution, becomes a disease that makes the republic go down the drain, instead of the cure.

Without responsible action, libertarianism sows the seeds of fascism, and the greatest communication tools ever invented become tools to spread the greatest lies ever invented, and eventually the greatest tools for repression.

Decent people should realize that a society where any lie can be the truth, isn’t a society that can lead the world, or one worth having.

Everyone lives in a bubble, and all models are overfitted

I beseech you, in the bowels of Christ, think it possible you may be mistaken. – Oliver Cromwell

Real knowledge is to know the extent of one’s ignorance. – Confucius

So, a lot of people are saying the media/elite got the election wrong because they live in a ‘bubble’.

And some are saying the pollsters and forecasters were all wrong and ‘data science’ is bullshit.

Well, this guy models, sometimes, and if you say that, you’re all as full of shit as the pundits and forecasters, and in most cases far more so.

Sure…media, analysts, Silicon Valley live in a bubble…unlike rural farmers who take time out of each day to consult a broad cross-section of Americans. (via this guy)

Sure…Nate Silver is full of BS…but an unemployed coal-miner who thinks Obama’s birth certificate is fake and Trump is going to build a wall and bring back his job is keeping it real.

The only way anyone can make sense of reality is by filtering it. We all create our own bubbles.

The only reality anyone knows is socially constructed, and subject to our own heuristics and behavioral biases.

Any sufficiently powerful model will overfit to past (in-sample) data, compared to the future (out-of-sample). The only way to prevent that is to pad the error bars for what you don’t know you don’t know, and build in a bias toward uncertainty.

What forecasters do is fight the curse of dimensionality and try to find the bias/variance sweet spot.

Fighting dimensionality means trying to explain a lot of variables with a few, finding simple patterns in complex data. The problem is, reality is incredibly messy, and the messier it is, the easier it is to find spurious patterns.

You can have an ultra-simple model, like a simple linear regression, and it may not fit the data very well, because the underlying reality is not linear, or because multiple predictors impact the response.

These models are biased in the sense that they start from an opinion about the data, a linear response to a single predictor, that is not true in reality.

So then you add variables and relax the linearity assumption, and if you do a good job your model starts to fit the in-sample data well and predicts the future a little better.

But if you add enough complexity to your model, you start to fit the quirks in your data too well, and your out-of-sample prediction gets worse. Instead of being overly wedded to the bias of your a priori model, you are overly sensitive to random variance in the data you happen to have encountered.

The tradeoff looks like this, via Scott Fortmann-Roe:

biasvariance

The data scientist is looking for that trough in the black line, the right balance between underfitting and overfitting, and trying to understand reality as well as possible to make the trough as deep as possible.

One thing data scientists do is divide data into training and test sets. You fit a model on the training set and then use the test set to measure the training error. Now maybe you go back and try a bunch of models and see which one performs best. Well, guess what, now you’ve selected a model using the test set, so you are in that sense fitting to the test set. By regression to the mean, you are more likely to select a model that got at least a little lucky, and your future performance will not match the test set.

So now you split into 3 sets, training to fit your model, cross-validation to select and fine-tune the model, and a test set, which you never look at until you are ready to predict your out-of-sample error. That should work in theory. But in practice, after that you will at some point go back and make adjustments based on out-of-sample performance. It’s very hard to stick 100% to that principle, although most scientists do so well enough that most results are correct. (Cough…Not! And yet that’s the nature of good science.)

Pollsters ask questions to determine whether respondents are likely voters. Then they predict whether they will actually vote based on past election data, and adjust sample weights accordingly. And there’s error from sampling variation, from your prediction of likely voters, from your sampling of past polls that you use to estimate the error of that prediction. It’s turtles all the way down. And if one particular type of voter is particularly excited to vote this election based on something that never happened in the past, you’re just not going to catch it. You just hope all the errors cancel out.

One of the funny concepts that I think is recent in machine learning is an emphasis on ‘worse is better.’

  • Regularization can add a penalty to really significant parameters on the theory that the most significant parameters got a bit lucky.
  • Dropout trains a neural network using only e.g. 50% of the neurons each iteration, so that the network develops independent paths to the correct result. Sounds nuts. Maybe it’s like shooting a basketball until you’re so tired you can’t see or feel your fingers, and it becomes automatic.
  • Random forests use an algorithm that builds a large number of decision trees which each use a randomly selected subset of predictors, and have them vote on the outcome. Kind of like a bunch of people who each see a different side of a jar of jelly beans vote on how many black beans there are. Again, sounds nuts until you see it work.

(Machine learning feels like street-fighting statistics. If it works in a well-designed out-of-sample test, use it. Throw out any opinionated model about what data looks like and where it comes from, and don’t worry about proofs or elegance.)

Nate Silver gave Trump 35% and thoroughly explained the limitations of that analysis. He said Trump was within a normal polling error. Does that make him an idiot? If his probability prediction is always perfect, he’s going to be an idiot 1 election out of 3 and a genius 2 out of 3. Others maybe not so much.

The Nate Silvers, and media, and Silicon Valley, are the guys confronting reality, creating it, with the best tools they have. Maybe they’re in a bubble, but they try to make it the most self-aware, attentive, deliberate bubble they can.

Those in the so-called ‘media/elite bubble’ get a lot of flak for both being too mainstream, and too sensitive to minority views. (i.e. both too much bias, and too much variance.)

Everyone’s models of the world are overfitted to their own experience…unless they make an intense, deliberate effort to appreciate others’ experiences…to back off a little from assuming our experience is the complete reality. If we’d been born where they were born, taught what they were taught, and lived what they live, we would live in their bubble, and believe what they believe.

I believe Trump and his followers should, for example, accept the reality of Obama’s birth certificate, and firmly reject the endorsement of the KKK. If they don’t, it seems like they need to get out of their own bubble and tolerate others. My acknowledging and appreciating your reality cannot always extend to conforming mine to yours.

The beautiful thing about science, and markets, and democracies, is that for all their faults, they harness the potential and decision-making of all participants, and when they screw up, they eventually self-correct.

It is a mistake to try to look too far ahead. The chain of destiny can only be grasped one link at a time. – Sir Winston Churchill

(inspiration credit to @firoozye)

A politics bullet-storm / linkfest

  • The Dems didn’t exactly get crushed or demolished in the Presidential. Hillary may have won the popular vote by up to 2%.
  • Trump got the fewest popular votes of any GOP candidate since W in 2000 [edit: this was based on early returns, no longer true]. Low approval. Crushed in home states that know him well. Like Waterloo, the nearest run thing you ever saw in your life.
  • But of course Dems have gotten systematically dismantled in Congress, at the local level.
  • Bill and Hillary Clinton moved Dems to center, curtailed redistributionist rhetoric, aligned more with elites. One would think that should have reduced political polarization and instability, right?
  • Then GOP moved to the right, positioned as anti-elite. Picked up some poor whites.
  • Now you have alt-right racist BS, unstable cynicism-inducing dynamic where left is the party of establishment, right anti-elite.
  • Right positions as anti-elite while pursuing policies that in practice, as a first-order approximation, are not anti-elite at all.
  • Trump comes in and fails, Dems return to anti-elite role, restoring a more traditional left-right dynamic, with both parties now on a far more populist axis.
  • When I say ‘fails’, eventually all political movements fail, sometimes they change the world and eventually peter out, sometimes they are disasters from the get-go.
  • But frankly no one has ever been less prepared, more of a political outsider, than Trump. Reagan was a two-term governor of California, had a team of some cronies and some heavyweights, like Jim Baker (picked up from GW Bush’s team), Don Regan, George Shultz.
  • That was the choice, the ultimate insider machine politician (who can’t even fathom why Obama didn’t handcuff Comey) against the ultimate outsider. You have chosen, poorly IMHO.
  • You never know how a President will govern, but Trump is a real wildcard. Can he really ally with establishment Republicans and movement conservatives and cobble together an agenda that doesn’t betray his populist base? Is he really going to let Ryan and McConnell run the country, kill Obamacare, reform entitlements, privatize Medicare and lay it on him?
  • A tea-party / liberal populist coalition? Between improbable and impossible.
  • Ineffectiveness, gridlock coupled with intensifying populist rhetoric seems a distinct possibility.
  • Finding the lost Apprentice N-word tapes and pushing Trump out seems like an avenue some on both sides of the aisle will vigorously pursue…which would inflame the populists even more.
  • On the Dem side, hard to see how anyone who follows the Clintons would not be more populist, anti-establishment like Warren or Sanders.
  • It’s hard to be party of poor whites and poor blacks at the same time, racial resentment effs up normal right-left dynamics. More so in tough times than prosperity.
  • But Trump seems racist enough that Dems will pick up the decent poor whites, the ‘non-deplorables.’ When you don’t repudiate David Duke, guys who taunt reporters about sending them to the ovens, that’s pretty bad.
  • 2-party politics is kind of like the Hotelling problem, or 2 vendors on a boardwalk.
  • The beach tends to have the vanilla lovers on one side and strawberry lovers on the other.
  • The tendency will be for both vendors to position close to each other in the center.
  • Products will tend to undifferentiate, but the one with better strawberry will be on the side with the strawberry lovers.
  • If the two vendors perversely switch places, word might spread on one side that the strawberry is better on the other side.
  • They walk the extra few steps, while maybe the vanilla lovers just go to the closer one. One vendor gets crushed.
  • Or if one just has crappier product overall, everyone goes to the other.
  • Then maybe the crappier one has to move more to the left or right, its natural side, to get any customers at all.
  • And with 3 vendors there is no stable solution. I don’t know what happens in practice, presumably 2 pair up or one goes out of business.
  • Now, I don’t identify as a liberal, I don’t like identifying as anything or joining any movements, I tend to be more of an economic realist than many liberals, but I share liberal values. I think one has to balance economic efficiency with fairness and freedom. And I’m not pleased at this election outcome.

    Trump is a promoter, not an operator. He’s all id and ego, no superego. You just can’t take the politics out of politics. Like “The Wire,” “the game is the game.” The things that breed cynicism are to some degree built into the game. It’s a divided country and the things Trump had to do to get his base more excited than Hillary’s base make it impossible to hire decent people or get anything done, or at least anything that doesn’t piss off more people than it makes happy. So Trump is already running away from his campaign at top speed.

    I’m not really a believer in the “Wall Street Trump bro relief rally.” Some of his proposed policies are highly stimulative, some are highly recessionary (rolling back globalization, tariffs etc.). Crisis could come from a variety of places and the system is fragile. If Trump doesn’t bring the yuge growth and jobs and America winning that he promised, what’s next? Will he get even more populist? If he goes down, how extreme will the true populists be who come after Trump?

    I’m hoping for the best but bracing for the worst.

    A few good links, don’t agree 100% with any of them but worth thinking about:


37 queries in 0.305 seconds.