惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

GbyAI
GbyAI
云风的 BLOG
云风的 BLOG
Microsoft Azure Blog
Microsoft Azure Blog
F
Fortinet All Blogs
A
About on SuperTechFans
月光博客
月光博客
酷 壳 – CoolShell
酷 壳 – CoolShell
博客园 - 司徒正美
P
Proofpoint News Feed
D
Docker
Jina AI
Jina AI
Apple Machine Learning Research
Apple Machine Learning Research
The Cloudflare Blog
I
InfoQ
Recorded Future
Recorded Future
爱范儿
爱范儿
Last Week in AI
Last Week in AI
J
Java Code Geeks
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
M
MIT News - Artificial intelligence
L
LINUX DO - 热门话题
腾讯CDC
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
IT之家
IT之家
博客园_首页
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
C
CXSECURITY Database RSS Feed - CXSecurity.com
L
Lohrmann on Cybersecurity
The Last Watchdog
The Last Watchdog
V
Visual Studio Blog
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
P
Privacy International News Feed
博客园 - 三生石上(FineUI控件)
Schneier on Security
Schneier on Security
Simon Willison's Weblog
Simon Willison's Weblog
Martin Fowler
Martin Fowler
雷峰网
雷峰网
Latest news
Latest news
Scott Helme
Scott Helme
T
Tenable Blog
Vercel News
Vercel News
宝玉的分享
宝玉的分享
PCI Perspectives
PCI Perspectives
Help Net Security
Help Net Security
L
LINUX DO - 最新话题
Attack and Defense Labs
Attack and Defense Labs
Spread Privacy
Spread Privacy
量子位
H
Heimdal Security Blog

ashishb.net

A day in Luxembourg - the richest country in the world I was asked to install malware during a fake interview Book summary: Breakneck - China's quest to engineer the future by Dan Wang Book summary: How to Teach Your Baby to Read Book Summary: The Discontented Little Baby Book by Pamela Douglas Introducing Amazing Sandbox - run third-party tools and AI agents securely on your machine Why software outsourcing gets a bad reputation? Book summary: The Natural Baby Sleep Solution by Polly Moore A day in Antwerp, Belgium Journey of online influencers Two days in Brussels, Belgium Shortcuts - when we love them and when we don't A visit to Rakhigarhi Three days in overhyped Paris Empty Japan, crowded Tokyo The real lock-in in GitHub is not the code, but the stars 11-day Norwegian Breakaway East Caribbean cruise Sanskrit and Sri Lankan Air Force Use REST with Open API The Achilles heel of American capitalism Costa Rica in 4 days At a juice stall in Sri Lanka A short stay at Warsaw, Poland Best practices for using Python & uv inside Docker Two days in Vilnius, Lithuania How IntelliJ IDEs waste disk space Pregnancy Why there aren't many digital nomads from India Two days in Riga, Latvia To keep your machine secure, run third-party tools inside Docker Family Ties in Your DNA: Some relatives are closer than others Doctors per capita Two days in Tallinn, Estonia Ship tools as standalone static binaries Made in America Two days in Helsinki, Finland Maintaining an Android app is a lot of work The land of good deals Two days in Oslo, Norway FastAPI vs Flask performance comparison Google Search is losing to Perplexity Two days in Dublin, Ireland Continuous integration ≠ Continuous delivery World's simplest project success heuristic London in 5 days It is hard to recommend Python in production Inflation, IRS, Credit cards, and Vendors Temu and the Chinese approach Things to do in Miami Florida Revenue vs Cost Axis Language learning as an adult The unanchored babies of the green card limbo Price variance in the United States A day in Louisville, Kentucky A surprisingly positive experience with Air India Unhospitable Airports Android: Don't use stale views USA = Union of Sales and Advertisement A day in Nashville, Tennessee Minimize Javascript in your codebase A day in Birmingham, Alabama In defense of ad-supported products Real vs artificial world The science behind Punjabi singers Hiking Mt. Fuji The Indian startup bubble is insane Repairing database on the fly for millions of users Book Summary: One up on Wall Street by Peter Lynch It is hard to recommend Google Cloud At the Prague airport Kyoto in three days Migrating from WordPress to Hugo Book summary: Sick Societies by Robert B. Edgerton Statistical outcomes require statistical games Illegal immigrants to Europe via Cairo Tokyo in three days Mobs are Status Games Writing Script matters as much as the spoken language Sri Lanka in 5 days LLMs: great for business but bad business Book Summary: Safe Haven by Mark Spitznagel Mac shortcut for typing Avagraha symbol On a bus with an asylum seeker Nicaragua in 5 days When to commit Generated code to version control Why I always buy a local SIM in a foreign country Use Makefile for Android Four days in Guadalajara, Mexico Android Navigation: Up vs Back Hotels vs Airbnb vs Hostels Currency issues in Argentina Abstractions should be deep not wide Some data on podcasting Always support compressed response in an API service A day in El Calafate - Patagonia, Argentina Hermetic docker images with Hugging Face machine learning models American Elections The sound of "ch" API services should always have usage Limits Hiking in El Chaltén - trekking capital of Argentina
Book Summary: The Black Swan by Nassim Nicolas Taleb
Ashish Bhatia · 2020-05-02 · via ashishb.net

The book posits a unique viewpoint to understand randomness and unpredictability in the world around us. Rather than trying to predict the improbable black swans, it focuses more on how not to be adversely impacted by them.

Mediocristan vs Extremistan

This is the central concept of the book. Let’s illustrate this with human height vs. human wealth. If you are trying to find the average height of humans by sampling 100 humans, then no single sample will have a significant impact. The tallest human is about 8 ft tall; the shortest is about 2 ft. That’s a difference of only 4x. A single sample of 8 ft where the average is 6 ft can skew the overall ratio by only 0.08 ft (1 inch). Once your sample set is large, no single instance will significantly change the aggregate.

Now, consider the wealth; you can encounter 99 humans with a net worth of ~100$, and if you meet Bill Gates next, the average will become $1 Billion!!! A single sample can skew the average drastically. The average net worth of an American household is ~100,000. Bill Gates’s net worth is a million times more than that. One single observation can disproportionately impact the aggregate.

  1. Wealth comes from extremistan. Height comes from mediocristan. Most outcomes of the biological process like height, weight, age expectations belong to mediocristan. Standard statistics like Gaussians are valid here. The regression to mean is standard here. That’s not true of extremistan. If you encounter someone 7 ft tall, the likelihood of encountering another 7 ft tall person is low. However, if the stock market, which belongs to extremistan, goes down by 20%, then the likelihood of another 20% fall the next day isn’t any lower. Mediocristan does not care about extremes. They severely impact Extremistan.
  2. We see Black Swans in extremistan, single incidents which have huge influences on history.
  3. It isn’t that whether Black Swans are more frequent than our imagination or not. The critical fact is that they are more consequential.
  4. Positive black swans take time to show up while the negative ones (destruction) happens very quickly. This perception of causation has a biological foundation.
  5. Mediocristan has a “typical” extreme. For example, a 7 ft human is a typical extreme in terms of height. There is no “typical” rich, 100 million, 1 billion, 10 billion, all have a very different lifestyle. Similarly, there is no “typical” war. Since we don’t know what a typical extreme in extremistan is, we cannot estimate its impact.
  6. We overestimate the positive and negative effects of significant events like buying a car, the death of a loved one, etc. on our lives. But that’s OK, these errors are in mediocristan and don’t cost much. However, such self-deceptions can cost a lot in extremistan.
  7. In mediocristan, samples regress to mean. Consider, for example, if the total height of two men is 14 ft, then you would imagine that both are about 7 ft (and not 12 ft and 2 ft). While if there are two authors with book sales with the total book sales of 100 million, then it is more likely than one of them had ~100 million in book sales, and the other one has none. The effect is even more dominant if the total book sales are 1 billion since very few books reach that scale.
  8. Preferential attachment and cumulative advantage play a significant role in extremistan. People read a famous author even more, and the benefits cumulate. Winning today increases the chance of winning in the future.
  9. People love using Gaussian since once you assume gaussian, it yields its properties with sufficient sampling. Gaussian does not apply in extremistan. Gaussian can be used in mediocristan, and even if it is wrong, the harm done is usually not devastating.
  10. Classical thermodynamics produces Gaussian variations, while informational variations are from Extremistan. For example, food is not just calorie in-calorie out, but it is complex signaling to your body.
  11. When dealing with simple binary payoffs in mediocristan, for example, casino bets, then the standard statistical methods work best. When dealing with complex binary payoffs in mediocristan, for example, epidemics, the usual statistical methods might work with some limitations.
  12. When dealing with simple payoffs in extremistan, since the harm is limited, it is fine to be wrong. The real danger is with complex payoffs in extremistan. Avoid any adverse outcomes here. Do not model the probability of the outcome in this case. No models are better than the wrong models for forecasting. If possible, cut your exposure by say buying insurance.

Narrative Fallacy

We are vulnerable to overinterpretation. We love compact stories over accepting raw truth. We should enjoy history as an enumeration of accounts instead of theorizing it.

Confirmation Bias

We not only form hypothesis early on but try to only look at the data in a way that it only confirms our existing belief.

Belief Perseverance

Belief Perseverance is our tendency to discard any data that discredits our existing hypothesis as outliers. Combine this with Confirmation Bias, and it can quickly create a scenario where someone presented with more data (and hence more noise) ends up forming strong incorrect opinions, which are harder to refute later. For example, consider people spinning a wheel and then predicting the number of countries in Africa. Those that got a lower number, despite knowing no relation, estimated a smaller number of countries in Africa and vice versa.

Silent Evidence

We only look at the evidence which is visible to us. For example, we look at successful authors and try to create a narrative of what traits lead to success. We don’t stop and think that maybe the other 99% of authors who failed might have the same characteristics as well. We may enjoy what we see, but there is no point in reading too much into the success stories since we don’t see the full picture. Consider, for example, radiating rats, which will kill the weak ones, and the surviving dominant rats will be left to roam in New York City. It might make people feel that the radiation made those rats powerful when in itself, radiation just filtered the most powerful ones out. Or consider bubonic plague it is “the most dangerous epidemic we know of” and not the “most dangerous epidemic”. There might have been a more dangerous one, but no observer ever survived to narrate it.

Beginner’s Luck

Beginner’s luck does exist, and it’s because of the silent evidence. If 100 people try gambling for the first time, the ones who win big will try again, the ones who don’t drop out almost immediately. And those who continue will narrate the story of the good old days when they were lucky.

Ludic Fallacy

The ludic fallacy is the mistaken belief that one can use gamified models with well-defined probabilities for modeling real-life situations.

Inverse Problems

Most problems we encounter in life are inverse. For example, imagine a few ice cubes which are melting on a floor, and you finally came in to see the puddle of water, your job is to find out the number and the shapes of ice cubes. This inverse problem is much harder than trying to imagine the puddle, which will be formed given a set of ice cubes. Similarly, we don’t see the underlying rules of the stock market, we only see a pattern, and we try to form a thesis out of what could be happening.

Randomness

  1. When you are employed, hence, dependent on other people’s judgment, then looking busy can help you claim the results of a random outcome.
  2. The more information you give someone, the more hypotheses they will form along the way, and the worse off they will be. They see more random noise and mistake it for information.
  3. The more routine the task is, the better you learn to forecast.
  4. Professions that deal with the future base their studies on the non-repeatable past have an expert problem.
  5. There is a difference between techne (technical) and episteme (Epistemological) knowledge. The first one can have experts, for example, brain surgeons, the second one do not, for example, it is not obvious who will make a better economic forecast, a finance Ph.D. vs. a business writer. Tetlock studied and concluded that those who had a more prominent reputation in Economics were worse predictors. Experts underestimate the duration of the war. While human history is full of violent conflicts, large-scale wars are events new to human history.
  6. Corporations survive not because they make good forecasts but because they may have been the lucky ones.
  7. Luck both made and unmade Carthage. It made and unmade Rome.
  8. Randomness in mediocristan is predictable. It isn’t in extremistan.

Nature

  1. Mother nature did not attend high school courses on Geometry or read the books of Euclid at the library of Alexandria.
  2. Mother nature does not like anything too big for resilience. For example, killing an elephant does not impact nature, while a falling bank can crumble the whole financial system.
  3. Mother nature does not like too much connectivity or globalization.
  4. Mother nature loves redundancies. Defensive redundancy which allows one to survive under adversities. Functional redundancy which allows one multiple different structures to perform the same task.

Forecasting

There are three significant issues with forecasting.

  1. The accuracy or the variance of the projection matters more than just the projected value.
  2. The duration matters. The longer the duration of the forecast, the lower is the accuracy.
  3. The unexpected outcomes. Randomness is extremistan allows for far major optimistic or pessimistic scenarios than one can imagine, especially when the worst-case outcome is more consequential than the forecast itself.
  4. Generalizing a theory out of samples requires intuition and common sense. We don’t have a well-developed intuition for Black Swans.
  5. In practice, randomness due to lack of information is the same as randomness due to unpredictability.
  6. Borrowing money makes you more vulnerable to forecast errors.
  7. Regular events from the past can predict regular events. While extreme events are so rare and acute, that the past can never be used to predict them. The next bestseller might beat all the existing ones, and the subsequent stock market loss might be worse than any loss seen before.

Gray Swan

  1. A Gray Swan is about modelable extremes while a Black Swan is about unknown unknowns.
  2. The 1987 crash of the stock market was a Gray Swan. The 9/11 attack on the World center was a Black Swan.
  3. Fractals are scale-invariant measures (unlike Gaussian), which can be used for modeling Gray Swans.
  4. A rug at an eye-level corresponds to Mediocristan. A coastline, however, is a fractal, it looks the same from the ground level or the airplane.
  1. Black Swan has three attributes - unpredictability, consequences, and retrospective explainability. The last one allows experts to look back and pretend that they expected it.
  2. The history is going to be dominated by an improbable event. I just don’t know which one.
  3. We build toys. Some of them like Laser, Internet, Compact Disks, change the world.
  4. Rank beliefs not based on their plausibility but by the harm they may cause. It is fine to trust weather predictions for the picnic even if they might turn out to be wrong but don’t trust the government’s forecast for social security in 2040.
  5. It is better to govern society based on awareness of ignorance than knowledge.
  6. The government keeps putting money in the market, and nothing happens, and then suddenly, one day, it is hyperinflation.
  7. We need to avoid exposure to small probabilities in specific domains. We simply cannot compute them.

Miscellaneous

  1. It is a contagion that determines the fate of a theory in social science, not its validity.
  2. Life exists in pre-asymptote. Many theories that are right in the long-run (asymptote) are meaningless from an applicability perspective.
  3. Categorizing is necessary for humans, but it becomes pathological when the category is seen as definitive. It prevents people from considering the fuzziness of boundaries, let alone revising those categories.
  4. Good news is good first. How good matters rather little.
  5. Most of the world around us is non-linear. We study linearity in the classroom since it is easier.