惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

T
Threatpost
aimingoo的专栏
aimingoo的专栏
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
T
Tailwind CSS Blog
J
Java Code Geeks
博客园_首页
Google Online Security Blog
Google Online Security Blog
Hugging Face - Blog
Hugging Face - Blog
C
CXSECURITY Database RSS Feed - CXSecurity.com
I
Intezer
P
Palo Alto Networks Blog
V
Vulnerabilities – Threatpost
雷峰网
雷峰网
O
OpenAI News
SecWiki News
SecWiki News
小众软件
小众软件
酷 壳 – CoolShell
酷 壳 – CoolShell
美团技术团队
N
News | PayPal Newsroom
Project Zero
Project Zero
Forbes - Security
Forbes - Security
IT之家
IT之家
A
Arctic Wolf
WordPress大学
WordPress大学
Jina AI
Jina AI
T
Tor Project blog
博客园 - 三生石上(FineUI控件)
S
Secure Thoughts
Google DeepMind News
Google DeepMind News
Attack and Defense Labs
Attack and Defense Labs
博客园 - 聂微东
Exploit-DB.com RSS Feed
Exploit-DB.com RSS Feed
P
Privacy International News Feed
Cloudbric
Cloudbric
G
GRAHAM CLULEY
博客园 - 叶小钗
H
Hacker News: Front Page
腾讯CDC
量子位
Help Net Security
Help Net Security
人人都是产品经理
人人都是产品经理
C
Cyber Attacks, Cyber Crime and Cyber Security
月光博客
月光博客
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
宝玉的分享
宝玉的分享
爱范儿
爱范儿
L
Lohrmann on Cybersecurity
Hacker News - Newest:
Hacker News - Newest: "LLM"
Recorded Future
Recorded Future
C
CERT Recently Published Vulnerability Notes

WhatIs

Strategic IT outlook: Tech conferences and events calendar | TechTarget 8 AI use cases in manufacturing Enterprises are making an AI native transformation Zero trust in the IT ops stack: Securing hybrid workloads How algorithmic value sets enhance clinical decision-making Top methods for collecting customer feedback Build a data governance team that delivers results How to calculate the total cost of ownership of ERP software Communities call for transparency in AI data center deals Scalable IT infrastructure: Balancing speed with stability How health systems are tackling 'Kill the Clipboard' obstacles Understanding the science behind AI-based hiring assessments Tape's strategic role in modern data protection How to choose an HR software system in 2026: A complete guide The UC stack gets the policy job Top zero-trust use cases in the enterprise 13 top IT infrastructure conferences in 2026 SNMP vs. CMIP: What's the difference? 3 essential network analytics use cases AI Security Risks Force CIOs to Rethink Strategy Red Hat Summit 2026 news and conference guide | TechTarget What is HR technology (human resources tech)? Understand, optimize and track customer journey touchpoints Should IT use Apple Business Manager without MDM? Build and organize an effective machine learning team The storage modernization imperative in a fast-changing IT landscape Procurement automation use cases for CSCOs to consider 3 steps for health system leaders to drive patient safety culture What is DevOps? Meaning, methodology and guide Enterprises Face New Storage Bottlenecks as AI Grows A guide to Intune Suite licensing for endpoint management Epic controls 42% of the US EHR market. Does that help or hurt interoperability? SAP Sapphire 2026 news, trends and analysis | TechTarget How to develop a data governance strategy: 7 key steps 12 generative AI tools for marketing and sales teams Top 9 smart contract platforms to consider in 2026 Top 8 e-signature software providers for 2026 Rise with SAP vs. S/4HANA Cloud: What are the differences? How businesses use KPIs to measure AI's performance 5 clues your network has shadow AI How do digital signatures work? Collaboration security and governance must be proactive Compare SAP greenfield vs. brownfield approach for S/4HANA Merck, Home Depot tap Gemini Enterprise for AI agent development Rural challenges may dampen digital healthcare's potential Build an ethical AI framework: 12 top resources The great workload reshuffle: Choices for AI and analytics How to remove a device from Intune enrollment Cisco unveils quantum network advancements 3 BYOD security risks and how to prevent them 10 of the top carbon accounting software 8 trends powering machine learning's dynamic new roles Network engineers must take the lead to push DDI to the cloud How does Microsoft 365 Copilot pricing and licensing work? ONC highlights behavioral health EHR adoption trends, data exchange barriers LLMs struggle with clinical reasoning, study finds Democratizing AI in business: The good, bad and ugly What can organizations do to address BYOD privacy concerns? Fix the service path before you optimize it with AI How AI reshapes upselling in customer experience platforms When collaboration starts becoming operational drag Balancing health AI management with growing vendor sprawl Career cure for AI phobia: Be a beekeeper, not a worker bee 16 top applicant tracking systems for 2026 How a rural community hospital deploys AI to detect heart disease 8 examples of document version control Guide to 30+ sustainability certifications for professionals AI agents are only as smart as the data that feeds them AI could earn trust in transactional work first How to fix keyboard connection issues on a remote desktop How to add and enroll devices to Microsoft Intune 11 DevSecOps best practices to prioritize in 2026 6 key components of a successful data strategy How to enable Copilot in Microsoft 365: A step-by-step guide What CIOs need to know about Meta's proposed CEO AI agent Top AI recruiting tools and software of 2026 How contact centers detect and prevent fraud 10 essential skills for modern contact center agents Beyond the chatbot: Engineering the agentic enterprise AI in business intelligence: How to manage it effectively Why legacy networks are a growing liability Failure is an option as an IT leadership tool How HR can create a successful change management strategy HR AI is becoming a change management story Digital transformation: Balancing speed and governance RSAC 2026 Conference: Key news and industry analysis | TechTarget 8 best practices for a bulletproof IAM strategy 5 customer journey phases businesses should understand 12 top HR software and tool options to consider in 2025 6 contact center trends shaping the future of customer service Contact center monitoring best practices for CX leaders Cloud vs. local backup: Which is right for your organization? 6 steps for when remote desktop credentials are not working How governance maturity affects M&A integration outcomes Inside the push to turn AI agents into suite functionality How should contact centers use AI today? Accenture global health lead on scaling AI in healthcare with governance and intent 10 best free DevOps certifications and training courses in 2026 What is compensation management? What CIOs must know about bossware strategy
AI existential risk: Is AI a threat to humanity?
2026-05-11 · via WhatIs

Rapid innovation in AI has fueled debate among industry experts about the existential threat posed by so-called intelligent machines that can perform tasks previously done by humans.

Artificial general intelligence (AGI) refers to machines that are able to think and experience the world like a human. Doomsayers argue that AGI will be upon us sooner than expected and have the capability to outwit people. Shorter term, they warn our overreliance on AI systems could spell disaster: Disinformation will flood the internet, terrorists will craft dangerous and cheap weapons, superintelligent AI models will lead to mass unemployment, automated AI could start a nuclear war and killer drones could run rampant.

Even Geoffrey Hinton, who's referred to as the "godfather of AI" for his seminal work on neural networks, has expressed growing concerns over AI's threat to humanity, He issued a warning in 2023 about the rapidly advancing abilities of generative AI chatbots, like ChatGPT. "Right now, they're not more intelligent than us, as far as I can tell," he told the BBC. "But I think they soon may be." Hinton predicted it would take 5 to 10 years, rather than his previous timeline of 30 to 50 years.

Rising concerns about AI's existential risks have led to calls for moratoriums on AI R&D -- an AI pause -- from industry and academic experts, including executives at many companies fueling AI innovation. Yet even staunch American AI doomerists like Sam Altman and Elon Musk have accelerated their AGI efforts.

Davi OttenheimerDavi Ottenheimer

\

Others argue that this AI doom narrative distracts from more likely AI dangers enterprises need to heed: AI bias, inequity, inequality, hallucinations, new failure modes, privacy risks and security breaches. A big concern among people in this group is that a pause might create a protective moat for major AI companies, like OpenAI, maker of ChatGPT and an AI pause advocate.

"Releasing ChatGPT to the public while calling it dangerous seems little more than a cynical ploy by those planning to capitalize on fears without solving them," said Davi Ottenheimer, vice president of trust and digital ethics at Inrupt, a secure data-sharing platform provider. A bigger risk might lie in enabling AI doomsayers to profit by abusing our trust, he said.

The existential threat discourse "turned out to be a new form of mysticism, to rope in true believers who wouldn't question the premise," Ottenheimer argued. Every major AI company gave what he called "saccharin safety pledges," then abandoned them. For enterprise leaders evaluating vendors, Ottenheimer offered a diagnostic: "The tells are architectural. When a vendor says 'safety,' ask where the controls are enforced." If they can't show evidence within a trend line, the commitment is decorative.

Responsible AI or virtue signaling?

Signed letters calling for an AI pause get a lot of media play, but they beg the question of what happens next.

The late Abhishek Gupta, who founded the Montreal AI Ethics Institute, was one of many prominent AI ethicists who viewed the calls for an AI pause ineffective, if not disingenuous. "I find it difficult to sign letters that primarily serve as virtue signaling without any tangible action or the necessary clarity to back them up," Gupta said in an interview in 2023. Such letters are often counterproductive as they consume attention cycles without leading to any real change, he noted.

Media-fueled doomsday narratives consume valuable time and resources and, in the process, they confuse the discourse and potentially put a lid on the levelheaded conversation required to make sound policy decisions, according to Gupta and like-minded ethicists. People seeking to manage AI risks are better off educating themselves on the actual risks instead of focusing on muddled musings about existential threats. They need to collaborate with technical experts who have practical experience in developing production-grade AI and machine learning systems, as well as with academic professionals who work on the theoretical foundations of AI.

Building consensus on what needs to be tackled to ensure responsible AI programs shouldn't be that hard.

What are realistic AI risks?

Brian GreenBrian Green

If AI doomerism isn't likely to prove useful in controlling AI risks, how should enterprises be thinking about the problem?

It's helpful to frame AI risks as those that come from AI itself and risks that come from the use of AI by humans, said Brian Green, director of technology ethics at the Markkula Center for Applied Ethics at Santa Clara University.

Risks from the AI technology itself range from simple errors in computation that lead to bad outcomes, to the existential threat of AI gaining a will of its own and deciding to attack humankind, Green said. At present, there's no clear way for the latter to happen, he said. Green is the co-author of Ethics in the Age of Disruptive Technologies: An Operational Roadmap, a handbook that lays out what he considers practical steps businesses can take to make ethical decisions

Risks stemming from the human use of AI cover every action people can imagine. They could be automated and made more efficiently evil with AI, such as more centralized nuclear weapons with hair-trigger controls, more powerful disinformation campaigns, deadlier biological weapons and more effective planning for social control.

"Everything horrible that human intelligence can do, artificial intelligence might be programmed to do as well as or better than humans," Green said. "There are vastly more chances that humans might use AI for existentially risky purposes than there are chances that AI would just pursue these goals on its own."

Green said he believes the realistic AI existential risks we face are more mundane: AI-generated content trained to catch our attention that inadvertently blinds us to important issues, or AI-based marketing apps trained to lure us into buying products and services detrimental to our well-being.

"I would argue that both of these things are already happening, so this possible existential AI risk is already upon us and is, therefore, 100% real," Green said.

Nell Watson, an AI ethics researcher and author, identified a risk that most frameworks miss: AI-automated psychological decomposition at scale. AI could use tactics similar to the Zersetzung approach of East Germany's Stasi secret police, she said, in which gaslighting, reputation destruction and social isolation were used to break individuals covertly.

Agentic AI makes this scalable and personalized in ways that were previously impossible, Watson added. The infrastructure is already built; recommender systems optimized for engagement can profile individual vulnerabilities and craft personalized content to exploit them, she explained. This approach becomes effective when the optimization target shifts from attention to influence. A new field of psychosecurity is needed, according to Watson, dedicated to protecting cognitive integrity, in the same way cybersecurity protects digital infrastructure.

It's important to keep on top of known problems, such as AI bias and misalignment with organizational objectives, Green said: "Immediate problems that are ignored can turn into big problems later, and conversely, it's easier to solve big problems later if you first get some practice solving problems now."

Could AI stir up social unrest by displacing workers?

Andrew PeryAndrew Pery

How AI technology is changing the nature of work is an issue companies should focus on now, said Andrew Pery, AI ethics evangelist at Abbyy, an intelligent automation company. "With the commercialization of generative AI, the magnitude of labor disruption could be unprecedented," he said. The International Monetary Fund predicted that nearly 40% of global employment is exposed to AI.

"Such a dramatic displacement of labor is a recipe for growing social tensions by shifting millions of people to the margins of society with unsustainable unemployment levels and without the dignity of work that gives us meaning," Pery said. Labor displacement could give rise to more nefarious and dangerous uses of GenAI technology that subvert the foundations of a rule-based order, he added.

Fostering digital upskilling for new jobs and rethinking social safety-net programs will play a pivotal role in safely transitioning into an age of AI, Pery said. However, he added that there's an even deeper structural concern: GenAI is replacing apprenticeships, eliminating the roles through which people learn how to become experts. The 2026 Anthropic labor market study found evidence that hiring of younger workers has slowed in highly exposed occupations, even though aggregate unemployment hasn't yet significantly increased.

Reskilling, Pery argued, has become the moral placebo of the AI age: It sounds responsible but often means little. Workers need wage insurance, transition stipends, portable benefits and mechanisms for sharing automation dividends. California's SB 53: Transparency in Frontier Artificial Intelligence Act and New York's Responsible AI and Safety Education Act impose obligations on frontier developers to publish safety frameworks and disclose catastrophic risk assessments, Pery said. They function as a de facto national benchmark in the absence of comprehensive federal legislation.

How enterprises can manage AI risks

A key component of responsible AI is identifying and mitigating risks arising from AI systems. These risks can manifest in various forms, including data privacy breaches, biased outputs, AI hallucinations, deliberate attacks on AI systems, and concentration of power in compute and data.

AI experts recommended enterprises and stakeholders take a holistic and proactive approach that considers the potential effect of each AI risk across different domains and stakeholders to prioritize these risk scenarios effectively. This approach requires a deep understanding of AI systems and their algorithmic biases, the data inputs used to train and test the models, and the potential vulnerabilities and attack vectors that hackers or malicious actors can exploit.

AI risk heat map

A practical approach applies the methods used in cybersecurity that evaluate risks according to their probability and severity. Many risks haven't been identified yet, so businesses would need to distinguish between areas of uncertainty and known potential risks to build their heat maps. Uncertainty considers the unknown unknowns, while risk refers to assessment based on known unknowns.

Trustworthy AI pledge

Pery suggested that enterprises make a top-down organizational commitment to trustworthy AI principles and guidelines. Trustworthy AI includes human-centered values of fairness of AI outcomes, accuracy, integrity, confidentiality, security, accountability and transparency associated with the use of AI.

Organizations that offer frameworks for trustworthy AI include the following:

  • Organization for Economic Cooperation and Development.
  • Berkman Klein Center at Harvard University.
  • Stanford Center for Human-Centered Artificial Intelligence.
  • AI Now Institute.

In addition, the NIST AI Risk Management Framework provides a comprehensive roadmap for implementing responsible AI best practices and a model for mitigating potential AI harms. Other standards that businesses might consider include the ISO/IEC 23894:2023 framework and the EU-sponsored AI governance framework by the European Committee for Standardization and the European Electrotechnical Committee for Standardization.

Checklist of questions for monitoring AI

Enterprises should implement measures to ensure continuous human monitoring of AI system performance. These steps can include identifying potential deviations from expected outcomes, taking remediation steps to correct adverse results and including processes for overriding automated decisions by AI systems.

'Kimberly Nevala, strategic advisor at SAS, recommended companies consider the following questions:

Kimberly NevalaKimberly Nevala

  • How will and could this solution go astray or make errors?
  • Is intentional misuse probable and in what circumstances?
  • How might the system be inadvertently misunderstood or misapplied?
  • What are the impacts, and how do they scale?
  • Does the system's design exacerbate or attenuate the potential for misuse and misunderstanding?
  • How might this system be integrated into or influence others within and beyond our scope of control?
  • What might be the second- or third-order effects thereof?

Will AI regulation help or harm?

Governments worldwide are starting to draft AI regulations that might prevent some of the worst AI risks. But poorly drafted regulations could slow the adoption of AI applications that solve some of our more pressing problems in healthcare and sustainable development -- or they could create new problems. Some industry observers have argued that regulation could stifle innovation and create a regulatory moat around incumbent companies, limiting competition and disruption from startups.

Effective AI regulation requires being specific about the processes and requirements to make AI transparent, understandable and safe. Drafting broad laws that focus on whether a process is ultimately harmful won't take us nearly far enough.

Discussions about regulating AI could take a cue from the regulations that helped us transition through the Industrial Revolution, such as minimum AI safety standards for work conditions, minimum pay requirements, child labor restrictions and environmental standards. The AI equivalent would address how processes should be regulated, including how they use data, whether and how they drive decisions, and how we can ensure AI's processes remain transparent, understandable and accountable.

Tackling existential AI risks will require identifying and addressing the present dangers of the AI systems we're deploying today, Nevala reasoned. "This is an issue that will only be addressed through a combination of public literacy and pressure, regulation and law, and -- history sadly suggests -- after a yet-to-be-determined critical threshold of actual harm has occurred," she said.

George Lawton is a journalist based in London. Over the last 30 years, he has written more than 3,000 stories about computers, communications, knowledge management, business, health and other areas that interest him.