New Report: Don’t Be Evil – A Survey of the Tech Sector’s Stance on Lethal Autonomous Weapons
Contents
As we move towards a more automated world, tech companies are increasingly faced with decisions about how they want — and don’t want — their products to be used. Perhaps most critically, the sector is in the process of negotiating its relationship to the military, and to the development of lethal autonomous weapons in particular. Some companies, including industry leaders like Google, have committed to abstaining from building weapons technologies; Others have wholeheartedly embraced military collaboration.
In a new report titled “Don’t Be Evil,” Dutch advocacy group Pax evaluated the involvement of 50 leading tech companies in the development of military technology. They sent out a survey asking companies about their current activities and their policies on autonomous weapons, and used each company’s responses to categorize it as “best practice,” “medium concern,” or “high concern.” Categorizations were based on 3 criteria:
- Is the company developing technology that could be relevant in the context of lethal autonomous weapons?
- Does the company work on relevant military projects?
- Has the company committed to not contribute to the development of lethal autonomous weapons?
“Best practice” companies are those with explicit policies that ensure their technology will not be used for lethal autonomous weapons.
“Medium concern” companies are those currently working on military applications of relevant technology but who responded that they were not working on autonomous weapons; or companies who are not known to be working on military applications of technology but who did not respond to the survey.
“High concern” companies are those working on military applications of relevant technology who did not respond to the survey.
The report makes several recommendations for how companies can prevent their products from contributing to the development of lethal autonomous weapons. It suggests that companies make a public commitment not to contribute; that they establish clear company policies reiterating such a commitment and providing concrete implementation measures; and that they inform employees about the work they are doing and allow open discussion around any concerns.
Pax identifies six sectors considered relevant to autonomous weapons: big tech, AI software and system integration, autonomous (swarming) aerial systems, hardware, pattern recognition, and ground robots. The report is organized into these categories, and then subdivided further by country and product. We’ve instead listed the companies in alphabetical order. Find basic information about all 50 companies in the chart, and read more about a select group below.
Companies involved in developing military technology
Company | HQ | Relevant Technology | Relevant Military / Security Projects | Level of Concern |
---|---|---|---|---|
AERIALX | Canada | Counter-drone systems | DroneBullet | High |
AIROBOTICS | Israel | Autonomous drones | Border security patrol bots | Medium |
AIRSPACE SYSTEMS | US | Counter-drone systems | Airspace interceptor | High |
ALIBABA | China | AI chips; facial recognition | Medium | |
AMAZON | US | Cloud; drones; facial and speech recognition | JEDI; Rekognition | High |
ANDURIL INDUSTRIES | US | AI platforms | Project Maven; Lattice | High |
ANIMAL DYNAMICS | UK | Autonomous drones | Skeeter | Best practice |
APPLE | US | Computers; facial and speech recognition | Medium | |
ARBE ROBOTICS | Israel | Autonomous vehicles | Best practice | |
ATOS | France | AI architecture; cyber security; data Management | Medium | |
BAIDU | China | Deep learning; pattern recognition | Medium | |
BLUE BEAR SYSTEMS | UK | Unmanned maritime and aerial systems | Project Mosquito/LANCA | High |
CAMBRICON | China | AI chips | Medium | |
CITADEL DEFENSE | US | Counter-drone systems | Titan | High |
CLARIFAI | US | Facial recognition | Project Maven | High |
CLOUDWALK TECHNOLOGY | China | Facial recognition | Medium | |
CORENOVA TECHNOLOGIES | US | Autonomous swarming systems | HiveDefense; OFFSET | High |
DEEPGLINT | China | Facial recognition | Medium | |
DIBOTICS | France | Autonomous navigation; drones | ‘Generate’ | Medium |
EARTHCUBE | France | Machine learning | ‘Algorithmic warfare tools of the future’ | High |
US | Social media; pattern recognition; virtual Reality | Medium | ||
GENERAL ROBOTICS | Israel | Ground robots | Dogo | Best practice |
US | AI architecture; social media; facial recognition | Best practice | ||
HERON SYSTEMS | US | AI software; machine learning; drone applications | ‘Solutions to support tomorrow’s military aircraft’ | High |
HIVEMAPPER | US | Pattern recognition; mapping | HiveMapper app | Best practice |
IBM | US | AI chips; cloud; super computers; facial recognition | Nuclear testing super computers; ex-JEDI | Medium |
INNOVIZ | Israel | Autonomous vehicles | Medium | |
INTEL | US | AI chips; UAS | DARPA HIVE | High |
MEGVII | China | Facial recognition | Medium | |
MICROSOFT | US | Cloud; facial recognition | HoloLens; JEDI | High |
MONTVIEUX | UK | Data analysis; deep learning | ‘Revolutionize human information relationship for defence’ | High |
NAVER | S. Korea | ‘Ambient Intelligence’; autonomous robots; machine vision systems | Medium | |
NEURALA | US | Deep learning neural network software | Target identification software for military drones | Medium |
ORACLE | US | Cloud; AI infrastructure; big data | Ex-JEDI | High |
ORBITAL INSIGHT | US | Geospatial analytics | Medium | |
PALANTIR | US | Data analytics | DCGS-A | High |
PERCEPTO | Israel | Autonomous drones | Medium | |
ROBOTEAM | Israel | Unmanned systems; AI software | Semi-autonomous military UGVs | High |
SAMSUNG | S. Korea | Computers and AI platforms | Medium | |
SENSETIME | China | Computer vision; deep learning | SenseFace; SenseTotem for police use | High |
SHIELD AI | US | Autonomous (swarming) drones | Nova | High |
SIEMENS | Germany | AI; Automation | KRNS; TRADES | Medium |
SOFTBANK | Japan | Telecom; Robotics | Best practice | |
SPARKCOGNITION | US | AI systems; swarm technology | ‘Works across defense and national security space in the U.S.’ | High |
SYNTHESIS | Belarus | AI- and cloud-based applications; pattern recognition | Kipod | High |
TAIWAN SEMICONDUCTOR | Taiwan | AI chips | Medium | |
TENCENT | China | AI applications; cloud; ML; pattern recognition | Medium | |
THARSUS | UK | Robotics | Medium | |
VISIONLABS | Russia | Visual recognition | Best practice | |
YITU | China | Facial recognition | Police use | High |
Company profiles
Company names are colored to indicate concern level: best practice, medium concern, high concern.
AerialX
- Developing the DroneBullet, a kamikaze drone that can autonomously identify, track, and attack a target drone
- Working to modify DroneBullet “for a warhead-equipped loitering munition system”
Airobotics
- In response to survey, stated that its “drone system has nothing to do with weapons and related industries
- Has clear links to military and security business; announced a Homeland Security and Defense division and an initiative to perform emergency services in 2017
- Involved in border security, in particular US-Mexico border; provides patrol bots
- Co-founder has stated that it will not add weapons to its drones
Airspace Systems
- Utilizes AI and advanced robotics for airspace security solutions, including “long-range detection, instant identification, and autonomous mitigation—capture and safe removal of unauthorized or malicious drones”
- Developed Airspace Interceptor, a fully autonomous system that can capture target drones, in collaboration with US Department of Defense
Alibaba
- China’s largest online shopping company
- Recently invested in seven research labs that will focus on areas including AI, machine learning, network security, and natural language processing
- Established a semiconductor subsidiary, Pingtouge, in September 2018
- Major investor in tech sector, including in Megvii and SenseTime
Amazon
- Likely winner of JEDI contract, a US military project that will serve as universal data infrastructure linking Pentagon and soldiers in the field
- Developed Rekognition program, used by police; testing by ACLU revealed that nearly 40 percent of false matches involved people of color
- CEO has stated, “If big tech companies are going to turn their back on the U.S. Department of Defense, this country is going to be in trouble.”
- Received backlash since exposure of partnership with government agencies, including ICE
- Has since proposed guidelines for responsible use of tech
Anduril Industries
- Co-founded by a former intelligence official
- Has vocally supported stronger ties between tech sector and Pentagon: “AI has paradigm-shifting potential to be a force-multiplier […] it will provide better outcomes faster, a recipe for success in combat.”
- Involved in Project Maven
- Has offered support for the Pentagon’s newly formed Joint Artificial Intelligence Center
- Developed the Lattice, an autonomous system that provides soldiers with a view of the front line and can be used to identify targets and direct unmanned vehicles into combat; has been used to catch border crossers
- Co-founder has stated that Anduril is “deployed at several military bases. We’re deployed in multiple spots along the U.S. border […] We’re deployed around some other infrastructure I can’t talk about.”
Animal Dynamics
- Spin-off company originating in Oxford University’s Zoology Department
- Develops unmanned aerial vehicles
- Stork, a paraglider with autonomous guidance and navigation, has received interest from both military and humanitarian aid/disaster relief organizations
- Skeeter, “disruptive drone technology,” was developed with funding from UK government’s Defense Science and Technology Laboratory
- In March 2019, took over software developer Accelerated Dynamics, which has developed ADx autonomous flight-control software
- Use of ADx with Skeeter allows it to be operated in a swarm configuration, which has military applications
- In response to survey, CEO stated that “we will not weaponize or provide ‘kinetic’ functionality to the products we make,” and that “legislating against harmful uses for autonomy is an urgent and necessary matter for government and the legislative framework to come to terms with.”
Arbe Robotics
- Began in military and homeland security sectors but has moved to cars
- In response to survey, stated that it “will sign agreements with customers that would confirm that they are not using our technology for military use.”
Baidu
- Largest provider of the Chinese-language Internet search services
- Highly committed to artificial intelligence and machine learning and is exploring applications for facial recognition technology
- Opened Silicon Valley AI research lab in 2013, where it has been heavily investing in AI applications; has scaled down this research since US-China trade war
- In charge of China’s Engineering Laboratory for Deep Learning Technologies, established March 2017
- Will contribute to National Engineering Laboratory for Brain-Inspired Intelligence Technology and Applications
Blue Bear Systems
- Research company involved in all aspects of unmanned systems and autonomy, including big data, AI, electronic warfare, and swarming systems
- In March 2019, consortium it headed was awarded UK Ministry of Defense contract worth GBP 2.5 million to develop drone swarm technology
Citadel Defense
- “Protects soldiers from drone attacks and surveillance in enemy combat” and “creates a force multiplier for Warfighters that enables them to get more done with the same or fewer resources”
- Contracted by US Air Force to provide systems that can defeat weaponized drones and swarms
- Developed autonomous counter-drone system called Titan
Corenova Technologies
- Offers “military-grade solutions to secure autonomous operations,” according to website
- Developed HiveDefense, “an evolving swarm of self-learning bots”
- Works with DARPA on OFFSET, facilitating unmanned missions without human control
Dibotics
- Works on autonomous navigation
- Supported by Generate, a program for French defense start-ups
- Founder/CEO signed FLI’s 2017 open letter to the UN
EarthCube
- “Developing monitoring solutions based on an automated analysis of geospatial information”
- Has been described as “conceiving of the algorithmic warfare tools of the future.”
- CEO has stated, “With the emergence of new sensors—whether they are satellite, UAV or plane—we have seen here a great opportunity to close the gap between AI in the lab and Activity Based Intelligence (ABI) in the field.”
General Robotics
- Robotics company focused on defense and security
- Founder previously worked in Israeli defense ministry’s R & D authority
- Supplies “advanced robotics systems to counter-terrorist units worldwide,” many of which are designed for “urban warfare”
- Developed Dogo, said to be “the world’s first inherently armed tactical combat robot,” but controlled remotely rather than autonomously
- In response to survey, CEO stated that “our position is not to allow lethal autonomous weapons without human supervision and human final active decision […] In general, our systems are designed to provide real-time high quality information and to present it to a trained human operator in an intuitive manner; this insures better decision making by the human and thereby better results with less casualties.”
Heron Systems
- Provides “leading-edge solutions for national security customers”
- States that its mission is “to strengthen America’s defense by providing innovative laboratory testing and simulation solutions”
Hivemapper
- Software provides mapping, visualization, and analytic tools; uses video footage to generate instant detailed 3-D maps and detect changes; could potentially be used by Air Force to model bombing
- Founder/CEO has stated that he “believes Silicon Valley and the US government have to work together to maintain America’s technological edge—lest authoritarian regimes that don’t share the US values catch up.”
- Founder/CEO signed FLI’s 2015 open letter; In his survey response, he stated that “we absolutely want to see a world where humans are in control and responsible for all lethal decisions.”
IBM
- Bid for Jedi contract and failed to qualify
- Actively working towards producing “next-generation artificial intelligence chips” for which it is building a new AI research center; over the next 10 years, expect to improve AI computing by 1,000 times
- Long history of military contracting, including building supercomputers for nuclear weapons research and simulations
- Currently involved in augmented military intelligence research for US Marine Corps
- 3 dozen staff members, including Watson design lead and VP of Cognitive Computing at IBM Research, signed a 2015 open letter calling for a ban on lethal autonomous weapons
- Developed Diversity in Faces dataset using information from Flickr images; claims the project will reduce bias in facial recognition; Dataset available to companies and universities linked to military and law enforcement around the world
- In response to survey, confirmed it is not currently developing lethal autonomous weapons systems
Innoviz
- Produces laser-based radar for cars
- Founded by former members of the IDF’s elite technological unit, but does not currently appear to be developing military applications
Intel
- Develops various AI technologies, including specific solutions, software, and hardware, which it provides to governments
- Selected by DARPA in 2017 to collaborate on DARPA HIVE, a data-handling and computing platform utilizing AI and ML
- Announced in 2018 that it will work with DARPA on developing “the design tools and integration standards required to develop modular electronic systems”
- Has invested significantly in unmanned aerial vehicles and flight control technology
Megvii
- AI provider known for facial recognition software Face++
- Reportedly uses facial scans from a Ministry of Public Security photo database that contains files on nearly every Chinese citizen
- Has stated, “We want to build the eyes and brain of the city, to help police analyze vehicles and people to an extent beyond what is humanly possible.”
Microsoft
- Competing with Amazon for JEDI contract
- Published “The Future Computed” in 2018, which defines core principles necessary for the development of beneficial AI
- According to employees, “With JEDI, Microsoft executives are on track to betray these principles in exchange for short-term profits.”
- Company position on lethal autonomous weapons systems unclear
- First tech giant to call for regulations to limit use of facial recognition technology
Montvieux
- Developing a military decision-making tool that uses deep learning-based neural networks to assess complex data
- Receives funding from the UK government
Neurala
- Sells AI technology that can run on light devices and helps drones, robots, cars, and consumer electronics analyze their environments and make decisions
- Military applications are a key focus
- Works with a broad range of clients including the US Air Force, Motorola, and Parrot
- Co-founder/CEO signed FLI’s 2017 open letter to the UN
Oracle
- Provides database software and technology, cloud-engineered systems, and enterprise software products
- Website states, “Oracle helps modern defense prepare for dynamic mission objectives”
- Bid for JEDI contract and failed to qualify; filed several complaints, in part related to Pentagon’s decision to use only one vendor
Palantir
- Data-analysis company founded in 2004 by Trump advisor; has roots in CIA-backed In-q-Tel venture capital organization
- Producer of “Palantir Intelligence,” a tool for analyzing data that is used throughout the intelligence community
- Has developed predictive policing technology used by law enforcement around the US
- In 2016, won a Special Operations Command contract worth USD 222 million for a technology
- In March 2019, won a US Army contract worth over USD 800 million to build the Distributed Common Ground System, an analytical systems for use by soldiers in combat zones
Percepto
- Developed the Sparrow, an autonomous patrol drone with security applications
- Focuses explicitly on industrial, rather than military or border security, applications
- In response to survey, stated “Since we develop solutions to the industrial markets, addressing security, safety, and operational needs, the topic of lethal weapon[s] is completely out of the scope of our work”
Roboteam
- Founded by two former Israeli military commanders with “access to the Israel Defense Forces as our backyard for testing”
- Specifically serves military markets, including the Pentagon
- Developed Artificial Intelligence Control Unit (AI-CU), which brings autonomous navigation, facial recognition, and other AU-enabled capabilities to control and operation of unmanned systems and payloads
- Exposure of links to Chinese investment firm FengHe Fund Management appears to have cost them a series of US Army robotics contracts last year
Samsung
- One of world’s largest tech companies
- Developing AI technologies to be applied to all its products and services in order to retain its hold on telephone/computer market
- Samsung Techwin, Samsung’s military arm known for SG1A Sentry robot, was sold in 2014
SenseTime
- Major competitor of Megvii
- Sells software that recognizes objects and people
- Various Chinese police departments use its SenseTotem and SenseFace systems to analyze video and make arrests
- Valued at USD 4.5 billion, it is “the world’s most valuable AI start-up” and receives about two-fifths of its revenue from government contracts
- In November 2017, sold its 51 percent stake in Tangli Technology, a “smart-policing” company it helped found
Shield AI
- States that its “mission is to protect service members and innocent civilians with artificially intelligent systems”
- Makes systems based on Hivemind, AI that enables robots to “learn from their experiences”
- Developed Nova, a “combat proven” robot that autonomously searches buildings while streaming video and generating maps
- Works with Pentagon and Department of Homeland Security “to enable fully autonomous unmanned systems that dramatically reduce risk and enhance situational awareness in the most dangerous situations.
Siemens
- Europe’s largest industrial manufacturing conglomerate
- Known for medical diagnostics equipment (CT scanners), energy equipment (turbines, generators), and trains
- Produces MindSphere, a cloud-based system that helps enable the US of AI in industry
- In 2013, won a USD 2.2 million military research contract with Carnegie Mellon University and HRL Laboratories to develop improved intelligence tools
- Collaborating with DARPA on the TRAnsformative DESign (TRADES) program
- In response to survey, stated: “Siemens in not active in this business area. Where we see a potential risk that components or technology or financing may be allocated for a military purpose, Siemens performs a heightened due diligence. […] All our activities are guided by our Business Conduct Guidelines that make sure that we follow high ethical standards and implement them in our everyday business. We also work on responsible AI principles which we aim to publish later this year.”
SoftBank
- Invests in AI technology through its USD 100 billion Vision Fund, including BrainCorp, NVIDIA, and Slack Technologies; Owns some 30 percent of Alibaba
- Works in partnership with Saudi Arabia’s sovereign wealth fund and is part of Saudi strategy for diversifying away from oil
- In 2017, took over Boston Dynamics and Schaft, both connected with DARPA
- Developed the humanoid Pepper robot
- In response to survey, stated, “We do not have a weapons business and have no intention to develop technologies that could be used for military purposes”
SparkCognition
- Collaborates “with the world’s largest organizations that power, finance, and defend our society to uncover their highest potential through the application of AI technologies.”
- Has attracted interest from former and current Pentagon officials, several of whom serve on the board or as advisors
- Works “across the national security space—including defense, homeland security intelligence, and energy—to streamline every step of their operations”; has worked with the British Army on military AI applications
- Founder/CEO has stated that he believes restrictions on autonomous weapons would stifle progress and innovation
Synesis
- Developed Kipod, a video analytics platform used by law enforcement agencies, governments, and private security organizations to find faces, license plates, object features, and behavioral events
- In use by law enforcement in Belarus, Russia, Kazakhstan, and Azerbaijan
Tencent
- China’s biggest social media company
- Created Miying platform to assist doctors with disease screening and more
- Focused on research in machine learning, speech recognition, natural language processing, and computer vision
- Developing practical AI applications in online games, social media, and cloud services
- Investing in autonomous vehicle AI technologies
- Has described its relationship to public in terms of a social contract: “Billions of users have entrusted us with their personal sensitive information; this is the reason we must uphold our integrity above the requirements of the law.”
VisionLabs
- Developed Luna, software package that helps businesses verify and identify customers based on photos or videos
- Partners with more than 10 banks in Russia and the Commonwealth of Independent States (CIS)
- In response to survey, stated that they “explicitly prohibit the use of VisionLabs technology for military applications. This is a part of our contracts. We also monitor the results/final solution developed by our partners.”
Yitu
- Developed “Intelligent Service Platform,” an algorithm that covers facial recognition, vehicle identification, text recognition, target tracking, and feature-based image retrieval
- Its DragonFly Eye System can reportedly identify a person from a nearly two-billion-photo database within seconds
- Technology utilized by numerous public security bureaus
- In February 2018, supplied Malaysia’s police with facial recognition technologies; partners with local governments and other organizations in Britain
About the Future of Life Institute
The Future of Life Institute (FLI) is a global non-profit with a team of 20+ full-time staff operating across the US and Europe. FLI has been working to steer the development of transformative technologies towards benefitting life and away from extreme large-scale risks since its founding in 2014. Find out more about our mission or explore our work.