Skip to content

New Report: Don’t Be Evil – A Survey of the Tech Sector’s Stance on Lethal Autonomous Weapons

Published:
August 19, 2019
Author:
Kirsten Gronlund

Contents

As we move towards a more automated world, tech companies are increasingly faced with decisions about how they want — and don’t want — their products to be used. Perhaps most critically, the sector is in the process of negotiating its relationship to the military, and to the development of lethal autonomous weapons in particular. Some companies, including industry leaders like Google, have committed to abstaining from building weapons technologies; Others have wholeheartedly embraced military collaboration. 

In a new report titled “Don’t Be Evil,” Dutch advocacy group Pax evaluated the involvement of 50 leading tech companies in the development of military technology. They sent out a survey asking companies about their current activities and their policies on autonomous weapons, and used each company’s responses to categorize it as “best practice,” “medium concern,” or “high concern.” Categorizations were based on 3 criteria:  

  • Is the company developing technology that could be relevant in the context of lethal autonomous weapons?
  • Does the company work on relevant military projects?
  • Has the company committed to not contribute to the development of lethal autonomous weapons? 

“Best practice” companies are those with explicit policies that ensure their technology will not be used for lethal autonomous weapons.

“Medium concern” companies are those currently working on military applications of relevant technology but who responded that they were not working on autonomous weapons; or companies who are not known to be working on military applications of technology but who did not respond to the survey.

“High concern” companies are those working on military applications of relevant technology who did not respond to the survey. 

The report makes several recommendations for how companies can prevent their products from contributing to the development of lethal autonomous weapons. It suggests that companies make a public commitment not to contribute; that they establish clear company policies reiterating such a commitment and providing concrete implementation measures; and that they inform employees about the work they are doing and allow open discussion around any concerns. 

Pax identifies six sectors considered relevant to autonomous weapons: big tech, AI software and system integration, autonomous (swarming) aerial systems, hardware, pattern recognition, and ground robots. The report is organized into these categories, and then subdivided further by country and product. We’ve instead listed the companies in alphabetical order. Find basic information about all 50 companies in the chart, and read more about a select group below.

Companies involved in developing military technology

CompanyHQRelevant TechnologyRelevant Military / Security ProjectsLevel of Concern
AERIALXCanadaCounter-drone systemsDroneBulletHigh
AIROBOTICSIsraelAutonomous dronesBorder security patrol botsMedium
AIRSPACE SYSTEMSUSCounter-drone systemsAirspace interceptorHigh
ALIBABAChinaAI chips; facial recognitionMedium
AMAZONUSCloud; drones; facial and speech recognitionJEDI; RekognitionHigh
ANDURIL INDUSTRIESUSAI platformsProject Maven; LatticeHigh
ANIMAL DYNAMICSUKAutonomous dronesSkeeterBest practice
APPLEUSComputers; facial and speech recognitionMedium
ARBE ROBOTICSIsraelAutonomous vehiclesBest practice
ATOSFranceAI architecture; cyber security; data ManagementMedium
BAIDUChinaDeep learning; pattern recognitionMedium
BLUE BEAR SYSTEMSUKUnmanned maritime and aerial systemsProject Mosquito/LANCAHigh
CAMBRICONChinaAI chipsMedium
CITADEL DEFENSEUSCounter-drone systemsTitanHigh
CLARIFAIUSFacial recognitionProject MavenHigh
CLOUDWALK TECHNOLOGYChinaFacial recognitionMedium
CORENOVA TECHNOLOGIESUSAutonomous swarming systemsHiveDefense; OFFSETHigh
DEEPGLINTChinaFacial recognitionMedium
DIBOTICSFranceAutonomous navigation; drones‘Generate’Medium
EARTHCUBEFranceMachine learning‘Algorithmic warfare tools of the future’High
FACEBOOKUSSocial media; pattern recognition; virtual RealityMedium
GENERAL ROBOTICSIsraelGround robotsDogoBest practice
GOOGLEUSAI architecture; social media; facial recognitionBest practice
HERON SYSTEMSUSAI software; machine learning; drone applications‘Solutions to support tomorrow’s military aircraft’High
HIVEMAPPERUSPattern recognition; mappingHiveMapper appBest practice
IBMUSAI chips; cloud; super computers; facial recognitionNuclear testing super computers; ex-JEDIMedium
INNOVIZIsraelAutonomous vehiclesMedium
INTELUSAI chips; UASDARPA HIVEHigh
MEGVIIChinaFacial recognitionMedium
MICROSOFTUSCloud; facial recognitionHoloLens; JEDIHigh
MONTVIEUXUKData analysis; deep learning‘Revolutionize human information relationship for defence’High
NAVERS. Korea‘Ambient Intelligence’; autonomous robots; machine vision systemsMedium
NEURALAUSDeep learning neural network softwareTarget identification software for military dronesMedium
ORACLEUSCloud; AI infrastructure; big dataEx-JEDIHigh
ORBITAL INSIGHTUSGeospatial analyticsMedium
PALANTIRUSData analyticsDCGS-AHigh
PERCEPTOIsraelAutonomous dronesMedium
ROBOTEAMIsraelUnmanned systems; AI softwareSemi-autonomous military UGVsHigh
SAMSUNGS. KoreaComputers and AI platformsMedium
SENSETIMEChinaComputer vision; deep learningSenseFace; SenseTotem for police useHigh
SHIELD AIUSAutonomous (swarming) dronesNovaHigh
SIEMENSGermanyAI; AutomationKRNS; TRADESMedium
SOFTBANKJapanTelecom; RoboticsBest practice
SPARKCOGNITIONUSAI systems; swarm technology‘Works across defense and national security space in the U.S.’High
SYNTHESISBelarusAI- and cloud-based applications; pattern recognitionKipodHigh
TAIWAN SEMICONDUCTORTaiwanAI chipsMedium
TENCENTChinaAI applications; cloud; ML; pattern recognitionMedium
THARSUSUKRoboticsMedium
VISIONLABSRussiaVisual recognitionBest practice
YITUChinaFacial recognitionPolice useHigh

Company profiles

Company names are colored to indicate concern level: best practicemedium concernhigh concern.

AerialX

  • Developing the DroneBullet, a kamikaze drone that can autonomously identify, track, and attack a target drone
  • Working to modify DroneBullet “for a warhead-equipped loitering munition system”

Airobotics

  • In response to survey, stated that its “drone system has nothing to do with weapons and related industries
  • Has clear links to military and security business; announced a Homeland Security and Defense division and an initiative to perform emergency services in 2017
  •  Involved in border security, in particular US-Mexico border; provides patrol bots
  • Co-founder has stated that it will not add weapons to its drones

Airspace Systems

  • Utilizes AI and advanced robotics for airspace security solutions, including “long-range detection, instant identification, and autonomous mitigation—capture and safe removal of unauthorized or malicious drones”
  • Developed Airspace Interceptor, a fully autonomous system that can capture target drones, in collaboration with US Department of Defense

Alibaba

  • China’s largest online shopping company
  • Recently invested in seven research labs that will focus on areas including AI, machine learning, network security, and natural language processing
  • Established a semiconductor subsidiary, Pingtouge, in September 2018
  • Major investor in tech sector, including in Megvii and SenseTime 

Amazon

  • Likely winner of JEDI contract, a US military project that will serve as universal data infrastructure linking Pentagon and soldiers in the field
  • Developed Rekognition program, used by police; testing by ACLU revealed that nearly 40 percent of false matches involved people of color
  • CEO has stated, “If big tech companies are going to turn their back on the U.S. Department of Defense, this country is going to be in trouble.” 
  • Received backlash since exposure of partnership with government agencies, including ICE
  • Has since proposed guidelines for responsible use of tech

Anduril Industries

  • Co-founded by a former intelligence official
  • Has vocally supported stronger ties between tech sector and Pentagon: “AI has paradigm-shifting potential to be a force-multiplier […] it will provide better outcomes faster, a recipe for success in combat.”
  • Involved in Project Maven
  • Has offered support for the Pentagon’s newly formed Joint Artificial Intelligence Center
  • Developed the Lattice, an autonomous system that provides soldiers with a view of the front line and can be used to identify targets and direct unmanned vehicles into combat; has been used to catch border crossers
  • Co-founder has stated that Anduril is “deployed at several military bases. We’re deployed in multiple spots along the U.S. border […] We’re deployed around some other infrastructure I can’t talk about.”

Animal Dynamics

  • Spin-off company originating in Oxford University’s Zoology Department
  • Develops unmanned aerial vehicles
  • Stork, a paraglider with autonomous guidance and navigation, has received interest from both military and humanitarian aid/disaster relief organizations
  • Skeeter, “disruptive drone technology,” was developed with funding from UK government’s Defense Science and Technology Laboratory
  • In March 2019, took over software developer Accelerated Dynamics, which has developed ADx autonomous flight-control software
  • Use of ADx with Skeeter allows it to be operated in a swarm configuration, which has military applications
  • In response to survey, CEO stated that “we will not weaponize or provide ‘kinetic’ functionality to the products we make,” and that “legislating against harmful uses for autonomy is an urgent and necessary matter for government and the legislative framework to come to terms with.”

Arbe Robotics

  • Began in military and homeland security sectors but has moved to cars
  • In response to survey, stated that it “will sign agreements with customers that would confirm that they are not using our technology for military use.”

Baidu

  • Largest provider of the Chinese-language Internet search services
  • Highly committed to artificial intelligence and machine learning and is exploring applications for facial recognition technology
  • Opened Silicon Valley AI research lab in 2013, where it has been heavily investing in AI applications; has scaled down this research since US-China trade war
  • In charge of China’s Engineering Laboratory for Deep Learning Technologies, established March 2017
  • Will contribute to National Engineering Laboratory for Brain-Inspired Intelligence Technology and Applications

Blue Bear Systems

  • Research company involved in all aspects of unmanned systems and autonomy, including big data, AI, electronic warfare, and swarming systems
  • In March 2019, consortium it headed was awarded UK Ministry of Defense contract worth GBP 2.5 million to develop drone swarm technology

Citadel Defense

  • “Protects soldiers from drone attacks and surveillance in enemy combat” and “creates a force multiplier for Warfighters that enables them to get more done with the same or fewer resources”
  • Contracted by US Air Force to provide systems that can defeat weaponized drones and swarms
  • Developed autonomous counter-drone system called Titan

Corenova Technologies

  •  Offers “military-grade solutions to secure autonomous operations,” according to website
  • Developed HiveDefense, “an evolving swarm of self-learning bots”
  • Works with DARPA on OFFSET, facilitating unmanned missions without human control

Dibotics

  • Works on autonomous navigation
  • Supported by Generate, a program for French defense start-ups
  • Founder/CEO signed FLI’s 2017 open letter to the UN

EarthCube

  • “Developing monitoring solutions based on an automated analysis of geospatial information”
  • Has been described as “conceiving of the algorithmic warfare tools of the future.”
  • CEO has stated, “With the emergence of new sensors—whether they are satellite, UAV or plane—we have seen here a great opportunity to close the gap between AI in the lab and Activity Based Intelligence (ABI) in the field.”

General Robotics

  • Robotics company focused on defense and security
  • Founder previously worked in Israeli defense ministry’s R & D authority
  • Supplies “advanced robotics systems to counter-terrorist units worldwide,” many of which are designed for “urban warfare”
  • Developed Dogo, said to be “the world’s first inherently armed tactical combat robot,” but controlled remotely rather than autonomously
  • In response to survey, CEO stated that “our position is not to allow lethal autonomous weapons without human supervision and human final active decision […] In general, our systems are designed to provide real-time high quality information and to present it to a trained human operator in an intuitive manner; this insures better decision making by the human and thereby better results with less casualties.” 

Heron Systems

  • Provides “leading-edge solutions for national security customers”
  • States that its mission is “to strengthen America’s defense by providing innovative laboratory testing and simulation solutions”

Hivemapper

  • Software provides mapping, visualization, and analytic tools; uses video footage to generate instant detailed 3-D maps and detect changes; could potentially be used by Air Force to model bombing
  • Founder/CEO has stated that he “believes Silicon Valley and the US government have to work together to maintain America’s technological edge—lest authoritarian regimes that don’t share the US values catch up.”
  • Founder/CEO signed FLI’s 2015 open letter; In his survey response, he stated that “we absolutely want to see a world where humans are in control and responsible for all lethal decisions.”  

IBM

  • Bid for Jedi contract and failed to qualify 
  • Actively working towards producing “next-generation artificial intelligence chips” for which it is building a new AI research center; over the next 10 years, expect to improve AI computing by 1,000 times
  • Long history of military contracting, including building supercomputers for nuclear weapons research and simulations
  • Currently involved in augmented military intelligence research for US Marine Corps
  • 3 dozen staff members, including Watson design lead and VP of Cognitive Computing at IBM Research, signed a 2015 open letter calling for a ban on lethal autonomous weapons
  • Developed Diversity in Faces dataset using information from Flickr images; claims the project will reduce bias in facial recognition; Dataset available to companies and universities linked to military and law enforcement around the world
  • In response to survey, confirmed it is not currently developing lethal autonomous weapons systems

Innoviz

  • Produces laser-based radar for cars
  • Founded by former members of the IDF’s elite technological unit, but does not currently appear to be developing military applications

Intel

  • Develops various AI technologies, including specific solutions, software, and hardware, which it provides to governments
  • Selected by DARPA in 2017 to collaborate on DARPA HIVE, a data-handling and computing platform utilizing AI and ML
  • Announced in 2018 that it will work with DARPA on developing “the design tools and integration standards required to develop modular electronic systems”
  • Has invested significantly in unmanned aerial vehicles and flight control technology

Megvii

  • AI provider known for facial recognition software Face++
  • Reportedly uses facial scans from a Ministry of Public Security photo database that contains files on nearly every Chinese citizen
  • Has stated, “We want to build the eyes and brain of the city, to help police analyze vehicles and people to an extent beyond what is humanly possible.”

Microsoft

  • Competing with Amazon for JEDI contract
  • Published “The Future Computed” in 2018, which defines core principles necessary for the development of beneficial AI 
  • According to employees, “With JEDI, Microsoft executives are on track to betray these principles in exchange for short-term profits.” 
  • Company position on lethal autonomous weapons systems unclear
  • First tech giant to call for regulations to limit use of facial recognition technology

Montvieux

  • Developing a military decision-making tool that uses deep learning-based neural networks to assess complex data
  • Receives funding from the UK government

Neurala

  • Sells AI technology that can run on light devices and helps drones, robots, cars, and consumer electronics analyze their environments and make decisions
  • Military applications are a key focus
  • Works with a broad range of clients including the US Air Force, Motorola, and Parrot
  • Co-founder/CEO signed FLI’s 2017 open letter to the UN

Oracle

  • Provides database software and technology, cloud-engineered systems, and enterprise software products
  • Website states, “Oracle helps modern defense prepare for dynamic mission objectives”
  • Bid for JEDI contract and failed to qualify; filed several complaints, in part related to Pentagon’s decision to use only one vendor

Palantir

  • Data-analysis company founded in 2004 by Trump advisor; has roots in CIA-backed In-q-Tel venture capital organization
  • Producer of “Palantir Intelligence,” a tool for analyzing data that is used throughout the intelligence community
  • Has developed predictive policing technology used by law enforcement around the US
  • In 2016, won a Special Operations Command contract worth USD 222 million for a technology 
  • In March 2019, won a US Army contract worth over USD 800 million to build the Distributed Common Ground System, an analytical systems for use by soldiers in combat zones

Percepto

  • Developed the Sparrow, an autonomous patrol drone with security applications
  • Focuses explicitly on industrial, rather than military or border security, applications
  • In response to survey, stated “Since we develop solutions to the industrial markets, addressing security, safety, and operational needs, the topic of lethal weapon[s] is completely out of the scope of our work”

Roboteam

  • Founded by two former Israeli military commanders with “access to the Israel Defense Forces as our backyard for testing”
  • Specifically serves military markets, including the Pentagon
  • Developed Artificial Intelligence Control Unit (AI-CU), which brings autonomous navigation, facial recognition, and other AU-enabled capabilities to control and operation of unmanned systems and payloads
  • Exposure of links to Chinese investment firm FengHe Fund Management appears to have cost them a series of US Army robotics contracts last year

Samsung

  • One of world’s largest tech companies
  • Developing AI technologies to be applied to all its products and services in order to retain its hold on telephone/computer market
  • Samsung Techwin, Samsung’s military arm known for SG1A Sentry robot, was sold in 2014

SenseTime

  • Major competitor of Megvii
  • Sells software that recognizes objects and people
  • Various Chinese police departments use its SenseTotem and SenseFace systems to analyze video and make arrests
  • Valued at USD 4.5 billion, it is “the world’s most valuable AI start-up” and receives about two-fifths of its revenue from government contracts
  • In November 2017, sold its 51 percent stake in Tangli Technology, a “smart-policing” company it helped found

Shield AI

  • States that its “mission is to protect service members and innocent civilians with artificially intelligent systems”
  • Makes systems based on Hivemind, AI that enables robots to “learn from their experiences”
  • Developed Nova, a “combat proven” robot that autonomously searches buildings while streaming video and generating maps
  • Works with Pentagon and Department of Homeland Security “to enable fully autonomous unmanned systems that dramatically reduce risk and enhance situational awareness in the most dangerous situations.

Siemens

  • Europe’s largest industrial manufacturing conglomerate
  • Known for medical diagnostics equipment (CT scanners), energy equipment (turbines, generators), and trains
  • Produces MindSphere, a cloud-based system that helps enable the US of AI in industry
  • In 2013, won a USD 2.2 million military research contract with Carnegie Mellon University and HRL Laboratories to develop improved intelligence tools
  • Collaborating with DARPA on the TRAnsformative DESign (TRADES) program
  • In response to survey, stated: “Siemens in not active in this business area. Where we see a potential risk that components or technology or financing may be allocated for a military purpose, Siemens performs a heightened due diligence. […] All our activities are guided by our Business Conduct Guidelines that make sure that we follow high ethical standards and implement them in our everyday business. We also work on responsible AI principles which we aim to publish later this year.”

SoftBank

  • Invests in AI technology through its USD 100 billion Vision Fund, including BrainCorp, NVIDIA, and Slack Technologies; Owns some 30 percent of Alibaba
  • Works in partnership with Saudi Arabia’s sovereign wealth fund and is part of Saudi strategy for diversifying away from oil
  • In 2017, took over Boston Dynamics and Schaft, both connected with DARPA
  • Developed the humanoid Pepper robot
  • In response to survey, stated, “We do not have a weapons business and have no intention to develop technologies that could be used for military purposes”

SparkCognition

  • Collaborates “with the world’s largest organizations that power, finance, and defend our society to uncover their highest potential through the application of AI technologies.”
  • Has attracted interest from former and current Pentagon officials, several of whom serve on the board or as advisors
  • Works “across the national security space—including defense, homeland security intelligence, and energy—to streamline every step of their operations”; has worked with the British Army on military AI applications
  • Founder/CEO has stated that he believes restrictions on autonomous weapons would stifle progress and innovation

Synesis

  • Developed Kipod, a video analytics platform used by law enforcement agencies, governments, and private security organizations to find faces, license plates, object features, and behavioral events
  • In use by law enforcement in Belarus, Russia, Kazakhstan, and Azerbaijan

Tencent

  • China’s biggest social media company
  • Created Miying platform to assist doctors with disease screening and more
  •  Focused on research in machine learning, speech recognition, natural language processing, and computer vision
  • Developing practical AI applications in online games, social media, and cloud services
  • Investing in autonomous vehicle AI technologies
  • Has described its relationship to public in terms of a social contract: “Billions of users have entrusted us with their personal sensitive information; this is the reason we must uphold our integrity above the requirements of the law.”

VisionLabs

  • Developed Luna, software package that helps businesses verify and identify customers based on photos or videos
  • Partners with more than 10 banks in Russia and the Commonwealth of Independent States (CIS)
  • In response to survey, stated that they “explicitly prohibit the use of VisionLabs technology for military applications. This is a part of our contracts. We also monitor the results/final solution developed by our partners.”

Yitu

  • Developed “Intelligent Service Platform,” an algorithm that covers facial recognition, vehicle identification, text recognition, target tracking, and feature-based image retrieval
  • Its DragonFly Eye System can reportedly identify a person from a nearly two-billion-photo database within seconds
  • Technology utilized by numerous public security bureaus
  • In February 2018, supplied Malaysia’s police with facial recognition technologies; partners with local governments and other organizations in Britain
This content was first published at futureoflife.org on August 19, 2019.

About the Future of Life Institute

The Future of Life Institute (FLI) is a global non-profit with a team of 20+ full-time staff operating across the US and Europe. FLI has been working to steer the development of transformative technologies towards benefitting life and away from extreme large-scale risks since its founding in 2014. Find out more about our mission or explore our work.

Our content

Related content

Other posts about 

If you enjoyed this content, you also might also be interested in:

The FLI Book Guide

Below you’ll find some of our favorite books on existential risk, existential hope, technology, society and more. These are essential reads for any aspiring philosopher, concerned humanist or curious thinker. Click on a book to learn more.
December 11, 2019

Sign up for the Future of Life Institute newsletter

Join 40,000+ others receiving periodic updates on our work and cause areas.
cloudmagnifiercrossarrow-up linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram