AI regulation and the case for slowing innovation
Arguments over AI regulation pull two ways: towards holding AI back or overseeing it until its risks are understood, and towards sparing innovation from rules that could inhibit it. Each argument belongs to its date, from the 2019 call for oversight and the 2023 call for a pause, through objections that European data and device rules and the proposed AI Act could inhibit innovation, to a defence of the Act as a step in the right direction.
Calls for oversight and a pause
In 2019 AI was already helping to diagnose cancer and understand climate change, but one overview of the field held that regulation and oversight were needed to stop the technology being abused. Guidelines were then being developed for the responsible design and implementation of AI systems.
By 2023 AI, once a niche domain for data scientists and AI technologists, had suddenly become widely accessible, and it was being used for everything from drafting emails to answering exam questions. Two writers held that this explosion had been so large and rapid that it had outpaced working practices and even legislation. On their account, it led to the kind of concerns that triggered the open letter in which technology leaders urged a pause in AI development to allow some checks and regulations to be put in place.
Data and device rules as a brake on innovation
In 2020 the head of a US device regulatory consultancy argued that European data and device rules could inhibit innovation in software devices. Machine-learning algorithms need vast amounts of high-quality training data, and organisations face barriers that limit their access to the data they need. Because AI relies heavily on data, he held, the General Data Protection Regulation's rules on data have substantial implications for the development and use of AI, especially machine learning.
Article 5 of the GDPR requires data to be collected for specified, explicit and legitimate purposes and to be adequate, relevant and limited to what is necessary. On his reading, these two restrictions, purpose specification and data minimisation, significantly limit innovation with data, because they keep organisations from collecting new data before they understand its potential value and from reusing existing data for novel purposes.
Article 22 gives individuals the right not to be subject to a decision based solely on automated processing, including profiling, that produces legal effects concerning them or similarly significantly affects them. He read this as giving anyone about whom a company uses AI to make a decision the right to have a human review it. He held that this makes it difficult and impractical to automate many processes, because a process must be developed for those who opt out of the automated one.
Articles 13 to 15 require meaningful information about the logic involved in automated decisions. The EU's guidelines have clarified that this does not necessarily mean full disclosure of the algorithm, but the information should be sufficiently comprehensive for the person to understand the reasons for the decision. He held that organisations cannot always comply with the requirement to explain the logic involved in an algorithmic decision-making process. Even where they can offer an explanation, he held, they may not be able to give it concisely and in the plain language the GDPR requires, so these rules will force many businesses not to use certain AI systems, especially more sophisticated ones, even where these may be more accurate, safer and more efficient than the alternatives.
On his reading, the GDPR has created an artificial scarcity of data, has made it harder to use AI that automates decisions about individuals using personal information, and has put the EU at a competitive disadvantage in the development and use of AI, its restrictions on collecting and using data putting EU firms at a disadvantage against firms in countries such as China, where companies have access to data on hundreds of millions of internet and mobile phone users. Unless the GDPR is amended, he expected it to have a negative impact on the development and use of AI in Europe.
He concluded that the EU's guidelines on device cybersecurity and on which software counts as a medical device may impose an undue burden on bringing devices to market in the EU. In his view, restrictions on how personal data may be used for AI algorithm development, and requirements for clinical validation that may be lengthy and costly, could inhibit innovation. A developer of software for the EU market must therefore, he held, weigh the cost of development against these rules, find the least burdensome approach and consider the requirements of other regulators. Developers may choose to market new devices first in regions with less stringent requirements than the EU, and, he suggested, the development of software devices may benefit from a globally harmonised set of requirements.
Objections to the proposed AI Act
Objections reported in 2022 held that the proposed AI Act may over-regulate. The proposal defined an AI system as software developed with one or more of machine-learning approaches, logic- and knowledge-based approaches, statistical approaches, Bayesian estimation, and search and optimisation methods, which can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations or decisions influencing the environments it interacts with. A professor of artificial intelligence criticised this definition for classing almost all existing and future software as AI, which may lead to overregulation. BVMed held that the definition urgently needed narrowing to prevent market access barriers for device manufacturers. The professor argued that additional regulation should address only novel use cases not yet covered by existing regulation, and that the proposed regulation was not needed, given existing regulation, and lacked delimitation from it.
The professor also considered the proposed requirements for developing or using AI in safety-critical areas disproportionate and innovation-inhibiting for healthcare, particularly those on technical documentation (Article 11), the EU database for stand-alone high-risk AI systems (Article 60) and the reporting of serious incidents and malfunctioning (Article 62). He held the proposal's requirements on data sharing and documentation (Articles 64 and 53) unfeasible because of lacking infrastructure, intellectual property conflicts and potential liability issues. He feared that the regulation would make the use or development of AI in safety-critical areas such as healthcare almost impossible in the EU, further strengthening the leadership of Chinese and US providers, who also have the financial power to implement GDPR-compliant services and to weather fines and lengthy trials. A further objection held that, given the already very tight post-marketing control of health-related risks under the MDR's vigilance system, an additional control and intervention possibility based on the AI Act seemed superfluous and not justified.
The AI Act and holding innovation back
Asked in May 2024 whether any AI legislation covers the idea that just because you can does not mean you should, and whether some kind of cap is needed to keep the brakes on before the risks are understood more deeply, a machine-learning specialist held that the AI Act gives some pathways to limit the impact of AI and to hold it back a bit. He added that, with AI a hype being used for everything, developers should keep in mind that it is not always the right tool for everything, which he thought most of them are aware of. But he did not think innovation needs holding back, because people will use AI for anything, and those who want to use it for something unsafe will do so, perhaps illegally, he suggested. In his view, getting some regulations is a step in the right direction, and if guidance comes some day it will be clearer what exactly to do.
Summing up the risks of AI, the same specialist held that there are a lot of them, that people have to stay aware of the limitations of these systems and the corruption that can happen to them, and that AI should not be used for just anything. He held that the AI Act was long overdue, that it is good that the Act is there to hold the providers and deployers of these systems accountable for the ethical and risk-conscious implementation of AI in practice, and that holding them accountable is very important. In his opinion, more good than bad can generally come from the use of AI, as long as good data modelling practices are held to and people keep being critical of where and how they use it.