Algorithms are computer code that forms the heart of all computer software, including artificial intelligence (AI) software.  AI algorithms have the ability to continuously refine their analyses as additional data are introduced to the software. AI algorithms are embedded in a wide range of software now in widespread use for commercial applications such as financial services, computer network security, and processing of employment applications.  In this environment, many users now routinely apply AI algorithm analysis to data they provide, yet they do not have a complete understanding of how the embedded algorithms use the data. To the extent that some of the data processed through the algorithms may be sensitive personal data, it may fall within the scope of the growing number of personal data protection laws, such as Virginia’s Consumer Data Protection Act (CDPA). Those laws place obligations on data controllers and data processors to assess the potential adverse impact of sensitive data use on consumers and to try to minimize that impact. Those privacy obligations may eventually force data controllers to obtain audits of some of their AI software as part of an effort to document that they are attempting, in good faith, to protect the interests of consumers as they use consumer personal data.

Consider one currently popular use of AI algorithms, screening of job applicants. AI systems are routinely used to help compare job applicants, yet this widespread use poses potential legal concerns if the algorithms used are biased in some way. New York City is concerned enough about the potential of hiring bias caused by unintended AI bias that it is in the process of implementing a requirement that all algorithms applied for hiring in New York City must be audited by an appropriate neutral party.  As attention to issues such as algorithm bias is relatively recent, there is currently no universally accepted framework for algorithm audits of the sort contemplated by New York City, thus implementation of the audit requirement will likely be a slow process. However, New York City’s acceptance of the concept of algorithm audits suggests that organizations should now be on notice that they may, in the future, be expected to have a more thorough understanding of how the embedded algorithms they rely upon work (including their biases and limitations), and what the implications of the increased use of AI algorithms may be for individuals.

Another current use of AI software is to facilitate the processing of loans. Embedded algorithms now play an important role in many consumer credit and lending decisions. If, however, you are a business that has collected sensitive personal data from a consumer and then used some of that data in an AI loan evaluation system, under the terms of the Virginia CDPA, you will be expected to have made an assessment that the use of the sensitive data for the loan process did not pose an unreasonable risk of harm to the consumer. It may be difficult to make such an assessment if you do not have a thorough understanding of how the embedded algorithms operate, and what constraints they may be operating under, including the presence of implicit or other types of bias.

Prudent organizations should begin to inventory the different embedded AI algorithms they are currently using and should try to understand how those algorithms operate and what limitations they face. Remember that the algorithms in use may be embedded in software resident in your system or they may be located in the cloud network your organization uses. Organizations should also begin to consult with their software vendors and data services providers to explore the future potential for audits and certifications regarding AI algorithms. Alliance Law Group is prepared to assist your organization as it begins to assess the scope of its current use of AI algorithms and its plans for future AI use, to ensure that your organization can make effective use of AI’s positive capabilities while avoiding legal liability or harm to consumers.

*This article is not intended to provide legal advice. Individual facts and circumstances vary. Accordingly, please consult Alliance Law Group or other legal counsel for issues with respect to your legal rights and responsibilities concerning the use of AI algorithms.