NYC Issues Updated Proposed Regulations on AI Employment Decision Tools

On December 15, 2022, the New York City Department of Consumer and Worker Protection issued updated proposed regulations for Local Law 144. We reported on Local Law 144 here. Briefly, this law requires NYC employers to conduct Bias Audits on any Automated Employment Decision Making Tools (“AEDT”).

As we previously reported on September 19, 2022, the Department issued the first round of proposed regulations. This blog highlights the revisions and changes from the last draft of regulations. The Department will conduct a public hearing on the second round of regulations on January 23, 2023.

Below is a breakdown of the second round of regulations and how they differ from the first round.

NEW DEFINITIONS

The Update Regs include four (4) new definitions:

  • Code. This refers to the Administrative Code of NY.

    • NOTE: This is an extremely minor addition. The actual statute, known as Local Law 144 , is part of the Code. This is just clarifying that when the regulations mention “Code” that means the Local Law 144.

  • Historical Data. Data collected during an employer or employment agency’s use of an AEDT to assess candidates for employment or employees for promotion.

    • NOTE: As explained below, a Bias Audit conducted pursuant to this law must use Historical Data.

  • Scoring Rate. The rate at which individuals in a category receive a score above the sample’s median score, where the score has been calculated by an AEDT.

    • NOTE: This is important when calculating the Impact Ratio. Briefly, a Bias Audit calculates the (1) Selection Rate for each category, (2) the Impact Ratio for each category, and (3) the impact of the AEDT on (a) sex categories, (b) race/ethnicity categories, (c) intersectional categories.

  • Test Data. This is data used to conduct a bias audit that is not historical data.

    • NOTE: This addition is important. Test Data can be used to conduct a Bias Audit, when Historical Data is nonexistent. This is a helpful addition to the regulations. There will undoubtedly be situations where an employer does not have Historical Data, given that they may have just begun utilizing an AEDT.

REVISED DEFINTIONS

The following definitions have been revised. In an effort to highlight the changes, we struck through the old language:

  • Independent Auditor. [M]eans a person or group that is not involved in using or developing an AEDT that is responsible for conducting a bias audit of such AEDT. capable of exercising objective and impartial judgment on all issues within the scope of a bias audit of an AEDT. An auditor is not an independent auditor if the auditor:

    • is or was involved in using, developing, or distributing the AEDT;

    • at any point during the bias audit, has an employment relationship with an employer or employment agency that seeks to use or continue to use the AEDT or with a vendor that developed or distributes the AEDT; or

    • at any point during the bias audit, has direct financial interest or a material indirect financial interest in an employer or employment agency that seeks to use or continue to use the AEDT or in a vendor that developed or distributed the AEDT.

      NOTE: This is a big change from the first round of regulations.

      • EMPLOYER CONSIDERATIONS: Employers must budget for hiring Independent Auditors to conduct the Bias Audits. Per the proposed regulations, the Independent Auditors cannot be employees of the employer.

      • VENDOR CONSIDERATIONS: The Independent Auditors cannot have any financial interest in the companies who develop the AEDT. In other words, companies who develop and sell these AEDT cannot conduct the Bias Audits on their own product. AEDT developers cannot have a team of Auditors that audit their own product.

  • Impact Ratio. [M]eans either (1) the selection rate for a category divided by the selection rate of the most selected category, or (2) the average score of all individuals in a category scoring rate for a category divided by the average score of individuals in the scoring rate for the highest scoring category.

    • VENDOR CONSIDERATIONS: Vendors conducting Bias Audits must note this change.

REVISED DEFINITION OF A BIAS AUDIT

  • AEDT’S MUST UNDERGO A BIAS AUDIT ANNUALLY

Before getting into the definition of a Bias Audit, the proposed regulations offer a new restriction: Employers MUST conduct a Bias Audit on an annual basis.

The regs are clear that, “an [employer] may not use or continue to use an AEDT if more than one year has passed since the most recent bias audit of the AEDT.”

Next, the Revised Proposed Regulations include a more detailed definition of a Bias Audit. The proposed regulations provide two subsections: (1) a section for candidates considered for hire, and (2) a section relating to employees being considered for promotion. Below is the updated language. Again, we will strikethrough the old language to highlight the difference:

  • HIRING

    [The new requirements the definition are in bold.]

    • When a AEDT selects candidates to be considered for hiring OR when an AEDT classifies individuals into groups, a Bias Audit must:

      • Calculate the selection rate for each Category;

      • Calculate the Impact Ratio for each Category; and

      • The calculations above must separately calculate the impact of the AEDT on: (1) sex categories, (2) race/ethnicity categories, and (3) intersectional categories of sex, ethnicity and race.

      • Where an AEDT classifies candidates into groups (ie- leadership styles), the calculations identified above must be performed for each group.

    • EMPLOYER CONSIDERATIONS: Prudent employers must anticipate that even when classifying individuals by non-protected classes (ie- leadership style, communication style etc.), calculations must be conducted to ensure that there is no bias within these “other” classification groups.

    • VENDOR CONSIDERATIONS: Vendors must be aware that employers are going to be held accountable for bias within these non-protected classifications.

  • PROMOTION

    [The new requirements the defintion are in bold.]

    • When a AEDT selects candidates to be considered for hiring OR when an AEDT classifies individuals into groups, a Bias Audit must:

      • Calculate the selection rate for each Category;

      • Calculate the Impact Ratio for each Category; and

      • The calculations above must separately calculate the impact of the AEDT on: (1) sex categories, (2) race/ethnicity categories, and (3) intersectional categories of sex, ethnicity and race.

    • NOTE: Notice that the subsection related to classifying employees into groups, is not included in the “Promotion” defintion . Per these proposed regulations, employers/vendors may not need to conduct bias audits for internal promotions based on non-protected classifications (ie- leadership styles, communication styles etc.)

BIAS AUDIT DATA REQUIREMENTS

The Department provided clarity on what data must be used for Bias Audits by adding an entire section. Below is the section text along with some commentary:

  • A bias audit conducted…must use historical data of theAEDT. If insufficient historical data is available to conduct a statistically signfiicant bias audt, test data may be used instead.

  • If a bias audt uses test data, the summary of results of the bias audit must explain why historical data was not used and describe how the test data was generated and obtained.

  • A bias audit of an AEDT used by multiple employers or employment agencies may use historical data of any employers or employment agencies that use the AEDT. However, an employer or employment agency may rely on a bias audit of an AEDT that uses the historical data from its use of the AEDT to the independent auditor for the bias audit or if it has never used the AEDT.

NOTE: The third subsection is interesting. As written, it means that employers may use historical data of other employers that use a particular AEDT. However, this is only allowed in a specific situation:

  • FIRST: The employer must have provided historical data from its use of the AEDT to the Independent Auditor (defined above); OR

  • SECOND: If the employer has never used the specific AEDT.

This is a bit confusing….

As discussed above, an Independent Auditor must be completely separate from the AEDT developer/vendor. It seems that if an employer wishes to conduct a Bias Audit using other employers’ historical data, the employer must obtain the data from the AEDT developer/vendor . Then, the data would have to be given to the Independent Auditor to conduct the Audit. The logistics of this transfer would need to be contractually crafted.

Employers who do not yet have their own Historical Data to conduct a Bias Audit, may want to opt for the option of using other employers’ Historical Data. Conducing a Bias Audit based on a Test Data may affect the results of the Bias Audit. Using others’ Historical Data may be the preferred option for employers who have just begun using an AEDT. Employers should consult with their vendor and Independent Auditors to assess the best option.

PUBLISHED RESULTS

The Department clarified employers’ obligation to publish the results of the Bias Audits. As previously reported, employers must post the (1) date of the most recent biad audit, and (2) the distribution date of the AEDT. The Department embedded an additional requirement (new text in bold):

  • …[E]mmployers and employment agencies in the city must make the following publicly available on the careers or jobs section of their website in a clear and conspicuous manner:

    • the date of the most recent bias audit of such AEDT and a summary of the results, which shall include the selection rates and impact ratios for all categories source and explanation of the data used to conduct the bias audit and the selection rates and impact ratios for all categories; and

    • the distribution date of the AEDT to which such bias audit applies.

The first draft of the regulation required employers to simply publish the results of the Audits, in the form of the selection rates and impact ratios. As you can imagine, this would have resulted in posted information that the general public cannot digest. The second revision requires employers to dig deeper and explain the basis for the Selection Rate and Impact Ratios.

NOTICE TO CANDIDATES AND EMPLOYEES

Lastly, the Department bulks up the Notice Requirement for Employers. As we know, the first draft of the regulations instructs employers to provide notice to candidates who reside in NYC. The basic requirements of notice remain unchanged. However, the Department rearranged the sections. One in beginning of the section, and the other at the end. Below are the re-worked sections with commentary:

  • FIRST REVISION: The notice required by…the Code must include instructions for how an individual can request an alternative selection process or a resonable accommodation under the laws, if available. Nothing in this subchapter requires an employer or employment agency to provide an alternative selection process.

    • ANOTHER NOTE: This section ends by reminding employer that they are not required to provide an alternative selection process. Employers must be reminded that this last sentence does not negate their obligations under the ADA.

  • SECOND REVISION: …[A]n employer or employment agency must:

    (1) Provide information on the emmployment section of its website in a clear and conspicuous manner about its AEDT data retention policy, the type of data collected for the AEDT, and the source of the data,

    (2) Post instructiosn on the employment section of its website in a clear and conspicuous manner for how to make a written request for such information, and if a written request is received, provide such information within 30 days; or

    (3) Provide an explanation to a candidate for employment or employee being considered for promotion why disclosure of such information would violate local, state, or federal law, or interere with a law enforcement investigation.

    • NOTE: The first round of regs allowed for the scenario where employers do NOT post a notice on their website. This second draft does not provide for such scenario. In other words, employers must put some sort of notice on the website regarding the use of AEDT.

FINAL THOUGHTS

A public hearing on these proposed regulations and Local Law 144 will be held on January 23, 2023 at 11:00 am. Prudent employers, AEDT developers, and Auditors should stay up-to-date on law and its fast approaching effective date of April 15, 2023.

Susie Cirilli