NYC Dept. of Consumer & Worker Protection Provides Glimpse of What’s to Come with the NYC AI Bias Law

On September 19, 2022, the New York City Department of Consumer and Worker Protection (“DCWP”) issued Proposed Rules in an attempt to clarify and expand upon Local Rule 144, the NYC law relating to Automated Employment Decision Tools. (NOTE: It may be helpful to check out my previous blog discussing Local Rule 144.) Now, these Rules are merely “proposed” and are currently open to public comment. In fact, on October 24, 2022, there will be a public hearing on these rules.

While these rules are not final, they offer a glimpse of what employers may need to prepare for come January of 2023. Even more, employers must review the Proposed Rules in order to make an informed comment or question.

Definitions

DCWP attempts to bring some clarity through definitions:

  • Automated Employment Decision Tool (AI Tool). Nothing new here. The Proposed Rules keep the same definition included in Local Rule 144.

  • Bias Audit. DCWP doesn’t deviate on this definition either. The Proposed Rules stick with the same definition included in Local Rule 144.

  • Employment Decision. Nothing to see here either. The Proposed Rules refer to Local Rule 144 for the definition.

  • Category. DCWP keeps it simple here as well. The Proposed Rules do not deviate from the definition provided in the Local Rule 144, which is simply any Component 1 Category.

  • Candidate for Employment. This is a person who applied for an employment position by submitting the necessary information and/or items in the format required by the employer or employment agency.

    • NOTE: This definition does not mention the residence of the candidate. Remember, Local Rule 144 includes a notice requirement for Candidates who reside in NYC. The above definition encompasses any individual, regardless of residence. This definition also specifies that “applying for a position” means submitting all necessary info and materials as required by the employer/agency. Based on this definition, merely submitting a resume when a transcript, letter of recommendation and references are required is insufficient to be considered a Candidate for Employment.

  • Distribution Date. This is the date when the Employer/Agency began using the AI Tool.

    • NOTE: Sec. 20-871(b) requires employers to provide notice to candidates/employees who are NYC residents 10 days prior to the use of the AI Tool. The Distribution Date may provide guidance to employers as to when the 10 clock begins to run.

  • Employment Agency. The DCWP takes the definition from the NYC Rules of the City. This definition encompasses career coaches, recruiters, and even entities that “maintain…or lists…job openings or positions…”

    • NOTE: This definition may rope in job boards and apps or engines like Linkedin, Monster, Indeed and the like.

  • Independent Auditor. A person or group that is not involved in using or developing an AI Tool that is responsible for conducting a bias audit of such AI Tool.

    • NOTE. This is interesting. According to this definition, the entity performing the Audit cannot be the same entity that created the AI Tool.

  • Impact Ratio. This is either (1) the selection rate for a category divided by the selection rate of the most selected category, or (2) the average score of all individuals in a category divided by the average score of individuals in the highest scoring category.

    • NOTE: DCWP Proposed Rules requires the Impact Ratio be included in the Bias Audit. (See below.)

  • Machine learning, statistical modeling, data analytics, or artificial intelligence. DCWP includes quite an extensive definition for a, “group of mathematical, computer-based techniques.” Check out the actual text to get the details.

  • Screen. DCWP felt the need to include this definition even though it is a bit redundant. This means to “make a determination about whether someone should be selected or advanced in the hiring or promotion process.”

    • NOTE: The definition of “Employment Decision” under Local 144 means, “to screen candidates for employment or employees for promotion within the city.” The definition of “screen” does not include geographic limitations. It is unclear as to whether this is intentional or just a redundant definition.

  • Selection Rate. The rate at which individuals in a category are either selected to move forward in the hiring process or assigned a classification by an AI Tool.

    • NOTE: The Proposed Rules require Bias Audits to calculate the selection rate for each category.

  • Simplified Output. The prediction/classification of a candidate’s likelihood of success, or score for skills, tagging of a resume, or ranking.

What is a Bias Audit?

Minimum Requirements

The proposed rules explain that at a minimum, a Bias Audit must include: (1) the Selection Rate for each category, (2) the Impact Ratio for each category, and (3) if the AI Tool classifies individuals into groups, then the calculations must be performed for each such classification. (Check out the Examples provided by the DCWP.)

Applicant/Candidate Scores

The Proposed Rules go further in that when an AI Tool assigns a score to an applicant/candidate, then the Bias Audit must include: (1) the average score for the individuals, and (2) the Impact Ratio for each category.

Publication of Results of Bias Audits

DCWP gets specific here. Employers who use an AI Tool must make the following publicly available:

  1. The date of the most recent Bias Audit;

  2. Summary of the results of the most recent Bias Audit, inclusive of the Selection Rates and Impact Ratios (see definitions above); and

  3. Distribution Date of the AI Tool to which the Bias Audit applies.

The above referenced information must be posted on the careers or job section of the employer/employer agency website in a clear and conspicuous manner. DCWP cuts employers some slack in allowing the information to be available via hyperlink, so long as the link is clearly identified as a “Link to Results of the Bias Audit.”

Retention of the Bias Audit Results

Employers/Employment Agencies must keep the summary of results and the Distribution Date posted for at least 6 months after last using the AI Tool for an employment decision.

Notice to Candidates and Employees

Remember, Local 144 requires employers/employment agencies to provide notice to candidates/employees who reside in NYC. The Proposed Rules provides three ways to compliantly provide notice:

Website. Employers may include Notice on the careers or jobs section of the website. This must be clear and conspicuous. this must be posted 10 business days prior to the use of the AI Tool.

Job Posting. The Notice can be in the actual job posting at least 10 business days prior to the use of the AI Tool.

E-Mail or U.S. Mail. The Notice can be sent via e-mail or regular mail at least 10 business days prior to the use of the AI Tool.

Upon Request. If the Notice is not included in the website, job posting, or via Mail, then Local Rule 144 requires that employers provide information about the AI Tool upon request. Employer must post instructions on how individuals can request the Notice. Employers must respond to any requests within 30 days (note that it is not “business days”).

NYC DCWP HEARING: OCTOBER 24, 2022

Employers doing business in NYC may want to check out the hearing on October 24, 2022. It is open to the public and individuals can attend virtually. While these rules are not final, they certainly provide a little more guidance than the initial version of the law.

Susie Cirilli