Rails - Active Record: Find all records which have a count on has_many association with certain attributes

A user has many identities.

class User < ActiveRecord::Base
    has_many :identities
end

class Identity < ActiveRecord::Base
    belongs_to :user
end

An identity has an a confirmed:boolean column. I'd like to query all users that have an only ONE identity. This identity must also be confirmed false.

I've tried this

User.joins(:identities).group("users.id").having( 'count(user_id) = 1').where(identities: { confirmed: false })

But this returns users with one identity confirmed:false but they could also have additional identities if they are confirmed true. I only want users with only one identity confirmed:false and no additional identities that are have confirmed attribute as true.

I've also tried this but obviously it's slow and I'm looking for the right SQL to just do this in one query.

  def self.new_users
    users = User.joins(:identities).where(identities: { confirmed: false })
    users.select { |user| user.identities.count == 1 }
  end

Apologies upfront if this was already answered but I could not find a similar post.