The bill targets chatbots posing as licensed professionals. As written, it makes almost any useful answer in a regulated field a liability, with no defense for accuracy nor giving context.
Great summary, but I think you’re being too charitable when you say the bill is well intended. It seems like it was written or pushed by lawyers because they benefit from reducing competition for clients and by getting settlements from suing them. This will be a nightmare for software developers because it means they need to restrict which answers they give for a subset of users based on location, both of which are hard to determine. I mean you can now download an open source model that runs locally on your own phone or computer, so will a bunch of unscrupulous lawyers install these on their machines, ask a restricted question and then file lawsuits against the developers?
Lawmakers are quick to come up with regulations to prevent all kinds of things that they shouldn’t, and when the targets are global software platforms and the lawmakers are in a bunch of various nations, states and localities it will stifle all progress.
Oh interesting, I hadn’t thought of open source models. I figured there the deployer is the person installing it and setting it up (harming themselves?) but there’s not clarity around that either.
I’ve been watching the lawmaker’s videos about it - I think misuse of this bill for profit motives is something that’s more clear in right-leaning or abundance-centric-frameworks. I don’t think she’s tracking? Hence I wanted to write up a kind analysis of it. I’m planning on sending it to her in some form.
Thanks Greg - I think I’m misunderstanding something. As in, people would emigrate because limited AI will impact their ability to research and do their jobs? I’m kind of expecting people to use VPNs or just not realize what they’re missing. :(
VPNs are a possibility, yes, but over the long term if life has more friction in NY than, say, Texas, people will move. Word of mouth. You are right that it will not be instant.
Thanks Angela - it’s unclear to me whether it applies to frontier AI companies. That language was ambigious so I left out frontier labs out. Is your take that it clearly does? I couldn’t tell.
The law is only two pages and says: “Proprietor" shall mean any person, business, company, organization, institution or government entity that owns, operates or deploys a chatbot system used to interact with users. Proprietors shall not include third-party developers that license their chatbot technology to a proprietor.
Proprietor is who is liable for damages,
This is poorly written, but to me, it means OpenAI, Anthropic or Google are proprietors because they provide the chatbot (as well as the model) in their ChatGPT, Claude and Gemini apps. Since they don’t license the models to themselves, they aren’t exempt. Apple now provides a chat interface to OpenAIs model and is planning on adding Google also, so if the lawyers want to chase the fattest target, I guess it will be Apple.
Claude fact-checking skill agrees with you, that frontier labs seem to be liable. Burnell Law Firm seems to disagree, noting that it's downstream applications that would be liable. But if there's no downstream application, and it's just the LLM, it does seem to me like it would apply to Anthropic/OpenAI. I'm still on the fence because the lawmaker herself says in interviews that you using a chatbot like normal isn't included.
I focused on what was clear: the harms to downstream applications that pop up trying to make tailored resources.
This won’t slow down progress, worst case the big labs say you can’t use their apps in NY, but keep working. And no matter what the labs do, you can bet the government labs in the PRC don’t care about US law, let alone NY state law.
Oh interesting. True! Though it also depends on whether NY would try to govern foreign LLMs, in which case I think DeepSeek might block NY access potentially?
Great summary, but I think you’re being too charitable when you say the bill is well intended. It seems like it was written or pushed by lawyers because they benefit from reducing competition for clients and by getting settlements from suing them. This will be a nightmare for software developers because it means they need to restrict which answers they give for a subset of users based on location, both of which are hard to determine. I mean you can now download an open source model that runs locally on your own phone or computer, so will a bunch of unscrupulous lawyers install these on their machines, ask a restricted question and then file lawsuits against the developers?
Lawmakers are quick to come up with regulations to prevent all kinds of things that they shouldn’t, and when the targets are global software platforms and the lawmakers are in a bunch of various nations, states and localities it will stifle all progress.
Oh interesting, I hadn’t thought of open source models. I figured there the deployer is the person installing it and setting it up (harming themselves?) but there’s not clarity around that either.
I’ve been watching the lawmaker’s videos about it - I think misuse of this bill for profit motives is something that’s more clear in right-leaning or abundance-centric-frameworks. I don’t think she’s tracking? Hence I wanted to write up a kind analysis of it. I’m planning on sending it to her in some form.
Bills like this will force emigration to jurisdictions that do not have such laws. I wonder, is that the intended consequence?
Thanks Greg - I think I’m misunderstanding something. As in, people would emigrate because limited AI will impact their ability to research and do their jobs? I’m kind of expecting people to use VPNs or just not realize what they’re missing. :(
VPNs are a possibility, yes, but over the long term if life has more friction in NY than, say, Texas, people will move. Word of mouth. You are right that it will not be instant.
The real intention of the legislation is to slow down AI progress and thus buy us a bit more time before AI drives humanity to extinction.
Thanks Angela - it’s unclear to me whether it applies to frontier AI companies. That language was ambigious so I left out frontier labs out. Is your take that it clearly does? I couldn’t tell.
The law is only two pages and says: “Proprietor" shall mean any person, business, company, organization, institution or government entity that owns, operates or deploys a chatbot system used to interact with users. Proprietors shall not include third-party developers that license their chatbot technology to a proprietor.
Proprietor is who is liable for damages,
This is poorly written, but to me, it means OpenAI, Anthropic or Google are proprietors because they provide the chatbot (as well as the model) in their ChatGPT, Claude and Gemini apps. Since they don’t license the models to themselves, they aren’t exempt. Apple now provides a chat interface to OpenAIs model and is planning on adding Google also, so if the lawyers want to chase the fattest target, I guess it will be Apple.
Of course, adding a disclaimer that I no a lawyer or chatbot, or in NY, so don’t sue me
Claude fact-checking skill agrees with you, that frontier labs seem to be liable. Burnell Law Firm seems to disagree, noting that it's downstream applications that would be liable. But if there's no downstream application, and it's just the LLM, it does seem to me like it would apply to Anthropic/OpenAI. I'm still on the fence because the lawmaker herself says in interviews that you using a chatbot like normal isn't included.
I focused on what was clear: the harms to downstream applications that pop up trying to make tailored resources.
This won’t slow down progress, worst case the big labs say you can’t use their apps in NY, but keep working. And no matter what the labs do, you can bet the government labs in the PRC don’t care about US law, let alone NY state law.
Oh interesting. True! Though it also depends on whether NY would try to govern foreign LLMs, in which case I think DeepSeek might block NY access potentially?